Markov Decision Process (MDP) models are commonly used tools for optimizing sequential decisions under uncertainty in medical decision making. If the parameters of an MDP satisfy certain assumptions, the optimal policy is guaranteed to be monotone. Unfortunately, these assumptions are not always satisfied. In this video, I define the price of interpretability (PI), which measures the gap between the optimal and an interpretable policy. My coauthors and I assess the PI for the best-performing monotone policy (BMP) and the novel class-ordered monotone policy (CMP), which preserves interpretability along user-defined state and action classes. Within the context of hypertension treatment, the CMP can be computed faster and achieves greater total quality-adjusted life years across a population of 66.5 million people in the US, compared to the BMP.

In this video, I introduce a new framework to obtain ranges of near-optimal actions for finite horizon Markov decision processes (MDPs). I apply the method for the management of hypertension and found that it outperforms the 2017 hypertension guidelines from the American College of Cardiology and American Heart Association by up to 37%, while being within 3% from optimality (assuming that the MDP accurately represents patients' disease progression).