Probabilistically distorted risk-sensitive infinite-horizon dynamic programming

Historically, the study of risk-sensitive criteria has focused on their normative applications - i.e., what should be done. The classic example is expected utility functions which produce deterministic policies. More recently, the literature on dynamic coherent risk measures has broadened the choices for risk-sensitive performance evaluation. However, coherent risk measures must be convex. This paper presents an alternative to both the expected utility and coherent risk measure approaches. This new approach, inspired by cumulative prospect theory (CPT), is nonconvex and has substantial empirical evidence supporting its descriptive power for human decisions, i.e., what is actually done. A key unique feature of the CPT-based approach, essential for modeling human decisions, is probabilistic distortion. Hence, CPT should be used instead of both expected utility and coherent risk measures when modeling human decisions, which requires a higher level of expressiveness than allowed by previous work. In addition, although both coherent risk measures and CPT produce randomized policies, which are more robust against inaccurate probabilistic descriptions of systems, CPT generates policies that are significantly different from those of coherent risk measures.