Shortfall as a risk measure: properties, optimization and applications
Introduction
The standard deviation of the return of a portfolio is the predominant measure of risk in finance. Indeed mean-variance portfolio selection using quadratic optimization, introduced by Markowitz (1959), is the industry standard. It is well known (see Huang and Litzenberger, 1988 or Ingersoll, 1987) that the mean-variance portfolio selection paradigm maximizes the expected utility of an investor if the utility is quadratic or if returns are jointly normal, or more generally, obey an elliptically symmetric distribution.1 It has long been recognized, however, that there are several conceptual difficulties with using standard deviation as a measure of risk:
- (a)
Quadratic utility displays the undesirable properties of satiation (increase in wealth beyond a certain point decreases utility) and of increasing absolute risk aversion (the demand for a risky asset decreases as the wealth increases), see e.g., Huang and Litzenberger (1988).
- (b)
The assumption of elliptically symmetric return distributions is problematic because it rules out possible asymmetry in the return distribution of assets, which commonly occurs in practice, e.g., due to the presence of options (see, e.g., Bookstaber and Clarke, 1984). More generally, departures from ellipticity occur due to the greater contagion and spillover of volatility effects between assets and markets in down rather than up market movements, see, e.g., King and Wadhwani (1990), Hamao et al. (1990), Neelakandan (1994), and Embrechts et al. (1999). Asymmetric return distributions make standard deviation an intuitively inadequate risk measure because it equally penalizes desirable upside and undesirable downside returns. In fact, Chamberlain (1983) has shown that elliptically symmetric are the only distributions for which investor's utility is a function only of the portfolio's mean and standard deviation.
Motivated by the above difficulties, alternative downside-risk measures have been proposed and analyzed in the literature (see the discussion in Section 3.1.4). Though intuitively appealing, such risk measures are not widely used for portfolio selection because of computational difficulties and problems with extending standard portfolio theory results, see, e.g., a recent review by Grootveld and Hallerbach (1999).
In recent years the financial industry has extensively used quantile-based downside risk measures. Indeed one such measure, Value-at-Risk, or VaR, has been increasingly used as a risk management tool (see e.g., Jorion, 1997; Dowd, 1998; Duffie and Pan, 1997). While VaR measures the worst losses which can be expected with certain probability, it does not address how large these losses can be expected when the “bad”, small probability events occur. To address this issue, the “mean excess function”, from extreme value theory, can be used (see Embrechts et al., 1999 for applications in insurance). More generally, Artzner et al. (1999) propose axioms that risk measures (they call them coherent risk measures) should satisfy and show that VaR is not a coherent risk measure because it may discourage diversification and thus violates one of their axioms. They also show that, under certain assumptions, a version of the mean excess function, which they call tail conditional expectation (TailVaR), is a coherent measure (see Section 3.1.5 below).
Our goal in this paper is to propose an alternative methodology for defining, measuring, analyzing, and optimizing risk that addresses some of the conceptual difficulties of the mean-variance framework, to show that it is computationally tractable and has, we believe, interesting and potentially practical implications.
The key in our proposed methodology is a risk measure called shortfall, which we argue has conceptual, computational and practical advantages over other commonly used risk measures. It is a variation of the mean excess function and TailVaR mentioned earlier (see also Uryasev and Rockafellar, 1999). Some mathematical properties of the shortfall and its variations have been discussed in Uryasev and Rockafellar (1999), and Tasche (2000).
Our contributions and the structure in this paper are:
- 1.
We define shortfall as a measure of risk in Section 2, and motivate it by examining its natural connections with expected utility theory and stochastic dominance.
- 2.
We discuss structural and mathematical properties of shortfall, including its relations to other risk measures in Section 3. Shortfall generalizes standard deviation as a risk measure, in the sense that it reduces to standard deviation, up to a scalar factor depending only on the risk level, when the joint distribution of returns is elliptically symmetric, while measuring only downside risk for asymmetric distributions. We point out a simple explicit relation of shortfall to VaR and show that it is in general greater than VaR at the same risk level. Moreover, we provide exact theoretical bounds that relate VaR, shortfall, and standard deviation, which indicate how big the error in evaluating VaR and shortfall can possibly be. We obtain closed form expressions for the gradient of shortfall with respect to portfolio weights as well as an alternative representation of shortfall, which shows that shortfall is a convex function of the portfolio weights, giving it an important advantage over VaR, which in general is not a convex function. This alternative expression also leads to an efficient sample mean-shortfall optimization algorithm in Section 5. We propose a natural non-parametric estimator of shortfall, which does not rely on any assumptions about the asset's distribution and is based only on historical data.
- 3.
In Section 4, we formulate the portfolio optimization problem based on mean-shortfall optimization and show that because of its convexity it is efficiently solvable. We characterize the mean-shortfall efficient frontiers and, in the case when a riskless asset is present, prove a two-fund separation theorem. We also define and illustrate a new, risk-level-specific beta coefficient of an asset relative to a portfolio, which represents the relative contribution of the asset to the portfolio shortfall risk. When the riskless asset is present, the optimal mean-shortfall weights are characterized by the equations having the CAPM form involving this risk-level-specific beta. For the elliptically symmetric, and in particular normal, distributions, the optimal mean-shortfall weights, the efficient frontiers, and the generalized beta for any risk level all reduce to the corresponding standard mean-variance portfolio theory objects. However, for more general multivariate distributions, the mean-shortfall optimization may lead to portfolio weights qualitatively different from the standard theory and, in particular, may considerably vary with the chosen risk level of the shortfall, see simulated and real data examples in Section 6.
- 4.
In Section 5, we show that the sample version of the population mean-shortfall portfolio problem, which is a convex optimization problem, can be formulated as a linear optimization problem involving a small number of constraints (twice the sample size plus two) and variables (the number of assets plus one plus the sample size). Uryasev and Rockafellar (1999) have independently made the same observation in the context of optimizing the conditional Value-at-Risk. This implies that the sample mean-shortfall portfolio optimization is computationally feasible for very large number of assets. Together with observations in item 3. above, this also importantly implies that the mean-shortfall optimization may be preferable to the standard mean-variance optimization, even if the distribution of the assets is in fact normal or elliptic, because in this case it leads to the efficient and stable computation of the same optimal weights and does not require the often problematic estimation of large covariance matrices necessary under the mean-variance approach.
- 5.
In Section 6, we present computational results suggesting that the efficient frontier in mean-standard deviation space constructed via mean-shortfall optimization outperforms the frontier constructed via mean-variance optimization. We also numerically demonstrate the ability of the mean-shortfall approach to handle cardinality constraints in the optimization process using standard linear mixed integer programming methods. In contrast, the mean-variance approach to the problem leads to a quadratic integer programming problem, a more difficult computational problem.
Section snippets
Definition and motivation of shortfall
In this section, we adopt the expected utility paradigm and theorems of stochastic dominance to motivate our definition of shortfall. Consider an investment choice based on expected utility maximization with an investor-specific utility function u(·). This means that an investment with random return X is preferred to the one with random return Y if E[u(X)]⩾E[u(Y)], where the expectations are taken with respect to the corresponding distributions of X and Y respectively.
As it is very difficult to
Properties of shortfall
The purpose of this section is to deepen our understanding of shortfall. We discuss its relation to other risk measures and outline various properties of shortfall.
The efficient shortfall frontier
In this section, we consider the mean-shortfall portfolio optimization problemwhere is the column vector of 1s. Problem (11) is defined analogously to the mean-variance portfolio optimization:
Because of the convexity of a solution of Problem (11) exists, and the graph of as a function of rp gives the minimum α-shortfall frontier. We next show that this frontier is a convex curve in the (rp,sα)
Sample mean-shortfall optimization
In this section, we outline efficient algorithms for the sample mean-shortfall optimization problem. The advantage of our approach is that we do not make any assumptions on the distribution of returns , but rather work directly with the historical data. We show that the sample mean-shortfall optimization can be formulated as a linear optimization problem involving a small number of constraints (twice the sample size plus two) and variables (the number of assets plus one plus the sample size).
Numerical examples
Our goal in this section is to shed light to the questions: (a) How different are the allocations produced by the mean-variance and mean-shortfall optimization under varying degree of asymmetry of the distribution of returns? and (b) How viable and effective is the idea that we can solve mean-variance quadratic optimization problems as linear optimization problems and in particular, in the presence of cardinality constraints?
We address the first question for symmetric return distributions and
Conclusions
We have shown that shortfall naturally arises as a measure of risk by considering distributional conditions for second-order stochastic dominance. We examined its properties and its connections with other risk measures. We showed that optimization of shortfall leads to a tractable convex optimization problem and to a linear optimization problem in its sample version. Interestingly, portfolio separation theorems as well natural definitions of beta can be derived in direct analogy to standard
Acknowledgements
We thank Chris Darnell for interesting discussions and providing us data for the asset allocation experiment reported in Section 6.3 and Stu Rosenthal for assistance with the computations in this section. We thank Roy Welsch for useful discussions and support. We thank the reviewers of the paper for several insightful comments. This research was partially supported by NSF grants DMS-9626348, DMS-9971579, DMI-9610486, and grants from Merill Lynch and General Motors.
References (32)
Optimal rule for ordering uncertain prospects
Journal of Financial Economics
(1975)- et al.
Capital market equilibrium in a mean-lower partial moment framework
Journal of Financial Economics
(1977) A characterization of the distribution that imply mean-variance utility functions
Journal of Economic Theory
(1983)- et al.
Variance vs. downside riskis there really that much difference?
European Journal of Operations Research
(1999) - et al.
Coherent measures of risk
Mathematical Finance
(1999) Safety-first, stochastic dominance, and optimal portfolio choice
Journal of Financial and Quantitative Analysis
(1978)- Bertsimas, D., Popescu, I., 1999. Optimal inequalities in probability: a convex programming approach. Working paper,...
- et al.
Option portfolio strategiesmeasurement and evaluation
Journal of Business
(1984) - Delbaen, F., 2000. Coherent risk measures on general probability spaces. Technical Report, ETH...
Beyond Value at Risk
(1998)
An overview of value at risk
The Journal of Derivatives
Mean-risk analysis with risk associated with below-target returns
The American Economic Review
Correlations in price changes and volatility across international stock markets
The Review of Financial Studies
Asset pricing in a generalized mean-lower partial moment frameworktheory and evidence
The Journal of Financial and Quantitative Analysis
Cited by (165)
Early Warning Systems for identifying financial instability
2023, International Journal of ForecastingEvaluation of the quantiles and superquantiles of the makespan in interval valued activity networks
2023, Computers and Operations ResearchEvaluation of VaR and CVaR for the makespan in interval valued blocking job shops
2022, International Journal of Production EconomicsCitation Excerpt :The assessment of the risk can be obtained taking advantage of forecasting of time attributes, under the assumption that they are described by means of prediction intervals (Agostino et al., 2020; Makridakis et al., 2008; Zurita et al., 2018). To this aim, risk measures often employed are the Value-at-Risk (VaR) and the Conditional Value-at-Risk (CVaR) (Bertsimas et al., 2004; Heckmann et al., 2015; Sarin et al., 2014; Tao et al., 2018). We deal with the evaluation of both VaR and CVaR for the makespan of BJSS schedules when the actual time attributes of jobs are uncertain at the scheduling or optimization stage and belong to a given interval provided by an available information/prediction system.
Risk of delay evaluation in real-time train scheduling with uncertain dwell times
2021, Transportation Research Part E: Logistics and Transportation ReviewManaging the shortfall risk of target date funds by overfunding
2024, Journal of Pension Economics and FinanceRisk-Efficient Coverage Selection Strategies for the Pasture, Rangeland, Forage (PRF) Insurance Program
2022, Journal of Agricultural and Applied Economics