Shortfall as a risk measure: properties, optimization and applications

https://doi.org/10.1016/S0165-1889(03)00109-XGet rights and content

Abstract

Motivated from second-order stochastic dominance, we introduce a risk measure that we call shortfall. We examine shortfall's properties and discuss its relation to such commonly used risk measures as standard deviation, VaR, lower partial moments, and coherent risk measures. We show that the mean-shortfall optimization problem, unlike mean-VaR, can be solved efficiently as a convex optimization problem, while the sample mean-shortfall portfolio optimization problem can be solved very efficiently as a linear optimization problem. We provide empirical evidence (a) in asset allocation, and (b) in a problem of tracking an index using only a limited number of assets that the mean-shortfall approach might have advantages over mean-variance.

Introduction

The standard deviation of the return of a portfolio is the predominant measure of risk in finance. Indeed mean-variance portfolio selection using quadratic optimization, introduced by Markowitz (1959), is the industry standard. It is well known (see Huang and Litzenberger, 1988 or Ingersoll, 1987) that the mean-variance portfolio selection paradigm maximizes the expected utility of an investor if the utility is quadratic or if returns are jointly normal, or more generally, obey an elliptically symmetric distribution.1 It has long been recognized, however, that there are several conceptual difficulties with using standard deviation as a measure of risk:

  • (a)

    Quadratic utility displays the undesirable properties of satiation (increase in wealth beyond a certain point decreases utility) and of increasing absolute risk aversion (the demand for a risky asset decreases as the wealth increases), see e.g., Huang and Litzenberger (1988).

  • (b)

    The assumption of elliptically symmetric return distributions is problematic because it rules out possible asymmetry in the return distribution of assets, which commonly occurs in practice, e.g., due to the presence of options (see, e.g., Bookstaber and Clarke, 1984). More generally, departures from ellipticity occur due to the greater contagion and spillover of volatility effects between assets and markets in down rather than up market movements, see, e.g., King and Wadhwani (1990), Hamao et al. (1990), Neelakandan (1994), and Embrechts et al. (1999). Asymmetric return distributions make standard deviation an intuitively inadequate risk measure because it equally penalizes desirable upside and undesirable downside returns. In fact, Chamberlain (1983) has shown that elliptically symmetric are the only distributions for which investor's utility is a function only of the portfolio's mean and standard deviation.

Motivated by the above difficulties, alternative downside-risk measures have been proposed and analyzed in the literature (see the discussion in Section 3.1.4). Though intuitively appealing, such risk measures are not widely used for portfolio selection because of computational difficulties and problems with extending standard portfolio theory results, see, e.g., a recent review by Grootveld and Hallerbach (1999).

In recent years the financial industry has extensively used quantile-based downside risk measures. Indeed one such measure, Value-at-Risk, or VaR, has been increasingly used as a risk management tool (see e.g., Jorion, 1997; Dowd, 1998; Duffie and Pan, 1997). While VaR measures the worst losses which can be expected with certain probability, it does not address how large these losses can be expected when the “bad”, small probability events occur. To address this issue, the “mean excess function”, from extreme value theory, can be used (see Embrechts et al., 1999 for applications in insurance). More generally, Artzner et al. (1999) propose axioms that risk measures (they call them coherent risk measures) should satisfy and show that VaR is not a coherent risk measure because it may discourage diversification and thus violates one of their axioms. They also show that, under certain assumptions, a version of the mean excess function, which they call tail conditional expectation (TailVaR), is a coherent measure (see Section 3.1.5 below).

Our goal in this paper is to propose an alternative methodology for defining, measuring, analyzing, and optimizing risk that addresses some of the conceptual difficulties of the mean-variance framework, to show that it is computationally tractable and has, we believe, interesting and potentially practical implications.

The key in our proposed methodology is a risk measure called shortfall, which we argue has conceptual, computational and practical advantages over other commonly used risk measures. It is a variation of the mean excess function and TailVaR mentioned earlier (see also Uryasev and Rockafellar, 1999). Some mathematical properties of the shortfall and its variations have been discussed in Uryasev and Rockafellar (1999), and Tasche (2000).

Our contributions and the structure in this paper are:

  • 1.

    We define shortfall as a measure of risk in Section 2, and motivate it by examining its natural connections with expected utility theory and stochastic dominance.

  • 2.

    We discuss structural and mathematical properties of shortfall, including its relations to other risk measures in Section 3. Shortfall generalizes standard deviation as a risk measure, in the sense that it reduces to standard deviation, up to a scalar factor depending only on the risk level, when the joint distribution of returns is elliptically symmetric, while measuring only downside risk for asymmetric distributions. We point out a simple explicit relation of shortfall to VaR and show that it is in general greater than VaR at the same risk level. Moreover, we provide exact theoretical bounds that relate VaR, shortfall, and standard deviation, which indicate how big the error in evaluating VaR and shortfall can possibly be. We obtain closed form expressions for the gradient of shortfall with respect to portfolio weights as well as an alternative representation of shortfall, which shows that shortfall is a convex function of the portfolio weights, giving it an important advantage over VaR, which in general is not a convex function. This alternative expression also leads to an efficient sample mean-shortfall optimization algorithm in Section 5. We propose a natural non-parametric estimator of shortfall, which does not rely on any assumptions about the asset's distribution and is based only on historical data.

  • 3.

    In Section 4, we formulate the portfolio optimization problem based on mean-shortfall optimization and show that because of its convexity it is efficiently solvable. We characterize the mean-shortfall efficient frontiers and, in the case when a riskless asset is present, prove a two-fund separation theorem. We also define and illustrate a new, risk-level-specific beta coefficient of an asset relative to a portfolio, which represents the relative contribution of the asset to the portfolio shortfall risk. When the riskless asset is present, the optimal mean-shortfall weights are characterized by the equations having the CAPM form involving this risk-level-specific beta. For the elliptically symmetric, and in particular normal, distributions, the optimal mean-shortfall weights, the efficient frontiers, and the generalized beta for any risk level all reduce to the corresponding standard mean-variance portfolio theory objects. However, for more general multivariate distributions, the mean-shortfall optimization may lead to portfolio weights qualitatively different from the standard theory and, in particular, may considerably vary with the chosen risk level of the shortfall, see simulated and real data examples in Section 6.

  • 4.

    In Section 5, we show that the sample version of the population mean-shortfall portfolio problem, which is a convex optimization problem, can be formulated as a linear optimization problem involving a small number of constraints (twice the sample size plus two) and variables (the number of assets plus one plus the sample size). Uryasev and Rockafellar (1999) have independently made the same observation in the context of optimizing the conditional Value-at-Risk. This implies that the sample mean-shortfall portfolio optimization is computationally feasible for very large number of assets. Together with observations in item 3. above, this also importantly implies that the mean-shortfall optimization may be preferable to the standard mean-variance optimization, even if the distribution of the assets is in fact normal or elliptic, because in this case it leads to the efficient and stable computation of the same optimal weights and does not require the often problematic estimation of large covariance matrices necessary under the mean-variance approach.

  • 5.

    In Section 6, we present computational results suggesting that the efficient frontier in mean-standard deviation space constructed via mean-shortfall optimization outperforms the frontier constructed via mean-variance optimization. We also numerically demonstrate the ability of the mean-shortfall approach to handle cardinality constraints in the optimization process using standard linear mixed integer programming methods. In contrast, the mean-variance approach to the problem leads to a quadratic integer programming problem, a more difficult computational problem.

Section snippets

Definition and motivation of shortfall

In this section, we adopt the expected utility paradigm and theorems of stochastic dominance to motivate our definition of shortfall. Consider an investment choice based on expected utility maximization with an investor-specific utility function u(·). This means that an investment with random return X is preferred to the one with random return Y if E[u(X)]⩾E[u(Y)], where the expectations are taken with respect to the corresponding distributions of X and Y respectively.

As it is very difficult to

Properties of shortfall

The purpose of this section is to deepen our understanding of shortfall. We discuss its relation to other risk measures and outline various properties of shortfall.

The efficient shortfall frontier

In this section, we consider the mean-shortfall portfolio optimization problemminimizesα(x)subjecttoxμ=rp,ex=1,where e is the column vector of 1s. Problem (11) is defined analogously to the mean-variance portfolio optimization:minimizexΣxsubjecttoxμ=rp,ex=1.

Because of the convexity of sα(x) a solution xα(rp) of Problem (11) exists, and the graph of sα(rp)=sα(xα(rp)) as a function of rp gives the minimum α-shortfall frontier. We next show that this frontier is a convex curve in the (rp,sα)

Sample mean-shortfall optimization

In this section, we outline efficient algorithms for the sample mean-shortfall optimization problem. The advantage of our approach is that we do not make any assumptions on the distribution of returns R, but rather work directly with the historical data. We show that the sample mean-shortfall optimization can be formulated as a linear optimization problem involving a small number of constraints (twice the sample size plus two) and variables (the number of assets plus one plus the sample size).

Numerical examples

Our goal in this section is to shed light to the questions: (a) How different are the allocations produced by the mean-variance and mean-shortfall optimization under varying degree of asymmetry of the distribution of returns? and (b) How viable and effective is the idea that we can solve mean-variance quadratic optimization problems as linear optimization problems and in particular, in the presence of cardinality constraints?

We address the first question for symmetric return distributions and

Conclusions

We have shown that shortfall naturally arises as a measure of risk by considering distributional conditions for second-order stochastic dominance. We examined its properties and its connections with other risk measures. We showed that optimization of shortfall leads to a tractable convex optimization problem and to a linear optimization problem in its sample version. Interestingly, portfolio separation theorems as well natural definitions of beta can be derived in direct analogy to standard

Acknowledgements

We thank Chris Darnell for interesting discussions and providing us data for the asset allocation experiment reported in Section 6.3 and Stu Rosenthal for assistance with the computations in this section. We thank Roy Welsch for useful discussions and support. We thank the reviewers of the paper for several insightful comments. This research was partially supported by NSF grants DMS-9626348, DMS-9971579, DMI-9610486, and grants from Merill Lynch and General Motors.

References (32)

  • D. Duffie et al.

    An overview of value at risk

    The Journal of Derivatives

    (1997)
  • Embrechts, P., McNeil, A., Straumann, D., 1999. Correlation and dependency in risk management: properties and pitfalls....
  • P.C. Fishburn

    Mean-risk analysis with risk associated with below-target returns

    The American Economic Review

    (1977)
  • Garman, M., 1997. Taking var to pieces. Risk (10),...
  • Y. Hamao et al.

    Correlations in price changes and volatility across international stock markets

    The Review of Financial Studies

    (1990)
  • W.V. Harlow et al.

    Asset pricing in a generalized mean-lower partial moment frameworktheory and evidence

    The Journal of Financial and Quantitative Analysis

    (1999)
  • Cited by (165)

    • Early Warning Systems for identifying financial instability

      2023, International Journal of Forecasting
    • Evaluation of VaR and CVaR for the makespan in interval valued blocking job shops

      2022, International Journal of Production Economics
      Citation Excerpt :

      The assessment of the risk can be obtained taking advantage of forecasting of time attributes, under the assumption that they are described by means of prediction intervals (Agostino et al., 2020; Makridakis et al., 2008; Zurita et al., 2018). To this aim, risk measures often employed are the Value-at-Risk (VaR) and the Conditional Value-at-Risk (CVaR) (Bertsimas et al., 2004; Heckmann et al., 2015; Sarin et al., 2014; Tao et al., 2018). We deal with the evaluation of both VaR and CVaR for the makespan of BJSS schedules when the actual time attributes of jobs are uncertain at the scheduling or optimization stage and belong to a given interval provided by an available information/prediction system.

    • Risk of delay evaluation in real-time train scheduling with uncertain dwell times

      2021, Transportation Research Part E: Logistics and Transportation Review
    • Managing the shortfall risk of target date funds by overfunding

      2024, Journal of Pension Economics and Finance
    View all citing articles on Scopus
    View full text