Performance Measure – Don’t Stand So Close To Sharpe
Universidad de Alicante
Universidad CEU Cardenal Herrera – Department of Business and Economics
University of Alicante
June 9, 2016
Abstract:
We analyze the use of alternative performance measures for ranking and selecting assets. Previous literature on performance evaluation is basically centered on studying the effects of non-normality on rank correlations between orderings. Instead, we analyze the out-of-sample returns and investigate the economic reasons for their differences of portfolios that include the assets selected according to each performance measure. The overall empirical findings show that the performance measure is definitively relevant for the posterior portfolio returns. Particularly good results are found for the assets selected by the Farinelli-Tibiletti (FT) ratio. The FT-based portfolio return distribution dominates stochastically the others and produces high differential cumulative returns after the 2008 downturn. We find that the ordering recommended by the FT ratio is consistent with a rebalancing (counter-cyclical) investment strategy.
Don’t Stand So Close To Sharpe – Introduction
The securities selection is one important part of the investment process for which the so-called screening rules are useful. These rules aim to restrict the investment universe to a reasonably limited set of assets with the best characteristics but without specifying assets allocations. Performance measures (PMs, hereafter) are examples of screening rules.
Most PMs used for ranking assets are risk?reward ratios since this is a very important concern for risk managers. The well?known Sharpe (1966) ratio, that relates the mean return with the standard deviation, has been used as a standard for this aim. It is based on the meanvariance paradigm which requires either elliptical1 (e.g. Gaussian distribution) returns or quadratic preferences. However, it is well documented that deviations from normality of some financial asset return distributions are statistically significant and then, in such cases, the standard deviation underestimates the total risk and generates biased investment rankings. Therefore, ratios that consider a more general framework such as, among others, one?sided reward?risk measures have been proposed.2 Simultaneously, the debate about the significance in investment applications of these new PMs regarding the Sharpe ratio (SR) is still open.
The usual way to compare the use of different PMs as screening rules is based on the Spearman correlations between the rankings that these measures generate. Although the information provided by each PM could be different, the correlation between two rankings might be large. If this were the case, one of the two measures might be redundant as a screening rule. Results from papers that compare PM rank correlations induce controversial conclusions. On the one hand, Eling and Schuhmacher (2007) or Eling (2008), among others, conclude that the PM choice becomes irrelevant since they all produce very similar rankings. However, these papers relay on a small subset of PMs and for a specific class of assets: mutual or hedge funds. Guo and Xiao (2015) reinforce this previous result showing that if return distributions belong to the location?scale family, then different PMs generate identical rank ordering. On the other hand, other studies show that rankings can be very different depending on the selected measure. Zakamouline (2011) shows that severe deviations of normality lead to significant shifts in the rankings for hedge funds. León and Moreno (2016), by assuming the Gram?Charlier distribution for the stock returns, also agree that the selection of PMs becomes relevant. Caporin and Lisi (2011) find evidence of low rank correlations by using a huge set of different PMs and argue that the results depend on both the type of assets and the sample period. Additionally, they also show that the rank correlations are time varying and influenced by the sample size. Finally, Magron (2014), using a sample of 24,766 individual investors from a French brokerage, shows that alternative PMs to the SR, and specifically the Farinelli?Tibiletti family, result in different rankings.
Our contribution is on the understanding of the economics behind the selection of assets that each PM does. We do not pay attention to the rankings but we analyze the consequences of different rankings on the investment results by looking at the out?of?sample (OOS) returns of the portfolios that contain the assets recommended by each PM. On the one hand, we analyze if different portfolios (PMs) generate different subsequent returns. In this way, our study complements previous empirical evidence about the relevance of using PMs more general than the standard Sharpe ratio. On the other hand, we investigate the economic reasons that explain different OOS returns. As far as we know, this is the first paper providing results that link the statistical properties embedded in the PMs with the basics of asset pricing.
Specifically, we consider 32 PMs computed daily for all stocks in the Standard and Poor (S&P) 500 index by using a 264?days rolling window of past returns. The individual stocks are daily ranked on the basis of each PM and the 5 percent best stocks are selected and combine equally?weightily in a portfolio.3 Then, the next day return is computed for each portfolio. Caporin and Lisi (2011) point out the need of working in a dynamic framework because of the instability of PMs over time. Therefore, the rolling window approach is an additional goal of our paper. And we employ a sufficiently large window to avoid inconsistencies due to the sample size.
First, we compare the portfolios (PMs) looking at the percentage of individual stocks that are selected simultaneously by two of them. It is shown that the portfolio composition is rather similar across portfolios based on measures from the same family. This fact allows for reducing the number of portfolios (and the dimension of results) by applying the principal component technique to all portfolios in each family and projecting the OOS returns. In contrast, we find a high distance between the portfolios compositions obtained through PMs from different families.
Second, we compare the characteristics of the distribution of the out?of?sample (OOS) returns between different portfolios in several ways. We analyze the descriptive statistics for the whole OOS period; we implement the dynamic conditional correlation (DCC) model of Engle (2002) between pairs of OOS portfolio returns; we analyze the spread in cumulative returns between each portfolio and the benchmark (the portfolio based on the Sharpe ratio); and, finally, complete empirical distributions of returns are compared by stochastic dominance. All the results indicate that the screening rule influences the subsequent portfolio returns that are, in some portfolios, quite different from the benchmark (the Sharpe ratio).
See full PDF below.
