Sample size matters

Post with image

Trying to predict the result of rolling a single dice is notoriously difficult, but predicting the average result from 1 000 000 dice is much easier.

This is the essence of the ‘law of large numbers’ discovered by statisticians as far back as the 16th century: the larger the sample size, the more it will resemble the average. Statisticians are very familiar with this effect, and most are careful to use large samples when results are tight.

In financial markets, we repeatedly see the same kind of effect: small samples of securities or market conditions are full of surprises, while larger samples generate results that are closer to predictions. For instance, the equity risk premium states that on average, investing in equities is worthwhile over the long term (large sample), but equities can still go down for a few years in a row (small sample). Value stocks (large sample) might outperform expensive stocks on average, but Apple* (small sample) might be a counter example.

One consequence of this ‘law of large numbers’ is that, in general, a quantitative strategy works better in broader asset universes. A larger universe of stocks being closer to the ‘law of large numbers’ in its behaviour than a reduced universe.

Below we use our Diversified Equity Factor Investing (DEFI) strategy to demonstrate this principle. This is a multi-factor equity strategy where the portfolio is exposed to value, quality, low-risk and momentum factors, each contributing equally to the tracking error of the strategy. Exhibit 1 shows the back-tested Information Ratio of the DEFI strategy applied to different global developed stock universes. The reduction in the strategy’s efficacy resulting from a splitting of the universe is clear.

Exhibit 1: Reduction of Information Ratio when splitting the universe

sample size

Source: BNP Paribas Asset Management, as of January 2017

More recently, however, psychological science added another finding. In ‘Belief in the law of small numbers’, an article published in the Psychological Bulletin in 1971, Amos Tversky explains an interesting cognitive bias of the human species: we tend to underestimate the importance of the law of large numbers and to believe, against all statistical evidence, that even evidence based on small numbers is enough to assess an average value.

In asset management, the effects of this ‘law of small numbers’ are manifold. The most obvious is that it shortens the horizon that investors deem necessary to judge the efficacy of investment strategies, compared to what should be significant. For instance, five years of live performance is considered to be good practice in selecting managers, although 60 months is not only short but can typically be restricted to just one type of market regime. At the moment, for instance, five years of history consists essentially of a bull market for equities, giving an advantage to riskier high beta equity investment strategies.

Another effect is the industry obsession with short-termism, expecting frequent explanations for short-term (under)performances. Even if month-to-month performances are mostly explained by the noise of small samples, most investors still expect those short-term performances to be in line with the global behaviour of the strategy, and thus expect the manager to take action if it underperforms, when in fact patience is the best strategy.


Written on 7 February 2017 in Paris

 See also: Performance Measures – CFP | Investopedia

* The above mentioned securities are for illustrative purpose only and do not constitute any investment recommendation.

Etienne Vincent

Head of Global Quantitative Management, THEAM

Leave a reply

Your email adress will not be published. Required fields are marked*