High Sigma Events—They’re Not All Black Swans

Updated: Jul 27th, 2016 | Vance Harwood

After every crash or major geopolitical event that roils the market we are exposed to graphics like this one containing sigma numbers:

Black Swan


The message associated with these charts is usually, “We should be very worried because the events that just occurred were really unlikely.”

The reader, on the other hand, should be thinking: the person that wrote this really doesn’t understand statistics or Black Swans.

What is Sigma?

The common usage of the term “sigma” in statistics stems from the use of the lower-case Greek letter sigma to denote volatility.  A standard deviation calculation is usually used to compute volatility, so sigma and one standard deviation have become interchangeable.  A standard deviation is an always positive measure of how much variation there is in a data set. If the data doesn’t move around much (e.g., daily high temperature in Hawaii) the standard deviation will be low.  If the data moves around a lot (e.g., the CBOE’s VIX), then the standard deviation will be higher.

The calculation (slightly simplified) of the standard deviation is not all that scary:

  1. Compute the average of the data set
  2. Subtract the average from every data point (which will give you both positive and negative differences) and then square it.
  3. Average the squared differences for all the data points, and then take the square root

Once computed the standard deviation provides a succinct way to describing how widely a specific data point differs from the average. For example, if the standard deviation of daily high temperatures in Hawaii is 4 degrees, then a cold day with a high 12 degrees below average would be a 3 standard deviation, or “3 Sigma” event.

Sigma and the Normal Distribution—Not Siamese Twins

The standard deviation calculation makes NO assumptions about how the data points are arranged around the average—what statisticians call the distribution. The standard deviation calculation does not assume, or require, that the distribution is normal (Gaussian) or any one of the many other different possible distributions (e.g., Laplace, Cauchy, Parabolic Fractal).

Unfortunately, even though there’s no mathematical linkage between the standard deviation and the normal distribution, most people assume the two are locked together.  For example, you may have heard of “6-sigma quality”, which strives to achieve quality levels of fewer than 3.4 defective features per million opportunities. This percentage assumes a normal distribution—not a bad assumption for flaws in a complex manufacturing process. However if the distribution of failures happens to be Laplacian, then 6-sigma quality might exhibit 860 defective features per million–not such a great achievement.

Nassim Taleb in his book “The Black Swan” calls this tendency to assume the normal distribution the Ludic Fallacy.  He suggests this belief stems from people inappropriately extending the structured randomness of games and gambling into the real world processes where complexity and human psychology reign.

Financial Indexes are Typically Not Normally Distributed

Most of the time variations in financial markets look like what you’d expect from a normal distribution.  The chart below shows returns of the S&P 500 since 1950 compared to what the normal distribution predicts.


Although the central peak of the historic data is higher than the normal distribution, and the shoulders are a bit narrower, the match is usually considered good enough. Financial analysts usually assume the normal distribution because they are familiar with it and there’s a rich tool set for it (e.g., Excel supports it directly with built-in functions).  For typical situations it works well enough.

The Devil is in the Assumptions

 If a process has 30 or more sub-processes or components within it and these components are independent from each other then it’s very likely the distribution of that process will be normal.

While it’s common for financial markets or metrics to have 30 or more components it’s incorrect to assume they’re independent—especially when emotions are running high.  In very stressful times market participants abandon any pretense of independence and start moving in lockstep. The chart below shows the correlation (a statistical measure of independence) between two exchange traded funds that track sectors in the S&P 500. A correlation of one means the sectors are in lockstep, a correlation of zero indicates they’re independent.

XLI-XLP-correl histo

As you can see the sectors are usually not independent.  And as a result of these correlations, the S&P’s returns are not normally distributed. Not only is the central peak too high, and the shoulders too narrow, but the tails of the distribution on both the positive and negative side (impossible to see on most histograms) are much fatter than the normal distribution.

So if the normal distribution is not a good match to financial returns which one should we be using?

Using the Laplace Distribution to Assess Probabilities

Mathematically it’s pretty easy to come up with distributions that are better matched to the S&P’s distribution than the normal, but it’s not possible to prove which one is best—there’s just not enough data to settle the matter.

One distribution that works well is the Laplace distribution (which I describe here). It has a good match to historical data and it serves to illustrate how different some distributions are from the normal. The chart below shows how the classic Laplace distribution is a better fit to the central peak of the historic S&P 500 data.

Act vs Normal vs Classic Laplace


The big disconnects between the historic data and the normal distribution are in the tails.  The table below shows the differences for a range of sigma levels. The “Matched Laplace” distribution used is one that’s tuned to better match the tails of the S&P 500 distribution.

Greater Than
Plus / Minus
Sigma Levels
Expected Frequency

S&P 500

Matched Laplace

Expected Frequency



>+-1 76 events per year 80 events per year
>+-2 23 events per year 12 events per year
>+-3 7 events per year 0.7 trading events per year
>+-4 2.2 events per year 0.016 events per year
>+5 7 events in 10 years 0.0014 events in 10 years
>+-6 2 events in 10 years 5×10-6 times in 10 years
>+-7 6  events in 100 years 6×10-8 times in 100 years
>+-8 2 events in 100 years 3×10-11 times in 100 years
>+-9 6 events in 1000 years 6×10-14 times in 1000 years
>+-10 2 events in 1000 years 4×10-18 times in 1000 years
>+-11 5 events in 10,000 years 9×10-22 times in 10,000 years
>+-12 2  events in 10,000 years 9×10-27 times in 10,000 years


For sigma levels lower than four the normal and Laplacian distributions aren’t dramatically different, but the expected frequencies diverge rapidly as we evaluate higher sigma levels. For events with sigmas as low as six or seven, the normal distribution predicts their odds of occurrence as incredibly low. Even with the Laplace distribution the probability of a sigma event higher than seven or eight is unlikely in a lifetime.

 Interpreting Sigma Events Above Seven

Common probability distributions predict that events with sigma levels above seven are very unlikely—and yet we see lots of them.   I think the potential causes of these events fall into the following four categories:

  • System Failures
  • Externally Driven
  • Low Liquidity Metrics
  • Stuff Happens

System Failures (e.g., Flash Crashes)

 The 2010 Flash Crash and the 1987 Black Monday Crash (21 sigma) were both system failures where the combination of flawed technology and run-away human emotion triggered unprecedented downswings.  In both cases the market makers stepped away from the market during the worst times—exacerbating an already bad situation.

Everyday variations in markets can’t be used to predict system failures—there’s no connection between those variations and the things (e.g., software defects, unanticipated interactions between subsystems) that cause system crashes. When systems do crash the obvious questions are how much damage was done, what was the root cause, and what can be done to prevent future failures.  While huge amounts of money are lost during these failures (at least on paper), some people benefit (e.g., Taleb made his first fortune from the 1987 crash) and the long-term effects are often minor. Despite the 22% drop during the October 19th crash the 1987 market ended up with a small overall gain for the year.

Externally driven (e.g., Forex shocks, Bank Rates)

In 2015 when the Swiss National Bank stopped trying to cap the Swiss Franc exchange rate to the Euro the Franc jumped 19% (180 sigma) essentially instantaneously relative to the Euro. The British Pound dropped 8.1% (15 sigma) in the confusion after the Brexit vote.  The everyday volatility of foreign exchange rate or an overnight interbank lending rate is clearly not a good predictor of the macroeconomic forces that ultimately drive those rates.  It’s like trying to use daily vibration levels in a car to predict auto accidents.

Low Liquidity Metrics (e.g., VIX, PE ratios)

The CBOE’s VIX index is fertile ground for producing high sigma events. Prognosticators pore over single and multiple day increases looking and often finding high sigma events useful for supporting their oft-repeated contention that the system is about to collapse.  Not only does the VIX spike up frequently, it often collapses after emotions subside—offering even more high sigma events to work with.

A key characteristic of the VIX is that unlike stocks or currencies it impossible to invest in it directly.  The best you can do to follow the VIX is trade large quantities of SPX options spread over dozens, or even hundreds of strike prices.  Things are further complicated by the fact that the VIX is calculated using mid prices, halfway between ask and bid prices, but real trades aren’t constrained to trade at that point.  Especially during times of market stress buyers must pay significantly more, and sellers get significantly less than the midpoint price.

Because there’s not an easy way to hedge or monetize it, the VIX acts like a market with low liquidity.  High demand during fearful times (usually for S&P 500 puts) rapidly drives up the index, and when the crisis has passed it collapses.

It hasn’t happened yet, but I wouldn’t be surprised to see a daily +100% move in the VIX (15 sigma).  The index can get as low as 10 or 12 when the market is quiet and has averaged around 20 over the long term.   A major geopolitical event / disaster after the VIX closes at a low level could quite conceivably result in the VIX opening in the 20 to 24 range—doubling in value and yet very close to its long-term average.

The VIX generates high sigma events when people rush to insure their portfolios, and also when they lose interest and wander off.  This behavior offers entertainment value but is not particularly informative, and definitely not predictive.

Stuff Happens

Physicists have a saying: “If something can happen it will.”  Given enough time low probability events will happen.  Thirty-two consecutive reds occurred on a roulette table in 1943—the odds of that happening are around 24 billion to one.   However, I’m not aware of any high sigma financial event that is best explained as a truly random event.  All the ones I’ve looked at appear to be explained best by one or more of the first three causes mentioned.

Are High Sigma Events Ever Predictive?

At least one high sigma event did foreshadow the 2008 meltdown—the 2007 jump in the 3-Month LIBOR OIS spread (6-sigma).  In Jack Schwager’s book Hedge Fund Market Wizards there’s a fascinating interview with Colm O’Shea. When the LIBOR spike occurred (more than a year before the 2008 travails) O’Shea saw it as a clear signal of the beginning of the end for the housing / financial bubble and shifted his investments.

It’s prudent to evaluate high sigma events as possible leading indicators of troubles ahead but recognize that virtually all these events reflect disruptions that have just occurred or are meaningless noise.

High Sigma Exhaustion

It’s an inherent survival skill to notice unusual events. For a savanna dweller, an atypical movement in the savanna grass might be a snake or a lion—well worth some focused attention. In the same way, investors should be monitoring their financial environment for threats. But focused attention requires a lot of energy—always a scare resource.

In the financial world attending to too many cries of “Wolf!!” will just leave us exhausted, paranoid, and vulnerable. We need to filter the cacophony of alerts/alarms we receive to determine real significance.


  • Are probabilities being assessed correctly? Most financial processes are not normally distributed.
  • Is the event a reasonable response to a system failure or a macroeconomic development?
  • Is the new value extreme, or is it within historic ranges? If values remain within historic ranges it’s likely that the damage isn’t that severe.
  • Is there an informed analysis showing the linkage between the event and real economic factors? If not, then you should suspect there’s an ideology being pushed, often unaccompanied by any coherent economic analysis or any accountability on why an oft-repeated prediction is being rolled out once again.

Being Antifragile

Saving some energy by filtering will allow us to focus on what Taleb recommends relative to Black Swans—rather than try to predict them, figure out how to prevent them, or mitigate their effects.  Build systems/portfolios that are tolerant of failures/unexpected events (Taleb calls systems with this attribute “antifragile”).

And probably most importantly, don’t assume something won’t happen because the computed probabilities are very low.  Those computations often severely underestimate the true risk.  We only need to look the failures of Long-Term Capital Management and the collapse of the Mortgage Backed Security market in 2008 to see the impact of underestimating risk.

Monthly and Yearly Decay Rates for Long Volatility Funds

Updated: Sep 29th, 2016 | Vance Harwood

While it’s certain that short-term volatility exchange traded products (ETPs) like VXX, TVIX, and UVXY are doomed to march towards zero, their decay rates are not consistent. Things like bear markets and big corrections can cause big upward swings. On the downside, the term structure of VIX futures and the volatility of volatility can significantly impact decay rates in monthly and yearly time frames.

The charts below give a graphical history of their decay rates. With each of them I’m making trade-offs between reducing noise (averaging out normal market action) and the ability to see medium term trends. Each data point on the first chart is computed using the cumulative gain or loss over the previous six months and a monthly compounding period.


1 + 2X vol monthly

None of these ETPs existed before 2009 (the first one, VXX started in January 2009), so I used my simulations based on historical VIX futures data to generate the pre-inception data.

The yearly percentage chart below uses a monthly moving average for all the price points and then uses the result for each date and the price a year prior to compute the simple percentage increase or decrease.

Vol Yearly Percent Loss


Despite eye-watering decay rates these funds are increasing in popularity.  I’m not sure why (except for the possibility of shorting them).

They are one of the few investments that will reliably go up when the market has a significant drop, but unless your timing is exquisite the decay rates on these ETPs will make you wish you never laid eyes on them.


Quant Corner

  1. The formula used in the first chart is the classic compounding equation:   Compounded Growth = ((End Price/ Start Price)^(1/Period))-1   To obtain monthly compounded returns I use a start price 6 months prior to the end price.  The start price I use is the average of the prices 21 days prior to and including the start date.  The period used in the formula is, of course, six. I multiplied the results by negative one in order to make decay rates show up as positive numbers.
  2. The second chart uses moving average prices (22 trading days) and computes the percentages using End Price / Start Price -1, with the start price one year previous to the end price.  I multiplied the results by negative one in order to make decay rates show up as positive numbers.

Guest Post: Modified Ned Davis Method—April 2016 Update, by Frank Roellinger

Updated: Apr 25th, 2016 | Vance Harwood

Over the years I have developed a lot of respect for the condition of breadth divergence, when an index such as the S&P 500 is rising and the NYSE daily cumulative advance-decline line is not.  In January 2016 I listed the performance of my method on the short side as a function of the number of consecutive weeks of divergence at the time of the short signal.  The optimum interval was 41 to 79 weeks, and as that interval was being approached, I speculated that a profitable short could be on the horizon.

It didn’t work out that way.  The last short began on December 11, 2015 at 31 consecutive weeks of divergence.  That short ended with a 3.8% profit on Feb. 26, 2016 and the method returned to 100% long.  That trade is now ahead by about 10.6%, but what may be more significant is that divergence ended on April 22, 2016. Thus a new short beginning any time soon does not have an especially good chance of ending profitably (but of course I will take it—one never knows for sure about these things).  See this post for my track record in calling corrections.

Might divergence ending after such an extended period (49 weeks as of April 15, 2016) have bullish implications?  I wondered, so checked, and discovered the following dates at which divergence periods from 31 to 109 consecutive weeks had ended.  I then noted how long it had been from that date to the date of the next small cap market peak which had been followed by a decline of 19% or more.  Results are as follows:

Small Cap
17-Feb-61 1-Dec-61 287 days
17-May-63 22-Apr-66 2 Years 341 days
25-Sep-64 22-Apr-66 1 Year 209 days
10-Dec-76 8-Sep-78 1 Year 272 days
19-Sep-80 12-Jun-81 266 days
22-Oct-82 24-Jun-83 245 days
15-Feb-85 21-Aug-87 2 Years 187 days
30-Jan-87 21-Aug-87 203 days
28-Apr-89 6-Oct-89 161 days
30-Aug-91 17-Apr-98 6 Years 232 days
24-Dec-92 17-Apr-98 5 Years 115 days
2-May-03 13-Jul-07 4 Years 73 days
24-Jul-09 23-Apr-10 273 days

There was quite a variation, from slightly over 5 months to more than 6 years.  The amount of gain also varied widely, suggesting that an average or median of either would be of questionable value.  However, the import clearly seems to be that breadth reaching a new high after a considerable period of deterioration is a bullish sign.

Another bullish sign has been daily NYSE breadth since the February 11, 2016 low.  There are several ways to measure this, but one meaningful way is to sum the number of advancing issues and the number of declining issues over N days and then compute the ratio of the former sum over the latter.

On April 22, 2016 this ratio was 1.46.  Since 1948, only 5 times has this ratio been this high or higher:


Market historians will recognize all of these as times when significant bull markets had been established and outsized gains lay ahead before the next severe bear market began; each time the beginning of that bear phase was a long way off.

Of course, none of this can be any guarantee of what the market will do, but over the years I have found breadth to be one of the best indicators available.  I am confident to be 100% long now, but as always, I have my eye on the exit sign.

Below is the latest chart of my method’s performance from 1960 through April 22, 2016 (click to enlarge).

MDM 042416


Related Posts

Predicting Stock Market Returns—Lose the Normal and Switch to Laplace

Updated: Jun 24th, 2016 | Vance Harwood

Everyone agrees the normal distribution isn’t a great statistical model for stock market returns, but no generally accepted alternative has emerged.  A bottom-up simulation points to the Laplace distribution as a much better choice.

A well-known problem in financial risk assessment is the failure of the normal distribution (also known as the Gaussian distribution) to correctly predict big up or down days on the stock market.  Even though these volatile days are infrequent they can make a big difference in the performance of an investment portfolio.  At least publically the financial industry has not moved on to better models, probably because no alternative has been accepted as superior.

The Issue

 My interest in this area stems from the frequent use of sigma notation by commenters when the market experiences big swings.  For example, there was a multi-day10 sigma increase in the value of the VIX index associated with the August 2015 correction and an 8 sigma single day downswing a couple of weeks later.

The term “sigma” is equivalent to the statistical term “standard deviation”, one of the two key descriptors in a distribution (the other is the mean or average).  Sigma can be used as a shorthand way of indicating the relative magnitude of a market move.  For example, if the S&P 500 drops 2.92% in a day (doubtless inciting headlines with the word “crash” and “depression” in them) we can determine the sigma level of this event as three by dividing the percentage drop (2.92%) by the S&P’s historic standard deviation (0.973%).

The normal distribution assesses the odds of a -3 sigma day like this at 0.135%, which assuming a 252 day trading year predicts a drop this size or greater should occur about once every 3 years of trading.

The odds associated with 8 to 10 sigma events for a normal distribution are truly mind boggling.  The chart below illustrates how often events of various sigma levels should be expected.


Plus / Minus Sigma Level Probability of occurring on any given day How often event is expected to occur Associated S&P 500 percentage move Actual S&P 500 occurrences (Jan 1950-2016) vs (expected from normal distribution)
>+-1 31.73% 80 trading days per year +-0.973% 3534  (expected 5276)
>+-2 4.56% 12 trading days per year +-1.95% 776    (expected 758)
>+-3 0.27% 1 event every 8 months +-2.92% 229    (expected 44)
>+-4 6.33×10-3 % Once in 62 years +-3.89% 98   (expected 1)
>+5 5.73×10-5 % One in 6900 years +-4.86% 50   (expected 0)
>+-8 1.22×10-13 % Once in 3.2 trillion years +-7.78% 8     (expected 0)
>+-9 2.25×10-17 % Twice in 20000 trillion years +-8.76% 7     (expected 0)
>+-10 1.53×10-21 % Once in 2.6 x 10^20 years +-9.73% 3     (expected 0)


When high sigma events do occur analysts often use their outrageous unlikeliness to promote their agenda—usually predicting the imminent collapse of the financial system and/or indict their favorite bad guys (e.g., Federal Reserve, Evil Bankers) for manipulating the markets.

I disagree.  The reasonable conclusion from seeing 5 or higher sigma events in the markets should not be that things are falling apart or that the market is rigged, instead, we should recognize that we are using the wrong probability model.


Where the Normal Distribution Works—And Where it Doesn’t

Graphically comparing the S&P 500 returns since 1950 with the matched normal distribution looks like this:



The actual returns have a higher central peak and the “shoulders” are a bit narrower, but the historic frequency of events occurring within the +-3 sigma range isn’t significantly different from what is predicted by the normal distribution. The big mismatches are in the tails of the distribution—which you really can’t see on a linear scale. Changing the vertical axis to use logarithmic scaling allows us to zoom in on the tails.



Once you get past +-4 sigma moves it’s hard to visualize how vast the differences are between the S&P 500 actual returns and the predictions of the normal distribution. The vertical dotted black line on the right side of the chart illustrates the problem.  In the last 67 years, the S&P has had two days with returns between +5.5% and +5.6%.  The normal distribution estimates the probability of one event of that magnitude as 0.0001 in 67 years (where the dotted black line touches the red line) —in other words, we should expect a move near +5.5% once in 67,000 years on average—and in reality, we’ve had two of them in 67 years!

From a risk analysis standpoint, this is analogous to building your house above the 67,000-year flood line and being flooded out twice in the last 67 years.

There are many academic papers proposing alternate distributions to address this problem (e.g., truncated Cauchy, Student T distribution, Gamma distribution, stochastic volatility), but no generally accepted alternative has emerged.

To my knowledge, none of these papers suggested a causal mechanism to explain why stock market returns should have their proposed distribution of returns.  It’s relatively easy to torture a set of equations until it delivers the sort of distribution you desire.  What’s hard is proving that these distributions will do a good job of predicting future returns.  Since new return data comes in only one day at a time, it will take decades before any of these alternate proposals can emerge as superior.


So What Can Be Done?

 In this section, I’ll provide an abbreviated discussion of the steps that led me to an alternative to the normal distribution. At the bottom of the post, in the “Quant Corner” section I’ll provide some details for those that want the next level of detail.

Verifying that a new model is superior to the normal distribution is tough since new historical data dribbles in too slow to be helpful.  However, all is not lost—another way to verify is to come up with a realistic bottom-up model for how the stock market works and then run that model many times with realistic random inputs to generate lots of expected results.  If the simulated results show a convincing match to the actual results then those results can be used to evaluate various theoretical solutions.  It’s not a proof, but it’s more convincing (at least to me) than a top-down historic data matching exercise.

My approach to modeling the bottom-up behavior of the stock market incorporates one of its key characteristic—intraday correlations.  Depending on the day, stocks may move randomly with respect to each other, in lock-step, and occasionally in complete opposition (e.g., oil drops in price, energy companies go down, transportation stocks go up).  We can quantify the degree of synchronicity of these moves with a statistical measure called Pearson’s Correlation Coefficient, which returns values ranging from -1 for patterns moving in opposition to 1 for patterns in lockstep.  I measured historical patterns of correlation between stocks by comparing the moves of one sector of the S&P 500 with another.  I decided to use sectors instead of stocks to minimize the impact of corporate actions and company specific idiocentricities.  Using 16 years of data I computed the correlation between the S&P 500 (energy and consumer) sectors and produced the following histogram:

XLI-XLP-correl histo


As you can see lock-step (correlation of one) on the far right is the most common situation—but there’s a fair amount of variation.  This chart is representative; I looked at multiple combinations of different sectors and they all looked very similar to this result. I hypothesize that on many days the buyers and sellers of the stocks in a sector behave very differently from the buyers and sellers in other sectors, however on some days (e.g., panics, market rallies), the behaviors of all the participants in all sectors become synchronized.  The behavior of crowds/rallies/mobs might be a good analogy (thanks to Asad Aziz for that observation).

Rather than try to derive a theoretical relationship from this data I took the easy way out and just used the data itself in a Monte Carlo simulation of the stock market to model 5 million days of trading with variable correlation.  For each day of the simulation, I randomly picked a trading day between December 23rd, 1998 and January 25th, 2016 and used XLI’s correlation with XLP on that day to generate randomized returns for 75 different stocks with that same level of correlation.  The daily returns for those 75 stocks were then averaged to generate the daily return of the “market” itself.

The chart below compares my simulated results (green line) to the actual S&P 500 and the predictions of the normal distribution:



The distribution of simulated results matches the S&P 500 actuals significantly better than the normal distribution.  This simulation has the leptokurtic shape (high central peak, narrower upper shoulders) characteristic of stock market returns.

The next chart uses a logarithmic vertical scale to show that the simulated distribution also closely matches the S&P 500’s tail distribution


Comparing events in the tails is tricky because you can’t have partial events.  Instead, what happens is that as you move out on the tails you have more and more bins with zero events.  With the 5 million day simulation I had enough data to extend out the tails considerably, but with only 16630 actual data points empty bins occur pretty quickly in the tails. I used moving averages (purple lines on the chart below) to convert these occasional events into an events-per-bin metric—providing a more nuanced picture of how the actual tails compare to the simulated ones.



The data for the right tail closely matches the simulation. The left tail actuals are fatter than the simulation, but it’s still not a bad match
The Obvious Solution

 Given the impressive match between the actual and simulated results, the next question is whether there’s a theoretical distribution that matches these non-Gaussian results.

One characteristic that jumped out at me when I looked at the logarithmically scaled histograms of actual data was the linearity of the slopes.  A straight line on a log chart is an exponential relationship.  Two back-to-back exponential decay curves, centered at the mean, should closely match the data. This kind of distribution is called a double exponential, or Laplace distribution.

The Laplace distribution is similar to the normal distribution in that it has two parameters, the location, and the scale factor.  For a set of returns matching an ideal Laplace distribution, the location parameter is equivalent to the mean, and the scale factor is equal to the standard deviation of the population divided by the square root of two.  Below an ideal Laplace distribution (black lines) is overlaid on the chart of S&P 500 actual and simulated data.



Not surprisingly there isn’t an exact match between the S&P 500 distribution and the ideal Laplace distribution; the tails of the S&P are somewhat wider.  Since the goal of this exercise was to come up with a way to more accurately estimate the big up/down days I adjusted the Laplace’s scale parameter such that the number of 5 sigma or greater events was roughly the same between the predicted and actual distributions.  The result looks like this:



This adjusted Laplace distribution closely matches the S&P 500 historical data in predicting that every year there’s around a 75% chance of having a 5 sigma or higher event—a far cry from the normal distribution’s prediction of once per 6900 years.

The adjustment for wide tails increased the scale factor by 19% and gives a central peak prediction within 8% of the actual value.  The equivalent tail adjustment for the normal distribution requires a 70% adjustment and leaves the central peak 63% lower than the actuals.

I haven’t looked at a lot of cases, but I suspect the adjustment factor will be relatively consistent.  The scale adjustment for IWM (Russell 2000) is 17% and 12% for Apple.


So What?

The art of science and engineering is using concepts and relationships we know to be not quite true to generate reliable results.  For analyzing market risk the normal distribution does not pass the “not quite true” test; it is seriously flawed when used to predict / analyze the more extreme moves of the market that historically happen every couple of years.  It’s time to start using the Laplace distribution.


Quant Corner

  1. The first pass of my Monte Carlo simulation assumed that the individual stocks had normally distributed returns. A closer look indicated that individual stocks, not just indexes exhibited Laplace distributions, so I used Laplace distributions on the final simulation for all of the simulated stocks.  My intuition is that the actions of the various buyer/sellers of a stock are highly correlated some days while being much more random or anti-correlated on other days—generating a histogram similar to the XLI / XLP correlation chart.
  2. The simulation required me to generate pairs of random numbers correlated at a specified level. This correlation changed for every simulated trading day.  Generating correlated random numbers is not a trivial process.  Contract the author if you would like more information.
  3. The Monte Carlo simulation generated returns with a standard deviation of around 14%. In order to realistically compare this data set to the S&P 500 daily returns I linearly normalized the simulation results such that its standard deviation matched the S&P 500’s.
  4. The equation used for the Laplace distributions shown in the charts, the probability density function is:



    •  Location is the mean
    •  Scale specifies the spread of the distribution
    •  ABS is the absolute value function
  5. The equation used for generating random variables according to the Laplace distribution is:

    Laplace-random variable


    • The function “sign” returns -1 if the argument is negative, +1 if it is positive,  0 for zero
    • rand() returns a uniformly distributed random number between 0
      and 1 non-inclusive
  6. The ideal scale factor for a Laplace distribution is the standard deviation of the population divided by the square root of two. The calibrated scale factor I used to match the event frequency in the 5 sigma or larger tails was the ideal scale factor times 1.19.

Frank Calls the Corrections with His Modified Ned Davis Method

Updated: Apr 24th, 2016 | Vance Harwood

In September 2013, I published a post written by Frank Roellinger on his stock market trading system—a modification of the Ned Davis system first published in the 80s. Since Frank’s work was first published here he has shifted his Russell 2000 positions 11 times, each move reported here and on my twitter account.   His hypothetical portfolio value has increased  from 1658 to 1936 (+16.7%)—impressive given that the Russell 2000 is down 1.7% over that period.   However the most impressive thing has been his method’s accuracy is predicting the last four significant market downturns.


The arrows indicate when his published strategies switched from long to short.

Kudos to Frank for an impressive track record.

For more information:

Related Posts