Volatility Is Rough
And rolls like a wave
Recently I was reading an article called “The Realized Volatility Puzzle”, by Harel Jacobson, a writer here on Medium. While I enjoyed the reading a lot, I was feeling the need for more visualizations. So let me directly start with a quote of the just-mentioned article:
The selection of length/frequency is probably the first and foremost important factor of the realized volatility measurement. While we sometimes put little to no thought into the realized volatility length/frequency, this is far from being a trivial question. Let’s say that we want to analyze the 1-month volatility. Which lookback window should we look at? should we compare that to the realized 10-day/20-day/40-day or even 100-day? what can assure us that the recent 20-day volatility is going to be a suitable comparison for the future 1-month? […] We can use different realized volatility estimators to detect mean-reversion/trend dynamic. […] The idea behind this test is rather intuitive, as mean reverting dynamic will exhibit higher intraday variance, while low close-close variance. [1]
Now I was just too curious how this would look like in a chart. Like how does it look like if we plot different window sizes used to estimate a standard deviation? And what would different Methods look like? Is there some kind of structure or are the data points just all over the place?
My first attempt to plot something to answer this questions was this (using some End-of-Day data of the S&P 500 Index):
Let’s call this plot the HF/LF plot, as we plot a high frequency and a low frequency volatility estimators over x different window sizes. I am not sure whether you are able to draw any conclusion from this plot right away, I however had no idea how I should interpret this.
Plot More Volatility Plots
Since this single plot does not give me much of a clue, I wanted to dig deeper to get a feeling for questions like these:
Does every one of such plots look similar? What kind of plot could we expect usually? How does this relate to implied volatility i.e. the VIX Index? How does this plot look at different points in time? Is there a difference at high volatility periods vs cool-off periods?
Turns out, to visualize all this questions I actually had to produce some kind of movie to understand how this plot evolves over a given period of time. And to make it a little bit more interesting probably over a period of time where the stock market experienced some stress, like February 2020.
What you can see in the following plot are two subplots. On the top there is basically the same HF/LF plot as we just saw but using a fixed scale for the y-axis. The second plot then shows the VIX as a proxy indicator of market stress. Finally a moving red line indicates at which timestamp we get the HF/LF plot from the top.
Look how beautiful this renders! It looks very much like a wave powering up smoothly and releasing the energy with quite some disturbance. And just because of the way the how the volatility cools off after a spike I immediately had to think about fractals and roughness.
So let’s talk a bit about fractals and roughness. I think most of you already know what fractals are, but just in case: A fractal is a never-ending repeating pattern which is self-similar across different scales. Such patterns can render very pretty like the famous Mandelbrot Set which you should have seen at least once. However, mathematicians like Mandelbrot and the hydrologist Hurst came up the concept of Fractional Dimension and a way how such a fractal dimension could be measured, like the Hurst Exponent H.
Now following the definition of the Hurst Exponent, a value of H=0.5 can indicate a completely uncorrelated series of data points where each data point is independent of the previous data point. Then a H > 0.5 can indicate auto correlation where each data point depends somehow on previous data points, we could call it a trend. Finally a H < 0.5 can indicate rough swings which we could call mean reverting. In the world of fractals a H > 0.5 would be smooth while a fractal series with H < 0.5 would be considered as rough.
Now the question arises whether we can put a label like “trending” (aka ”smooth”) or “mean reverting” (like “rough”) onto our volatility measurements right?
Lets take another look at the quote from the beginning of this article:
We can use different realized volatility estimators to detect mean-reversion/trend dynamic. […] The idea behind this test is rather intuitive, as mean reverting dynamic will exhibit higher intraday variance, while low close-close variance. [1]
So the idea is to use the different volatility estimation methods to get some insights. In another article (by the same author) we can find a clue on “how” we should do this :
Once we have both the HF variance and the close-close variance we divide the HF by the c-c variance. A ratio larger than 1 means that the asset has some degree of mean reversion, as the asset tends to have lower volatility on a c-c basis compared to intraday sampling. a ratio smaller than 1 will suggest some degree of trend. [2]
The Ratio Between High-Frequency Variance and Low-Frequency Variance
So here we are, back to questioning our self’s. How does such a ratio look like and how does it change over time? Can we draw any conclusions by plotting this data?
In order to answer this questions I want to stick with the movie solution, but this time we render it as a movie file instead of a gif. This allows us to skim over particular points in time more easily. Also we have to add the HF/LF Ratio to the plot as this is actually what we are interested in now.
I guess the interpretation is clear when all ratios (of all the different window sizes) are on the same side, meaning positive or negative. However it gets a bit tricky if some window sizes say “trending” while others suggest “mean reverting”. Now I am wondering if we can do better i.e. by directly estimating the Hurst Exponent as a measure of roughness.
Volatility is Rough
In a paper called “Volatility is rough” by Jim Gatheral, the author suggests to to estimate H by using simple regression tasks. [3] By following the paper I was able to write a piece of code and to get an estimate of the Hurst Exponent H over a moving window of a given size.
Note that due to the nature of the underlying regressions we already use multiple different time intervals (lags) and therefore only one single number per point in time. This means we don’t need to make a video anymore but can produce a simple line plot again.
As you can see the estimation of Hurst Exponent itself is pretty much robust. The number of lags as well as the window size do not have too big of an impact. Of course the shorter our window size the more “reactive” H is. However, in a video on YouTube Jim Gatheral pointed out, that so far no one has found a H being greater then 0. 5 — hence volatility is rough. In fact all tested financial indices produced similar numbers somewhere between .13 and .20.
However we can also see that H is lagging a lot. While it reacts quickly into the direction of “trending”, it takes a long time to drop back to “mean reverting”. At this point I am not sure whether H by itself is a useful indicator or not.
Conclusion
Simply reading H would not give me the reactiveness whether Vola is going to “mean revert” or “trend” over the next couple of time steps I was looking for. I think for now I can get better insights by using the High Frequency / Low Frequency Ratio plot. And if you read other articles I have posted like “The Importance of Feature Engineering for Financial Time Series Forecasting” [4] you might get an idea what we are going to try next.
Excurse: Volatility Forecasting via Fractional Brownian Motion
However estimating the Hurst Exponent is not a useless task at all. We can use H and the vol of vol estimator ν (nu) and plug it into the volatility forecasting model proposed by Jim Gatheral. [3]
Here we plot such a volatility forecast next to the implied volatility index VIX.
Note! Up until now we have used the VIX Index only as a proxy to illustrate how stressful a period of time in the stock market was. Volatility forecasting is not meant to predict the VIX. However, since I just did that and since IV is derived from option prices I would expect IV most of the time to be higher than the realized volatility. Which btw. seems to be the case (at lest for this plot).
[1] The Realized Volatility Puzzle, https://medium.com/swlh/the-realized-volatility-puzzle-588a74ab3896
[2] Delta Hedging made simple (sort of…), https://volquant.medium.com/delta-hedging-made-simple-sort-of-34441d1d1db8
[3] Volatility is rough, https://arxiv.org/abs/1410.3394
[4] The Importance of Feature Engineering for Financial Time Series Forecasting, https://towardsdatascience.com/the-importance-of-feature-engineering-for-financial-time-series-forecasting-a1163efe8b8a
Notebook used to create this charts: https://gist.github.com/KIC/dfb8284b667c3bbbeea025b8a580cd1c
