The Invisible-hand Explanation by Nozick

This is not a quant post, nor an Econ 101 revisit. In Robert Nozick’s political philosophy book Anarchy, State and Utopia, he gives us a precise and elegant explanation of what “invisible-hand explanations” actually are epistemologically. Analogous to the way we model market behavior using information theory. I believe this concept is monumentally important, but we are not spending enough time on it in our education.

There is a certain lovely quality to explanations of this sort. They show how some overall pattern or design, which one would have thought had to be produced by an individual’s or group’s successful attempt to realize the pattern, instead was produced and maintained by a process that in no way had the overall pattern or design “in mind”. After Adam Smith, we shall call such explanations invisible-hand explanations….Consider now complicated patterns which one would have thought could arise only through intelligent design, only through some attempt to realize the pattern. One might attempt straightforwardly to explain such patterns in terms of the desires, wants, beliefs, and so on, of individuals, directed toward realizing the pattern. But within such explanations will appear descriptions of the pattern, at least within quotation marks, as objects of belief and desire. The explanation itself will say that some individuals desire to bring about something with (some of) the pattern-features, that some individuals believe that the only (or the best, or the . . .) way to bring about the realization of the pattern features is to . . . , and so on. Invisible-hand explanations minimize the use of notions constituting the phenomena to be explained; in contrast to the straightforward explanations, they don’t explain complicated patterns by including the full-blown pattern-notions as objects of people’s desires or beliefs. Invisible-hand explanations of phenomena thus yield greater understanding than do explanations of them as brought about by design as the object of people’s intentions. It therefore is no surprise that they are more satisfying.

An invisible-hand explanation explains what looks to be the product of someone’s intentional design, as not being brought about by anyone’s intentions. We might call the opposite sort of explanation a “hidden-hand explanation.” A hidden-hand explanation explains what looks to be merely a disconnected set of facts that (certainly) is not the product of intentional design, as the product of an individual’s or group’s intentional design(s). Some persons also find such explanations satisfying, as is evidenced by the popularity of conspiracy theories.

Someone might so prize each type of explanation, invisible hand and hidden hand, that he might attempt the Sisyphean task of explaining each purported nondesigned or coincidental set of isolated facts as the product of intentional design, and each purported product of design as a nondesigned set of facts! It would be lovely to continue this iteration for a bit, even through only one complete cycle.

After listing dozens of invisible-hand explanations across different fields including evolutionary biology, economics, sociology, psychology, etc, Nozick goes on and explains the 2 types of invisible-hand explanations and their properties related to causality.

We can mention here two types of invisible-hand processes by which a pattern P can be produced: filtering processes and equilibrium processes. Through filtering processes can pass only things fitting P, because processes or structures filter out all non-P’s; in equilibrium processes each component part responds or adjusts to “local” conditions, with each adjustment changing the local environment of others close by, so that the sum of the ripples of the local adjustments constitutes or realizes P (Some processes of such rippling local adjustments don’t come to an equilibrium pattern, not even a moving one.) There are different ways an equilibrium process can help maintain a pattern, and there also might be a filter that eliminates deviations from the pattern that are too great to be brought back by the internal equilibrating mechanisms. Perhaps the most elegant form of explanation of this sort involves two equilibrium processes, each internally maintaining its pattern in the face of small deviations, and each being a filter to eliminate the large deviations occurring in the other.

Here’s the most interesting part.

We might note in passing that the notion of filtering processes enables us to understand one way in which the position in the philosophy of the social sciences known as methodological individualism might go wrong. If there is a filter that filters out (destroys) all non-P Q’s, then the explanation of why all Q’s are P’s (fit the pattern P) will refer to this filter. For each particular Q, there may be a particular explanation of why it is P, how it came to be P, what maintains it as P. But the explanation of why all Q’s are P will not be the conjunction of these individual explanations, even though these are all the Q’s there are, for that is part of what is to be explained. The explanation will refer to the filter. To make this clear, we might imagine that we have no explanation of why the individual Q’s are P’s. It just is an ultimate statistical law (so far as we can tell at any rate) that some Q’s are P; we even might be unable to discover any stable statistical regularity at all. In this case we would know why all Q’s are P’s (and know there are Q’s, and perhaps even know why there are Q’s) without knowing of any Q, why it is P! The methodological individualist position requires that there be no basic (unreduced) social filtering processes.




Leveraged ETF – A Simulation

This post is a token of appreciation for Faisal Habib who taught us structured products this summer.

As commonly known among people who are familiar with leveraged ETFs, the tracking error of those products tend to be larger than what we intuitively expected. This phenomenon has been explored and explained by Avellaneda and Zhang (2009). More information can also be found here. In a nutshell, the discrepancy resides in the borrowing cost incurred in a replication portfolio and extra realized variance in a continuous model. The resources linked above have more details, here I think it’ll be interesting just to model it out and take a look at the simulated results.

To do this I simulated four time-series.

  1. An underlying that follows GBM.
  2. An imaginary product that provides perfect leverage simply by multiplying the underlying return by the leverage ratio.
  3. A leveraged product that achieves leveraged return by constructing a static replication portfolio with risk-free rate r.
  4. A leveraged ETF modeled in continuous time, i.e., drift term adjusted using Ito’s lemma.

Here we have the results after running it once with 2x leverage and daily frequency (delta_t = 1/252). I know it strikingly resembles the shape of S&P500 but I swear this is pure coincidence (I thought it’s cool, too).


And annualized returns:

underlying_process simple_multiplication static_replication leveraged_etf
12.0% 25.4% 23.6% 22.3%

A quick note. The annualized return of the simple multiplication process (# 2) is way above 2 times of the underlying because of the compounding effect in a raging bull market we just created. The static replication (# 3) underperforms # 2 roughly by the same amount of the annual risk-free rate. The leveraged ETF (# 4) underperforms # 3 by the annual variance of the underlying. These results are consistent with previous studies.

There are a lot of ways to play with this so I’ll post the source code here for anyone who’s interested in trying it out (or let me know if I made any mistakes).

LETF <- function(S_0 = 100,
                 r_f = 0.015,
                 borrow_cost = 0.01,
                 mu = 0.05,
                 sigma = 0.1,
                 end_t = 5,
                 delta_t = 1/252,
                 leverage = 2) {
  # create a vector of the undelrying process
  n <- end_t / delta_t
  underlying_process <- zoo(0, 1:n)
  underlying_process[1] <- S_0      if (leverage > 1) {borrow_cost <- 0}

  # create vectors of simple_multiplication, static replication
  # and leveraged ETF without expense ratio
  simple_multiplication <- underlying_process
  static_replication <- underlying_process
  leveraged_etf <- underlying_process

  for (i in 2:n) {
    # the random component of the GBM model
    rdn <- rnorm(1)

    # model the underlying with GBM
    underlying_process[i] <- coredata(underlying_process[i - 1]) *
      exp((mu - sigma ^ 2 / 2) * delta_t + sigma * sqrt(delta_t) * rdn)
    underlying_log_return <- log(coredata(underlying_process[i]) /
                                   coredata(underlying_process[i - 1]))

    # model a leveraged process by simplely multiplying underlying returns
    simple_multiplication_log_return <- leverage * underlying_log_return
    simple_multiplication[i] <- simple_multiplication[i - 1] *

    # model static replication return with equation (1) and (2) in
    # Avellaneda and Zhang (2009)
    static_replication_log_return <- underlying_log_return * leverage -
      ((leverage - 1) * r_f - borrow_cost * leverage) * delta_t
    static_replication[i] <- coredata(static_replication[i - 1]) *

    # model leveraged etf return with equation (10) in
    # Avellaneda and Zhang (2009)
    leveraged_etf[i] <- coredata(leveraged_etf[i - 1]) *
      exp(underlying_log_return) ^ leverage *
      exp(-((leverage - 1) * r_f +
              sigma ^ 2 * leverage * (leverage - 1) / 2) * delta_t)
  return(merge(underlying_process, simple_multiplication,
               static_replication, leveraged_etf))



Ex-ante and Ex-post Risk Model – An Empirical Test

Whenever constructing a quant portfolio or managing portfolio risk, the risk model is at the heart of the process. A risk model, usually estimated with a sample covariance matrix, has 3 typical issues.

  1. Not positive-definite, which means it’s not invertible.
  2. Exposed to extreme values in the sample, which means it’s highly unstable through time and will be exploited by the optimizer.
  3. Ex-post tracking error is always larger than the ex-ante tracking error, given a stochastic component in the holdings, which means investors will suffer from unexpected variances, either large or small.

Issue 1 is a pure math problem, but issue 2 and 3 are more subtle and more related to each other. A common technique called shrinkage has been devised to solve these issues. The idea behind is to add more structure to the sample covariance matrix by taking a weighted average between itself and a more stable alternative (e.g. a single-factor model or a constant correlation covariance matrix). Two main considerations are involved in the usage of shrinkage: 1. what’s the shrinkage target, i.e. the alternative? 2. what’s the shrinkage intensity, i.e. the weight assigned to each matrix?


Links above provide details about these considerations. I did several tests to show how the differences between ex-ante and ex-post tracking errors vary when using different shrinkage targets and intensities.The test is done with 450 stocks that were both in the S&P500 by the end of Oct 2016 and had been listed for at least 10 years. An equal-weighted portfolio is formed using 2-year weekly data and is rebalanced every month.

The test is done with 450 stocks that were both in the S&P500 by the end of Oct 2016 and had been listed for at least 10 years. An equal-weighted portfolio is formed using 2-year weekly data and is rebalanced every month. The shrinkage intensity changes from 10% to 90% by 10% throughout the test. The spreads between the ex-ante and ex-post variances are recorded each week .



As shown above, the Ledoit-Wolf approach (single-factor, optimal intensity as derived by L&W) creates the least estimation error among all other approaches tested. Interestingly, the sample covariance matrix approach shows higher ex-ante risks than ex-post, which violates the theory mentioned above. This is possibly because in this test the ex-ante variances always stay constant for four weeks while the ex-post variances change every week, which amplifies the actual spread if we believe that they should move together over time.



An Empirical Mean Reversion Test on VIX Futures

VIX mean reversion trade gets popular when the market experiences big ups and downs. You hear a lot of talks about how much money people make from trading VXX, XIV and their leveraged equivalents. However, is VIX truly mean reverting, or it seems more lucrative than it is just because people only like to talk about it when they make money and keep quiet when they lose?

In this post I use daily returns of S&P 500 VIX Short-Term Futures Index from December 2005 to August 2015 (2438 observations) to find if there’s empirical evidence that supports short-term VIX MR. It’s the most suitable vehicle for this test becuase there’s no instrument that tracks VIX spot and it is the benchmark for VXX and XIV.

VIX ST Futures Index holds VIX 1-month and 2-month futures contracts and rolls them on daily basis. Its performance suffers from contango effect like commodity futures ETFs do but it’s an inevitable cost in this case.

To find out if extreme VIX returns lead to strong short-term rebound, I group all daily returns by deciles and summarize the distributions of the accumulative future returns of each group up to 5 trading days (1 week). If VIX is truly ST MR, we should see the future returns following the 1st group (lowest) significantly higher than 0 on average, and the returns following the 10th group (highest) significantly lower than 0 on average. Future returns that go beyond the sample time period are recorded as 0.


As shown above, group 1 and group 10 are the two groups we want to focus on. If someone can systematically make money by putting in MR trade on VIX, we should see the next day (or next 2, 3, maybe 5 days) returns following these two groups distributed like this:


The actual data look like this:

Day_1_Ret_Dist Day_5_Ret_Dist

It’s hard to spot any major difference between group 1 and 10 on the next day. However, on the 5th day, accumulated returns in group 1 largely outperform group 10. To better illustrate, I perform t-test on both groups from day 1 to day 5, as reported below (H0 = average return equals 0).


As it turns out, future returns in Group 1 systematically outperform Group 10 within the next 5 days. The results are not blessed by overwhelmingly strong t-stats and p-values but it’s hard to argue that we are looking at random noises here. Additionally, this test is done by end of day prices. Intraday movements, which possibly constitute the bulk of VIX MR trades, are completely ignored by this test. Therefore the results we see are likely a mitigated version of market reality.


Constructing an Alpha Portfolio with Factor Tilts

In this post I’d like to show an example of constructing a monthly-rebalanced long-short portfolio to exploit alpha signals while controlling for factor exposures.

This example covers the time period between March 2005 and 2014. I use 477 stocks from S&P500 universe (data source: Quandl) and Fama-French 3 factors (data source: Kenneth French’s website) to conduct backtests. My alpha signal is simply the cross-sectional price level of all stocks – overweighting stocks that are low on price level and underweighting the ones that are high. By doing this I’m effectively targeting the liquidity factor so it worked out pretty well during the 2008 crisis. But that’s beside the point, for this post is more about the process and techniques than a skyward PnL curve.

At each rebalance, I rank all stocks based on the dollar value of their shares, then assign weights to them based on their ranks inversely, i.e., expensive stocks are getting lower weights and vice versa. This gives me naive exposure to my alpha signal. However, my strategy is probably exposed to common factors in the market. By the end of the day, I could have a working alpha idea and a bad performance driven by untended factor bets at the same time. This situation calls for a technique that gives me control for factor exposures while still keeping the portfolio close to the naive alpha bets.

Good news: the basic quadratic programming function is just the tool for the job – its objective function can minimize the sum of squared weight differences from to the naive portfolio while the linear constraints stretching factor exposures where we want them to be. For this study I backtested 3 scenarios: naive alpha portfolio, factor neutral portfolio and a portfolio that is neutral on MKT and HML factor but tilts towards SMB (with a desired factor loading at 0.5). As an example, the chart below shows the expected factor loadings of each 3 backtests on the 50th rebalance (84 in total). Regression coefficients are estimated with 1-year weekly returns.


After the backtests, I got 3 time-series of monthly returns for 3 scenarios. Tables below show the results of regressing these returns on MKT, SMB and HML factors. All three strategies yield similar monthly alpha, but the neutral portfolio mitigated factor loadings from the naive strategy significantly, while the size tilt portfolio kept material exposure to the SMB factor.


Tables below summarize the annualized performance of these backtests. While the neutralized portfolio generates the lowest annualized alpha, it ranks the highest in terms of information ratio.


Interpretation: the naive and size portfolio gets penalized for having more of their returns driven by factor exposures, either unintended or intentional. The neutral portfolio, with slightly lower returns, gets a better information ratio for representing the “truer” performance of this particular alpha idea.

The idea above can be extend to multiple alpha strategies and dozens of factors, or even hundreds if the universe is large enough to make it feasible. The caveat is that there is such thing as too many factors and most of them don’t last in the long run (Hwang and Lu, 2007). It’s just not that easy to come across something that carries both statistical and economic significance.


Non-linear Twists in Stock & Bond Correlation

Stocks and bonds are negatively correlated. Translation: they must move against each other most of the time. Because intuitively, stocks bear higher risk than bonds so investors go to stocks when they want to take more risks and flee to bonds when they feel a storm is coming. Plus the numbers tell the same story, too – correlation coefficient between SPY and TLT from 2002 to 2015 is -0.42, year-over-year correlation on daily returns are:


However, this effect was very week from 2004 to 2006. This makes sense because in a credit expansion like that, it was hard for any asset class to go down (except for cash, of course).

But this observation reveals that the conventional stock & bond correlation might be conditional or even deceptive. One might ask, is this stock & bond relationship significantly different in bull and bear markets? Does it also depend on market returns? Or does it just depend on market directions?

To keep it simple, I will stick to SPY and TLT daily returns. If I split my data into bull (2002-2007 & 2012-2015) and bear (2008-2011) periods, and divide each of them into two groups (market goes up & market goes down), then dice each group by quantiles of market returns, I will get:

corr_bull bear_corr

The graphs show that these two assets tend to dramatically move against each other when the market is going extremely up or down. Also this effect seems more pronounced when a bull market is having an up day or a bear market is having a down day. But there’s nothing significantly different between the bear and bull groups.

Next I can try not to split the data into bear & bull, instead I’ll just divide it by market direction, then quantile of performance.


This graph clearly shows that stocks & bonds mostly only move against each other when the market is having a extremely up or down day, either in a bull or bear market. Of course, one could argue that this is a self-fulfilling prophecy because big correlation coef’s feed on big inputs (large market movements), but in the chart the correlation coef’s do not change proportionally through quantiles, which confirms a non-linear relationship.


A New Post

First of all, apologies to anyone who were expecting new posts or left a comment here but didn’t get a reply from me. There were quite a few changes in my life and I simply had to move my time and energy on blogging somewhere else. Now I’m trying to get back to it.

Because of reasons I will avoid writing about specific investment strategies, factor descriptors or particular stocks. I will write more about my thoughts (or thoughts stolen from people smarter than me) on generic techniques and theories. In an attempt to be rigorous (or more realistically, less sloppy), I will try to stay on one main track: hypothesis -> logical explanations -> supporting data or observations. This time I will use math and programming to make sense of things instead of just getting results on paper.