This is not a quant post, nor an Econ 101 revisit. In Robert Nozick’s political philosophy book Anarchy, State and Utopia, he gives us a precise and elegant explanation of what “invisible-hand explanations” actually are epistemologically. Analogous to the way we model market behavior using information theory. I believe this concept is monumentally important, but we are not spending enough time on it in our education.
There is a certain lovely quality to explanations of this sort. They show how some overall pattern or design, which one would have thought had to be produced by an individual’s or group’s successful attempt to realize the pattern, instead was produced and maintained by a process that in no way had the overall pattern or design “in mind”. After Adam Smith, we shall call such explanations invisible-hand explanations….Consider now complicated patterns which one would have thought could arise only through intelligent design, only through some attempt to realize the pattern. One might attempt straightforwardly to explain such patterns in terms of the desires, wants, beliefs, and so on, of individuals, directed toward realizing the pattern. But within such explanations will appear descriptions of the pattern, at least within quotation marks, as objects of belief and desire. The explanation itself will say that some individuals desire to bring about something with (some of) the pattern-features, that some individuals believe that the only (or the best, or the . . .) way to bring about the realization of the pattern features is to . . . , and so on. Invisible-hand explanations minimize the use of notions constituting the phenomena to be explained; in contrast to the straightforward explanations, they don’t explain complicated patterns by including the full-blown pattern-notions as objects of people’s desires or beliefs. Invisible-hand explanations of phenomena thus yield greater understanding than do explanations of them as brought about by design as the object of people’s intentions. It therefore is no surprise that they are more satisfying.
An invisible-hand explanation explains what looks to be the product of someone’s intentional design, as not being brought about by anyone’s intentions. We might call the opposite sort of explanation a “hidden-hand explanation.” A hidden-hand explanation explains what looks to be merely a disconnected set of facts that (certainly) is not the product of intentional design, as the product of an individual’s or group’s intentional design(s). Some persons also find such explanations satisfying, as is evidenced by the popularity of conspiracy theories.
Someone might so prize each type of explanation, invisible hand and hidden hand, that he might attempt the Sisyphean task of explaining each purported nondesigned or coincidental set of isolated facts as the product of intentional design, and each purported product of design as a nondesigned set of facts! It would be lovely to continue this iteration for a bit, even through only one complete cycle.
After listing dozens of invisible-hand explanations across different fields including evolutionary biology, economics, sociology, psychology, etc, Nozick goes on and explains the 2 types of invisible-hand explanations and their properties related to causality.
We can mention here two types of invisible-hand processes by which a pattern P can be produced: filtering processes and equilibrium processes. Through filtering processes can pass only things fitting P, because processes or structures filter out all non-P’s; in equilibrium processes each component part responds or adjusts to “local” conditions, with each adjustment changing the local environment of others close by, so that the sum of the ripples of the local adjustments constitutes or realizes P (Some processes of such rippling local adjustments don’t come to an equilibrium pattern, not even a moving one.) There are different ways an equilibrium process can help maintain a pattern, and there also might be a filter that eliminates deviations from the pattern that are too great to be brought back by the internal equilibrating mechanisms. Perhaps the most elegant form of explanation of this sort involves two equilibrium processes, each internally maintaining its pattern in the face of small deviations, and each being a filter to eliminate the large deviations occurring in the other.
Here’s the most interesting part.
We might note in passing that the notion of filtering processes enables us to understand one way in which the position in the philosophy of the social sciences known as methodological individualism might go wrong. If there is a filter that filters out (destroys) all non-P Q’s, then the explanation of why all Q’s are P’s (fit the pattern P) will refer to this filter. For each particular Q, there may be a particular explanation of why it is P, how it came to be P, what maintains it as P. But the explanation of why all Q’s are P will not be the conjunction of these individual explanations, even though these are all the Q’s there are, for that is part of what is to be explained. The explanation will refer to the filter. To make this clear, we might imagine that we have no explanation of why the individual Q’s are P’s. It just is an ultimate statistical law (so far as we can tell at any rate) that some Q’s are P; we even might be unable to discover any stable statistical regularity at all. In this case we would know why all Q’s are P’s (and know there are Q’s, and perhaps even know why there are Q’s) without knowing of any Q, why it is P! The methodological individualist position requires that there be no basic (unreduced) social filtering processes.