This Is Not The Cause You're Looking For
This Is Not The Cause You're Looking For
Can the hunt for cause and effect in economics ever be successful?
… the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters.
Jorge Luis Borges, “On The Exactitude of Science” (Spanish version here)1
Earlier this month, the Boston Fed published a paper titled “Cost-Price Relationships in a Concentrated Economy”, which deals with the relationship between market concentration and inflation, mediated by the passing through of higher costs to consumers. Using empirical estimates, it finds that the increase in market concentration since 2000 has increased inflation. The paper’s findings came under heavy scrutiny, including this exchange between two IO economists:
“For every cross-industry price-HHI regression there is an equal and opposite cross-industry price-HHI regression”. This is a fascinating statement and gets at an issue at the heart of economics itself: Which models are valuable and which ones aren’t? How can you tell them apart?
A similar, related question that’s come up as a result of the Fed’s rate hike is the following: can we know if monetary policy works, if we don’t precisely know how it works? Is this even a relevant question?
This is a post about the epistemology of economics, once again, because it’s really interesting to me and because it’s applicable to this one scenario (and also because it’s a way to trick myself into studying for an exam). I really recommend reading my first piece on it, although only really some parts are relevant.
A Long Time Ago In a Galaxy Far, Far Away...
The first economist to take epistemology, i.e. the study of the philosophy of science and knowledge, seriously in a relevant manner was John Stuart Mill. Mill is a fascinating character personally, and probably a brilliant political thinker (though not that good a writer). But his key contribution to what I’m after here comes from his 1844 work “Essays on Some Unsettled Questions of Political Economy, Essay V: On the Definition of Political Economy and on the Method Proper to It”. What an appealing title! But the topic is fascinating and very relevant: what is economics (then called “political economy”), what are its methods, and how should it reach conclusions?
To begin with, Mill makes a really counterintuitive distinction within what we normally called science - between “science” and “art”. In Mill’s terms “science” is the search for true, universal causal laws, and “art” is the application of science to specific contexts. Mill argues that economics (political economy, remember) is a science because it rigorously searches for the causes underlying certain phenomena - in this case, after some back and forth, he settles on economics being:
The science which traces the laws of such of the phenomena of society as arise from the combined operations of mankind for the production of wealth, in so far as those phenomena are not modified by the pursuit of any other object.
Concerns about readability aside, Mill is making a really big claim here, that becomes one of the central characteristics of his epistemology of economics: that, for a theory to explain “the creation and distribution of wealth” within economics, it only has to contain economic factors. This stems from Mill’s understanding of the world at large: the world is very complex, and there are endless causes to explain each phenomenon, so to develop universal laws, you have to perform analyses that isolate causes. To this extent, both between disciplines and within them, you simply remove explanations besides the one you want to explore, and then proceed to develop cause and effect. The role of assumptions, in Mill’s view, is to simplify reality so that causal inference is possible. After each cause is isolated, you combine them.
Now there’s the issue of the how: how do you isolate a cause? The difference might seem between data and one theory, but to Mill’s credit, he rejects this (ridiculous) dichotomy: science is completely sterile without theory, and completely pointless without data - nobody has ever used just one. The difference is actually much more subtle: the empiricist (a term Mill doesn’t use) analyzes the facts of each instance and determines the common laws they show (i.e. reasons upwards) while the theorist determines general laws from first principles under certain assumptions (i.e. reasons downwards). Verification isn’t really the point, but it’s still highly valuable: if an economist wants to examine anything, they can’t just list off models and their conclusions - they have to get their hands dirty and look at some facts to match real circumstances with the models that have the best fitting assumptions. Without doing this, "he may be an excelent professor of abstract science”.
The natural sciences can actually use the first method because they have full control over the circumstances of the data they utilize: the natural scientist can claim to have universal knowledge of the real world with his experiments because, since the lab environment is perfectly controllable, then they can perform crucial experiments that determine whether or not a hypothesis is true by simply creating a “world” where the only difference between two scenarios is that a potential cause is present or absent. So, for example, you give half of patients a medicine and half a placebo - so you can know if the effect from the medicine is from the placebo or if it’s actually effective, and because both groups are chosen to be as identical as possible, you know with certainty. But a social scientist doesn’t have two countries that are completely identical but have one different policy, so therefore we can’t actually make that sort of high-confidence prediction in most cases - you have to rely on theory, primarily.
The assumptions economics has to use have to be really strict and limited because the reality it studies is so complicated that untangling a single cause requires egregious amounts of simplifications. Instead, economics (and all social sciences) has to settle for theoretical models where the reality seeps in through the foundations and not through verification. In consequence, while the laws found by economists are not true in the same way as Newton’s or Galileo’s, this doesn’t mean they’re not true outside of textbooks: provided that each causal mechanism is developed correctly, they have to be correct in the abstract, and that their correctness in the real world depends on how closely their initial assumptions match reality. In a problem where assumptions are a close match, the conclusions drawn from abstraction and simplification are valuable.
Confidence in empirical work is not boundless, however. The problem when applying them to the real world can be best explained by Donald Rumsfeld: the unkown unkowns, potential causes that weren’t included in the model but play a major role in it. Even if you can reach reasonable degrees of confidence for your causal mechanisms due to their realism, and you develop a good working understanding on how they all work together - meaning your causal claims are really strong in theory - you can still blunder due to not including enough (relevant) variables. In a rare banger Mill quote:
“A person may be warranted in feeling confident, that whatever he has carefully contemplated with his mind’s eye he has seen correctly; but no one can be sure that there is not something in existence which he has not seen at all.”
There is another
Mill’s epistemology is clearly old-fashioned, and a lot of his claims don’t really make a lot of sense in a world where no serious economist would be treated seriously without a large amount of econometrics training. Fortunately for the “Mill Side of the Force”, there is another: physicist and philosopher Nancy Cartwright. Cartwright’s epistemology is broadly similar to Mill’s, but has major differences - especially as it pertains to the use of experiments, assumptions, and theoretical models.
The world Cartwright lives in is, in general, similar to Mill’s: messy, with each phenomenon determined by a multitude of causes. The “hunt” for causes, therefore, is also messy - and simplifications (via assumptions) are therefore required. The isolation of causes is realized through additional, more specific assumptions, which, in turn, results in the validity of a model depending less on the evidence supporting it and more on the applicability of its assumptions. Consequently, there is a fundamental tension in all models: between their internal validity, that is, the accuracy with which they capture causes, and their external validity, i.e. their applicability. There is a hard tradeoff between the two because of the need to utilize unrealistic assumptions - to further isolate an object from other potential causes, you need more and more assumptions that restrict the applicability of the model to various contexts.
Much like Mill, Cartwright considers that models and experiments are equally worthwhile when possible - but for reasons unlike Mill’s. Models and experiments are both valid not because they can both isolate causes, but because both are fundamentally the same: models are “word experiments” and experiments are “material models”. Both represent the world through simplified systems which act as shorthandby eliminating extraneous factors to isolate causes. This is equally unrealistic whether it happens in a lab or in a sheet of paper: in both cases, the isolated causal relationships are true within the context of the experiment but not within the context of the real world. Much like firms don’t actually maximize profits, apples don’t fall in a vacuum. The laws of physics and the laws of economics are both equally false.
Cartwright does place limitations on assumptions. Assumptions that simplify the world and help isolate causes can be as unrealistic as they want, since they only limit the scope of applicability of the model itself. Assuming profit maximization or that there is no friction, are both valid assumptions that simply allow you to narrow down causes However, assumptions that only serve the purpose of simplifying models (such as rational expectations, or the Efficient Markets Hypothesis) are all a step too far and don’t serve the purpose of pruning the causal forest - only of ensuring certain conclusions or cleaning up the math.
Begun, the cause wars have
In my first post on the epistemology of economics, I mostly explained what I’ll call the Popper-Friedman view of economic inquiry. The two views are generally at odds with each other for a number of reasons:
Truth: Popper and Friedman think finding truth is impossible; Mill and Cartwright that it’s possible and “straightforward”. The wrongness of a model is not determined by the same means - validity is determined through the origin of the model in the first case, and through its empirical application in the latter.
Theory and evidence: both agree that theory doesn’t exist without evidence, and that evidence doesn’t exist without theory. But Mill-Cartwright put evidence at the beginning, while Popper-Friedman at the end.
Assumptions: Mill and Cartwright use assumptions to simplify a complicated world so they can isolate causal relationships; Popper and Friedman to ensure predictions that can be empirically verified. The realism of assumptions matters for Team Mill but very much not for Team Popper2.
Models: for Team Popper, all models are wrong, but some are useful; for Team Mill, all models are right, but some are useless. Popper-Friedman make universally contestable claims; Mill-Cartwright make situationally valid ones.
Application: Team Mill considers that applying a model to specific real world circumstances is useful but not crucial - in crude terms, evidence is useful to estimate beta hat and to figure out whether you’re applying the right model. Team Popper, however, see application as the crucial step for validity.
Science. Economic is a science, according to Mill and Cartwright, because it uses generally broadly empirical methods to “find” universal laws. According to Friedman and Popper, because it rejects theories that make false predictions.
Now, Mill and Cartwright’s views on the epistemology of economics is interesting, powerful, and well-articulated. I think that Cartwright’s version in particular can be used to defend economics from its dumbest, least thought out criticisms (homo economicus and the like) much more convincingly than Friedman’s. I also think that the implications for public policy of the Mill-Cartwright view are very interesting, and much “richer” than of the Friedman-Popper view.
It’s over, I have the high ground
There are some problems inherent to the Mill view: the first is that it’s not especially clear how to check if assumptions match reality, because Cartwright (much moreso than Mill) allows for extensive use of unrealistic assumptions as long as they isolate causes. The second one is that two models with completely opposite conclusions can be both true in Mill World, if their assumptions are different enough, but cannot in Popper World. The velocity at which an object falls either is or isn’t proportional to its weight, and a change in the price level either is or isn’t equal to the growth of the supply of money. The issue is that similar models with opposing conclusions can never be ruled out under the Mill paradigm. The Popper view has the opposite problem, that different models with equal predictions cannot be ruled out, but it’s much smaller because you’re just holding them to be non false, not true.
A major concession to make to Cartwright (not so much Mill), and one which Milton Friedman actually makes, is that some models are only really useful under certain circunstances. Something like Sargent & Wallace (1981) (a pure theory paper but whatever), even when empirically verified a la Friedman, is not useful in all circumstances. The paper specifically refers to economies that are undergoing fiscal dominance, i.e. have central banks that semi-independent and might have to step in and finance the government. The conclusion that fiscal, not monetary, policy rules inflation is only true because fiscal dominance holds. Team Mill would say that under these circumstances the traditional Monetarist model is not true (even though the Quantity Theory holds by assumption) and that the deficit is the cause of inflation. But the issue here is that the fundamental cause of inflation is exactly the same, monetary policy, and the mechanism through which it influences prices is different (nominal spending in Monetarist World and government finances in Fiscal Dominance World). The underlying causal relationships are the same, but the fundamental difference is that different environments rely on different mechanisms.
The Friedman view is not flawless either, and theories can sometimes be rejected with insufficient data - for an example, look back at an old blog post topic: by the 1990s, the neoclassical growth theory’s prediction of convergence seemed falsified. Something like the Lucas paradox also could have pointed to neoclassical growth theory being highly flawed3. However, convergence started happening literally at the same time economists decided it wasn’t happening, ulitmately settling the question in the opposite direction. So prematurely rejecting a hypothesis, or rejecting it due to an incomplete falsification of its key implications, is a huge problem.
The problem of being unable to reject either of two competing hypothesis is also present in Friedman’s model, especially because many theories have multiple conclusions that are up for verification, and multiple manners to prove each one. The “rival” theory of neoclassical growth, endogenous growth, specifically didn’t result in convergence, but did have “scale effects” - the idea that bigger countries would grow faster, because their bigger markets amplified externalities. Scale effects are not observed in the real world - so here’s a dilemma: if there’s two major conclusions, one true and one false, is the theory falsified or not? Are both falsified, or neither is? And which theory is “more” falsified?
Falsification itself is fundamentally unstable as a criterion for validity, for a very simple reason: how do you know if a theory is being rejected wholesale or if there’s a specific causal channel at play? Take James Buchanan’s response to Card & Krueger (1994), a now legendary paper that found no statistically significant disemployment effects from a minimum wage increase - contrary to (then) standard labor economics theory predictions. Buchanan had this to say about the paper:
Just as no physicist would claim that “water runs uphill,” no self-respecting economist would claim that increases in the minimum wage increase employment.4 Such a claim, if seriously advanced, becomes equivalent to a denial that there is even minimal scientific content in economics, and that, in consequence, economists can do nothing but write as advocates for ideological interests. Fortunately, only a handful of economists are willing to throw over the teaching of two centuries; we have not yet become a bevy of camp-following whores.
At first glance, it seems Buchanan is completely misrepresenting the paper’s claims and being a sore loser because he disagrees with the paper’s results (which, to be fair, he kind of was), but he’s also right - it’s not really sufficient to use a single negative empirical finding to disprove a long-standing theory. If, for instance, most of the literature over the following 20 years had found the opposite results, would the theory be falsified or not? It’s not really clear, and a fundamental inability to discern true from false theories seems like a pretty bad flaw.5
This is where the fun begins
We started with two sets of questions: is the Boston Fed paper any good? And does it prove that market concentration “caused” inflation? Answers: it’s bad and it doesn’t. I’m not going to rehash this excellent Price Theory substack piece about it, so just agree with me that it’s not a good paper - and even if it did, it would be besides the point of this piece. The other set concerned monetary policy: does not knowing the precise transmission channels of interest rate hikes on inflation mean that monetary policy is an inadequate tool? And is this even a reasonable thing to discuss? Since this is an epistemology of economics piece, let’s tackle it from an epistemology standpoint.
The Boston Fed paper doens’t really make many causal claims, and mostly estimates an effect, but it has sometimes been used to justify “greedflation” theory. Greedflation means, roughly, that market concentration caused inflation by increasing corporate profits for any number of reasons. In principle, the paper doesn’t say that inflation accelerated because of concentration; it says that market concentration made underlying inflationary trends worse, which is not the same at all. This means that it’s not asserting any causal relationship between inflation and concentration - inflation has its causes (say, for instance, higher costs) and those phenomena are then amplified by market concentration.
So, at first, Team Mill wouldn’t be on board with the greedflationists - this isn’t causal first base, let alone a home run. You’re just estimating stuff, and that’s okay, but you’re not really finding any universal laws. Team Popper would obviously not be on board either, but for different reasons. What is the main empirical implication of the greedflation thesis? Well, that corporate profits have to grow faster than inflation, and that more concentrated industries would have more inflation. The paper doesn’t have much to say about the latter6, but plenty about the former. The Boston Fed paper actually kind of falsifies the concentration-causing-inflation claim, because it shows that inflation is higher because firms charge more to offset costs, which means they’re making at best the same amount of profit and that they are making more than in the no-concentration-with-shocks scenario but not necessarily in a concentration-without-shocks scenario, which is the claim being advanced by greedflationists.
What about the other question, the one about monetary policy? As a reminder, many people have said “since the channel through which monetary policy affects the economy is unclear, then the efficacy of monetary policy must also be unclear as well”, in more or less words. Empirically, it’s generally agreed upon that monetary policy works, and both affects prices and outputs across different lenghts of time. Once again, both views disagree with this question, and both times for different reasons.
Going back to Nancy Cartwright’s views, that it’s fundamentally a useless question: isolating the causal channels through which monetary policy affects the economy would require assuming a lot of things that make it so the model cannot capture the full effect of monetary policy. For instance, a recent paper showed that higher interest rates can increase inflation by bringing down mortgages and making rents higher. But even if this is unarguably true, then it’s still not clear how this channel interacts with all the others. If you could get reliable estimates for how interest rates affect every single product in the economy, you still wouldn’t be able to determine the macroeconomic effects because you wouldn’t be able to know the interactions between every single one of them under every single possible configuration of events. There is a trade-off between internal and external validity for a reason, and knowing only and exclusively whether monetary policy works or how it works in any single context is precisely what’s expected under “real science”.
Under Karl Popper and Milton Friedman’s position, the question is even less relevant. The channels through which monetary policy affect the economy are an interesting empirical question, but they are only relevant insofar as they can result in different predictions. So higher rates pushing up rents and thus increasing inflation are relevant, but only if you have any reason to expect them to overcome all other effects of inflation - which can happen, maybe. But fundamentally their take on the question of “neutrality” isn’t “are our models of inflation and monetary policy correct”, it’s “are our models of inflation and monetary policy useful”, and the evidence shows that monetary policy is generally useful and effective (or rather, that it’s not useless and ineffective) in the kind of context we’re talking about.
Conclusion
What can we make of this?
The assumptions of economic models should as capable as possible of simplifying reality to isolate causal relationships. In a complicated, messy world, it’s impossible to determine causality certainly, so causes must be isolated by simplification of reality (in theory).
Economics should use primarily analytic methods due to its incapacity of experimenting. Since the social sciences cannot directly isolate irrelevant causes as the natural sciences do in lab experiments, they must work through rigorous theoretical modelling. Verification occurs by matching assumptions with real world circumstances.
Models are experiments and experiments are models. Both the natural and social sciences simplify reality to isolate causes; the natural sciences do it through experiments, and the social sciences through models. In either case the simplifications must be in service of isolating causality, not of specific conclusions or of simplicity in the mathematics.
Every model has a tradeoff between precision and usefulness. The more assumptions used, the better the model is at isolating causal ties, but the less applicable to different circumstances it is.
Both views have their strong suits. The Mill view is stronger for its emphasis on causality and its allowance of different mechanisms to operate for the same cause under different circumstances. The Popper view is stronger for its emphasis on empirical verification of conclusions, and of simplicity in modelling.
Causality not being an empirically verifiable phenomenon makes the Mill-Cartwright view inherently unstable.
The Mill-Cartwright view is limited by its reliance on assumptions for verification. If models with similar, similarly plausible assumptions have radically different conclusions, there is no way to determine which one is true through either theory or practice. Additionally, there is no clear methods to directly verify assumptions, let alone to decide which ones are worth verifying.
The Popper-Friedman view is limited by the ambiguous nature of verification through outcomes. If a model advances multiple claims and is only partially validated, it is unclear whether it’s falsified. If two competing, incompatible models are both partially falsified, it’s unclear which one to reject. It’s unclear if one falsification is enough to reject a model.
Sources
My previous blog post on the epistemology of economics
Mill and Nancy
Mill (1844), “Essays on Some Unsettled Questions of Political Economy, Essay V: On the Definition of Political Economy and on the Method Proper to It” (link to the essay)
Cartwright (1999), “The Vanity of Rigour in Economics”
Cartwright (2010), “Hunting Causes and Using Them: Is There No Bridge from Here to There?" (ungated summary)
Maki (2005), “Models are experiments, experiments are models”
Friedman vs Mill
Friedman (1953), “The Methodology of Positive Economics”
Patel, Sandefur, & Subramanian (2021), “The New Era of Unconditional Convergence”
McLeay & Tenreyro, "Optimal inflation and the identification of the Phillips curve"
Fun fact: this piece was used as an introduction in every single macroeconomics class I’ve had, plus economic growth and IO. A university plagued by JS Mill believers, sadly.
Worth pointing out that Friedman wouldn’t consider that if a model’s predictions are not falsified, its assumptions must be true. For example, if you assume the housing market is perfectly competitive, rent control doesn’t lower rents. If you found this to be non-false, Friedman wouldn’t say that the housing market is competitive, just that perfect competition is a useful way of defining it under certain circumstances.
Cartwright and Mill would have actually solved the Lucas paradox in the same way Lucas did, by clarifying additional circumstances - such as risk. Funnily enough, Lucas himself is an avowed “Friedmanite [in the issue of methodology]”
Card & Krueger didn’t actually claim that, and David Card himself was abundantly clear that he didn’t believe that this was the case in interviews after he won the Nobel Prize. It’s also why he’s not written much about things like the minimum wage and immigration lately.
You could use a very “Lakatosian” view of science and say a theory is only rejected when both its major predictions are consistently falsified and when a rival theory that accounts for those findings emerges, but this is more descriptive than prescriptive.
Let’s get it out of the way: if you use better data for on market concentration, it’s not true that more concentrated industries are more inflationary; measuring by traditional concentration measures shows basically no relationship, and measuring the outcome of concentration, markups, shows a negative relationship.
Great post! We often think about causality in economics, but I haven't seen many posts taking it a step further and talking about the intersection of economics and epistemology.