At Stanford summer camp a couple months ago, a presenter started his talk noting that structural estimation was the gold standard for identification but natural experiments were the platinum standard. I could not agree more. After all, platinum is about the same price per gram as gold but was hyped up by clever marketing during 70s and 80s to differentiate the discriminating elite from gold, which had become a staple for the middle class.
Yet, more seriously, natural experiments seemingly have a hedge in clean identification of an exogenous shock, so let me ask the question:
Where does identification come from in structural work?
I think the answer to this is best obtained by considering identification in the natural experiment: there is an observable exogenous shock and we see responses to this shock. But, what if we do not directly observe the shock, should we give up on identification? This is exactly where structural estimation comes in: under certain situations, we can fill in the gaps with plausible theory to analyze consequences of explicitly described exogenous, but not directly observable, shocks. That is, exogenous shocks are central to the entire process of structural estimation. People would sometimes call these shocks “the data-generating process.”
I am slightly curious about extremists claiming that we should only rely on observable exogenous shocks. Let me do a quick off-topic parenthesis. I wonder how much of modern physics would survive if we required any fundamental particle to be directly observed, or processes that happen at the core of cosmological objects to be seen. I’m not sure I know anybody who’ve seen the Big Bang directly but I’d venture to say it’s well accepted.
These assumptions are precisely where structural estimation shines, because – when you think about it – it is truly a joint identification: identification of the hidden exogenous shock and of the economic mechanism to obtain the true equations of how the world works. By contrast, very few natural experiments tell us much about the mechanism, the ‘Why.’
Consider the excellent paper by B. Gipper from U. of Chicago (here), who examines the consequences of a shock to compensation disclosure, observing that compensation levels go up given greater disclosure. He pitches that this is because firms will now try to hire away underpaid CEOs. Certainly possible. Or, I have a paper (here) that shows that disclosure could make higher-powered pay a signal of future performance, but then causing a higher payment to compensate for greater risk. Or, it could be that that matching was inefficient prior, and the greater transparency has increased the efficiency of the labor market which will translate in shared benefits for all.
Any attempt to pick an explanation is speculation and, yet, these explanations have near-opposite policy implications (so it does matter!). In the first explanation, disclosure is largely redistributive. In the second, disclosure is probably bad because it increases signalling costs. In the third, disclosure is good because it makes CEOs work for the company where they are the most useful. So, the evidence is clearly good to know but it hasn’t reached an answer form that could be used for policy, simply because its welfare implications are completely obscure.
I heard the economist U. Malmendier offer a good resolution of the, mostly non-constructive debates about which methods is the most precious. She notes that the view that natural experiments should be better or worse than structural work presumes that they are substitutes but (more plausibly) they are complements. Nothing prevents a paper that has a natural experiment from using a structural model to recover the details of the theory – in fact, such should be necessary, if on a second step, to fully understand the mechanism as she has done in her own work (here). And, vice-versa, a theory that is being used in a setting where shocks are unobservable should be validated in a different setting with observable exogenous shocks.
So this is my personal call for future research: let us leverage on the institutional settings in which shocks are exogenous and try to identify the mechanisms. It’s a great place to envision structural work.