Critics of transit investment – especially rail investment – frequently cite a failure to achieve a budgeted ridership estimate as evidence of the ineptitude or corruption of the agencies planning the lines in question. While I never wish to discourage due criticism, ridership estimates are constructed via theoretical models, and critiquing a model for being wrong is tautological, akin to critiquing a human for being mortal. Frustrated at popular confusion over the nature of modeling, I thought I’d write a post on the limitations and capabilities of models. To my mind there are four main points:
More after the jump…
(1. Ridership models attempt to quantify what would otherwise be qualitative phenomena; they translate dynamic human behaviors into static numerical inputs. Any number of input factors (endogenous variables) may be included in the construction of the model, such as tolerable walking distance, seasonal weather patterns, political affiliations, other transit connections, demographics, while other inputs are excluded (exogenous variables), such as unforeseen economic recessions. In short, modeling is all about throwing numbers at sentiments in search of rigor. (For philosophical problems with this approach, see Nancy Cartwright’s “The Vanity of Rigour in Economics”)
(2. Models aggregate particular behaviors upwards to create generalized assumptions. Such “bottom-up” science is inductive, and it has long been recognized that such reasoning is useful but tricky. You may see 1,000 black cats and conclude that all cats are black, only to see a white one. Similarly, all transit riders are willing to walk ½ a mile, until they aren’t. Etc etc…
(3. Simpler models are better. One tricky feature of models is that they are better at sketching a picture than painting it. The more variables one includes, the more sensitive (i.e. touchy and error-prone) the model becomes. Thus modelers try to select only the core components driving behavior. Never let anyone impress you by boasting about how complex their model is.
(4. Models don’t do any real work; rather, they merely actualize the assumptions they contain. The incredible ability of models to describe system dynamics far exceeds human capacity, but the inputs the models contain are usually entirely human-derived. A model is thus like a psychiatrist; you litter it with anecdotes and it tells you what they mean. But they don’t give you any information about the veracity of the anecdotes themselves.
Despite these many flaws, models nevertheless tend to outperform human judgment alone. While they may never be right, they’re almost always close. Models do a wonderful job of establishing baseline approximations from which to tweak, experiment, and innovate. When we use predictive models to aid the formation of transportation policy, we should only do so with a clear understanding of their limitations. Variances of 10-20% from modeled estimates are par for the course, and patience should be afforded to transit agencies when their models either under- or (more often) overestimate ridership demand. The real success of a transit service is its relative and ongoing performance once its baseline is empirically (rather than theoretically) established, and by this criterion Link is doing very well.