By Les Dunseith
In the realm of transportation planning, significant time, effort and money go into the process of forecasting, but the gap between predicted outcomes and reality remains a persistent problem for many projects.
“Forecasts don’t always get it right,” said Joseph Schofer, professor of civil and environmental engineering and associate dean of faculty affairs at Northwestern University. Schofer spoke on the topic of forecasting the future during the 10th annual Martin Wachs Distinguished Lecture, held at the UCLA Luskin School of Public Affairs on April 4, 2017.
The Wachs Distinguished Lecture features prominent and innovative scholars and policy makers who draw on many years of research and thinking in the field of transportation. Created by the students in honor of Emeriti Professor Wachs, the lecture rotates between UCLA and UC Berkeley, respectively.
This year’s lecture invitee, Joe Schofer, provided a wide-ranging view about forecasting – a prominent feature of transportation planning. In Schofer’s talk titled “When Forecasting Fails: Making Infrastructure Decisions in an Uncertain World,” he explained that learning to accept the inherent limitations of the forecasting process is a necessary first step in helping planners improve their predictions of cost, utilization, performance and impact.
“Don’t expect that the gap between predicted outcomes and reality is going to get really small,” Schofer told a crowd of more than 50 scholars, planning professionals and transportation decision-makers who came to hear him. “The world is changing at a faster and faster pace. And those big sources of uncertainty — sources of risk — often are outside the transportation system.”
Schofer’s lecture focused less on the shortcomings of forecasting than on “improving decisions by systematic learning from experience,” as Brian Taylor, director of the Institute of Transportation Studies and professor of urban planning, described the topic during his introduction of Schofer.
Schofer acknowledged the significance of the occasion during his opening remarks, taking a moment to recognize the presence in the audience of his “dear friend and colleague of a lot of years. This is not just another lecture. It’s me giving the Martin Wachs lecture,” said Schofer, who also cited Wachs’ “immeasurable impact on ideas in transportation, transportation planning, transportation policy and transportation finance.”
On Limitiations
In his lecture, Schofer focused on “what we can do in a situation where we don’t always get it right.”
For starters, he said, planners need to understand that they can never know everything there is to know about the dynamics of human behavior. It’s also important to keep in mind the rapid pace of change in today’s world.
“Changes that are going on right now literally make it impossible to forecast what the future is going to be like,” Schofer said. He pointed to examples such as the proliferation of cellphones, which enhance the speed of communication but negatively impact the capacity to do telephone-based polling research.
Schofer also pointed to other factors that limit forecast accuracy. “Data aren’t complete. There might be better models that we can use. Perhaps those models are not even available to us yet,” he said.
Although transportation experts are making strides and “using better and better data all the time, it’s not a calculus problem; we will not get infinitely close to zero error,” Schofer said.
He also noted that it’s common for forecasts to be impacted by unforeseen factors. For instance, major infrastructure projects often experience cost overruns and construction delays when previously unknown grave sites of historical or cultural significance are found during excavation.
On Being Grounded
Dealing with uncertainty may be avoided if planners make an effort to ground their projects firmly in the reality of previous experience. When forecasting a new project, planners must “ground that by finding out what someone else has experienced,” Schofer said.
The idea of looking at case studies and data related to past events is an essential element of evidence-based decision making, he said.
Some projects face the added complication of being based on visionary thinking — the “visionary ideas of interesting people,” he said. “It’s very difficult in a forecasting situation to go against that because you are dealing with somebody who has a firmly held vision, who is really committed to a particular idea.”
The goals of a visionary leader may outweigh an expert’s forecasts in the decision-making process, Schofer noted. The upside, he said, is that a diligent and resourceful planner can seize the opportunity in these situations to approach that visionary leader directly.
“You may be able to get his or her attention, which may be an opportunity to talk about a more realistic forecast,” Schofer said.
In most circumstances, however, it’s data that drives forecasting, and Schofer said he has seen some promising signs in getting access to better and more useful information.
Among the notable efforts he cited was a federal effort to mine existing administrative data, not to collect new information, to make better-informed decisions during evaluation of social programs.
In the medical field, he noted an effort known as the Cochrane Collaboration that is a loose confederation of people in medical research around the world who have an agreement to produce evidence-based information and to advocate for sharing of that information.
“A bunch of people around the world who have agreed to share data, agreed to work together, are bringing together data from a variety of studies to amplify the impact of that data,” Schofer said.
It’s a model that could easily translate to transportation planning, he said, an “opportunity to look at cases, to bring cases together, and to codify that.”
Schofer envisions a sharing of information among scholars, doctoral researchers, professionals and, perhaps, even journalists, in which information about the success or failure of infrastructure projects would be gathered into a database that could be accessed by “every one of us who wants to ask the question, “How well is this going to work in my town?’”
The shared data would be available for forecasters to evaluate, either analytically or qualitatively, and decide if there’s something useful from which they can learn.
For this type of case-based reasoning, it’s important to have a large dataset from which to draw conclusions. It’s also important for the cases to be kept up-to-date.
“The cases that we studied two years ago or 10 years ago, those are dead,” Schofer said. “We have to look at what’s happening right now.”
On Being Flexible
Given the limitations they face, Schofer said, it’s also important for transportation forecasters to be flexible in their thinking. In his lecture, he called this strategic incrementalism.
Think of it as hedging against uncertainty, he said, “getting ready for something different to happen that you didn’t expect to happen, and maybe putting some dollars against it, so that you are ready for it. So you can preserve future flexibility.”
In practical terms, this might mean erecting a building at a certain height but with the foundation and structure to allow it to become taller should the need for additional space later arise. It could mean building a bridge with one roadway but adequate architectural support to add a second deck later.
It means taking a long view when building major infrastructure projects, then monitoring, collecting data and watching closely to see how the new project actually gets used. If a project has design flexibility in the beginning, any future expansions can proceed at greater speed and at lower cost.
“We have to convey the notion of flexibility and adaptability and real options with the public and decision-makers,” Schofer said. “What you need to say is: “Let’s be a little looser about this, a little more flexible, to get what you really need.’”
Making better decisions in an uncertain world, Schofer said, involves collecting, analyzing and sharing as much data as planners can. Better information leads to better forecasting.
“In the end,” Schofer said, “it’s all about learning.”
