Press release: Crain and Crain study based on fundamentally flawed economic model and faulty data

Claims by researchers Nicole and Mark Crain that government regulations cost $1.75 trillion are fundamentally flawed, says a new critique released today by the Economic Policy Institute.  Flaws call for rejecting Crain and Crain model by research and policy director John Irons and Andrew Green, is a thorough critique of the deficient model Crain and Crain developed to estimate the cost of “economic regulations,” which account for 70 percent of the large cost of regulations they found in the 2010 study they wrote for the Small Business Administration’s Office of Advocacy that is often cited by critics of regulations.

“The Crain and Crain model contains basic conceptual mistakes and relies on extraordinarily poor data,” said Irons. “Its results should neither be used as a valid measure of the economic costs of regulation nor as a guide for policy.”

Flawed methodology

Flaws call for rejecting Crain and Crain model criticizes the Crain and Crain methodology that contains the following fundamental errors for determining the cost of economic regulations:

• It fails to capture the timing between changes in the determinants of economic growth and the amount of ensuing growth. The estimation does not properly take into account time-series dynamics.
• It misses the potential reverse causality between factors associated with economic growth and the growth itself.  For example, the Crain and Crain model considers only how such factors as education, the extent of broadband use, and regulations affect economic growth without considering how economic growth can affect education, broadband use, and regulations.
• It uses a composite World Bank measure of “regulatory quality” that may capture a range of factors that could lead to higher levels of economic activity but have nothing to do with the stringency of regulations in a country. Indeed, one of the authors of the World Bank’s “Index of Regulatory Quality” disputes the way it is used in the Crain and Crain study.

As broad evidence of its conceptual flaws, the Crain and Crain regression analysis finds that a country’s economy shrinks as the level of education of its population grows. To unquestioningly accept Crain and Crain’s finding that economic regulations cost $1.2 trillion, one must also believe that more education somehow undermines economic growth.

Flawed data

In addition to flawed methodology, Flaws call for rejecting Crain and Crain model criticizes the results of the Crain and Crain study for its use of an erratic data set. In particular:

• The Crain and Crain data set is missing close to half of the potential observations. The study purports to use data describing various indicators in Organization for Economic Cooperation and Development countries from 2002‒2008 in order to determine the relationship between regulation and GDP. This implies a potential total of 210 (seven years times 30 countries) “observations.”  Yet so much information is missing that 92 (44%) of the observations had to be dropped from the regression model.
• Missing data is primarily due to the education measure used, namely primary school completion rates.
• Among the observations dropped are all observations from five countries and one entire year (2008).  Countries such as Austria are retained, but only partially represented due to incomplete data. For example, while Austria’s information for 2002, 2004, and 2007 are used, information for 2003, 2005, 2006, and 2008 had to be dropped out of the data set.

The use of erratic data sets often yields surprising and statistically fragile findings.  EPI’s report updates the Crain and Crain study with data for 2008 and fills in nearly all of the missing data points.  When this more complete data set is applied, there is no statistically significant relationship between regulatory quality and GDP, meaning that even Crain and Crain’s own flawed model does not provide reliable evidence that regulations impede economic activity.