The economic impact of so-called “right-to-work” (RTW) laws has become a hotly contested issue in recent years. These laws restrict the ability of unions to collect dues from workers whose interests they represent. Advocates for these laws claim that RTW status can boost employment in a state, because, they argue, it will attract businesses with lower labor costs. Those opposed to RTW laws claim that by hamstringing the power of unions, these laws can lower workers’ wages, disproportionately so for low- and moderate-wage workers.

The latest attempt to assess the economic impact of RTW laws is a study from Deskins, Bowen, and Christiadi (2015) associated with the West Virginia University School of Business (the “WVU study” henceforth). In this latest study, the authors claim to identify the causal effect of RTW laws on employment growth rates by examining a panel of state-level data on employment from 1990 to 2013. In doing so, they claim that RTW laws lead to faster employment growth. While they would appear on first glance to have an impressive dataset to tackle an ambitious question, the WVU study is fraught with several problems, outlined here and described in more detail below:

1. The WVU study does not have sufficient variation in RTW status within states during the study period to support the strong causal claims it makes about RTW laws. Convincing analysis of employment trends (either levels or growth rates) should rely on identification from states that switch from RTW to non-RTW or from non-RTW to RTW in the panel data. Between 1990 and 2010, there is only one state that meaningfully changed its RTW status, which does not provide enough variation to make a determination on employment growth rates.
2. The WVU study authors mistakenly claim that Texas became RTW in 1993 and Utah became RTW in 1995. While there is no evidence why they claim that Utah became RTW in 1995 rather than 1955, in 1993 Texas did pass legislation that modified the RTW law that had been on the books since 1947. Correctly reclassifying those states as RTW during the entire study period greatly reduces both the magnitude and statistical significance of the posited relationship between employment growth and RTW status.
3. The WVU study fails to include state fixed effects in its analysis. State fixed effects are the industry standard in conducting analysis of this type because they account for characteristics that are particular to a given state and not controlled for in other variables in the model. Employing fixed effects, Jepsen et al. (2014) and Eren and Ozbeklik (2016) find no such causal relationship running from RTW status to improved economic outcomes across states.

On their own, those first three arguments are enough to allow serious researchers and conscientious policymakers to disregard the WVU study results. When the appropriate adjustments are made to its model specification, the relationship between RTW and employment growth disappears.

Even outside these fatal considerations, there remain other problems with the WVU analysis:

4. The WVU regression analysis fails to acknowledge the fact that state data are highly correlated from one year to the next, so even in its multivariate regression model, it is likely that the growth rate (conditional on the controls) is correlated over time, as is the policy “treatment” variable (RTW status). If one erroneously treats state-year observations as fully independent of one another, then one will very likely underestimate standard errors and may overstate the statistical significance of any regression results.
5. The WVU study appears to use some incorrectly measured data and improperly employs others. For instance, the authors suggest their variable of interest is total employment, but they appear to have limited their data to private-sector employment. However, when we pull state-level employment data from the same source they employ, we find different employment levels than they report. This calls into question their proper collection of reliable data on other variables.

There is insufficient variation in RTW status to assess growth rate differences

The WVU study authors claim to be able to identify the causal effect of RTW laws on employment growth rates by examining a panel of state-level data on employment from 1990 to 2013. The first thing to note about this claim is how different it is from most other assessments of the effect of RTW legislation. Most previous assessments have posited a relationship between levels of employment (or wages) and RTW laws, not growth rates.

For example, the most detailed studies on the correlation of RTW laws and wage levels across U.S. states are Gould and Shierholz (2011) and Gould and Kimball (2015), who find that wage levels in RTW states are roughly 3 percent lower than in other states even after controlling for a comprehensive range of wage determinants besides RTW status. As we will highlight below, the nonstandard use of growth rates instead of levels in the WVU study becomes particularly problematic upon realizing that all of the positive effects of RTW status in the WVU dataset come from states that adopted RTW laws decades before the dataset begins.

Such effects seem hard to credit. States either are or are not RTW, and they can only choose to adopt RTW status once. Whatever economic mechanism links RTW status with employment should really be a one-time, discrete shock. If RTW status, for example, leads to lower wage levels because the hamstringing of union power reduces workers’ bargaining power, then perhaps the lower wage level should lead some employers to migrate to RTW states. But this employment migration to lower-wage states should in turn lead only to a one-time, level shift in employment (as opposed to a change in the rate of growth). Granted, as states move to the new wage or employment levels resulting from RTW status, there could be a temporary change in relative growth rates relative to non-RTW states. But, as we point out below in more detail, the timespan over which the WVU study implicitly posits that these adjustments would have to take place is just enormously, even implausibly, long.

Finally, besides the issues raised by using growth rates rather than levels as the dependent variable, a number of statistical issues remain. In theory, the long panel dataset assembled by the WVU authors (48 states over 20 years) could provide enough observations to enable economic relationships to be well-estimated. However, their data set does not actually solve the most vexing problem in trying to assess the effect of RTW status on employment growth: the relatively small number of independent observations on states’ pre- and post-RTW performance.

Contrary to study claims, Texas and Utah did not switch RTW status during the study period

In their sample, the WVU study authors posit that Utah, Oklahoma, and Texas switched from non-RTW to RTW status between 1990 and 2010. It should be noted that Texas merely “modified” its already existing RTW law in 1993 (Collins 2014). The original legislation was enacted in 1947; only minor changes were made in 1993, which did not expand RTW during the period in question.1 Thus, it’s inappropriate to label Texas as a “switcher” state in the study period because it became RTW in 1947. In fact, the 1993 act was titled “non-substantive changes to the Texas Labor Code.” And then there is the question of Utah. It is unclear why the WVU study lists Utah’s RTW legislation as occurring in 1995, when the National Right to Work Legal Defense Foundation, among other sources, clearly lists the enactment of Utah’s RTW law as 1955 (NRTW).

Before moving on to the multivariate regressions, the WVU study illustrates employment growth in switching states, including those outside the scope of its regression analysis. Bartik (2016) calls into question the WVU study on the grounds that 10 states that switched to RTW saw no improvement in employment growth in subsequent years. In an effort to better understand the WVU authors’ data and results, we replicate the figures in question for the three states that they claimed to have switched in their period of analysis, using the Current Employment Statistics and Federal Reserve Economic Data.

As shown in Figure A, Texas had only trivially faster employment growth (well under half a percentage point more rapid growth) in the 20 years after the authors posited that RTW status was passed. Figure B shows that Utah saw average growth of 1.4 percentage points lower following the supposed RTW switch. The actual remaining switcher, Oklahoma (Figure C), saw employment growth in the 10 years after RTW status was passed that was nearly 1.8 percentage points slower than what prevailed in the 10 years before. We acknowledge that there are a number of reasons why employment growth would be lower in the latter years in Oklahoma. For instance, it includes the years of the Great Recession and this is why multivariate regression analysis is important. However, this naive look at the employment changes in the one state that actually switched its RTW status calls into question the WVU study results.

What this means is that the WVU study’s results are driven by a number of states that adopted RTW decades before the dataset even begins. For example, the vast majority of RTW states (18 of 25) adopted RTW status before 1960 (Collins 2014). The idea that adopting right-to-work status in the 1940s or 1950s strongly influenced employment growth in the 1990s and 2000s does not seem particularly persuasive given the huge range of other potential influences on employment growth across states in the second half of the 20th century.

Therefore, the WVU study claim that right-to-work status generates faster employment growth is totally based on the employment performance of states that have had right-to-work status for many decades. Yet, the adoption of right-to-work legislation would be expected, at best, to have a one-time effect occurring over a few years, not a decades-long impact.

The need for state fixed effects

The authors regress employment growth in a given state in one year against RTW status three years before. While this nonstandard regression framework essentially means that only a state’s RTW status before 2011 will be assessed, their regression results show a positive and statistically significant relationship between employment growth in a state/year observation and RTW status three years previous. While this relationship holds even when a number of controls are included, the authors are missing key state explanatory variables, notably state fixed effects.

Previous studies have demonstrated that RTW states have a number of characteristics besides their RTW status that are similar, making the potential problem of omitted variables driving the results particularly worrisome. In relatively long panel data a very common sensitivity test to probe the durability of regression results is the inclusion of state fixed effects—indicators that control for characteristics that are particular to a given state and not controlled for specifically in the remaining control variables.

A recent study at the University of Kentucky (Jepsen et al. 2014) examined economic development trends across states and found no impact of right-to-work status. The model used in this study used fixed effects “to capture unobserved differences in states that may affect economic growth. These effects include things such as a state’s climate, access to overseas markets or its citizens’ work ethic.”

Furthermore, Timothy Bartik (2016), a leading economic development expert at Upjohn Institute, recently reviewed the WVU study and concluded that it “does not provide any convincing evidence that a state that adopts RTW laws will as a result experience faster job growth.” Bartik reached this conclusion in part because the WVU study failed to include state fixed effects. Bartik also notes that “the most rigorous recent study that looks at Right-to-Work,” which focuses on Oklahoma, found no significant effect on employment growth (Eren and Ozbeklik 2016).

The WVU study authors argued against the inclusion of state fixed effects in their regression analysis on the grounds that because only three (arguably only one) states “switched” from non-RTW to RTW status between 1990 and 2010, that statistical identification in the fixed-effect regressions is based on only this very small number of “switcher” states. But this is the entire point of why their panel dataset is the wrong tool to use to assess the causal impact of RTW status on employment growth—despite the large number of state/year observations, it just doesn’t have enough pre- and post-RTW observations within states to make strong inferences.

To assess the effect of the correct classification of Texas and Utah and the inclusion of state-level fixed effects, we begin by replicating Table 3 of the WVU study to the best of our ability. We were able to match its results in Models 1 and 2, but did not have the additional variable to replicate Model 3 in its entirety. Our regression results are found in Table 1.2

The first and fourth columns replicate the WVU study results. When state fixed effects are included in the WVU study’s baseline regression as shown in the second column, the relationship between employment growth and (three-years-previous) RTW status gets smaller and becomes statistically insignificant.

This failure to find a statistically significant relationship between RTW and employment growth when a standard control such as state fixed effects is included is the most salient thing to know about the WVU study. State fixed effects are a completely standard robustness check for panel data and the WVU results fail this check, meaning one should be quite skeptical about their final results.

In the third column, we go one step further and classify Texas and Utah correctly as RTW states during the entire period. The coefficient gets even smaller. The fifth column of Table 1 displays Model 2 with state fixed effects. Here, the magnitude of the result increases and remains statistically significant. However, once we correctly reclassify Texas and Utah, the economic and statistical significance disappears.

Uncorrected autocorrelation of errors overstates statistical significance

The WVU regression analysis fails to acknowledge the fact that state data are highly correlated from one year to the next so even in its multivariate regression model, it is likely that the growth rate (conditional on the controls) and the policy variable (RTW) is correlated over time. For example, if employment growth for one state in one year is above its conditional mean, then it’s likely that it will be above its mean in the next period. This suggests that there is serial correlation in the error terms. Traditionally when this is corrected, this increases standard errors. If standard errors go up, then confidence intervals get wider and statistical significance goes down.

The WVU study makes no mention of adjusting its standard errors for this serial correlation. Correcting standard errors in the presence of serial correlation is standard practice and likely would have increased standard errors. If one erroneously treats state-year observations as fully independent one another, then one will very likely underestimate standard errors and may overstate the statistical significance of any regression results.

We used a simple (and almost surely insufficient) correction to their Model 2 specification, adding in both state fixed effects and robust standard errors (accounting for the serial correlation).3 Those results are shown in the last column of Table 1. Even employing the model that wrongfully includes Texas and Utah as switchers, the WVU study results are nullified.

Other issues with the WVU data and methodology

The WVU study appears to use some incorrectly measured data and improperly employs others. For instance, the authors suggest their measured key variable of interest is total employment. But, the published Current Employment Statistics data on state-level employment has differed by 15–30 percent from their data.4 When we investigated further, it appears that they limited their data to private-sector employment. When we tried to match their employment levels using the Current Employment Statistics, we still found significant differences. We found their state employment levels vary anywhere from 0 percent to 7.8 percent from what is found in the CES. Given the lack of clarity and discrepancy in measurement, it calls into question the validity of other variables in their model.

Other studies that have attempted to identify correlations between RTW status and economic variables have tended to look at cross-sectional variation—sometimes in growth rates over relatively long (5- or 10-year) periods, rather than annually. We experimented with regressing state-level employment growth over the 1990–2000 and 2000–2007 periods on a number of controls. We also included some control variables that other studies have indicated may be correlated with employment growth across states, including the number of sunny days per year, the relative median rent in a state, and the starting level of employment in a state. In these regressions (available upon request) the RTW variable often became statistically insignificant depending on the precise constellation of other controls included.

Conclusion

In the United States, 25 of the 50 states had adopted RTW policies by 2015. This could conceivably give one enough data to assess the causal impact of RTW policies by looking at economic variables before and after RTW adoption and using controls such as state fixed effects. However, 18 of the RTW states had adopted these laws before 1960. This means that state-level data on economic outcomes (employment or wages, for example) would need to be found that extended quite far back in history. Until such data is assembled, it will be hard indeed to make strong claims about the causal impact of RTW status. But the latest WVU study does not acknowledge these difficulties and presents claims about the impact of RTW status that are just not supported by the data.

In short, while a naive assessment of the WVU dataset seems to indicate a large number of observations (48 states by 25 years), the small number of RTW “switchers” in the data (namely one) make it impossible to reliably identify the causal impact of RTW on employment growth. Our reasonable set of robustness test yields no relationship between RTW status and employment growth.

Endnotes

1. EPI analysis of Texas Labor Code, Title 3, Chapter 101, revisions effective Sept. 1, 1993. Also, see: http://www.lrl.state.tx.us/legis/billsearch/billdetails.cfm?billtypedetail=HB&billnumberdetail=752&legSession=73-0

2. Replication possible with dataset supplied by WVU study authors upon request, December 2015

3. The very limited extent of their panel data (specifically, the small number of states “switching” RTW status) does not allow the type of rigorous correction for the problem of serial correlation suggested in Bertand et al. (2004). In fact, the WVU study’s panel dataset is even too small to allow for the test Bertrand et al. (2004) submit as appropriate for a panel dataset with a “small” number of switchers.

4.  Email correspondence with WVU study authors, December 17, 2015.

Appendix: Measuring employment in states

—This report was amended January 19 to include the text of this appendix.

The WVU study’s most important findings are those regarding the impact of right-to-work status on state employment growth. However, we tried but could not replicate the study authors’ data with what is available from the Bureau of Labor Statistics website.

The WVU study’s authors describe their work by saying “we examine overall employment” (page 11). Their data appendix says they measure “Employment Growth” by “Year t to t+3 growth rate in total employment, by state.” They list their data source as “Current Employment Statistics Survey, US Bureau of Labor Statistics.”

We interpret “total employment” to be total nonfarm employment since that is the most aggregate category available from BLS. Appendix Table 1 provides a comparison of the WVU study’s data for 1990 and 2013, the beginning and end of the period of their study, with the data we developed from BLS for those years, both private sector and nonfarm (which should match their “total employment”). For the U.S. as a whole, total nonfarm employment in 1990 was 108.3 million, nearly 20 million more than the WVU study’s employment total of 89.8 million. This is why we also benchmarked their data to private-sector employment, which, for 1990, turns out to be close, just over 200,000 off. So, it appears that the WVU study may simply have mislabeled their results and relied on private-sector employment rather than total employment. However, the data comparison in the latest year, 2013, is not as close; there is a 1.5 million difference between the WVU study’s employment number (111.7 million) and our retrieval of BLS data on private-sector employment (113.2 million). BLS data on state employment comes rounded to the nearest hundred; however, the WVU study employment data was provided without rounding. We tried to replicate its data by averaging three years, which could produce unrounded values, but this didn’t explain the discrepancy.

What matters for this study is whether employment growth (not levels) is measured correctly. Simple tabulations of the growth of employment from 1990 to 2013 in each state using the WVU study’s data and the BLS private-sector employment data show that there are several large discrepancies. For instance, our downloaded BLS private-sector employment data show that West Virginia employment grew by 21.6 percent while the WVU study’s data suggest a 17.1 percent growth. There are also sizeable discrepancies of at least four percentage points in Arizona, California, and Idaho. The average discrepancy is only 1.1 percent but this reflects some states where employment growth is overstated by the WVU study and some states where employment growth is understated.

We conclude that at minimum their data are mislabeled as “total employment” when they should be labeled “private employment.”

References

Bartik, Timothy. 2016. “What do we know about right-to-work laws and state prosperity? Not much, because of limited variation over time in which states are RTW states.” Investing in Kids (blog). January 5.

Bertrand, Marianne, Esther Duflo, and Sendhil Mullainathan. 2004. “How Much Should We Trust Difference-in-Difference Estimators?” Quarterly Journal of Economics, 119(1), pp. 249–275

Bureau of Labor Statistics (U.S. Department of Labor) Current Employment Statistics Program. Various years. Regional and State Employment and Unemployment [database].

Collins, Benjamin. 2014. Right to Work Laws: Legislative Background and Empirical Research. Congressional Research Service report.

Deskins, John, Eric Bowen, and Christiadi. 2015. The Economic Impact of Right to Work Policy in West Virginia. Bureau of Business and Economic Research report.

Eren, Ozkan, and Serkan Ozbeklik. 2016. “What Do Right-to-Work Laws Do? A Case Study Analysis Using Synthetic Control Method.” Journal of Policy Analysis and Management, vol. 35, 173–194.

Federal Reserve Bank of St. Louis. Federal Reserve Economic Data (FRED) [database].

Gould, Elise, and Heidi Shierholz. 2011. The Compensation Penalty of “Right-to-Work” Laws. Economic Policy Institute, Briefing Paper No. 299.

Gould, Elise, and Will Kimball. 2015. “Right-to-Work” States Still Have Lower Wages. Economic Policy Institute, Briefing Paper No. 395.

Jepsen, Christopher; Sanford, Kenneth; and Troske, Kenneth R. 2008. Economic Growth in Kentucky: Why Does Kentucky Lag Behind the Rest of the South?. Center for Business and Economic Research, University of Kentucky.

National Right to Work Legal Defense Foundation, Inc. (NRTW) “Right to Work States: Utah.”