1. Introduction

The impetus behind the papers that appear in this special issue was to examine how patterns of Ostrom’s design principles relate to outcomes across diverse CPR settings. Poteete et al. (2010) noted the lack of comparative CPR studies, especially ones involving different types of CPRs, such as irrigation systems, fisheries, forests, and so on. The starting point for the research project was the data collected by Cox et al. (2010) “A Review of Design Principles for Community-based Natural Resource Management”, a meta-analysis of 91 studies that applied E Ostrom’s design principles to instances of local level, self-governance of common pool resources. The purpose of the meta-analysis “was to conduct a review of the relevant literature to document its findings and re-evaluate the principles” twenty years after E Ostrom (1990) first proposed them in Governing the Commons (Cox et al. 2010, 2). Cox et al. (2010) found that the design principles held up after multiple applications by numerous scholars across many settings. The design principles were associated with successful outcomes, leading Cox et al. (2010) to conclude that the design principles “are a sound basis for future research”.

The authors of the manuscripts in this special issue, led by Marty Anderies and Marco Janssen of Arizona State University, launched a project to extend the research and data of Cox et al. (2010) in three critical ways. First, they wanted to engage in an explicit comparison of design principles and outcomes across CPR types. Second, they wanted to examine the combinations of design principles associated with success and whether the combinations varied by CPR type. Third, the original CPR data set collected by Ostrom et al. (1989) did not include data on the design principles. The effort was to use the original CPR coding forms to construct measures of the design principles so that the original CPR data set (Ostrom et al. 1989) could be merged with the Cox et al. (2010) data set. In setting out to achieve these goals, the scholars ended up addressing a number of issues that took them beyond their original idea of comparative analysis of CPR systems. This special issue reflects the research team’s experiences and findings in engaging in a complex research program whose starting point was data collected by a different team of scholars. The manuscripts make important methodological and theoretical contributions to a major research program on common pool resources begun by Elinor Ostrom in the 1980s.

This introduction provides an over view of each of the special issue papers situating them within the larger CPR research program and the important contributions made to that research program. In addition, how transparent methods contribute to theory building and how theory building suggests new avenues of research to be addressed by using transparent methods is explored. Finally, the introduction concludes with a discussion of frameworks, particularly the CIS and IAD, and the contribution of frameworks to supporting the scientific enterprise and the cumulation of knowledge, and how the Complex Infrastructure Systems Framework relates to the IAD framework.

2. Contributions to the CPR research program

The special issue begins with Baggio et al. (2016) “The Puzzle to Govern the Commons”, which explores whether and how configurations of design principles are associated with success and failure in governing common pool resources. Examining configurations of design principles aligns with the configural nature of the IAD framework (Ostrom 2005). The core of the IAD framework is the action situation, which is structured by a configuration of rules – boundary, position, choice, information, aggregation, scope, and payoff – and of community characteristics and biophysical dimensions (Ostrom 2005). Configurations, or patterns, of rules, interactions, and outcomes are often contingent upon the specific type of common pool resource (Baggio et al. 2016). Each of the design principles are configurations of rules, consequently, a number of the design principles can be conceptualized as action situations, some occurring at the operational level of action, such as well matched and equitable rules, and others, such as monitoring and conflict resolution, at the collective choice level of action. Thus, in examining configurations of design principles, Baggio et al. (2016) are exploring combinations of linked action situations in 69 distinct settings.1

Systematically comparing configurations of design principles across cases requires the use of data analysis tools sensitive to configural or case analysis. Baggio et al. (2016) use a variety of tools that maintain the integrity of the cases while comparing configurations of design principles. In particular, they use Qualitative Comparative Analysis to examine how different patterns of design principles lead to successful or unsuccessful outcomes. One of the pioneers of QCA, Ragin (1987, 20) calls this “deciphering order in complexity”. The goal is to understand the cases in their entirety and in relation to one another rather than reducing them to collections of variables and examining the effects of a single variable on an outcome, holding constant, or subtracting out, the effects of other variables.

What Baggio et al. (2016) find is important. First, no single design principle is necessary and sufficient for the successful governance of a common pool resource. Rather, configurations of design principles are necessary for success. Overall, the more design principles present in a case, the more likely the case is successful. Second, particular patterns of design principles are associated with success, and those patterns vary by type of resource. Third, particular patterns of the absence of design principles are associated with unsuccessful cases. The first two points most likely elicit a shoulder shrug from readers. So what? Tell us something we didn’t know. But, I find it reassuring that the first design principle – well defined boundaries – is necessary but not sufficient for success. Well defined boundaries present the possibility that resource users will be able to capture the benefits from investing in and implementing the remaining design principles. Without the ability to capture those benefits, why invest? Furthermore, I was agnostic about the numbers of design principles associated with success. Ostrom (1990) argued that the numbers and combinations of design principles that support successful outcomes was an open question. Baggio et al. (2016) shed light on this open question. Finally, the findings around unsuccessful cases are intriguing. If cases lack rule congruence (principles 2A and 2B), accountable monitors (principle 4B), and graduated sanctions (principle 5) they are unsuccessful. Without well designed rules that are appropriately monitored and enforced, resources users are unlikely to experience success. Rules matter, and so does their absence!

As anyone who worked closely with or who trained under E Ostrom is quick to point out – design principles are not a panacea (Ostrom et al. 2007). But what students and scholars who work in the Ostrom tradition need to engage with more are the different combinations of design principles and how they relate to different CPR settings. The Baggio et al. (2016) paper represents an important and much needed step in this direction. An early paper (Schlager et al. 1994) theorizing on differences in the governance of access and use among different types of common pool resources, argued that differences in storage and mobility contributed to differences in governance. These dimensions of the resource system and resource units affected the quality of information resources users could develop to understand the dynamics of resource systems and their ability to control access and use. In systems characterized by storage or stationarity, resource users are more likely to develop quantity restrictions on use because they can exercise control over resource units. However, in systems characterized by lack of storage and mobility resource users are more likely to develop spatial and temporal restrictions on access and use because users may exercise some control over space, but little control over mobile resource units (Schlager et al. 1994, 301). Rather than exploring differences in access and use rules, Baggio et al. (2016) examine differences in patterns of design principles as a function of mobility and the intensity of hard human made infrastructure. In systems with low mobility of resource units, monitoring and graduated sanctions are prevalent in successful cases. In systems with high mobility, well defined boundaries of resource users, or design principle 1, appears necessary for success. Why these patterns of design principles are present across different types of CPRs is not clear, but points to a productive line of research.

The second manuscript, Ratajczyk et al. (2016) “Challenges and opportunities in coding the commons: problems, procedures, and potential solutions in large-N comparative case studies”, provides the methodological foundation for the Baggio et al. (2016) paper. From the very beginning of the common pool resource research program, case studies and ethnographies of efforts by resource users to sustainably govern CPRs have been important sources of data for the study of the commons (National Research Council 1986). However, as Ratajczyk et al. (2016), point out, developing reliable and valid measures of variables from secondary sources present a number of challenges, from many different variables and operationalizations used to measure similar concepts, to varying scopes and scales of the cases studied, to the quality and completeness of the data presented.

One widely accepted way of imposing consistency and reliability on secondary sources of data is through the use of coding forms composed of well defined sets of measures of theoretically relevant variables. The use of coding forms to develop coherent data sets from diverse sources has an extensive history in the social sciences. For instance, Harold Lasswell, considered one of the founders of modern political science, claimed to have pioneered the use of content analysis in his 1927 dissertation examining the uses of propaganda during World War I (Neuendorf 2017). Several well known contemporary projects grounded in explicit and transparent coding forms and protocols with the express purpose of encouraging the cumulation of scientific knowledge include the Correlates of War project and the Comparative Policy Agendas project, both of which have made extensive data sets widely available to scholars and practitioners.2 Thus, the common pool resource project sits in good company, and the project itself, under the leadership of Elinor Ostrom, directly led to the development of two additional research projects, one on irrigation systems in Nepal, and one on community forests located in numerous countries.3 The effort on the part of Marty Anderies and Marco Janssen, through the Center for Behavior, Institutions, and the Environment, at Arizona State University to host and make available the coding forms and data sets of the different common pool resource projects is an important step in supporting rigorous scholarship on the study of the commons.

The original CPR coding forms, used by the scholars represented in this special issue, were developed by Elinor Ostrom and colleagues in the mid-1980s (Ostrom et al. 1989). The coding forms predate the design principles and were explicitly grounded in the IAD framework. Action situations at the operational and collective choice levels, outcomes, rules-in-use, community characteristics, and biophysical features, are all captured in the coding forms. Considerable effort was devoted to developing questions applicable to all types of common pool resources and not to just a particular type of resource, although questions unique to irrigation systems were included to capture the distinct infrastructure of canals, diversions, dams, and tanks and reservoirs. The ASU research team, in coding a subset of cases from Cox et al. (2010), used a selection of questions from the common pool resource project coding forms that most closely measured the different dimensions of the Ostrom design principles, and biophysical and social outcomes. In addition, they coded for the presence or absence of the design principles (see Tables 2 and 3 in Ratajczyk et al. 2016).

Explicitly identifying variables and how they are measured are two of several activities researchers should take to ensure transparency of methods. Ratajczyk et al. (2016) carefully identify and explain best practices for engaging in content analysis of cases. The best practices are commonly exhorted in a variety of methods textbooks, but as Ratajczyk et al. (2016) note, many of these practices are not commonly practiced in studies of the commons, in particular, scholars do not share coding forms or protocols and they do not report intercoder reliability scores. The two go hand in hand – achieving acceptable intercoder reliability scores is not possible without well developed and commonly understood coding protocols that are carefully followed as cases are coded. Reliability in the coding of documents is determined by intercoder reliability scores. As Neuendorf (2017, 19), a recognized expert in content analysis states, “…reliability is paramount. Without acceptable levels of reliability, content analysis measures are meaningless.”

Readers of this special issue who are not familiar with content analysis may be surprised by the methodological rigor of the best practices and the time and labor needed to consistently and reliably code dozens of cases. Neuendorf (2017, 7–8) discusses several myths of content analysis, one of them being that “it doesn’t take any special preparation”. High quality analysis of secondary sources requires substantial research planning, as Ratajczyk et al. (2016) carefully spell out. They provide an important service to commons scholars by documenting the steps taken by the ASU research team in coding dozens of cases. Commons scholars should follow in their footsteps and journal editors should encourage the dissemination of best practices by requiring authors to publish their coding protocols, report intercoder reliability scores, and share their data. One result of these painstaking steps is transparency. This allows other scholars to assess the quality of the research, and it provides the possibility of reproducibility.

Reproducibility, or replication is a highly salient topic. In the past two years (2015 and 2016) Science has published the results of two major efforts to reproduce psychology and economics laboratory experiments (Open Science Collaboration 2015; Camerer et al. 2016). Replication rates were notably lower than expected. For instance, for economics, just over 60% of the replicated studies exhibited significant effects in the same direction as the original studies (Camerer et al. 2016, 1434). The papers’ authors raise questions of whether knowledge in psychology and economics is as firm as it is assumed. As the Open Science Collaboration (2015, 943) points out, “there is still more work to do to verify whether we know what we think we know”.

Replication is considered the gold standard in science. Science, among other things, makes claims about how the world works. Confidence in those claims increases as they are replicated. However, before replication is possible, scientists must engage in a series of activities that not only permit replication, but that provide the foundation for the development and cumulation of science based knowledge. Without these activities, which include well developed research designs that explicitly and transparently identify how data is gathered and analyzed, hypotheses cannot be tested, theory cannot be developed, extended, and revised; and without making the research designs, data gathering instruments, and data publicly available, studies cannot be replicated. More importantly, the legitimacy and credibility of the scientific enterprise is called into question without the ability and opportunity to examine and question the basis for scientifically produced claims.4

The third manuscript, Barnett et al. (2016) “An iterative approach to large-N studies: insights from qualitative analysis of quantitative inconsistencies” draws on the methods of Ratajczyk et al. (2016) to build off of the findings of Baggio et al. (2016) and illustrate the value of complementing quantitative analysis with qualitative analysis. Recall, Baggio et al. (2016) found that successful cases of CPR governance exhibited 8 or more design principles. Barnett et al. (2016), using qualitative methods, examine two sets of what appear to be anomalous cases. Type 1 anomalies are cases that exhibit most of the design principles but are not characterized by successful outcomes. Type 2 cases exhibit few design principles but are successful. Are these cases really anomalies or are they artifacts of human biases? Or, are they due to missing data? An important issue that Barnett et al. (2016) address is how to handle and interpret missing data. As they explain a missing design principle means that the author of the case did not report on it. It may be present, or absent, but the coder cannot make a determination from the evidence provided. Type 2 cases (few design principles, but successful outcomes) are especially prone to the missing data issue. If they exhibit few design principles because of missing data, then uncertainty over how to classify them is high. They could, in fact, fit with CPR theory, if the design principles are present (but not reported). Or, they could be anomalies if design principles are absent. An analyst cannot distinguish between the two. Besides missing data, Barnett et al. (2016) also note different types of human biases that may affect the interpretation of cases. These include investigator bias – the authors of the case study fail to attend to some design principles or neglect alternative explanations of outcomes, and procedural error – mistakes are made in coding the case.

Barnett et al. (2016) carefully examine each of the Type 1 and Type 2 cases to determine whether they may be explained by biases and errors or if they are anomalies and why. Among the cases they find evidence of the different types of biases, but they also find theoretically interesting issues. For instance, in a couple of the Type 1 cases it appears that a single absent design principle is key for explaining the lack of success, demonstrating how design principles are configural – they don’t just occur together, they interact to produce outcomes. In addition, a methodological issue that requires attention is time. How analysts handle the temporal dimension of a case affects how outcomes are assessed. Dividing a case into multiple time periods and treating each period separately, as was done in a couple of cases, affects the assessment of the cases. Finally, some of the Type 2 cases likely should not have been included in the analysis according to Barnett et al. (2016) as they did not represent or exhibit common pool resource dilemmas.

The two empirical manuscripts (Baggio et al. 2016; Barnett et al. 2016) contribute theoretical and methodological insights to the Ostrom based common pool resource program. Different configurations of design principles are associated with successful and unsuccessful outcomes. The configural nature of the design principles means that in some instances, the absence of a single design principle will lead to failure. Furthermore, the design principles are appropriate for settings in which resource users experience dilemmas that require coordination and cooperation to address. Institutional arrangements are costly and challenging to devise, as E Ostrom (1990, 2005) repeatedly noted. Resource users are unlikely to invest in institutional arrangements unless they face significant issues that they believe they can resolve through cooperation. The design principles help account for the outcomes in CPR dilemma settings, but are not appropriate to explain settings without dilemmas. In addition, both Baggio et al. (2016) and Barnett et al. (2016) handle missing data explicitly and in appropriate ways. Missing data is a major issue in analyzing data gathered from secondary sources and the two manuscripts demonstrate different approaches for dealing with it. Finally, Ratajczyk et al. (2016) provide a much needed presentation and discussion of best practices for coding secondary sources.

3. The role of frameworks in advancing knowledge

For the social sciences, however, good science practices may not be sufficient if knowledge cumulation and sage advice to policy makers is the goal. As both Vincent and Elinor Ostrom repeatedly noted throughout their careers, human language is the source of much uncertainty and confusion. People use different terms for the same concept or object and similar terms for different concepts and objects. For instance, in a landmark work, Crawford and Ostrom (1995, 589) identified the many different usages and meanings of the term institutions. Without common agreement on concepts, what they represent, and how they may be measured, it is difficult to develop research projects that address shared questions and produce results that are comparable. In other words, without shared language, it is difficult to cumulate knowledge.

One of the motivations for developing the Institutional Analysis and Development Framework, according to Elinor Ostrom (1999, 2005) was to provide a common set of concepts and terms that scholars from many different disciplines could use to study institutional arrangements. These concepts and terms also form a meta theoretical language that allow for communication among research projects and comparability of research results. For instance, the IAD framework forms the foundation of the original CPR research project and the coding forms referenced in and used by Ratajczyk et al. (2016). The Nepal Irrigation Institutions Study used the same coding forms. The coding forms were extended and revised for the International Forestry Resources and Institutions Project. All three research projects share a common foundation, the IAD framework, and shared and overlapping coding forms, allowing cases from the projects to be analyzed and compared using similar variables (Ostrom et al. 2014).

Just as important as providing a shared language, a framework also organizes the study of a particular topic. The concepts and terms are organized in relation to one another. For the IAD, the action situation and its constituents parts are the starting point for engaging in institutional analysis. Action situations take place at different levels of action – operational, collective choice, and constitutional choice. But no matter the level of action they are structured by biophysical and community characteristics, and rules-in-use. Thus, the assumption built into the IAD is that the appropriate way to analyze institutional arrangements is to examine how they structure, guide, and constrain human interactions in interdependent situations. The outcome an individual actor achieves depends on her choices as well as the choices of others in the situation (Ostrom et al. 1994). As Anderies et al. (2016) note, the conventional application of the IAD framework is to examine common pool resource situations at a point in time, allowing scholars to examine how a fixed configuration of biophysical and community characteristics along with rules in use structure the interactions among actors and the outcomes they achieve (Tang 1992, Schlager 1994).

The IAD framework with its feedback loops, however, holds the potential to study common pool resources as dynamic systems with the biophysical and community characteristics, rules in use, and action situations co-evolving (Anderies et al. 2016). Anderies et al. (2004) first proposed re-organizing the IAD framework to emphasize dynamic systems in the form of the Robustness Framework (Anderies et al. 2004). The Robustness Framework treats the common pool resource situation as a system of complex interactions among a resource, resource users, public infrastructure, and public infrastructure providers. It allows scholars to focus on explaining and understanding linked action situations as dynamic systems. No longer is the primary emphasis on institutional arrangements, as is the case with the IAD Framework, rather attention is devoted to system dynamics and robustness.

The fourth manuscript in this special issue, Anderies et al. (2016) “Institutions and the performance of coupled infrastructure systems” further extend and develop the Robustness Framework by conceptualizing the parts as infrastructures. Examples of infrastructures include knowledge; ecosystem processes that provide services such as water cleansing or flood control; roads and canal systems; and institutional arrangements. Infrastructures are foundational, providing opportunities for actors to use infrastructures as inputs to create valued outputs and outcomes. In creating valued outcomes, such as productive irrigation systems, people draw on multiple infrastructures, not a single type of infrastructure. The Coupled Infrastructure Systems Framework captures the dynamics of common pool resource systems by focusing on the opportunities and possibilities, or what Anderies et al. (2016) call affordances, that emerge from the interactions of infrastructures.

How do the design principles fit into the Coupled Infrastructure Systems Framework? The design principles largely represent different types of institutional arrangements that serve a variety of roles in support of collective action, from conflict resolution to rule making. As such, the design principles are a form of soft, human made infrastructure. Anderies et al. (2016) argue the design principles allow for the processing of information that regulates complex systems. More than anything the CIS Framework demonstrates the importance of the co-occurrence of the design principles, why they must co-occur, and why robust systems are characterized by most of the principles.

4. Conclusion

As the Open Science Collaboration (2015, 943) concluded in reflecting on their effort to replicate psychology experiments:

Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not.

This special issue on “The importance of context, scale, and interdependencies in the understanding and applying Ostrom’s design principles for successful governance of the commons” represents both likely and possible paths in better understanding coupled infrastructure systems. It confirms the critical roles that the design principles play in supporting the emergence of effective governance and it suggests future research directions exploring the spillover effects among different types of natural and soft and hard human created infrastructures (Baggio et al. 2016). It demonstrates the value of combining quantitative analysis and qualitative analysis and the role both play in theory testing and extension (Baggio et al. 2016; Barnett et al. 2016). It provides important guidance for scholars who study coupled infrastructure systems to be sufficiently clear and transparent in the design, collection, and analysis of data so that the process may be replicated (Ratajczyk et al. 2016). And it provides a framework, grounded in the IAD framework, that assists scholars in conceptualizing and explaining dynamic complex systems (Anderies et al. 2016). The manuscripts composing this special issue suggest a number of innovative research paths that are likely to keep scholars busy for years to come.