Carliss Baldwin - Mirroring Hypothesis
A series of 2008 Working Papers has been released by Harvard Professor Carliss Baldwin. (Click here to her page where all ten can be downloaded.) This first paper is "Exploring the Duality Between Product and Organizational Architectures: A Test of the Mirroring Hypothesis." This paper provides keen insight into many of the topics we need to better to understand in developing the People, Ideas & Objects application modules. As discussed in her paper we reviewed here, the mirroring hypothesis was a part of that paper, we now have the opportunity to review the mirroring hypothesis.
Before we begin I want to put forward the Cognitive & Motivational Paradoxes into the discussion as background for the discussion of the justification for radical change of the oil and gas industry, as considered in the Draft Specification. It is suggested in this paper that the rewriting of software applications from scratch has not been done. And I would suggest in the short period of time that software has been used in corporations limits the full scope of our understanding and experience of software. We have also not gone through a comprehensive market meltdown, as the one that we face today, an economic situation that I believe is the result of our inability to make the necessary, wholesale changes to organizations. And as such these changes are not only necessary for the economies to resume their positive attributes, but critical. If as I say in the above referenced blog entry, the constraints of code and customers are too large of a compromise in approaching this situation from an otherwise clean slate. These compromises are too significant to overcome the cognitive and motivational paradoxes. Now lets begin to review this very interesting paper. From a technology implementation point of view, the build up from a blank slate is the easiest approach to providing this value to the industry.
A variety of work has sought to examine the link between a product’s architecture and the characteristics of the organization that develops it. The roots of this work come from the field of organization theory, where it has long been recognized that organizations should be designed to reflect the nature of the tasks that they perform (Lawrence and Lorsch, 1967; Burns and Stalker, 1961). In a similar fashion, transaction cost theory predicts that different organizational forms are required to effectively solve the contractual challenges associated with tasks that possess different levels of uncertainty and interdependency (Williamson, 1985; Teece, 1986). To the degree that different product architectures require a different set of tasks to be performed, this work suggests that organizations and architectures must be aligned. p. 5Alignment of the Joint Operating Committee's (JOC) legal, financial, operational decision making, communication and cultural frameworks with the compliance and governance which has been the sole domain of the bureaucracy, are what is achieved as a secondary benefit of this software development project. Our primary objective is to move the producer firm from a banking mentality to that which is based on the earth science and engineering disciplines; and to innovative off that base of knowledge. This is necessary in order to provide the energy consumer with the energy they demand. Referring back to the motivational & cognitive paradoxes, I would assert that the industry has been unable to meet the markets demands for energy, and almost all producers production profiles are in decline. This is further justification for the radical redesign of the oil and gas industry that is proposed in the Draft Specification.
While the studies above begin with the premise that a development organization must be designed to match the desired structure of a new product, a second stream of work adopts the opposite perspective. It assumes that a development organization’s structure is fixed in the short-medium term, and seeks to understand the impact on new product designs. This view was first articulated by Conway (1968) and is sometimes known as “Conway’s Law.” He states, “Any organization that designs a system will inevitable produce a design whose structure is a copy of the organization’s communication structure.” The dynamics of this law are best illustrated in Henderson and Clark’s (1990) study of the photolithography industry, in which they show that market leadership changed hands each time a new generation of equipment was introduced.I italicized the quotation of Conway's Law to highlight the fact that communication in the oil and gas industry is through the JOC. As we have achieved this alignment in the Draft Specification with the JOC's communication structure, the alignment of the organization will be better able to serve the primary (enabling the earth science and engineering) and secondary reasons (the enhanced innovativeness and performance) from this alignment of the industry. This is a material change to the Draft Specification in that the Communication Structure will be added as the fifth framework that the JOC provides the innovative oil and gas producer.
These observations are traced to the successive failure of leading firms to respond effectively to architectural innovations, which involve significant changes in the way that components are linked together. Such innovations challenge incumbent firms given they destroy the usefulness of the architectural knowledge embedded in their organization structures and information-processing routines, which tend to reflect the existing dominant design (Utterback, 1996). When this design is no longer optimal, they find it difficult to adapt. pp. 5 - 6Again I assert that the reason for the rewrite is that the bureaucracy is unable to make the necessary changes to ensure the producer firms remain innovative and profitable. The inability to adapt to the increased amount of earth science and engineering necessary for each barrel of oil, is the beginning of the end of these bureaucratic organizational structures. I can not see them surviving these changes in the greater economy. And I am certain that the economic meltdown we are currently experience will ensure their demise. That is why we must begin the process of developing the software as described in the Draft Specification.
2.1 Product Architecture and Measures of Modularity
We have purposely defined a modular design structure from the work of Professor Baldwin but more specifically through Professor Richard Langlois. These are accurately summarized as follows.Modularity is a concept that helps us to characterize different product architectures. It refers to the way that a product design is decomposed into different parts or modules. While there are many definitions of modularity, authors tend to agree on the concepts that lie at its heart; the notion of interdependence within modules and independence between modules (Ulrich, 1995). The latter concept is referred to as “loose-coupling.” Modular designs are loosely-coupled in that changes made to one module have little impact on the others. Just as there are degrees of coupling, hence there are degrees of modularity. p. 6
6. Discussion
I am particularly proud of the size of this community. It has been many years in the building and each day I am pleasantly surprised by its scope and scale. The most important aspect of this community at this time is their vested interest in this system and particularly their understanding of the basic ideas and issues. When you have this many people following the ideas in this blog, it reflects that we are on the right track. One other important point that may be off topic a bit, but the size of this blog is well over 600,000 words and reflects the basic idea of using the Joint Operating Committee as the key organizational construct of the oil and gas industry. So many words for just one idea. I can not wait to see what this community does with these ideas when they get finished with it. In this next quote Professor Baldwin notes the products architecture is comprised of more then the functions. Our results make an important contribution to the academy in several ways. First, they reveal substantial differences in the levels of modularity between software systems of similar size and function. The pairs we examine vary by a factor of eight, in terms of the potential for a design change to propagate to other system components. This result has significant implications for those who must design such systems. It shows that a product’s architecture is not wholly determined by function, but is also influenced by a variety of other factors, including the characteristics of the organization within which development occurs. The space of possible designs within which solutions are sought appears to be constrained by the nature of the context within which search occurs. p. 20This communities influence on the Draft Specification and the building of this system will be like no other we have seen to date.
We should note that the mirroring phenomenon is consistent with two rival causal mechanisms. The first is that designs evolve to reflect their development environments. In closed source projects, dedicated teams employed by a single firm and located at a single site develop the design. Problems are solved by face-to-face interaction, and performance “tweaked” by taking advantage of the access that module developers have to the information and solutions developed in other modules.Once introduced to the ideas of this software development project people can begin to see how things fit in naturally. Using the JOC is a very natural way in which the industry operates. The technologies today provide the ability to mitigate the effects of location specific activities. The virtual JOC being the ultimate manifestation of the way in which oil and gas investors can manage their operations.
Even if not an explicit managerial choice, the design naturally becomes more tightly-coupled. By contrast, in open source products, a large and widely distributed team develops the design. Face-to-face communications are rare given most developers never meet, hence fewer connections between the modules are established. The architecture that evolves is more modular as a result of the inherent limitations on communication. p. 21
Alternatively, our observations may be a product of purposeful choices made by the system architects. For closed source products, the sole aim is to develop a product that maximizes performance at a point in time.By defining the modular specification we have what I consider the break from the "old way" of doing things. It is necessary for people to see how and where the system they are going to be involved in is going to be different. Without the overall vision of the Draft Specification we may have regressed into the "old ways" without thinking how this system could truly be different. I like to think that the design of the eleven modules makes it difficult to operate in the "old way" as its inefficiencies and frustrations are always in the way.
The benefits of modularity, given the competitive context, may not be viewed as significant. By contrast, for open source products, the benefits of modularity are far greater. Without a modular design, there is little hope that contributors can understand enough of a design to contribute to it, or develop new features and fix defects without affecting many other parts of the system.
Open source products therefore need to be modular to both attract a developer community and also to facilitate the work of this community. Our data can be explained by either of these causal mechanisms. In practice, both are likely to work in parallel. p. 21
Our work suggests that managers of the innovation process must strive to understand the influences on their design choices that stem directly from the way they are organized. The challenge is that these influences are seldom explicit, but are a result of the complex interplay between a firm’s normal problem solving and information processing routines, and the space of designs that must be searched to arrive at a new solution. While a firm can look backwards and see what kinds of designs it is predisposed to produce, it is hard to look forward, and imagine what new designs might be possible. The commercial software managers we work with almost always think their designs are highly modular. When shown these results however, they realize how much more can be achieved. pp. 21 - 22It should also be evident that the constraints (code and customers) and the motivational and cognitive paradoxes be eliminated from the mindset of the community. To do this I have established a very high bar in which participants in this community need to conduct. This does not preclude anyone from contributing, it only seeks to break the ties with the past so that the unencumbered and unconstrained methods of community involvement are optimized to the best solution. The up to 2,500 word essay expects the community member to apply their experience in the oil and gas industry to the specification in its current state. I believe that this is enough of an exercise to truly have the community optimize the solution. And for like minded individuals to find one another on the wiki. (Closed to the general public.)
Our findings have important implications for development organizations given the recent trend towards “open” innovation and the increased use of partners in R&D projects (Chesbrough, 2003; Iansiti and Levian, 2004; MacCormack et al, 2007). In particular, they imply that these new organizational arrangements will have a distinct impact on the nature of the designs they produce, and hence may affect product performance in unintended ways. In essence, our work suggests that R&D partnering choices, as well as the division of tasks that these choices imply, cannot be managed independently of the design process itself (von Hippel, 1990). Decisions taken in one realm will ultimately affect performance in the other. Managers must understand the implications of these organizational choices, in terms of the constraints they place on the solution space.There is much to do and much to learn in this new project. I can't suggest strongly enough that the future does not include the structured hierarchy in any business operation. That is what is being eliminated in this market meltdown. Our first issue is related to the fact that new organizations are unable to form themselves in productive and efficient ways without the software being in place first. This is why the Baldwin, Lanlgois and others analysis is so necessary to find our way through this future.
Companies today have had the opportunity to change and build this system and they have chosen to ignore it. And that is the expected response. Bureaucracies do not change and it is foolhardy to think so. The change can not be implemented in the manner that is necessary without the complete destruction of the old. To change direction, you must first stop. Does anyone believe that the structured hierarchy will be used in 2025, what about 2015? I suggest it may be sooner then 2011 that we plan to have the retirement party of the last millennium in honor of the bureaucracy.
Our work opens up a number of areas for future study. With respect to methods, we show that dependency analysis provides a powerful lens with which to examine product architecture. While we focus on only a few types of dependency, our methods can be generalized to others, assuming that they can be identified from source code. With respect to studies of modularity, our work provides visibility of a phenomena which was previously hidden, and metrics with which to compare different products. This approach promises to facilitate the study of a variety of important research questions that have previously been answered only via purely descriptive or conceptual work. pp. 22 - 23Professor Baldwin is on the right track here. Her analysis of transactions was the means in which the Accounting Voucher was developed. With the expressed intent to have transaction design be the area of real value generation in oil and gas. Transaction processing has developed to a reasonably high level such that the ability to differentiate ourselves based on transaction processing does not exist. It is a necessity, whereas using Baldwins analysis and tools provides the means in which to design transactions.
I close with two paragraphs of Professor Baldwin that put in perspective the context of this software development project. This is an opportunity that provides the community with significant ability to make the changes and increase the performance of the oil and gas industry.
Does greater modularity require trade-offs with other aspects of performance? Intriguingly, our work suggests that, in practice, many designs are not at the performance “frontier” where a trade-off exists, but lie below it due to architectural inefficiencies or “slack” (MacCormack et al, 2006). If this is true, there may be scope to improve a design along multiple dimensions without a performance penalty.Please, join me here.
Exploring such issues via the measurement of architecture and product performance will help reveal managerial strategies for moving designs towards the frontier. And they will help us understand the trade-offs involved in moving along it. Herbert Simon (1962) was the first to argue for the systematic study of design more than 40 years ago, claiming, ‘…the proper study of mankind is the science of design.’ However, his ambitious vision for the field has proven elusive. The study of design has been constrained by, among other things, limited theory, methods and tools that can deal with the complexity of everyday designs, and more importantly, to make them visible, allowing us to compare their structures. The methods we have developed promise to open up a host of questions that, until now, were beyond our analytical capabilities. p. 23