Governance seems to be one of those frightening words that threatens to stop an Agile transformation effort dead in its tracks. I’ve been hearing it whispered, and even screamed once or twice, quite a lot recently. There’s no big surprise here. As the big corporations and Government agencies get increasingly fascinated by frameworks like Scrum, they are mandating their IT departments to “Go agile” and then, sooner or later…governance!
There is operational governance represented in the defined processes that organizations and teams are expected to follow when software is in production. There is project management governance, perhaps dictated by PRINCE2 or similar, while the product is in development. PRINCE, by the way, is an acronym standing for ‘PRojects In a Controlled Environment’.
Project management governance is often a subset of a wider IT governance. According to the TOGAF version 9.1, IT governance supposedly provides the framework and structure that links IT resources and information to enterprise goals and strategies. “IT governance institutionalizes best practices for planning, acquiring, implementing and monitoring IT performance…” Standards like COBIT, which stands for Control OBjectives for Information and related Technology, might be in place. And then, of course, there is a range of issues to do with compliance to the requirements of external regulators in all public bodies, as well as commercial institutions in sectors such as insurance and banking. So, is this a case of an irresistible force (Agile) meeting an unmovable object (governance)?
Let’s go back to first principles. What is governance? Here’s a definition I came across recently in the TOGAF 9.1: “The discipline of monitoring, managing and steering a business to deliver the business outcomes required.” As an Agilist, I have no problem with governance described in this way. An obstinate focus on the delivery of business outcomes is the very stuff of Agile. It is the way governance is applied that is the problem. Typically specialist governance bodies are set up in a hierarchy at each level of which procedures are mandated, documents are required for sign-off, and auditing procedures abound. The Agile principle of trusting professionals to get the job done is about as welcome as a trap door in a canoe. Fear of non-compliance drives very different behaviors from those we are trying to grow with Agile.
When these kinds of bodies and procedures get imposed on software development teams there is one guaranteed result: a delay in the delivery of value to the customer (often a protracted delay at that). Phase-gates block the development path. Waiting for sign-offs builds queues of work items. Sometimes the whole development ‘track’ is idle, like a train sitting at a signal waiting for it to turn green.
The scenarios I have just described are in no way conducive to the achievement of business outcomes. In fact, very often, it is the governance procedures which are the major obstacle to their delivery? Why is this?
Business value is much more likely to be delivered in the Agile Model because of its fast feedback cycles. These are necessary because software development is inherently unpredictable. Its practical processes are more like what goes on in the design rooms of product development than it is like the assembly lines of mass manufacture.
“Design is not passive. It is wise for designers to harmonize with the ways of nature. But the ways of nature follow context and change.”(1) Mass manufacture is predictable. The uncertainties have been ironed out and removed in the design “phase”. But software development is not.
While not everything in software development is design (problem analysis should shape design, after all), its ‘implementation’ is creative. Even when we are crafting code we are only designing the instructions that the computer will run. Design is dominated by uncertainty. New information emerges during the process itself and is incorporated as quickly as possible. Feedback cycles surface that information, allowing the development team to converge on a solution that delivers business value to the customer.
Traditional governance procedures follow the motto “Plan the work; work the plan”. They assume predictability. There are many corporate procedures where governance for predictability is appropriate. Software development is not one of them. Governance for feedback; governance for responsiveness is what is required. In many Agile-adopting organizations, a “two-speed” solution is evident. Where predictability reigns, traditional governance procedures are maintained. Where feedback drives success, different approaches operate.
In Scrum, the Product Owner is responsible for achieving the business outcomes inherent in the product development. She owns the product for the business, and is accountable for, amongst other things, its alignment with the strategic goals of the business. She is also a peer member of the Scrum team. One of the reasons you rarely see formal, outline business cases, followed by detailed business cases and post-project audits against them in Scrum is because it is unnecessary. The Product Owner role and the Sprints’ inspect-and-adapt cycle takes care of that stuff in a more ‘light touch’ way.
Similarly, the quality of the product is best ensured by embedding testers as developers in the Scrum team. The goal of testing then itself becomes feedback. The ‘Whack-The-Mole’ pattern means defects can be removed by the Development team before the increment gets to the Sprint Review, let alone Release. The ping-pong between programmers and testers that so characterizes (and delays) waterfall development is eliminated by making the Scrum team responsible for quality.
While any overall solution to the governance issue will be situational, and may yet require some external oversight, the general line of approach seems obvious to me. Business stewardship and quality assurance are, in my opinion, stronger in Scrum than they are in waterfall precisely because the specialist skills needed have been brought into the Scrum team, and the team is accountable for them. The Scrum Development team is supposedly fully cross-functional: it should contain all the skills necessary to deliver the product. If that requires specialists in governance, then include them in the team.
Let the team figure out, through conversation and collaboration with whoever it needs – external regulators included – the best way to ensure the proper delivery of business outcomes.
(1) J.O. Coplien and L.Zhao Towards a General Formal Foundation of Design: Symmetry and Broken Symmetry. Monograph