The BBC’s Dr. Who is one of the most popular TV science fiction series world-wide. I recently took advantage of a trip to a client’s site in Cardiff Bay to visit the Dr. Who Experience there, and found myself drawing analogies between aspects of the series and Agile development. I’ve been a fan since I watched the very first Dr.Who episode in November 1963. What hooked me in that first broadcast was the moment when William Hartnell, the actor who played the first Doctor (the incumbent, Peter Capaldi is the twelfth) revealed to his first companions, Ian Chesterton (played by William Russell) and Barabara Wright (Jacqueline Hill), that the Tardis is bigger on the inside than it is on the outside. Tardis stands for Time And Relative Dimensions in Space and is The Doctor’s time-travelling spaceship. Bigger on the inside than it appears on the outside? Scrum is like that. Let me explain.
Bigger on the Inside
The Scrum framework is a very simple set of rules, easy to memorize and easy to understand. Elsewhere I’ve referred to it as the 3-4-5 rule (1): three roles (Scrum Master, Product Owner and Development Team); four artifacts (Product Backlog, Definition of Done, Sprint Backlog and the increment of Potentially Shippable Product) and five Events (the Sprint, the Sprint Planning Meeting, the Daily Scrum, the Sprint Review and the Sprint Retrospective). There are rules governing the interrelationship of these essential elements, of course, and these are spelled out in The Scrum Guide (2).
Scrum is a minimalist framework for self-managing teams that develop complex products. The current version of The Scrum Guide contains even less than the earlier versions. That’s because its whole purpose is to give maximum room for Scrum teams to make their own decisions about how to develop and deliver their product or service. In many ways, this is what makes the framework difficult to apply. Other than the mandatory events built into its inspect-and-adapt cycle there are no predetermined steps to follow. It is up to the Scrum team to figure out what to do. In effect, it makes up its own process as it goes along. The self-management that frees the team’s creativity and enables it to become potentially hyperproductive in its delivery of business value, is also what makes Scrum hard. There is a whole universe of potential decisions a team could make that are hiding behind its simple edifice: which ones to choose?
Back to the Future
The Doctor and his companions have often used the Tardis to travel back in time to fix things in the future. In doing so they set us an example. As past attendees of Learning Tree’s top-selling course 918 Agile Project Management with Scrum will know, patterns were part of the genesis of Scrum. I authored that course seven years ago and maintain it. Its study guide has always included an appendix concerning organisational patterns and Scrum. Studying the past of a phenomenon can give us great insights into its present and future trajectory, as I’m sure the good Doctor would tell us.
A paper called “SCRUM: An extension pattern language for hyperproductive software development” (3) was workshopped at the PLoP ’98 conference (4), presenting the Scrum framework as a collection of organisational patterns. This was not the first public presentation of Scrum. Its debut was at the 1995 OOPSLA convention (5) . The PLoP paper openly drew on prior work by Jim Coplien (6). “Developer Controls Process” is one of Cope’s orgpatterns, and that concept is, of course, fundamental to the Scrum framework. Others that were explicitly drawn upon were “Engage Customer”, “EngageQA” and “NamedStableBases”. The paper’s authors, Mike Beedle, Martine Devos, Yonat Sharon, Ken Schwaber and Jeff Sutherland (Schwaber and Sutherland are also the authors of The Scrum Guide, by the way) added their own patterns such as “ScrumMaster”, “Backlog” and “Sprint” to create the extension pattern language. Figure 1 depicts the relationship between Coplien’s patterns and the new ones.
Further Back in Time
The connection between organisational patterns and Scrum goes back even further. Most software developers have used, or at least heard of, design patterns (7). But the first volume of Pattern Languages of Program Design included a chapter by Coplien (8) , and it was to this that the authors of the Scrum pattern language had referred.
However, if the Tardis could take us back a year before Sutherland and Schwaber revealed Scrum to the world, we would find an article in Dr. Dobb’s Journal in which Cope described a technique, adapted from CRC cards, for visualizing and analysing software development processes (9). Together with Brendan Cain, he had applied this technique while undertaking research for Bell Laboratories into highly productive software development teams. The project was called Pasteur. “We have found many…recurring patterns in these visualizations, many of which bear out common management folklore, others which are counterintuitive, and still others which deserve further study….By understanding these patterns, we hope to develop principles from which new, highly productive organizations can be built.” (10)
The exemplar case study in the Pasteur research was Borland’s Quattro Pro for Windows (QPW) project. It assimilated requirements, completed design, implemented more than a million lines of code during which two prototypes were built and discarded as well as a final product created, and was fully tested in less than 31 months. The coding was done by four to a maximum of eight people. “The project capitalized on its small size by centering development activities around daily meetings where architecture, design and interface issues were discussed.”(11) Coplien wrote that the achievements of the Borland QPW team were “off the charts” compared to most of the other processes that the Pasteur team had studied .
Cope’s famous Dr. Dobbs article recently had its 20 year anniversary, prompting a blog post by the leader of the QPW team, Bob Warfield (12). Entitled “How I Helped Start the Agile/Scrum Movement 20 Years Ago” the post recounts the Pasteur research, and reveals that Jeff Sutherland got many of the ideas for Scrum from an early draft of Coplien’s article that he found circulating on the web. In other words, when Beadle et al., cloaked Scrum in the clothes of organizational patterns, they were in fact going back to its very roots.
Fast Forward to the Present…and the Future
We don’t have space here to describe the concept of patterns in detail. Suffice it to say that they form part of an historical knowledge base that developers can draw upon and adapt to fix their own issues, and that they are discovered not invented. The original Pasteur research patterns informed and became part of the Scrum framework itself, but patterns can also help Scrum teams make their choices once the framework has been established.
Returning to the present, The Doctor can point out that there is a rather larger resource to draw upon than we might initially imagine. Bob Warfield’s blog post describes fourteen “fundamental tenets” each one of which contains numerous patterns. For example, tenet number eight, “Interactive Middle-Out Programming”, describes the need to build a small and skinny version of the product before detailing it out, and keeping something working every build. Precursors, he quite rightly points out, to both Continuous Release and Minimal Viable Product.
Another resource I always recommend to teams I am working with (and their management) is the book, Organizational Patterns of Software Development (13) by Coplien and his long-time colleague, Neil Harrison. The book contains four complete pattern languages: a project management pattern language, a piecemeal growth pattern language, an organizational style pattern language and a people and code pattern language. Between them these languages combine 104 different patterns, and there is a bunch of other stuff in the book that Scrum teams will find useful too.
And there’s more good news. Because for some years now there has been a small community uncovering patterns specific to Scrum teams. It is called ScrumPLoP. Both Jim Coplien and Jeff Sutherland are members of this community, as is Neil Harrison, together with a highly talented and committed group of like-minded people. They held a number of workshops in Denmark and Sweden which have published 26 patterns for Scrum. There are many more patterns in the pipeline. These patterns will both define Scrum, and help practitioners to understand its meaning by drawing, as all patterns do, on the experience of real people working on real products in real organizations. If only Dr. Who and his Tardis were real, perhaps he could tell us how many will be in the book these guys are putting together, and how many books containing Scrum patterns will be published. In the absence of The Doctor, my best advice is to begin your exploration of Scrum’s “inner space” by heading over to the ScrumPLoP website. “Allons-y”, as the tenth Doctor used to say.
(1) See my videoscribe. http://www.emerald-hill.co.uk/4-minute-scrum-overview/4%20minute%20scrum_player.html
(2) Jeff Sutherland and Ken Schwaber. The Scrum Guide. July 2013. http://www.scrumguides.org
(3) Mike Beedle, Martine Devos, Yonat Sharon, Ken Schwaber and Jeff Sutherland. SCRUM: An extension pattern language for hyperproductive software development. http://www.jeffsutherland.org/scrum/scrum_plop.pdf
(4) PLoP is an acronym for Pattern Languages of Programming. The PLoP conferences are supported by the patterns movement. http://hillside.net/conferences
(5) Ken Schwaber. 1995. “Scrum Development Process” In OOPSLA Business Object Design and Implementation Workshop. J. Sutherland (editor)Object-Oriented Programming, Systems and Applications (OOPSLA) convention. Austin, Texas, USA.
(6) James O. Coplien. 1995. “A Generative Development-Process Pattern Language” in Pattern Languages of Program Design, J.Coplien and D.Schmidt (editors). Addison-Wesley
(7) E.g., Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides 1995. Design Patterns. Addison Wesley
(8) James O. Coplien.1995. op. cit.
(9) James O. Coplien 1994. “Examining the Software Development Process”. Dr. Dobb’s Journal. October 1, 1994. http://www.drdobbs.com/examining-the-software-development-proce/184409329
(10) James. O. Coplien 1994 op. cit.
(11) James. O. Coplien 1994 op. cit.
(12) Bob Warfield. 2014. “How I Helped Start the Agile/Scrum Movement 20 Years Ago”. SmoothSpan Blog. https://smoothspan.wordpress.com/2014/10/02/how-i-helped-start-the-agilescrum-movement-20-years-ago/
(13) James O. Coplien and Neil B. Harrison. 2004. Organizational Patterns of Software Development. Prentice Hall