I still continue to be amazed (thought ‘shocked’ and ‘dumbfounded’ will be more appropriate words here) by the amount of dogma in agile circles. Do this! Don’t do this! Wasn’t agile meant to liberate us from the tyrannies of the so-called big monolithic non-agile white elephant processes, and create a more nimble mindset, flexible culture and adaptive process framework where ‘inspect and adapt’ was valued more than ‘dogma and prescription’? I sometimes think the poor old waterfall, for whatever it was worth (and I do believe it was worth a lot more than most people are willing to give it credit for!), was more open-ended and invited innovation simply because it was not perfect enough, and was very clumsily built for a rainy day, and depending on how sunny your day was, you were allowed (rather encouraged and expected) to pack only that much ration that you felt might be needed for the jungle trip.
For example –
- You could have had any team size.
- You could choose to locate your team members anywhere they wanted to be.
- You could tailor your process in whatever way that made sense for you.
- You could choose to slice your functionality in any which ways that made sense.
- You could stagger the releases in whichever way you felt offered better yield.
- and so on.
In several ways, such innovation required one to master the nuances of software development. And for those who would apportion reasons behind failure of such project on the method, my response would be – when there is so much flexibility in the system, why blame the system for being so ‘rigid’?
I have experimented more with waterfall methods in my career than with agile methods (which also I continue to do, much to purists chagrin :)), and here’s a small list of key exeriments that I remember doing – something that gave me immense joys because I had the liberty to try out stuff to see if that solved my problems better –
- In 1995, when we realized that we are in a technologically evolving and complex domain (Asynchronous Transfer Mode switches), we didn’t build castles in air with the ‘non-negotiable’ waterfall-based product development process that the company had mandated, but decided to build an early prototype what would allow us to validate some key assumptions about our architecture. Yes, the company’s process didn’t support us, but yes, we broke the rules :).
- In 1997, when we ‘discovered’ that standard waterfall won’t help us speed up the development cycle while we wait for the previous phase to complete, we didn’t blame the process for it, we simply ‘invented’ sashimi model and kept going.
- In 1998, when conventional estimations didn’t work out in a domain that was completely new and unknown to us (digital set top boxes), I was not obligated to follow some obsolete standard process (though we were a CMM Level 5 company), but encouraged to try out estimations using complexity weights using methods like PROBE to mitigate the risks in estimations.
- In the same project in 1998, when the project’s technology was new to us, I was able to home-brew and define a process with five increments that recognized the experimental nature of the problem we were solving and the learning curve of the team rather than sticking to a one-size single-release process.
- During 2000 to 2003, I liberally experimented with waterfall methods to build teams that delivered large products in telecom and datacom domains with high success rates. At one time, I had 190+ engineers on a single product in my team organized around 14 parallel projects running on a common timeline and delivered on-time product in complex 3G Softswitch space. Yes, all in waterfall :). At that time, we were ranked last in global market. Today that company is global leader in that space, and I can proudly say some of our efforts were behind that turnaround.
- In 2004-05, I experimented with our conventional enterprise service pack release model by liberally adding the weekly cadence from Gilb’s Evo process to create a weekly delivery model, and by accidentally stumbling on the concept of limiting work in progress to create one of the world’s first kanban implementations without knowing kanban – to be fair, it didn’t exist at that time – all without any prescriptions but just with a liberal dose of enthusiasm and undying spirit of experimentation.
- …and the journey continues.
And what has been my take on the agile theory and practice? Not so open to experimentation or innovation. Sad, but true. Take some simple things for a start:
- Agile methods recommend a small team size. That’s good common sense, and backed by scientific studies and acendotal data from ages, and is a generally good advise. What’s no so good is then we insist that agile teams can only be in a certain numerical range, and any team size more than that is blasphemy! In fact, my extreme view is that the best team size is what you have right now and not an ideal something from the literature, howsomuch backed by data that might be! In several ways, it is same as the ideal body weight – most of us will never have it, but what we have ‘here and now’ is the most important number to start with. So, why waste time over building an ideal team and lose all precious learning opportunities in that process? If my team has a true ‘inspect and adapt’ DNA, irrespective of where I start, I will get to the finish line. Somehow. Isn’t’ that more important and being truly agile rather than finding the perfect take-off point?
- Take user stories. The notion of moving to user stories makes lot of sense give the constant pace of world around us. PRDs could never cope up with documenting such copious amount of details – and if anything, they would only succeed in documenting history of what customer wanted a year back! Now on one end we want our user stories to be ‘negotiable’ (from the acronym INVEST) so that we can create meaningful conversations between product owner and the development team. This again makes a lot of sense in an imperfect world where documenting every single requirement with its myriad corner conditions might be practically impossible, and has diminishing rewards beyond a point. So, if we can create a quick and cheap way to get started and have both, the process and the humility to listen to development team come up with more questions and options, then this premise holds very high promise. However, as a philosophy, something that is non-negotiable might not be so good in the same spectrum. For example, the Scrum process that we want them to follow must be non-negotiable. Why is that? If Fosbury had listened to the best way of high jumping, he might have never broken the proverbial sound barrier in high jumping.
Hey…what happened to the big promise of team being allowed to figure out its own ways and means? Once we ‘tell’ them, shouldn’t we step aside and let the team find its true north? Do I hear you mention ‘Shu-Ha-Ri’ thingy? Do yourself a favor – go and find a student (even better – try it on a second-grader) and then keep telling him/her that they are still a ‘Shu’ and hence must obediently listen to whatever you are telling them. They are supposed to follow your instructions to the hilt and not even think of wavering a bit. Good. Now take a deep look at their reaction. Count yourself lucky if they choose to ignore you and decide to move on, for there are far more violent ways they could have chosen to respond to such dogmas. In short – this is not the time and age for dogmas. Kingdoms, Colonies and Communism are all long dead. Accept it and change your own coaching methods, if you want to be counted.
To me, agile is a state of mind that tells me how to proceed in an imperfect world. Not to somehow make a ‘perfect’ world and then proceed. To me, a successful agile implementation is not about finding the perfect team + perfect process + perfect customer + perfect timeboxing + perfect sprint planning + perfect retrospectives + perfect product owner + perfect scrummaster + perfect
engineering practices + many more perfections = perfect landing. To me, a successful agile means starting with team that you have at hand, with the process that you have under the constraints you have, with the requirements that you have on a best effort basis and a many such real-world realities that works under a mindset of taking things one after other and improving the journey with the hope to get to the destination better than without such effort. Remember, we are being melioristic, not idealistic. We are being adaptive, not laying down pre-conditions for take-off. And in that pursuit, the most important guide for decision-making is our own judgment. Everything else is just that – everything else, and while it might work at times, it might not work at other times. So, like the Swiss Army manual that says – when there is a gap between map and the terrain, trust the terrain, go ahead boldly and experiment. In the worst case, you will lose some time and dollars, but if your DNA is built on the premise of self-improvement, you will quickly recover and eventually find your own path. If you are not able to ‘find’ it, you will ‘build’ it. Even better…
In many ways, there were no royal guards so zealously guarding waterfall model that made it sexy enough to be experimented with and experimented upon. On that same scale of flexibility, I don’t find agile methods sexy enough. It appears to be a lot like dogma on steroids. And I think that’s a serious problem.
Is your agile still a lot like dogma on steroids?