Category Archives: Agile

How fast you can change?

In my talks, I often ask a trick question – what is the most important part in a bicycle and a formula one racing car?

I get all kinds of answers – wheels, engine, chasis, tyres, steering, even the driver…. No doubt, they are all right answers.

However, my favorite right answer is the brakes! Why? Because they make us go faster! 

Let me explain.

Imagine you are riding a bicycle without brakes. You won’t be able to go faster because you have an in-built fear in your mind that if you pick up pace beyond some ‘reasonable limit’, you won’t be able to control it lest it becomes important for some reason. Now imagine riding the same bicycle with the best brakes that your money can buy (or, even better, that you can build!). You not only can ride the same bicycle with much more confidence, you can actually imagine going much faster than before. Of course, you can take the analogy much further and argue that you could go even faster if there was a better helmet, a better kneecap, jacket, leather gloves, even the insurance to make the rider free themselves up from the worries of a possible accident…then you might zoom even faster and potentially reach closer to the Peltzman Effect, but we will stop at just having the right brakes in this blog post.

Now the mental model about brakes is that they keep a stationary object at rest (at least on a downslope), or slow down or stop an object in motion, but never make anything go faster. And yet, that’s what they do – they give the courage and confidence to the rider to go faster than without those brakes. In that sense, the brakes are the feedback and the control mechanism rolled into one. Without a brake, you still have access to the same power (or the engine power in case of a formula one car) as before, but your decision to use it judiciously as opposed to using it as unbridled raw power is based on your ability to get that feedback and adapt to it. In general, the sharper the ability to get the feedback, make a decision and execute it, the faster one could go.

Of course, one can (rightly) extend the argument that it eventually depends on the rider/driver of the vehicle – for the real ability to handle a machine lies not in the machine but in the mind (and the hands) that operates it. We will get to that in a moment.

A lot of people think agile is all about faster deliveries, and then they set unreal expectations which often sound something like this – let’s do agile so we can ship the product in 3 months which today takes 9 months. Like they say, you can’t put ten pounds in a five pounds sack, I have a bit of a bad news for them – the quantum of work that takes 9 months to ship can’t be shipped in 3 months without seriously undercutting either the breadth (i.e., the scope or quantity of work) or the depth (i.e. the quality, performance, usability, reliability, etc.) of the deliverables. Agile isn’t quackery (even though it has been oversold as one by some overenthusiastic souls!). 

Speed is not fast you can go, but how fast you can change!

Speed is not fast you can go, but how fast you can change!

For me, brakes are the metaphor of agile. They provide the feedback and enhance the ability to manoeuvre, even more so at high speeds. The more real-time and actionable (i.e., more “byte-sized” than “brick-sized” to clearly understand and identify the cause-and-effect relationships that allow the feedback to be “meaningfully actionable”) a feedback is, coupled with the internal ability of the machine to rapidly absorb and assimilate it, the sooner it can respond and adapt to a potential event. Remember – making a third-stage spacecraft change its position even by a few micro-degrees is much more difficult and perhaps valuable than a car changing its lane on a highway. Just like any serious competitor who will adapt the brakes based on weather and other operating conditions, there is no one-size-fit-all brake here. The term “agile process” is a fairly useless one, for it creates a mental model of a laminated process that will enable people to operate with (guaranteed?) agility out-of-the-box, and yet, it is hardly ever going to be that! If you have already nailed down every single nook and cranny so that the resultant process is a certified “agile” one, than one must only be smoking pot. Neither the creator of that uber-agile process could have anticipated every possible future condition (the last time I checked, I was told that job description was reserved for God!) nor anyone knows enough to prescribe a single templated solution to all kinds of problems. Net-net, everyone must create their own “agile process”. Of course, they can start with what we know today, but remember – what is know today can only be a (better) starting point and not the limiting rate factor for what you must eventually accomplish. In that sense, an “agile process” is like “best practices” – you can’t simply copy someone else’s best practices and expect them to 10x your results. Rather, you must painstakingly solve the hard problems, and evolve your own best practices – to that end, best practices are things that are created as an outcome of a problem-solving activity by smart people, and not really something that others can use as a shortcut. Sure, some of these might be very universal, but I guess most all have been already discovered long back, and we are mostly rebottling and recycling them these days.

So, there you have it. There is no such thing as a universal agile process that will solve all maladies. Like Edison said we are able to see further by standing on the shoulder of giants, we simply take what exists today as a starting point and create our own agile process. Nothing more, nothing less.

And just like the picture in this blog, agility is not something that takes you faster, but it enables taking you to new, unknown and exciting places when you think there might be something interesting there. If you don’t find the cheese interesting, you can always come back to the last stable state, and start another journey, till you find something better!

Speed is not how fast you can drive and deliver. It is how fast you can change and adapt. And life and product development have more hairpin bends than you think… 

Hard work is killing people. Literally!

Recently, I was reading an article on how Japanese-style manufacturing isn’t working out quite so well in India (despite having been successfully around since 80s), and that there have been several strifes lately. There was an interesting reference to what the Japanese call as Karoshi and Karojisatsu. Now, unlike many other Japanese words, these are really gross words because they mean

  • Karoshi = Death from Overwork
  • Karojisatsu = Suicide from Overwork (and stressful working conditions)

Curious to know more about it, I chanced upon this graphic from ILO page on workplace safety:

Karoshi and Karojisatsu cases in Japan, 1997-2011, ILO This data comes from Japan, and while the numbers might not (yet!) look staggering to many a folks, the trend is unquestionably disturbing, and our own deniability might only be getting compounded by lack of data from our own respective countries or industries, not to mention the social stigma that might be associated with someone refusing to ‘work hard’ on medical grounds lest they suffer an injury, suffer a burnout or commit suicide!

In the last few years, I have heard of small but increasing number of such sudden deaths among senior IT professionals – not just in Bangalore, but also globally. You can read about them here, here, here, and here.

Burnout is a serious issue for several countries, industries and people, even if we don’t acknowledge in as many words. In our industry, where heroism, cowboy programming and all-nighters are considered cool and an integral part of the software subculture, there has been a (really) small effort to address work-life balance by identifying that software development should be ably to proceed “indefinitely” on “sustainable pace” with XP explicitly advocating 40hrs a week (though some might disagree with this interpretation of “sustainable pace“). However, anyone who has ever done any non-trivial piece of software development will point out how hollow this expectation is in reality. When it is the release time, families learn to deal with their family members coming late or staying back office overnight, or working through the night even if home. Those in startup phase don’t have the luxury of ‘closing the day’, and those who are entrepreneurs actually thrive in such environment. So, is the thrill-seeking behavior at work here? Could it even be the case that we are only unknowingly aggravating the problem by hiring more people who think, talk and work like us – thereby creating a tarpit-kind of environment where there is no escaping it?

It is well-known in manufacturing that we never utilize machines more than 80-90% of their rated capacity (global stats on utilization are more like 80%). And yet, we don’t think twice before ‘loading’ human beings to much beyond their 100% capacity! Unfortunately, the data suggests that working more reduces productivity, as below:

Working hours vs. Productivity

Or, our ‘professionalism’ stops us from admitting that if I put out my case for a better work-life balance, I might be considered a loser and be considered unfavorably in appraisals, promotions and salary raises?  

So, how do you deal with it, or rather, ensure that you don’t get burnt out? Or, if you are a people manager or a leader, how do you ensure those supporting you are not getting burnt out?

Important to call out here that the old harmless joke “Hard work never killed anyone. But, why take chances?” suddenly doesn’t look so innocuous anymore. 

Worth thinking…

Let’s free up agile teams…

Agile in general, and Scrum in particular are all about self-organizing teams collocated for maximum face to face communication that improves agility and real-time collaboration. This was a great utopian idea in ~2000s because of three primary factors back then – economics, technology and psychology. Back then, we were still trapped in the enterprise-mindset – all collaboration happening within the confines of an organizational boundary – be it research or product development because it was much more cost-effective to lead such efforts internally either due to massive costs of R&D or closed IP protection or simply being the sole magnets for top talent.

However, as Henry Cherbrough’s Open Innovation has challenged, and Lafley’s Connect & Develop program at P&G and several others have subsequently demonstrated, there is perhaps more to gain from opening up meaningless and irrelevant organizational boundaries than protect false economics of a closed innovation system.

Globalization, even with all its ugly side effects, has shown us repeatedly in industry after industry that working across a global supply chain is not a zero-sum game after all! So, why are we so parochial in software industry about not recognizing the bigger economic sense rather than limiting ourselves to a singular idea that collocated teams are the best option?

Similarly, the technology to collaborate over the wire is dramatically more potent, even if underutilized in mainstream due to various reasons, than ever before. Collaboration over internet is the most natural thing for the millennials – back in 2007, a good 97% owned a computer, 94% owned a cellphone, a good 74% used instant messaging and 94% of those multitasked while messaging. In 2013, they are spending over 25hours every week online – be it social media or online shopping, or consuming infotainment or what have you. So, it is natural that this generation is extremely comfortable – even prefers – in being connected online and getting its job done. How will you make them sit in the same office day after day, and why? Little wonder that companies such as Automattic, MySQL, 37Signals and Github are starting out as massively distributed teams – not just on day one, but are continuing with their thought process of virtual workforce that still values teamwork but prefers newer ways to stay connected with the mothership – online instead of being their physically. As they say – the technology favors the geek!

And finally, the human psychology. The more specialized the talent, the rarer it is to find than ever before, and the best talent might not always be ready to work in your office even if they are willing to work for or with you. Just like people want democracy but not necessarily politicians, they might want to work with you on specific projects or technologies, but without being part of your office politics or hierarchy. They may not aspire to a long-term career growth with you but still value working with you on that one small problem that really fires them up. However, the very thought of being on your rolls and then moving the family to your location might not make sense to them, especially if you also happen to be in the similar business of delivering services online for your own customers (and who isn’t these days?). So, will you still ‘force’ your agile teams to be collocated inside a single mothership, or take a very large pragmatic leap of faith and explore oasis wherever they are, and build distributed teams around them? To me, a process is a reflection of the social values and norms of the time when they were written. If the world has moved on since ~2000, why should we not revaluate what should ‘being agile’ mean for this world?

Let’s free up agile teams…

 

...and he might be a 'free bird' and work remotely!

…and he might be a ‘free bird’ and work remotely!

I draw upon several ideas for my hypothesis. Authors Ori and Rod give a compelling vision of autonomous teams in “The Starfish and the Spider” – while spider teams are more vulnerable, the starfish teams could be autonomous and perhaps more effective in “VUCA” times. These thoughts are getting echoed in leaderless teams in Holacracy, and are well-grounded in Complex Adaptive Systems. There is a whole new movement of “Office Optional” new-age companies where they work from home or Starbucks, and even the people who probably show up in a physical office have to dial-in for a call from a meeting room so that everyone is equally ‘remote’.

How will the currently established mental models of agile teams – specifically on business people and developers working ‘together’ daily, which is invariably assumed and expected to mean ‘face to face’ – scale up in such a scenario? If the people are changing the way they think and interact, with or without agile, and the organizations are changing the way they conduct business, can we expect our processes to hold out for us? Think about it – when our customers expect work to happen at their convenience when they need it, they are asking for a business model that is distributed in time and space – and all that is already happening around us! So, how could the teams designing and developing such solutions not put themselves in the shoes of customers and internalize such learning environment?

I think there is a strong business case of freeing up agile teams from the physical boundaries of collocation because the collaboration methods are both – efficient and effective – and create compelling business case – even if only a minority at this point. I definitely see a wave building up at the horizon…

How do you design agile feedback?

Feedback is perhaps the most important aspect of the overall agile lifecycle – without a proper, honest and timely feedback, there is no ‘adapt’ step in the inspect-adapt cycle. The absence of such feedback only ensures there is no early opportunity to ‘respond to changes’ and teams will have no option but to simply keep ‘following the plan’ thereby violating a key agile value. Starting with the TDD loop to the CI systems, we are constantly seeking feedback on our outputs – in ever shortening feedback cycles as technologically possible. However designing a proper feedback instrument for a human-human interaction, like a training program, is a totally different thing because it entails imprecise measurements that are often influenced by people’s mental models, skills and experiences, and not to mention – their calendars! Needless to say, these feedbacks could mean anything to different people on different days.

Feedback needs to be agileIf the feedback required is too ‘wide and shallow’, it can be obtained very quickly but it won’t give enough actionable feedback. On the other hand, a ‘narrow and deep’ feedback could be more actionable but might take relatively more time, and it might also fail to register feedback outside its focus area (what if you were focusing on the wrong problem?). So, how does one go about designing feedbacks that enable agile learning? We call them agile feedbacks.

In Agile India 2014, I am presenting the experience report on this topic, and will our experience from designing agile feedbacks for agile trainings and workshops. The objective was to get most critical feedback in shortest amount of time to enable quick action planning. We created feedback that took a maximum of just 5 minutes per respondent, and enabled the most important learning in both, focused as well as open-ended manner that allowed us to focus on the most critical items. We employed elements of Design Thinking and Rapid Iterative Testing and Evaluation (RITE) to improve the process and quality of feedback themselves.

The experience report is accessible here.

How do you design agile feedback?

Dude, where’s my customer?

I am delivering this talk at the Lean India Summit, Bangalore, Nov 15-16. Ahead of my talk, this Q&A captures some of the key ideas behind the talk.

Abstract of the talk: Startups that operate under a stealth mode achieve over 90% failure rates. While they might have bright ideas, access to top talent, adequate funding, etc., they typically fail to accomplish their original objectives. A key reason behind such spectacular failure rates is premature scaling at each stage of the startup. In this talk, we will examine the mistakes that startups make, and what we can learn from the Customer Development model proposed by Steve Blank to improve better chances of survival and growth.

Q1. You mention that premature scaling at each stage of the startup leads to failure. What is the right time to start scaling for the startup? How can the startup course correct if they realize that they have started scaling prematurely?

A:   As Steve Blank says, a startup exists to search for its business model and not to execute it, I think the right time to scale is when you have ‘discovered’ the ‘business model’. What that means is that unless you have gone through Customer Discovery and Customer Validation process and turned your hypothesis into facts based on actual market feedback, you don’t really have any need (and hence the justification) to scale up.

Why Startups Fail

If you have started scaling prematurely, I think you should stop and introspect. Ideally, stop all other marketing and sales and other activities of scaling up the product development organization, and just focus on validating your initial hypothesis. One important point to remember is that sometimes having excess cash could force one to scale up, so avoid getting more cash than you really need!

Q2. There are couple of questions we asked one of our other speakers Wes Williams, which we think is relevant to your topic as well. Would like to hear your thoughts on this – a) How does the size of an organization impact its scalability? Would you need to think of a different scalable model for large sized organizations?

A: Large organizations have typically been cash-rich and hence were more eager to scale-up in the past. However, the harsh realities of today’s economic environment have forced them to be more conservative. I believe even the largest, most cash-rich organizations are thinking twice before scaling up prematurely. I know how hard it is in most product companies to get an additional resource even, so clearly, the old rules don’t apply so well anymore.

I would build a much shorter runway where I can demonstrate tangible results in terms of customer discovery and customer validation that allow me to pitch my case for incremental funding that is backed by solid data from the market. That way, my expenses and promised revenues are growing in tandem rather than making a wild promise about the revenue without a scientific basis and piling up a large expense plan in anticipation of revenues.

Q2. b)What do you see as the difference between scalability, flexibility and agility?

A: Agility is the ability to achieve (or exceed) the set goals by being flexible about how to accomplish them. Flexibility is the means to accomplish agility as an end-goal. Even there, let me correct myself – agility is not an end-goal. It is once again a means to accomplish the next higher-order goal, viz., create a successful business.

Scalability is very different, though an effective scalability might require them both. In traditional manufacturing, it was all about building something at a large-scale so that economics of manufacturing or production could become much more affordable. The problem is that accomplishing scalability invariably requires lots of sunk costs and irreversible decisions and might be fraught with risks if the underlying assumptions turn out to be false. Henry Ford was able to reduce car prices by creating a process that eliminated user customization (“you can have any color of car as long as it is black”) to create a standard one-size-fit-all car that could be mass produced to achieve unprecedented economics of scale. However to achieve such economics of scale, he had to pump in millions of dollars in manufacturing plant and a car design that might hopefully sell in the market. So, in manufacturing, mass production brings down prices, and hence, scaling up is good, but one needs to make sure they have the product that the market wants, lest they be stuck with deadwood!

In software, scalability needs to be understood differently from manufacturing. Presently, there are two key manifestations of scalability – one is scaling-up product development process across the organization, e.g., going outside a single scrum team and aligning to the strategic portfolio, etc. One such framework that attempts to integrate all organizational functions together is Scaled Agility Framework. The other one is scaling-up your software product or services, e.g., offering the app on all major OS versions of mobile or tablets, or offering the software in all languages, or offering the product in all geographies, etc. The first one is all about being internally efficient and reduce friction and hand-offs across different departments in achieving a seamless supply chain to ‘scale up’ your agility throughout the organization and not just limiting to software teams alone. The second is all about scaling up your product and services from the market coverage or product capability standpoint and is conceptually more akin to traditional notions of scalability. One needs to be pragmatic about market potential before committing to any such expenses that involve sunk costs or are irreversible in nature. To that end, the principles of agility are a must to establish early proof points about the real assessment of market potential.

Q3. The Customer Development Manifesto suggested by Steve Blank mentions that “A Startup Is a Temporary Organization Designed to Search for A Repeatable and Scalable Business Model”. In your experience, how much of scaling is good for the organization? At what point does the startup need to pause and introspect on whether to scale any bigger, or to look for some other way to scale?

A: There is some very good data from Startup Genome Project that captures such data points. While I think such guidance is very comprehensive and highly relevant for all startups, it must not be construed as universally applicable. Take the data as a guidance and question before committing any fixed expense item whether you really need it at this point? To me, it is not any different from how one would commit to such expenses in their home or even as a group manager in a company.

 

Customer Development model provides a framework to systematically validate all hypothesis before going out and scaling up the startup prematurely

I meet so many entrepreneurs who build a large team – VP of Sales, VP of Marketing and even 8-10 developers before they have built a serious product to market! If you follow Steve Blank, he is very unambiguous in his guidance – during customer discovery phase, it should only be the founder of the company to validate the ‘leap of faith’ hypotheses, and no one else! I also find that sometimes getting funding early in the day could make people scale-up prematurely and that’s just as bad.

Is there an alternative to scaling up? Yes, first off, build an MVP and keep your expenses low. A key thing about the MVP is that you must aim for selling the MVP as a means to validate your Business Model – freebies don’t give the proof point that you need to turn your hypothesis into facts. Once you have a reasonable confidence backed by solid data about customer validation, then you might be better-placed scaling up.

Q4. Could you talk a bit more about the Customer Development model proposed by Steve Blank and how you are going to extrapolate from this model in your talk?

A: Traditional startups, for example during dotcom, were known for the so-called ‘stealth mode’ development where they could work for an extended period of time, often away from public glare or any meaningful form of real-world customer feedback. However, we now know such a model is fraught is too many risks. It is the Hail Mary pass of entrepreneurship! On the other hand, Customer Development model is a highly scientific approach to entrepreneurship that recognizes the need to have real-world market data before going out and building the product. In today’s time and age, technology and consumer preferences change at warp-speed, and it is no longer realistic to build a product with a long-lead time. The worst thing that can happen is to build a product that no one wants! This might appear as poetic exaggeration, but the world is literally full of such experiments. Think of McDonalds Pizza or Colgate Breakfast Entrees or Frito-Lay’s lemonade or hundreds of such products – they were all created by highly successful companies that definitely knew what it took to built new products. And yet, they all failed. Why? Because essentially, they built something that the market didn’t need. Surely, they must have done market research throughout the lifecycle, but clearly they missed something much more fundamental – product development is not simply developing a nifty idea but also entails a parallel cycle of customer development, especially in new markets. Customer Development model is one such model that helps figure out what and how to do to do customer development – whether one is a true-blue startup or a brand-new idea in an established company.

Q5. New research by Harvard Business School’s Shikhar Ghosh shows that 75% of all start-ups fail. In your experience, generally what must the remaining 25% be doing right?

A: One thing we could be safely assured they are doing well is that they are listening to early feedback and adapting to it rather than going out and building the complete product or service based on untested hypothesis. And to be able to do such adaption both effectively and efficiently, they are using short iterations where they are able to give functional products in the hands of the customers so that they can get some valuable customer feedback. Popularly known as an MVP (or a Minimum Viable Product), it allows a high degree of prioritization of features from users point of view, and an agile development process that allows creating rapid iterations of high-quality and well-test features.

Q6. If I am part of a start up like organization, operating within a giant organization with its set business models and processes (that are also required to keep the big brother organization up and operational), how does one navigate the plethora of subsidiary business groups like facilities, infrastructure, HR, etc. to enable a lean start up mode for my particular group?

A: No easy answers here! In fact, history has repeatedly shown us that the more successful an organization is, the more it is difficult to get a radically new idea going. No wonder why Sony, the undisputed leader in Trinitron technology, completely missed the Plasma and LCD display bus (before eventually joining the LED display party, even if if a bit late) or the Internet leader Google who has never quite figured out its social strategy (well, some might argue that with Google+, they do have it, but that’s one point of view). So, traditional methods of pitching an idea and expecting the upper management to bless might work in theory, but I wouldn’t rely on them.

Over year, I have found one approach that perhaps works much better compared to anything else – skunkworks. If you are truly passionate about the idea, let that passion help you ‘recruit’ other co-creators (always remember – it takes a village to raise a child) and then put your spare cycles to build a prototype to demonstrate your idea. If you are not willing to take risks and spend days and weeks in even trying the idea, why should you expect that the management will give you budget for it? It’s like going to bank asking for money, whether to buy a house or to build a company – they want you to put down some money first in the form of your contribution or a guarantee, and the logic is same – if you are not willing to take any amount of risk behind your idea, why should others do it?

Java language came out of skunkworks at Sun. And so did the first Apple Macintosh. I have seen several good ideas come out like this in large companies, though I don’t think anyone can guarantee skunkwork projects. However, on a relative basis, it is much better demonstration of the power of your idea (compared to a powerpoint deck) and your passion that might improve your chances of getting the necessary organizational support to take your idea to its next leg of journey.

Q7. Finally when is it the right time for an organization to realize that it has grown too big to be in a startup mode anymore, and might need to revisit its way of working to adapt to its current size?

A: This might be the next million-dollar challenge and I wish I knew the cookbook answer! Unfortunately, we build the bureaucracy one day at at time and before we realize, we have built an impregnable wall of self-preservation that blunts any attack on status-quo. We don’t realize it because it is like slow-boiling the frog. Initially it looks innocuous and we tend to trivialize it, until one day it becomes so huge that we can’t get rid of it even when we try so hard!

I think an organization has stopped being a startup when people stop complaining and simply give up on you, when no one volunteers and there is generally a diffusion of responsibility, when people build territories and silos, when we are intolerant of failures and anything perceived disruptive, when people want to join because of the perks of the company rather than the challenges they can work on and the impact they can create, when more time is spent in waiting for decisions rather than learning from actions following implementing those decisions, and so on…

Q8. Who is the target audience for your talk? Only startup organizations?

A: Anyone who is undertaking a new endeavor under conditions of extreme uncertainty.

Why is your agile still a lot like dogma on steroids?

I still continue to be amazed (thought ‘shocked’ and ‘dumbfounded’ will be more appropriate words here) by the amount of dogma in agile circles. Do this! Don’t do this! Wasn’t agile meant to liberate us from the tyrannies of the so-called big monolithic non-agile white elephant processes, and create a more nimble mindset, flexible culture and adaptive process framework where ‘inspect and adapt’ was valued more than ‘dogma and prescription’? I sometimes think the poor old waterfall, for whatever it was worth (and I do believe it was worth a lot more than most people are willing to give it credit for!), was more open-ended and invited innovation simply because it was not perfect enough, and was very clumsily built for a rainy day, and depending on how sunny your day was, you were allowed (rather encouraged and expected) to pack only that much ration that you felt might be needed for the jungle trip.

For example –

  • You could have had any team size.
  • You could choose to locate your team members anywhere they wanted to be.
  • You could tailor your process in whatever way that made sense for you.
  • You could choose to slice your functionality in any which ways that made sense.
  • You could stagger the releases in whichever way you felt offered better yield.
  • and so on.

In several ways, such innovation required one to master the nuances of software development. And for those who would apportion reasons behind failure of such project on the method, my response would be – when there is so much flexibility in the system, why blame the system for being so ‘rigid’?

I have experimented more with waterfall methods in my career than with agile methods (which also I continue to do, much to purists chagrin :)), and here’s a small list of key exeriments that I remember doing – something that gave me immense joys because I had the liberty to try out stuff to see if that solved my problems better –

  • In 1995, when we realized that we are in a technologically evolving and complex domain (Asynchronous Transfer Mode switches), we didn’t build castles in air with the ‘non-negotiable’ waterfall-based product development process that the company had mandated, but decided to build an early prototype what would allow us to validate some key assumptions about our architecture. Yes, the company’s process didn’t support us, but yes, we broke the rules :).
  • In 1997, when we ‘discovered’ that standard waterfall won’t help us speed up the development cycle while we wait for the previous phase to complete, we didn’t blame the process for it, we simply ‘invented’ sashimi model and kept going.
  • In 1998, when conventional estimations didn’t work out in a domain that was completely new and unknown to us (digital set top boxes), I was not obligated to follow some obsolete standard process (though we were a CMM Level 5 company), but encouraged to try out estimations using complexity weights using methods like PROBE to mitigate the risks in estimations.
  • In the same project in 1998, when the project’s technology was new to us, I was able to home-brew and define a process with five increments that recognized the experimental nature of the problem we were solving and the learning curve of the team rather than sticking to a one-size single-release process.
  • During 2000 to 2003, I liberally experimented with waterfall methods to build teams that delivered large products in telecom and datacom domains with high success rates. At one time, I had 190+ engineers on a single product in my team organized around 14 parallel projects running on a common timeline and delivered on-time product in complex 3G Softswitch space. Yes, all in waterfall :). At that time, we were ranked last in global market. Today that company is global leader in that space, and I can proudly say some of our efforts were behind that turnaround.
  • In 2004-05, I experimented with our conventional enterprise service pack release model by liberally adding the weekly cadence from Gilb’s Evo process to create a weekly delivery model, and by accidentally stumbling on the concept of limiting work in progress to create one of the world’s first kanban implementations without knowing kanban – to be fair, it didn’t exist at that time – all without any prescriptions but just with a liberal dose of enthusiasm and undying spirit of experimentation.
  • …and the journey continues.

And what has been my take on the agile theory and practice? Not so open to experimentation or innovation. Sad, but true. Take some simple things for a start:

  • Agile methods recommend a small team size. That’s good common sense, and backed by scientific studies and acendotal data from ages, and is a generally good advise. What’s no so good is then we insist that agile teams can only be in a certain numerical range, and any team size more than that is blasphemy! In fact, my extreme view is that the best team size is what you have right now and not an ideal something from the literature, howsomuch backed by data that might be! In several ways, it is same as the ideal body weight – most of us will never have it, but what we have ‘here and now’ is the most important number to start with. So, why waste time over building an ideal team and lose all precious learning opportunities in that process? If my team has a true ‘inspect and adapt’ DNA, irrespective of where I start, I will get to the finish line. Somehow. Isn’t’ that more important and being truly agile rather than finding the perfect take-off point?
  • Take user stories. The notion of moving to user stories makes lot of sense give the constant pace of world around us. PRDs could never cope up with documenting such copious amount of details – and if anything, they would only succeed in documenting history of what customer wanted a year back! Now on one end we want our user stories to be ‘negotiable’ (from the acronym INVEST) so that we can create meaningful conversations between product owner and the development team. This again makes a lot of sense in an imperfect world where documenting every single requirement with its myriad corner conditions might be practically impossible, and has diminishing rewards beyond a point. So, if we can create a quick and cheap way to get started and have both, the process and the humility to listen to development team come up with more questions and options, then this premise holds very high promise. However, as a philosophy, something that is non-negotiable might not be so good in the same spectrum. For example, the Scrum process that we want them to follow must be non-negotiable. Why is that? If Fosbury had listened to the best way of high jumping, he might have never broken the proverbial sound barrier in high jumping.

Hey…what happened to the big promise of team being allowed to figure out its own ways and means? Once we ‘tell’ them, shouldn’t we step aside and let the team find its true north? Do I hear you mention ‘Shu-Ha-Ri’ thingy? Do yourself a favor – go and find a student (even better – try it on a second-grader) and then keep telling him/her that they are still a ‘Shu’ and hence must obediently listen to whatever you are telling them. They are supposed to follow your instructions to the hilt and not even think of wavering a bit. Good. Now take a deep look at their reaction. Count yourself lucky if they choose to ignore you and decide to move on, for there are far more violent ways they could have chosen to respond to such dogmas. In short – this is not the time and age for dogmas. Kingdoms, Colonies and Communism are all long dead. Accept it and change your own coaching methods, if you want to be counted.

To me, agile is a state of mind that tells me how to proceed in an imperfect world. Not to somehow make a ‘perfect’ world and then proceed. To me, a successful agile implementation is not about finding the perfect team + perfect process + perfect customer + perfect timeboxing + perfect sprint planning + perfect retrospectives + perfect product owner + perfect scrummaster + perfect

When does experience get ossified into dogma?

When does experience get ossified into dogma?

engineering practices + many more perfections = perfect landing. To me, a successful agile means starting with team that you have at hand, with the process that you have under the constraints you have, with the requirements that you have on a best effort basis and a many such real-world realities that works under a mindset of taking things one after other and improving the journey with the hope to get to the destination better than without such effort. Remember, we are being melioristic, not idealistic. We are being adaptive, not laying down pre-conditions for take-off. And in that pursuit, the most important guide for decision-making is our own judgment. Everything else is just that – everything else, and while it might work at times, it might not work at other times. So, like the Swiss Army manual that says – when there is a gap between map and the terrain, trust the terrain, go ahead boldly and experiment. In the worst case, you will lose some time and dollars, but if your DNA is built on the premise of self-improvement, you will quickly recover and eventually find your own path. If you are not able to ‘find’ it, you will ‘build’ it. Even better…

In many ways, there were no royal guards so zealously guarding waterfall model that made it sexy enough to be experimented with and experimented upon. On that same scale of flexibility, I don’t find agile methods sexy enough. It appears to be a lot like dogma on steroids. And I think that’s a serious problem.

Is your agile still a lot like dogma on steroids?

Why user stories make sense?

User story is a popular mechanism used by most agile methodologies to communicate user requirements. Actually, this is only partially true. User stories are meant to be a placeholder to initiate an early conversation between the product owner and the team on key hypotheses so that a quick validation of them could be done to refine/confirm/reject our understanding of what the customer really wants. To that end, user stories are not anything like the requirements of yesterday – they are not even remotely meant to be complete/comprehensive and so on. 

The reason user stories (and not ‘well-formed’ requirements, if that is ever possible) makes sense is because we human beings are not experts in following the written word, and are likely to misinterpret requirements even when they are explicitly written down, especially when the communication is a one-time event and not an ongoing process. (if you don’t believe this, just visit this interesting site http://www.cakewrecks.com/ that celebrates real-life examples of how something as simple as icing on the cake can go horribly wrong despite really simple and absolutely clear instructions!). And we are talking about building complex mission-critical systems that must operate 24/7 with top speed and efficiency. If only the written word could be consistently understood by the receiver as intended by its author… 

On the other hand, if there is a continuous two-way dialog between the product owner and the agile team, such purposeful brevity leads to curiosity which then leads to a constantly improving and better shared understanding that refines as the product gets developed incrementally and gets periodic user feedback. 

User stories reflect the reality that we might not have 100% clarity about the requirements on day one, but with constant dialog, we might be able to improve our shared understanding of customer's wants and needs.

User stories reflect the reality that we might not have 100% clarity about the requirements on day one, but with constant dialog, we might be able to improve our shared understanding of customer’s wants and needs.

The key benefit of user stories is that instead of waiting for weeks (even months!) to get a fully-bloated and rather unprioritized PRD which is supposed to have fully-baked requirements while the development team sits idle all this time, is that identifying highest priority user stories and using them  to develop a prioritized product backlog allows the teams to get started much earlier and start developing features which could then be put out in front of customers to get real feedback (as opposed to individual ‘opinions’ that might/not best guide us in an uncertain environment). This is especially important because in the past, when the world was little less complex and much more underserved. I purposefully use this term ‘underserved’ because it is not that people have suddenly become much more demanding about what they like or not, but were just told there were exactly three carmakers to choose from, or just two operating systems to choose from, and so on. However, with rapid advancement of computing paradigms and constantly lowering cost of ubiquitous communication devices, they suddenly have the opportunity to demand what they want, and hence classical ‘forecast’ models of requirements elicitation, design or production (at least in the manufacturing sense) don’t work so effectively as much as the newer ‘feedback’ driven models that allow for developing key hypotheses (and not hard requirements carved in stone) which could then be quickly and cheaply delivered as ‘done’ features so that customers could tell us if they like them or not. Based on such valuable customer feedback, the team could iterate to either refine the feature, or to pivot, as the case might be. In the past, this might take the entire product gestation cycle, by which time, many, if not most, things might have undergone significant changes, and the entire development costs being sunk by that time, not to mention complete loss of any window of opportunity.

As Facebook says – Done is better than perfect. User stories allows developing a small slice of the product without really perfecting the entire product, but facilitates the process of validated learning that eventually helps develop a product with much higher chances of meeting customer needs.

In today’s world, you probably might not have the luxury of having investors wait multiple years for an exit, or customer waiting several quarters for a perfect product when the competition is willing to serve them faster (even letting them lay their hands on an earlier version and give feedback thus making customers a co-creator of the product), or employees working month after month without getting any meaningful feedback from customers. And we know that absence of any feedback eventually leads to erosion of trust. Incremental product development could help you bridge such trust deficit by delivering highest value to customers early and periodically, and user stories might help you get your project a quickstart.

The Joys of Designing Agile Solutions for New-Age Problems

Pune is hosting India’s very first Scrum Gathering this year in July with the theme of “Scrum – Kal, Aaj aur Kal“, meaning the past, present and future of Scrum. I appreciate all efforts by Madhur Khaturia and his team in making it happen. I hope and believe it will be yet another milestone in India’s journey towards agility, and will stimulate a whole lot of new ideas and conversations thus ultimately leading to increased awareness, higher adoption and more breakthrough products from India.

Scrum Gathering India Regional 2013

Scrum Gathering India Regional 2013

I got confirmation mail yesterday that my workshop on “The Joys of Designing Agile Solutions for New-Age Problems” has been accepted for Scrum Star Trek, which is the track for discussing The future of Agile Philosophy and Scrum Framework. This would focus on new techniques, methods , innovations and concepts being used by practitioners. This would have workshops, concept presentations and round-tables on future of Scrum. The other two tracks are Scrum From the Trenches and Scrum Accomplished.

I am really looking forward to this workshop. Here are a few details about it. 

Agile methods, and scrum in particular, help organize the work by prioritizing items could be done and delivered in a small timebox. While its simplicity and effectiveness is based on timeless experiences that make it an effective approach for a certain class of problems where a product backlog exists in some shape or form, and there is some basis (scientific, data-driven or otherwise) for the product owner to prioritize features, it doesn’t really offer any clear guidance for a newer class of problems where, for example, the work is so radically new that there might not be a product backlog. For that matter, there might not even be a market, a customer or a product to begin with. Think of you specifying and designing ‘something’ when no such product ever existed – not just in the minds of customer but even in the dreams of its designers! How can one have a ‘product owner’ or a product backlog when there is even no source of such information? All we have is a loose set of ‘ideas’ which might or not work. Distilling such innovative ideas in a product or a sprint backlog might be too premature. We clearly need a different approach than a thoroughbred Scrum. 

In the last few years, the work on Customer Development and Lean Startup by Steve Blank and Eric Ries has led to putting a framework in place for solving such class of “VUCA = Volatile, Uncertain, Complex and Ambiguous” problems in a more systematic and result-oriented manner. These have greatly helped startups and entrepreneurs where failure rates tend to be extremely high traditionally. In this half-day workshop, we will experience to joys of creating a product starting with Design Thinking and apply principles of Customer Development and Lean Startup and explore the nuances of designing agile solutions for such new-age problems.

Do you have experiences to share in this class of new-age problems? How do you design effective solutions that allow you to thrive on ambiguity, deal with unknowns and keep moving at lightning speeds despite maintaining a holistic sense of agility? Share back your ideas and experiences…

So, does agile really kill innovation?

In continuation of my earlier blog post on ‘Does Agile Kill Innovation?’, I had a great time moderating the panel discussion at Agile India 2013 with Henrik Kniberg, Owen Rogers, Sujatha Balakrishnan, Udayan Banerjee, Praful Pillay and Sudipta Lahiri. The panel discussion was literally the last program at the end of two long days of management conference – but despite that, we had 60-70 folks throughout the session.

Agile methods, with their rather short-term focus on achieving a ‘done’ feature seem to give away an impression that the eventual goal of a sprint is delivering such a full-baked feature. However, the new-age thought process seems to be more like ‘done is better than perfect‘. Question is, does agile implicitly become the rate-limiting step for your innovation process by self-imposing a stringent and non-negotiable completion criteria that possibly can’t be met by innovation-led ‘stories’ where the discovery and experimentation is generally a long-haul process and failure is such an acceptable outcome that we want to fail early and fail often!

We had some interesting perspectives ranging from ‘agile kills innovation’ to ‘agile accelerates innovation’ to ‘it depends’. Some of the ideas that emerged out of the panel discussion were:

  • agile, and scrum in particular, seems to be well-suited to the class of problems where a reasonably well-defined product backlog exists to begin with. To that end, there is some level of agreement on ‘what’ is to be done in the stacey’s matrix. For such problems, maybe ‘20% innovation’ user stories could be designed that allow implementing features with lesser uncertainty on ‘what’ and ‘how’ and leaving the uncertain aspects to the innovation user stories. This could ensure that the sprints are not losing steam just because of taking up innovation-led tasks, and neither are those tasks suffering in the anxiety of maintaining (and improving) team’s velocity.
  • for problems that are so radically new that don’t have any notion of a product backlog to begin with, perhaps ideas from lean startup (and design thinking) make more sense where the notion of ‘velocity’ is not so pronounced as compared to the ability quickly iterate through the build-measure-learn loop in shortest amount of time, and test various hypothesis around customer discovery or MVP, or any other moving part
Does your process let you think outside the box?

Does your process let you think outside the box?

  • one attendee from the session remarked that in their organization, they give a ‘break’ in-between the sprints to take up such innovation-led tasks, and it is working for them.
  • in organizations and teams that get too obsessed with meeting predictability goals and focus on velocity, there is a risk that teams don’t attempt big bets but just chipping away at the small bets. So, that version of agile is probably not going to lead to any radical or breakthrough or disruptive innovation, but more likely suitable for incremental innovation.
  • in services companies, the essence of business value is predictability of service, and hence there is some level of disconnect with the whole notion of experimenting and failing that might impede any innovation. To that end, achieving agility could be hurting innovation. However, there are always possibilities to innovate in such customer-facing processes, and one needs to deliver proof-points and earn the charter.

These perspectives made for some good conversation, questions and counterpoints. After all, what is a good panel discussion without some arguments and controversies!

So, does your agile really kill innovation? You tell…

Role of Integrative Thinking in Project Management

I just finished reading the brilliant “The Opposable Mind” by Roger Martin. He introduces the notion that we all possess the so-called  “opposable mind” which has this amazing capability to simultaneously hold two contradictory views about a problem.

The conventional wisdom is to try to find a via media but that is perhaps meekly surrendering to complexity by taking a short-cut to a suboptimal solution. He argues that the some of the most exceptional leaders do not succumb to the obvious “either/or” thinking but rather work patiently towards synthesizing the best from both of these opposing views to create a best-of-breed solution that is far superior to either of these. He calls it “integrative thinking”.

Martin writes, and I quote:

“…the leaders I have studies share at least one trait, aside from their talent for innovation and long-term business strategy. They have the predisposition and the capacity to hold two diametrically opposing ideas in their heads. And then, without panicking or simply settling for one alternative or the other, they’re able to produce a synthesis that is superior to either opposing idea. Integrative thinking is my term for this process – or more precisely this discipline of consideration and synthesis – that is the hallmark of exceptional business and the people who run them.…Human beings, it’s well known, are distinguishes from nearly every other creature by a physical feature known as the opposable thumb. Thanks to the tension we can create by opposing the thumb and fingers, we can do marvelous things that no other creature can do – write, thread a needle, carve a diamond, paint a picture, guide a catheter up through an artery to unblock it. All those actions would be impossible without the crucial tension between the thumb and fingers.…Similarly, we were born with an opposable mind that we can use to hold two conflicting ideas in constructive tension.”

The book is a treasure trove of how go develop integrative thinking, but I was struck by how much it is relevant to each one of us at all levels, not just for the leaders at the top. The field of project management is all about making sensible but often hard trade-offs. Effective management of Project Management Triangle is literally the holy grail of project management – one that not only is the nemesis of every project manager but is also, in fact, the genesis of modern project management. In layman terms, what we mean is that out of Scope, Cost and Time, you may pick up any two, but can never meet all the three of them at the same time. So, if the requirement is to build replica of Eiffel Tower in just three months and ten thousand dollars, then yes, it can be built but it might look more like a school project. If it is a fixed-scope project like migrating all tax payers to a new software by close of the current financial year, then one might need to look at costs involved without which either one of the scope of the time might not be possible to meet.

However, what ends up happening in reality in many situations is that a project still ends up taking the amount of money that is takes, and yet the scope delivered is too little, too late!

If one were to look at modern project management (though most of us in software industry would probably call it as ‘traditional project management’), I think we just don’t have any similar notion of integrative thinking. The only treatment I have seen is perhaps in the Theory of Constraints where the approach is more holistic.

In software development, the conventional project management is modeled around industrial model of a production process – the so-called waterfall. In a linear, sequential flow done by silos that specialize in a single functional area, there is natural tendency and inclination to optimize the area of responsibility. For example, a design team might be most concerned with how elegant their design is, whether is meets the future issues of maintainability or portability or not. Similarly, the high wall between developers and testers is industry-famous – which is a major reason why developers don’t trust testers (“they will come up with only Sev 3 bugs and kill us with volumes towards the release, and expect us to fix them all before GA”) and in turn, testers don’t trust developers (“they can’t write clean code”, “if only they did better code reviews and unit testing and found all code-level defects, we could do such a better job of finding all Sev 1 bugs and focus on performance and reliability aspects”). Clearly, such a process breeds a mutual trust deficit, and the narrowly focused project roles simply perpetuate it further. What happens is the tragic story of optimization of parts at the cost of sub-optimizing the whole – completely anti-lean approach!

Agile software development (the concept, not a particular methodology) helps by offering to eliminate such artificial boundaries that force “either/or” thinking. Instead of taking a monolithic approach of meeting 100% of the project triangle, it offers to simultaneously meet part of the scope at part of the cost in fraction of time that what would normally be needed in a waterfall model. However being recently introduced to this concept of integrative thiking, I am still thinking if agile philosophy fully addresses it, and on the surface, it doesn’t look like. However, I have been proven wrong in the past, so we will see…

Seeking submissions on New Product Development and Product Management in Agile world for #AgileIndia2012

Agile Alliance has announced Agile India 2012 in Bangalore, India in Feb 2012. First of all, Naresh deserves kudos for all his untiring efforts last several years bringing up and building a robust multi-city agile software community in India, and deserves all congratulations bringing the flagship Agile conference to India. As the Chair for Agile India 2012, we couldn’t have had anyone better than Naresh to lead this.

AgileIndia2012 comes to Bangalore, India in Feb 2012
AgileIndia2012 comes to Bangalore, India in Feb 2012

I had proposed the stage on Agile for New Product Development (NPD). Annu Augustine (@annua) proposed a stage on Product Management in Agile world. The organizing team proposed merging these two and we all agreed that was a great idea to create a stage on “New Product Development and Product Management in Agile world”. I believe the time is ripe to elevate agile thought process outside the software development team and apply it to the big picture – the end to end product development. We often see significant ‘productivity’ claims from software teams adopting agile practices. However, there are several other key functions involved in product development – product management, user design, marketing, sales, technical documentation, beta programs, manufacturing (at least for systems companies), and so on. If software team optimizes its software process, it is good. However, it is necessary but not sufficient to achieve end-to-end agility. If rest of the organization doesn’t change its thought process, we just end up moving the ‘constraint’ from software development to something else, and the true gains of agility might not get realized at the organization level.

In this stage, we are now seeking perspectives, experiences, insights and groundbreaking ideas from practitioners and thinkers on how they have applied the spirit of agility to create new products that have led to unprecedented and extraordinary market success compared to their previous conventional practices. Specifically, we are looking for proofpoints from the marketplace to demonstrate how software team’s agility was directly visible in business results. We want to know what you learnt from your experiences applying agility at every step of the new product development to create customer delight experiences. We want to know how did you manage to crash your time to market to deliver innovative solutions that delighted your customers, or made significant impact to their topline.

We are still working on setting up the infrastructure for submitting the ideas, but in true agile letter and spirit, we want to evolve this stage as we go. Write back to Annu or me to share your views and ideas so that we can co-evolve this stage. We want to dogfood the spirit of this session by incorporating your feedback to make this stage more useful to everyone. You can find more details on Agile India 2012 here.

Interest to submit a proposal? Click here.

Let’s get going…

Project Management vs. Program Management

Program Management is often seen as the next logical step for seasoned project managers looking to take on bigger challenges. While project management is more about managing within boundaries of a project and gatekeeping it against anything and everything that threatens the status quo, program management is typically all about breaking those very boundaries and managing across them by taking up anything and everything that threatens the status quo. In this post, I will examine how they differ in its approach on two important aspects – scope management and people management.

Scope Management

Like I mentioned above, a project’s success depends on its ability to retain focus against all odds. Once a project scope is defined, its estimates made, resources  allocated and commitments made, the project manager is pretty much focused on gatekeeping everything else out of the scope lest the project success is threatened. In reality, most projects would have a CCB, or a Change Control Board, of some levels of formal authority, there is still a tendency to bulk up the entire requirements in first-pass and do as much as possible in one breath irrespective of how much time it takes and whether the final outcome is acceptable to the customer or not. While most of the traditional world still likes this model for various reasons, software development community identified this as a bottleneck and created the suite of so-called ‘agile methodologies’ that exploit software’s ability to incorporate late changes to specs without seriously endangering the project or its deliverables. Still, at a high level, a project must work around a reasonably stable set of requirements to ring-fence itself against any potential changes to the ‘core’ of a project – the premise being how can you build a successful product on a shaky and wobbly foundation. After all, don’t we pay product managers to do a better job of defining those requirements upfront rather than changing their mind later in the project and calling it as customer change request to cover up what they failed to think of in the first place! Surely, everyone understands changes in workflow or bells and whistles, but I am talking about the core architecture – the fundamental DNA of a product that must be understand before any further allocation of time, money or resources is made. So, clearly, scope is sacrosanct to a project.

A program is a different beast. As the highest level of body chartered to translate an organizational’s strategic intent to reality, it can’t box itself inside any boundaries of defined or undefined scope. Anything that could impact a program’s ability to accrue full ‘benefits’ envisaged from it must be taken up. While in theory, a program must have a defined scope to plan its activities and resources around its deliverables, in practice it is not so trivial. Any reasonably large program has sufficient number of moving parts, uncertainties, conflicting requirements and rapidly changing priorities. It is very typical in a program for the component project managers to carve out their pieces very sharply. A lesser reported fact of life is that developer’s motivation to work in newer and sexier technology often dictates the choice behind a project taking up (or refusing to take up) a given problem. Program organizational structure and governance plays a very important role in ensureing that component projects are not only cleanly defined, they also identify inter-group dependencies and secure commitments to address them. To that end, a project might safeguard itself by rejecting an inter-group request, but eventually the program needs to address it! Similarly, a project manager might complete the work within her boundaries but it takes much more for the program to be ‘done’, let alone be successful. I have seen many situations where project managers would be so focused on their project that they won’t recognize that unless the program was successful as a single entity, their individual progress was meaningless. This could get aggravated when teams are geographically or organizationally dispersed. However, none of those can hide or discount the fact that a program is only as  successful as it ability to influence things that might not be in its line of control but whose impact is definitely in a project’s line of success, especially if those things were to backfire.

While a project is like a fortress that must protect itself against all invasions to survive and eventually be successful, a program is more like a university, a rose garden, or a mission – they all deal in ‘soft power’ and maximize the ROI of their mission by keeping their doors open and by teaming up with their potential adversaries.

People Management

A project manager in a functional organization wields significant ‘positional power’ and thus has very high ability to influence team members behavior. While today’s organizations are highly flat and democratic, there is still an asymmetric balance of power, for the manager who writes your focal and annual salary revisions also potentially has the power to decide when you should update your resume! Surely we have come a long way in management-worker power-sharing from Taylor and Ford era, make no mistake that there is no such thing as a perfectly symmetric world where a manager and her team member have equal rights. Thus, a project manager has much higher ability to define the work and assign people to it as she feels appropriate. She also has a much higher level of responsibility towards training and career development of her team members, and being the closest face of management to the team, she is the official spokesperson of the upper management. A team will likely listen more to her than to the CEO. She must balance two opposing sets of expectations that, if not aligned properly, can set the project on fire. To that end, people management for a project management must be one of the toughest job.

In high contrast, a program manager manages at boundaries of participating organization in a highly matrix environment, and hence must manage by influence and not by any formal authority. In a software team, a program manager needs to get Development, QA, Product Management, Usability, Documentation, Marketing, and several other functions on the same page. In most organizations, they are organized along the functional lines and hence report to a solid-line manager in the same skill-set pool rather than program manager. Given that many of these resource might be timesharing on a program, their eventual loyalties are still with their respective line managers and hence a program manager can only rely on collaboration and influence as the key measures to get everyone onboarded. In some cases, there might even be a conflict between a component’s goals and the program’s goals and if such issues remain unresolved, the only recourse to make the program successful might be to move the problem up the command chain. Still, there is a big value in managing large and complex endeavors as a program, for it allows an organization to manage its resources and inter-group dependencies and conflicts in a more systemic and transparent manner. Some of the best program managers I have seen were not the ones who stopped managing the interfaces, but went over and above what their jobs required. They established a direct contact with key team members in component projects and created an alternate informal channel to validate project risks and plans, and to feel the pulse of the organization. They would do it very unintrusively without creating any friction between the line manager and them.

How does your organization view these two important functions?

 

What are agile practitioners thinking in 2011?

Shortly before the yearend holidays, couple of us from product development companies got together to discuss how software development process philosophy and methodology is changing. We had representation from global embedded companies, healthcare products, internet domain, automotive, semiconductor – you name it. What was interesting was that irrespective of the type of software product being created, some common themes emerged in terms of practices that seem to make sense:

  1. Continuous integration: integrate early, integrate often seemed to be high on priority list for most.
  2. Test Automation was considered equally important agile practice
  3. Collocating cross-functional teams was found critical for end-to-end product development
  4. Being nimble to requirements was critical not just for customer-facing products but even in enterprise world
  5. Some (many ?) engineers are not used to reporting daily – they think it is an unnecessary intrusion into their work, and perhaps even a productivity impediment! Making them think otherwise is an important culture change
  6. Techies don’t want to become scrummaster – they would rather work knee-deep in coding! Some companies tried to hire ‘scrummasters’ – career project managers without specific domain skills but that was not very successful. The challenge is to find right set of people to play the role of scrummaster.
  7. Daily stand up seems to be a great way to make sure teams stay on the same page. I recounted my own experiences – way back in ’98 at Philips, we used to have daily stand-ups and it was highly effective.
  8. Simulating interface components was a high need for companies in hardware-software co-creation cycle. This especially became critical as the hardware was generally never available when doing software
  9. Can’t work from home in a truly agile world – have to co-locate teams. This seems to be a rather side-effect of using agile for teams – since teams have work closely and for things like daily stand-ups, everyone should (preferably) be in the same room. However, given the realities of modern day life, working from home is not only inevitable few days a month, but might actually be a productivity booster (ask those of us from Bangalore!). So, having a rigid agile discipline seems to be at crossroads to people’s ability to balance their work-life.
  10. Blank sprint after 3-4 sprints is a generally used practice. People felt back-to-back sprints would fatigue the teams, and make the work monotonous, and hence a blank sprint. A blank sprint was simply another timebox with housekeeping activities, vacations, etc.
  11. There seems to be an growing chorus for having internal agile coaches – however, no one in the group was using them. There is a general disdain for external coaches who might only give bookish prescriptions – after all, you need someone within the system to own the action items and not someone who simply makes powerpoint of the status. There was a great discussion that such agile coaches can’t simple be a single-axis professionals whether process guru, people manager, or techie. Rather, they need to be a bit of everything and then some more – project manager + process guru + people coach + techie + communication expert + …
  12. What is the role of manager in an agile world? This question has never been addressed well by agile community. How do people grow in their careers in an agile world. In the traditional system, whether good or bad, there is a hierarchy to aspire or grow into, but how do you acquire different skills that prepare you for taking on higher-level responsibilities in the career? If scrummaster is the closest role for a manager, then what next? Scrummaster or scrummasters? There was no clear direction or best practices that have stood the test of time.
  13. Pretty much no one believed that bookish or a biblical approach to Agile is the right thing – everyone seems to tailor agile as per the unique combination of business needs, nature of products and business and culture, etc.

This was definitely an interesting session that gave an opportunity take stock of some of the things that are working or not working. We also discussed the fact that Agile is really addressing a subset of the entire business problem – to address the entire problem, we need to embrace systems thinking and lean thinking.

What are you thinking in 2011?

Is Planning an old idea whose time is up?

In the age of nano attention-spans of people, the tendency and respect for planning things upfront has taken a serious beating. The mainstream logic is to simply “play by the ear” because there are far too many moving parts to be completely accounted for and properly factored-in – and in any case, by the time the plan goes to execution, ground realities would have changed beyond recognition, thereby rendering the plan completely useless by that time. In software development, an inaccurate predictive long-range model such as Waterfall has been replaced by more accurate adaptive short-range Agile methods that solve the line-of-sight problem but don’t address the original problem – that of planning a large project with its own share of uncertainties.

While none of those arguments might be wrong par se, we conveniently ignore the fact that that is the nature of the beast, and we need to understand that planning is not a single-pass 100% process that stays current forever. Further, just because things will eventually change is not a good enough reason to abandon any systematic efforts to understand it, if not to tame it altogether!

Pillar on a railway trackWhile preparing my lecture notes for a recent class on planning, I decided to explore the subject of planning through the ages using various quotations over a period of time. I sorted them in chronological order in order to understand how human mind has evolved the thought process behind planning. What came out from this exercise surprised me – it seems like the values associated with planning are as timeless as the fundamental human nature. The age-old wisdom was that a “stitch in time saves nine” and hence there was a focus that “prevention is better than cure”. Some might argue their world was not so complex and dynamic. I disagree.

There world was highly uncertain, and hence inherently complex. With perhaps 95% of reasons behind mortality unexplainable, with perhaps 95% of scientific events being attributed to the Gods (mostly of the angry variety!) and no protection from animals, predators and even their own tribal enemies, how much more complexity do you need in a day? When faced such levels of uncertainty, the modern man is more likely to live life one day at a time. And yet, the thought process from yesteryears seems to indicate that they didn’t simply abandon long-range planning, on the contrary, they reinforced the values of such long-range planning as the only real way to deal with such uncertainties!

Here are some of the quotations that I discovered during my research:

  • If you are planning for a year, sow rice; if you are planning for a decade, plant trees; if you are planning for a lifetime, educate people – Chinese Proverb
  • A man who does not plan long ahead will find trouble at his door – Confucius (551-479 BC)
  •  Plan for what is difficult while it is easy, do what is great while it is small. The difficult things in this world must be done while they are easy, the greatest things in the world must be done while they are still small. For this reason sages never do what is great, and this is why they achieve greatness – Sun Tzu, Chinese General, The Art of War, 400 BC
  •  Before beginning, plan carefully – Marcus Tulius Cicero, Roman statesman and orator (106-43 BC)
  • Long-range planning works best in the short term – Euripides, poet (480-406 AD)
  • Forewarned, forearmed; to be prepared is half the victory – Miguel de Cervantes Saavedra, Spanish writer and author of Don Quixote (1547-1616). 
  • By failing to prepare, you are preparing to fail – Benjamin Franklin
  • In preparing for battle I have always found that plans are useless, but planning is indispensable – Dwight D. Eisenhower, general and president (1890 – 1969)
  • A plan which succeeds is bold, one which fails is reckless – General Karl von Clauswitz
  • Always plan ahead. It wasn’t raining when Noah built the ark – Richard Cushing, novelist
  • Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought – Bernard Baruch, A  stock broker, advisor to presidents Woodrow Wilson, Harry S. Truman, (1870-1965)
  • Those who plan do better than those who do not plan even thou they rarely stick to their plan – Winston Churchill, British Prime Minister
  • In the space of two days I had evolved two plans, wholly distinct, both of which were equally feasible. The point I am trying to bring out is that one does not plan and then try to make circumstances fit those plans. One tries to make plans fit the circumstances – General George Patton (1947).
  • A good plan today is better that a perfect plan tomorrow – George S. Patton
  • Planning is a process of choosing among those many options. If we do not choose to plan, then we choose to have others plan for us – Richard I. Winwood
  • Plans are only good intentions unless they immediately degenerate into hard work – Peter Drucker
  • Prediction is difficult, especially about the future – Yogi Berra, baseball catcher (1925-present)

Another interesting discovery was that planning didn’t mean people simply fell in love with their original plans. On the contrary, there are instances that people advocated not only adapting to changes, but rather incorporating mechanism to accommodate them upfront. Look at some of the quotations on this:

 

  • It’s a bad plan that admits of no modification – Publilius Syrus, Roman slave and poet (circa 100 BC)
  • Observe always that everything is the result of change, and get used to thinking that there is nothing Nature loves so well as to change existing forms and make new ones of them – Marcus Aurelius, emperor of Rome (121-180 AD)
  • It is not the strongest of the species that survive, not the most intelligent, but the one most responsive to change – Charles Darwin, scientist

So, is the concept of ‘planning’ an overrated and old idea who time is up? I don’t think so. It only means that there is more need to refresh our understanding and explore how to evolve long-range planning methods to incorporate short-term changes such that agility is not lost at the tactical level, while near-sight changes don’t create a bullwhip effect on strategic plans.

Do you think inaccurate and uncertain proactive planning should be completely abandoned in favor of a highly accurate and definite reactive ‘planning’ (if that can still be called as ‘planning’, that is)?

How do you schedule tasks in a project?

How do you decide what tasks to schedule first: the complex ones or the easy ones? the short ones or the long ones? the risky ones or the sure-shot ones? Most often, this task sequence is determined by hard logic, soft logic, or some other external constraints. However, how do you decide when there are no such contraints?

If we look at the risk driving the project lifecycle and scheduling, then it is natural to expect high-risk tasks being tackled at the start just so that we are systematically driving down risks in the project and achieve higher certainty levels as we get close to the project. However, it seems inconceivable that someone will cherry-pick the easy tasks first and leave all high-risk ones for the end! Clearly, that is setting up the project for a grand finale of all sorts!

Project SchedulingCould complexity be a good measure then? What will happen if we take high complex tasks first? Surely, that will lead to tackling some of the most difficult problems first, and there might also be a high level of correlation between complexity and risk. So, taking this approach is also likely to significantly lower a project risks. However, not all tasks are created equal. It is likely that high-risk tasks not the longest, so while the project makes appreciable gains in lowering the risk, but doesn’t make a whole lot of progress.

Agile approaches, Scrum in particular, rely on driving the tasks that make biggest sense to the customer – tasks that deliver maximum value to the customer. However, there is no guarantee that this will help lower the project risks, or manage the schedule better. Similarly, Kanban helps manage tasks but doesn’t really spell out how they should be scheduled.

So, what is the best approach?

I read an interesting blog, The Art of the Self-Imposed Deadline, where I specially liked this one from the blog post:

3. Avoid the curse of the “final push.” Scope and sequence a project so that each part is shorter than the one that precedes it. Feeling the work units shrink as you go gives you a tangible sense of progress and speeds you toward the end. When you leave the long parts for last, you’re more likely to get worn out before you finish. Besides, if you’re “dead at the deadline,” those other projects you’re juggling will stagnate.

This seems like a very practical suggestion – often the tendancy is to ‘cherry pick’ while scheduling the tasks – we tend to pick up tasks that are favorite to people, or appear to be easier to take up. However, very often, we end up taking more time, and project gets delayed. If the tasks are scheduled such that big/complex tasks are scheduled first, that might mitigate lot of risk pertaining to last-minute issues (could be due to underestimation, or integration issues, etc.). However, I am not 100% sure that purely scheduling tasks on “longest-task first” basis is the best policy to lower risks in a project. Shouldn’t we be using a combination of top-risk items that are longest?

I haven’t used this approach yet, but seems like something to try. How about you – what’s your favorite way to schedule tasks?