Category Archives: Agile

Your estimates or mine ?

For decades now, the project management world is divided between top-down estimation and bottom-up estimation. While a top-down approach might have limitations, it perhaps is the only way to get some meaningful estimates at the start of a project. A bottom-up approach might be a great way to get more accurate and reliable estimates but you might have to wait for a problem to be whittled down to that small a level to get such reliable estimates. Both are required for a comprehensive and useful project planning, but unfortunately, most people see one over other, and the Agilists abandoning top-down in favor of the highly accurate but low-lookahead bottom-up methods.

A top-down estimation is a great way to abstract the problem without worrying about its nitty-gritties, and come up with a workable estimates when there is no other source of getting any better estimates. At the start of a project, when making a realistic WBS is a remote possibility, the only way one could get some estimates is by top-down estimation techniques such as

  • Expert Judgment
  • Wideband Delphi Method
  • Analogous Estimation
  • Parametric Estimation

Estimates represent a range of possibilities

 

They allow an ‘order of magnitude’ estimate to be made available to set the ball rolling. We know that such estimates suffer from cone of uncertainty, but it is better to get imperfect estimate now than to get perfect estimates much later in the project – which we still need to get – the point is that we also need to get some estimates here and now!

Another key reason why we must use such top-down techniques at the start is because if we straightaway jumped into bottom-up estimation, we might face challenges because

  • Details about requirements are still emerging, and hence likely to get refined as our understanding about the problem gets better
  • Requirements might keep changing, thereby changing the scope of work all the time
  • Too many low-level details could introduce too many moving parts in the scope of work, thereby forcing one to skip “vital few” and start focusing on “trivial many” clearly not a prudent approach at that stage

So, we need to accept the ground conditions and accept those high-level estimates with a boulder of salt and keep moving. As we get deeper into the project, we decompose the problem better and have a more granular WBS that helps us zoom into the problem and get more refined estimates. At that point, a bottom-up approach is much more relevant, as it allows us to plan and track the work much better.

Agile methodologies clearly favor bottom-up estimation methods and the idea is to slice down every task on the project into ‘stories‘ that could be ideally done by an individual team members within a couple of days. It helps you to win daily battles but takes your sight off the big war that you must win to eventually succed. We can’t really scale up these stories for large projects even if we assume that every product scope could actually be broken down to meaningful stories of that grain size. So yes, in theory, that’s great. But the real world is not obligated to obey the theory!

A prudent approach is to adopt rolling-wave planning – start with big grain size when details are not very clear and gradually move down to small grain size as you get closer to the task. However, you don’t just abandon the overall planning simply because you are now doing task-level estimates and planning! On the contrary, it is even more important to keep track of dozens of small-task estimates becuase, to quote the great Fred Brooks in The Mythical Man-Month, it is not the tornadoes but the termites that eat up your project’s schedule! So, a delay of one day here and one day there repeated for a couple of tasks over next few sprints and you suddently are sitting with a month-long delay on your overall project. You might be hitting each timebox but that’s one problem with timeboxes – it could give you a false sense of comfort that all is well, when clearly you are being forced to jetsam things that you are not able to accomplish within a timebox. On the other hand, if you go by a task-driven schedule, you atleast have a clear idea of how much late you are to complete that task.

A frequent criticism of top-down estimates is that they are at best a wishful thinking, and at worst a highly unrealistic estimate forced upon a team against their free will. Again, we should be careful in such judgment because the reality is not always what is presented to us, but rather what we do with it! So, if you take a top-down estimate as the word of god and make it ultra-resistant to changes, than you have yourself to blame when those estimates don’t reflect the reality anymore. They should be taken as a map and should be continuously refined from the data coming from the terrain. To that end, top-down estimates and bottom-up estimates play a complmentary role and an effective project manager blends them together. A good top-down estimate covers the ground fairly well, thereby allowing realistic bottom-up estimates to happen (by allocating them sufficient, meaning as close to the actuals, time and resource available), and good bottom-up estimates at task level help reinforce assumptions made in the top-down estimates of the overall project.

So, at the end of the day, it is not about which estimation technique is superior. It is about using most appropriate tool for every type of problem.

Next time, think again before you ask the question your estimates or mine?

32 Reasons Why People Don’t Plan Projects

The concept of planning has a very intuitive appeal, irrespective of the type and size of endeavor. Who wouldn’t want to undertake an activity without a little bit of upfront planning? Will you take your car and just head out for the next family weekend in the neighboring town – without first checking if your car is roadworthy for the long drive, has enough gas, or needs a visit to the garage before the long drive? Will you buy a house without first checking your cash flows and other commitments for the next couple of years? Will you get your kids admitted to the nearest school you come across, or you will do little bit of investigation and plan is based on your specific needs and child’s comfort?

Still then, it comes as a great surprise when many among us disregard the benefits of planning things upfront. While they might still succeed in a trivial pursuit, for it might present very predictable outcome, require relatively risk-free approach, is generally easy to recover from a bad position, have virtually all resources at one’s disposal, etc. While such trivial ‘projects’ might not demand a rigorous planning, we should also note that often such planning is a very intuitive process, one that happens very effortlessly and unconsciously in our minds. Such planning, or several of its elements, have been executed so many times that one doesn’t have to think of anything else – it just happens. To an untrained eye, it might appear like a careless approach where nothing is being planned, and yet the actors seem to be in supreme control – it is nothing short of magic! However, let the vision not get the better of logic.

However, for any non-trivial problem, or a trivial problem with a few key changes, or some last-minute unpleasant surprises, the life is not that easy. They transform into wild beasts that don’t obey their masters anymore, they behave randomly (or at least unpredictably), and unless you address them proactively, they are only more likely to make things increasingly difficult for you. What follows then is the endless loop of firefighting (often leading to ‘immortal projects’ that I conceptualised in my earlier blog post “From Project Immortality to Project Moksha“). So, why is it that some people refuse to plan their projects, choosing instead to clean up their mess over long evenings and frustrating weekends over the next several months? Here are some of the reasons that I have come across:  

  1. Our project is too large to plan
  2. Our project is too small to plan
  3. Planning is waste of time and effort – I’d rather be coding right now!
  4. Planning commits me to something that I must stick to at all times, even when the world around me has changed.
  5. Publishing the plan will expose buffers in the plan to our customers
  6. Publishing the plan will expose buffers in the plan to project team members
  7. If we publish our plan, customer will question our productivity being low
  8. The project is evolving and the situation is changing on a daily basis
  9. We are too deep fire-fighting the project – have no time to plan at this point
  10. Why plan things that are too simple, and as for things that are too complex – well, you anyway can’t manage them, so why plan?
  11. We believe in adaptive processes rather than a predictive planning
  12. Planning involves tracking which means supervision of a person’s time. We think that creates mistrust in the team.
  13. We do burn-down chart which is better for projecting the delivery date
  14. The sub-contractor is responsible for project plan
  15. This is just a simple porting work
  16. Next 2-3 months we will be doing real research. There is no way I can manage that effort to a plan!
  17. Plans are for the weak-hearted, the brave hearts face the tornado head-on
  18. We typically estimate and pad it by 30% to make it a realistic plan
  19. We can’t estimate number of bugs we will get in the testing phase, and hence can’t plan for those phases
  20. Someone or other is always leaving our team or the company. No use building plans around them.
  21. All real work happens in nights and over weekends. Plans have always failed us.
  22. Our customer only cares for working software
  23. Requirements are always changing
  24. Customer has given us fixed end-date, so our plan doesn’t count
  25. My management will always underprovision team resources
  26. Doing estimation won’t speed up the project – it will take the same time even without an estimation
  27. We do Agile
  28. We do Scrum
  29. Plans are anyway changing all the time
  30. I have the project plan – here it is in the gantt chart
  31. Our project is diferent – planning will stifle the innovation and creativity
  32. Any version 1.0 product will be like that only!

You might have seen variations of these, or brand-new excuses. Feel free to share them here. We need to deglamorize these lame excuses for they serve no one’s interests. At best, they handcuff us into existing methodology, and at worst, they ensure project immortality (discussed above). Both are fatal for your project’s success.

What’s your excuse not to plan your project ?

Our methodology is 100% pure, our result is another thing!

What is worse then an anarchy ? You might say that is the absolute abyss, but I think blind allegiance is even more dangerous (and that includes following the letter but tweaking the spirit – things like ‘creative accounting‘ or its parallels in every field). Anarchy at least allows for things to become ‘better’ in order to survive – whether it is the idealogy, resistance, or even musclepower, or any other ills (and hopefully at some point, social forces of constructive destruction take over). But in a land where unquestionable compliance and blind allegiance rule the roost, IMNSHO, is like a terminal patient off the ventilator support. When people are on their deathbed, they don’t regret things that they did but much rather the things they did not do!

In project management, life is no less colorful. We have process pundits (read “prescription police”) shouting from the rooftop with a megaphone on how heavens will strike them bone dead with lighting if they ever as much as strayed from the ‘standards’. When projects are being postmortemed, we don’t often ask what or why the project did something that they did, but why they did not do things that they did not do. And quite often, you find answer in the map itself – because the map did not factor-in those conditions that were actually encountered on the terrain, the blind followers just followed the Pied Piper and danced their way into the river of death. What a terrible waste of human talent.

Why do we get stuck with methods so much that our result-orientation takes a back seat ? I think there might be many answers, but some that deserve merit:

We fall in love with ‘our’ methods

Let’s face it – introducing a new methodology in an organization, or a team, is not easy. People have their favorites, and it already is a fairly serious political game to get such groups to a decision (whether or not there is a consensus). After you have sold the executive sponsor to go for a particular methodology, you have to now somehow sell to middle-managers and the teams – without their support, you are toast. Sometimes you might have legitimate power (“position power”) to thrust the decision down their unwilling throats, but most often that doesn’t work anymore (even in jails!). So, you have to use your charm offensive to somehow make the other stakeholders believe that this methodology is the god’s greatest gift to mankind, and how it will take their next project to unprecendented glories. If only that were half-true! What follows next is a qunitessentially Bollywood storyline – the project is in tatters and the project manager and teams are exhausted and some even ready to bail-out. You go and put some more pressure on them, ask for weekly reports to be given on daily basis and take away even more productive hours into endless meetings to discuss yesterday’s weather. In short, we do everything but realize that our methods might be wrong…perhaps…for a change? Most people don’t get it, but sometimes, the smarter ones do get it. But by then, they are so much neck-deep in love with their methods that they can’t afford to take most obvious and the only correct decision – take a U turn, and admit their mistakes. Our professional ego checkmates us. Falling in a bad relation is bad, staying in it even after realizing one’s folly is worse.

We believe blindly in marketing claims

These days, there are dozens of marketing gurus, each extolling the virtues of their methods. In a way, it is like entering a village market – each vendor selling the same portfolio of vegetables, but each has to ‘differentiate’ from his neighboring vendor, and hence some make wild performance claims, quite often unbacked by any reliable data, and almost always trusted by gullible buyers at face value. So, some of us just go by the tall claims in glossy marketing brochures. You don’t believe me – tell me, how else you can explain those knowledgeable investors getting duped by Madoff’s elaborate Ponzi scheme in this age and day! Tell me how most banks fell for easy money in the subprime market. And if we were to assume, for a moment, that everything these marketing brochures said was indeed true, how come rest of the world had not quite figured it out already ? I think intelligence is not the elite preserve of a learned few – the rest of us lesser mortals also deserve to be treated with some professional respect that we can figure out what works and what doesn’t. In a previous blog, I once did an interesting analysis of one such marketing claim Blame your flaccid developers and your flaccid customers for your poor quality products !.

We are are too scared to experiment and take sensible risks

This is a real classic. Many organizations, certainly many more than we think there are, are so stuck in ‘command and control’ that they create a system where compliance is rewarded and creativity is shunned. This might work well in some industries or companies, but certainly doesn’t work for most of us. A beauty about such companies is that such outright disdain to new ideas might not just start and end at the top – depending on how deep-rooted the indoctrination is, it might be running in every employee at all levels. In the zest to display unflinching loyalty and to project oneself as a corporate citizen second to none, many employees (and many managers) simply abort new ideas because that might threaten the ‘status quo’ and makes them look very bad in front of the powers that be.

Because everyone else is doing it

This is groupthink at its best. Just because every Tom, Dick and Harry about town is doing it, I must also adopt (simply continue following mindlessly) the latest management fad. For example, we have all heard of (often unverified) stories how investors in the bay area during the dotcom boom era would only look at business plans that had an India component. Just because everyone is doing offshoring might not be good enough why I should also do it. Similarly, just because everyone I know is into Agile, does that make a strong case for my business also ? I think we should stop and think before succumbing to trade journals that routinely publish such forecasts and doomsday prophecies.

We are looking for easy cookbook solutions

Let’s accept it – some people are just plain lazy, or just too risk averse to do any meaningful research on what works for them. Instead of investing in figuring out what’s best for them, they are looking for some quick wins, some jumpstart, some fast food, some room service, some instant karma. They believe they can learn from other’s mistakes (which is definitely a great way to learn – definitely a lot better than not learning from anyone else’s mistakes!) and sometimes they might be successful, if they have copied the right recipe, but very often, that only results in wrong meal to wrong people at wrong time. What people don’t realize is that every problem is cast in a different mold, and whatever one says, there simply is no way one could airlift a solution and drop on the face of a second problem – in its totality – and expect the solution to work. Similarly, there is no way a cookboo solution might work. For example, Agile was designed around small collocated teams that have so high communication among itself that trust replaces formal contracts and communication replaces documentation – I mean they thrive on such a high-energy environment. But, sadly, simply to prove that Agile can fix any problem in the world, management consultants have stretched (should I say ‘diluted’) those very Agile princinples and they now try to forcefit those methods on a distributed, large team, that often has a heterogeneous ‘salad bowl’ culture than a homogeneous ‘melting pot’. So, you land in the situation that just because it worked for some random cases, some among us just naively believe it could work for our random case too. Does it get any more smarter than this 🙂

We believe compliance leads to repeatable success

Standards are often treated like insurance against catastrophes – both the termites and the tornadoes types (think of tornadoes as something that just comes out of the left field and kills a project overnight – like a meteor hits a neighborhood, and think of termites as slow and steady erosion that goes on and on and goes undetected until the time the damage is complete, comprehensive and beyong damage control). They ensure that no one deviates from the beaten track and tries something adventerous (without getting rapped on the knuckles), because that could lead to unpredictable results. And we all look for standards and certifications – ISO-this and ISO-that, CMM and CMMI (and the sunsequent obsession with all other 5-level models), Scrum, PMP, PRINCE2, MBA, PhD, IEEE, …so much so that even in the field of creative arts, there are schools that specify what creativity is allowed and what is out! There are success formulas for Bollywood movies – rich girl meets poor boy and the anti-social forces strike boy’s family. Eventually, our hero saves the day. Similarly, in Hollywood movies, it has to be the hero saving the nation from external threat (very often, coming from the space). In software development, ISO championed the cause of non-negotiable compliance and blind CMM practitioners only perfected it. Agilists were born out of the promise to create a world free of compliance, but it seems they have also ended up growing their tribes with their own mini-rules that give an instant gratification by useless things like Scrum Test to massage one’s ego that my Scrum is more Agile than your Scrum! Do, if you score a certain score in the Scrum Test, you are almost ‘guaranteed’ to get some x level of productivity improvement! Does that remind you of yesteryears, or does it make the future look more scarier?

Conclusion

Human behavior never ceases to amaze. For every one rational thinker, the world has to produce thousands of blind followers. Our schools and colleges teach us to learn the knowledge but they don’t always teach us how to convert it into wisdom. So, when we reach workplace, we are deeply apprehensive of trying out new stuff. We are excellent followers, but simply shudder at the mere thought of questioning status quo. We often behave like the monkey whose hand is stuck in the cookie-jar but refuses to release the cookies even when it knows that the only way he can extricate his hand out of the jar is without cookies. When those workplaces ignore result-orientation and only worship the compliance, the story only gets murkier. Think of a state where compliance is handsomely rewarded and questioning it seen as full and frontal attack, and its timid citizens are only too happy to oblige. They think a lifetime of blind obedience to methodology is far more superior than a moment of experimentation, even if leads to bad results.

After all…our methodology is 100% pure, our result is another thing!

(Inspired by a slogan on a tee: Our vodka is 90% pure, our reputation is another thing. very inspiring, indeed :))

Does your project management methodology lets you free think?

Its not about the project management methodology anymore. Frankly, it never was, even though it has triggered off some of the most senseless wars in the history of project management. Starting with Frederick Winslow Taylor‘s Scientific Management to Henry Ford‘s assembly-line based mass production system and eventually landing in a flavor-of-the-day methodology (CMM, ISO, Agile, XP, Scrum, Lean, Kanban…and add your favorite one here), project management community, especially in software field, has seen it all…and still counting! All these project management methodologies have been eulogized as silver bullets in their heydays (and some still continue to be worshipped as we speak), and have subsequently been improved upon by the next wave of innovation driven by ever-evolving business needs, state of technology and the sociological changes at the workplace. However, each predecessor has been uncharitably rejected and unceremoniously relegated to trash by every successive methodology champs. However, that doesn’t seem to have stopped project woes, certainly not – going by the claims made in their marketing brochures :). So, whom are we to trust – the overzealous champs or their ever-evolving methodologies ?

For most practitioners, novices and experienced folks alike, project management methodology became this one large target to shoot at, the advertisement to get the project deal, the crutches to hold the project on to, the lame excuse against change in project specs, the insurance against failures, perhaps the raison d’tre for project managers ? “Sorry, the manual says do it this way, we can’t change that”.The process handbook says we can’t take any changes anymore – tell customers to wait until the next release which is just six months away”. “You are not approved to prototype, so stop that effort”. “Our company’s org structure doesn’t allow an engineer to manage the project – the risks are too high”. “Our metrics are within the control limits, so I don’t understand why engineers fear a project delay”. Goes without saying, they come in all hues.

Little did we realize that the “problem” was a moving target. We continued to evolve our solution blissfully unaware that the problem was also upgrading itself. Every new fix has led to a newer generation of problem that seems to have outpaced the development of solutions so far, and I see no good reason why we will ever have a perfect solution for every type of problem. So, it doesn’t amuse me when people on Agile / Scrum discussion boards try to indiscriminately apply those principles to just about every type of problem under the sun, and then when, predictably, things don’t work, they blame that Agile / Scrum is not being applied in its spirit. Have you ever seen a project manager so baptized that he won’t think beyond the book ? I think those blind preachers are just living like a frog in a well.

Try to imagine this scene pictorially in your mind: You are walking down the road with a map in hand. You see a puddle of water that’s not on the map. What do you do ? Agilists will have you believe that waterfallists will merrily walk through (or drown through, depending on depth of the puddle) whereas Agilists will “inspect and adapt” a sophisticated term for “trial and error” (which is, by no means, a bad term). So, they will step into the puddle a little every time, and based on the results of that, determine if they want to go ahead or change the course. I will ask you something: forget the books (its actually much better if you burn them literally), what will you do if no one told you anything. You will simply avoid it. I think humans are infinitely more capable of acting on their own, especially in matters of their own interest, that they don’t require anyone to tell them what to do. So, let’s not try to be their parents. Right? Or wrong?

In my small world, we are all grown-ups who have a right to view the problem with our own private lenses without owing anyone an iota of apology about the unique problems we alone face and struggle with, whether we understand or not. And we are free to blend any major or minor, known or unknown, new or old, and right or wrong methdologies to solve our own problems. As real-life practitioners solving non-glamorous, project management template-agnostic problems, we don’t need to display our camarederie and unflinching loyalty to some obscure methodology whatever anyone says. After all, they don’t sign your paychecks, but your customers do! Your solutions should only be aligned to what your customer wants – and not what a methodology wants!

So, it is simply not about methodology. If anything, its all about:

  • Refusing to get ringfenced by a process document, howsoever great that might be
  • Outrightly rejecting the idea that things can’t be improved any further
  • Optimizing things that can be done better than in the past when a given process was written
  • Eliminating deadwood from a process simply becuase something makes no sense
  • Embracing change because your problems were not designed keeping your process in mindIn fact, it is not even about a project anymore. If anything, it is about your fundamental right to free thinking. But then, my view should not come in your way of your free thinking :).

So, think again….does your project management methodology lets you free think?

Codifying Agile Skills or creating more checklists ?

Did you check out the Agile Skills Projects yet ? It seems to be a new and interesting initiative to “… establish a common baseline of the skills an Agile developer needs to have, including a shared vocabulary and understanding of fundamental practices”.

They talk about Agile Skills Matrix that has seven essential skills, or the Seven Pillars, organized into five skill levels.

Seven Pillars include:

  1. Product: Agile teams must share a vision of what they are working to achieve; the business problem they are trying to solve.  When a developer understands the problem domain, they can help the Product Owner evaluate upcoming features.  Understanding of the product is the first step to creating a fully cross-functional team.

  2. Collaboration: Teamwork is the heart of Agile software development.  A truly collaborative team shares all its information freely.  They share their individual knowledge and skills by working closely with their teammates.  Teammates who fully rely on one another become much more effective as a whole, yet simultaneously increase individual skill levels.

  3. Business Value: The purpose of any software development effort is to create features that deliver business value.  A good Agile developer will focus on delivering that value.  They do not add complexity just to make their program ‘cooler’.  Instead they will base their work on business priority.  They produce a steady stream of running, tested software with real business value.  They will build only what is needed to solve today’s problem, knowing that building too little or too much is waste. 

  4. Supportive Culture: Any highly productive enterprise is founded in learning. Practitioners in an effective agile team view everything they do as an opportunity to learn. Every step is an experiment intended to make real progress, and to clear our vision of what to do next. We accept and embrace the fact that when a task is done, we’ll see better what we should have done. That helps us see what to do next.

  5. Confidence: Those on an Agile project strive to know the state of their code and the state of their project.  They do not share their work until they can prove it functions properly and is well designed and implemented.  They report progress based upon the actual rate at which fully tested features of real business value are being created.

  6. Technical Excellence: Developers understand, and choose from, many possible technical ways to satisfy business needs–choices that reflect a craft that balances design, use, and support.  They provide the technical underpinnings that enable us always to move forward at a steady pace.  They do this using principles of truly simple design, combined with a grasp of technical debt and the means to keep it under control. They use the best techniques for keeping the design under control without excessive work or rework.

  7. Self Improvement: An Agile team member seeks new ways of doing things, while keeping existing skills as sharp as possible.  They know that ours is an ever changing world and they strive to be prepared to take advantage of anything new.  They are both introspective and aware of what’s going on around them, be it in the team, or the larger business context. They will take action to fix things that aren’t right, and to help those who are in trouble. 

And the skill levels are:

  1. Learning: Individuals at this level have been exposed to a skill, but do not yet have firsthand experience with it.  This may come through reading or conversation, through a formal class or casual presentation at a local user’s group. A training course should provide at least this level of attainment.

  2. Practitioner: At this level, one has practiced the skill in a ‘safe’ environment.  They may have taken a course with hands-on exercises, or recreated the examples from a book.  The Novice level represents the first big step from abstract to concrete.  A wise organization will not let a Practitioner loose on their own.  They do not yet know what they do not know. They do not yet know what they do not know how to do. A training course with a sufficient hands-on component could provide this level of attainment.

  3. Journeyman: A Journeyman is known to possess the specific skill.  They have demonstrated a practical knowledge of the skill in various environments.  They can work on their own, or to increase the competence of a team.  A Team member of a lower skill level will learn from them, a team member of a higher skill level will recognize their expertise. One cannot be taught to be a Journeyman, one can only learn it.  Regardless of the skill category, maintaining Journeyman status, one must practice their current techniques and seek out new and better ones.

  4. Master: A Master must possess unquestioned competence in the particular skill.  They know it cold; it is second nature to them. But, it does not stop there.  To be accepted as a Master, they must also accept the additional job of bringing up the skill level of their team mates.  They do this in several ways. They serve as an example of proper practice.  If you want to know how it is done, observe a Master.  They partner with other team members and actively share their knowledge and experience. They seek out teachable moments.  When asked a question, they prefer to engage in a dialog that will lead the questioner to an answer, rather than making a pronouncement.

  5. Contributor: To be a Contributor, one must have added to our community’s understanding of how to practice our craft.   Contributors bring new ideas to light.  They develop and evaluate new techniques.  They are the silverbacks and young turks, who advance the state of the art.  They may be seen as today’s heretics.

I think this is a good effort to break-down the craft into little more tangible traits that developers need to acquire, thus making the body of knowledge available, one step at a time. Further, it is also refreshing to see things other than technical excellence being part of the Seven Pillars. I have long believed that Sociology should be part of the computer science curriculum now that software development is a serious team effort, and eventual success being largely influenced by human dynamics in the team than individual programming effort alone, howsoever supreme and important that might be.

However, all good things have a shelf life, and all good ideas become fertile grounds for next set of consulting and assessment industry to take-off (and Agile Skills Project makes no secret of its desire to support any effort to certify developers, even though they won’t officially endorse nor condemn any particular certification). I suspect it won’t be long before people will be certified on this (yet another!) 5-level scale of individual proficiency. Look at how successfully we have reduced Scrum to a series of checklist items in the so-called Nokia Test. Then there is this perpetual debate raging in the Agile / Scrum community on merits and demerits of certification, with everyone sort of nodding their heads that certification can’t guarantee competence, but even then everyone continues to offer some form of certification. I even suspect this might soon find its way into job descriptions. However, my bigger question is: will having a bunch of certified developers on your next Agile team ensure, or at least improve chances of success ? Are we unwittingly creating a class-hierarchy in Agile teams where developers will have to be of a certain class (or caste, if you like that way) to be part of the team ? How does one judge things like ‘Confidence’ or ‘Self Improvement’ ? Does high rating guarantee domain competence, or portability of skills? What is better in a 5-people team: 5 Masters, or 3 Masters and 2 Practitioners? Does someone at the skill level ‘Learning’ stand a chance to be in the team of ‘Contributors’ without adversely impacting the team productivity? Further, this is by no means an exhaustive checklist of critical success factors, and hence, chances are that people will fill up some checklists and declare themselves some 86% compliant (on some arbitrary and controversial scale) and their managers will expect the next project to be a resounding success. However, you and I know better than that :).

I think these are highly contentious issues that could further create more checklists and thus degenerate a good intent into a big mess.

The best way to use this tool will be to self-use it and set personal targets to acquire higher levels of competency as one goes by. Any effort to use this as a yardstick to measure and compare individuals is, IMHO, against the basic ethos of a human endeavor, let alone software development. So, shun the checklist-driven software development. 

Software trends survey…

This is an interesting update on software trends:

A recent survey of more than 6,000 senior-level business leaders and software development executives found optimism for higher IT budgets and a preference for outsourcing, agile methods, and enterprise applications. In the survey, sponsored by software consulting firm SoftServe, 60 percent of respondents reported increases in 2009 IT development budgets, despite the uncertain global economic climate. Some 26 percent indicated that budgets had increased by more than 10 percent over 2008. Some 38 percent said they use some type of software development outsourcing. Of those, 67 percent used locations in India, followed by the emergence of Ukraine, China, and other Eastern European countries.

Nearly three-quarters (71 percent) listed new product or software development as a top priority, while cost- and expense-cutting followed with 51 percent, and improving usability ranked third with 49 percent. Forty-two percent favored agile methodologies as their chosen development model, with only 18 percent preferrng the waterfall method. About one-third (36 percent) employed Capability Maturity Model Integration (CMMI) as their process maturity and quality model with Six Sigma used by 25 percent. Some 62 percent of respondents were focused on enterprise applications, and 51 percent on Web-based applications.

Based on this survey, these are my observations:

  • Investments in IT budgets continue to rise, despite economic conditions. I guess the fact that any serious IT development is a long-led-time endeavor, the motivation might be to sit in the labs and work on getting stuff out just when the market comes up.
  • 67% are using India as outsourcing destination, which is a significant number, what will all predictions of anti-outsoucing sentiments, or a shrinking cost arbitrage, or low design skills in low-cost destinations such as India. They all can’t be wrong at the same time. I think hard numbers are almost always a stronger evidence.
  • CMMI is (still !) not dead! If one-third of industry uses CMMI, clearly there still must be benefits of it, and I think the non-believers might be well-advised to stop harping on CMMI as being a so 20th-centurish heavyweight, beaurecratic and document-intensive process and start learning how it adds value to the the compaies that still believe in it.
  • 42% favoring Agile methods is a good news, and I guess the 18% favoring Waterfall is the last post of resistance that will unfortunately still continue to wiggle its severed tail for many many years to come. I am more interested to know how these 42% of companies / professionals use Agile and how effective it is for them. If the only reason they are doing Agile is because that is the new ‘in thing’, and they have no clue how effective it is to them, I would argue that those 18% Waterfall loyalists are perhaps better than those neo-Agilists becuase they probably have a strong enough reason to cling on the their bad Waterwall ways – one of them could even be that it works for them! After all, these is no such thing as one single right way to Software Nirvana, and no one single method can claim to have monopoly over all the best practices 🙂
  • I could not understand how some 25% were using Six Sigma in the same breath as 36% were using CMMI as their process maturity and quality model. I think Six Sigma is not a ‘model’ in the same sense as CMMI (or even Agile for that matter). Secondly, if you are a Motorola making cellphones (perhaps not anymore), then six sigma production means less than 3.4 defects per million. If you are using CMMI for software development, you know that you are using a set of staircased practices (at each maturity level) that guide you on ‘what to do’ at each step. I can also understand using Six Sigma principles and methods to measure and statistically management processes like a peer review or a testing process, but I don’t undersand how Six Sigma qualifies for a full-lifecycle quality model in the same league as a CMMI. Or, am I completely missing the point???
  • I am surprised to find no mention of Lean, unless the Agile umbrella includes all shades of original Agile methods, as well as Post-Agile methods.

All surveys are sampling exercise and must be discounted for its inherent flaws, design constraints and execution issues. Despite this, it is always interesting to analyse trends over a period of time, as they indicate a movement and help us assess the wind direction and speed.

Addressing the issue of “social loafing” in large teams

 Large teams might be inevitable in certain large endeavors, but there are several benefits of small teams. A small team can build and maintain a strong culture and a character that gets better with time. Small teams quickly learn the invaluable skills in teamwork and interdependence that lead to higher efficiencies while ensuring that individual team members don’t end up competing against each other but rather collaborate on the common objectives. Small teams also mean small egos 🙂

One of the biggest motivations of making smaller teams is to provide higher levels of transparency and task accountability to individual team members. A large team tends to hide inefficiencies, both of its structure and of its people. One particular problem in a large team is the problem of “social loafing” – something that is perhaps best described in this poem by Charles Osgood:


There was a most important job that needed to be done,
And no reason not to do it, there was absolutely none.
But in vital matters such as this, the thing you have to ask
Is who exactly will it be who’ll carry out the task?

Anybody could have told you that Everybody knew
That this was something Somebody would surely have to do.
Nobody was unwilling; Anybody had the ability.
But Nobody believed that it was their responsibility.

It seemed to be a job that Anybody could have done,
If Anybody thought he was supposed to be the one.
But since Everybody recognized that Anybody could,
Everybody took for granted that Somebody would.

But Nobody told anybody that we are aware of,
That he would be in charge of seeing it was taken care of.
And Nobody took it on himself to follow through,
And do what Everybody thought that Somebody would do.

When what Everybody needed so did not get done at all,
Everybody was complaining that somebody dropped the ball.
Anybody then could see it was an awful crying shame,
And Everybody looked around for Somebody to blame.

Somebody should have done the job
And Everybody should have,
But in the end Nobody did
What Anybody could have.

This is a great description of how so many ‘obvious’ things don’t get done – either due to miscommunication, or misunderstanding, wrong assumptions, or sometimes just shirking away from the responsibility. One of my best personal examples is working for a community team as a volunteer. I believe working for a community as a volunteer is the greatest way to hone one’s teamwork – if you can get people who are not motivated by money or power or promotions to work together for a job, you can do anything! So, I was part of this team of a really nice bunch of 15-odd people who was required to serve this 400+ families. This so-called executive committee was required to plan social events for the community. What I found was that 80% of the people on this team were there just for the meaningless social prestige. Over 90% of the work was done by just one individual (and I was doing another 5% and rest of the entire team doing the remaining 5%). Those 80% of the people were otherwise regular nice people, but when part of this large team, they could not be counted upon to deliver the goods. We organized some wonderful events, but it was mostly the two or three of us who did maximum running around and the rest of the team just travelling First Class. After a few months, I was ready to quit (I eventually quit that team at the next team election. What I find is that the new executive committee team has very similar effort distribution – so it was clearly not me who was an aberration :)).

In my professional experience, I have been involved in some really large software teams, up to 190+ people on a single product. While these efforts were large and simply required those many number of people, we used small teams, not more than 7-9 people each, to divide and manage the work. Each of these programs was divided into such a number of small project teams, each a self-contained and autonomous unit that could deliver its functionality with minimum external dependency. Small teams have smaller number of communication paths, and allow fostering of meaningful teamwork rather than poisionous politics. It also is a great way to groom technical leadership and managerial expertise in the teams. A large team is no fun for people to volunteer and train for roles and tasks that require building special skills. But a small team must often replicate several skills in each team, and hence is a great way to groom future leadership apart from also acting as a derisking strategy to counter impact of attrition. Agile practices advocate small teams for achieving high team throughput, and the issue of social loafing is indirectly addressed by things like daily scrum meetings where social shame and the feeling of letting down the team ‘forces’ team members to get their act together. An excellent discussion of social loafing can be found here.

Conclusions

Social Loafing is a real team dysfunction not restricted to a given country, culture, society, industry or team size. It has been observed in all types of societies and all kinds of groups. Making the team size small is one of the ways to address social loafing. In the context of software teams, everything depends on how individuals make commitments and live up to them. And hence, it becomes extremely important for a manager to be aware of this team dysfunction and evolve strategies to deal with it. Having a small team with clear roles and responsibilities, and setting common standards for work evaluation are some of the ways that can reduce the extent of social loafing in teams and improve morale and team productivity.

How Lean thinking improves productivity in software teams?

At its core, productivity for a software team (often wrongly termed as programming productivity because software development is much more than mere programming) looks like a great idea. In its simplest form, it compares output of the team (amount of useful and usable software created, amount of unnecessary rework done, number of defects produced, etc.) to the input (time, effort and resources invested in producing software) factored-in by the uniqueness of the given software  endeavor and other external environmental factors (complexity of the software being produced, impact due to team sizes, nature of team spread and its familiarity with the problem at hand, problem domain, and so on). Unless you are willing to discount software development as a non-business critical activity undertaken purely for the labor of love that should not be ‘measured’ lest it dilutes the very ethos of software development as a creative cognitive endeavor, one might agree, albeit reluctantly, that some measure of how well we are doing is clearly in order. Can you think of a single business activity where such a measure should not be done?

In last few decades, researchers and statisticians have tried to create quantitative measures of productivity. However, due to an often unilaterally percieved inherent ‘uniqueness’ of every software endeavor, practitioners have consistently rejected such measures contending that there are either far too many assumptions in its definition and far too many variations in practice, or both. Instead of focusing on the ‘vital few’ commonalities, it is only unfortunate that practitioners have virtually rejected productivity measures in all forms by considering the ‘trivial many’ differences. What could have gained the industry by considering a big-grain definition of productivity was unfortunately lost because of focusing on decimal-digit precision of productivity measures.

In this blog, I will discuss the subject of productivity from a conceptual plane, and explore how Lean thinking helps us understand it little better by establishing a correlation between wastes in software development and productivity of the team. Lean thinking is much more than identification and elimination of the seven types of wastes, but I will only take up the issue of wastes in this blog.

In the simplest form, we can call a process or an activity productive if it creates an output higher than its input. Though the input and output are often in two different and inequitable currencies, we are still mostly able to compare them. What that means at a conceptual level is that:

  • If Output < Input, we can say that the productivity is low
  • If Output > Input, we can say that the productuvuty is high
  • If rework is high, we can say that the productivity is low
  • If rework is low, we can say that the productivity is high 

These are highly generic views of productivity that most of us understand, but unfortunately they are not very usable to understand the factors that impact an output to become low, or the rework to become high. Developers might give one view, and testers might give another view as to why the output is low. There is no common vocabulary that can explain at a fundamental level what is it that is causing the productivity to dip. Context-specific variations (like differences in programming environment, percieved differences in complexities, developer competency, tester efficiency, etc.) might be too many to be usable to form any meaningful conclusions.

Lean thinking offers a more specific understanding into the wastes that typically make a process costlier (in terms of time or effort or both) than what it ought to be. Lean identifies seven types of wastes in a manufacturing setup. These wastes are known to introduce avoidable wastes, and Toyota pioneered a production system and an organizational culture that seeks to sysmetically eliminate root causes behind these wastes. Software practitioners, led by Mary and Tom Poppendieck, have investigated borrowing from Lean thinking into software development in last couple of years. Let’s examine how Lean encourages eliminations of following seven types of wastes in software development, and how they individually impact productivity of a team:

  1. Overproduction: Implementing a feature is an irreversible decision that takes lots of time and effort from the product engineering team, and delivery of a feature must wait until the next release cycle (whether the big-bang release in conventional software development, or the next real shippable release in agile development). If there are extra features in a software that remain unused because they are not of high priority but the engineering team is anyway spending effort to develop and test it, it is probably lowering the productivity of the team that could have used its effort elsewhere to develop other important features. The customer is also not benefiting from such overproduction of features, because he not only must wait for the desired features to be delivered, he must also pay the delivery team for those unimportant features that are not being used.
  2. Inventory: Anything that has not yet been delivered to the customer is work-in-progress (WIP) or inventory. The higher the inventory in a system, the higher is the number of resources locked-up and unavailable for any other activity. Even the work that has been completed but still sitting on the shelf is an inventory that is not creating any tangible value either to the team or the customer. If there are too many requirements yet to be implemented, it involves doing requirements analysis, design, development and testing without any of that effort being delivered to the customer, and hence increasing the work in progress and thus lowering the productivity at a business level. Even if a critical feature has beem completed, but it waiting simply because several other not-so-important features must be included alongwith in the next release, it is also adding to inventory because that important feature is not immediately delivered to the customer.
  3. Extra Processing: If the software is implemented in a complex manner that requires extra processing (e.g., a design that requires more storage, or duplicates data or code needlessly, etc.), it is either increasing the effort required to develop and test the software, or increasing the usability complexity, or both. Either ways, it is lowering the productivity. However, the opposite of extra processing is not smart programming, which might solve the problem in short run, but create maintenance problems in the long run.
  4. Motion: If the information is not readily available to the engineer, he/she has to spend unnecessary and avoidable time and effort that it takes to fetch the required information (which could anything – clarification about requirements, latest design specs, or locating the correct module from another team for integration, etc.), thus effectively lowering the productivity.
  5. Defects: Any defect that is passed on to the customers requires the engineering team to once again open up the code and spend effort to fix the problem. Since no new output is generated in this process, and the only output is fixing the functionality that should not have been broken in the first place, the net productivity over the entire product engineering lifecycle gets impacted adversely.
  6. Waiting: Perhaps no other project resource is as irrecoverable as lost time. Brooke’s Law reminds us how difficult, if not impossible, it is to recover delay from a project. Waiting for any resource for any reason creates unpredictability in software development apart from creating activities of zero or low output, which again reduces the productivity.
  7. Transportation: Handoffs are perhaps a Fordism legacy to the waterfall development where a team of business analysts would complete their work in isolation and handover the specs to dev team. Dev teams in turn would work in isolation and pass on the software to test team for testing, and so on. If all these teams are working in silos, such practice of working in isolation would often result in information distortion (or loss) down the stream, which could result in missed requirements, misunderstood requirements, unclear design, and so on. All these will eventually require extra (and unplanned) effort to implement the right functionality or fix the broken functionality, thus lowering net productivity over the entire engineering lifecycle.

 

As we can see, analyzing wastes in software development using the Lean thinking provides us a technology-independent and function-neutral view of the inefficiencies of the software process that can be used to achieve substantial improvements in productivity of the teams. None of these ‘attack’ an individual for lower productivity, and only look at the systemic deficiencies that must be eliminated for a team to perform at higher level.  However, of what possible business value would be the process of elimination of such wastes if we can’t picture ‘before’ and ‘after’ ? It is extremely important to assess the state before and after introducing a change – how else do we know if our efforts are yielding the desired results ? So, measuring the productivity in some absolute numbers is like the feedback whether an improvement is happening in the right direction (or not).

Lean thinking or not, do you measure productivity of your software teams? If yes, what for, and if not, why not?

Why Agile doesn’t sell with Management ?

Agile thought movement has been around in different stages of evolution (and not to mention, different vocabulary from different preachers and schools of thought)  from 1960s – perhaps it always co-existed with its better-known but half-effective cousin, Waterfall, all those decades without ever becoming a mainstream / preferred method of solving complex problems. Declaration of Agile Manifesto in 2001 was a watershed event, and the body of literature that has grown since then is simply mindboggling. There is a huge experience base on various Agile methodologies and methods that seems to be gaining firm ground globally every single passing day.

Divide and Rule

There can’t be any doubt that the fundamental idea to approach a problem in smaller pieces is great – it is logical, it is result-oriented, it is economical and it is highly intuitive. After all, the Imperial British conquered half the world by their brilliantly cunning doctrine of ‘divide and rule’. It allows the human mind to focus on what is ‘visible’ here and now as opposed to worrying endlessly about the entire length and breadth of the problem which may never ever be crisply definable, or might require a disporportionately large amount of time (and effort) to clearly define it – by which time it is quite likely that the problem might have undergone changes due to natural ageing or changes in environment around it, or an improved clarity into the problem has given rise to another thought process about its solution, or both. Frankly, none of them is bad in the sense it is always better to know more clear definition of the problem or a more clear definition of the solution. The only problem is that it takes an inordinately large amount of time to reach there, with no mechanism to know if what is being considered will actually work as envisaged. While executing, one could encounter several more unknowns on the way that force changing the laid-out strategy. However, when you slice the problem in meaningful increments, you allow the solution to grow, one slice at a time. Of course, you still need to have the strategy to accomplish the top-level vision – but the working plans must not attempt to address that entire strategy in one single-pass. There must be a mechanism to (a) breakdown a problem into meaningful slices that (b) can be accomplished individually and incrementally to (c) make a definitive progress towards the eventual goals (d) without unduly front-loading the progress by later-day uncertainties. Agile encourages establishing a beachhead and then reinforcing it, slowly and steadily as opposed to landing in the middle of a battlefield and getting fired at from all directions. As a concept, this is splendid.

Sensible Trade-offs

It also allows a sensible trade-off between cost (resources), time and scope of work to be performed – the good old ‘triple constraints’ of any project. A small team could be highly efficient and could be easily composed of experts that don’t have to worry about a lot of non-work related problems that will eventually happen to any larger team. However, a small team can only achieve so much in a given time period. So, if you want to accomplish more work than what a bunch of experts can undertake at a time, you either must be willing to have a larger team to deliver in the same short time period, or allow the highly-efficient team to work for longer period. There is enough historical evidence to suggest that small teams are the best in solving just about any problem that requires intense human interaction and usage of individual human cognitivate skills. However, not every large task could wait for a small team to complete over prolonged time (as also attested to by Fred Brooks in his classic, The Mythical Man-month). Commercial pressures (“time to market” to meet the shopping season and to beat the competitor), rapid obsolesence of technology, fluctuating customer needs in short-term and uncertain preferences over long time, etc. are just some of the market-driven reasons why running a long development-cum-release cycle is often a bad idea. Add to that indirect factors like manpower attrition risks over long-term, uncertain economic situation in the coming quarter, etc. and you have a complete set of risks that have nothing to do with the problem at hand! However, when we work with long-cycle single-pass development, we tend to create a Frankenstien of project management because most such problems will now have to be ‘supported’ just because of choosing a particular problem-solving method! Mind you, most of these were not the problems originally stated as the technical challenges, but we piled them on because of our management decisions. Developing products incrementally allows us to filter out such indirect risks and only focus on core risks related to problem at hand.

Short feedback cycle

Agile promotes ‘doing it now’ over ‘deliberating it forever’ and rely on continuous feedback on working software as a faster and more reliable means to improve software than doing a rather late integration and a back-loaded testing cycle that is always ‘too little, too late’. Agile makes it possible for teams to develop (and deliver) software faster thereby providing early visibility to Customers by way of ‘working software’ and get an early feedback on important aspects like functionality, usability, scalability, workflow, and so on. Customers are happy to see a working software because they are mostly used to documents that don’t mean much to them. Management is happy because the customer is happy, and the teams are happy because they get an immediate feedback on their work that allows them to improve their products. There is clearly a merit in having a short feedback cycle.

Painless Change

Agile handles changes elegantly throughout the development cycle. Short of handling a mid-flight change, it allows just about any change to be introduced at any stage of development. If we believe that it is beyond the human capability to correctly specify every single requirement of a system unambiguously – not just in one go but ever – then this tweak of handling changes on the go makes very smart sense. As we have seen from RUP, the ‘phases’ of a project don’t really mimic engineering activities undertaken at a given point in the project lifecycle – it might as well be the case of shifting weights in each of the phases, but it can never be the case that we could be done with all Requirements in any phase and then move on (if that were the case, Waterfall would be little more effectives, perhaps). Of course, this strategy to handle changes flexibly means that there is never a clear and firm requirements baseline, and hence it might be near impossible to define a fixed-price work contract (or an outsourcing contract), but if the customer and the delivery organization both understand the new way of doing business, it will just come out as good, or better. The point is, a work contract exists to support the project and not to dictate how a project must handle changes. If it is the fundamental nature of the beast, we better have contracts that recognize this hard truth and allow both the parties to engage in a win-win relationship than a contractual relationship that frustrates everyone.

Team Ownership

Traditional teams have long followed the model of ‘division of labor’, or some such variant. In a small team, everyone does everything, but the larger the teams grow, the more is need for ‘vertical differentiation’ and ‘horizontal differentiation’ and it allows people to make decisions for themselves commensurate to their functional skills and their level of competency in them. Again, this is not bad, for we can’t confuse between ‘desire’ and ‘competency’. So, in theory, the traditional distribution of responsibility works well, and perhaps worked well in yesteryears (or maybe it didn’t, but that is not the point here). It certainly doesn’t work anymore. The historical advent of ‘trade unions’ in traditional industrial environment was largely motivated by the desire to participate in management process, and we see similar traits in knowledge industry as well. I think this is a positive thing, and allows ‘workers’ to become better informed, and hence better engaged in decision-making process, have better ownership and accountability of the results. Agile has done tremendous service institutionalizing this so important issue.

Agile improves team ownership of tasks and creates ‘self-organizing’ teams. A team that collectively makes its decision is more likely to own it and support its team members in realizing their individual goals that add up to the team goals. A ‘mutually exclusive and collectively exhaustive’ cross-functional team will also be highly interdependent on its members to achieve their shared goals, hence likely to have a heathy mutual respect. All these enhance a team’s sense of pride and ownership. While I strongly believe just a mere adoption of Agile practices is not likely to make teams self-organizing overnight, it still is a tremendous improvement from the traditional management structure.

So then, what is the problem ?

These are great strategies to make software development more meaningful and value-adding to customers, and less painful to developers. Still, how come most companies (read ‘managers’) have not warmed up to the idea, let alone adopted it ? In a recent blog, Allan Kelly says the adoption of Agile is probably in the range of 5%-15%. I would imagine the number more closer to lower bound than upper bound, and even then, I am little sceptic about it real implementation and actual effectiveness. How do we know that the Agile implementation in those companies is not just limited to peripheral pilot projects ? How do we know that when a 10,000 people company says they adopt Agile, they are not saying that only some projects use Agile ? Joe Little maintains list of firms practicing Scrum, clearly the favorite Agile methodology, but there are just 160+ companies in it. Definitely, many companies doing Agile are not listed here, but I also know of companies listed in that list who just pay lip service to Agile. There were also recent discussions on implementations of Agile in large companies in some Agile / Scrum lists, and even though the debate is inconclusive (which is not so relevant), the fact remains that there are real issues in adoption of Agile in larger world. My question is, why is it so?

I think there are several reasons, and in this blog, I will not include the most favorite punching bag 🙂 but rather focus internally. I believe the way Agile buffet has been laid out, it offers considerable challenges that self-limits any further growth, at least in its current avatar. For one, Agile is clearly portrayed as anti-establishment. Now, this is not necessarily bad, but an outright contempt of management, so much so that they are not even wildly considered in the category of ‘chickens’, in my view, makes ‘management’ uninterested in knowing about it any further. If the so-called command and control was one-sided (towards management) and bad, Agile is equally one-sided (towards teams) and equally bad. I am pro-empowerment, so this statement needs to be understood well before I am misunderstood. There is all merit in the team owning up its own decisions and being accountable for its decisions, but they also owe to the management to be answerable and accountable for their decisions, progress and results. In traditional projects, the management had a grip on the overall project scope and schedule, even if there was an element of uncertainty in those commitments. In Agile projects, we took away the big picture, denied the sense of control to management (again, neither good nor bad – just stating the facts as they are) and worse, we are in no better condition to give them a sense of the overall project. Yes, we have a better grip in the current iteration, but are we any better informed with similar margin of errors as before, in predicting the overall projects ? I doubt.

So, if the result of an Agile adoption is that management is not involved, is getting no better commitment from teams on the final project delivery, is kept away from periodic progress, is not allowed to be part of project meetings, and has no say on the work being undertaken or when dropped from a sprint without any prior two-way discussion, I don’t see how that same management is going to bless the adoption of Agile in his teams? If it doesn’t help him having a better confidence about his business’s collective ability to solve problems, howsomuch it might help his teams, I don’t think any human being is going to be enthusiastic about it. So, it is not surprising that the Agile adoption continues to be so low, even when the benefits clearly outweigh the costs of adopting Agile. I think the bottleneck is not management, we are the bottleneck. We have failed to include management as part of the solution team. We just conveniently decided (unilaterally, I must add) that management is the classic villian who must be kept away if the team has to achieve something worthwhile.

I think in our zeal to fix the ‘problem’, we just let the pendulum swing to the other side, and it must eventually settle down somewhere in-between to be of use because that is the real world. Perched on extremes, it has no value because of the extreme positions it must take to stay put there which are not closely related to the real world. I believe that a New Agile will evolve (in my view, it is already there – just that some people are so steadfastly holding on to the remnants of the old Agile) to become a better fit in the real world and include all parts of the workforce as respectable members of the solution team. Smart among us will freely drop the dogmatic aspects of Old Agile and quickly adopt things that work for them, whether prescribed or not.

We already see people experimenting with Lean, Kanban, Scrumban…the smart ones will not limit their potential with someone else’s prescription.

The rest will perish together.

Do you follow Project Management ‘religiously’ ?

Project Management is perhaps one of the most fiercely debated and grossly misunderstood disciplines in the software field currently, hence let me throw in a disclaimer first: if you are a small team of experts and/or people well-known to each other (e.g., have worked together as a team earlier), and situated in a collocated fashion, doing a lot of ‘creative work’ that can’t be very ‘accurately’ scoped, let alone managed; you probably will find ideas of formal project management a huge overkill (on time, effort, money and might even seem to stifle creativity), and you might be better off considering ‘lightweight’ methods like Agile Project Management / Scrum in the context of software development (well, nothing stops you from deploying pieces of Agile / Scrum in a non-software context – it is based on common sense after all). Small projects, small teams can afford to define a process that exploits the tailwind:

  • A small and collocated team means more direct interaction among team members and lesser management overheads in formalizing project communication (e.g., project status reporting, team meetings, etc.). This reduces the time it takes to collate, transmit and share important project information and also decreases the possibility of information distortion or confusion. A small team means there is high signal to noise ratio in team communications, is also leads to a better utilization of time spent in transmissing, receiving or digesting communication. Further, to schedule any event, one can always convene impromptu meetings around a whiteboard or around the coffee table without worrying about people’s already double-booked calendars or finding a place large enough to fit the team for the next meeting, and so on.
  • Having a group of experts means there is low(er) need for managerial supervision and people are generally ‘hand-picked’ for their knowledge, skills and abilities that could typically be ‘mutually exclusive, collectively exhaustive’. This ensures tight interdepency and very high mutual respect among team members. There is lesser ‘competition’ among team members and everyone knows that “united we stand, divided we fall” which fosters teambuilding.
  • For a small project, a large swing in any of the project constraints (time, cost, scope) could seriously and irretrievably impact the project in a downward spiral. If the work is new to the team, it could spell bigger trouble, especially if there is no way to do mid-course corrections. Executing projects in an iterative fashion helps shorten the feedback loop and give a chance to make necessary corrections to improve its ability to stay on-course and eventually hit the target. Additionally, if there is a customer involvement on these short iterations, the quality and value of these feedbacks could improve substantially lest any decisions are made that are hard to change if required.

However, if your project is anything more than a handful of people, takes more than a few months and entails issues like external dependencies with multiple vendors and ISVs (Independent Software Vendors), substantial technical and managerial risks, subcontracting / co-development, etc., you probably might improve chances of success using a formal project management methodology like PMBoK or PRINCE2 than without it. Agilists will argue and disagree with me that Agile / Scrum is not suited for large projects, but if you are new to Agile as well as to large projects, you might want to consider all options and associated risks – don’t believe in what worked for others, it might or might not work for you (remember the golden rule – we are all different), and your company might have a different set of constraints and tolerance to performance. 

The size, complexity, criticality, and risk profile of a project, and organizational appetite for project risk and uncertainty, and organizational culture would generally determine the level and need of management control required, and hence one must always right-size the project management methodology based on the context. Both, PMBoK / PRINCE2 and Agile methods evolved in response to challenges seen in executing projects successfully. While PMBoK / PRINCE2 took the approach that projects most often failed due to lack right levels of management oversight, planning, execution and control; Agillists felt the key reason was the fundamental nature of software being a ‘wicked problem‘ that just could not be managed using an obsolete waterfall model of developing software – it needed a better way to develop software in short iterations with very close-knit self-managed teams.

Without getting my loyalties needlessly divided betwen the two of these approaches, I believe there are merits in both the thoughts. I refuse to believe that any non-trivial project can be managed by piecemeal iterations that give no commitment whatsoever on the overall project performance, nor give any method to assess if the overall project is indeed on the right track (so, even if first few iterations are achieving the desired ‘veolcity’, what is the guarantee that subsequent iterations will also move at the same pace?). On the other hand, doing an iterative development is highly intiutive and very effective way to learn from past experiences to plan next steps, especilly on any non-routine project that is full of unknowns and assumptions. To me, these represent ‘mix-and-match’ ideas that one should pick up and apply wherever required, without getting baptized to the school of thought. I find most process wars have taken the disproportionate dimensions of falling just short of religious wars – you have to belong to one of the camps, and must talk evil about the other camp to be seen as a law-abiding corporate citizen of that camp. In life, we borrow ideas around us – our housing society, professional volunteer society we volunteer for, our kids playground, gardening, home improvement, nature, science, religion, other cultures…without necessarily converting to another faith or religion or society just because we like an idea that we want to borrow. So why should that be required in project management methods ???

I think this process war will be won by pragmatics. Those who take a position on extreme ends of the continuum and insist on always applying a particular style as the panacea might be in for a complete surprise because no two problems were created equal. A prudent approach is to evaluate all options and then decide which is the best response to a problem, as opposed to blindly follow a project management approach ‘religiously’.

 

Do you follow your project management aproach ‘religiously’ ?

Blame your “flaccid developers” and your “flaccid customers” for your poor quality products !

This is the text from a recent announcement for a course by Ken Schwaber on “Flaccid Scrum – A New Pandemic?” (text underlining is mine):

Scrum has been a very widely adopted Agile process, used for managing such complex work as systems development and development of product releases. When waterfall is no longer in place, however, a lot of long standing habits and dysfunctions have come to light. This is particularly true with Scrum, because transparency is emphasized in Scrum projects.

Some of the dysfunctions include poor quality product and completely inadequate development practices and infrastructure. These arose because the effects of them couldn’t be seen very clearly in a waterfall project. In a Scrum project, the impact of poor quality caused by inadequate practices and tooling are seen in every Sprint.

The primary habits that hinder us are flaccid developers and flaccid customers who believe in magic, as in:

Unskilled developers – most developers working in a team are unable to build an increment of product within an iteration. They are unfamiliar with modern engineering and quality practices, and they don’t have an infrastructure supportive of these practices.

Ignorant customer – most customers are still used to throwing a book of  requirements over the wall to development and wait for the slips to start occurring, all the time adding the inevitable and unavoidable changes.

Belief in magic – most customers and managers still believe that if they want something badly enough and pressure developers enough to do it, that it will happen. They don’t understand that the pressure valve is quality and long term product sustainability and viability.

Have you seen these problems? Is your company “tailoring” Scrum to death? Let Ken respond to your issues and questions!

Ken will describe how Scrum addresses these problems and will give us a preview of plans for the future of the Scrum certification efforts.

Here are my observations and thoughts from this synopsis:

  • line 2: what does “when waterfall is no longer in place” mean ? So, when waterwall was still in place, there were no issues ??? Somehow, one gets a feeling all the problems came to light only when Waterfall was “no longer in place”…so, why not get waterfall back in its place 🙂
  • line 5-6: In a Scrum project, the impact of poor quality caused by inadequate practices and tooling are seen in every Sprint ? …in every Sprint ?….now, wait a minute….I thought  Scrum project did not have any of these problems because there were no inadequate practices and tooling issues. And you certainly don’t expect to find such issues in every Sprint, or do you ?
  • line 7: OK, so now we have an official reason: “flaccid developers” and “flaccid customers” for all ills of the modern world. Wow! I am not sure if that is the best way to build trust either with teams or with customers by fingerpointing and squarely blaming them…without giving them a chance to even speak for themselves. And I thought Srcum was cool about trusting developers, not fixing blame on individuals, interacting with customers…but flacid developers and flaccid customer ??? Does it get any better than this ?
  • OK, I will probably agree ‘unskilled developers’ in the context of building an increment of product in an iteration, but what the hell is an ‘ignorant customer’ ? Try telling a customer that you are an ignorant customer…Scrum or no Scrum, you are guaranteed some unsurprising results ! And which Customer paying through his nose waits for the “slips to start occurring” ? In these tough economic times ?  
  • Is your company ‘tailoring’ Scrum to death ? I thought there were only two shades of Scrum – either you did Scrum-per-the-book or you did not. Since Scrum is the modern-day Peter Pan which refuses to grow up, we have only one version of Scrum to play around with, and unless some group of people decide now Scrum can grow to the next version (perhaps more because of commercial interests than anything else, whenever that happens). So, how can one ‘tailor’ Scrum – by any stretch of imagination, that is NOT Scrum. I mean, that is not allowed ! Right  ? So, why are we discussing it and wasting time – we might actually be acknowledging and unknowningly legitimizing that there is a secret world out there where Scrum can be ‘tailored’ and still be called Scrum ! That is sacrilege !

I always hate marketing messages that overpromise miracles, offer snake oil, belittle the intelligence of people, ridicule people…why can’t people simply stick to facts and figures ? Why don’t we talk in a non-intimidating language that encourages people to look up to what is being talked about ? And in the context of current subject, I doubt Scrum community is going down the right path if its next target is developers and customers. One of them does the work and the other one pays for it. Who gives us the right or the data to talk about any of them, in absentia ? Customers are customers, and even if they are irrational, they are still the customers, and whether paying or not, they alone are going to decide how they will behave. Yes, we might not like it, but is ridiculing them as ‘ignorant customers’ going to turn things into our favor ? Other industries like retail, hospitality, transportation, etc. have millions of years of collective experience in managing customers, yet they never break the singlemost cardinal rule of customer service: the customer is always right! And the first chance we software developers get to explain a poorly designed or a bad quality product and rightaway we blame the customers for it ! As if that is not enough, we then look inwards and blame the developers ! GREAT !

I think I will just develop software without blaming anyone else for my mistakes and limitations 🙂 

So you think you are a top-notch Project Manager ?

Building Pyramids is one of the most common examples and metaphors in project management, and should I also add, perhaps one that is almost always discussed about by people who have practically no knowledge of construction engineering, let alone experience in building a pyramid (after all, I don’t know too many people who can claim to have built a pyramid). There is always a classic problem of what is the right way to build a pyramid: if you go by the much-maligned and clearly fallen out of favor ‘waterfall’ way, what happens if the King dies midway and the Pyramid is still unusable ? Worse, what if the King comes half-way down the project and sees bottom half of the pyramid and remains unimpressed with the sight and either wants to change the project specs, or worse, wants to stop the funding because he is not happy with what he is getting. OK, you want to build the top first. What if the King finds the explanation of building the Pyramid top-down funny (remember, there is no past precedence of your idea, and you might be ridiculed for your ideas – it is 2500 BC, after all) and all he sees on the ground is a miniature Pyramid ? What happens if a rival King starts constructing a bigger pyramid – can you now change the designs and make your pyramid bigger than the rival King’s ? Surely, there are obvious limiations of this model.

The other alternative is building it the ‘Agile’ way - starting with a miniature fully-functional pyramid and then incrementally building it inside-out. Theoretically, it sounds like the right thing to do – if and when King comes for inspection (and you can always count on that !), he always sees a ‘complete product’ albeit a miniature one. If he is impressed by the ‘plan’, he might release more funds and theoretically, the construction could go on perpetually, i.e., until when the King one days breathes his last. That day, right after the next ‘iteration’, his body could be laid in eternal peace in the majestic pyramid. However, there is a major problem with using Pyramid metaphor to justify Agile way of construction. Most of us who are simply ignorant about construction engineering treat the problem as building a Lego Pyramid – there is only the civil component of construction that is considered in any well-meaning debate. What about plumbing, what about electricity (ok, there was no electricity those days but you surely need air and light ducts to serve the purposes that electricity would have served otherwise), what about the state of technology that was available back then, what about stairways, what about interiors, and so on. Can all these be built inside-out ? Construction engineering might not entirely agree with building a Pyramid incrementally.

Well, there are critics on both sides of the thick and high wall that separates these two worlds (with some of them, perhaps, perched atop the wall as well) and there seem to be strong emotional views about which is better and why. In this post, I will not add anymore fuel to fire, but share an interesting thing that I came across. Chances are, irrespective of what camp you belong to, you will find this simluation exercise interesting, and guess what, you actually get to build a pyramid ! There is this great game on BBC site that tests how good a project manager you are – by playing a game to build The Great Pyramid ! Here is what the text reads:

Journey back four and a half thousand years to Egypt’s Old Kingdom, to the Pyramid Age.

As the vizier, or head of state, you are about to undertake the most important project of your career – the building of the king’s pyramid.

To succeed in this task, you must be a good all-rounder. Not only should you be able to motivate your workforce, but you must have good observational skills and the ability to steer a barge up the Nile, avoiding hippos and crocodiles.

Have you got what it takes to be a pyramid builder?

What I specially liked about the project is that it places emphasis on just about everything you require as a project manager to complete you project successfully – domain skills, planning and scheduling, executing skills and the most important of them all – the people issues, like what type of people to hire (and how many) and how to determine the reward strategy. This is not just a drawing board game – you actually get to ‘execute’ project (at least I got to execute a part of it, and I guess the game might allow you to do more if you pass the first stage).

If this sounds like fun, click here to take the Pyramid Challenge.

PS: I took it once and lost the game. My ‘project’ failed miserably :(. Hmmmm….

PRINCE2 handles Project Tolerances better

Most project management frameworks and methods advocate (and actually require) ‘point-estimates’ in planning and scheduling. By ‘point-estimates’, I mean there is a ‘hard number’ that seems to be etched in stone, leaving with no ‘tolerance’ or ‘leeway’ for the project manager and his team. Even though we all understand that estimates are never point-estimates (and hence the project commitments are never a point-commitment), we still expect a firm estimate (and consequently a firm commitment) from a project manager.

For example, a given feature must be required by 23-June, but there is no recognition of the fact that 23-June might still be a couple of months away, and several things could go wrong or could change meanwhile thus affecting the validity of this date. In real-life, there are always tolerances, some allowable, some acceptable, some tolerable and some simply unacceptable. This is not limited to schedule along, but also apply to cost, quality, scope, project benefits, etc. In some cases, the tolerances are defined, for example for the project scope, requirements might be prioritized as ‘M-S-C’ (or Most, Should and Could). In a previous organization I worked for, we had a great system for prioritizing all product requirements as ‘A’ (for current version), ‘B’ (implement if time is available) or ‘C’ (long-term requirement, but is being identified right now just so that designers and developers are available of how the system will evolve over time – so that, hopefully, they could build their designs futureproof). However, most product managers are unwilling to commit to such tolerances (or atleast unwilling to acknowledge existance of such scope tolarances), perhaps fearing that development team will straighaway prune down the list to bare minimum ones. Hence, in most organizations and projects, there seem to be two sets of commitments: one set is for the customer and one set is for the team. The one for team are the more stringent ones, but in the heart of the hearts, upper management knows that not all of them will be achievable and hence makes a little less stringent commitment (perhaps a more realistic one !) to the customer, thereby cushioning themselves against Murphy and the real-world. The project team slogs over months, often spending long nights and weekends alone in office just to try and meet the set of commitments that have been given to them, hardly ever realizing that there is another set of more realistic commitments that perhaps don’t acknowledge the cost of their commitment and efforts the team members are currently paying.

Agile methods, and Scrum in particular, take a major step forward and acknowledge that long-term plans are prone to later-day variations (and thus a waste of time and effort for all practical purposes) and hence favor short iterations to minimize the possibility of a project drifting or overcommiting itself to impossible deadlines due to such wild and optimistic estimates for long-range plans. But, it goes a little too far by simply accepting a world where no project tolerances are recognized, let alone identified for a project or its iterations. When a team member comes back and informs that he needs more time to complete a task than originally estimated, it simply makes suitable changes in the sprint burndown chart without placing an upper-boundary condition on how much off-estimation is permissible? As per the revised commitments, if the engineer is able to complete the task in the current sprint, well and good, but if for some reason, he is not able to complete, you simply push it out to the next sprint. It may not be necessarily bad, but definitely doesn’t place any upper-bound on the allowances available to a project team, and hence might not find good favor from upper management or product management in the long run.

PRINCE2 not only legitimizes the existance of tolerances at project level, it also allows a project to inherit a portion of those tolerances at stage plans and team plans at various stages thus giving a reasonable and mutually-agreed upon leeway to project manager and team managers to use those tolerances to manage the projects better (than, say, worrying about some 2-day delay that they can’t avoid or not able to meet a scope requirement because some peripheral feature can’t be met in given time). The mere act of legitimizing project tolerances creates a healthy understanding among all stakeholders that in real-world, there will be variations in future due to factors that might not be known or understood fully, or might simply change over time. A prudent project manager might not restrict himself to his current project management framework, and might want to look at what PRINCE2 has to offer in this regard.

How do you handle project tolerances ?

How good is your ‘bad news management’ ?

No one likes surprises, least of all executives who might have given high-level commitments on project delivery to their peers in customer organizations, investors or other key stakeholders. I hold Project Managers largely responsible for poor ‘bad news management’. In most cases, there are half-baked estimates that never are a firm basis for project planning to begin with, or there are inaccurate project metrics that lead to an ambitious planning. During project execution, Murphy strikes and then the fun begins – the project manager is fighting a losing battle but is not willing to admit that (professional’s ego?), nor is he asking for help! The result is a blindly predictable situation where it is too late to do anything anymore. Could this have been avoided? Well, its possibility surely could have been lowered, even if not eliminated 100%.

My advise on how to minimize ‘bad news’ events would consist of:

  • Don’t rely on bravados. They might be ok for a weekend to complete a build, release some critical feature or deliver some part functionality, but to rely on them for anything larger and non-trivial would be setting up for grand failure. It definitely is not sustainable in the long-run.
  • Remember Fred Brooks when he said “how does a project gets late by a month – one day at a time”. No one likes to know about a project that suddenly pops up with a one-month delay. If that’s what you report, don’t expect any sympathies or support. Period. This typically happens when the project manager continues to have blind hope that the schedule he lost earlier would be recovered subsequently by some stroke of luck, magic or overtime, or all three. However, in reality, it just doesn’t happen. While you might be able to bring back some of the tasks on track, the time lost one day at a time just can’t be gained one month at a time without doing something radical like dropping a key feature, but not definitely by adding people (because that’s when you meet Brook’s Law).
  • Don’t overstuff your plate. It is very tempting to fill up the plate because that shows manager and team in a very good light – you know, like shining as a ‘good corporate citizen’ who understands company’s obligations to its customers and so on. However, non-essential features eventually take up lot of design, development and test effort relative to the value they provide to the customers. They also unnecessarily elongate the development cycle without creating an opportunity to get real customer feedback if those features are required at all. A project manager must do effective gatekeeping. 
  • Use enough prototyping and feedback cycles (like Agile development methodologies in software development).  

There are perhaps many more, but you get the idea – it is possible to reduce ‘bad news’ possibility for your project without hurting project goals.

That said, if there is a bad news, no point in covering it up, or externalizing the faults. Own it up as the Project Manager in-charge  – no one is going to hang you up for that. However, be prepared with an extremely thorough analysis of how the problem could be fixed, and the support you require from senior management to make it happen. Involve your team into a complete commitment to the revised proposal before discussing with the upper management. Also, try to explore multiple options and not a single-point solution – giving multiple solutions not only shows your analytical ability and intent, it also gives flexibility to upper management to take the best possible option. Finally, don’t go to your management as a postman and deliver bad news – go to your management with solutions and explain your preferred solution and various pros and cons.

A few years back, while working with a group in Germany, a project manager colleague of mine overcommitted on a delivery and decided to deliver the code against recommendations of his team. It was actually ‘code’ only – as it was not even compiling. I was the Quality Manager and I was not made aware of the delivery. When my boss came to know, he was naturally furious. However, before talking to our peer in the German office, we first sat down and made a careful plan to fix the problem. We got the team involved in it, and once we had a plan, we called up our German colleague. The fact that he was upset would be a gross understatement, but we worked sincerely to make up for our mistakes. In another case, during early phases in another project, the technical team delivered a high-level design that was a complete fiasco. The team in headquarters was extremely doubtful if we even understood the problem. In the morning, my boss pulled me in to that project and that evening, we were in the flight to Amsterdam. Next morning, we had transparent discussions with our peers in the Business Unit, and by afternoon, we were discussing high-level project plan. In both these cases, we had serious project problems, in one case in the delivery phase and in the second one during initiation phase itself. However, in both cases, we were confronted the issues as soon as we came to know about them, and without neither defending nor externalizing our initial actions that resulted in this problem, we worked towards finding a solution to the problem.

Failing first time is not a crime and almost everyone is entitled to a second life, but don’t expect a third life. That happens only in video games. In real-life, bad news will happen, but it can be managed much better if people are willing to keep project goals in the forefront and do not go about taking hardline positions. With sincerity and a willingness to fix the problem, we can improve our ‘bad news management’.

So, how good is your ‘bad news management’ ?

When are you planning to fail ?

Yes, you read it right…when are you planning to fail ?

In the world where insatiable hunger for ‘success’ is an obsessive-complusive disorder (OCD), we don’t think of ‘failure’ much. It is shunned, scoffed at, systematically eliminated (or mitigated, at least), avoided, bypassed, ignored….everything but embraced with open mind and open arms. All management ideas are directed towards the age-old wisdom of “if you fail to plan, you plan to fail” and not on something like…”what won’t kill you only makes you stronger”. All project management philosophies are centred around safeguarding the projects from any possible failures…but has that stopped projects from grand failures? Every risk management action is towards making the project safe from failing…and yet we still see so many projects biting the dust, struggling for survival. Failure appears to be a social embarrassment that is best avoided at dinner table conversations. New-age enterpreunership, especially in Internet world, has helped a lot to eliminate the stigma that eventually comes with people associated with any well-known failure, but in everyday lives, we still continue to play safe, rather extremely safe. Of course, I am not talking about breaking the law and driving without seat-belts on or driving in the middle of the road jeopardizing everyone’s life. I am talking about thinking like Fosbury and challenging the established way a high jump is done – even at the risk of failing because what you are about to propose hasn’t been tested and certified to succeed. I am talking about taking those small daily gambles that strenghten you when they fail. I am not interested in those small daily gambles that are supposed to strenghten you if they succeed – honestly, they don’t teach you anything. In fact, those small successes might limit your ability to reach for higher skies because you remain contended by those sweet-smelling early successes. In my view, people who don’t want to risk gentle failures must prepare themselves for grand failure !

The word ‘fail’ is such a four-letter word that is evokes very strong emotional responses. In an achievement-oriented society and success-intoxicated corporate culture, fear of failure drives people to seek safer havens. When choosing what subjects to take in college, we ‘force’ our children to take the ‘safest’ subjects – they are the subjects that have maximum job potential ensure maximum longevity in job market. (I agree that ‘force’ is not the right word here, but it is not in its literal sense that I use this word here. The ‘force’ can come from parental expectations, societal pressure, peer pressure, ‘coolnees’ of a job, perks of the job, etc.). Traditionally, they have been Engineering and Medicine and anything else that ensured a government job (in India, and I am sure every country has had its own fixations at different times). So, the foundation to seek safety from failure gets laid right at the start of one’s career (well, in my view it is erected right after birth and we are still curing it by the time we start our careers, but that’s for another blog post). After graduating, there is once again a massive derisking operation: find some company that has a ‘big’ name (even if one is doing a fairly mechanical job there). Basically, trade any hopes or ambitions to do something new and creative in life with rock-solid jobb safety in a mundane assignment! As a rookie, you then become a link in this enormous chain where your job is dictated by the volumes of SOP (Standard Operating Procedures) that trade your urge to experiment / innovate with a higher ‘percieved’ safety of the given process. The logic being: this is the way we know it has been done before, and since it worked the last time, we expect it will always work and hence this is the standard procedure. Wow !

You then become a manager and have to now run a project. You have the ‘process police’ breathing down your neck challenging every single thought, let alone a decision, to deviate from those SOPs. You want to try an innovative recruitment…no, that is too risky. Why not first prototype this sub-system…no, the finance won’t allow that because we can’t bill the customer since nothing tangible has been delivered. How about a 200% make-or-break bonus for this team that is up against Mission Impossible….no way ! we will have mutiny in other projects. How about this….and how about that… Finally you give up and give in…and the project somehow starts. Most optimizations, if any, have been seriously watered down by now, and are highly local at best. The result is predictable – no substantial improvement from the previous project, at best, and an utter fiasco, at worst.

Let’s pause for a moment and quickly run through the script here. We are taking every beaten path that has individually been successful in the past (or so we have been told), and yet there is no guarantee of success in its next run. In each of these situations, the project manager faces the music – after all, the process did work the last time and if it is not working this time, this must be a capability issue (or, even worse, attitude issue) with the manager. Being at short end of the stick and facing an imminent loss of professional credibility, the manager pushes his team to burn themselves over really late evenings and long weekends and somehow gets past the finish line…but not before incurring emotional and professional scars, not before pruning down some of the original features, not without some ‘known’ bugs in the release and definitely not before going way above the original budget.

What went wrong ? We did everything to prevent ‘failure’ but we still failed to deliver a decent product on-time, within budget (even if overtime is unpaid, it still means shooting out of the budget), blah, blah, blah …???

This is what went wrong – in order to safeguard our project from any (all?) possible failures, we fortified the project. We poured money like water to avoid being mousetrapped. We used innovative processes that allowed us to take baby steps and gain early successes. Those early successes, after all, helped us validate our assumptions and only then did we go all out. And yet? In this process of scaffolding the project, we actually erected artificial life-support systems to make the project stand on its feet. Most assumptions were then tested in isolation for its validity under ‘standard test conditions’. Unfortunately, those early successes gave a false illusion that everything was fine, even though several of the chinks in the armor were not fully exposed. Absence of ‘gentle failures’ early in the project lifecycle ensured that the team thought they were on rock-solid footing. However, in reality, we could never fully guarantee that, especially when those early cycles were done with the intent to make something work and not make something fail ! Imagine you were testing a tool to detect landmines. To do acceptance testing of the equipment, you operate it under ‘specified test conditions’ and accept it. However, the enemy won’t lay mines under those very standard test conditions! Chances are extremely high that enemy’s plan will fail your equipment. I am only asking you to fail it yourself before someone else does it for you.

Here is something you might want to do on your next project. Identify all possible and potential ‘points of failure’ in the project (there are always more than one). Challenge everything and everyplace where Mr. Murphy can strike. Design the project plan to fail, fail softly, fail early and fail as often as required to ensure there is no grand failure (or at least a significantly lower chances of a grand failure). Design your test criteria such that success is measured by how many of those assumptions have failed. Cull out important lessons from those failures and now build your project plan to avoid grand failures. Of course, you won’t completely eliminate grand failures, but will have moved a couple of steps closer to avoid them or minimize their impact compared to what an early success approach would have given you.

So…when are you planning to fail ?