Tag Archives: Waterfall

Project Management vs. Program Management

Program Management is often seen as the next logical step for seasoned project managers looking to take on bigger challenges. While project management is more about managing within boundaries of a project and gatekeeping it against anything and everything that threatens the status quo, program management is typically all about breaking those very boundaries and managing across them by taking up anything and everything that threatens the status quo. In this post, I will examine how they differ in its approach on two important aspects – scope management and people management.

Scope Management

Like I mentioned above, a project’s success depends on its ability to retain focus against all odds. Once a project scope is defined, its estimates made, resources  allocated and commitments made, the project manager is pretty much focused on gatekeeping everything else out of the scope lest the project success is threatened. In reality, most projects would have a CCB, or a Change Control Board, of some levels of formal authority, there is still a tendency to bulk up the entire requirements in first-pass and do as much as possible in one breath irrespective of how much time it takes and whether the final outcome is acceptable to the customer or not. While most of the traditional world still likes this model for various reasons, software development community identified this as a bottleneck and created the suite of so-called ‘agile methodologies’ that exploit software’s ability to incorporate late changes to specs without seriously endangering the project or its deliverables. Still, at a high level, a project must work around a reasonably stable set of requirements to ring-fence itself against any potential changes to the ‘core’ of a project – the premise being how can you build a successful product on a shaky and wobbly foundation. After all, don’t we pay product managers to do a better job of defining those requirements upfront rather than changing their mind later in the project and calling it as customer change request to cover up what they failed to think of in the first place! Surely, everyone understands changes in workflow or bells and whistles, but I am talking about the core architecture – the fundamental DNA of a product that must be understand before any further allocation of time, money or resources is made. So, clearly, scope is sacrosanct to a project.

A program is a different beast. As the highest level of body chartered to translate an organizational’s strategic intent to reality, it can’t box itself inside any boundaries of defined or undefined scope. Anything that could impact a program’s ability to accrue full ‘benefits’ envisaged from it must be taken up. While in theory, a program must have a defined scope to plan its activities and resources around its deliverables, in practice it is not so trivial. Any reasonably large program has sufficient number of moving parts, uncertainties, conflicting requirements and rapidly changing priorities. It is very typical in a program for the component project managers to carve out their pieces very sharply. A lesser reported fact of life is that developer’s motivation to work in newer and sexier technology often dictates the choice behind a project taking up (or refusing to take up) a given problem. Program organizational structure and governance plays a very important role in ensureing that component projects are not only cleanly defined, they also identify inter-group dependencies and secure commitments to address them. To that end, a project might safeguard itself by rejecting an inter-group request, but eventually the program needs to address it! Similarly, a project manager might complete the work within her boundaries but it takes much more for the program to be ‘done’, let alone be successful. I have seen many situations where project managers would be so focused on their project that they won’t recognize that unless the program was successful as a single entity, their individual progress was meaningless. This could get aggravated when teams are geographically or organizationally dispersed. However, none of those can hide or discount the fact that a program is only as  successful as it ability to influence things that might not be in its line of control but whose impact is definitely in a project’s line of success, especially if those things were to backfire.

While a project is like a fortress that must protect itself against all invasions to survive and eventually be successful, a program is more like a university, a rose garden, or a mission – they all deal in ‘soft power’ and maximize the ROI of their mission by keeping their doors open and by teaming up with their potential adversaries.

People Management

A project manager in a functional organization wields significant ‘positional power’ and thus has very high ability to influence team members behavior. While today’s organizations are highly flat and democratic, there is still an asymmetric balance of power, for the manager who writes your focal and annual salary revisions also potentially has the power to decide when you should update your resume! Surely we have come a long way in management-worker power-sharing from Taylor and Ford era, make no mistake that there is no such thing as a perfectly symmetric world where a manager and her team member have equal rights. Thus, a project manager has much higher ability to define the work and assign people to it as she feels appropriate. She also has a much higher level of responsibility towards training and career development of her team members, and being the closest face of management to the team, she is the official spokesperson of the upper management. A team will likely listen more to her than to the CEO. She must balance two opposing sets of expectations that, if not aligned properly, can set the project on fire. To that end, people management for a project management must be one of the toughest job.

In high contrast, a program manager manages at boundaries of participating organization in a highly matrix environment, and hence must manage by influence and not by any formal authority. In a software team, a program manager needs to get Development, QA, Product Management, Usability, Documentation, Marketing, and several other functions on the same page. In most organizations, they are organized along the functional lines and hence report to a solid-line manager in the same skill-set pool rather than program manager. Given that many of these resource might be timesharing on a program, their eventual loyalties are still with their respective line managers and hence a program manager can only rely on collaboration and influence as the key measures to get everyone onboarded. In some cases, there might even be a conflict between a component’s goals and the program’s goals and if such issues remain unresolved, the only recourse to make the program successful might be to move the problem up the command chain. Still, there is a big value in managing large and complex endeavors as a program, for it allows an organization to manage its resources and inter-group dependencies and conflicts in a more systemic and transparent manner. Some of the best program managers I have seen were not the ones who stopped managing the interfaces, but went over and above what their jobs required. They established a direct contact with key team members in component projects and created an alternate informal channel to validate project risks and plans, and to feel the pulse of the organization. They would do it very unintrusively without creating any friction between the line manager and them.

How does your organization view these two important functions?

 

Is Planning an old idea whose time is up?

In the age of nano attention-spans of people, the tendency and respect for planning things upfront has taken a serious beating. The mainstream logic is to simply “play by the ear” because there are far too many moving parts to be completely accounted for and properly factored-in – and in any case, by the time the plan goes to execution, ground realities would have changed beyond recognition, thereby rendering the plan completely useless by that time. In software development, an inaccurate predictive long-range model such as Waterfall has been replaced by more accurate adaptive short-range Agile methods that solve the line-of-sight problem but don’t address the original problem – that of planning a large project with its own share of uncertainties.

While none of those arguments might be wrong par se, we conveniently ignore the fact that that is the nature of the beast, and we need to understand that planning is not a single-pass 100% process that stays current forever. Further, just because things will eventually change is not a good enough reason to abandon any systematic efforts to understand it, if not to tame it altogether!

Pillar on a railway trackWhile preparing my lecture notes for a recent class on planning, I decided to explore the subject of planning through the ages using various quotations over a period of time. I sorted them in chronological order in order to understand how human mind has evolved the thought process behind planning. What came out from this exercise surprised me – it seems like the values associated with planning are as timeless as the fundamental human nature. The age-old wisdom was that a “stitch in time saves nine” and hence there was a focus that “prevention is better than cure”. Some might argue their world was not so complex and dynamic. I disagree.

There world was highly uncertain, and hence inherently complex. With perhaps 95% of reasons behind mortality unexplainable, with perhaps 95% of scientific events being attributed to the Gods (mostly of the angry variety!) and no protection from animals, predators and even their own tribal enemies, how much more complexity do you need in a day? When faced such levels of uncertainty, the modern man is more likely to live life one day at a time. And yet, the thought process from yesteryears seems to indicate that they didn’t simply abandon long-range planning, on the contrary, they reinforced the values of such long-range planning as the only real way to deal with such uncertainties!

Here are some of the quotations that I discovered during my research:

  • If you are planning for a year, sow rice; if you are planning for a decade, plant trees; if you are planning for a lifetime, educate people – Chinese Proverb
  • A man who does not plan long ahead will find trouble at his door – Confucius (551-479 BC)
  •  Plan for what is difficult while it is easy, do what is great while it is small. The difficult things in this world must be done while they are easy, the greatest things in the world must be done while they are still small. For this reason sages never do what is great, and this is why they achieve greatness – Sun Tzu, Chinese General, The Art of War, 400 BC
  •  Before beginning, plan carefully – Marcus Tulius Cicero, Roman statesman and orator (106-43 BC)
  • Long-range planning works best in the short term – Euripides, poet (480-406 AD)
  • Forewarned, forearmed; to be prepared is half the victory – Miguel de Cervantes Saavedra, Spanish writer and author of Don Quixote (1547-1616). 
  • By failing to prepare, you are preparing to fail – Benjamin Franklin
  • In preparing for battle I have always found that plans are useless, but planning is indispensable – Dwight D. Eisenhower, general and president (1890 – 1969)
  • A plan which succeeds is bold, one which fails is reckless – General Karl von Clauswitz
  • Always plan ahead. It wasn’t raining when Noah built the ark – Richard Cushing, novelist
  • Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought – Bernard Baruch, A  stock broker, advisor to presidents Woodrow Wilson, Harry S. Truman, (1870-1965)
  • Those who plan do better than those who do not plan even thou they rarely stick to their plan – Winston Churchill, British Prime Minister
  • In the space of two days I had evolved two plans, wholly distinct, both of which were equally feasible. The point I am trying to bring out is that one does not plan and then try to make circumstances fit those plans. One tries to make plans fit the circumstances – General George Patton (1947).
  • A good plan today is better that a perfect plan tomorrow – George S. Patton
  • Planning is a process of choosing among those many options. If we do not choose to plan, then we choose to have others plan for us – Richard I. Winwood
  • Plans are only good intentions unless they immediately degenerate into hard work – Peter Drucker
  • Prediction is difficult, especially about the future – Yogi Berra, baseball catcher (1925-present)

Another interesting discovery was that planning didn’t mean people simply fell in love with their original plans. On the contrary, there are instances that people advocated not only adapting to changes, but rather incorporating mechanism to accommodate them upfront. Look at some of the quotations on this:

 

  • It’s a bad plan that admits of no modification – Publilius Syrus, Roman slave and poet (circa 100 BC)
  • Observe always that everything is the result of change, and get used to thinking that there is nothing Nature loves so well as to change existing forms and make new ones of them – Marcus Aurelius, emperor of Rome (121-180 AD)
  • It is not the strongest of the species that survive, not the most intelligent, but the one most responsive to change – Charles Darwin, scientist

So, is the concept of ‘planning’ an overrated and old idea who time is up? I don’t think so. It only means that there is more need to refresh our understanding and explore how to evolve long-range planning methods to incorporate short-term changes such that agility is not lost at the tactical level, while near-sight changes don’t create a bullwhip effect on strategic plans.

Do you think inaccurate and uncertain proactive planning should be completely abandoned in favor of a highly accurate and definite reactive ‘planning’ (if that can still be called as ‘planning’, that is)?

Why are we in this mess?

Prior to the industrial age, the world was essentially an agrarian and a trading economy. Production methods were often a craft and top secret, fiercely protected within a family and handed down from a master craftsman to his sons, and with no machinery for mass production, pretty much every product was handmade and unique, perhaps also customized, for its intended user. Industrial revolution made mass production and rapid movement of goods possible, and among other things, catapulted Britain into forefront of global economies. Gutenburg’s printing press was perhaps the first mass production system built by man. Subsequent inventions like harnessing of steam power made railways possible, spinning machines, and other advances in iron founding and chemicals pushed the envelope. However, a lot of these advances were limited to Europe and even more within the UK, which thrived on these advances and became the economic (and imperial) superpower well until the start of twentieth century.

However, by the start of twentieth century, America too woke up to industrial advancements and contributed to some of the most important advancements that still continue to touch our lives. The pioneering work by Eli Whitney on ‘interchangeable parts’ on his now classic cotton gin introduced the concept of modular design, followed up by Frederick Winslow Taylor’s groundbreaking work on scientific management led to the concepts of standard work and division of labor  (even if somewhat questionable and controversial in today’s context) and created the foundation for Henry Ford to envisage a mass production system with a moving assembly line where finished goods could be assembled from standard parts by semi-trained operators, e.g. Ford’s most famous Model T car in black color (“Any customer can have a car painted any colour that he wants so long as it is black”).

In essense, a production run, say in an automative parts production or even a car assembly line, is the repetition of a process that produces similar (or similar-looking) objects. Once the ‘process’ is ‘designed’, the job entails repeating the process till the desired number of objects have been produced. Clearly, the faster you produce those objects, the sooner you can put them up in the market for sale and start getting money. The more you produce, the more you are able to amortize the capex, and get lower per unit price over the long run. Intuitively, if you have to produce objects that are exactly alike in properties, shape, size, color or any other physical attributes achieved by the production process, you can ensure that your production machinery will need to be ‘programmed’ once and re-used several times later. So, if you run the paint shop (which seems to be the largest bottleneck in terms of time in a modern production setup) and need to produce purple colored chasis, once you set the process, stock the paint to desired levels, you are pretty much all set. Now imagine if you have to produce first 200 cars in purple, then next 30 in wine red, then next 70 in pearl white colors. Surely there needs to be some way (manual or otherwise) to alter the production process to suit such job order. Similarly, instead of producting 300 sedans, if you have to produce a mix of automobiles – say, 50 SUVs, 50 compacts, 150 sedans and 50 hybrids, your process will have to be different from the one that just produces 300 sedans in a single production run. While the customer desires options (don’t we all?), the manufacturer incurs additional time, money and resources in creating such options. From the manufacturer’s point of view, producing each piece exactly similar as the last one makes such great economic sense that he can create huge economies of scale made possible by principles of mass production. It simplifies the machine (and machine operations) required in plants, it standardizes the components required, there is no downtime to alter the production process, people don’t have to be retrained every now and then on different type of products, and all this make the entire production process very ‘controllable’ from throughput and quality perspective, and hence highly predictable. Elaborate statistical charts can be created based on prior experience on how much time it takes for a given production run, how much men (and women) and materials are needed to meet a given production target, and what levels of quality can be achieved based on statistical experiences. 

After WWII, an economically-broken Japan trying to rebuild itself too threw its hat in the ring and set out to Europe and US to learn about principles of mass production. For reasons that are well-research and well-chronicled in books starting from “The Machine that Changed the World” by Womack et al, companies like Toyota created a brand-new way of mass production that focused on just-in-time production as opposed to utilizing the production machinary as the way to achieve economices of scale. But we are getting ahead of ourselves here a bit, so let’s just ignore it for a while (because the world would not ‘discover’ these techniques until late 80s or early 90s).

Around the same time, computer science was born, and the earliest of software started to be written. Writing software was an extremely hard job, what with enough complexities of huge (and very costly) machines. Software had to be written in the so-called low-level or machine language and required very high level of cognitive abilities. With large endeavors, software creation soon became a techno-managerial problem involving several dozens or even hundred of people. However, unlike the semi-trained operators in Ford’s factories, these were highly educated and perhaps the first generation of knowledge workers of the digital economy.

At that time, there were perhaps the two best methods to organize men and machines: military was where men were employed in thousands and mass production was where machines were deployed. So, what happened next was only the most logical way to extend the thinking: software was treated as a ‘production’ problem and the techniques of mass production were used to develop a ‘waterfall’ model that allowed for enough (?) time and effort at the start of a project to gather all requirements and do the ‘design’ and then rest of the development was contrued as the ‘production’ and hence principles of mass production could be deployed. To organize men, the traditional command and control model was pereived as the best way to separate decision-making executive from the heavy-lifting workers.

Today we debate and critisize these as the worst things that could have happened to software development, but I don’t want to be unfair to what was done more than five decades back based on the state of art knowledge, tools and practices back then. I am sure people back then also wanted to do the best thing – as do the people today claim :).

This was a short overview of how some of the important advancement had impact on thought process on software development (and there are many more important ones that have influenced our world). In the next article, we will discuss how it actually impacted the software development, both positively and negatively.

Does your project management methodology lets you free think?

Its not about the project management methodology anymore. Frankly, it never was, even though it has triggered off some of the most senseless wars in the history of project management. Starting with Frederick Winslow Taylor‘s Scientific Management to Henry Ford‘s assembly-line based mass production system and eventually landing in a flavor-of-the-day methodology (CMM, ISO, Agile, XP, Scrum, Lean, Kanban…and add your favorite one here), project management community, especially in software field, has seen it all…and still counting! All these project management methodologies have been eulogized as silver bullets in their heydays (and some still continue to be worshipped as we speak), and have subsequently been improved upon by the next wave of innovation driven by ever-evolving business needs, state of technology and the sociological changes at the workplace. However, each predecessor has been uncharitably rejected and unceremoniously relegated to trash by every successive methodology champs. However, that doesn’t seem to have stopped project woes, certainly not – going by the claims made in their marketing brochures :). So, whom are we to trust – the overzealous champs or their ever-evolving methodologies ?

For most practitioners, novices and experienced folks alike, project management methodology became this one large target to shoot at, the advertisement to get the project deal, the crutches to hold the project on to, the lame excuse against change in project specs, the insurance against failures, perhaps the raison d’tre for project managers ? “Sorry, the manual says do it this way, we can’t change that”.The process handbook says we can’t take any changes anymore – tell customers to wait until the next release which is just six months away”. “You are not approved to prototype, so stop that effort”. “Our company’s org structure doesn’t allow an engineer to manage the project – the risks are too high”. “Our metrics are within the control limits, so I don’t understand why engineers fear a project delay”. Goes without saying, they come in all hues.

Little did we realize that the “problem” was a moving target. We continued to evolve our solution blissfully unaware that the problem was also upgrading itself. Every new fix has led to a newer generation of problem that seems to have outpaced the development of solutions so far, and I see no good reason why we will ever have a perfect solution for every type of problem. So, it doesn’t amuse me when people on Agile / Scrum discussion boards try to indiscriminately apply those principles to just about every type of problem under the sun, and then when, predictably, things don’t work, they blame that Agile / Scrum is not being applied in its spirit. Have you ever seen a project manager so baptized that he won’t think beyond the book ? I think those blind preachers are just living like a frog in a well.

Try to imagine this scene pictorially in your mind: You are walking down the road with a map in hand. You see a puddle of water that’s not on the map. What do you do ? Agilists will have you believe that waterfallists will merrily walk through (or drown through, depending on depth of the puddle) whereas Agilists will “inspect and adapt” a sophisticated term for “trial and error” (which is, by no means, a bad term). So, they will step into the puddle a little every time, and based on the results of that, determine if they want to go ahead or change the course. I will ask you something: forget the books (its actually much better if you burn them literally), what will you do if no one told you anything. You will simply avoid it. I think humans are infinitely more capable of acting on their own, especially in matters of their own interest, that they don’t require anyone to tell them what to do. So, let’s not try to be their parents. Right? Or wrong?

In my small world, we are all grown-ups who have a right to view the problem with our own private lenses without owing anyone an iota of apology about the unique problems we alone face and struggle with, whether we understand or not. And we are free to blend any major or minor, known or unknown, new or old, and right or wrong methdologies to solve our own problems. As real-life practitioners solving non-glamorous, project management template-agnostic problems, we don’t need to display our camarederie and unflinching loyalty to some obscure methodology whatever anyone says. After all, they don’t sign your paychecks, but your customers do! Your solutions should only be aligned to what your customer wants – and not what a methodology wants!

So, it is simply not about methodology. If anything, its all about:

  • Refusing to get ringfenced by a process document, howsoever great that might be
  • Outrightly rejecting the idea that things can’t be improved any further
  • Optimizing things that can be done better than in the past when a given process was written
  • Eliminating deadwood from a process simply becuase something makes no sense
  • Embracing change because your problems were not designed keeping your process in mindIn fact, it is not even about a project anymore. If anything, it is about your fundamental right to free thinking. But then, my view should not come in your way of your free thinking :).

So, think again….does your project management methodology lets you free think?

Do you follow Project Management ‘religiously’ ?

Project Management is perhaps one of the most fiercely debated and grossly misunderstood disciplines in the software field currently, hence let me throw in a disclaimer first: if you are a small team of experts and/or people well-known to each other (e.g., have worked together as a team earlier), and situated in a collocated fashion, doing a lot of ‘creative work’ that can’t be very ‘accurately’ scoped, let alone managed; you probably will find ideas of formal project management a huge overkill (on time, effort, money and might even seem to stifle creativity), and you might be better off considering ‘lightweight’ methods like Agile Project Management / Scrum in the context of software development (well, nothing stops you from deploying pieces of Agile / Scrum in a non-software context – it is based on common sense after all). Small projects, small teams can afford to define a process that exploits the tailwind:

  • A small and collocated team means more direct interaction among team members and lesser management overheads in formalizing project communication (e.g., project status reporting, team meetings, etc.). This reduces the time it takes to collate, transmit and share important project information and also decreases the possibility of information distortion or confusion. A small team means there is high signal to noise ratio in team communications, is also leads to a better utilization of time spent in transmissing, receiving or digesting communication. Further, to schedule any event, one can always convene impromptu meetings around a whiteboard or around the coffee table without worrying about people’s already double-booked calendars or finding a place large enough to fit the team for the next meeting, and so on.
  • Having a group of experts means there is low(er) need for managerial supervision and people are generally ‘hand-picked’ for their knowledge, skills and abilities that could typically be ‘mutually exclusive, collectively exhaustive’. This ensures tight interdepency and very high mutual respect among team members. There is lesser ‘competition’ among team members and everyone knows that “united we stand, divided we fall” which fosters teambuilding.
  • For a small project, a large swing in any of the project constraints (time, cost, scope) could seriously and irretrievably impact the project in a downward spiral. If the work is new to the team, it could spell bigger trouble, especially if there is no way to do mid-course corrections. Executing projects in an iterative fashion helps shorten the feedback loop and give a chance to make necessary corrections to improve its ability to stay on-course and eventually hit the target. Additionally, if there is a customer involvement on these short iterations, the quality and value of these feedbacks could improve substantially lest any decisions are made that are hard to change if required.

However, if your project is anything more than a handful of people, takes more than a few months and entails issues like external dependencies with multiple vendors and ISVs (Independent Software Vendors), substantial technical and managerial risks, subcontracting / co-development, etc., you probably might improve chances of success using a formal project management methodology like PMBoK or PRINCE2 than without it. Agilists will argue and disagree with me that Agile / Scrum is not suited for large projects, but if you are new to Agile as well as to large projects, you might want to consider all options and associated risks – don’t believe in what worked for others, it might or might not work for you (remember the golden rule – we are all different), and your company might have a different set of constraints and tolerance to performance. 

The size, complexity, criticality, and risk profile of a project, and organizational appetite for project risk and uncertainty, and organizational culture would generally determine the level and need of management control required, and hence one must always right-size the project management methodology based on the context. Both, PMBoK / PRINCE2 and Agile methods evolved in response to challenges seen in executing projects successfully. While PMBoK / PRINCE2 took the approach that projects most often failed due to lack right levels of management oversight, planning, execution and control; Agillists felt the key reason was the fundamental nature of software being a ‘wicked problem‘ that just could not be managed using an obsolete waterfall model of developing software – it needed a better way to develop software in short iterations with very close-knit self-managed teams.

Without getting my loyalties needlessly divided betwen the two of these approaches, I believe there are merits in both the thoughts. I refuse to believe that any non-trivial project can be managed by piecemeal iterations that give no commitment whatsoever on the overall project performance, nor give any method to assess if the overall project is indeed on the right track (so, even if first few iterations are achieving the desired ‘veolcity’, what is the guarantee that subsequent iterations will also move at the same pace?). On the other hand, doing an iterative development is highly intiutive and very effective way to learn from past experiences to plan next steps, especilly on any non-routine project that is full of unknowns and assumptions. To me, these represent ‘mix-and-match’ ideas that one should pick up and apply wherever required, without getting baptized to the school of thought. I find most process wars have taken the disproportionate dimensions of falling just short of religious wars – you have to belong to one of the camps, and must talk evil about the other camp to be seen as a law-abiding corporate citizen of that camp. In life, we borrow ideas around us – our housing society, professional volunteer society we volunteer for, our kids playground, gardening, home improvement, nature, science, religion, other cultures…without necessarily converting to another faith or religion or society just because we like an idea that we want to borrow. So why should that be required in project management methods ???

I think this process war will be won by pragmatics. Those who take a position on extreme ends of the continuum and insist on always applying a particular style as the panacea might be in for a complete surprise because no two problems were created equal. A prudent approach is to evaluate all options and then decide which is the best response to a problem, as opposed to blindly follow a project management approach ‘religiously’.

 

Do you follow your project management aproach ‘religiously’ ?

So you think you are a top-notch Project Manager ?

Building Pyramids is one of the most common examples and metaphors in project management, and should I also add, perhaps one that is almost always discussed about by people who have practically no knowledge of construction engineering, let alone experience in building a pyramid (after all, I don’t know too many people who can claim to have built a pyramid). There is always a classic problem of what is the right way to build a pyramid: if you go by the much-maligned and clearly fallen out of favor ‘waterfall’ way, what happens if the King dies midway and the Pyramid is still unusable ? Worse, what if the King comes half-way down the project and sees bottom half of the pyramid and remains unimpressed with the sight and either wants to change the project specs, or worse, wants to stop the funding because he is not happy with what he is getting. OK, you want to build the top first. What if the King finds the explanation of building the Pyramid top-down funny (remember, there is no past precedence of your idea, and you might be ridiculed for your ideas – it is 2500 BC, after all) and all he sees on the ground is a miniature Pyramid ? What happens if a rival King starts constructing a bigger pyramid – can you now change the designs and make your pyramid bigger than the rival King’s ? Surely, there are obvious limiations of this model.

The other alternative is building it the ‘Agile’ way - starting with a miniature fully-functional pyramid and then incrementally building it inside-out. Theoretically, it sounds like the right thing to do – if and when King comes for inspection (and you can always count on that !), he always sees a ‘complete product’ albeit a miniature one. If he is impressed by the ‘plan’, he might release more funds and theoretically, the construction could go on perpetually, i.e., until when the King one days breathes his last. That day, right after the next ‘iteration’, his body could be laid in eternal peace in the majestic pyramid. However, there is a major problem with using Pyramid metaphor to justify Agile way of construction. Most of us who are simply ignorant about construction engineering treat the problem as building a Lego Pyramid – there is only the civil component of construction that is considered in any well-meaning debate. What about plumbing, what about electricity (ok, there was no electricity those days but you surely need air and light ducts to serve the purposes that electricity would have served otherwise), what about the state of technology that was available back then, what about stairways, what about interiors, and so on. Can all these be built inside-out ? Construction engineering might not entirely agree with building a Pyramid incrementally.

Well, there are critics on both sides of the thick and high wall that separates these two worlds (with some of them, perhaps, perched atop the wall as well) and there seem to be strong emotional views about which is better and why. In this post, I will not add anymore fuel to fire, but share an interesting thing that I came across. Chances are, irrespective of what camp you belong to, you will find this simluation exercise interesting, and guess what, you actually get to build a pyramid ! There is this great game on BBC site that tests how good a project manager you are – by playing a game to build The Great Pyramid ! Here is what the text reads:

Journey back four and a half thousand years to Egypt’s Old Kingdom, to the Pyramid Age.

As the vizier, or head of state, you are about to undertake the most important project of your career – the building of the king’s pyramid.

To succeed in this task, you must be a good all-rounder. Not only should you be able to motivate your workforce, but you must have good observational skills and the ability to steer a barge up the Nile, avoiding hippos and crocodiles.

Have you got what it takes to be a pyramid builder?

What I specially liked about the project is that it places emphasis on just about everything you require as a project manager to complete you project successfully – domain skills, planning and scheduling, executing skills and the most important of them all – the people issues, like what type of people to hire (and how many) and how to determine the reward strategy. This is not just a drawing board game – you actually get to ‘execute’ project (at least I got to execute a part of it, and I guess the game might allow you to do more if you pass the first stage).

If this sounds like fun, click here to take the Pyramid Challenge.

PS: I took it once and lost the game. My ‘project’ failed miserably :(. Hmmmm….

What is the Inventory of your Software Development project ?

 

Traditional software development follows a quintessential Waterfall model. Among its many limitations, originally discussed by Winston Royce in his 1970 classic paper “Managing the Development of Large Software Systems” and many more thereafter, it also forces a build-up of ‘work-in progress’, or inventory of unfinished work that has not yet been delivered to the customer, and even if it has been presented to the customer, it doesn’t add value until the final working software has been delivered. I define ‘value’ here as originally defined by Womack and Jones in their classic ‘Lean Thinking’, “…The critical starting point for lean thinking is value. Value can only be defined by the ultimate customer. And it’s only meaningful when expressed in terms of a specific product (a good or a service, and often both at once), which meets the customer’s needs at a specific price at a specific time”.  

 

Agile software development, as originally envisaged in the Agile Manifesto, aims to address several of the challenges faced in a Waterfall-type of software development by working on shorter delivery cycles using only a subset of requirements at a time. This subset of requirements (“Sprint Backlog” in Scrum) is prioritized by the customer or customer’s representative (“Product Owner”), so at the end of this short delivery cycle (“Sprint”), the customer gets ‘potentially shippable software’ having exactly those features that add ‘value’ to her. Frequent delivery of working software also has an additional impact on reducing ‘inventory’ levels in a software project, however, Agile doesn’t specifically address how is improves it over traditional software development. We can use concepts from Lean manufacturing to understand and explain inventory build-up in a software development, and how Agile practices help us manage them better compared to the traditional software development practices.

 

Little’s Law states that inventory in a process is the multiplication of throughput and the flow-time. In traditional manufacturing, there is a strong emphasis on plant capacity utilization as a core driver in cost management. However, a high plant capacity utilization requires (or rather leads to) high inventory to ensure the production doesn’t slow down for want of raw materials. High inventory in turn leads to a low inventory turnover, signifying poor sales, thus having high economic implications. Inventory is also identified as one of the seven wastes in Lean. I have discussed the concept of inventory in a typical manufacturing process and how Little’s Law can be used to analyze mathematical relationship between inventory, manufacturing lead time and cycle time in my article “Applying Little’s Law to Agile Project Management – part 1”.

 

In software development, inventory can be thought of as the development costs that continue to be incurred while the development activities are underway. Only when a successful delivery has been made and accepted by the customer, the development team can realize its developments costs, and free-up the ‘inventory’. Until such time, this ‘inventory’ accumulates and remains locked-up.

 

In traditional software development, there is no concept of interim releases to the customer. Some teams might provision for a proof of concept or a prototype, but most often, they are only meant to work under ‘test conditions’, meaning they are not fit for deployment in real world. At best, they give a feel of what it likely to come out of the development lab at the end of a generally long development cycle. In that long cycle, it is possible that some requirements become obsolete, or get clubbed with some other requirements, or just get de-prioritized. In effect, a ‘BRUF’ approach often might end up the team with a software built to original specs, but unfortunately for them, the customer’s view of the world has changed while they were busy developing the software in isolation. So, the ‘effort’ that doesn’t add to ‘value’ also becomes part of an already bloated inventory of development costs. Mary and Tom Poppendieck have discussed the subject of ‘inventory’ in software development in great details in their seminal work on Lean Software Development. They have taken a much wider view of the inventory, while I have limited my discussion in this article only to the development costs.

 

When we use Agile practices, we work on a prioritized subset of requirements for each increment, and at the end of each such increment, we deliver a working software (not a prototype) to the customer. The customer is in immediate position to test-drive the delivery and come back what any necessary changes that could be incorporated in the very next increment. At the end of each such successful increment that delivers ‘value’ to the customer, it is possible for the development teams to realize their development costs, thereby resetting the inventory levels to zero before starting the next increment. I have discussed this scenario taking a near-real-life example in my featured paper “Applying Little’s Law to Agile Project Management – part 2”.

 

Conclusions

Agile software development is a radical approach to improve productivity in software development teams while improving the value delivered to the customer by using shorter delivery cycles (and a lot of engineering practices, though not discussed in this article). Using Agile practices, it is possible for the development organization to create ‘value’ for the customer and also manage its own inventory levels. This is true for ISVs, or services companies working for a customer. It is also equally applicable for start-ups that often work for a prolonged gestation period in a ‘stealth mode’ and their only performance metric during that phase being the notorious ‘cash burn rate’. In today’s tough economic times, virtually every firm is trying to reduce its cash cycles, and Agile software development just provides one more financial incentive by reducing the inventory and shortening the cash cycle. 

 

 

 

 

What is your Software Development Religion today ? And where does the Customer fit in that ?

The Swiss Army manual says: When the map and the terrain disagree, trust the terrain. However, in software industry, there seems to be an unending effort to make sure the terrain is retrofitted so that it looks much more like the map in hand ! So, I have modified the Swiss Army saying to suit the reality in the software development community as: When the real-life and bookish process definition differs, it clearly shows you have not understood the bookish definition, and hence you must change the real-life until it looks like the bookish definition and then trust the bookish definition.

Statutory Health Warning: Reading this blog post further could be bad for health, especially for those who love the process vocabulary more than the process intent, see the means more clearly than the ends and anyday favor the established processes of the day even to solve newer class of problems that clearly require fresh thinking lest they end up shaking the establishment. Author takes no material responsibility for anyone proceeeding from this point beyond and suffering serious health problems because of the radical views presented in this blog.

The conventional definition, popular understanding and state of practice of Software Quality is seriously flawed. They have created a very lopsided perspective that adherence to certain standards is Software Quality, howsoever hard- or soft-baked those standards be. On one hand of the rather colorful spectrum are the die-hard process zealots who won’t stop short of anything less than an ISO, CMMi or the likes and on the other end of the spectrum are the neo-rebels who believe anything Waterfall is bad and unless anything is Agilized, it ain’t good enough. On the innumerable discussion boards that I subscribe and listen to, completely petrified to speak up lest be asked my credentials to back up my anti-establishment views, I find more productive hours being lost on what should be the exact definition of a ‘product owner’, what should an ‘iteration zero’ be better known as, and whether you pass the Nokia test or not. I find the neo-rebels falling in the same honeytrap that they once so detested and fought tooth and nail – compliance over creativity. I see more mail threads getting fatter and longer because there are linguistic differences that probably should be settled so that practitioner’s camp can have peace after all, but where is the Customer in all this ?

We are probably forgetting that software came first, the generic notion of Quality came much earlier and everything else is only a recent model that some wise men and women have put together – and is likely to change with time. In fact, if that doesn’t change, there is something seriously wrong. So, there is a shelf life associated with Waterfall, and there is a shelf life associated with Agile as well ! Granted that Waterfall has had a rather long-tail that simple refuses to die (much to the chagrin of agilists), and so is Agile likely to have, but the fact remains if the software community stops innovating at and after Agile, it is not just bad enough for Agile – it is bad enough for the entire industry for the wheels of innovation and continuous improvement would have come to a grinding halt. 

I believe quality is not about how much the goods or services that a manufacturer or a service provider produces confirms to requirements. They are also not about whether it is achieved using Waterfall or Agile, whether CMM or ISO, whether done in-house or oursourced, or qualified using random testing or automated testing. Quite frankly speaking, I couldn’t care less if it was done by a bunch of social misfits or comforming cousins as long as I get what I want. But in all the noise and the alphabet soup of neo-processes, I think “what customer wants” isn’t audible much these days.

So, what really is quality. Well, to me, Quality is THAT differentiator in a product or a service that

  • makes me drive a few extra miles just so that I could buy or experience something I really like even when other, relatively cheaper options are available nearby. (=sarcifice time, effort and comfort to get something I value)
  • makes me choose one over other even when, everything else being rather equal, the one I choose might be costlier but not exorbitantly priced. (= availability of other altarnatives, freedom and ability to choose what I want)
  • makes me patiently wait in a line for my turn to come (=sacrificing my comfort to get something that I believe it worth it)
  • makes me pick up a product blindfold (=blind trust, but not trust blindly; reliable everytime)
  • I can recommend to my friends and family (=what is good enough for me is good enough for people I care)”

When I view quality from this perspective, it becomes very easy to substantiate what I said earlier – who cares what process was used, where the software was written, which language was used and what metrics were collected as long as I got what I wanted ? When one is willing to take a customer-centric view of software quality like this, the choice between Waterfall or Agile is only a matter of what class of problem we are trying to solve and what is the best-proven technique to address that situation. There are no permanent ideologies nor permanent religions – you must be flexible to choose what suits the problem at hand, rather than view it from a fixed-focus lens and try to ‘retrofit’ the problem to your software development religion.

So, what is your Software Development religion today ? and where does the Customer fit in that ?

PS: This post happened in response to a question on my favorite Q&A forum – LinkedIn.