Tag Archives: Agile

Hard work is killing people. Literally!

Recently, I was reading an article on how Japanese-style manufacturing isn’t working out quite so well in India (despite having been successfully around since 80s), and that there have been several strifes lately. There was an interesting reference to what the Japanese call as Karoshi and Karojisatsu. Now, unlike many other Japanese words, these are really gross words because they mean

  • Karoshi = Death from Overwork
  • Karojisatsu = Suicide from Overwork (and stressful working conditions)

Curious to know more about it, I chanced upon this graphic from ILO page on workplace safety:

Karoshi and Karojisatsu cases in Japan, 1997-2011, ILO This data comes from Japan, and while the numbers might not (yet!) look staggering to many a folks, the trend is unquestionably disturbing, and our own deniability might only be getting compounded by lack of data from our own respective countries or industries, not to mention the social stigma that might be associated with someone refusing to ‘work hard’ on medical grounds lest they suffer an injury, suffer a burnout or commit suicide!

In the last few years, I have heard of small but increasing number of such sudden deaths among senior IT professionals – not just in Bangalore, but also globally. You can read about them here, here, here, and here.

Burnout is a serious issue for several countries, industries and people, even if we don’t acknowledge in as many words. In our industry, where heroism, cowboy programming and all-nighters are considered cool and an integral part of the software subculture, there has been a (really) small effort to address work-life balance by identifying that software development should be ably to proceed “indefinitely” on “sustainable pace” with XP explicitly advocating 40hrs a week (though some might disagree with this interpretation of “sustainable pace“). However, anyone who has ever done any non-trivial piece of software development will point out how hollow this expectation is in reality. When it is the release time, families learn to deal with their family members coming late or staying back office overnight, or working through the night even if home. Those in startup phase don’t have the luxury of ‘closing the day’, and those who are entrepreneurs actually thrive in such environment. So, is the thrill-seeking behavior at work here? Could it even be the case that we are only unknowingly aggravating the problem by hiring more people who think, talk and work like us – thereby creating a tarpit-kind of environment where there is no escaping it?

It is well-known in manufacturing that we never utilize machines more than 80-90% of their rated capacity (global stats on utilization are more like 80%). And yet, we don’t think twice before ‘loading’ human beings to much beyond their 100% capacity! Unfortunately, the data suggests that working more reduces productivity, as below:

Working hours vs. Productivity

Or, our ‘professionalism’ stops us from admitting that if I put out my case for a better work-life balance, I might be considered a loser and be considered unfavorably in appraisals, promotions and salary raises?  

So, how do you deal with it, or rather, ensure that you don’t get burnt out? Or, if you are a people manager or a leader, how do you ensure those supporting you are not getting burnt out?

Important to call out here that the old harmless joke “Hard work never killed anyone. But, why take chances?” suddenly doesn’t look so innocuous anymore. 

Worth thinking…

Let’s free up agile teams…

Agile in general, and Scrum in particular are all about self-organizing teams collocated for maximum face to face communication that improves agility and real-time collaboration. This was a great utopian idea in ~2000s because of three primary factors back then – economics, technology and psychology. Back then, we were still trapped in the enterprise-mindset – all collaboration happening within the confines of an organizational boundary – be it research or product development because it was much more cost-effective to lead such efforts internally either due to massive costs of R&D or closed IP protection or simply being the sole magnets for top talent.

However, as Henry Cherbrough’s Open Innovation has challenged, and Lafley’s Connect & Develop program at P&G and several others have subsequently demonstrated, there is perhaps more to gain from opening up meaningless and irrelevant organizational boundaries than protect false economics of a closed innovation system.

Globalization, even with all its ugly side effects, has shown us repeatedly in industry after industry that working across a global supply chain is not a zero-sum game after all! So, why are we so parochial in software industry about not recognizing the bigger economic sense rather than limiting ourselves to a singular idea that collocated teams are the best option?

Similarly, the technology to collaborate over the wire is dramatically more potent, even if underutilized in mainstream due to various reasons, than ever before. Collaboration over internet is the most natural thing for the millennials – back in 2007, a good 97% owned a computer, 94% owned a cellphone, a good 74% used instant messaging and 94% of those multitasked while messaging. In 2013, they are spending over 25hours every week online – be it social media or online shopping, or consuming infotainment or what have you. So, it is natural that this generation is extremely comfortable – even prefers – in being connected online and getting its job done. How will you make them sit in the same office day after day, and why? Little wonder that companies such as Automattic, MySQL, 37Signals and Github are starting out as massively distributed teams – not just on day one, but are continuing with their thought process of virtual workforce that still values teamwork but prefers newer ways to stay connected with the mothership – online instead of being their physically. As they say – the technology favors the geek!

And finally, the human psychology. The more specialized the talent, the rarer it is to find than ever before, and the best talent might not always be ready to work in your office even if they are willing to work for or with you. Just like people want democracy but not necessarily politicians, they might want to work with you on specific projects or technologies, but without being part of your office politics or hierarchy. They may not aspire to a long-term career growth with you but still value working with you on that one small problem that really fires them up. However, the very thought of being on your rolls and then moving the family to your location might not make sense to them, especially if you also happen to be in the similar business of delivering services online for your own customers (and who isn’t these days?). So, will you still ‘force’ your agile teams to be collocated inside a single mothership, or take a very large pragmatic leap of faith and explore oasis wherever they are, and build distributed teams around them? To me, a process is a reflection of the social values and norms of the time when they were written. If the world has moved on since ~2000, why should we not revaluate what should ‘being agile’ mean for this world?

Let’s free up agile teams…

 

...and he might be a 'free bird' and work remotely!

…and he might be a ‘free bird’ and work remotely!

I draw upon several ideas for my hypothesis. Authors Ori and Rod give a compelling vision of autonomous teams in “The Starfish and the Spider” – while spider teams are more vulnerable, the starfish teams could be autonomous and perhaps more effective in “VUCA” times. These thoughts are getting echoed in leaderless teams in Holacracy, and are well-grounded in Complex Adaptive Systems. There is a whole new movement of “Office Optional” new-age companies where they work from home or Starbucks, and even the people who probably show up in a physical office have to dial-in for a call from a meeting room so that everyone is equally ‘remote’.

How will the currently established mental models of agile teams – specifically on business people and developers working ‘together’ daily, which is invariably assumed and expected to mean ‘face to face’ – scale up in such a scenario? If the people are changing the way they think and interact, with or without agile, and the organizations are changing the way they conduct business, can we expect our processes to hold out for us? Think about it – when our customers expect work to happen at their convenience when they need it, they are asking for a business model that is distributed in time and space – and all that is already happening around us! So, how could the teams designing and developing such solutions not put themselves in the shoes of customers and internalize such learning environment?

I think there is a strong business case of freeing up agile teams from the physical boundaries of collocation because the collaboration methods are both – efficient and effective – and create compelling business case – even if only a minority at this point. I definitely see a wave building up at the horizon…

How do you design agile feedback?

Feedback is perhaps the most important aspect of the overall agile lifecycle – without a proper, honest and timely feedback, there is no ‘adapt’ step in the inspect-adapt cycle. The absence of such feedback only ensures there is no early opportunity to ‘respond to changes’ and teams will have no option but to simply keep ‘following the plan’ thereby violating a key agile value. Starting with the TDD loop to the CI systems, we are constantly seeking feedback on our outputs – in ever shortening feedback cycles as technologically possible. However designing a proper feedback instrument for a human-human interaction, like a training program, is a totally different thing because it entails imprecise measurements that are often influenced by people’s mental models, skills and experiences, and not to mention – their calendars! Needless to say, these feedbacks could mean anything to different people on different days.

Feedback needs to be agileIf the feedback required is too ‘wide and shallow’, it can be obtained very quickly but it won’t give enough actionable feedback. On the other hand, a ‘narrow and deep’ feedback could be more actionable but might take relatively more time, and it might also fail to register feedback outside its focus area (what if you were focusing on the wrong problem?). So, how does one go about designing feedbacks that enable agile learning? We call them agile feedbacks.

In Agile India 2014, I am presenting the experience report on this topic, and will our experience from designing agile feedbacks for agile trainings and workshops. The objective was to get most critical feedback in shortest amount of time to enable quick action planning. We created feedback that took a maximum of just 5 minutes per respondent, and enabled the most important learning in both, focused as well as open-ended manner that allowed us to focus on the most critical items. We employed elements of Design Thinking and Rapid Iterative Testing and Evaluation (RITE) to improve the process and quality of feedback themselves.

The experience report is accessible here.

How do you design agile feedback?

Why is your agile still a lot like dogma on steroids?

I still continue to be amazed (thought ‘shocked’ and ‘dumbfounded’ will be more appropriate words here) by the amount of dogma in agile circles. Do this! Don’t do this! Wasn’t agile meant to liberate us from the tyrannies of the so-called big monolithic non-agile white elephant processes, and create a more nimble mindset, flexible culture and adaptive process framework where ‘inspect and adapt’ was valued more than ‘dogma and prescription’? I sometimes think the poor old waterfall, for whatever it was worth (and I do believe it was worth a lot more than most people are willing to give it credit for!), was more open-ended and invited innovation simply because it was not perfect enough, and was very clumsily built for a rainy day, and depending on how sunny your day was, you were allowed (rather encouraged and expected) to pack only that much ration that you felt might be needed for the jungle trip.

For example –

  • You could have had any team size.
  • You could choose to locate your team members anywhere they wanted to be.
  • You could tailor your process in whatever way that made sense for you.
  • You could choose to slice your functionality in any which ways that made sense.
  • You could stagger the releases in whichever way you felt offered better yield.
  • and so on.

In several ways, such innovation required one to master the nuances of software development. And for those who would apportion reasons behind failure of such project on the method, my response would be – when there is so much flexibility in the system, why blame the system for being so ‘rigid’?

I have experimented more with waterfall methods in my career than with agile methods (which also I continue to do, much to purists chagrin :)), and here’s a small list of key exeriments that I remember doing – something that gave me immense joys because I had the liberty to try out stuff to see if that solved my problems better –

  • In 1995, when we realized that we are in a technologically evolving and complex domain (Asynchronous Transfer Mode switches), we didn’t build castles in air with the ‘non-negotiable’ waterfall-based product development process that the company had mandated, but decided to build an early prototype what would allow us to validate some key assumptions about our architecture. Yes, the company’s process didn’t support us, but yes, we broke the rules :).
  • In 1997, when we ‘discovered’ that standard waterfall won’t help us speed up the development cycle while we wait for the previous phase to complete, we didn’t blame the process for it, we simply ‘invented’ sashimi model and kept going.
  • In 1998, when conventional estimations didn’t work out in a domain that was completely new and unknown to us (digital set top boxes), I was not obligated to follow some obsolete standard process (though we were a CMM Level 5 company), but encouraged to try out estimations using complexity weights using methods like PROBE to mitigate the risks in estimations.
  • In the same project in 1998, when the project’s technology was new to us, I was able to home-brew and define a process with five increments that recognized the experimental nature of the problem we were solving and the learning curve of the team rather than sticking to a one-size single-release process.
  • During 2000 to 2003, I liberally experimented with waterfall methods to build teams that delivered large products in telecom and datacom domains with high success rates. At one time, I had 190+ engineers on a single product in my team organized around 14 parallel projects running on a common timeline and delivered on-time product in complex 3G Softswitch space. Yes, all in waterfall :). At that time, we were ranked last in global market. Today that company is global leader in that space, and I can proudly say some of our efforts were behind that turnaround.
  • In 2004-05, I experimented with our conventional enterprise service pack release model by liberally adding the weekly cadence from Gilb’s Evo process to create a weekly delivery model, and by accidentally stumbling on the concept of limiting work in progress to create one of the world’s first kanban implementations without knowing kanban – to be fair, it didn’t exist at that time – all without any prescriptions but just with a liberal dose of enthusiasm and undying spirit of experimentation.
  • …and the journey continues.

And what has been my take on the agile theory and practice? Not so open to experimentation or innovation. Sad, but true. Take some simple things for a start:

  • Agile methods recommend a small team size. That’s good common sense, and backed by scientific studies and acendotal data from ages, and is a generally good advise. What’s no so good is then we insist that agile teams can only be in a certain numerical range, and any team size more than that is blasphemy! In fact, my extreme view is that the best team size is what you have right now and not an ideal something from the literature, howsomuch backed by data that might be! In several ways, it is same as the ideal body weight – most of us will never have it, but what we have ‘here and now’ is the most important number to start with. So, why waste time over building an ideal team and lose all precious learning opportunities in that process? If my team has a true ‘inspect and adapt’ DNA, irrespective of where I start, I will get to the finish line. Somehow. Isn’t’ that more important and being truly agile rather than finding the perfect take-off point?
  • Take user stories. The notion of moving to user stories makes lot of sense give the constant pace of world around us. PRDs could never cope up with documenting such copious amount of details – and if anything, they would only succeed in documenting history of what customer wanted a year back! Now on one end we want our user stories to be ‘negotiable’ (from the acronym INVEST) so that we can create meaningful conversations between product owner and the development team. This again makes a lot of sense in an imperfect world where documenting every single requirement with its myriad corner conditions might be practically impossible, and has diminishing rewards beyond a point. So, if we can create a quick and cheap way to get started and have both, the process and the humility to listen to development team come up with more questions and options, then this premise holds very high promise. However, as a philosophy, something that is non-negotiable might not be so good in the same spectrum. For example, the Scrum process that we want them to follow must be non-negotiable. Why is that? If Fosbury had listened to the best way of high jumping, he might have never broken the proverbial sound barrier in high jumping.

Hey…what happened to the big promise of team being allowed to figure out its own ways and means? Once we ‘tell’ them, shouldn’t we step aside and let the team find its true north? Do I hear you mention ‘Shu-Ha-Ri’ thingy? Do yourself a favor – go and find a student (even better – try it on a second-grader) and then keep telling him/her that they are still a ‘Shu’ and hence must obediently listen to whatever you are telling them. They are supposed to follow your instructions to the hilt and not even think of wavering a bit. Good. Now take a deep look at their reaction. Count yourself lucky if they choose to ignore you and decide to move on, for there are far more violent ways they could have chosen to respond to such dogmas. In short – this is not the time and age for dogmas. Kingdoms, Colonies and Communism are all long dead. Accept it and change your own coaching methods, if you want to be counted.

To me, agile is a state of mind that tells me how to proceed in an imperfect world. Not to somehow make a ‘perfect’ world and then proceed. To me, a successful agile implementation is not about finding the perfect team + perfect process + perfect customer + perfect timeboxing + perfect sprint planning + perfect retrospectives + perfect product owner + perfect scrummaster + perfect

When does experience get ossified into dogma?

When does experience get ossified into dogma?

engineering practices + many more perfections = perfect landing. To me, a successful agile means starting with team that you have at hand, with the process that you have under the constraints you have, with the requirements that you have on a best effort basis and a many such real-world realities that works under a mindset of taking things one after other and improving the journey with the hope to get to the destination better than without such effort. Remember, we are being melioristic, not idealistic. We are being adaptive, not laying down pre-conditions for take-off. And in that pursuit, the most important guide for decision-making is our own judgment. Everything else is just that – everything else, and while it might work at times, it might not work at other times. So, like the Swiss Army manual that says – when there is a gap between map and the terrain, trust the terrain, go ahead boldly and experiment. In the worst case, you will lose some time and dollars, but if your DNA is built on the premise of self-improvement, you will quickly recover and eventually find your own path. If you are not able to ‘find’ it, you will ‘build’ it. Even better…

In many ways, there were no royal guards so zealously guarding waterfall model that made it sexy enough to be experimented with and experimented upon. On that same scale of flexibility, I don’t find agile methods sexy enough. It appears to be a lot like dogma on steroids. And I think that’s a serious problem.

Is your agile still a lot like dogma on steroids?

Why user stories make sense?

User story is a popular mechanism used by most agile methodologies to communicate user requirements. Actually, this is only partially true. User stories are meant to be a placeholder to initiate an early conversation between the product owner and the team on key hypotheses so that a quick validation of them could be done to refine/confirm/reject our understanding of what the customer really wants. To that end, user stories are not anything like the requirements of yesterday – they are not even remotely meant to be complete/comprehensive and so on. 

The reason user stories (and not ‘well-formed’ requirements, if that is ever possible) makes sense is because we human beings are not experts in following the written word, and are likely to misinterpret requirements even when they are explicitly written down, especially when the communication is a one-time event and not an ongoing process. (if you don’t believe this, just visit this interesting site http://www.cakewrecks.com/ that celebrates real-life examples of how something as simple as icing on the cake can go horribly wrong despite really simple and absolutely clear instructions!). And we are talking about building complex mission-critical systems that must operate 24/7 with top speed and efficiency. If only the written word could be consistently understood by the receiver as intended by its author… 

On the other hand, if there is a continuous two-way dialog between the product owner and the agile team, such purposeful brevity leads to curiosity which then leads to a constantly improving and better shared understanding that refines as the product gets developed incrementally and gets periodic user feedback. 

User stories reflect the reality that we might not have 100% clarity about the requirements on day one, but with constant dialog, we might be able to improve our shared understanding of customer's wants and needs.

User stories reflect the reality that we might not have 100% clarity about the requirements on day one, but with constant dialog, we might be able to improve our shared understanding of customer’s wants and needs.

The key benefit of user stories is that instead of waiting for weeks (even months!) to get a fully-bloated and rather unprioritized PRD which is supposed to have fully-baked requirements while the development team sits idle all this time, is that identifying highest priority user stories and using them  to develop a prioritized product backlog allows the teams to get started much earlier and start developing features which could then be put out in front of customers to get real feedback (as opposed to individual ‘opinions’ that might/not best guide us in an uncertain environment). This is especially important because in the past, when the world was little less complex and much more underserved. I purposefully use this term ‘underserved’ because it is not that people have suddenly become much more demanding about what they like or not, but were just told there were exactly three carmakers to choose from, or just two operating systems to choose from, and so on. However, with rapid advancement of computing paradigms and constantly lowering cost of ubiquitous communication devices, they suddenly have the opportunity to demand what they want, and hence classical ‘forecast’ models of requirements elicitation, design or production (at least in the manufacturing sense) don’t work so effectively as much as the newer ‘feedback’ driven models that allow for developing key hypotheses (and not hard requirements carved in stone) which could then be quickly and cheaply delivered as ‘done’ features so that customers could tell us if they like them or not. Based on such valuable customer feedback, the team could iterate to either refine the feature, or to pivot, as the case might be. In the past, this might take the entire product gestation cycle, by which time, many, if not most, things might have undergone significant changes, and the entire development costs being sunk by that time, not to mention complete loss of any window of opportunity.

As Facebook says – Done is better than perfect. User stories allows developing a small slice of the product without really perfecting the entire product, but facilitates the process of validated learning that eventually helps develop a product with much higher chances of meeting customer needs.

In today’s world, you probably might not have the luxury of having investors wait multiple years for an exit, or customer waiting several quarters for a perfect product when the competition is willing to serve them faster (even letting them lay their hands on an earlier version and give feedback thus making customers a co-creator of the product), or employees working month after month without getting any meaningful feedback from customers. And we know that absence of any feedback eventually leads to erosion of trust. Incremental product development could help you bridge such trust deficit by delivering highest value to customers early and periodically, and user stories might help you get your project a quickstart.

So, does agile really kill innovation?

In continuation of my earlier blog post on ‘Does Agile Kill Innovation?’, I had a great time moderating the panel discussion at Agile India 2013 with Henrik Kniberg, Owen Rogers, Sujatha Balakrishnan, Udayan Banerjee, Praful Pillay and Sudipta Lahiri. The panel discussion was literally the last program at the end of two long days of management conference – but despite that, we had 60-70 folks throughout the session.

Agile methods, with their rather short-term focus on achieving a ‘done’ feature seem to give away an impression that the eventual goal of a sprint is delivering such a full-baked feature. However, the new-age thought process seems to be more like ‘done is better than perfect‘. Question is, does agile implicitly become the rate-limiting step for your innovation process by self-imposing a stringent and non-negotiable completion criteria that possibly can’t be met by innovation-led ‘stories’ where the discovery and experimentation is generally a long-haul process and failure is such an acceptable outcome that we want to fail early and fail often!

We had some interesting perspectives ranging from ‘agile kills innovation’ to ‘agile accelerates innovation’ to ‘it depends’. Some of the ideas that emerged out of the panel discussion were:

  • agile, and scrum in particular, seems to be well-suited to the class of problems where a reasonably well-defined product backlog exists to begin with. To that end, there is some level of agreement on ‘what’ is to be done in the stacey’s matrix. For such problems, maybe ‘20% innovation’ user stories could be designed that allow implementing features with lesser uncertainty on ‘what’ and ‘how’ and leaving the uncertain aspects to the innovation user stories. This could ensure that the sprints are not losing steam just because of taking up innovation-led tasks, and neither are those tasks suffering in the anxiety of maintaining (and improving) team’s velocity.
  • for problems that are so radically new that don’t have any notion of a product backlog to begin with, perhaps ideas from lean startup (and design thinking) make more sense where the notion of ‘velocity’ is not so pronounced as compared to the ability quickly iterate through the build-measure-learn loop in shortest amount of time, and test various hypothesis around customer discovery or MVP, or any other moving part
Does your process let you think outside the box?

Does your process let you think outside the box?

  • one attendee from the session remarked that in their organization, they give a ‘break’ in-between the sprints to take up such innovation-led tasks, and it is working for them.
  • in organizations and teams that get too obsessed with meeting predictability goals and focus on velocity, there is a risk that teams don’t attempt big bets but just chipping away at the small bets. So, that version of agile is probably not going to lead to any radical or breakthrough or disruptive innovation, but more likely suitable for incremental innovation.
  • in services companies, the essence of business value is predictability of service, and hence there is some level of disconnect with the whole notion of experimenting and failing that might impede any innovation. To that end, achieving agility could be hurting innovation. However, there are always possibilities to innovate in such customer-facing processes, and one needs to deliver proof-points and earn the charter.

These perspectives made for some good conversation, questions and counterpoints. After all, what is a good panel discussion without some arguments and controversies!

So, does your agile really kill innovation? You tell…

Seeking submissions on New Product Development and Product Management in Agile world for #AgileIndia2012

Agile Alliance has announced Agile India 2012 in Bangalore, India in Feb 2012. First of all, Naresh deserves kudos for all his untiring efforts last several years bringing up and building a robust multi-city agile software community in India, and deserves all congratulations bringing the flagship Agile conference to India. As the Chair for Agile India 2012, we couldn’t have had anyone better than Naresh to lead this.

AgileIndia2012 comes to Bangalore, India in Feb 2012
AgileIndia2012 comes to Bangalore, India in Feb 2012

I had proposed the stage on Agile for New Product Development (NPD). Annu Augustine (@annua) proposed a stage on Product Management in Agile world. The organizing team proposed merging these two and we all agreed that was a great idea to create a stage on “New Product Development and Product Management in Agile world”. I believe the time is ripe to elevate agile thought process outside the software development team and apply it to the big picture – the end to end product development. We often see significant ‘productivity’ claims from software teams adopting agile practices. However, there are several other key functions involved in product development – product management, user design, marketing, sales, technical documentation, beta programs, manufacturing (at least for systems companies), and so on. If software team optimizes its software process, it is good. However, it is necessary but not sufficient to achieve end-to-end agility. If rest of the organization doesn’t change its thought process, we just end up moving the ‘constraint’ from software development to something else, and the true gains of agility might not get realized at the organization level.

In this stage, we are now seeking perspectives, experiences, insights and groundbreaking ideas from practitioners and thinkers on how they have applied the spirit of agility to create new products that have led to unprecedented and extraordinary market success compared to their previous conventional practices. Specifically, we are looking for proofpoints from the marketplace to demonstrate how software team’s agility was directly visible in business results. We want to know what you learnt from your experiences applying agility at every step of the new product development to create customer delight experiences. We want to know how did you manage to crash your time to market to deliver innovative solutions that delighted your customers, or made significant impact to their topline.

We are still working on setting up the infrastructure for submitting the ideas, but in true agile letter and spirit, we want to evolve this stage as we go. Write back to Annu or me to share your views and ideas so that we can co-evolve this stage. We want to dogfood the spirit of this session by incorporating your feedback to make this stage more useful to everyone. You can find more details on Agile India 2012 here.

Interest to submit a proposal? Click here.

Let’s get going…

Project Management vs. Program Management

Program Management is often seen as the next logical step for seasoned project managers looking to take on bigger challenges. While project management is more about managing within boundaries of a project and gatekeeping it against anything and everything that threatens the status quo, program management is typically all about breaking those very boundaries and managing across them by taking up anything and everything that threatens the status quo. In this post, I will examine how they differ in its approach on two important aspects – scope management and people management.

Scope Management

Like I mentioned above, a project’s success depends on its ability to retain focus against all odds. Once a project scope is defined, its estimates made, resources  allocated and commitments made, the project manager is pretty much focused on gatekeeping everything else out of the scope lest the project success is threatened. In reality, most projects would have a CCB, or a Change Control Board, of some levels of formal authority, there is still a tendency to bulk up the entire requirements in first-pass and do as much as possible in one breath irrespective of how much time it takes and whether the final outcome is acceptable to the customer or not. While most of the traditional world still likes this model for various reasons, software development community identified this as a bottleneck and created the suite of so-called ‘agile methodologies’ that exploit software’s ability to incorporate late changes to specs without seriously endangering the project or its deliverables. Still, at a high level, a project must work around a reasonably stable set of requirements to ring-fence itself against any potential changes to the ‘core’ of a project – the premise being how can you build a successful product on a shaky and wobbly foundation. After all, don’t we pay product managers to do a better job of defining those requirements upfront rather than changing their mind later in the project and calling it as customer change request to cover up what they failed to think of in the first place! Surely, everyone understands changes in workflow or bells and whistles, but I am talking about the core architecture – the fundamental DNA of a product that must be understand before any further allocation of time, money or resources is made. So, clearly, scope is sacrosanct to a project.

A program is a different beast. As the highest level of body chartered to translate an organizational’s strategic intent to reality, it can’t box itself inside any boundaries of defined or undefined scope. Anything that could impact a program’s ability to accrue full ‘benefits’ envisaged from it must be taken up. While in theory, a program must have a defined scope to plan its activities and resources around its deliverables, in practice it is not so trivial. Any reasonably large program has sufficient number of moving parts, uncertainties, conflicting requirements and rapidly changing priorities. It is very typical in a program for the component project managers to carve out their pieces very sharply. A lesser reported fact of life is that developer’s motivation to work in newer and sexier technology often dictates the choice behind a project taking up (or refusing to take up) a given problem. Program organizational structure and governance plays a very important role in ensureing that component projects are not only cleanly defined, they also identify inter-group dependencies and secure commitments to address them. To that end, a project might safeguard itself by rejecting an inter-group request, but eventually the program needs to address it! Similarly, a project manager might complete the work within her boundaries but it takes much more for the program to be ‘done’, let alone be successful. I have seen many situations where project managers would be so focused on their project that they won’t recognize that unless the program was successful as a single entity, their individual progress was meaningless. This could get aggravated when teams are geographically or organizationally dispersed. However, none of those can hide or discount the fact that a program is only as  successful as it ability to influence things that might not be in its line of control but whose impact is definitely in a project’s line of success, especially if those things were to backfire.

While a project is like a fortress that must protect itself against all invasions to survive and eventually be successful, a program is more like a university, a rose garden, or a mission – they all deal in ‘soft power’ and maximize the ROI of their mission by keeping their doors open and by teaming up with their potential adversaries.

People Management

A project manager in a functional organization wields significant ‘positional power’ and thus has very high ability to influence team members behavior. While today’s organizations are highly flat and democratic, there is still an asymmetric balance of power, for the manager who writes your focal and annual salary revisions also potentially has the power to decide when you should update your resume! Surely we have come a long way in management-worker power-sharing from Taylor and Ford era, make no mistake that there is no such thing as a perfectly symmetric world where a manager and her team member have equal rights. Thus, a project manager has much higher ability to define the work and assign people to it as she feels appropriate. She also has a much higher level of responsibility towards training and career development of her team members, and being the closest face of management to the team, she is the official spokesperson of the upper management. A team will likely listen more to her than to the CEO. She must balance two opposing sets of expectations that, if not aligned properly, can set the project on fire. To that end, people management for a project management must be one of the toughest job.

In high contrast, a program manager manages at boundaries of participating organization in a highly matrix environment, and hence must manage by influence and not by any formal authority. In a software team, a program manager needs to get Development, QA, Product Management, Usability, Documentation, Marketing, and several other functions on the same page. In most organizations, they are organized along the functional lines and hence report to a solid-line manager in the same skill-set pool rather than program manager. Given that many of these resource might be timesharing on a program, their eventual loyalties are still with their respective line managers and hence a program manager can only rely on collaboration and influence as the key measures to get everyone onboarded. In some cases, there might even be a conflict between a component’s goals and the program’s goals and if such issues remain unresolved, the only recourse to make the program successful might be to move the problem up the command chain. Still, there is a big value in managing large and complex endeavors as a program, for it allows an organization to manage its resources and inter-group dependencies and conflicts in a more systemic and transparent manner. Some of the best program managers I have seen were not the ones who stopped managing the interfaces, but went over and above what their jobs required. They established a direct contact with key team members in component projects and created an alternate informal channel to validate project risks and plans, and to feel the pulse of the organization. They would do it very unintrusively without creating any friction between the line manager and them.

How does your organization view these two important functions?

 

Is Planning an old idea whose time is up?

In the age of nano attention-spans of people, the tendency and respect for planning things upfront has taken a serious beating. The mainstream logic is to simply “play by the ear” because there are far too many moving parts to be completely accounted for and properly factored-in – and in any case, by the time the plan goes to execution, ground realities would have changed beyond recognition, thereby rendering the plan completely useless by that time. In software development, an inaccurate predictive long-range model such as Waterfall has been replaced by more accurate adaptive short-range Agile methods that solve the line-of-sight problem but don’t address the original problem – that of planning a large project with its own share of uncertainties.

While none of those arguments might be wrong par se, we conveniently ignore the fact that that is the nature of the beast, and we need to understand that planning is not a single-pass 100% process that stays current forever. Further, just because things will eventually change is not a good enough reason to abandon any systematic efforts to understand it, if not to tame it altogether!

Pillar on a railway trackWhile preparing my lecture notes for a recent class on planning, I decided to explore the subject of planning through the ages using various quotations over a period of time. I sorted them in chronological order in order to understand how human mind has evolved the thought process behind planning. What came out from this exercise surprised me – it seems like the values associated with planning are as timeless as the fundamental human nature. The age-old wisdom was that a “stitch in time saves nine” and hence there was a focus that “prevention is better than cure”. Some might argue their world was not so complex and dynamic. I disagree.

There world was highly uncertain, and hence inherently complex. With perhaps 95% of reasons behind mortality unexplainable, with perhaps 95% of scientific events being attributed to the Gods (mostly of the angry variety!) and no protection from animals, predators and even their own tribal enemies, how much more complexity do you need in a day? When faced such levels of uncertainty, the modern man is more likely to live life one day at a time. And yet, the thought process from yesteryears seems to indicate that they didn’t simply abandon long-range planning, on the contrary, they reinforced the values of such long-range planning as the only real way to deal with such uncertainties!

Here are some of the quotations that I discovered during my research:

  • If you are planning for a year, sow rice; if you are planning for a decade, plant trees; if you are planning for a lifetime, educate people – Chinese Proverb
  • A man who does not plan long ahead will find trouble at his door – Confucius (551-479 BC)
  •  Plan for what is difficult while it is easy, do what is great while it is small. The difficult things in this world must be done while they are easy, the greatest things in the world must be done while they are still small. For this reason sages never do what is great, and this is why they achieve greatness – Sun Tzu, Chinese General, The Art of War, 400 BC
  •  Before beginning, plan carefully – Marcus Tulius Cicero, Roman statesman and orator (106-43 BC)
  • Long-range planning works best in the short term – Euripides, poet (480-406 AD)
  • Forewarned, forearmed; to be prepared is half the victory – Miguel de Cervantes Saavedra, Spanish writer and author of Don Quixote (1547-1616). 
  • By failing to prepare, you are preparing to fail – Benjamin Franklin
  • In preparing for battle I have always found that plans are useless, but planning is indispensable – Dwight D. Eisenhower, general and president (1890 – 1969)
  • A plan which succeeds is bold, one which fails is reckless – General Karl von Clauswitz
  • Always plan ahead. It wasn’t raining when Noah built the ark – Richard Cushing, novelist
  • Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought – Bernard Baruch, A  stock broker, advisor to presidents Woodrow Wilson, Harry S. Truman, (1870-1965)
  • Those who plan do better than those who do not plan even thou they rarely stick to their plan – Winston Churchill, British Prime Minister
  • In the space of two days I had evolved two plans, wholly distinct, both of which were equally feasible. The point I am trying to bring out is that one does not plan and then try to make circumstances fit those plans. One tries to make plans fit the circumstances – General George Patton (1947).
  • A good plan today is better that a perfect plan tomorrow – George S. Patton
  • Planning is a process of choosing among those many options. If we do not choose to plan, then we choose to have others plan for us – Richard I. Winwood
  • Plans are only good intentions unless they immediately degenerate into hard work – Peter Drucker
  • Prediction is difficult, especially about the future – Yogi Berra, baseball catcher (1925-present)

Another interesting discovery was that planning didn’t mean people simply fell in love with their original plans. On the contrary, there are instances that people advocated not only adapting to changes, but rather incorporating mechanism to accommodate them upfront. Look at some of the quotations on this:

 

  • It’s a bad plan that admits of no modification – Publilius Syrus, Roman slave and poet (circa 100 BC)
  • Observe always that everything is the result of change, and get used to thinking that there is nothing Nature loves so well as to change existing forms and make new ones of them – Marcus Aurelius, emperor of Rome (121-180 AD)
  • It is not the strongest of the species that survive, not the most intelligent, but the one most responsive to change – Charles Darwin, scientist

So, is the concept of ‘planning’ an overrated and old idea who time is up? I don’t think so. It only means that there is more need to refresh our understanding and explore how to evolve long-range planning methods to incorporate short-term changes such that agility is not lost at the tactical level, while near-sight changes don’t create a bullwhip effect on strategic plans.

Do you think inaccurate and uncertain proactive planning should be completely abandoned in favor of a highly accurate and definite reactive ‘planning’ (if that can still be called as ‘planning’, that is)?

Your estimates or mine ?

For decades now, the project management world is divided between top-down estimation and bottom-up estimation. While a top-down approach might have limitations, it perhaps is the only way to get some meaningful estimates at the start of a project. A bottom-up approach might be a great way to get more accurate and reliable estimates but you might have to wait for a problem to be whittled down to that small a level to get such reliable estimates. Both are required for a comprehensive and useful project planning, but unfortunately, most people see one over other, and the Agilists abandoning top-down in favor of the highly accurate but low-lookahead bottom-up methods.

A top-down estimation is a great way to abstract the problem without worrying about its nitty-gritties, and come up with a workable estimates when there is no other source of getting any better estimates. At the start of a project, when making a realistic WBS is a remote possibility, the only way one could get some estimates is by top-down estimation techniques such as

  • Expert Judgment
  • Wideband Delphi Method
  • Analogous Estimation
  • Parametric Estimation

Estimates represent a range of possibilities

 

They allow an ‘order of magnitude’ estimate to be made available to set the ball rolling. We know that such estimates suffer from cone of uncertainty, but it is better to get imperfect estimate now than to get perfect estimates much later in the project – which we still need to get – the point is that we also need to get some estimates here and now!

Another key reason why we must use such top-down techniques at the start is because if we straightaway jumped into bottom-up estimation, we might face challenges because

  • Details about requirements are still emerging, and hence likely to get refined as our understanding about the problem gets better
  • Requirements might keep changing, thereby changing the scope of work all the time
  • Too many low-level details could introduce too many moving parts in the scope of work, thereby forcing one to skip “vital few” and start focusing on “trivial many” clearly not a prudent approach at that stage

So, we need to accept the ground conditions and accept those high-level estimates with a boulder of salt and keep moving. As we get deeper into the project, we decompose the problem better and have a more granular WBS that helps us zoom into the problem and get more refined estimates. At that point, a bottom-up approach is much more relevant, as it allows us to plan and track the work much better.

Agile methodologies clearly favor bottom-up estimation methods and the idea is to slice down every task on the project into ‘stories‘ that could be ideally done by an individual team members within a couple of days. It helps you to win daily battles but takes your sight off the big war that you must win to eventually succed. We can’t really scale up these stories for large projects even if we assume that every product scope could actually be broken down to meaningful stories of that grain size. So yes, in theory, that’s great. But the real world is not obligated to obey the theory!

A prudent approach is to adopt rolling-wave planning – start with big grain size when details are not very clear and gradually move down to small grain size as you get closer to the task. However, you don’t just abandon the overall planning simply because you are now doing task-level estimates and planning! On the contrary, it is even more important to keep track of dozens of small-task estimates becuase, to quote the great Fred Brooks in The Mythical Man-Month, it is not the tornadoes but the termites that eat up your project’s schedule! So, a delay of one day here and one day there repeated for a couple of tasks over next few sprints and you suddently are sitting with a month-long delay on your overall project. You might be hitting each timebox but that’s one problem with timeboxes – it could give you a false sense of comfort that all is well, when clearly you are being forced to jetsam things that you are not able to accomplish within a timebox. On the other hand, if you go by a task-driven schedule, you atleast have a clear idea of how much late you are to complete that task.

A frequent criticism of top-down estimates is that they are at best a wishful thinking, and at worst a highly unrealistic estimate forced upon a team against their free will. Again, we should be careful in such judgment because the reality is not always what is presented to us, but rather what we do with it! So, if you take a top-down estimate as the word of god and make it ultra-resistant to changes, than you have yourself to blame when those estimates don’t reflect the reality anymore. They should be taken as a map and should be continuously refined from the data coming from the terrain. To that end, top-down estimates and bottom-up estimates play a complmentary role and an effective project manager blends them together. A good top-down estimate covers the ground fairly well, thereby allowing realistic bottom-up estimates to happen (by allocating them sufficient, meaning as close to the actuals, time and resource available), and good bottom-up estimates at task level help reinforce assumptions made in the top-down estimates of the overall project.

So, at the end of the day, it is not about which estimation technique is superior. It is about using most appropriate tool for every type of problem.

Next time, think again before you ask the question your estimates or mine?

32 Reasons Why People Don’t Plan Projects

The concept of planning has a very intuitive appeal, irrespective of the type and size of endeavor. Who wouldn’t want to undertake an activity without a little bit of upfront planning? Will you take your car and just head out for the next family weekend in the neighboring town – without first checking if your car is roadworthy for the long drive, has enough gas, or needs a visit to the garage before the long drive? Will you buy a house without first checking your cash flows and other commitments for the next couple of years? Will you get your kids admitted to the nearest school you come across, or you will do little bit of investigation and plan is based on your specific needs and child’s comfort?

Still then, it comes as a great surprise when many among us disregard the benefits of planning things upfront. While they might still succeed in a trivial pursuit, for it might present very predictable outcome, require relatively risk-free approach, is generally easy to recover from a bad position, have virtually all resources at one’s disposal, etc. While such trivial ‘projects’ might not demand a rigorous planning, we should also note that often such planning is a very intuitive process, one that happens very effortlessly and unconsciously in our minds. Such planning, or several of its elements, have been executed so many times that one doesn’t have to think of anything else – it just happens. To an untrained eye, it might appear like a careless approach where nothing is being planned, and yet the actors seem to be in supreme control – it is nothing short of magic! However, let the vision not get the better of logic.

However, for any non-trivial problem, or a trivial problem with a few key changes, or some last-minute unpleasant surprises, the life is not that easy. They transform into wild beasts that don’t obey their masters anymore, they behave randomly (or at least unpredictably), and unless you address them proactively, they are only more likely to make things increasingly difficult for you. What follows then is the endless loop of firefighting (often leading to ‘immortal projects’ that I conceptualised in my earlier blog post “From Project Immortality to Project Moksha“). So, why is it that some people refuse to plan their projects, choosing instead to clean up their mess over long evenings and frustrating weekends over the next several months? Here are some of the reasons that I have come across:  

  1. Our project is too large to plan
  2. Our project is too small to plan
  3. Planning is waste of time and effort – I’d rather be coding right now!
  4. Planning commits me to something that I must stick to at all times, even when the world around me has changed.
  5. Publishing the plan will expose buffers in the plan to our customers
  6. Publishing the plan will expose buffers in the plan to project team members
  7. If we publish our plan, customer will question our productivity being low
  8. The project is evolving and the situation is changing on a daily basis
  9. We are too deep fire-fighting the project – have no time to plan at this point
  10. Why plan things that are too simple, and as for things that are too complex – well, you anyway can’t manage them, so why plan?
  11. We believe in adaptive processes rather than a predictive planning
  12. Planning involves tracking which means supervision of a person’s time. We think that creates mistrust in the team.
  13. We do burn-down chart which is better for projecting the delivery date
  14. The sub-contractor is responsible for project plan
  15. This is just a simple porting work
  16. Next 2-3 months we will be doing real research. There is no way I can manage that effort to a plan!
  17. Plans are for the weak-hearted, the brave hearts face the tornado head-on
  18. We typically estimate and pad it by 30% to make it a realistic plan
  19. We can’t estimate number of bugs we will get in the testing phase, and hence can’t plan for those phases
  20. Someone or other is always leaving our team or the company. No use building plans around them.
  21. All real work happens in nights and over weekends. Plans have always failed us.
  22. Our customer only cares for working software
  23. Requirements are always changing
  24. Customer has given us fixed end-date, so our plan doesn’t count
  25. My management will always underprovision team resources
  26. Doing estimation won’t speed up the project – it will take the same time even without an estimation
  27. We do Agile
  28. We do Scrum
  29. Plans are anyway changing all the time
  30. I have the project plan – here it is in the gantt chart
  31. Our project is diferent – planning will stifle the innovation and creativity
  32. Any version 1.0 product will be like that only!

You might have seen variations of these, or brand-new excuses. Feel free to share them here. We need to deglamorize these lame excuses for they serve no one’s interests. At best, they handcuff us into existing methodology, and at worst, they ensure project immortality (discussed above). Both are fatal for your project’s success.

What’s your excuse not to plan your project ?

Our methodology is 100% pure, our result is another thing!

What is worse then an anarchy ? You might say that is the absolute abyss, but I think blind allegiance is even more dangerous (and that includes following the letter but tweaking the spirit – things like ‘creative accounting‘ or its parallels in every field). Anarchy at least allows for things to become ‘better’ in order to survive – whether it is the idealogy, resistance, or even musclepower, or any other ills (and hopefully at some point, social forces of constructive destruction take over). But in a land where unquestionable compliance and blind allegiance rule the roost, IMNSHO, is like a terminal patient off the ventilator support. When people are on their deathbed, they don’t regret things that they did but much rather the things they did not do!

In project management, life is no less colorful. We have process pundits (read “prescription police”) shouting from the rooftop with a megaphone on how heavens will strike them bone dead with lighting if they ever as much as strayed from the ‘standards’. When projects are being postmortemed, we don’t often ask what or why the project did something that they did, but why they did not do things that they did not do. And quite often, you find answer in the map itself – because the map did not factor-in those conditions that were actually encountered on the terrain, the blind followers just followed the Pied Piper and danced their way into the river of death. What a terrible waste of human talent.

Why do we get stuck with methods so much that our result-orientation takes a back seat ? I think there might be many answers, but some that deserve merit:

We fall in love with ‘our’ methods

Let’s face it – introducing a new methodology in an organization, or a team, is not easy. People have their favorites, and it already is a fairly serious political game to get such groups to a decision (whether or not there is a consensus). After you have sold the executive sponsor to go for a particular methodology, you have to now somehow sell to middle-managers and the teams – without their support, you are toast. Sometimes you might have legitimate power (“position power”) to thrust the decision down their unwilling throats, but most often that doesn’t work anymore (even in jails!). So, you have to use your charm offensive to somehow make the other stakeholders believe that this methodology is the god’s greatest gift to mankind, and how it will take their next project to unprecendented glories. If only that were half-true! What follows next is a qunitessentially Bollywood storyline – the project is in tatters and the project manager and teams are exhausted and some even ready to bail-out. You go and put some more pressure on them, ask for weekly reports to be given on daily basis and take away even more productive hours into endless meetings to discuss yesterday’s weather. In short, we do everything but realize that our methods might be wrong…perhaps…for a change? Most people don’t get it, but sometimes, the smarter ones do get it. But by then, they are so much neck-deep in love with their methods that they can’t afford to take most obvious and the only correct decision – take a U turn, and admit their mistakes. Our professional ego checkmates us. Falling in a bad relation is bad, staying in it even after realizing one’s folly is worse.

We believe blindly in marketing claims

These days, there are dozens of marketing gurus, each extolling the virtues of their methods. In a way, it is like entering a village market – each vendor selling the same portfolio of vegetables, but each has to ‘differentiate’ from his neighboring vendor, and hence some make wild performance claims, quite often unbacked by any reliable data, and almost always trusted by gullible buyers at face value. So, some of us just go by the tall claims in glossy marketing brochures. You don’t believe me – tell me, how else you can explain those knowledgeable investors getting duped by Madoff’s elaborate Ponzi scheme in this age and day! Tell me how most banks fell for easy money in the subprime market. And if we were to assume, for a moment, that everything these marketing brochures said was indeed true, how come rest of the world had not quite figured it out already ? I think intelligence is not the elite preserve of a learned few – the rest of us lesser mortals also deserve to be treated with some professional respect that we can figure out what works and what doesn’t. In a previous blog, I once did an interesting analysis of one such marketing claim Blame your flaccid developers and your flaccid customers for your poor quality products !.

We are are too scared to experiment and take sensible risks

This is a real classic. Many organizations, certainly many more than we think there are, are so stuck in ‘command and control’ that they create a system where compliance is rewarded and creativity is shunned. This might work well in some industries or companies, but certainly doesn’t work for most of us. A beauty about such companies is that such outright disdain to new ideas might not just start and end at the top – depending on how deep-rooted the indoctrination is, it might be running in every employee at all levels. In the zest to display unflinching loyalty and to project oneself as a corporate citizen second to none, many employees (and many managers) simply abort new ideas because that might threaten the ‘status quo’ and makes them look very bad in front of the powers that be.

Because everyone else is doing it

This is groupthink at its best. Just because every Tom, Dick and Harry about town is doing it, I must also adopt (simply continue following mindlessly) the latest management fad. For example, we have all heard of (often unverified) stories how investors in the bay area during the dotcom boom era would only look at business plans that had an India component. Just because everyone is doing offshoring might not be good enough why I should also do it. Similarly, just because everyone I know is into Agile, does that make a strong case for my business also ? I think we should stop and think before succumbing to trade journals that routinely publish such forecasts and doomsday prophecies.

We are looking for easy cookbook solutions

Let’s accept it – some people are just plain lazy, or just too risk averse to do any meaningful research on what works for them. Instead of investing in figuring out what’s best for them, they are looking for some quick wins, some jumpstart, some fast food, some room service, some instant karma. They believe they can learn from other’s mistakes (which is definitely a great way to learn – definitely a lot better than not learning from anyone else’s mistakes!) and sometimes they might be successful, if they have copied the right recipe, but very often, that only results in wrong meal to wrong people at wrong time. What people don’t realize is that every problem is cast in a different mold, and whatever one says, there simply is no way one could airlift a solution and drop on the face of a second problem – in its totality – and expect the solution to work. Similarly, there is no way a cookboo solution might work. For example, Agile was designed around small collocated teams that have so high communication among itself that trust replaces formal contracts and communication replaces documentation – I mean they thrive on such a high-energy environment. But, sadly, simply to prove that Agile can fix any problem in the world, management consultants have stretched (should I say ‘diluted’) those very Agile princinples and they now try to forcefit those methods on a distributed, large team, that often has a heterogeneous ‘salad bowl’ culture than a homogeneous ‘melting pot’. So, you land in the situation that just because it worked for some random cases, some among us just naively believe it could work for our random case too. Does it get any more smarter than this 🙂

We believe compliance leads to repeatable success

Standards are often treated like insurance against catastrophes – both the termites and the tornadoes types (think of tornadoes as something that just comes out of the left field and kills a project overnight – like a meteor hits a neighborhood, and think of termites as slow and steady erosion that goes on and on and goes undetected until the time the damage is complete, comprehensive and beyong damage control). They ensure that no one deviates from the beaten track and tries something adventerous (without getting rapped on the knuckles), because that could lead to unpredictable results. And we all look for standards and certifications – ISO-this and ISO-that, CMM and CMMI (and the sunsequent obsession with all other 5-level models), Scrum, PMP, PRINCE2, MBA, PhD, IEEE, …so much so that even in the field of creative arts, there are schools that specify what creativity is allowed and what is out! There are success formulas for Bollywood movies – rich girl meets poor boy and the anti-social forces strike boy’s family. Eventually, our hero saves the day. Similarly, in Hollywood movies, it has to be the hero saving the nation from external threat (very often, coming from the space). In software development, ISO championed the cause of non-negotiable compliance and blind CMM practitioners only perfected it. Agilists were born out of the promise to create a world free of compliance, but it seems they have also ended up growing their tribes with their own mini-rules that give an instant gratification by useless things like Scrum Test to massage one’s ego that my Scrum is more Agile than your Scrum! Do, if you score a certain score in the Scrum Test, you are almost ‘guaranteed’ to get some x level of productivity improvement! Does that remind you of yesteryears, or does it make the future look more scarier?

Conclusion

Human behavior never ceases to amaze. For every one rational thinker, the world has to produce thousands of blind followers. Our schools and colleges teach us to learn the knowledge but they don’t always teach us how to convert it into wisdom. So, when we reach workplace, we are deeply apprehensive of trying out new stuff. We are excellent followers, but simply shudder at the mere thought of questioning status quo. We often behave like the monkey whose hand is stuck in the cookie-jar but refuses to release the cookies even when it knows that the only way he can extricate his hand out of the jar is without cookies. When those workplaces ignore result-orientation and only worship the compliance, the story only gets murkier. Think of a state where compliance is handsomely rewarded and questioning it seen as full and frontal attack, and its timid citizens are only too happy to oblige. They think a lifetime of blind obedience to methodology is far more superior than a moment of experimentation, even if leads to bad results.

After all…our methodology is 100% pure, our result is another thing!

(Inspired by a slogan on a tee: Our vodka is 90% pure, our reputation is another thing. very inspiring, indeed :))

Does your project management methodology lets you free think?

Its not about the project management methodology anymore. Frankly, it never was, even though it has triggered off some of the most senseless wars in the history of project management. Starting with Frederick Winslow Taylor‘s Scientific Management to Henry Ford‘s assembly-line based mass production system and eventually landing in a flavor-of-the-day methodology (CMM, ISO, Agile, XP, Scrum, Lean, Kanban…and add your favorite one here), project management community, especially in software field, has seen it all…and still counting! All these project management methodologies have been eulogized as silver bullets in their heydays (and some still continue to be worshipped as we speak), and have subsequently been improved upon by the next wave of innovation driven by ever-evolving business needs, state of technology and the sociological changes at the workplace. However, each predecessor has been uncharitably rejected and unceremoniously relegated to trash by every successive methodology champs. However, that doesn’t seem to have stopped project woes, certainly not – going by the claims made in their marketing brochures :). So, whom are we to trust – the overzealous champs or their ever-evolving methodologies ?

For most practitioners, novices and experienced folks alike, project management methodology became this one large target to shoot at, the advertisement to get the project deal, the crutches to hold the project on to, the lame excuse against change in project specs, the insurance against failures, perhaps the raison d’tre for project managers ? “Sorry, the manual says do it this way, we can’t change that”.The process handbook says we can’t take any changes anymore – tell customers to wait until the next release which is just six months away”. “You are not approved to prototype, so stop that effort”. “Our company’s org structure doesn’t allow an engineer to manage the project – the risks are too high”. “Our metrics are within the control limits, so I don’t understand why engineers fear a project delay”. Goes without saying, they come in all hues.

Little did we realize that the “problem” was a moving target. We continued to evolve our solution blissfully unaware that the problem was also upgrading itself. Every new fix has led to a newer generation of problem that seems to have outpaced the development of solutions so far, and I see no good reason why we will ever have a perfect solution for every type of problem. So, it doesn’t amuse me when people on Agile / Scrum discussion boards try to indiscriminately apply those principles to just about every type of problem under the sun, and then when, predictably, things don’t work, they blame that Agile / Scrum is not being applied in its spirit. Have you ever seen a project manager so baptized that he won’t think beyond the book ? I think those blind preachers are just living like a frog in a well.

Try to imagine this scene pictorially in your mind: You are walking down the road with a map in hand. You see a puddle of water that’s not on the map. What do you do ? Agilists will have you believe that waterfallists will merrily walk through (or drown through, depending on depth of the puddle) whereas Agilists will “inspect and adapt” a sophisticated term for “trial and error” (which is, by no means, a bad term). So, they will step into the puddle a little every time, and based on the results of that, determine if they want to go ahead or change the course. I will ask you something: forget the books (its actually much better if you burn them literally), what will you do if no one told you anything. You will simply avoid it. I think humans are infinitely more capable of acting on their own, especially in matters of their own interest, that they don’t require anyone to tell them what to do. So, let’s not try to be their parents. Right? Or wrong?

In my small world, we are all grown-ups who have a right to view the problem with our own private lenses without owing anyone an iota of apology about the unique problems we alone face and struggle with, whether we understand or not. And we are free to blend any major or minor, known or unknown, new or old, and right or wrong methdologies to solve our own problems. As real-life practitioners solving non-glamorous, project management template-agnostic problems, we don’t need to display our camarederie and unflinching loyalty to some obscure methodology whatever anyone says. After all, they don’t sign your paychecks, but your customers do! Your solutions should only be aligned to what your customer wants – and not what a methodology wants!

So, it is simply not about methodology. If anything, its all about:

  • Refusing to get ringfenced by a process document, howsoever great that might be
  • Outrightly rejecting the idea that things can’t be improved any further
  • Optimizing things that can be done better than in the past when a given process was written
  • Eliminating deadwood from a process simply becuase something makes no sense
  • Embracing change because your problems were not designed keeping your process in mindIn fact, it is not even about a project anymore. If anything, it is about your fundamental right to free thinking. But then, my view should not come in your way of your free thinking :).

So, think again….does your project management methodology lets you free think?

Codifying Agile Skills or creating more checklists ?

Did you check out the Agile Skills Projects yet ? It seems to be a new and interesting initiative to “… establish a common baseline of the skills an Agile developer needs to have, including a shared vocabulary and understanding of fundamental practices”.

They talk about Agile Skills Matrix that has seven essential skills, or the Seven Pillars, organized into five skill levels.

Seven Pillars include:

  1. Product: Agile teams must share a vision of what they are working to achieve; the business problem they are trying to solve.  When a developer understands the problem domain, they can help the Product Owner evaluate upcoming features.  Understanding of the product is the first step to creating a fully cross-functional team.

  2. Collaboration: Teamwork is the heart of Agile software development.  A truly collaborative team shares all its information freely.  They share their individual knowledge and skills by working closely with their teammates.  Teammates who fully rely on one another become much more effective as a whole, yet simultaneously increase individual skill levels.

  3. Business Value: The purpose of any software development effort is to create features that deliver business value.  A good Agile developer will focus on delivering that value.  They do not add complexity just to make their program ‘cooler’.  Instead they will base their work on business priority.  They produce a steady stream of running, tested software with real business value.  They will build only what is needed to solve today’s problem, knowing that building too little or too much is waste. 

  4. Supportive Culture: Any highly productive enterprise is founded in learning. Practitioners in an effective agile team view everything they do as an opportunity to learn. Every step is an experiment intended to make real progress, and to clear our vision of what to do next. We accept and embrace the fact that when a task is done, we’ll see better what we should have done. That helps us see what to do next.

  5. Confidence: Those on an Agile project strive to know the state of their code and the state of their project.  They do not share their work until they can prove it functions properly and is well designed and implemented.  They report progress based upon the actual rate at which fully tested features of real business value are being created.

  6. Technical Excellence: Developers understand, and choose from, many possible technical ways to satisfy business needs–choices that reflect a craft that balances design, use, and support.  They provide the technical underpinnings that enable us always to move forward at a steady pace.  They do this using principles of truly simple design, combined with a grasp of technical debt and the means to keep it under control. They use the best techniques for keeping the design under control without excessive work or rework.

  7. Self Improvement: An Agile team member seeks new ways of doing things, while keeping existing skills as sharp as possible.  They know that ours is an ever changing world and they strive to be prepared to take advantage of anything new.  They are both introspective and aware of what’s going on around them, be it in the team, or the larger business context. They will take action to fix things that aren’t right, and to help those who are in trouble. 

And the skill levels are:

  1. Learning: Individuals at this level have been exposed to a skill, but do not yet have firsthand experience with it.  This may come through reading or conversation, through a formal class or casual presentation at a local user’s group. A training course should provide at least this level of attainment.

  2. Practitioner: At this level, one has practiced the skill in a ‘safe’ environment.  They may have taken a course with hands-on exercises, or recreated the examples from a book.  The Novice level represents the first big step from abstract to concrete.  A wise organization will not let a Practitioner loose on their own.  They do not yet know what they do not know. They do not yet know what they do not know how to do. A training course with a sufficient hands-on component could provide this level of attainment.

  3. Journeyman: A Journeyman is known to possess the specific skill.  They have demonstrated a practical knowledge of the skill in various environments.  They can work on their own, or to increase the competence of a team.  A Team member of a lower skill level will learn from them, a team member of a higher skill level will recognize their expertise. One cannot be taught to be a Journeyman, one can only learn it.  Regardless of the skill category, maintaining Journeyman status, one must practice their current techniques and seek out new and better ones.

  4. Master: A Master must possess unquestioned competence in the particular skill.  They know it cold; it is second nature to them. But, it does not stop there.  To be accepted as a Master, they must also accept the additional job of bringing up the skill level of their team mates.  They do this in several ways. They serve as an example of proper practice.  If you want to know how it is done, observe a Master.  They partner with other team members and actively share their knowledge and experience. They seek out teachable moments.  When asked a question, they prefer to engage in a dialog that will lead the questioner to an answer, rather than making a pronouncement.

  5. Contributor: To be a Contributor, one must have added to our community’s understanding of how to practice our craft.   Contributors bring new ideas to light.  They develop and evaluate new techniques.  They are the silverbacks and young turks, who advance the state of the art.  They may be seen as today’s heretics.

I think this is a good effort to break-down the craft into little more tangible traits that developers need to acquire, thus making the body of knowledge available, one step at a time. Further, it is also refreshing to see things other than technical excellence being part of the Seven Pillars. I have long believed that Sociology should be part of the computer science curriculum now that software development is a serious team effort, and eventual success being largely influenced by human dynamics in the team than individual programming effort alone, howsoever supreme and important that might be.

However, all good things have a shelf life, and all good ideas become fertile grounds for next set of consulting and assessment industry to take-off (and Agile Skills Project makes no secret of its desire to support any effort to certify developers, even though they won’t officially endorse nor condemn any particular certification). I suspect it won’t be long before people will be certified on this (yet another!) 5-level scale of individual proficiency. Look at how successfully we have reduced Scrum to a series of checklist items in the so-called Nokia Test. Then there is this perpetual debate raging in the Agile / Scrum community on merits and demerits of certification, with everyone sort of nodding their heads that certification can’t guarantee competence, but even then everyone continues to offer some form of certification. I even suspect this might soon find its way into job descriptions. However, my bigger question is: will having a bunch of certified developers on your next Agile team ensure, or at least improve chances of success ? Are we unwittingly creating a class-hierarchy in Agile teams where developers will have to be of a certain class (or caste, if you like that way) to be part of the team ? How does one judge things like ‘Confidence’ or ‘Self Improvement’ ? Does high rating guarantee domain competence, or portability of skills? What is better in a 5-people team: 5 Masters, or 3 Masters and 2 Practitioners? Does someone at the skill level ‘Learning’ stand a chance to be in the team of ‘Contributors’ without adversely impacting the team productivity? Further, this is by no means an exhaustive checklist of critical success factors, and hence, chances are that people will fill up some checklists and declare themselves some 86% compliant (on some arbitrary and controversial scale) and their managers will expect the next project to be a resounding success. However, you and I know better than that :).

I think these are highly contentious issues that could further create more checklists and thus degenerate a good intent into a big mess.

The best way to use this tool will be to self-use it and set personal targets to acquire higher levels of competency as one goes by. Any effort to use this as a yardstick to measure and compare individuals is, IMHO, against the basic ethos of a human endeavor, let alone software development. So, shun the checklist-driven software development. 

Addressing the issue of “social loafing” in large teams

 Large teams might be inevitable in certain large endeavors, but there are several benefits of small teams. A small team can build and maintain a strong culture and a character that gets better with time. Small teams quickly learn the invaluable skills in teamwork and interdependence that lead to higher efficiencies while ensuring that individual team members don’t end up competing against each other but rather collaborate on the common objectives. Small teams also mean small egos 🙂

One of the biggest motivations of making smaller teams is to provide higher levels of transparency and task accountability to individual team members. A large team tends to hide inefficiencies, both of its structure and of its people. One particular problem in a large team is the problem of “social loafing” – something that is perhaps best described in this poem by Charles Osgood:


There was a most important job that needed to be done,
And no reason not to do it, there was absolutely none.
But in vital matters such as this, the thing you have to ask
Is who exactly will it be who’ll carry out the task?

Anybody could have told you that Everybody knew
That this was something Somebody would surely have to do.
Nobody was unwilling; Anybody had the ability.
But Nobody believed that it was their responsibility.

It seemed to be a job that Anybody could have done,
If Anybody thought he was supposed to be the one.
But since Everybody recognized that Anybody could,
Everybody took for granted that Somebody would.

But Nobody told anybody that we are aware of,
That he would be in charge of seeing it was taken care of.
And Nobody took it on himself to follow through,
And do what Everybody thought that Somebody would do.

When what Everybody needed so did not get done at all,
Everybody was complaining that somebody dropped the ball.
Anybody then could see it was an awful crying shame,
And Everybody looked around for Somebody to blame.

Somebody should have done the job
And Everybody should have,
But in the end Nobody did
What Anybody could have.

This is a great description of how so many ‘obvious’ things don’t get done – either due to miscommunication, or misunderstanding, wrong assumptions, or sometimes just shirking away from the responsibility. One of my best personal examples is working for a community team as a volunteer. I believe working for a community as a volunteer is the greatest way to hone one’s teamwork – if you can get people who are not motivated by money or power or promotions to work together for a job, you can do anything! So, I was part of this team of a really nice bunch of 15-odd people who was required to serve this 400+ families. This so-called executive committee was required to plan social events for the community. What I found was that 80% of the people on this team were there just for the meaningless social prestige. Over 90% of the work was done by just one individual (and I was doing another 5% and rest of the entire team doing the remaining 5%). Those 80% of the people were otherwise regular nice people, but when part of this large team, they could not be counted upon to deliver the goods. We organized some wonderful events, but it was mostly the two or three of us who did maximum running around and the rest of the team just travelling First Class. After a few months, I was ready to quit (I eventually quit that team at the next team election. What I find is that the new executive committee team has very similar effort distribution – so it was clearly not me who was an aberration :)).

In my professional experience, I have been involved in some really large software teams, up to 190+ people on a single product. While these efforts were large and simply required those many number of people, we used small teams, not more than 7-9 people each, to divide and manage the work. Each of these programs was divided into such a number of small project teams, each a self-contained and autonomous unit that could deliver its functionality with minimum external dependency. Small teams have smaller number of communication paths, and allow fostering of meaningful teamwork rather than poisionous politics. It also is a great way to groom technical leadership and managerial expertise in the teams. A large team is no fun for people to volunteer and train for roles and tasks that require building special skills. But a small team must often replicate several skills in each team, and hence is a great way to groom future leadership apart from also acting as a derisking strategy to counter impact of attrition. Agile practices advocate small teams for achieving high team throughput, and the issue of social loafing is indirectly addressed by things like daily scrum meetings where social shame and the feeling of letting down the team ‘forces’ team members to get their act together. An excellent discussion of social loafing can be found here.

Conclusions

Social Loafing is a real team dysfunction not restricted to a given country, culture, society, industry or team size. It has been observed in all types of societies and all kinds of groups. Making the team size small is one of the ways to address social loafing. In the context of software teams, everything depends on how individuals make commitments and live up to them. And hence, it becomes extremely important for a manager to be aware of this team dysfunction and evolve strategies to deal with it. Having a small team with clear roles and responsibilities, and setting common standards for work evaluation are some of the ways that can reduce the extent of social loafing in teams and improve morale and team productivity.