Category Archives: Software Quality

Software trends survey…

This is an interesting update on software trends:

A recent survey of more than 6,000 senior-level business leaders and software development executives found optimism for higher IT budgets and a preference for outsourcing, agile methods, and enterprise applications. In the survey, sponsored by software consulting firm SoftServe, 60 percent of respondents reported increases in 2009 IT development budgets, despite the uncertain global economic climate. Some 26 percent indicated that budgets had increased by more than 10 percent over 2008. Some 38 percent said they use some type of software development outsourcing. Of those, 67 percent used locations in India, followed by the emergence of Ukraine, China, and other Eastern European countries.

Nearly three-quarters (71 percent) listed new product or software development as a top priority, while cost- and expense-cutting followed with 51 percent, and improving usability ranked third with 49 percent. Forty-two percent favored agile methodologies as their chosen development model, with only 18 percent preferrng the waterfall method. About one-third (36 percent) employed Capability Maturity Model Integration (CMMI) as their process maturity and quality model with Six Sigma used by 25 percent. Some 62 percent of respondents were focused on enterprise applications, and 51 percent on Web-based applications.

Based on this survey, these are my observations:

  • Investments in IT budgets continue to rise, despite economic conditions. I guess the fact that any serious IT development is a long-led-time endeavor, the motivation might be to sit in the labs and work on getting stuff out just when the market comes up.
  • 67% are using India as outsourcing destination, which is a significant number, what will all predictions of anti-outsoucing sentiments, or a shrinking cost arbitrage, or low design skills in low-cost destinations such as India. They all can’t be wrong at the same time. I think hard numbers are almost always a stronger evidence.
  • CMMI is (still !) not dead! If one-third of industry uses CMMI, clearly there still must be benefits of it, and I think the non-believers might be well-advised to stop harping on CMMI as being a so 20th-centurish heavyweight, beaurecratic and document-intensive process and start learning how it adds value to the the compaies that still believe in it.
  • 42% favoring Agile methods is a good news, and I guess the 18% favoring Waterfall is the last post of resistance that will unfortunately still continue to wiggle its severed tail for many many years to come. I am more interested to know how these 42% of companies / professionals use Agile and how effective it is for them. If the only reason they are doing Agile is because that is the new ‘in thing’, and they have no clue how effective it is to them, I would argue that those 18% Waterfall loyalists are perhaps better than those neo-Agilists becuase they probably have a strong enough reason to cling on the their bad Waterwall ways – one of them could even be that it works for them! After all, these is no such thing as one single right way to Software Nirvana, and no one single method can claim to have monopoly over all the best practices 🙂
  • I could not understand how some 25% were using Six Sigma in the same breath as 36% were using CMMI as their process maturity and quality model. I think Six Sigma is not a ‘model’ in the same sense as CMMI (or even Agile for that matter). Secondly, if you are a Motorola making cellphones (perhaps not anymore), then six sigma production means less than 3.4 defects per million. If you are using CMMI for software development, you know that you are using a set of staircased practices (at each maturity level) that guide you on ‘what to do’ at each step. I can also understand using Six Sigma principles and methods to measure and statistically management processes like a peer review or a testing process, but I don’t undersand how Six Sigma qualifies for a full-lifecycle quality model in the same league as a CMMI. Or, am I completely missing the point???
  • I am surprised to find no mention of Lean, unless the Agile umbrella includes all shades of original Agile methods, as well as Post-Agile methods.

All surveys are sampling exercise and must be discounted for its inherent flaws, design constraints and execution issues. Despite this, it is always interesting to analyse trends over a period of time, as they indicate a movement and help us assess the wind direction and speed.

How Lean thinking improves productivity in software teams?

At its core, productivity for a software team (often wrongly termed as programming productivity because software development is much more than mere programming) looks like a great idea. In its simplest form, it compares output of the team (amount of useful and usable software created, amount of unnecessary rework done, number of defects produced, etc.) to the input (time, effort and resources invested in producing software) factored-in by the uniqueness of the given software  endeavor and other external environmental factors (complexity of the software being produced, impact due to team sizes, nature of team spread and its familiarity with the problem at hand, problem domain, and so on). Unless you are willing to discount software development as a non-business critical activity undertaken purely for the labor of love that should not be ‘measured’ lest it dilutes the very ethos of software development as a creative cognitive endeavor, one might agree, albeit reluctantly, that some measure of how well we are doing is clearly in order. Can you think of a single business activity where such a measure should not be done?

In last few decades, researchers and statisticians have tried to create quantitative measures of productivity. However, due to an often unilaterally percieved inherent ‘uniqueness’ of every software endeavor, practitioners have consistently rejected such measures contending that there are either far too many assumptions in its definition and far too many variations in practice, or both. Instead of focusing on the ‘vital few’ commonalities, it is only unfortunate that practitioners have virtually rejected productivity measures in all forms by considering the ‘trivial many’ differences. What could have gained the industry by considering a big-grain definition of productivity was unfortunately lost because of focusing on decimal-digit precision of productivity measures.

In this blog, I will discuss the subject of productivity from a conceptual plane, and explore how Lean thinking helps us understand it little better by establishing a correlation between wastes in software development and productivity of the team. Lean thinking is much more than identification and elimination of the seven types of wastes, but I will only take up the issue of wastes in this blog.

In the simplest form, we can call a process or an activity productive if it creates an output higher than its input. Though the input and output are often in two different and inequitable currencies, we are still mostly able to compare them. What that means at a conceptual level is that:

  • If Output < Input, we can say that the productivity is low
  • If Output > Input, we can say that the productuvuty is high
  • If rework is high, we can say that the productivity is low
  • If rework is low, we can say that the productivity is high 

These are highly generic views of productivity that most of us understand, but unfortunately they are not very usable to understand the factors that impact an output to become low, or the rework to become high. Developers might give one view, and testers might give another view as to why the output is low. There is no common vocabulary that can explain at a fundamental level what is it that is causing the productivity to dip. Context-specific variations (like differences in programming environment, percieved differences in complexities, developer competency, tester efficiency, etc.) might be too many to be usable to form any meaningful conclusions.

Lean thinking offers a more specific understanding into the wastes that typically make a process costlier (in terms of time or effort or both) than what it ought to be. Lean identifies seven types of wastes in a manufacturing setup. These wastes are known to introduce avoidable wastes, and Toyota pioneered a production system and an organizational culture that seeks to sysmetically eliminate root causes behind these wastes. Software practitioners, led by Mary and Tom Poppendieck, have investigated borrowing from Lean thinking into software development in last couple of years. Let’s examine how Lean encourages eliminations of following seven types of wastes in software development, and how they individually impact productivity of a team:

  1. Overproduction: Implementing a feature is an irreversible decision that takes lots of time and effort from the product engineering team, and delivery of a feature must wait until the next release cycle (whether the big-bang release in conventional software development, or the next real shippable release in agile development). If there are extra features in a software that remain unused because they are not of high priority but the engineering team is anyway spending effort to develop and test it, it is probably lowering the productivity of the team that could have used its effort elsewhere to develop other important features. The customer is also not benefiting from such overproduction of features, because he not only must wait for the desired features to be delivered, he must also pay the delivery team for those unimportant features that are not being used.
  2. Inventory: Anything that has not yet been delivered to the customer is work-in-progress (WIP) or inventory. The higher the inventory in a system, the higher is the number of resources locked-up and unavailable for any other activity. Even the work that has been completed but still sitting on the shelf is an inventory that is not creating any tangible value either to the team or the customer. If there are too many requirements yet to be implemented, it involves doing requirements analysis, design, development and testing without any of that effort being delivered to the customer, and hence increasing the work in progress and thus lowering the productivity at a business level. Even if a critical feature has beem completed, but it waiting simply because several other not-so-important features must be included alongwith in the next release, it is also adding to inventory because that important feature is not immediately delivered to the customer.
  3. Extra Processing: If the software is implemented in a complex manner that requires extra processing (e.g., a design that requires more storage, or duplicates data or code needlessly, etc.), it is either increasing the effort required to develop and test the software, or increasing the usability complexity, or both. Either ways, it is lowering the productivity. However, the opposite of extra processing is not smart programming, which might solve the problem in short run, but create maintenance problems in the long run.
  4. Motion: If the information is not readily available to the engineer, he/she has to spend unnecessary and avoidable time and effort that it takes to fetch the required information (which could anything – clarification about requirements, latest design specs, or locating the correct module from another team for integration, etc.), thus effectively lowering the productivity.
  5. Defects: Any defect that is passed on to the customers requires the engineering team to once again open up the code and spend effort to fix the problem. Since no new output is generated in this process, and the only output is fixing the functionality that should not have been broken in the first place, the net productivity over the entire product engineering lifecycle gets impacted adversely.
  6. Waiting: Perhaps no other project resource is as irrecoverable as lost time. Brooke’s Law reminds us how difficult, if not impossible, it is to recover delay from a project. Waiting for any resource for any reason creates unpredictability in software development apart from creating activities of zero or low output, which again reduces the productivity.
  7. Transportation: Handoffs are perhaps a Fordism legacy to the waterfall development where a team of business analysts would complete their work in isolation and handover the specs to dev team. Dev teams in turn would work in isolation and pass on the software to test team for testing, and so on. If all these teams are working in silos, such practice of working in isolation would often result in information distortion (or loss) down the stream, which could result in missed requirements, misunderstood requirements, unclear design, and so on. All these will eventually require extra (and unplanned) effort to implement the right functionality or fix the broken functionality, thus lowering net productivity over the entire engineering lifecycle.

 

As we can see, analyzing wastes in software development using the Lean thinking provides us a technology-independent and function-neutral view of the inefficiencies of the software process that can be used to achieve substantial improvements in productivity of the teams. None of these ‘attack’ an individual for lower productivity, and only look at the systemic deficiencies that must be eliminated for a team to perform at higher level.  However, of what possible business value would be the process of elimination of such wastes if we can’t picture ‘before’ and ‘after’ ? It is extremely important to assess the state before and after introducing a change – how else do we know if our efforts are yielding the desired results ? So, measuring the productivity in some absolute numbers is like the feedback whether an improvement is happening in the right direction (or not).

Lean thinking or not, do you measure productivity of your software teams? If yes, what for, and if not, why not?

Blame your “flaccid developers” and your “flaccid customers” for your poor quality products !

This is the text from a recent announcement for a course by Ken Schwaber on “Flaccid Scrum – A New Pandemic?” (text underlining is mine):

Scrum has been a very widely adopted Agile process, used for managing such complex work as systems development and development of product releases. When waterfall is no longer in place, however, a lot of long standing habits and dysfunctions have come to light. This is particularly true with Scrum, because transparency is emphasized in Scrum projects.

Some of the dysfunctions include poor quality product and completely inadequate development practices and infrastructure. These arose because the effects of them couldn’t be seen very clearly in a waterfall project. In a Scrum project, the impact of poor quality caused by inadequate practices and tooling are seen in every Sprint.

The primary habits that hinder us are flaccid developers and flaccid customers who believe in magic, as in:

Unskilled developers – most developers working in a team are unable to build an increment of product within an iteration. They are unfamiliar with modern engineering and quality practices, and they don’t have an infrastructure supportive of these practices.

Ignorant customer – most customers are still used to throwing a book of  requirements over the wall to development and wait for the slips to start occurring, all the time adding the inevitable and unavoidable changes.

Belief in magic – most customers and managers still believe that if they want something badly enough and pressure developers enough to do it, that it will happen. They don’t understand that the pressure valve is quality and long term product sustainability and viability.

Have you seen these problems? Is your company “tailoring” Scrum to death? Let Ken respond to your issues and questions!

Ken will describe how Scrum addresses these problems and will give us a preview of plans for the future of the Scrum certification efforts.

Here are my observations and thoughts from this synopsis:

  • line 2: what does “when waterfall is no longer in place” mean ? So, when waterwall was still in place, there were no issues ??? Somehow, one gets a feeling all the problems came to light only when Waterfall was “no longer in place”…so, why not get waterfall back in its place 🙂
  • line 5-6: In a Scrum project, the impact of poor quality caused by inadequate practices and tooling are seen in every Sprint ? …in every Sprint ?….now, wait a minute….I thought  Scrum project did not have any of these problems because there were no inadequate practices and tooling issues. And you certainly don’t expect to find such issues in every Sprint, or do you ?
  • line 7: OK, so now we have an official reason: “flaccid developers” and “flaccid customers” for all ills of the modern world. Wow! I am not sure if that is the best way to build trust either with teams or with customers by fingerpointing and squarely blaming them…without giving them a chance to even speak for themselves. And I thought Srcum was cool about trusting developers, not fixing blame on individuals, interacting with customers…but flacid developers and flaccid customer ??? Does it get any better than this ?
  • OK, I will probably agree ‘unskilled developers’ in the context of building an increment of product in an iteration, but what the hell is an ‘ignorant customer’ ? Try telling a customer that you are an ignorant customer…Scrum or no Scrum, you are guaranteed some unsurprising results ! And which Customer paying through his nose waits for the “slips to start occurring” ? In these tough economic times ?  
  • Is your company ‘tailoring’ Scrum to death ? I thought there were only two shades of Scrum – either you did Scrum-per-the-book or you did not. Since Scrum is the modern-day Peter Pan which refuses to grow up, we have only one version of Scrum to play around with, and unless some group of people decide now Scrum can grow to the next version (perhaps more because of commercial interests than anything else, whenever that happens). So, how can one ‘tailor’ Scrum – by any stretch of imagination, that is NOT Scrum. I mean, that is not allowed ! Right  ? So, why are we discussing it and wasting time – we might actually be acknowledging and unknowningly legitimizing that there is a secret world out there where Scrum can be ‘tailored’ and still be called Scrum ! That is sacrilege !

I always hate marketing messages that overpromise miracles, offer snake oil, belittle the intelligence of people, ridicule people…why can’t people simply stick to facts and figures ? Why don’t we talk in a non-intimidating language that encourages people to look up to what is being talked about ? And in the context of current subject, I doubt Scrum community is going down the right path if its next target is developers and customers. One of them does the work and the other one pays for it. Who gives us the right or the data to talk about any of them, in absentia ? Customers are customers, and even if they are irrational, they are still the customers, and whether paying or not, they alone are going to decide how they will behave. Yes, we might not like it, but is ridiculing them as ‘ignorant customers’ going to turn things into our favor ? Other industries like retail, hospitality, transportation, etc. have millions of years of collective experience in managing customers, yet they never break the singlemost cardinal rule of customer service: the customer is always right! And the first chance we software developers get to explain a poorly designed or a bad quality product and rightaway we blame the customers for it ! As if that is not enough, we then look inwards and blame the developers ! GREAT !

I think I will just develop software without blaming anyone else for my mistakes and limitations 🙂 

What is your Software Development Religion today ? And where does the Customer fit in that ?

The Swiss Army manual says: When the map and the terrain disagree, trust the terrain. However, in software industry, there seems to be an unending effort to make sure the terrain is retrofitted so that it looks much more like the map in hand ! So, I have modified the Swiss Army saying to suit the reality in the software development community as: When the real-life and bookish process definition differs, it clearly shows you have not understood the bookish definition, and hence you must change the real-life until it looks like the bookish definition and then trust the bookish definition.

Statutory Health Warning: Reading this blog post further could be bad for health, especially for those who love the process vocabulary more than the process intent, see the means more clearly than the ends and anyday favor the established processes of the day even to solve newer class of problems that clearly require fresh thinking lest they end up shaking the establishment. Author takes no material responsibility for anyone proceeeding from this point beyond and suffering serious health problems because of the radical views presented in this blog.

The conventional definition, popular understanding and state of practice of Software Quality is seriously flawed. They have created a very lopsided perspective that adherence to certain standards is Software Quality, howsoever hard- or soft-baked those standards be. On one hand of the rather colorful spectrum are the die-hard process zealots who won’t stop short of anything less than an ISO, CMMi or the likes and on the other end of the spectrum are the neo-rebels who believe anything Waterfall is bad and unless anything is Agilized, it ain’t good enough. On the innumerable discussion boards that I subscribe and listen to, completely petrified to speak up lest be asked my credentials to back up my anti-establishment views, I find more productive hours being lost on what should be the exact definition of a ‘product owner’, what should an ‘iteration zero’ be better known as, and whether you pass the Nokia test or not. I find the neo-rebels falling in the same honeytrap that they once so detested and fought tooth and nail – compliance over creativity. I see more mail threads getting fatter and longer because there are linguistic differences that probably should be settled so that practitioner’s camp can have peace after all, but where is the Customer in all this ?

We are probably forgetting that software came first, the generic notion of Quality came much earlier and everything else is only a recent model that some wise men and women have put together – and is likely to change with time. In fact, if that doesn’t change, there is something seriously wrong. So, there is a shelf life associated with Waterfall, and there is a shelf life associated with Agile as well ! Granted that Waterfall has had a rather long-tail that simple refuses to die (much to the chagrin of agilists), and so is Agile likely to have, but the fact remains if the software community stops innovating at and after Agile, it is not just bad enough for Agile – it is bad enough for the entire industry for the wheels of innovation and continuous improvement would have come to a grinding halt. 

I believe quality is not about how much the goods or services that a manufacturer or a service provider produces confirms to requirements. They are also not about whether it is achieved using Waterfall or Agile, whether CMM or ISO, whether done in-house or oursourced, or qualified using random testing or automated testing. Quite frankly speaking, I couldn’t care less if it was done by a bunch of social misfits or comforming cousins as long as I get what I want. But in all the noise and the alphabet soup of neo-processes, I think “what customer wants” isn’t audible much these days.

So, what really is quality. Well, to me, Quality is THAT differentiator in a product or a service that

  • makes me drive a few extra miles just so that I could buy or experience something I really like even when other, relatively cheaper options are available nearby. (=sarcifice time, effort and comfort to get something I value)
  • makes me choose one over other even when, everything else being rather equal, the one I choose might be costlier but not exorbitantly priced. (= availability of other altarnatives, freedom and ability to choose what I want)
  • makes me patiently wait in a line for my turn to come (=sacrificing my comfort to get something that I believe it worth it)
  • makes me pick up a product blindfold (=blind trust, but not trust blindly; reliable everytime)
  • I can recommend to my friends and family (=what is good enough for me is good enough for people I care)”

When I view quality from this perspective, it becomes very easy to substantiate what I said earlier – who cares what process was used, where the software was written, which language was used and what metrics were collected as long as I got what I wanted ? When one is willing to take a customer-centric view of software quality like this, the choice between Waterfall or Agile is only a matter of what class of problem we are trying to solve and what is the best-proven technique to address that situation. There are no permanent ideologies nor permanent religions – you must be flexible to choose what suits the problem at hand, rather than view it from a fixed-focus lens and try to ‘retrofit’ the problem to your software development religion.

So, what is your Software Development religion today ? and where does the Customer fit in that ?

PS: This post happened in response to a question on my favorite Q&A forum – LinkedIn.