Humphrey's Requirements Uncertainty Principle

Watts S. Humphrey (July 4, 1927 - October 28, 2010) passed away over the last few weeks and as a memorial to him I thought I'd post an article about his Requirements Uncertainty Principle, which is a cornerstone of Agile's approach to defining system requirements.

Watts Humphrey contributed significant thought leadership in the software engineering process and one of the principles he states is requirements are inherently uncertain. To quote and excerpt from his book "A Discipline for Software Engineering":

"This creative design process is complicated by the generally poor status of most requirements descriptions. This is not because the users or the system's designers are incompetent but because of what I call the requirements uncertainty principle:

For a new software system, the requirements will not be completely known until after the users have used it.

The true role of design is thus to create a workable solution to an ill-defined problem. While there is no procedural way to address this task, it is important to establish a rigorous and explicit design process that can identify and help to resolve the requirements uncertainties as early in the developmental process as possible."

Back in 1995 when this book was written (2 years after the founding of Scrum), Humphrey recognized that the software engineering process was broken and that repeated attempts at having a requirements document comprehensively describe a proposed system was met with failure many more times than with success. We are 15 years removed from this publication and there are many development teams still searching for this fictitious Holy Grail!

As we all know, Agile addresses Humphrey's Requirements Uncertainty Principle by:

1. Capturing what users' want in user stories
2. At the time of development, collaborating orally and through whatever documentation is required to fully understand what is to be developed
3. Designing and developing features to address the requirement(s)
4. Then immediately thereafter providing what's been developed to the user(s) so the requirements can thus be fully known.
5. Repeat

Why wait till the end of a project or many months after a feature has been developed to get feedback from our users. It is always most efficient to make the inevitable, yet unforeseen, changes to features immediately after they have been introduced.

The Uncertainty Principle in Software Engineering

Way back in 1996, three computer scientists, Hadar Ziv, Debra J Richardson, and Rene Klosch, wrote a paper that should be better known than it is. It's called The Uncertainty Principle in Software Engineering. Many have shortened the reference to Ziv's Uncertainty Principle. It states that "uncertainty is inherent and inevitable in software development processes and products". This principle sheds light on why Waterfall is well intentioned but flawed as a development methodology and Agile is better suited to deal with the uncertainty in software development.

Ziv's Uncertainty Principle models uncertainty in software engineering using Bayesian Belief Networks. In the software development world, Bayesian nets are most commonly known in its use in search algorithms used on large volumes of text and hypertext.

The authors focused on five areas of software engineering to demonstrate uncertainty;

1. requirements analysis
2. transition from requirements to design and coding
3. software re-engineering
4. software reuse
5. software testing

The authors also provided three example sources of uncertainty of which below are my paraphrased descriptions of each:

1. Uncertainty in the problem domain: The problem, for which an application is developed, exists in the real world. We all know the real world has many uncertainties, many of which are not, and/or cannot, be addressed by the application being developed.
2. Uncertainty in the solution domain: Building the application itself introduces uncertainty beyond the uncertainties in the problem domain. The example used in the paper is the act of debugging of race conditions from concurrent use. There is uncertainty in the exact conditions that cause it as well as how to observe the condition for reproduction. The authors use Heisenberg's uncertainty principle as similar a affect where by the mere attempt at observing an environment will change it. If you've had to debug a problem with concurrent use, then I'm sure you see this connection.
3. Human participation: Human involvement introduces uncertainty through business logic built into the application. Business logic coded into an application do no typically address explicit uncertainties. We code mostly based on certainty - not uncertainty.

The point is that attempting to address in advance all of the potential situations and conditions that will be faced in production is futile. The most effective way of dealing with the inevitable uncertainties is to get the application in the hands of the users as soon as possible and let the real world tell us what needs to change. This doesn't mean production alone. It means putting the application in the hands of users who can use the application in real world circumstances.

Waterfall makes the noble, yet inherently flawed, attempt at making the application as rock solid as possible on paper before commencing development. Agile's principle of frequent inspection requires that the team deliver working code frequently. Agile accepts and embraces the fact that once in production, the real world will show that significant imagined truths will be deemed either false or incomplete thus requiring non-trivial modifications and enhancements. These changes are most effectively implemented when features are introduced and not when the application is fully developed.

Another Explanation of Story Points

Story points seem to be the most misunderstood concept within Agile so I'm going to take my stab at helping others to understand what they are and their importance.

The typical question is: Why estimate user stories with these ambiguous and arbitrary things call story points when I can use hours which inherently make more sense to me?

Hours may make sense over story points at this very moment, but hopefully I'll be able to change your mind after you complete reading this article.

The major impediment to using hours is the variability between people and teams. I recently heard Jeff Sutherland state that a Yale University study has shown that what will take the best developer one hour to complete, the worst developer needs 10 hours to complete. When comparing best and worst teams that grows by an order of magnitude so an hour of the best teams translates to 2000 hour for the worst teams. The variability of hours is a real impediment and is typically a huge time consuming task on most projects.

Story points simplifies the estimation process by taking the developer/team variability out of the process by assigning a level of complexity to user stories. Many in the Agile community use a subset of the Fibonacci sequence as the units of measure; 1/2, 1, 2, 3, 5, 8, 13, 21, 34 and 45 per user story respectively. Think of story points as a more flexible version of assigning estimates as small, medium or large.

The story point estimation process begins by the team selecting the smallest user story and mutually agreeing on its complexity by assigning it a number from the Fibonacci subset of numbers. This user story is called the "keystone", meaning all other user story estimates are based on relative complexity to the keystone user story.

The byproduct of this is that upon sprint completion the team is able to report to the Product Owner in a more concise measurement of productivity - number of story points completed. The Product Owner sees things through the lens of project stories, features, etc. It makes more sense to communicate velocity, i.e. sprint productivity, in the form of story points than hours. Story points translate directly to what is manifested on screen when the working code is demonstrated and hours do not.

Here is an example of using hours as a measurement of sprint effectiveness. A team of four developers work on a project for a two week sprint. At 40hrs per week, the four developers have 320 hours of available time to work during the sprint. However, once the sprint is complete, their velocity shows that they only accomplished 60 hours of work based on the estimated hours assigned to the user stories they completed. That is a misleading depiction of productivity that leads to irrational conversation about whether the team is working hard enough.

Using story points, the team's velocity is expressed in terms of productivity related to the complexity of the tasks completed. For example, let's say the Product Owner knows in advance of the sprint that the team's goal is to complete 35 story points. At sprint completion, the effectiveness of the sprint is measured against the sprint goal and previous sprint velocities. Story point velocity is used to view the overall project productivity in terms of acceleration/deceleration, i.e. is velocity increasing or decreasing.

The Product Owner can also easily translate velocity into an estimated release date more easily than with estimated hours. As an example, if there are 350 total story points remaining in the product backlog, and the velocity of the team is at 35 story points per sprint, then the product backlog will be exhausted in 10 sprints.

Lastly, estimating projects is not only faster but more accurate than estimating in hours! Jeff Sutherland references two experiences related to this in one of his blog articles (read it here).

I hope this clears up what a story point is, why we use them, and provides a compelling reason to transition from hours to points.

Performing a Review of a Software Development Team

I've had several conversations this week about how difficult it is to provide reliable and consistent software development. Whether the team is an internal IT team, a product team, or a team of consultants, software development is rarely a pretty sight. It is sort of inline with the "making sausage" analogy. However, it doesn't have to be ugly so to help I thought I'd provide a checklist of attributes that every development team should consider.

Development Infrastructure

Each developer should be working in a virtual environment. Whether those VMs should live on the users computer or on the network is a debatable point. I believe the environments should live on the network to ensure they reside in the safest place possible. Residing VMs on the network also removes the dependency of a workstation being online.

Each developers’ local version of source code and folder structure should be identical so if someone were to go from developer to developer the location and folder structure would be identical.

There should be an integration environment to which a build is deployed at least once a day automatically. If the build fails the entire teams should be notified. At least one person should get an email when it succeeds. I’ve been in situations when everyone thought the build was succeeding reliably when in actuality the build machine was down. The integration environment should be identical to the production environment, e.g. web server, app server, database server. When the application is deployed it should tear down the servers to a base environment and rebuild it from scratch. At the very least, this environment provides developers with the ability to test their code in an environment that replicates production. Ideally, there should be automated QA testing that happens with each deployment. It doesn’t have to be full regression testing; an automated smoke test goes a long way.

Continuous integration is a must have. Finding out immediately when a developer has broken the build is vital. It’s so much easier to fix issues as they occur than to untangle them weeks after many developers have introduced their own flavors of bugs.

Development Methodology

This one is a big deal to me because I think very few shops actually have a development methodology. My evidence is anecdotal evidence from being a consultant talking with many clients and prospects and through interviewing software engineers. Most profess to having a methodology which turn out to be more of a set of development procedures than a methodology.

One definition of the term methodology is “the methods or organizing principles underlying a particular art, science, or other area of study.” Organizing principles is the key phrase. Organizing principles are more than a set of procedures. Your methodology should be well understood and those using it should evoke confidence that when the going gets rough that your organizing principles will protect you - as long as you stick to them. Abandoning your organizing principles, in whole or in part, when times get tough results in pain for you, your team, your management and your internal/external customers. In other words – everyone.

Estimation Techniques

My own little survival handbook is called “Demystifying the Black Art of Software Estimation” by Steve McConnell. In it, the author states that studies have shown that estimates are, on average, 30% of what the actual level of effort will be. Thus, know that your developers will significantly and consistently underestimate tasks. If you have never read a book on software estimation then it is a must.

The absence of formal estimation techniques could be the biggest failure point in software development. Communicate estimates in the form or ranges not in single values; e.g. 4-6 weeks. Include all the ancillary tasks such as user documentation, technical documentation, project management, etc. When developing the schedule consider vacations and holidays. And lastly, stand strong in the face of resistance from those who want the job done cheaper and/or sooner. You can negotiate rate, features, etc. but you can’t negotiate how long it’s going to take by magically lowering the number of hours and expecting good things to come from it.

Strong Project Management
A project manager, depending on the organization, is either just another resource keeping track of the project plan, or a leadership role on the project. The latter role requires the project manager to be an oracle and captain at the helm and is the one I prefer. “The captain” reference refers to that person as a significant influencer of decisions and direction. The oracle reference  refers to the ability to recognize patterns to identify risks and potential pitfalls. Project management is a combination of art and science. It also requires a strong personality who is willing to give bad news as soon as it is known. This isn’t easy for most people to do and lots shy away from it and futiley hoping things will work out.

Team Composition

We all want the best an brightest. But what’s often overlooked is how the team is composed. Being smart isn’t enough. Your team needs to be filled with smart people who fill specific roles and view the world in a specific way. People can be big picture oriented or not, detail oriented or not, organized or not, ambitious or not, etc. It is vital to architect your teams composition, meaning, defining how many of what types of people you want/need on the team. Teams need a variety of types to be successful. Know what those types are and fill them accordingly.

QA Process

The business should write the test plans and ideally they should be written before development begins. If you are an Agile shop, each user story should have test scenarios. There should be automated QA testing. If there are budget constraints there are free tools out there to leverage (Selenium and NUnit combined with Cruise Control will work fine in most cases).

Stakeholder Roles & Cross Functional Relationships

The business users should be heavily involved. My mantra to customers when negotiating for their time is “the quality of the end product correlates directly with the level of your direct involvement.” Given that they are often making a significant capital expenditure, they tend to get involved to whatever degree they are needed. The reality of the outcome as a result of their involvement, or lack thereof, just needs to be communicated.

Resource roles should be clearly defined at the beginning of the project. There should be no ambiguity with what’s needed from the business users.

Cross functional relationships should be nurtured during and outside of projects. it is important that the business and technical teams know that they are partners and together they will be succeed or fail.

Communication

The glue that hold all of these together individually and as a whole is communication. Everyone should be communicating to everyone often and transparently. When issues occur, make sure everyone knows. I’m not saying that there shouldn’t be a communication protocol. There should be. However, when Engineering disagrees that a bug found by QA is actually a bug, the engineer who developed the feature should have direct access to the QA engineer who logged the bug and the product manager who articulated the feature.

Customers should be involved on a daily basis (with Agile via the daily standup). This makes the weekly status report nothing more than a summary of what the customer already knows. Demo working code frequently to get feedback from the customer. At least one engineer should be in the demo to hear directly from the customer and be able to ask questions accordingly.

Transparency can be scary because we are afraid of the repercussions of allowing the customer to see our problems as they occur. What I’ve found is that customers are reasonable business people who understand the ups and downs of projects. The surprises are what get them angry. Transparency eliminates surprises and keeps everyone in the loop with everything that is happening with the project. Under this umbrella of openness the cross functional team inherently act as a unified team navigating the project towards a mutual success.