Category Archives: Agile

How should we estimate software sizing in a software development project when using Agile?

Caliper

In this post, I will focus on how should we do estimates in an Agile project.

Just to set the proper context, it is important to keep in mind the main tenets of Agile:

In any Agile project, the primary measure of progress is working software, which means that any other kind of deliverable is basically of low value.

Another main tenet is simplification: Maximize the amount of work not done!

During development, we need to be able to control the process, and we all know that we can’t control what we do not measure, so we do need some estimates.

But we should choose any software sizing estimation technique that is good enough for us to deliver working software: the focus should be on good enough!

When it comes to Agile estimation techniques, there’s a lot written about it.

On most books and articles on Agile Estimation techniques, the lion’s share goes to a technique popularly known as Planning Poker.

I agree with the idea that Planning Poker is fun to do, but I also agree with the well established fact that any team using Planning Poker will require a lot of time to complete this activity for a set of user stories. A lot of time spent on anything different from delivering working software is the wrong thing to do when you are using Agile.

Spending a lot of time on estimates is not consistent with the idea of maximizing the amount of work not done (more on this at the end of this post!).

There are alternative techniques, some of them are better versions of Planning Poker. Among the few techniques that have succeeded where Planning Poker has not is a technique known as Silent Grouping.

Even though Silent Grouping shares some elements with Planning Poker, it has avoided to include the elements that in PP contribute the most to the long duration of the sessions.

What is the basic “operation” of Silent Grouping?

The “game plan” for Silent Grouping has a Setup stage, two rounds of estimations and a Reflection stage.

The main elements are a card for each user story to estimate, a PP Board and a Parking Lot.

The most important aspect in the entire “game plan” of Silent Grouping is that the participants do not talk to each other during the two rounds of estimations (thus the “Silent” reference).

It is meant to eliminate, as much as it is possible, any kind of discussions among participants during the rounds.

At the Setup stage, the moderator will lay the ground rules of the session to the participants, and set expectations.

During the first round, the team members that participate in the session stand in a queue, so that each member has to wait for her/his turn. When it is the turn of a given team member, she/he will pick up the next card from the stack of cards to estimate, and put the card at the board within the “swim lane” that belongs to the number of points that the member considers to be the closest estimate for that given user story.

During the first round, each participant will give feedback on just one single card for each given turn at the board.

If the list of participants is small and the set of user stories is large, participants will have to iterate as many times as necessary so that each user story card is assigned an initial estimate.

Even though any given participant may give feedback on more than one card during the first round, any given card with have feedback from one and only one participant during the first round.

The board with “swim lanes” shows at the top the number of user story points for that group (thus the “Grouping” reference). Typically, the board will have 8 “swim lanes” or groups.

The first “swim lane” at the left side of the board has the number 1 at the top, and the number increases for each following “swim lane” to the right according to the sequence of Fibonacci numbers, typically until the number 34 (that 8th number in the Fibonacci sequence):

1, 2, 3, 5, 8, 13, 21, 34

(Did you ever wonder WHY is it that the Planning Poker Board NEVER has a number bigger than 34?)

During the first round, participants give feedback on card estimates in complete silence.

During the second round, the participants stand in a queue.

When it is the turn of a given team member, the participant is free to change the estimate of all the cards, by moving each card and placing it at the board within the “swim lane” that belongs to the number of points that the member considers
to be the closest estimate for that given user story.

It is possible that during the second round, some of the cards suffer a lot of moves. It is the responsibility of the moderator of the session to take note of these “controversial” cards (the cards that get shuffled at lot).

This second round is also done in complete silence: Participants give their feedback just by moving cards, one participant at a time.

During the second round, each and every participant is free to give feedback on the estimates of each and every card, but this time, each participant will have one and only one turn at the board.

Once the second round finishes, the moderator will place all the “controversial” cards in the Parking Lot (any kind of place holder for these “outlier” cards: a box, a basket, etc.).

At the reflection stage, if there are any “controversial” or “outlier” cards, participants will resolve any disputes of estimates for these cards only, but this part should we run without any kind of discussions or arguments. The goal is to either agree on a fast estimate for each card or else.

The Product Owners should not participate in the rounds of estimations, but could give feedback to participants during the reflection stage, if and only if any participant has questions regarding any given “outlier” user story: just short answers to simple questions.

Regarding this “outlier” cards, two possible outcomes could happen during the Reflection stage: the participants are able to easily resolve the dispute on the estimate of a given “outlier” card, or they don’t.

If they easily resolve the estimation dispute for a given “outlier” user story, everything’s fine with that user story.

If they are not able to resolve the estimation dispute for a certain user story, I can tell you that the estimate is not the ONLY hard question that the participants are not able to answer regarding that user story!.

In a sense, Agile is an adventure of exploration and discovery around a given set of user stories.

When the team members cannot agree on the estimation for a given user story, it could be a clear indicator that the team has not explored that user story enough at that point in time, so, the user story is not ripe enough for development.

If your team faces this kind of problem with any given “stubborn” user story at the end of the Reflection stage of a Silent Grouping session, the moderator should set aside each and every “stubborn” user story for further exploration and discovery.

As any Agile team is free to organize itself to solve any issue, the team should decide on how to deal with this situation: which of the team members should work on the exploration and discovery of the “missing” details of the “stubborn” user stories?

After the necessary stage of exploration and discovery is done for all user stories, they should be all ripe for rapid estimation and development.

On any given project, a team may need to run a few estimation sessions to cover all user stories, mainly because on any given project not necessarily all user stories are known from the get-go.

As a parting thought, we have come to the conclusion that any estimation session in Agile should always be a fast affair.

We should use techniques like Silent Grouping instead of Planning Poker, and most importantly, the team should never waste time feuding on estimates.

The proper way to deal with “stubborn” user stories is to set aside a group of team members to do some further exploration and discovery of the details of such user stories.

Kind regards, GEN

Agile and Estimates (I): the nature of projects that require the use of agile

Clock

Estimates are very important for any kind of project, including those projects  that require the use of agile, there is no question about that, but this brings to the foreground another set of important questions:

  • how do we use estimates in an agile project?
  • are there differences in the use of estimates with agile projects (when compared to the use of estimates in “traditional” projects)?

To be able to answer these questions, we need to explore some important characteristics of the nature of the projects that require the use of agile.

So, what’s the deal with the nature of certain types of projects that require the use of agile? Well, the kinds of projects that may need the use of agile have at least one of the following characteristics:

  • The company is exploring and developing an entirely new business model, so, business rules for this new business model are being explored and understood along with the project.
  • The company is developing a new type of product or service based on a new type of technology, so, there are no experts as this is a new field.
  • The company is using new software tools and technologies, or is adapting and customizing open source tools to its own needs, so the dev team has no experts on these tools and technologies.

If the company in question is a startup, the project will have all these characteristics, but even if the company is not as startup and/or the project has only one of these characteristics, it clearly will benefit from the use of agile.

So, the main factor in each one of these characteristics is uncertainty in a very high order of magnitude, very much in line with the kind of uncertainty you would expect in a typical Research and Development project.

In fact, agile projects and R&D projects share enough elements in common pertaining to the impact of uncertainty, that some of the project management tools and techniques that apply to R&D projects may also apply to agile projects.

One such tool that is useful for agile, just as it is useful for R&D projects, is the PERT estimation formula. Even though the historic background of the development of the project management tools collectively known as PERT is really interesting, it makes more sense to focus our attention on the estimation formula itself.

In agile projects as well as in R&D projects we may have a group of experienced professionals exploring how to solve a problem while “navigating through uncharted waters”. Under such circumstances, when trying to work out estimates, they will offer a range of values for any given task, from optimistic to pessimistic, and with some intermediate values being the most frequently proposed.

Taking this into consideration, the PERT estimation formula is based on a skewed distribution, the Beta Distribution, that properly takes into consideration the nature of such processes. Duration estimates are always positive, and it makes sense to use a balanced approach capable of compensating for overly optimistic or overly pessimistic values used in the estimation process.

The PERT estimation formula is very useful to provide “good enough” task duration estimates provided by experts under conditions of high uncertainty.

For this case, the estimation value is based on the median for this skewed distribution.

The PERT estimation formula expression is the following:

Te = (To + 4Tmp + Tp)/6

Where Te is the estimation for the duration of a given task, To is the most optimistic value for the duration of the given task, Tmp is the most probable value for the duration of the given task, and Tp is the most pessimistic value for the duration of the given task.

This estimation formula offers a balanced approach based on a weighted average, giving more importance (4 times more importance) to the most probable value, while still taking into account both optimistic and pessimistic values.

In an agile project, how do we get the values to apply to this formula?

It would be in meetings like the Planning Game, or in a Sprint Planning Meeting where the team can obtain estimates from the team members for each task, and out of the range of estimate values for each task it would be easy to determine the 3 input values for each given task needed for this estimation formula.

For each given task, the most optimistic and most pessimistic values are easy to determine, but the most probable value is not that obvious to determine. The most probable value is the mode, and the team must create a histogram for each task with the value of all the estimates for each given task. The mode corresponds to the class of the histogram that has the tallest bar. Sometimes, histograms for estimates look kind of weird, like with more than one hump (that’s a multimodal histogram): we will discuss about that in a follow-up post.

It is very easy to determine the mode for any list of values with a tool like Excel or Open Office.

This sheds some light on what approach to use for task duration estimation out of the range of values that the team proposes for each task during any planning meeting, but it does not offer many clues regarding the differences (if any) on how to handle estimates in an agile project. We will discuss this in a follow-up post.

Kind regards, GEN

Any Agile project is an ongoing conversation with the customer

Image

In my previous post, I mentioned that when in doubt on what to do to fix a problem in your agile process, think of The Agile Manifesto as your inspiration and guide on what to do.

But, as the Manifesto offers a list of guiding concepts, it may turn out a little tricky when a given problem to fix puts you in a dilemma between at least two of these concepts that may seem at odds.

Let’s suppose a situation to give an example to make the point more clear: in the next sprint, the customer is asking to add a new feature to the solution. Some of the existing functionality must be able to support that new feature (**)

From the experience in previous sprints, you have already learned that the customer is adamant about the idea of “retrofitting” as the way to give existing functionality the ability to support new features, specially if that retrofit (which looks a lot like re-work!), may require a substantial amount of the person-hours of a sprint.

 After you navigate these reqs with the customer, you realize that they feel oriented towards adding new routines, similar to the existing, that include the new feature, so, for any given functionality, you will have two versions of the routine: one version is the existing routine, and the second version will be a similar routine that also supports the new feature.

It is clear to you that this approach does not sit very well with quality attributes like Maintainability or DRY (Don’t Repeat Yourself), just to name a few (and Separation of Concerns, and Single Responsibility Principle, and …)

Besides this, you perceive that these reqs put you in a contradiction between “Working Software” and “Customer Collaboration“.

So, the Agile Manifesto, instead of helping you on how to find a solution to a problem, seems to be part of the problem. So, what should you do?

As I have already stated in (another) previous post, Agile is all about “giving value to the customer”, so, we should not find any contradiction at all, since “value to the customer” means that “Customer Collaboration“, first and foremost, will be your beacon that will lead you out of the woods.

If, after you explain to your customer all the pros and cons of each proposal, including the ones that do not seem OK but are more pleasing to them, the customer still insists on their idea, well, you should be flexible enough to accept their proposal for the time being, implement, and let them use this (not so perfect) implementation, so that they may eventually realize what you were talking about.

THE concept here is that any agile project has to evolve as an ongoing conversation with the customer: to be able to give value to your customer, you must understand your customer, and to be able to do so, you must keep that conversation open and flowing.

Kind regards, GEN

(**): An existing digital dashboard web app, with a set of charts that allowed drill down into tables with detailed info. The original dashboard had monetary figures in just one currency, and the customer wanted to include support for multiple currencies.

When in doubt, think of The Agile Manifesto as your inspiration

Image

You are the leader of an agile team, in the middle of a sprint, facing a problem with the process. You need to figure out a fix to the problem, but can’t find a clear solution. You reviewed many books and web pages on agile, but still can’t find a clear path to solve this problem.

It may sound like a deja vu situation to many readers simply because is a very frequent situation.

When we find ourselves in such a predicament, where do we turn to for council?

Well, when in doubt, look for inspiration in The Agile Manifesto, but focus your attention on the concepts on the left, that is:

Individuals and interactions  as it is people and their interactions that may be able to detect and solve the problems.

Working software“, as it is the main deliverable that represents real value to the customer.

Customer collaboration“, as the true approach to fix things up is “all together against the problem”.

Responding to change“, as change is the name of the game nowadays.

Kind regards, GEN

Recent sequence of tweets on Agile

This is an article made out of a recent sequence of tweets I sent on Agile.

Let’s suppose that twitter would have a 40 character limit: how would you describe Agile in a tweet? It’s all about value to the customer!

In Agile, “value to the customer” is a recipe with @ least 2 ingredients: one part delivery of user stories, one part wacking process waste

What does “process waste” mean in Agile? … It means just about the same that it means for either TPS or Lean Manufacturing! (<== TPS stands for Toyota Production System)

Just to be precise, let’s review Taiichi Ohno’s Seven Types of Waste …

1.- Waste of Overproduction

2.- Waste of Waiting …

3.- Waste of transportation

4.- Waste of processing in itself

5.- Waste of inventory

6.- Waste of motion

7.- Waste of making defects

OK, it sounds nice, but what does all that have to do with Agile? … A lot! … Actually, in a way, Agile is all about that!

1.- Waste of overproduction … this is an easy one: it is the same as YAGNI (http://c2.com/cgi/wiki?YouArentGonnaNeedIt …)

2.- Waste of waiting … Another easy one: all of us point this out as a problem at the daily scrum (or daily standup meeting)

3.- Waste of transportation …This happens a lot in many shapes on a daily basis, like when you need to FTP a DB backup from the customer

4.- Waste of processing itself … This also happens a lot, like when (you) have to do a lot of debugging to test your code: design simpler code.

5.- Waste of inventory … this is a tricky one, as in SW dev we do not have code inventory … What’s the equivalent? …

… Actually, that is as easy one: in manufacturing, inventory is the cushion, so, in SW dev, the cushion is cushion time in the estimates!

6.- Waste of motion … Another one that seems tricky, but it’s an easy one:this happens a lot, when you don’t have all the tools at your PC.

7.- Waste of making defects … self explanatory.

OK. Now we know that a lot in Agile is about getting ride of the waste as much as it is about delivering valuable code to the customer …

… on a frequent basis! (<=== on both counts: frequent wacking of process waste, and frequent delivery of valuable, working code to the customer!)

Kind regards, GEN

 

Globalization and some challenges Agile teams are facing

Image

When it comes to the IT industry, some of the more important consequences of Globalization are Outsourcing of IT operations and Outsourcing of many software development posts.

Regarding the Outsourcing of software development posts, in general this has turned out to be a positive move for software companies and IT professionals in developing countries, as they have experienced a growth in their business operations driven by this process.

But this movement also has brought its own share of ills into the mix.

Now, the typical software development project has a team with team members from many countries, working across many time zones. The most frequent process used by these teams is Agile.

Let’s review some of the challenges that these Agile teams are facing, in no particular order.

One of such challenges is the diverse cultural backgrounds of team members. Even though diversity in general, and diverse cultural background in particular, is a very positive factor for an Agile team, the challenge that most frequently shows up is pertaining to communication.

One on one conversations between team members are a major aspect of Agile as a process, and diverse cultural backgrounds, including diverse native languages, pose some important issues and obstacles to the natural flow of the conversation.

[New comment added] It would make sense to expand this idea a little bit: outsourced team members in general do not belong to the same organization as the rest of the team, so, it is very likely that outsourced team members (developers) might not know the details of the business of the customer, or the meaning of business terms used by the customer when describing functional requirements and business rules (this relates to the concept of “ubiquitous language” in Domain Driven Development).

Besides this, different cultures usually have different cosmovisions, that is, different mappings to the concepts and ideas that explain the universe all around us. One of the first mappings that each person incorporates is their mother tongue, so, different native languages is an important element of the diverse ways of understanding the universe all around us. [New comment added]

But diverse cultural backgrounds is not the only challenge to one on one conversations, as all of these teams have non-collocated team members, and  that also poses a major obstacle to the flow of the conversation, at least to some of the conversations that are required to happen on a daily basis.

Another challenge is caused by the fact that team members that represent The Voice of the Customer are usually on a different “shore” from outsourced team members, most of them being developers, so, “up-close” conversations between developers and users also face some obstacles to be solved.

Another pattern that also is becoming important is that outsourced team members are also non-collocated among them, that is, outsourced team members from different regions of a country or from different countries of the world.

This pattern in particular is a major obstacle to Agile. This pattern affects many Agile practices, like one on one conversations, or even pair programming.

Another pattern that is becoming important is that all these constraints that affect these Agile teams introduce certain types of discontinuities to the process that induce issues and problems to the scalability of the team.

For instance, these constraints and discontinuities have to be mitigated with many calls and meetings of the leadership of the project, which means that team leaders are busy each and every day during a significant percentage of their available time, which means that they are not as available to team members as it would be recommended.

The principles of Agile are such that if Agile needs any fixing, it should be done from within, that is, by the team. So, in a coming post, we will discuss some solutions to these challenges.

Kind regards, GEN

How should the developer interact with the Business User?

 

From an Agile view-point, the business requirements to the developers should be expressed as single sentences. These single sentence functional requirements are called user stories.

The Sprint Backlog for any given Sprint should list the user stories (by their IDs) that are planned to be developed during that Sprint.

Each and every user story is an invitation for the developer to talk with the Business User about the details of that specific requirement.

In my personal experience, the best expression of the business requirement they could come up with is a group of unit tests that describe the requirement in a “good enough” fashion.

As the team should be using “Test First Development”, all they should work on is just enough code to have all the unit tests for that requirement, which at that moment, these tests should not pass.

This should not take much time to accomplish, so the Business User will be free to talk to some other developer about some other user story, while the developer could start working on adding just enough code to have all the tests in “green”.

KR, GEN