The Software Purist |

TAG | management

Dec/09

14

Scrum: Agility and Practicality?

In this installment, I wanted to discuss the Scrum software process. I have worked on some Scrum projects and I intend to go through some of my experiences in the following post. Scrum is often referred to, as simply, Agile. However, calling Agile a software process is a bit of a misnomer. Agile is a classification of software processes that share a few common traits: Iterative styles of development and a more fluid ability to react to a change in requirements, strategy or customer needs. Answers.com defines the word agile as being the following: “1. Characterized by quickness, lightness, and ease of movement; nimble. 2. Mentally quick or alert: an agile mind.” So, this flows with the general spirit of Agile. With this in mind, I can share some of my experiences, particularly with the most recent large project I worked on that was using the Scrum methodology.

The first thing to note when discussing this project is that I have worked on other Agile projects before. Generally speaking, however, prior to this one, saying we were doing Agile was more of a catch phrase and Agile wasn’t really being done. Having been trained in Agile and having done my research, in these cases, I was well aware that we weren’t doing Agile, but I believe I was in the minority. Anyway, coming back from that tangent, in this most recent project, the group did make an effort to follow Agile, as it was being dictated by a particular customer. More specifically, we were using Scrum, although most of the way through the process, most of the group was unaware that Agile and Scrum aren’t simply synonyms. An interesting blog article discussing this common problem can be found here.

Anyway, my experience with it wasn’t really a positive one, although I don’t necessarily think I can fault the Scrum process itself. The idea of a daily morning scrum with a small group of people seems like a logical idea. In our case, it wasn’t so effective, because our group was as high as 16 people, including project managers, technical managers, developers and QA, in two different offices. For the overseas office, the meetings were in the afternoon for them. Because of the high quantity of personnel, the meetings took 30 minutes – 1 hour instead of the recommended 10 minutes or thereabouts. In addition, the meetings degraded quickly, with developers discussing specific issues instead of keeping things at a high level, following the general rule of three: 1) What I worked on yesterday, 2) What I will work on today, 3) Blocking issues. Attempts to redirect it back to these three 1-3 sentence overviews weren’t always appreciated.

Again, this isn’t condemnation of Scrum specifically, as I think it had a lot to do with the structure of the meeting. That being said, I think daily scrums can degrade pretty rapidly, and it’s especially at risk, the larger the group. In addition, we typically started in the 9-9:30 AM range, however, the meeting was often at 10 or 10:30 AM. I find I have very good concentration in the morning and breaking this routine had an impact on my morning workflow. The increased communication is important, but I question whether it would’ve been better to end the day with this meeting. I ran into this same problem at my previous company, when we worked with Scrum: The meetings would last too long. In that case, the group was much smaller, but again, developers are developers, and sometimes the moment they think of an issue or interesting concern, they want to discuss it right there, even if it’s not the appropriate forum.

The second critical piece of the Scrum methodology is sprint length. Scrum recommends one month sprints. In the most recent project, sprints were generally two or three weeks at various points during the process, which I felt was too short. The problem with the reduced scope of this sprint length is I find that there’s too quick a push to show results. Sometimes when designing a complicated and major feature that integrates with multiple different system components, this can be too short. Sprints this short require too often a time to leave the system in a stable state to display to the customer and to management. In general, I recommend if you decide to do Scrum that you stick to the suggested one month sprint length.

Some other important artifacts of the Scrum process are the product backlog, user stories and burn down chart. The product backlog makes a lot of sense; unfortunately in our process, the product backlog was not used by upper management. I believe this is due to it being the first time the company had really tried to use Scrum. I think things would’ve flowed a lot better had the product backlog been used effectively. User stories are a tool which seems to be the Scrum replacement for Waterfall-style requirements. The idea of user stories makes sense, but I found that they were very time consuming to write. From here, the user stories derived into the sprint backlog tasks, which were intended to be short tasks between 4 and 16 hours. The final piece is the burn down, which basically gives a simple representation to figure out if you’re going to hit your target for the sprint. It will give you an idea if you’re nearing the danger zone.

One problem I frequently encountered during this sprint backlog tasks is for the tasks, the development teams often over-committed. It’s very easy to look at a small feature and say it will take only four hours, but the four hour estimate was often a best case estimate, without taking any consideration for integration, testing and bug fixing. So, it would generally follow that the team was working 60-70 hour weeks to meet the goals, due to the mis-estimates. Once hitting the mis-estimates became commonplace by upper management, they pushed for the same amount of committing with all sprints, so the development team had dug themselves into a hole, due to the poor estimating.

A critical piece of the scrum process is the sprint planning meeting. During the sprint planning meeting, the goal is to plan the tasks that will be done for the sprint and decide on who will do them. However, part of this process is that the sprint meeting will take no more than 8 hours and that the team will sign up for tasks they will do. However, this frequently was not the process. Mostly, it was a few senior developers and management deciding on who would work on the tasks. As you might have guessed, I was among the deciding group, although I objected to not including everyone. Furthermore, because not everyone was included in the meeting, the meetings often ran way over; I recall one that started in the morning and basically ended about 15 hours later. I generally recommend that there needs to be prepared prior to this meeting and the tasks should flow from the product backlog. Everyone should come into the meeting prepared and with some goals in mind. If people come unprepared, the meeting will drag. If not all the right people are involved, again, the meeting will drag. Unfortunately, no matter what the outcome is, you’re almost guaranteed to lose a full day of development every sprint, just deciding what tasks will be worked on.

The final piece I’d like to mention is that during a sprint, Scrum dictates that tasks shouldn’t be changed. Management cannot dictate priorities for the current sprint. They can change requirements for future sprints, but not the current sprint. In our case, this advice was not heeded. Both management and the customer would frequently change direction during the sprint. This resulted in a lot of rework and a lot of late nights adjusting to the new requirements. I have a feeling that this is commonplace during Scrum, which is part of my distaste for the process itself.

Ultimately, I think that Scrum makes a lot of sense on paper. Having gone through multiple projects which have made some use of Scrum, including one that made heavy use of Scrum, I have not seen any of them work effectively. In a way, Scrum promotes poor development practices, because the focus on rapid change somewhat discourages the practice of thinking a design and architecture through, which can lead to premature gratification that ultimately makes a more difficult to maintain design later. I’m definitely no fan of a strict Waterfall process, but I am confident that Scrum, in the process of fixing some of the flaws of Waterfall, has created some new ones in its place. I do think that education in the usage of any software process, whether it’s Scrum, XP, RUP, CMMI, etc… is essential to things working smoothly. The fact is that when faced with deadlines and pressure, people will revert to their most familiar behavior, some of which does not play with a process like Scrum. On a side note, a few developers I’ve worked with in the past have been very fond of the XP process. I have some doubts about XP, as well, but I can say that when I have used it, I have seen some benefits.

· · · · · · · · · · · · · · · ·

Dec/09

4

Software Outsourcing, Part II

If you have not read part I of this series, please check it out first:  My Experiences With Software Outsourcing. My first entry in the series discusses some of my experiences with outsourcing. In this second installment, I discuss these issues a bit deeper and provide my overall thoughts on outsourcing.  I’ve already gotten some great comments on the first article, which I appreciate.  One thing I should clarify is I’m not against outsourcing, just against certain approaches and motivations by companies who decide to use outsourcing.  In this article, I present scenarios where I think it may work.

Overhead

As I mentioned in the previous edition, time difference can be a huge factor.  Teams that are intertwined often need to work closely and need more immediate feedback.  The only way to reduce this coupling is… well, to reduce the coupling.  Therefore, I am of the personal opinion, that in outsourcing, much like software, there should be a large desire for low coupling, high cohesion.  You have to trust your outsourced developers to do the work with the right instruction.  If you don’t have this sort of faith, then outsourcing is not for you, because the overhead can be high enough to defeat all discounts, and potentially, even be more costly.  Having dealt directly with this, I know from experience.  Ultimately, the goal is to desire economies of scale and avoid diseconomies of scale.

Low Coupling

High coupling occurs when any one person is free to contact any other person in the company.  Not only are they free to, but it is encouraged.  This can happen in one office.  The problem is if a single person takes direction from multiple sources, it’s more time cumulatively time consuming.  This can be deceiving, because to the person who made the request, they often got feedback quicker than if they had to go through channels.  However, for the person who served his 30th request of the day, it is well known that this is not efficient.  If this person who has been communicated to, by so many people, is a developer, all of this intercommunication prevents entry into “The Zone”.  This is one area where all of these channels of intercommunication can be the most costly.

The reason why this behavior is so prevalent is very simple.  It works extremely well when a company is small.  In a 10 person company, every person knows every other person and likely communicates daily.  But there’s a reason why this is so efficient: The company is small enough that there is likely no more than two managers, and everyone is strictly focused on work and completing their own tasks.  Then, each person merely talks to the person who he or she most needs to communicate with to complete his or her task.  The problem is that, as you likely realize, it does not scale well at all.  Putting project managers in place is a step to reduce that, since the project managers can act as the point of contact.  However, it doesn’t go far enough, unless the offshore project manager is given more authority, and the onsite project manager is more used for communication and just communicating ideas back and forth.  When it goes further that this, I believe it’s demotivating for the foreign employee to be working through multiple layers of management and messages being relayed.  And of course, it’s costly, resulting in a lot of rework, due to miscommunication.

My ultimate recommendation here is that each office should work on disjoint projects, with potentially different sets of standards and rules.  I think it’s tempting at first, to couple things tighter, because from a company perspective, it makes sense to not duplicate work.  As a purist, I feel that duplicate work is generally not cost-effective.  However, in this case, I think it’s sometimes acceptable, mostly because the alternative may be worse.  So, I generally feel that in order for the scheme to work, it has to be treated as almost two different companies.  The US location should act and be treated like a customer.  The foreign location should act and be treated like the supplier of the software.  In this scheme, I think it can be valuable, because you forego most of the overhead of the interoffice communication, except from the perspective of a client, who is concerned with functionality and not dictacting architecture.

Motivations

The last thing I wanted to talk about in this installment is the motivations behind outsourcing.  The obvious motivation is saving money.  There’s no doubt about this, and it’s justified.  However, I see outsourcing as an opportunity.  You are getting developers for a fraction of what they would cost in the US.  Some companies take this a step further and even look for a deal in those countries.  I have to say, I think this is going overboard.  If you’re already getting developers at a fraction of the cost, why would you then proceed to also get inexperienced developers to save a few extra thousand per person per year?  If I was me, I’d pay highly there to get the most talented. It’s often cost prohibitive to get the most talented developers in the US, but abroad, it isn’t. I also feel that by getting the most talented developers, you get people you can trust to do the job, which also flows well with the concept of keeping overhead minimal.

Conclusion

Studies have shown that the cost benefits for outsourcing are often minimal, especially for companies who are seeking a deal. In pure cost analysis, because of the inherent overheads, it can provide little cost benefit, or in some cases, even be more costly.  I discuss scenarios where outsourcing might work, but it needs to be done with care.  If you’re a discount shopper, you may be fooling yourself.  If you use outsourcing to simply increase the pool of talented individuals who you may employ, the strategy has potential to be effective.

· · ·

Theme Design by devolux.nh2.me