The Software Purist |

TAG | Development

Dec/09

27

Front-End vs. Back-End Development

Hi all. There was a bit of break because of the holidays, but now I’m back with a new post that I hope you will enjoy. In this edition, I wanted to discuss some of the differences between front-end and back-end software development. I would say that one of the interesting things about my career, to date, is that I’ve probably had a pretty even split between front-end and back-end software. As such, I’ve noticed some interesting differences between the two, particularly with how they are approached. When I speak of Front-End development, I’m really speaking more of visual-related code, particularly GUI code. As such, note that some of these observations don’t apply as heavily to related areas. When I speak of Back-End developmet, I’m generally speaking of the logic on a multi-user server application, such as a socket server, which handles requests from many users. With that, let’s get started.

So, the first thing to discuss here is ease of development. I’ll admit, outright, that this is a bit of an unfair category, because it will significantly vary depending on framework. For example, using some well-designed frameworks, setting up the GUI can involve a lot less programming. If you use one of these frameworks, a lot of the work can even be done by artists or designers, without much intervention by developers. Of course, if you’re doing more low-level GUI work, such as pure Win32 or XWindows/Motif, this doesn’t really apply to you. Having said that, for many applications, choosing a relatively high-level GUI framework is commonplace, unless there’s restrictions that prevent this from occurring, such as performance reasons or customer requirements. So, from the front-end side, you often wind up in a state, where the framework you’re using is already in place, and you’re making small extensions and the application basically derives from that. From a pure GUI standpoint, while designing the overall architecture is important useful, in some cases, especially in smaller applications, you can get away without it, or possibly define it later, through some refactoring techniques.

From the back-end side, the framework you’re using is often lower-level. Even if it’s higher-level, you still need to architect an application to integrate well for future needs and a lot of user requests, including making use of multiple cores and taking advantage of latency. Back-end work mixes both low-level and high-level concepts, and there is often a significant amount of work required to get something up and running. In many back-end applications, the logic on the back-end can be heavy, for the reason of security. At other levels, the data may not be trusted. On the back-end it is. Furthermore, back-end software typically operates with a database. The database has the interesting property of both making things simpler, in terms of coordinating many simulatenous requests, and also more complicated than typical front-end data storage. For example, on the front-end, it may be perfectly acceptable to store all data as XML and write all data to file immediately. This of course, would not scale to the back-end. Furthermore, the back-end has to worry about synchronization. Front-end applications can potentially get away with not making great use of idle processing time, but this is often not so on the back-end.

The front-end is a world where there is a lot more instant gratification. For example, you can design a new button, click a wizard option to add a handler, enable/disable the button at certain times and handle the button press, but initiating a file to be saved. You can code this and be testing, verify it and have new functionality to show the same day. This is usually not the case for back-end development. On the back-end you might be designing functionality such as saving some data to the database. So, you write the code to handle the request message from the client, you write the stored procedure you need (possibly to be tweaked by a DBA later), you code up the response to the message. After the same amount of time as spent on the GUI, are you done? Unfortunately, no. Firstly, having gone this far, the functionality isn’t actually verified; so it can potentially look like you haven’t done any work. This will continue until there is front-end code to interact with the back-end. However, sometimes the integration process can get messy, so it’s best to sprearhead problems before they can occur. Therefore, the next step is to write an automated test to restore the database to a known state, simulate the message occurring, verify the expected response and then verify the database has the correct data. You may require n of these types of tests, until you’re at a  point where you have confidence. From there, are you done? Alas, no. You still need to handle many users, so now it’s time to test many users performing the same thing. You would write an additional automated test, with a high number of users, repeatedly performing the same action: restore db to known state at beginning of test, simulate the message, verify the expected response, and verify correct data for that user… repeatedly for the number of users. Failure can happen at any time, so you might have cases where 8 out of 10,000 attempts fail, and you need to look at this and know why. Until your test passes, you’re not done. Then, until the front-end is making use of your code and integration has happened, nobody has seen it: which can be days, weeks, or longer. So, verification takes a very different route. This can be somewhat spearheaded by having the same developer work on both the front-end and the back-end for a particularly piece of functionality. There’s some merit to this approach, so I generally applaud this attempt.

Of course, this is not to say that there is no testing on the GUI. There often is, but for many companies, it’s about putting the work where there is the most value. Take the case where there’s 8 failures out of 10,000. This sort of scenario can happen on the GUI just as easily. The difference is that it may only occur when a particular user uses the software in a particular way. So, perhaps the issue is only noticed by 1 out of 1,000 users, because the other 999 don’t click as fast, or don’t click as repeatedly. As sad as it is to think about, being a software purist, these are the sorts of issues that are unlikely to get fixed, anyway. The type of testing that sometimes happens for GUI code is unit-tests when proper use of the Model-View Controller design pattern has occurred, and the model and control have been kept relatively framework-neutral. This can be difficult to achieve with some frameworks, so I don’t see unit testing happening on the GUI as much as it should. The second type of testing that often occurs on the GUI is using an automated framework that actually simulates a user clicking various buttons and runs through these scenarios. I see a lot of value in this, but often the software is very pricey. Most companies I’ve worked at have shyed away from this option because of the high cost involved.

So, anyway, hopefully this was an interesting discussion. This certainly isn’t the last discussion I will have about the differences and in future articles, I will talk about more steps to streamlining the process.

· · · · · · · · · · · · · · · ·

Dec/09

16

Characteristics of Software Company Size

I find that it’s something interesting in software companies, that often, depending on the company size, things work very differently. Of course, this can be true for any profession, but for software, it seems to be at a larger scale than other areas of development. In this post, I will go over some general categories and then my experiences and perception for each.  I will also put a general figure for each, as far as size,  which is the size of the company, not specifically the development team. This will go over what I feel is generally more of a typical company of that nature. E.g.: For a 100 employee software company, generally at least 10-20+ developers. If it falls way outside the range, it’s an atypical case, that I won’t be covering here.

Very Small/Tiny (5-15 employees)

Very small companies are generally run as your typical start-up style. That means that process is commonly limited or non-existent. Software tools are kept to only what is absolutely required and what is free. Teams are kept small, as is the company. Communication is widespread, e-mails will often go to the entire company, because there’s little point in excluding the one person who it doesn’t apply to. The group is often close-knit and may even have lunch together every day. Management is thin or non-existent. With these strategies, most overheads are eliminated, and because the team is so small, it can be justified and gotten away with. Software developers are often working on a large piece or the entire product by themselves. Getting code done fast and to market is often more important than designing it well, which may come at the cost of having to rewrite it in a few years, if the company grows, but it’s considered worth the risk

Small (16-80 employees)

Small companies borrow a lot of characteristics from their very small predecessor. Most small companies started out with years under their belt as a very small. This has given the company both good experience and some responsibility to rethink what worked in the past. Some of what worked in the past worked because they were so small and the scale of their product was smaller, so they got away with doing it quickly and without following best practices. In a small company, they will review some of what’s been done and potentially rewrite some pieces with future goals in mind. In small companies, e-mails start to become refined so they won’t go to everyone; the volume would be too high, as to where e-mails would just start getting ignored anyway. Software process goes from none to minimal. A process is generally put in place where management is there, but their job is mostly revolved around staying on top of development, and in some companies, interfacing with the customer. Usage of process is done, but usually not “by the book”, so to speak. Development generally takes a bit longer than very smalls, but the software is generally written a bit more robustly, as company goals become more defined.

Medium (81-300)

Medium-sized companies generally stem from small companies who have grown. Depending on what phase the company is, because there’s a wide-gap in there, it can exibit characters closer to a small company or closer to a large company. Process generally becomes defined better, and they start to try to follow software processes. Managament gains the additional responsibility of enforcing process. Rearchitecture general does not put the company at financial risk. E-mails are sent purely to those who “need to know”. Interaction starts to occur through layers; a software developer generally wouldn’t interact directly with a CEO.

Large (301+)

Large companies are characterized by, a generally, lengthy existence. They have tried many methods and have an idea what has worked for them and what has not. Process is well-defined, and there are a lot of layers of overhead, but these are often required to ensure successful management. Operating processes are generally less efficient, but in the end, they generally put out better tested products (not always). E-mails to the entire company are frowned upon, unless you’re a human resources, or have a title such as VP. Development cycles are generally longer and the company has a long-term roadmap, going out 5 years or more. Many employees will never meet the CEO or president. Developers will generally be specialists. Some developers will specialize in  a GUI technology, another will specialize in a specific-back end technology. They might work with this one technology for years.

Conclusion

So, that’s my overview. There are a few points I want to note for technical accuracy. There are well-defined sizes for companies defined by legal terms, that I specifically avoided adhearing to: This discussion is from my perception. Additionally, as we all should be aware, every company is different. Which means, that some of what I’ve said may not apply to your company. If so, don’t take offense: I don’t work at your company… unless I do, in which case, you should tell me who you are. 😉 Finally, some of these also vary by industry. A large gaming company is likely to be very different than a large defense contractor. I hope you enjoyed the post, and of course, I look forward to your feedback.

· · · · · · · ·

Dec/09

14

Scrum: Agility and Practicality?

In this installment, I wanted to discuss the Scrum software process. I have worked on some Scrum projects and I intend to go through some of my experiences in the following post. Scrum is often referred to, as simply, Agile. However, calling Agile a software process is a bit of a misnomer. Agile is a classification of software processes that share a few common traits: Iterative styles of development and a more fluid ability to react to a change in requirements, strategy or customer needs. Answers.com defines the word agile as being the following: “1. Characterized by quickness, lightness, and ease of movement; nimble. 2. Mentally quick or alert: an agile mind.” So, this flows with the general spirit of Agile. With this in mind, I can share some of my experiences, particularly with the most recent large project I worked on that was using the Scrum methodology.

The first thing to note when discussing this project is that I have worked on other Agile projects before. Generally speaking, however, prior to this one, saying we were doing Agile was more of a catch phrase and Agile wasn’t really being done. Having been trained in Agile and having done my research, in these cases, I was well aware that we weren’t doing Agile, but I believe I was in the minority. Anyway, coming back from that tangent, in this most recent project, the group did make an effort to follow Agile, as it was being dictated by a particular customer. More specifically, we were using Scrum, although most of the way through the process, most of the group was unaware that Agile and Scrum aren’t simply synonyms. An interesting blog article discussing this common problem can be found here.

Anyway, my experience with it wasn’t really a positive one, although I don’t necessarily think I can fault the Scrum process itself. The idea of a daily morning scrum with a small group of people seems like a logical idea. In our case, it wasn’t so effective, because our group was as high as 16 people, including project managers, technical managers, developers and QA, in two different offices. For the overseas office, the meetings were in the afternoon for them. Because of the high quantity of personnel, the meetings took 30 minutes – 1 hour instead of the recommended 10 minutes or thereabouts. In addition, the meetings degraded quickly, with developers discussing specific issues instead of keeping things at a high level, following the general rule of three: 1) What I worked on yesterday, 2) What I will work on today, 3) Blocking issues. Attempts to redirect it back to these three 1-3 sentence overviews weren’t always appreciated.

Again, this isn’t condemnation of Scrum specifically, as I think it had a lot to do with the structure of the meeting. That being said, I think daily scrums can degrade pretty rapidly, and it’s especially at risk, the larger the group. In addition, we typically started in the 9-9:30 AM range, however, the meeting was often at 10 or 10:30 AM. I find I have very good concentration in the morning and breaking this routine had an impact on my morning workflow. The increased communication is important, but I question whether it would’ve been better to end the day with this meeting. I ran into this same problem at my previous company, when we worked with Scrum: The meetings would last too long. In that case, the group was much smaller, but again, developers are developers, and sometimes the moment they think of an issue or interesting concern, they want to discuss it right there, even if it’s not the appropriate forum.

The second critical piece of the Scrum methodology is sprint length. Scrum recommends one month sprints. In the most recent project, sprints were generally two or three weeks at various points during the process, which I felt was too short. The problem with the reduced scope of this sprint length is I find that there’s too quick a push to show results. Sometimes when designing a complicated and major feature that integrates with multiple different system components, this can be too short. Sprints this short require too often a time to leave the system in a stable state to display to the customer and to management. In general, I recommend if you decide to do Scrum that you stick to the suggested one month sprint length.

Some other important artifacts of the Scrum process are the product backlog, user stories and burn down chart. The product backlog makes a lot of sense; unfortunately in our process, the product backlog was not used by upper management. I believe this is due to it being the first time the company had really tried to use Scrum. I think things would’ve flowed a lot better had the product backlog been used effectively. User stories are a tool which seems to be the Scrum replacement for Waterfall-style requirements. The idea of user stories makes sense, but I found that they were very time consuming to write. From here, the user stories derived into the sprint backlog tasks, which were intended to be short tasks between 4 and 16 hours. The final piece is the burn down, which basically gives a simple representation to figure out if you’re going to hit your target for the sprint. It will give you an idea if you’re nearing the danger zone.

One problem I frequently encountered during this sprint backlog tasks is for the tasks, the development teams often over-committed. It’s very easy to look at a small feature and say it will take only four hours, but the four hour estimate was often a best case estimate, without taking any consideration for integration, testing and bug fixing. So, it would generally follow that the team was working 60-70 hour weeks to meet the goals, due to the mis-estimates. Once hitting the mis-estimates became commonplace by upper management, they pushed for the same amount of committing with all sprints, so the development team had dug themselves into a hole, due to the poor estimating.

A critical piece of the scrum process is the sprint planning meeting. During the sprint planning meeting, the goal is to plan the tasks that will be done for the sprint and decide on who will do them. However, part of this process is that the sprint meeting will take no more than 8 hours and that the team will sign up for tasks they will do. However, this frequently was not the process. Mostly, it was a few senior developers and management deciding on who would work on the tasks. As you might have guessed, I was among the deciding group, although I objected to not including everyone. Furthermore, because not everyone was included in the meeting, the meetings often ran way over; I recall one that started in the morning and basically ended about 15 hours later. I generally recommend that there needs to be prepared prior to this meeting and the tasks should flow from the product backlog. Everyone should come into the meeting prepared and with some goals in mind. If people come unprepared, the meeting will drag. If not all the right people are involved, again, the meeting will drag. Unfortunately, no matter what the outcome is, you’re almost guaranteed to lose a full day of development every sprint, just deciding what tasks will be worked on.

The final piece I’d like to mention is that during a sprint, Scrum dictates that tasks shouldn’t be changed. Management cannot dictate priorities for the current sprint. They can change requirements for future sprints, but not the current sprint. In our case, this advice was not heeded. Both management and the customer would frequently change direction during the sprint. This resulted in a lot of rework and a lot of late nights adjusting to the new requirements. I have a feeling that this is commonplace during Scrum, which is part of my distaste for the process itself.

Ultimately, I think that Scrum makes a lot of sense on paper. Having gone through multiple projects which have made some use of Scrum, including one that made heavy use of Scrum, I have not seen any of them work effectively. In a way, Scrum promotes poor development practices, because the focus on rapid change somewhat discourages the practice of thinking a design and architecture through, which can lead to premature gratification that ultimately makes a more difficult to maintain design later. I’m definitely no fan of a strict Waterfall process, but I am confident that Scrum, in the process of fixing some of the flaws of Waterfall, has created some new ones in its place. I do think that education in the usage of any software process, whether it’s Scrum, XP, RUP, CMMI, etc… is essential to things working smoothly. The fact is that when faced with deadlines and pressure, people will revert to their most familiar behavior, some of which does not play with a process like Scrum. On a side note, a few developers I’ve worked with in the past have been very fond of the XP process. I have some doubts about XP, as well, but I can say that when I have used it, I have seen some benefits.

· · · · · · · · · · · · · · · ·

Nov/09

28

“The Zone”

One of the mysteries of software development, as well as many other professions is the concept of “The Zone”.  This is also known as “The Flow”. Wikipedia defines it as this, “Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity.” You can witness this in a sport such as basketball. I recall quotes from Michael Jordan, in the 90s, about a game he had in the NBA finals, and how he was in the zone, as he singlehandedly led his team to victory.

The fact is that when you’re in the zone, you’re performing way above your normal state, things come naturally, thought is mostly unconscious, and you have extreme focus, which ultimately lets you process things far more quickly and effectively. Ultimately, developers get in the zone too. I know when I’m in the zone, I’ve done multiple days work in what would otherwise take a few hours. This usually happens for a combination of reasons: being well rested, interest in what I’m doing, determination, and a concept of a goal that I want or need to achieve.

I know other developers I’ve worked with often talk about a similar process. A mentor of mine from years back once said that he often starts a project slowly. Then, when he’s working, suddenly things just “click” and he quickly makes up for lost time, and that’s why his performance was good. I think this happens to a lot of us. I believe that the moment when things “click” and suddenly in the course of a few days you get a massive amount of work done is “The Zone”. In software development, the zone doesn’t happen all the time, and it needs to be realized that it’s important when it does.

Breaking the Zone

One of the challenges is the zone is working in an office workplace, where there are potentially a lot of distractions to remove you from the zone.  Mostly, I find these to be other employees.  Ironically, the lower you are in the chain, the more zone opportunities you will have, because generally you have less points of contact.  Each time, as a developer, I’m interrupted, it can be as short as 10 minutes and as long as an hour before I re-enter “The Zone”.  This kind of productivity loss should be considered unacceptable for most companies.  If you’re a company saving a few dollars by keeping the employees crunched together, shame on you.  It provides more stress for your developers, who are often, not a social bunch to start with.

Keeping the Zone

Obviously, as Joel Spolsky suggests, giving each person their own office is a big help, because it reduces the temptation to distract others.  This is a huge benefit, but it’s probably not possible for every company.  In general, I think high wall cubicles are also fine.  Noise levels should be kept in check by management and policy: Keep outloud music to a minimum, personal conversations should be taken elsewhere, and impromptu meetings should be a rarity and should occur in conference rooms.  This policy should be enforced and if someone is violating it, they should be asked to be more considerate.  Having inconsiderate neighbors is never a good experience.

Another thing that helps is investing a good set of headphones.  For some people, it’s best to get a studio-style headset or gamer headset, which will block out almost all noise.  If you want to be aware so you can hear nearby voices, but ignore most sounds, you can use a cheaper headset.  Again, though, note that this can be an unnecessary distraction for you, but can definitely be a tool for the paranoid.

Conclusion

“The Zone” is an psychological state every developer loves getting into.  The work flows faster, better, and with higher quality and focus.  It is sometimes difficult to stay in the zone when working due to outside distractions, but with some careful techniques, companies and individuals can make it easier to stay in “The Zone”.


· · · · ·

Theme Design by devolux.nh2.me