I hope everyone is enjoying their holidays. With the holidays, I haven’t had much opportunity to write new posts, but I will have some additional new ones soon enough. Hope everyone is doing well. In the meantime, let me know if there’s any topics that you would like to see covered, in any areas of development, or perhaps reviews of dev. books, or anything like that.

No tags

I find that it’s something interesting in software companies, that often, depending on the company size, things work very differently. Of course, this can be true for any profession, but for software, it seems to be at a larger scale than other areas of development. In this post, I will go over some general categories and then my experiences and perception for each.  I will also put a general figure for each, as far as size,  which is the size of the company, not specifically the development team. This will go over what I feel is generally more of a typical company of that nature. E.g.: For a 100 employee software company, generally at least 10-20+ developers. If it falls way outside the range, it’s an atypical case, that I won’t be covering here.

Very Small/Tiny (5-15 employees)

Very small companies are generally run as your typical start-up style. That means that process is commonly limited or non-existent. Software tools are kept to only what is absolutely required and what is free. Teams are kept small, as is the company. Communication is widespread, e-mails will often go to the entire company, because there’s little point in excluding the one person who it doesn’t apply to. The group is often close-knit and may even have lunch together every day. Management is thin or non-existent. With these strategies, most overheads are eliminated, and because the team is so small, it can be justified and gotten away with. Software developers are often working on a large piece or the entire product by themselves. Getting code done fast and to market is often more important than designing it well, which may come at the cost of having to rewrite it in a few years, if the company grows, but it’s considered worth the risk

Small (16-80 employees)

Small companies borrow a lot of characteristics from their very small predecessor. Most small companies started out with years under their belt as a very small. This has given the company both good experience and some responsibility to rethink what worked in the past. Some of what worked in the past worked because they were so small and the scale of their product was smaller, so they got away with doing it quickly and without following best practices. In a small company, they will review some of what’s been done and potentially rewrite some pieces with future goals in mind. In small companies, e-mails start to become refined so they won’t go to everyone; the volume would be too high, as to where e-mails would just start getting ignored anyway. Software process goes from none to minimal. A process is generally put in place where management is there, but their job is mostly revolved around staying on top of development, and in some companies, interfacing with the customer. Usage of process is done, but usually not “by the book”, so to speak. Development generally takes a bit longer than very smalls, but the software is generally written a bit more robustly, as company goals become more defined.

Medium (81-300)

Medium-sized companies generally stem from small companies who have grown. Depending on what phase the company is, because there’s a wide-gap in there, it can exibit characters closer to a small company or closer to a large company. Process generally becomes defined better, and they start to try to follow software processes. Managament gains the additional responsibility of enforcing process. Rearchitecture general does not put the company at financial risk. E-mails are sent purely to those who “need to know”. Interaction starts to occur through layers; a software developer generally wouldn’t interact directly with a CEO.

Large (301+)

Large companies are characterized by, a generally, lengthy existence. They have tried many methods and have an idea what has worked for them and what has not. Process is well-defined, and there are a lot of layers of overhead, but these are often required to ensure successful management. Operating processes are generally less efficient, but in the end, they generally put out better tested products (not always). E-mails to the entire company are frowned upon, unless you’re a human resources, or have a title such as VP. Development cycles are generally longer and the company has a long-term roadmap, going out 5 years or more. Many employees will never meet the CEO or president. Developers will generally be specialists. Some developers will specialize in  a GUI technology, another will specialize in a specific-back end technology. They might work with this one technology for years.

Conclusion

So, that’s my overview. There are a few points I want to note for technical accuracy. There are well-defined sizes for companies defined by legal terms, that I specifically avoided adhearing to: This discussion is from my perception. Additionally, as we all should be aware, every company is different. Which means, that some of what I’ve said may not apply to your company. If so, don’t take offense: I don’t work at your company… unless I do, in which case, you should tell me who you are. Finally, some of these also vary by industry. A large gaming company is likely to be very different than a large defense contractor. I hope you enjoyed the post, and of course, I look forward to your feedback.

Company, Company Size, Development, Large Companies, Medium Companies, Programming, Small Companies, Software, Very Small Companies

In this installment, I wanted to discuss the Scrum software process. I have worked on some Scrum projects and I intend to go through some of my experiences in the following post. Scrum is often referred to, as simply, Agile. However, calling Agile a software process is a bit of a misnomer. Agile is a classification of software processes that share a few common traits: Iterative styles of development and a more fluid ability to react to a change in requirements, strategy or customer needs. Answers.com defines the word agile as being the following: “1. Characterized by quickness, lightness, and ease of movement; nimble. 2. Mentally quick or alert: an agile mind.” So, this flows with the general spirit of Agile. With this in mind, I can share some of my experiences, particularly with the most recent large project I worked on that was using the Scrum methodology.

The first thing to note when discussing this project is that I have worked on other Agile projects before. Generally speaking, however, prior to this one, saying we were doing Agile was more of a catch phrase and Agile wasn’t really being done. Having been trained in Agile and having done my research, in these cases, I was well aware that we weren’t doing Agile, but I believe I was in the minority. Anyway, coming back from that tangent, in this most recent project, the group did make an effort to follow Agile, as it was being dictated by a particular customer. More specifically, we were using Scrum, although most of the way through the process, most of the group was unaware that Agile and Scrum aren’t simply synonyms. An interesting blog article discussing this common problem can be found here.

Anyway, my experience with it wasn’t really a positive one, although I don’t necessarily think I can fault the Scrum process itself. The idea of a daily morning scrum with a small group of people seems like a logical idea. In our case, it wasn’t so effective, because our group was as high as 16 people, including project managers, technical managers, developers and QA, in two different offices. For the overseas office, the meetings were in the afternoon for them. Because of the high quantity of personnel, the meetings took 30 minutes – 1 hour instead of the recommended 10 minutes or thereabouts. In addition, the meetings degraded quickly, with developers discussing specific issues instead of keeping things at a high level, following the general rule of three: 1) What I worked on yesterday, 2) What I will work on today, 3) Blocking issues. Attempts to redirect it back to these three 1-3 sentence overviews weren’t always appreciated.

Again, this isn’t condemnation of Scrum specifically, as I think it had a lot to do with the structure of the meeting. That being said, I think daily scrums can degrade pretty rapidly, and it’s especially at risk, the larger the group. In addition, we typically started in the 9-9:30 AM range, however, the meeting was often at 10 or 10:30 AM. I find I have very good concentration in the morning and breaking this routine had an impact on my morning workflow. The increased communication is important, but I question whether it would’ve been better to end the day with this meeting. I ran into this same problem at my previous company, when we worked with Scrum: The meetings would last too long. In that case, the group was much smaller, but again, developers are developers, and sometimes the moment they think of an issue or interesting concern, they want to discuss it right there, even if it’s not the appropriate forum.

The second critical piece of the Scrum methodology is sprint length. Scrum recommends one month sprints. In the most recent project, sprints were generally two or three weeks at various points during the process, which I felt was too short. The problem with the reduced scope of this sprint length is I find that there’s too quick a push to show results. Sometimes when designing a complicated and major feature that integrates with multiple different system components, this can be too short. Sprints this short require too often a time to leave the system in a stable state to display to the customer and to management. In general, I recommend if you decide to do Scrum that you stick to the suggested one month sprint length.

Some other important artifacts of the Scrum process are the product backlog, user stories and burn down chart. The product backlog makes a lot of sense; unfortunately in our process, the product backlog was not used by upper management. I believe this is due to it being the first time the company had really tried to use Scrum. I think things would’ve flowed a lot better had the product backlog been used effectively. User stories are a tool which seems to be the Scrum replacement for Waterfall-style requirements. The idea of user stories makes sense, but I found that they were very time consuming to write. From here, the user stories derived into the sprint backlog tasks, which were intended to be short tasks between 4 and 16 hours. The final piece is the burn down, which basically gives a simple representation to figure out if you’re going to hit your target for the sprint. It will give you an idea if you’re nearing the danger zone.

One problem I frequently encountered during this sprint backlog tasks is for the tasks, the development teams often over-committed. It’s very easy to look at a small feature and say it will take only four hours, but the four hour estimate was often a best case estimate, without taking any consideration for integration, testing and bug fixing. So, it would generally follow that the team was working 60-70 hour weeks to meet the goals, due to the mis-estimates. Once hitting the mis-estimates became commonplace by upper management, they pushed for the same amount of committing with all sprints, so the development team had dug themselves into a hole, due to the poor estimating.

A critical piece of the scrum process is the sprint planning meeting. During the sprint planning meeting, the goal is to plan the tasks that will be done for the sprint and decide on who will do them. However, part of this process is that the sprint meeting will take no more than 8 hours and that the team will sign up for tasks they will do. However, this frequently was not the process. Mostly, it was a few senior developers and management deciding on who would work on the tasks. As you might have guessed, I was among the deciding group, although I objected to not including everyone. Furthermore, because not everyone was included in the meeting, the meetings often ran way over; I recall one that started in the morning and basically ended about 15 hours later. I generally recommend that there needs to be prepared prior to this meeting and the tasks should flow from the product backlog. Everyone should come into the meeting prepared and with some goals in mind. If people come unprepared, the meeting will drag. If not all the right people are involved, again, the meeting will drag. Unfortunately, no matter what the outcome is, you’re almost guaranteed to lose a full day of development every sprint, just deciding what tasks will be worked on.

The final piece I’d like to mention is that during a sprint, Scrum dictates that tasks shouldn’t be changed. Management cannot dictate priorities for the current sprint. They can change requirements for future sprints, but not the current sprint. In our case, this advice was not heeded. Both management and the customer would frequently change direction during the sprint. This resulted in a lot of rework and a lot of late nights adjusting to the new requirements. I have a feeling that this is commonplace during Scrum, which is part of my distaste for the process itself.

Ultimately, I think that Scrum makes a lot of sense on paper. Having gone through multiple projects which have made some use of Scrum, including one that made heavy use of Scrum, I have not seen any of them work effectively. In a way, Scrum promotes poor development practices, because the focus on rapid change somewhat discourages the practice of thinking a design and architecture through, which can lead to premature gratification that ultimately makes a more difficult to maintain design later. I’m definitely no fan of a strict Waterfall process, but I am confident that Scrum, in the process of fixing some of the flaws of Waterfall, has created some new ones in its place. I do think that education in the usage of any software process, whether it’s Scrum, XP, RUP, CMMI, etc… is essential to things working smoothly. The fact is that when faced with deadlines and pressure, people will revert to their most familiar behavior, some of which does not play with a process like Scrum. On a side note, a few developers I’ve worked with in the past have been very fond of the XP process. I have some doubts about XP, as well, but I can say that when I have used it, I have seen some benefits.

Agile, Backlog, Burn Down, Burndown, Development, Estimates, Estimation, management, Product Backlog, Programming, Scrum, Software, Sprint backlog, Sprint planning meeting, Sprints, Waterfall, XP

Hi all. In this edition, I want to discuss a C++ logging system a little bit. I’ve been through the process of developing loggers many times in my career. Each time they get iteratively better, and so I feel this would be an interesting topic to discuss. Now, of course, you could just use a logger from an already existing library, but then there would be no point in this article. So, let’s play along for my sake.

First Approach

Now, obviously, the first thing a logger should do is… log!  So, a basic first step might have our logger looking like this:

class Logger
{
public:
	void log(const std::string& p_output)
	{
		file_ << p_output;
	}
};

Logger& operator<<(Logger& p_logger, const std::string& p_output)
{
	p_logger.log(p_output);
	return p_logger;
}

Simple enough, and effective. However, there are obviously some improvements we could make:

1. Logging of additional information: file, method name, line.
2. Scoped logging.
3. Conditional logging. The point of this would be to log only if a message is important enough to log.
4. Effect on application performance. Asynchronous logging.
5. Registering different output sources (Difficulty: Intermediate-Hard. Mostly because it hits up a part of the standard most people avoid. Unfortunately, out of the scope of this article, but can write a follow-up if there’s interest.)

Logging of Additional Information

It’s always very helpful if we can get more information when we’re logging. A log message by itself is often useless, since it may have been called from a variety of different points or the message may be repeated. By providing the right information in the right places, we can help alleviate the first issue and remove any doubt for the second. By putting logging in areas of interest, you might have 5 log messages, which happen to precede the call location where the error message you’re really interested is occurring. Of course, a full stack trace in C++ is more effective, but this, while doable, is way beyond the scope of this article.

The output method might be changed to this:

class Logger
{
public:
	void log(const char* p_pFile, const char* p_pMethod,
		size_t p_line, const std::string& p_output)
	{
		file_ << p_pFile << ":" << p_pMethod << ":" << p_line << ": " << p_output;
	}
};

// Called like this:
logger.log(__FILE__, __FUNCTION__, __LINE__, ... msg ... );

This is certainly starting to get a bit verbose, but you could potentially cover this with a macro, if i’s a win in this case. Unfortunately, operator<< is eliminated, but there is potential trickery you could get to reuse the solution, but it’s probably fine to not have it in this case. Besides, log() is more readable than << for C++ newbies.

Scoped Logging

This is my favorite and most important feature for any logging system. In C++, it’s useful to log not only when logging of a scope started, but also when it ended. If you can scope the log calls, then information will be shared between the constructor and destructor of this auto log object and the destructor is guaranteed to be called. Here’s an example:

class AutoLogger
{
public:
	AutoLogger(Logger& p_.logger, const char* p_file,
		const char* p_function, size_t p_line, const std::string& p_msg) :
		logger_(p_logger), file_(p_file), function_(p_function),
		line_(p_line), msg_(p_msg)
	{
		logger_.log(file_, function_, line_, msg_);
	}

	~AutoLogger()
	{
		logger_.log(file_, function_, line_, msg_);
	}
};

AutoLogger autoLog(logger, __FILE__, __FUNCTION__, __LINE__, ... msg ...);

... do stuff ...

// destructor automatically called, so it logs at exiting the scope.

This has a lot of benefits. The main thing is the guaranteed and saved typing for executing potentially the same conditions. You also potentially gain the ability to have the scoped version log additional information, such as time deltas, memory deltas, etc… this can even be a weak form of profiling.

Conditional Logging

You could add an optional log type which is a conditional. When provided, it will check the logger’s conditional and see if it’s allowed. If not, it won’t log. This is purely  meant as an efficiency, and preventing log files from getting ridiculously large, while keeping old log messages in tact and not having to change otherwise working code. As an example:

class Logger
{
public:
	void log(const char* p_pFile, const char* p_pMethod, size_t p_line,
		LogCondition p_logCondition, const std::string& p_output)
	{
		if (p_logCondition is allowed)
		{
			file_ << p_pFile << ":" << p_pMethod <<
				":" << p_line << ": " << p_output;
		}
	}
};

// Called like this:
// The following line gets logged
logger.log(__FILE__, __FUNCTION__, __LINE__, MUST_LOG, ... msg ... );

// This one, however, is not logged
logger.log(__FILE__, __FUNCTION__, __LINE__, TRIVIAL, ... msg ... );

Conditional logging holds the advantages that I mentioned above and is very easy to create. Just fill in with the types you need. In this case, I mentioned MUST_LOG and TRIVIAL.

Asynchronous Logging

Now, if we implement what we’ve gone through so far, we’ve got a decent logging system. Unfortunately, put enough logging in place, and your application is slowed down to the speed of the hard disk, at best. This is absolutely unacceptable, of course. So, the obvious solution is to create a thread or pool of threads to handle disk writes to the system. When you make a log call, it actually pushes the “request” on the queue and then allows the logging system to take care of the request when it is free. This prevents your application being slowed down due to logging (if done with care).   Without this, any multithread application would become a single-threaded application which is basically synchronized on files. Now, at this point, you’re probably thinking that we’ve fixed the last major issue. Almost…

Maximum Queue Size

This hit me once when working on a server application. Basically, the story behind it is you almost always won’t be logging a ton of information in your regular application. You only start logging at key points and if things are starting to look “fishy” (e.g.: unexpected value in method Blah::Blah2()). However, on a server application, things scale quickly with load and then are limited based on hardware. Ultimately, a load test with 2000 simulated users worked in a testing environment. However, at around 800 users in a live environment, we started to hit what appeared to be increasing memory leaks, which is never good. To make a long story short, it turned out that the live system was using RAID and the testing environment wasn’t. So, because the hard disk was considerably slower, it hit hard disk write limits much quicker. This issue was easily corrected by providing a maximum queue size. If you’re queueing, say, 10k messages, you then hit a decision path for how you’d like to handle: toss away messages until the queue shrinks to say, 8k, or start having the logger sync with the app. The first solution is easier to code, so we went with that one.

Ultimately, this article discussed writing a basic logger from scratch with example code. Hopefully you found it useful.

C++, Design, Logging, OOP

If you’re familiar with Object-Oriented programming, and potentially, UML, these terms may be familiar to you. If not, I will present them below.

Inheritance is a relationship between two classes, where there is a parent-child relationship, in the sense that the parent appears higher on the tree than the children and a parent may have multiple children. However, the relationship follows than the children inherit all the parent’s state and behavior and may also additionally add their own. I’ll demonstrate an example with UML, done with SmartDraw, and some pseudo-code:

class Shape
{
public:
	virtual void draw() = 0;
};

class Circle : public Shape
{
public:
	Circle(const Pos2f& p_pos, float p_radius) :
		pos_(p_pos), radius_(p_radius)
	{
	}

	virtual void draw()
	{
		// perform draw operation based on position and radius
	}

private:
	Pos2f pos_;
	float radius_;
};

class Square : public Shape
{
public:
	Square(const Pos2f& p_topLeftPos, const Dim2f& p_dimensions) :
		topLeftPos_(p_topLeftPos), dimensions_(p_dimensions)
	{
	}

	virtual void draw()
	{
		// perform draw operation based on top-left position, width and height
	}

private:
	Pos2f topLeftPos_;
	Dim2f dimensions_;
};

This relationship makes sense as a class hierarchy because there’s a simple guideline to determine whether a class should be inherited from another. If you can justify that a class “is-a” more specific type of another type, then semantically, it makes sense to inherit, if not it doesn’t. Applying it to this case, a Circle is-a type of Shape. A Square is-a type of Shape. So, therefore, inheritance is appropriate.

This brings up the question, what do we do when inheritance isn’t appropriate? There is also object composition. Object composition basically means that a type or class is “composed” of other types or classes. If you make use of object composition it looks more like this:

Here’s the UML diagram, done with SmartDraw:

class Duck
{
public:
	void quack()
	{
		// logic to quack
	}

	void fly()
	{
		// logic to fly
	}

private:
	Bill duckBill_;
	std::vector<Feather> feathers_;
};

class Bill
{
...
};

class Feather
{
...
};

class Pond
{
...

private:
   Duck* pDuck_;
};

This example demonstrates what it might look like for code that involves a Duck. A duck has-a bill, and has-some feathers. When a relationship between two classes is a has-a relationship, then you should use composition. There are two types of composition, which are demonstrated in the code and diagram above. The more general form of composition implies full ownership, including responsibility for an object’s lifetime. This is true for the bill of a duck and its feathers, for without a instance of a duck object, neither the feathers nor bill exist. (Side note: This is assuming we can’t pluck the feathers of the duck or remove its bill, which would get into a concept called Transfer of Ownership, but that is beyond the scope of this article.) The second type of composition is called aggregation, which implies that while an object has another object, it doesn’t own it. In the example I showed above, the Pond has this sort of relationship with the Duck. An instance of a Pond object may keep a reference (or pointer) to the Duck object while the Duck is in the Pond, but isn’t responsible for managing the lifetime of the Duck object, so it is a aggregation relationship.

Now that I’ve clarified inheritance vs. composition a bit, it brings up some additional questions. In the case of the Duck, a common design problem I’ve seen before is that in the case of a one-to-one relationship, like there is between the Duck and the Bill, you could theoretically derive Duck from the Bill class. The argument I’ve heard is, “This violates the is-a rule, but really what is the big deal? I was going to promote the methods of Bill to be public on Duck anyway? This saves typing.” The problem is semantic understanding. This type of inheritance is generally meant to be a polymorphic relationship. Polymorphism requires a is-a relationship to make sense. Now, there are cases where it makes sense to inherit when there isn’t a is-a relationship, but generally you should avoid it, when it’s obviously wrong. A codebase that violates is-a/has-a rules is much more difficult to understand than one that follows it. Ultimately, it just flows more naturally, and you wind up avoiding some dead ends that code that violates this runs into.

In a future article I will discuss some of the more grey area cases. Hopefully, you found this to be a good introduction.

Aggregation, C++, Composition, Generalization, Inheritance, Object-Oriented Design, Object-Oriented Programming, OOD, OOP, UML

So, one thing you might be wondering is why I titled my blog,The Software Purist. One friend even surmised that it had to do with some clensing program where I would annihilate all the impure programmers. While a humorous suggestion, it wasn’t what I was aiming for.   The long and short of it is that at a previous job, two different managers used to refer to me as a C++ purist, for my take on approaching tackling issues when programming in the C++ language. It was generally meant more as a compliment, because I believe they respected me “sticking to my guns”, so to speak. My general mentality at the time is that all problems can be solved by using well-defined methods, best practices and always maintaining a preference for anything standard or that has the possibility of becoming standard in the future (such as some of the Boost libraries). So, if there was an approach, my general methodology was a question of, “How would Scott Meyers solve this problem?” It’s difficult to get more pure than following Scott Meyer’s advice in Effective C++, at least at the time.

Now that we’ve been through that intro, perhaps some definitions, from my perspective would be useful. There’s two camps of extremes in software development. First, there’s the purists. Purism is about following language standards, following best practices, avoiding legacy features, ignoring language loopholes and using the language as intended by its authors. For example, a purist, in C++, might completely avoid and ignore C++ macros, except when necessary for things, such as header guards. A C++ purist might also prefer to use std::copy instead of memcpy, even if either solution would work, performance was equal, but memcpy is outdated. A C++ purist would use std::string instead of MFC’s CString or Qt’s QString. I know, because I did this and generally stick to this advice, unless it gets in the way.

Pragmatism is exactly the opposite. Pragmatism dictates that the most important thing is getting the job done. If you’re a pragmatist, your goal is generally to get the product out the door. So, if macros streamline your process and make it quicker to code something, this is more important than following a set of recommendations, because you can get your product out the door faster. Likewise, memcpy is a few characters less typing than std::copy and you’re more used to it, so you use it over std::copy, even though iterators are considered “safer” than pointers. Finally, you might use CString, because it gives you direct access to the internal memory, so you don’t have to wrestle through an abstraction and you can use a C-style API if you choose.

Now, these are all fair and valid views. Yes, that’s right. Both views are fair, both are valid. Both are also extreme. We know from many aspects of life, that extremes are generally bad. A temperature of too hot or too cold is uncomfortable. Driving at a speed of too fast or too slow is uncomfortable. And so on… The same holds true for software.  Ultimately, we all evolve as developers. I have a theory that many developers start out as purists and start to migrate towards gaining more pragmatism each year, once they become more seasoned with more business experience. Anyway, as most developers know, it can be a problem to be either too pure or too close to pragmatism.

If you’re too pure, you will probably think every pragmatist’s code is terrible, even to the point of saying that they’re “under-engineering”. I know, because I was there, a long time ago. In some situations, what’s called for is simply getting the job done. Businesses have a need to have a product ship and a product that doesn’t ship is a failure, even if the code was beautiful. Purism often has a large focus on long term goals, which is definitely beneficial. The secret knowledge that purists don’t want to let out is by following purist methodology: 1) The coding becomes very mechanical and repetetive, which is great for developing, because if you can reuse patterns of development, it gets easier and becomes more natural each time. 2) The purist has a keen sense and sensitivity to maintaining the code later and they know if they take a shortcut now, they will be grunting and groaning when they have to look at it later. The truth is that these are really valid points, and in a lot of situations, this is the right approach. Of course, there’s some items that have to be compromised in the name of getting things done. On the flip side…

If you’re too pragmatic, you will probably think every purist is overengineering. Why build a packet class for every packet to the server? You can more easily code it inline, in one large source file, right? The problem with this approach is when you need to maintain it later, such as putting it in a test harness, all of this hardcoded logic becomes a mess to fit in. It’s difficult to extract the 80 out of 200 lines of a large function that you actually want to test, while stubbing out the other 120 — this necessitates refactoring. Both extremes find that refactoring is time consuming. Extreme purists find that in reality, it’s difficult to find time to refactor, so they try to code in such a way to avoid this step. Extreme pragmatists find that it’s difficult to find time to ever refactor, so they just never bother with it and the code is messy forever. Refactoring is one of those concepts that works is good in practice, but unless you get everyone to buy in, it doesn’t happen. Extreme pragmatists often don’t buy into refactoring; they got used to the mess, and have developed a mental map of the shortcuts, so they can often work effectively, despite the challenges. Extreme pragmatism creates a potentially difficult work environment for coworkers when done to extremes, because it becomes a mind field for the uninformed to trip over.

Ultimately, as we know, any sort of extremist views should generally be avoided. There is never always a single answer to any problem. Development has to be done with care and the beauty of the code is imoprtant. However, don’t lose sight of actually shipping the product. There must be a balance. If you feel like you are in the middle and you are accused of either overengineering or underengineering, it’s very possible that the person you’re talking to is an extremist. As for The Software Purist, my general approach now is to stay somewhere in between, but I still lean a bit towards the purist side, because ultimately, I have a high appreciation for the beauty of code.

Boost, C++, Java, methodology, MFC, packet, Pragmastist, Pragmatism, Purism, Purist, Qt, server, STL

I’ve decided to start a new column for Mondays to discuss some music that I find that helps me concentrate while developing. The first entry in the series is the album, Once, by Nightwish. Nightwish is a symphonic metal and power metal band from Finland. Once came out in 2004.  This is the last cd that Nightwish did with then, lead singer, Tarja Turunen, who was later let go from the group, while on tour, and has since now gone solo.  Most Nightwish fans will agree that Tarja Turunen has one of the most amazing voices.  Along with Tarja, the album also features Marco Hietala, who does some additional vocals and compliments Tarja very well in some of the songs.  Of all of Nightwish’s albums, this one has the most songs that you can just listen to again and again.  I don’t find myself skipping anything on this cd.

The track listing is as follows:

1.     ”Dark Chest of Wonders”
2.     ”Wish I Had an Angel”
3.     ”Nemo”
4.     ”Planet Hell”
5.     ”Creek Mary’s Blood”
6.     ”The Siren”
7.     ”Dead Gardens”
8.     ”Romanticide”
9.     ”Ghost Love Score”
10.     ”Kuolema Tekee Taiteilijan”
11.     ”Higher Than Hope”
12.     ”White Night Fantasy”
13.     ”Live to Tell the Tale”

I’ve found that software developers are especially into good music. Many of us sit at desks all day, so we need something that helps keep us focused, while also making the day just a little less monotonous. Symphonic metal is an interesting genre. With Nightwish and this genre, you will find a combination of vocals, keyboards, piano, guitars, drums, percussion, bass. They flow together in an enchanting way and just seem to make the day go by a bit more smoothly.

On the cd, I like all of the songs, but I’ll go thorugh a few of my favorites. Ghost Love Score is utterly epic.  It’s beautiful. Tarja meshes her beautiful voice with some amazing background music. The song itself is 10 minutes long. What’s great about songs like this is you don’t really notice them when you’re programming, it just flows nicely and you find yourself a bit happier while it’s playing. Nemo is the first song off of the album. The song alludes to either Captain Nemo or possibly to a name which can be translated to “Nobody” (according to a wiki entry). I originally thought perhaps it was referring to Nemo from Finding Nemo, but this does not appear to be the case.  Anyway, it is one of the most successful songs for Nightwish, andis very powerful. The vocals are just amazing. Finally, I will mention Wish I Had an Angel, which has a slightly different feel of some of the other songs on the album, but is still great in it’s right.  To date, I believe this song charted the highest on the US charts of any Nightwish song and also was featured in a few movie soundtracks.

Ultimately, this is a great album and one that hopefully you will enjoy.  I’m hoping to make this a weekly column on Mondays, so let me know if you like it. For this sort of a column, I’m also willing to consider articles written by others. If you want to write an article recommending an interesting album that you find eases your daily development tasks here, just send me an email and we can discuss further.

MM, Music, MusicMonday, Nightwish

If you’re an experienced programmer, this probably won’t be new information, but I hope to at least present it in a new way.  When developing software programs, there’s different ways to think about the problems you’re trying to solve, which affect the entire process from initial design to how it’s coded, even to how it’s tested.  I discuss a few of these in this article.

Unstructured

These days, unstructured styles of programming are generally frowned upon. In the old days, you might have programmed unstructured in older dialects of Fortran, COBOL, Basic, etc…, and used GOTO to move between sections of code. All variables were global to the program and so you had to be very careful about the usage and naming. This type of code was simple to code at first, but very difficult to read. In addition, it didn’t scale well at all. As programs got larger, it became exponentially more difficult to maintain. There isn’t much to talk about here, because coding in this manner is rare nowadays, except in specialized fields. You can imagine a program looking very sequential, though. Something like this:

if my height is less than 4'
goto short
elif my height is greater than 6'
goto tall
else
goto normal

short:
print "You are too short to go on this ride"
tall:
print "You must wear a helmet."
... offer helmet ...
if accept
goto getonride
else
print "No dice..."
normal:
goto getonride

getonride:
print "Welcome onto the ride."

exit:
print "You must leave.  NOW!"

As you can see, this can quickly get out of control. Reuse was almost non-existent.

Procedural Programming/Imperative Programming

Procedural programming was the first type of structured programming and it started to become widespread in the late 60s and early 70s.  It was probably the first major step towards programming we do today.  Structured programming is still used quite a bit and is the basis for some of the later programming paradigms.  Structural programming is responsible for mostly eliminating the widespread use of GOTO.  This methodology was more commonly taught at the time I was in college, as Object-Oriented programming was newer at the time, and not very well understood outside of a smaller community.  The main concepts behind is that any task can be broken up into sub-tasks.  Emphasis is placed on functionality and data structures.  With this, it is became easy to break down a workflow with direct relationships and traceability for a functional specification, often, provided by a customer.  This directly can be derived into software functional requirements, and then directly derived into software code, and then directly derives into tests.  Because all of the emphasis is on functionality, and the code is structured in that manner, it provides a lower barrier to entry than newer techniques such as Object-Oriented Programming. Due to this, the common tool for figuring out where a piece of code is implemented is simply a matter of using grep (on Linux/Unix) or find (on Windows).  Data definitions can be provided by systems engineers, because once the functionality is defined, the data required is also easily derived.

Some of the common procedural-oriented programming languages are Ada-83, Algol, Pascal and C.  Of course, at different points, many of the procedural programming languages later gained Object-Oriented features with new revisions (Ada-95, C++, etc…).  One of the main problem with structural programming concerns reuse.  You can successfully meet functional requirements, but later notice that different components of the system have 95% similar functionality.  It becomes difficult to directly express these relationships in your code.  To try to handle reuse of sections of code where there can be different types used, you wind up with large if and/or switch statements.  The problem then becomes that for each new type that supports this relationship, you wind up modifying working code, which is always risky.  Modifying working code makes it difficult to supply a working library because elements in the library will often be changed.

As an example, let’s take the case of a vehicle and then provide multiple types of vehicles, a car and a bus. Example in C:

void drive(int type, void* obj)
{
	switch (type)
	{
		case CAR:
		{
			Car* car = (Car*)obj;
			// ... Logic to accelerate car
		} break;
		case BUS:
		{
			Bus* bus = (Bus*)obj;
			// ... Logic to accelerate bus
		} break;
		default:
		{
		} break;
	}
}

Later, we provide a boat:

void drive(int type, void* obj)
{
	switch (type)
	{
		case CAR:
		{
			Car* car = (Car*)obj;
			// ... Logic to accelerate car
		} break;
		case BUS:
		{
			Bus* bus = (Bus*)obj;
			// ... Logic to accelerate bus
		} break;
		case BOAT:
		{
			Boat* boat = (Boat*)obj;
			// ... Logic to accelerate boat
		} break;
		default:
		{
		} break;
	}
}

As you can probably see, it’s relatively easy to figure out where to insert the code, but the maintenance of this can increase quickly, if you take into account that each function, such as drive, park, accelerate, addFuel, etc… would each need this sort of switch statement. You would wind up changing a lot of working code.

Object-Oriented Programming

Object-Oriented programming could be considered the next phase in the evolution of programming languages.  It largely gained popularity due to C++ (formerly C with Classes).  Object-Oriented development changes the emphasis.  The emphasis in Object-Oriented programming is not with defining the functionality of the system and the data.  Instead of putting the emphasis on the data of the system, you start out by identifying the objects in the system.  So, imagine a game, such as the original Super Mario Bros.  You could identify objects such as your main character (Mario, Luigi), the enemies in the world (Goombas, Koopa Troopas, Bowser, etc…), the blocks, the pipes, moving platforms, and even the world itself.  The functionality is tied in when the objects communicate with each other. In technical terms, this communication is called messaging. The functions are owned by objects, and are called methods, instead of functions. This ownership is based on something being able to do something else.  For example, Mario can jump, so Mario might have a method called jump().  Mario can also shoot fireballs, so he would have a method called shoot().  Since Mario and Luigi are the same, they might simply be two separate object instances of the same class called Player.  The enemies have some similarities, so they could be structured with a base class called Enemy and derived classes, which implement the different functionality.  It’s a different way of thinking about things.

Now what I’ve described so far might not make sense if you’re not proficient with Object-Oriented programming, so let me go back to the Vehicle example.  Here’s what it would look like in OO-terms:

class Vehicle
{
public:
	virtual void drive() = 0;
};

class Car : public Vehicle
{
public:
	virtual void drive()
	{
		// ... Logic to accelerate car
	}
};

class Bus : public Vehicle
{
public:
	virtual void drive()
	{
		// ... Logic to accelerate bus
	}
};

class Boat : public Vehicle
{
public:
	virtual void drive()
	{
		// ... Logic to accelerate boat
	}
};

In it’s most simple form, OO is simply a reorganization of code. However, it is obviously much more than this and this is a very simple example, which doesn’t touch all of the depths of how far things can go, but I think is fine to start. When you say that a Car is derived from a Vehicle, you are effectively saying that a Car is-a Vehicle. This is the basis for this type of inheritance. You should only derive if you can logically say that something is something else. For example, you shouldn’t have Bus derive from Boat, because a bus is-not-a boat.

So, if you look at the above example, I think you can see how this flows really nicely for things like GUIs. That’s why you can have a framework where every visual element might be derived from Control, Widget, or even Window (Side note: except in Actionscript, where for legacy reasons, everything is nonsensically derived from MovieClip). There’s logic behind this. A button is-a control. A push button is-a type of button. A list box is-a type of widget or control. And so on… It gets a little trickier when the base class is called Window (or CWnd in MFC), but if using this type of framework, you can try to accept the notion that each control could be considered a window, even though there were better name choices.

Object-oriented suffers from a set of problems of its own, even though it is an improvement over procedural programming. The first is that there’s more typing, at least upfront. As developers, we try to reduce typing, but OO can often be more verbose than necessary. Of course, wizards and newer programming languages aim to reduce this overhead more than languages like C++ or Java, which can often be overly verbose. The OO theory, though, is that through reuse, you avoid much of this typing as you develop higher-level things because you’re basically picking from a toolbox. OO also suffers from not having the same traceability that you have in procedural programming, because functional requirements do not map directly into design anymore, nor into code, nor into testing. When doing Object-Oriented Design, Object-Oriented Programming and Object-Oriented Testing, the traceability becomes less direct, and so newer processes try to take this into account a bit more.

Actionscript, Ada, C++, COBOL, Design, Fortran, Java, Object-Oriented, Object-Oriented Programming, OO, OOP, Practices, Procedural, Programming, Structural, Structured, Testing, Unstructured

I wanted to write in here with some of my favorite Visual Studio Tools, that I find useful in my every day work.

VSFileFinder

VSFileFinder has a very simple concept: You have a lot of files, but it takes a while to navigate for which project they’re in.  Not so with VSFileFinder.  This helpful tool allows you to type a few letters of a file or even use wildcards in your search and helps you find the files you’re looking for.  The premise is simple, but this tool definitely meshes well with my workflow.

CppDoc Comment Maker

CppDoc comment maker gives you an easy to fill out template for generating a new comment that conforms to the JavaDoc standard.  Again, there isn’t much to say here, other than it streamlines the workflow making this process easier.  I will admit, I haven’t used this one as much lately, since I discovered Visual Assist X (below).

Visual Assist X

Visual Assist X is really a lifesaver, especially if you do a lot of C++ work.  It has an incredible amount of features, including syntax highlighting, improved intellisense and refactoring.  The syntax highlighting in Visual Assist X is more intuitive and easier to follow than the default in Visual Studio.  The intellisense performs better, and actually seems to do more than Visual Studio Intellisense typically does.  I find it’s less prone to getting “confused”.  Finally, the refactoring is a godsend.  Refactoring tools in C++ are almost non-existent.  So, while Visual Assist X doesn’t provide as many refactoring capabilities as you get in Java IDEs, it does a good job.  Furthermore, you can edit and configure the snippets, to further customize your refactoring.  For this reason, I actually don’t use CppDoc Comment Maker so much, because I can use Visual Assist X to generate a CppDoc-style comment once I edit the template.  At a later date, I plan to share some of my useful Visual Assist X snippets that I use.

C++, Plugins, Visual C++, Visual C++ .Net, Visual Studio .Net

If you have not read part I of this series, please check it out first:  My Experiences With Software Outsourcing. My first entry in the series discusses some of my experiences with outsourcing. In this second installment, I discuss these issues a bit deeper and provide my overall thoughts on outsourcing.  I’ve already gotten some great comments on the first article, which I appreciate.  One thing I should clarify is I’m not against outsourcing, just against certain approaches and motivations by companies who decide to use outsourcing.  In this article, I present scenarios where I think it may work.

Overhead

As I mentioned in the previous edition, time difference can be a huge factor.  Teams that are intertwined often need to work closely and need more immediate feedback.  The only way to reduce this coupling is… well, to reduce the coupling.  Therefore, I am of the personal opinion, that in outsourcing, much like software, there should be a large desire for low coupling, high cohesion.  You have to trust your outsourced developers to do the work with the right instruction.  If you don’t have this sort of faith, then outsourcing is not for you, because the overhead can be high enough to defeat all discounts, and potentially, even be more costly.  Having dealt directly with this, I know from experience.  Ultimately, the goal is to desire economies of scale and avoid diseconomies of scale.

Low Coupling

High coupling occurs when any one person is free to contact any other person in the company.  Not only are they free to, but it is encouraged.  This can happen in one office.  The problem is if a single person takes direction from multiple sources, it’s more time cumulatively time consuming.  This can be deceiving, because to the person who made the request, they often got feedback quicker than if they had to go through channels.  However, for the person who served his 30th request of the day, it is well known that this is not efficient.  If this person who has been communicated to, by so many people, is a developer, all of this intercommunication prevents entry into “The Zone”.  This is one area where all of these channels of intercommunication can be the most costly.

The reason why this behavior is so prevalent is very simple.  It works extremely well when a company is small.  In a 10 person company, every person knows every other person and likely communicates daily.  But there’s a reason why this is so efficient: The company is small enough that there is likely no more than two managers, and everyone is strictly focused on work and completing their own tasks.  Then, each person merely talks to the person who he or she most needs to communicate with to complete his or her task.  The problem is that, as you likely realize, it does not scale well at all.  Putting project managers in place is a step to reduce that, since the project managers can act as the point of contact.  However, it doesn’t go far enough, unless the offshore project manager is given more authority, and the onsite project manager is more used for communication and just communicating ideas back and forth.  When it goes further that this, I believe it’s demotivating for the foreign employee to be working through multiple layers of management and messages being relayed.  And of course, it’s costly, resulting in a lot of rework, due to miscommunication.

My ultimate recommendation here is that each office should work on disjoint projects, with potentially different sets of standards and rules.  I think it’s tempting at first, to couple things tighter, because from a company perspective, it makes sense to not duplicate work.  As a purist, I feel that duplicate work is generally not cost-effective.  However, in this case, I think it’s sometimes acceptable, mostly because the alternative may be worse.  So, I generally feel that in order for the scheme to work, it has to be treated as almost two different companies.  The US location should act and be treated like a customer.  The foreign location should act and be treated like the supplier of the software.  In this scheme, I think it can be valuable, because you forego most of the overhead of the interoffice communication, except from the perspective of a client, who is concerned with functionality and not dictacting architecture.

Motivations

The last thing I wanted to talk about in this installment is the motivations behind outsourcing.  The obvious motivation is saving money.  There’s no doubt about this, and it’s justified.  However, I see outsourcing as an opportunity.  You are getting developers for a fraction of what they would cost in the US.  Some companies take this a step further and even look for a deal in those countries.  I have to say, I think this is going overboard.  If you’re already getting developers at a fraction of the cost, why would you then proceed to also get inexperienced developers to save a few extra thousand per person per year?  If I was me, I’d pay highly there to get the most talented. It’s often cost prohibitive to get the most talented developers in the US, but abroad, it isn’t. I also feel that by getting the most talented developers, you get people you can trust to do the job, which also flows well with the concept of keeping overhead minimal.

Conclusion

Studies have shown that the cost benefits for outsourcing are often minimal, especially for companies who are seeking a deal. In pure cost analysis, because of the inherent overheads, it can provide little cost benefit, or in some cases, even be more costly.  I discuss scenarios where outsourcing might work, but it needs to be done with care.  If you’re a discount shopper, you may be fooling yourself.  If you use outsourcing to simply increase the pool of talented individuals who you may employ, the strategy has potential to be effective.

management, Outsourcing, Software, Zone