4 Common Follies Made In Unit/Integration Testing

I was having a discussion with a colleague last week and we got on the topic of unit testing. What unfolded is what stemmed this post… It started like this…

Colleague: “There is no need to test getter’s and setter’s in a property, because ‘I know they work’.”

Me: “WHOA WHOA WHOA!! You know they work? How? Please inform me, because I KNOW my code works too, but I also know I MAKE MISTAKES, A LOT OF STUPID MISTAKES… ”

The conversation continued and eventually spawned this post … but before we get started, I’d first like to say I’ve been burned by all four of these follies, so I know first hand how important they are.

1. Not Testing Components
2. Using a shared database
3. Testing code that depends on external objects
4. Writing a test stub and not finishing the test

 

1. Not Testing Components

“I don’t need to test that, I know they work.” – you, me, everyone (at one point in time) 🙂  

Why its a Folly: A few questions in regards to this should set us up nicely … Why are you even trying to perform testing of any kind if you’re not testing even the basic stuff? This is the REAL SIMPLE stuff that should not be ignored. Seriously, If you write code and you KNOW its solid, then why write tests for it if your so confident? Think about that… That doesn’t make sense! WE’RE ALL HUMAN, WE ALL MAKE MISTAKES. That’s why. No one’s code is perfect. I don’t care who you are. Also, what happens after you check in your code and then tomorrow Developer X checks out the code, makes a change to a setter and then checks it back in. How do you know if that did or did not break anything? YOU DON’T.

Example:

Your original code – a simple read only property that returns the order total:

public double OrderTotal { get; } 

 

Developer X’s code after he changed it. He added the new “customer level pricing discount” feature that was requested by your boss.

private double orderTotal; 
public double  OrderTotal 
{ 
	get 
	{ 
		if(customerLevel == CustomerLevel.Platinum)
		{ 
			orderTotal = TotalOrderAfterCustomerDiscountAppliedFor(customerLevel); 
		}
		return orderTotal; 
	} 
}

 

Unfortunately, because no test was written to act as a parachute to catch us (to make us fail early) we are now possibly introducing a bug into the system. Also we have no unit test backing up the new business functionality. Unfortunately because “you know it works” doesn’t cut it in this case. The QAT team fires up the app and notices the system has some “odd behavior” at certain times of execution. A bug is submitted and you eventually trace down the source of the error to the property accessor change. Oops.

 

What To Do: Test your properties. Its easy, its quick and it will save your *ss in a bind. Trust me, I’ve been that guy that said “I know this works, its part of the framework.” . If you want to make life real easy on yourself you can even set up a ReSharper Template for it. Its not that hard.

 

Notes: For example – I’m not saying to go out and test that the FileInfo class actually does what it says. If your code or the framework offer the ability for a bug to exist, then test for it. Regarless if its there OR NOT. It’s your parachute. You don’t see skydivers skimping out on their equipment do you? They do have an EMERGENCY pack. Yeah, THEY KNOW their main pack works. But what if it doesn’t? We all need a back up plan to help us out.

 

 

2. Using a  Shared Database

At times you will NEED to connect to a database to perform some type of integration testing.

 

Why its a Folly: Most teams that I’ve worked have begun working from a shared “dev” database. This introduces so many problems with the dev workflow environment that only an example is the best way to describe the madness going on here.

It usually goes like this (you’ve probably seen this before too):

• DevA updates a SprocX on the Dev DB server
  • Changes are validated by “running the app”
• DevB updates a table definition on the Dev DB Server
  • Changes are validated by “running the app”
• DevA develops some new functionality, runs app to debug and then it breaks on some code that he DID NOT CHANGE.
  • He gets a SQL Exception stating that column “CustomerNo” is not found in SprocX.
    • What? Huh? I just validated this 20 minutes ago. WTF?!
    • He looks at the code, talks to his peers, finds out that DevB had to make a change for Feature123 that made him change the column name to “CustomerNumber”.

It’s complete madness working with a shared model. DB Objects get overwritten, changed, updated, you name it – it WILL HAPPEN. Murphy loves to come visit during this time. The amount of un-managed change is too much to handle for any team. Following a shared model usually creates far too much friction for any team to succeed in any time boxed environment (and who doesn’t live in one of those).

 

What To Do: Implement a Database Sandbox for each developer. In this example we provide a separate development or test database for each developer.

Note: Managing changes in a database is a challenging task all in itself. This can be intensified in any multi-developer environment. Read K. Scott Allen’s series on version control for a database.

1. Three Rules for database work (note the first bolded topic in this post… )
2. The Baseline
3. Change Scripts
4. Views, Stored Procedures and the like
5. Branching and Merging

 

 

3. Testing Code that depends on external objects (network, file, web service)

Continuing with the previous item, when writing unit tests that depend on an external service (in this case – a database) states that you are making the following assumptions: 

1. The Network is up and working
2. The remote machine your connecting to up and running
3. The database server is running
4. The database server is accepting remote connections
5. The database in question is ACTUALLY ON THAT SERVER
6. The connection string is correct
7. No one has messed with what you’re about to test
8. and … the planets align correctly

We’re assuming that the database is in the correct state. What if this is a shared environment? What  if someone changed something? What if the network hiccups? Trust me, if it can happen, it will happen. I know first hand because all of this has happened to me.

 

Why this is a folly: Again, you’re betting on the fact that 1-7 above (if not more) are going to succeed. What if this is a web service? What types of things are you betting on there?

1. The network is up
2. The the Internet connection is up
3. The remote server is up
4. The web service address is correct
5. The web service interface has not changed
6. The web service in face does what you expect it to do
7. blah, blah, blah – too much!

The same follies again! A file is no different. Heck, even another class can be this way. Imagine working with a class that requires HttpContext to be available? It would be a mess to try to reconstruct this object graph to ensure that my test result was the same.

 

What To Do: This is a lot harder than it sounds – break your dependencies. Testing is not the only thing you’ll get from this. You’ll get flexibility, functionality and the ability to become more agile with your software. How do you break your dependencies… implement some dependency injection techniques into your code. After you break these dependencies you can mock them out with fake implementations to streamline your testing.

 

4. Writing a Test Stub & Not Finishing The Test

I’ll leave the easy one for last… this one is easy to fix. 🙂

Ok, I’ll be the first to admit, I’m guilty of this from time to time. However, I have become older and wiser and I have now started implementing an initial not implemented exception throw upon the test getting created.

Here’s what I’m talking about when we forget (this test will pass):

[Test]
public void Customer_must_be_21_to_drink_booze()
{
	ICustomer customer = new Customer("FirstName", "LastName"); 
	// TODO: Finish Test. Ran out of time this afternoon! :) 
}

 

Why this is a folly: Its not a completed test. It does not do anything.

 

What to do: Here is what I do to eliminate this (this test will fail):

[Test]
public void Customer_must_be_21_to_drink_booze()
{
	throw new NotImplementedException(); 
}

 

 

 

Conclusion

We all make mistakes, I hope that this post helps at least a few people NOT make these mistakes. 🙂