Testing, Testing and Testing

You hear it in real estate: location, location, location.  You hear it (not as much) in software: test, test, test.  However, in software the tests are all different.

A recent project that I'm on involves a venerable app with as much technical debt as the US Government... and we need to add a feature.  Solution: tests, tests and tests.  More specifically if you don't know the different kinds of tests:

• Good old Unit Testing, easy to implement and understand, unless...
• You are getting used to mock objects (e.g. JMock or Mockito)
• You are testing database functionality and are new to DBUnit
• You are testing a web app or Servlet and are new to HttpUnit
• You're confusing unit testing (ONE module) with integration testing (multiple modules)
• Example: testing your Data Access Layer (DAL) with a dummy DB, like a local HSQL
• Integration Testing, usually implemented with a framework that ends in "Unit" so it's easily confused with unit testing.  Integration testing integrates two or more components, everything from two classes to the entire app with a dummy DB.  Due to this you can have multiple levels of integration tests, each testing more modules together.  Examples include:
• Testing a servlet in HttpUnit's ServletUnit instead of your actual web container
• Testing your DAL with an "actual" database (whatever actual means to your project)
• Testing a fully created Spring bean with the real beans injected instead of mocks
• System Testing, this is tests of the entire system
• QA usually conducts manual system tests
• There can be automated ones as well with things like Selenium
• As an example for a web app, using a supported browser to use the app in the final web container and using the real database.

There is of course, much more to software testing, but these three are usually a good start.  Heck, in many organizations / projects even using unit testing is new to people.  Like Regan said: trust, but verify.

PS: There's a good post at EvilTester with a more philosophical approach to testing.

NOW I See: The Realized Benefits of Unit Testing

We all know the things that we are supposed to do, but don't as much as we should: eat fiber, cut down on sugar, don't be sedentary, exercise, eat fruit, unit test.  Blegh!

Not my exact point, but pictures about
unit testing are hard to find!
(Image from Reportnet)

For the last part at least, I came across some actual useful results.  Before, whenever I wanted to unit test a class, I didn't write a test for it before and usually had to ramp up on another technology to boot (JUnit3 vs JUnit4, HttpUnit, DBUnit, Mocking) or realize that I really wanted an integration test (jMeter, Selenium).

This time however, I spotted what looked like incorrect code and there was already a unit test for the class!  I was able to write a short test that verified the bug (i.e. wrote a failing test), fixed it and reran the test successfully!  All this in less time it would take to run mvn verify (including deployment to WebLogic)!

I'll be a bit less loathe to write tests in the future now, since I actually SAW that "it's better in the future" thing they say about testing.