Techniques for stubbing out dependencies when unit testing Java code (part 2/2)

Part 1 of this series examined the problem of how to build unit tests when adding new features to legacy code that may not be structured perfectly, that is, typical real-world legacy code. How do you structure your new code or refactor the legacy code so that you can stub out the legacy services needed by your code when testing?

For most cases, we found techniques that work, but with a bit of pain. You can extract interfaces for your legacy services and write your new code against those interfaces. But, every time you add a new method to the legacy service, you must also add it to the interface and add it to each of your dummy test stubs. That’s a bit tedious. When your legacy service requires you to subclass, you can subclass with a simple adapter class and then use composition so that the adapter delegates to your real class, which is no longer a subclass of the legacy service. While that works, it’s certainly a bit awkward and confusing.

And then there’s the case of legacy code with static methods. We found no good approaches to stubbing out static methods for testing. Your only options are either 1) do an expensive and high risk refactoring of the legacy code to remove the statics or 2) don’t stub out the legacy statics in your tests, requiring you to startup the full system with each test.

In this article, I’ll discuss a different way to build your new code that allows you to easily stub out legacy code without any need to refactor it, no matter how badly it is written.

A confession. First a confession: I don’t like Java. In fact, I really, really dislike Java. Do you ever have those dreams where you’re running as hard as you can, gasping for breath, straining every muscle in your body, but yet you’re only moving at a snail’s pace because some invisible force is holding you back? That’s Java development in a nutshell.

When I code Java, it seems like for every one useful thing I write, I have to tack on 10 extra pieces of bureaucratic baggage. It doesn’t feel agile. It doesn’t feel modern. Rather, it feels stifling and bureaucratic. Frankly, I feel like I should be wearing a suit when I code in Java.

I have become a big fan of dynamically typed languages, Python in particular. At work, I began looking for ways to incorporate Python into my company’s Java code base. I discovered Jython and began playing around with developing new code with it in hopes of developing new features faster and making our code base more agile.

When experimenting with Jython, my most interesting discovery was that building unit tests for new features was significantly easier than building unit tests for features written in Java. Why? Because Jython makes it trivial to stub out legacy services, even poorly written ones.

Reexamining YourNewFunctionality in Jython. Recall from part 1 that we wrote a new class called YourNewFunctionality that uses the legacy class ThingyManager, which is a real class, not an interface. The equivalent Jython class is as follows:

class YourNewFunctionality(object):

def __init__(self, thingy_manager):

self.thingy_manager = thingy_manager

def get_dependent_thingy_names(self, thingy_name):

thingy = self.thingy_manager.getThingyByName(thingy_name)

return thingy and [dep_thingy.name for dep_thingy in thingy.dependentThingies] or []

(As an aside, notice how Python took 6 lines of code to do the exact same thing as Java with 16 lines of code… bureaucratic baggage!)

As in the Java code, we construct the new class with a reference to the legacy ThingyManager, which is then stored as a member variable. Method get_dependent_thing_names() calls the Java ThingyManager to get a Java Thingy object. It then calls method getDependentThingies() on this object to get another list of Thingy objects. It’s pretty cool how Jython makes it possible to seamlessly integrate Python and Java.

Now let’s think about testing this new code (with apologies to Kent Beck for doing this in the wrong order). Even though the Java ThingyManager class may have 50 methods, we’re only calling one, getThingyByName(). Similarly, even though the Thingy class may have 20 methods, all we’re calling is getDependentThingies() and getName(). If we’re going to stub out these two classes for testing, all we need are classes with those three methods. And, we don’t have to worry about interfaces at all; Jython doesn’t care about them. Here are some stubbed out implementations:

class DummyThingy(object):

def __init__(self, name, dependent_thingies=[]):

self.name = name

self.dependentThingies = dependent_thingies

class DummyThingyManager(object):

def __init__(self, thingies):

self.thingy_map = {}

for thingy in thingies:

self.thingy_map[thingy.name] = thingy

def getThingyByName(self, name):

return self.thingy_map.get(name)

With these two dummy classes in place, we can easily test our new functionality:

def test_get_dependent_thingy_names(self):

thingy1 = DummyThingy(‘thingy1’)

thingy2 = DummyThingy(‘thingy2’)

thingy3 = DummyThingy(‘thingy3’, [thingy1, thingy2])

thingy_manager = DummyThingyManager([thingy1, thingy2, thingy3])

new_funct = YourNewFunctionality(thingy_manager)

self.assertEquals([‘thingy1’, ‘thingy2’], new_funct.get_dependent_thingy_names(‘thingy3’))

self.assertEquals([], new_funct.get_dependent_thingy_names(‘thingy2’))

self.assertEquals([], new_funct.get_dependent_thingy_names(‘bogus thingy’))

Done. Simple. We didn’t have to refactor anything. In fact, we didn’t even have to stub out 70 methods, most of which are irrelevant to us! We’ve eliminated lots and lots of bureaucratic baggage.

What about static methods? Now we’ll look at the case where static methods on Thingy replace ThingyManager. Remember that we found no good solution for this case in Java. But static methods are easy to deal with in Jython. Rather than constructing YourNewFunctionality with an instance of the ThingyManager, we’ll now construct it with the class representing Thingy objects:

from com.yourcompany.legacy import Thingy

class YourNewFunctionality(object):

def __init__(self, thingy_class=Thingy):

self.thingy_class = Thingy

def get_dependent_thingy_names(self, thingy_name):

thingy = self.thingy_class.getThingyByName(thingy_name)

return thingy and [dep_thingy.name for dep_thingy in thingy.dependentThingies] or []

I’ve thrown in a little Python trick here for convenience. When you construct class YourNewFunctionality, if you pass no parameters, it will use the Java Thingy class. Alternatively, you can pass the constructor any Java or Jython class, such as a dummy class for testing.

Method get_dependent_thingy_names() is much the same as before. The only difference is that now it calls getThingyByName() as a static method on the Thingy class.

Here’s how to create a dummy Thingy implementation for testing (I find Python’s syntax for static methods a bit awkward):

thingy_map = {}

def set_thingies(thingies):

for thingy in thingies:

thingy_map[thingy.name] = thingy

class DummyThingy(object):

def __init__(self, name, dependent_thingies=[]):

self.name = name

self.dependentThingies = dependent_thingies

def getThingyByName(name):

return thingy_map.get(name)

getThingyByName = staticmethod(getThingyByName)

Your test now looks like this:

def test_get_dependent_thingy_names(self):

thingy1 = DummyThingy(‘thingy1’)

thingy2 = DummyThingy(‘thingy2’)

thingy3 = DummyThingy(‘thingy3’, [thingy1, thingy2])

set_thingies([thingy1, thingy2, thingy3])

new_funct = YourNewFunctionality(DummyThingy)

Using statics is still a poor way to code, but at least we now have a strategy for stubbing out legacy code that uses them.

What about subclassing legacy code? Recall from part 1 that we need to write class YourNewFunctionalityProcessor that subclasses legacy Java class ProcessorBase. Here, we can take advantage of two Python tricks. First, in Python, class names are just variables. Therefore, we can pass them to methods, assign them to other variables to create synonyms, use the synonym variables in place of the real classes, and so on. (Indeed, we took advantage of this in the YourNewFunctionality constructor in the previous section.)

Second, Python supports closures, which basically means that you can create new functions or classes inside a function, and those new functions or classes have read access to all the variables of the enclosing function.

Combining these two tricks, we’ll create a factory function that creates and returns the YourNewFunctionalityProcessor class. The base class will be passed as a parameter to this factory function:

from com.yourcompany.legacy import ProcessorBase

def make_processor_class(base_class=ProcessorBase):

class YourNewFunctionalityProcessor(base_class):

def someMethodRequiredByProcessorBase(self):

self.someUtilityMethodProvidedByProcessorBase()

return YourNewFunctionalityProcessor

We’ve pushed the definition of class YourNewFunctionalityProcessor into an enclosing method so that we can control its base class at run-time. If we call make_processor_class() with no parameters, our new class will inherit from ProcessorBase, which is what we want for production.

Alternatively, we can create a dummy base class that defines method someUtilityMethodProvidedByProcessorBase(). With that, we can construct a new processor class that inherits from the dummy base class so that we isolate our new functionality for testing:

class DummyProcessorBase(object):

def someUtilityMethodProvidedByProcessorBase(self):

processor_class = make_processor_class(DummyProcessorBase)

processor = processor_class()

Explicitly testing Java dependencies. Jython has certainly simplified our testing. However, our tests have a major flaw. We’ve stubbed out all of our dependencies on the legacy Java code. What if someone changes the interface to some of the legacy Java code? Our tests will keep passing happily using the stubs, but our code will crash and burn when run from within the real product. And what’s worse, because our product is huge, we can’t possibly do manual regressions of all functionality with each release. So, in all likelihood, if someone changes an interface to the legacy code, a customer will call up complaining that our new feature no longer works.

Of course, it’s a bad idea to ever change interfaces in such a way that you could break other code. So, in theory, this situation should never occur. But here’s what happens in reality. Over your objections, your company hires some pimply kid who just graduated from your rival university. We’ll call him Wally. Three months later, crunch mode for the next release hits. Everybody is working ridiculous hours. Everybody is stressed. Nobody has time to code review Wally’s latest change where he innocently modified an interface without telling anybody. That’s reality.

Being a proud, protective parent of our new functionality, we clearly need to build some test that automatically catches Wally red-handed when he sets out to screw things up. But how?

If we enumerate all our dependencies on Java, we can easily build a small Java class that uses all those dependencies. The class will never run because it doesn’t worry about things like setup and initialization. But, it will compile. And if any of the interfaces ever change, it won’t compile. That is, we’re going to write an automated test of our dependencies that is strictly a compile-time test; if it compiles, it passes; otherwise, it fails.

public class TestJavaDependencies {

public void testThingyManager(ThingyManager thingyManager) throws Exception {

Thingy thingy = thingyManager.getThingyByName(“some string”);

}

public void testThingy(Thingy thing) throws Exception {

List<Thingy> depThingies = thingy.getDependentThingies();

String name = thingy.getName();

}

public static void TestProcessor extends ProcessorBase {

public void someMethodRequiredByProcessorBase() {

someUtilityMethodProvidedByProcessorBase();

}

}

}

Not too hard at all. Take your best shot Wally!

Conclusion. My company has a has a large suite of automated tests. That’s good. Most of these tests must configure and startup our core system. That is, they’re really integration tests, not unit tests. That’s bad. Most individual tests take a minute or more to run. The entire suite takes about 6 hours to run. These long run times severely hamper our ability to do effective test-driven development, where you’re literally changing and running tests every minute or two while developing code.

The long run times also make it impossible for us to run the test suite from Cruise Control with each commit to Subversion. Instead, we only run the test suite nightly, making it much more difficult to isolate which commit during the day broke the tests.

But using the techniques in this article, I’ve built a new feature with nearly 100% test coverage. And the test suite for this new feature runs sub-second, opening the door for Cruise Control and real test-driven development. Throw in the additional coding efficiencies of Python over Java (many web sites claim Python developers code 5-10X faster than Java developers, though I suspect that 2X is more realistic), and Jython looks like a clear winner!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: