AvocadoSoftware.com

Chad Myers over at LosTechies, posted some interesting questions about TDD. I don't consider myself to be a TDD master - actually, I've had the same questions that Chad has, over the last year - and I do have some opinions, now, and thought the world might want to know. :)

I'm repeating Chad's questions here, and numbering them so I can respond accordingly.

TDD Practice Questions

1. How much up-front design do you tolerate? How do you know when to stop (i.e. 'when algorithms start getting discussed, it's testing time')?
2. What about certain "I just know we'll need this" type stuff (i.e. putting a try/catch{Console.WriteLine(ex);} in your console main() method?)
3. When writing a test, in order to even get it to compile, you have to build an interface or two, a concrete class with some NotImplementedExceptions in it, etc. How far do you allow that to go?
4. If, during the middle of testing on a story, you realize that your up-front design wasn't correct, do you stop and discuss with your pair right then, do what you need to do and proceed, or do you pull back and go back to full pre-test design mode on that story?
5. When proceeding to a new story, you discover that a test you wrote for a previous story no longer reflects the requirements. Do you refactor that test immediately, mark it as ignored until you finish the current story and cycle back to the ignored one? Something else?
6. If the new story's requirements involve a slight tweaking to an existing test, do you tweak it, or make a new one and discard the old one (i.e. 'No changing existing tests!' or 'Only change if it's a minor compiler issue, but otherwise don't change it')?
7. If you're, say, building up your model and it passes tests, but you're seeing that it's infantile and that the next few upcoming stories will produce significant changes to produce a new emergent model, is it appropriate to step back and consider a larger refactoring, or do you keep plugging and make the changes into the existing model even if it could benefit from some housekeeping that is otherwise out of scope?

My Answers

1. Enough to know what I'm supposed to build.
   
   You really need to answer this question based on your individual project. I recently built an enterprise integration project that took our core maintenance system and brought some disconnected operations into it. On this project, I did not have a lot of up front design. I knew that the system needed to operate in a disconnected mode, I knew it needed to be operational even when there was no one logged into the computer or running the software, and I knew that it needed to do bi-directional communication with the master maintenance system, to perform a limited set of maintenance related tasks. I started with a few assumptions / up front design - build a Windows Service to run the core process, build a client-server architecture to host the UI, build on top of a messaging system to support the communication needs. Beyond this, the functional requirements of those tasks drove the design of the project during the construction of the software.
   
   Conversely, I've worked with a team that was building a system for a laboratory, recently. With the requirements from the customer, the process that they requested, and the required conversion of existing data that the customer had; we required a large up-front design, to determine if the data conversion and functional requirements were going to be met by our proposed solution. The major design of the system was known up front, but the implementation was still a work-in-progress during the construction of the software.
   
2. This is one of the questions that I still battle with, and I answer it differently every time I ask the question. I try not to add features and functionality unless I know that I need them right now - when it comes to the domain model and core infrastructure services of my application. However, with items like the Console.WriteLine(ex) question, I don't see any issue with adding those into the application's Main method. After all - you are not going to unit test the App's Main Method... you are going to integration test it.
   
3. In terms of implementing interfaces, I try not to manually code stub objects anymore. Instead, I try to use RhinoMocks whenever possible so that I can avoid that question. I used to code stub objects all the time, and would have dozens of Not Implemented methods - the end result is that my unit tests were horrendously difficult to maintain and I had dozens of duplicate stub objects to provide a certain method to a certain unit test. RhinoMocks helped me clean that up and keep my unit test code stub object requirements manageable size. There are times when you really do need a stub, and when you do need one, you build the methods you need and allow the Not Implemented methods when you really don't need that method for your tests.
   
   If you're asking about real objects that you are stubbing out the method signatures for, in your unit tests - you shouldn't allow NotImplemented methods to live in your code for very long... you should only be adding the methods that you need, for your software to perform it's required functionality. If you have a real object with a Not Implemented exception being thrown, you are probably looking at a method that can / should be deleted from the code base.
   
4. Once again, it comes down to your specific situation.
   
   I spent some time doing pair programming with a coworker, a while back. During this time, we continuously ran into situations where the next functional requirement caused a change in our existing designs. When this occurred, we would only make changes as we needed to. We would write a new unit test for some new functionality, and if the previous designs did not allow us to make the unit test pass, we would go back and modify our previous designs - making changes and updates to the unit tests in those changing areas, as we go. The great thing about unit tests in this situation is that you can consider them to be a "to do list" of sorts. During the times that we were changing our existing designs, we would often forget about an area or the new functional requirements that we were trying to satisfy. Fortunately, a complete run of the entire unit test suite would show the now broken tests and be a reminder to us on what we needed to change.
   
   During this pair programming time, there were a few occasions where we got to a new requirement and looked back at our design and implementation with the realization that the entire model was wrong and needed to be re-written from the ground up, with the knowledge of this specific requirement in mind. At those times, we did what a lot of people would never even consider - we deleted every class and every unit test in our system and started from scratch. The lessons learned along the way and the knowledge of other requirements lead us to a new design that accounted for the functionality that we would need - but we were still able to code the existing functionality on an as-needed basis. We provided extensibility through additional interfaces and abstractions, without creating "future-code"... stuff that "we know we'll need this in a bit, so let's do it now".
   
   I've gone through these "design, fix, redesign, fix, rebuild from scratch, fix" cycles numerous times. Every time it happens, it scares the heck out of me to delete existing code and start over. In my experience, though, the end result is worth the effort - fewer bugs in the system, a model that more accurately represents the business needs, and better extensibility and flexibility that allows for change in the future.
   
5. Fix it as early as possible. If you don't fix it now, you're likely to create more work for yourself because you'll have something new that interacts with that broken piece, and will have to fix the broken piece and the new piece. If you fix the broken piece now, you won't have to re-write the new piece later.
   
6. If you are changing the meaning of a test (i.e., you have to rename the test to accurately reflect what is being tested and why), then you are automatically destroying the old test and creating a new one - even if you are only destroying it semantically and you are really just renaming and changing a few details. If the meaning of the test remains the same, and you are only changing some of the implementation details (because you changed the design, via question #4 or whatever), then leave the test where it is and just change the implementation to meet the new implementation needs.
   
7. See my answer to question #4.

TDD Style Questions

1. Assert.That(x, Is.EqualTo(y)) or Assert.AreEqual(y,x)?
2. How many asserts/test? Any caveats?
3. How do you know what to put in the SetUp method?

My Answers

1. Syntax is a personal choice and almost irrelevant. If they are both technically correct, then choose the one that most correctly represents what you are trying to say. I prefer to code in english as much as possible (read the first 9 chapters in Steve McConnell's "Code Complete 2", if you haven't). However, I also prefer simple APIs. I don't like the Assert.That(x, Is.EqualTo(y)) syntax because it creates a complex API set that I have to know. RhinoMocks is a great example of what you are suggesting, here. I love RhinoMocks and will continue to use it... but I don't always like the strange API calls that I have to make.
   
2. If you have a single object and a single method call that is being tested, with a single output value; the answer is easy - you have one output value, so you need one assert. In the real world though, your have a system under test that interacts with two or more sub-objects and interfaces, each with multiple behaviors and / or values changes that get invoked when you make a single method call on your system under test. The real answer is that you need one assert for every behavior / value that you expect when that one method is called. If your method returns seven values and call two methods on a child object via an interface, you need nine asserts for that one test.
   
3. Anything that is used by EVERY test in your test fixture. If there is even one test in your test fixture that does not need the information provided by the Setup method, then you should not have that information in the Setup test - move it to a common method that can be called by each of the tests that does need the information.
   

TDD Mechanics Questions

1. Is a refactoring tool absolutely necessary?
2. If so, what is the bare minimum features that the tool would need to facilitate decent TDD?
3. Integrated IDE test runner or a background source-watching auto-runner?

My Answers

1. Absolutely, 100% NOT required. In fact, I would recommend that you exercise your mind now and then, and purposely work without a refactoring tool from time to time. Remind yourself what it really takes to refactor something - understand the true cost of refactoring by doing it manually. When you understand this cost, you'll see why you really do want a refactoring tool. Personally, I can't stand coding without Resharper. However, it doesn't support Visual Studio 2008, at the moment, so I'm stuck refactoring everything by hand when I code in VS2008. It really makes me appreciate Resharper that much more and I'm looking forward to the EAP / release of Resharper 4.
   
2. Resharper has a built in unit test runner, which is really nice. I use it whenever I'm in VS2005 and writing unit tests. However, TestDriven.Net also has some really nice integration into VS2005/2008 for unit test running. I use this in VS2008, since I can't use Resharper at the moment.
   
3. Both. See #2, and build yourself a Continous Integration Server (We use CruiseControl (the CC.NET flavor)) with automated unit test running. We schedule builds on our CCNet server to run every 15 minutes, and every time it builds, it runs all of our unit tests for us.

Hopefully this information will be useful to someone else out there, who may be asking the same questions. I'm glad to see Chad asking these questions, and I know that I've asked the same questions in the past. The learning curve on this stuff can be very steep and very painful. If we all pitch in and try to answer these questions when they pop up, hopefully we can reduce the learning curve and entry cost.