Alternative Methods of Testing

This was a topic of discussion at Smalltalk Solutions after Niall Ross' talk on concurrency testing. The floor was opened up for other methods of testing a system. Michael-Lucas Smith brought up how he used SmallLint (a tool that exposes a plethora of code smell symptoms) to prevent certain code smells from entering the system. I went on to explain how I used what I called "Source Code Tests". Basically, I make tests that check compiled code for forbidden method calls in the system.

It's a topic I've been thinking a lot about lately and I'm planning on presenting some of my thoughts at the next Smalltalk User's Group Meeting. Basically, I feel that testing is crucial to a successful project. Unit tests are great for finding errors quickly. Wouldn't it be great if we could use tests to help us find other problems rapidly? Michael-Lucas Smith's usage of Lint in his unit tests stamp out code smells before they even get started. I started to write "Source Code Tests" to stop the usage of APIs that cause performance problems from getting out of hand. It's worked great because it stops bad code from starting or continuing. What other ways could we possibly test? Here's some additional testing methods:

• Source Code Management provides historical information on code that could be problem sources. Think about it. If we see that one class is changed more frequently than any other class, then that might indicate a potential god class or at the very least a class that needs to be closely examined. Why does it change so often? We could ensure our design guards properly against unstable classes in the system and so that they do not effect the stable portions. You could test for rate of change (if there's a huge jump, then your tests could warn you. Maybe they shouldn't fail, but a warning color of yellow might be indictative that the code is correct, but there is a design problem.


• Metrics calculations can be useful for uncovering design smells. You could place thesholds in your tests for whatever metric calculations your team wants to track. If a certain threshold is passed, then a test fails. This makes you look at the offending code right away and stops design smells in their infancy.


• Method wrappers during execution of unit tests and verifying the results. For example, mark your private methods (via pragmas or method categories) and check if any code called it externally. This is just one simple example. I know there's more lurking.

And that's just a few, I'm sure there's more. The point is to look at what problems would you like to know about early and fix? The earlier we find problems, the easier they are to fix. Anything that will help find code and design smells is a good thing. What other tools do we have now (like Lint and Metrics) that we could use from our tests? It's a fun topic to think about! Of course, I welcome any feedback because I want to know how I can not only better the correctness of my code, but my design as well.