Today’s text intends to explore a question that we sometimes hear from clients. It’s common to hear things like: “Now I understand the importance of having automated tests! The tests which are 100% manual are sacrificing our time and the quality of the release. But where do we start? What types of tests should we use to start automating?”
Starting…
As a preamble, it’s good to see that the need to automate tests is clear! It’s not reasonable to keep people working in our projects with the sole purpose of running manual tests. This type of work can be automated with a high level of reliability, freeing people (those in the testing area, for instance) to play nobler and more productive roles. For example, creating and executing exploratory tests in cases that have not yet been investigated, such as:
- Preventing some fields from accepting dates such as February 29 in non-leap years (This type of error is for good testers only! Not everyone is so wicked!)
- Important information that should be on the screen but isn’t
- Language errors in labels, messages, etc.
- “Corner-cases“
There are surely several types of automated tests that can be used: unitary, integration, functional, or acceptance tests. But where to start testing your system? Definitely where the ROI (Return on Investment) is higher – usually, in cases of systems that have been under development for a while, the ROI will be higher if we target acceptance tests first. Why? Because acceptance tests are, by nature, black-box tests, and these are the tests that more easily express what clients would like to do or to have in their software. For this reason, they tend to be highly aligned with business rules (expressed, for instance, through User Stories).
As input to write acceptance tests, we can (or should!) use the acceptance criteria described by the Product Owners in the User Stories. When these tests have been implemented, are running and passing successfully, we’ll have good coverage and a guarantee that at least the most important functionalities are really being covered by our tests. Most importantly, all this feedback is provided in just a few minutes and without any need for human intervention!
This type of test can/should run on a Continuous Integration robot (Jenkins, TravisCI, Hudson, Team City, etc.), which will do the work of running tests for you according to a predefined strategy (every day, for each change made to the source code, etc.).
And what about the autonomy of an acceptance test?
Basically, this type of test may or may not be behavior-driven (BDD), must manipulate the interface as if it were a user (using tools such as Selenium WebDriver), and must guarantee the result found in the database even before running the test (to avoid firefly tests, which sometimes pass and sometimes don’t).
Nowadays, this type of test can be implemented in virtually any “commercial” language (Java, Python, C#, Delphi , Ruby etc.). However, because it’s a black-box test, you will know that there is a problem, but you won’t know exactly in which layer of your system it resides. Tests of this type indicate that there is an error (which is great), but leave it to the developers to locate the exact point of the problem (which can be a nuisance, requiring huge debugs). An analogy I often use is that acceptance tests are a great “zoom out” of the system, while unitary (white-box) tests are great “zoom-ins” of a problem. In summary, if the legacy has no tests:
- Begin with acceptance tests
- Find out where coverage is lacking
- If you discover weak spots, attack them with unit tests
- Automate the build and the CI (with tests)
The final tip is that, with rare exceptions, it’s not worthwhile to begin with unitary tests. There are many legacy systems which are highly coupled and developed without the preoccupation of making them “testable” as a unit. Unitary testing means isolating all other classes/components so that a given piece of the code can be tested as a unit. But how to do that if the software often resembles a house of cards? That definitely isn’t the best strategy.
I hope you’ve enjoyed this post, and we would like to receive feedback from you, our readers, on the next topics we should address! Please send your suggestions to our email or to our Facebook page.
And another important thing! All the topics mentioned here are discussed in more detail and at greater length in our Certified Scrum Developer (CSD) training. If you’re interested in these topics, we’ll be waiting for you in one of our classes!
