You don’t need complex test automation

2 min readApr 18, 2020

While there’re lots of information published on the internet about how one should develop/buy a Test Automation solution, there’s an elephant in the room — which is the fundamental question “Do I need to do it at all?”.

That is a good question. While I think that there’re some legitimate contexts where product/project/team can benefit from having a sophisticated test automation solution/framework, my general rule of thumb is “If you need a framework — you do it wrong”.

There’re at least several decent papers showing the benefit of unit-testing ([1],[2], [3]), while I could not find any research proving that system level tests could bring better results. Please let me know if you can prove me wrong.
There’s at least one half-scientific paper showing the benefits of unit tests in comparison to integration/system level tests [4]
There’s a good paper bringing a notion of running/writing and maintaining test vs. the benefit of having/running it at all [5]. If we know that, by definition, system level tests are more complex in maintenance, expensive in running and inspection, why do we insist on having so many of them?
Inability to write small and reliable unit tests is almost universally considered an indication of poor product architecture.

However, there’re legitimate contexts where you might want/need to invest in a complex test automation solution. My personal opinion is that it is when both of the following is true:

Product under test does not have enough reliable unit-test coverage;
The cost of writing/maintaining test automation solution is lower than the cost of refactoring/investing in product architecture and unit-test coverage for the expected planning horizon.

Or where this is true:

The quality is critical to the extent where we don’t really care about how expensive QA would be.

All things considered, at the time of writing I think we have only one working hypothesis, which is:

While all type of automated verification has its own benefits and areas of applicability, unit testing seem to be the most efficient way of automatic verification.