Is Test-Driven Development effective?
During some interesting discussion, one very respectful gentlemen said “there are plenty of studies demonstrating the effectiveness of TDD”. But is it so?
Test-driven development (TDD) has been a hot topic for a long time. There’re proponents and opponents of TDD. There seems to be no middle ground for those opposing groups — TDD works well for those who claim that TDD is beneficial and spectacularly fail to work for those who think otherwise.
Note: I am no longer publishing professional posts on Medium, so this one may be outdated. Please check my blog at Hashnode: https://testclub.hashnode.dev/
Is there any scientific evidence?
There’re many pieces of research about TDD effectiveness, and many papers conclude that TDD brings good results, however, their findings are not consistent.
For example, in the paper called “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development”, published in 2007 it was concluded that while TDD increases efficiency there was no evidence of quality improvement [1].
In a separate paper, which reviewed and summarized several related pieces of research, “Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls”, published in 2017 it was found that test-first approach can improve quality, while decrease productivity. Besides, it was claimed that most developers feel test-last approach as more natural [2].
During the “A Replicated Experiment on the Effectiveness of Test-First Development”, 2013 they could not find any effect of using Test-First approach on quality or productivity with comparison to Test-Last approach [3].
Finally, in paper “Overview of the Test Driven Development Research Projects and Experiments”, 2012, which tried to find a decisive conclusion based on several pieces of research and papers, it was concluded that there’s no persuasive evidence of the positive impact of TDD neither on quality nor productivity [4].
So, why so many people would still insist that TDD is so effective technique?
As a huge proponent of TDD, I can offer only one possible answer — the effectiveness of TDD hugely depends on the developer. It was not possible to get a consistent result on the TDD because it was not [only] TDD impacting quality and productivity, but developers themselves. It seems that there’re developers for whom TDD brings very good results, there’re developers for whom it does not have any positive (or negative) effect and there’re developers whose productivity and effectiveness is deteriorated by TDD.
I can offer only one possible answer — the effectiveness of TDD hugely depends on the developer.
The only thing now would be to consider if we can design an experiment to prove or disprove this hypothesis. In my views, this hypothesis is a natural advancement of the idea that development is a social activity. The only problem with social phenomenons is that they are not-deterministic, which would explain so different results.
PS
Did you enjoy this post? I think you might also find this video useful: https://www.youtube.com/watch?v=RVel7MZaSA0
It describes: 3 rules of test-driven development, TDD benefits, TDD drawbacks
References
[1] A. Gupta, P. Jalote “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development” presented at First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 2007
[2] Z. Khanam, M. Ahsan, “Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls” International Journal of Applied Engineering Research vol. 12, no 18, p. 7705, 2017. Available: https://www.ripublication.com/ijaer17/ijaerv12n18_81.pdf. [Accessed May 13, 2020].
[3] D. Fucci, B. Turhan, “A Replicated Experiment on the Effectiveness of Test-first Development” presented at ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, USA, 2013
[4] A. Bulajic, S. Sambasivam and R. Stojic “Overview of the Test Driven Development Research Projects and Experiments” presented at Proceedings of Informing Science & IT Education Conference (InSITE), 2012. Available: https://pdfs.semanticscholar.org/3278/971bb53a25822171df127d47a57243dd3bbd.pdf. [Accessed May 13, 2020].
Originally published at https://senpay.github.io.
Russian translation: https://habr.com/ru/post/501900/