What is Test Automation Framework?
Some time ago I wrote this post describing my understanding at that time of the architecture patterns used to create Test Automation Framework and Solutions. While there’s some information I still agree with, my understanding has evolved and I want to share this evolved understanding with others.
First, lets probably coin the terminology I use (which is not necessarily would be the right one — feel free to suggest and use something different)
Test automation framework
It is a framework that allows one to write automated tests. Usually one means the specialized framework, i.e. framework that is specialized for one or several related applications under test.
Framework usually does not include tests themselves
An example of a framework can be found here: https://github.com/senpay/layered-test-framework-example
Test automation solution
It is a complete solution used for test automation. It includes everything needed to perform automated tests and usually consists of:
- Test Automation Framework
- Test Automation Infrastructure
- Automated Tests
Solution may be based on a framework, however, it is not always the case.
Architecture
Architecture may be defined as a mental model that dictates how code of the framework and/or solution is structured and communicates between its parts. Usually, architecture is enforced by the physical distribution of code parts, but again, that is not always the case.
Once I was asked why we need an architecture. For me, it was that obvious so I couldn’t even articulate this properly.
Among other things, we use architecture to:
1) Increase maintainability
2) Decrease complexity
Increase maintainability
In order to be able to change code easily, we need to divide parts that change often from those which change rarely. In order to do that we need to understand which parts are going to be changing more often than others. Also, breaking things into parts allows us to change them with a better sense of safety.Decrease complexity
By dividing code into simple, digestible parts we increase maintainability through simplification of things for the one who works on the code.
Most prominent test automation framework architecture types are:
- No architecture
- Layered
- Layered pluggable
- Hexagonal
The most used and well-known pattern is Layered, also known as N-Tier pattern — so I will describe it in more details.
Layered architecture pattern
Layered architecture pattern is so well known that on job interviews for some companies when they ask you about Test Automation Architecture you are supposed to describe this one. If you don’t — they think you know nothing about the architecture altogether.
I suggest first reading a brilliant description of the pattern at the OReilly web page because in this post I am going to describe the pattern in a way it is applied to building a test automation solution without deep-enough introduction to the pattern itself.
Usually, there would be three distinct layers, which may have different names, but mostly follow the same logic:
- Test layer
- Business-layer
- Core layer
There’s no standard naming convention, so chances are that on your project different names are in use!
Key rules for layered architecture are:
- the dependency direction (each level depend on the level below)
- call direction (no level can call/reference code described in the level above).
The rough structure of such framework is shown in the picture below (notice arrows that indicate dependency/call direction).
In such architecture, Test layer typically contains test scenarios. They may be written in a programming language or DSL (for example, Gherkin in BDD inspired solutions).
The key requirements for this layer would be:
- Test scenarios communicate test intention, not the implementation
- Test scenarios structured in a way it doesn’t takes ages to find a test for a specific feature
Business layer makes it possible to exercise actions specific for a system under test (SuT). For example, if we’re talking about online shopping, such actions may be:
- log in
- add something to cart
- check out
- e.t.c
This layer is a glue, mediator that allows translating complicated interaction with the SuT to high-level actions, that will be used by test layer.
Core layer deals with orchestration, reporting and usually also provides low-level API to communicate with the tested application itself, like web-service facades, Selenium Web Driver wrappers, e.t.c.
In a BDD inspired framework, test layer will typically contain feature files, business layer will contain steps definitions, while core layer contains BDD-framework configuration and core component.
For a data-driven framework, test layer will contain data files while business layer will contain mid-level application specific abstractions.
You may also want to have a look at the implementation example of such framework: https://medium.com/@alexspush/test-automation-framework-architecture-part-2-1-layered-architecture-example-62a0011d3329
And the Test Automation Framework Architecture FAQ: https://senpay.github.io/ta/taf/taf_faq
Have another question? Ask in comments and I will come up with an answer!
PS
Did you enjoy this post? I think you might also find this video useful: https://www.youtube.com/watch?v=RVel7MZaSA0
It describes: 3 rules of test-driven development, TDD benefits, TDD drawbacks