Technical

Upcoming releases with Automation workbench

Meliora Team | 4 June, 2019

In this post, we will discuss a bit about our upcoming release schedule and offer a glimpse of the features in upcoming releases.

Challenges with test automation

Business challenges related to interpreting results of automated tests are obvious. More often, tests are created by developers with various different motives, the number of tests is often numerous and traceability to the specification is often lacking. Features are needed to make sense of these results and gain the business benefits outside the scope of regression or smoke testing – which in practice, often tends to be the use for automated tests.

If you are able to map and scope the tests efficiently, you

have a better understanding of what is currently passing or failing in your system under testing,
have your team collaborate better,
ensure that key parties in your project can make educated decisions instead of being reactive and
have a tool to really transition from manual to automated testing as existing test plans can be easily automated by mapping automated tests to existing test cases.

Automation in Testlab

In Testlab, results from automated tests can be mapped to test cases in your project. This way of working with your automated tests has benefits as the results can be tracked and reported in exactly the same way as the manual tests in your current test plan. What happens is that when results for automated tests are received (for example from a Jenkins’ job), a test run is added with the mapped test cases holding the appropriate results. The logic of how the mapping works is currently fixed so that the test cases can be mapped with pre-defined fixed logic: If the ID of the automated test starts with an ID bound to a test case, it will receive results of this automated test.

Upcoming Automation workbench

Future versions of Testlab will include a new Test automation view with a workbench aimed at managing the import and mapping of your automated tests. A screenshot of the current prototype can be seen below (click for a peek):

The workbench

allows you to define “sources” for your tests: a source is a new concept which allows you to configure how the incoming results from such source are handled. You might want to think the source as your “Jenkins job” or your Jenkins installation as a whole if your jobs follow the same semantics and practices. A source is defined with “rules”, which define how the automated tests are mapped. The workbench
features a rule engine, which will allow comprehensive mapping rules to be defined with
- ignores (which tests are ignored),
- creation rules (how test cases for tests are automatically created) and
- mapping rules (such as “starts-with”, “ends-with”, regular expressions – defines how the results for the tests are mapped to your test plan) and
the workbench allows you to
- easily upload new results by dragging and dropping,
- execute any rules as a dry run to see how the results would be mapped before the actual import,
- create any test cases by hand – if preferred over the creation rules – and
- use pre-built wizards which help you to easily create the needed rules by suggesting appropriate rules for your source.

We feel this is a unique concept in the industry to help you get better visibility in your automation efforts.

The prototype shown is work-in-progress and is due to be changed before release.