TestNG and Cucumber

TestNG:

TestNG is a testing framework inspired from JUnit and NUnit, but introducing some new functionalities that make it more powerful and easier to use. TestNG is an open source automated testing framework; where NG means Next Generation.

Cucumber:

Cucumber is a tool based on Behavior Driven Development (BDD) framework which is used to write acceptance tests for a web application. It allows automation of functional validation in easily readable and understandable format (like plain English) to Business Analysts, Developers, Testers, etc.

Why TestNG:

• Well, support page object model (POM) automation design.
• There is no more need for a static main method in our tests. The sequence of actions is regulated by easy-to-understand annotations that do not require methods to be static.
• Useful when you have to automate a large number of test case.

Listeners and Reporters

• It is pretty easy to write your own listeners (which act in runtime) and reporters (which are invoked after the execution).
• TestNG can generate reports based on our Selenium test results.
• WebDriver has no native mechanism for generating reports.
• Embeds BeanShell for further flexibility.
• Default JDK functions for runtime and logging (no dependencies).
• Dependent methods for application server testing.

Concurrency

• No need to create own Thread objects! Use of rich library of annotations and that is it!

An example of running same test method simultaneously (20 times, by 3 threads):

@Test(threadPoolSize = 3, invocationCount = 20)

public void concurrencyTest() {

System.out.print(” ” + Thread.currentThread().getId());

}

Result:

13 12 13 11 12 13 11 12 13 12 11 13 12 11 11 13 12 11 12 13

 

• Support of Data Driven Testing

Group of Tests

• each test class (and method) can belong to multiple groups,
• it is pretty easy to run only selected group:
• g. you can run only tests from groups “smoke”, or only “gui” tests,
• it is possible to define dependencies between methods/groups (see next slide).
• all done with annotations (e.g. @Test(groups = “smoke”))
• Run threads using multiple threads for faster test execution

Parametrized Tests

• Passing test parameters

Parametrized tests are very simple with TestNG (no constructor awkwardness).

You can have as many data providers in one class as you wish. You can reuse them (call them from other classes), and you can make them “lazy”, so each set of parameters is created when required.

Once you migrate to TestNG you will notice that you start writing a lot of parametrized tests. Because they are so very useful!

 

Test Fixture Settings

• TestNG offers well-thought @Before… and @After… annotations (on suite, class, group and method level).

Dependencies between Tests

• Fail Fast: Means that the feedback will be much quicker in case of failed tests
• Logical dependencies: gives you a much more realistic error information – you learn that 1 tests has failed and 99 has been skipped, which is much easier to fix than the information about 100 failed tests

Exception Handling

• Uncaught exceptions are automatically handled by TestNG without terminating the test prematurely. These exceptions are reported as failed steps in the report.

Documentation

• TestNG documentation is in very good shape, and always up-to-date

Integration with other tools

With TestNG you still can:

• run tests with Maven, Ant, Gradle, …
• execute tests with your IDE: Eclipse, IntelliJ IDEA, …
• use Jenkins, Sonar, Cobertura, Mockito, FEST Fluent Assertions, Hamcrest, …
• enjoy Spring support for testing,

Cucumber:

• Cucumber is a tool based on Behavior Driven Development (BDD) framework
• Tool for mainly acceptance testing
• Format

given (some context)

when (something happens)

then (some behavioral validation)

 

• Cucumber helps facilitate the discovery and use of a ubiquitous language
  within the team, by giving the two sides of the linguistic divide a place where
  they can meet.
• Cucumber tests interact directly with the developers’ code,
  but they’re written in a medium and language that business stakeholders
  can understand.
• Initially, Cucumber was implemented in Ruby and then extended to Java framework

Report

• Cucumber generates its own HTML format.
• Cucumber is used to test the system rather than testing the particular piece of code

Integration with other tools

• Cucumber can be used along with Selenium, Watir, and Capybara
• Cucumber supports many other languages like Perl, PHP, Python, Net

Features

1. Feature File:

• Feature files are essential part of cucumber which is used to write test automation steps or acceptance tests
• This can be used as live document
• The steps are the application specification.
• All the feature files ends with .feature extension.

2. Feature:

• This gives information about the high-level business functionality (Refer to the previous example) and the purpose of Application under test
• Everybody should be able to understand the intent of feature file by reading the first Feature step

3. Scenario:

• Basically, a scenario represents a particular functionality which is under test
• By seeing the scenario user should be able to understand the intent behind the scenario and what the test is all about
• Each scenario should follow given, when and then format. This language is called as “gherkin”.

4. Scenario Outline:

• Scenario outlines are used when the same test has to be performed with different data set

5. Tags

• Cucumber by default runs all scenarios in all the feature files. In real time projects there could be hundreds of feature file which are not required to run at all times.

For Ex: Feature files related to smoke test need not run all the time. So if you mention a tag as smokeTest in each feature file which is related to smoke test and runs cucumber test with @SmokeTest tag

Example of use of single tags: @SmokeTest

Example of use of multiple tags: @SmokeTest@LoginTest

 

Cucumber vs TestNG: Which is better?

 

Cucumber

TestNG

Cucumber is a collaboration tool, which lets non-technical people write executable specifications. Those executable specifications test your app from the outside – like a black box. Cucumber is not meant to be used as a unit testing tool. It allows to write automated acceptance tests, functional requirements and software documentation into one format that would be understandable by non-technical people as well as testing tools. You can implement your tests using the same language you use to discuss them with the business. Cucumber adds the overhead of plain English (or other native language) to executable code conversion. Good for acceptance testing.

TestNG are unit testing tools. They are great for testing individual classes, but not great for executable specifications that are readable (and writable) by non-technical people. It facilitates to test individual classes. You can group tests using tags. TestNG supports a lot of complicated practices like priorities, grouping, listener etc. Useful when you have to automate large number of test case.