Selenium

Exploring TestNG .xml File

In last chapter, we discussed about Single test case class of two types which can be a soft or hard assert.
In industry, we will never have a single class, instead we will have many classes for every functionality such as ‘loginTestCases.java’, ‘ForgotPasswordTestcases.java’ and many more. So it’s not possible to run each test class separately.
We need to do integration between test classes and so now testng.xml comes into the picture. We need testng.xml for integrating test case classes and many other things which are explained later in this chapter.
To start our project we will follow the below given steps:
- Create ‘LoginTestCases.java’ and ‘ForgotPasswordTestCases.java’.
- Then start putting test cases inside every class.
- Make one test case pass and the other fail in each class which means there will be total 2 test cases that will pass [one in every class].

See the below given code.

package com.javabykiran; 
import org.testng.Assert;
import org.testng.annotations.Test; 
public class LoginTestCases {
    @Test
    public void checkBlankUname() {
        Assert.assertEquals("blank not allowed", "blank not allowed");
    }
    @Test
    public void checkSpacialCharUname() {
        Assert.assertEquals("sp char not allowed", "sp char allowed");
    }
}

package com.javabykiran; 
import org.testng.Assert;
import org.testng.annotations.Test;
public class ForgotPasswordTestCases { 
    @Test
    public void enterUnameLabel() {
        Assert.assertEquals("Enter Uname as jbk", "Enter Uname as jbk");
    }
    @Test
    public void enterPasswdLabel() {
        Assert.assertEquals("javabykiran is passsword", "javabykiran is not passsword");
    }
}

First right click on project >> go to ‘convert to testng’ >> next >> finish.
Now we can see “testng.xml” generated in our project.

This file will look like: testng .xml file

Now right click on ‘testng.xml’ >> run as >> testNG suite.

See console or go to window as shown below: testng suite

See test-output folder and open “index.html” in a browser.
In live company projects this file is created only once.
Every automation engineer has to place his class name in the file once test class is created so that it will start running in integration suite.
One more thing is that we need to explore “@Test” annotation in detail.

@Test
(priority = 0, alwaysRun = true, dependsOnMethods = ”checkSpacialCharUname”)
    public void checkBlankUname() {
        Assert.assertEquals(“blank not allowed”, “blank not allowed”);
}

In the above snap priority is attribute of test annotation, we have many attributes like this for test annotation.

Attribute Names	Description
alwaysRun	If set to true, this test method will always be run even if it depends on a method that failed.
dataProvider	The name of the data provider for this test method.
dataProviderClass	The class where to look for the data provider. If not specified, the data provider will be looking at the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static in the specified class.
dependsOnGroups	The list of groups this method depends on.
dependsOnMethods	The list of methods this method depends on.
description	The description for this method.
enabled	Whether methods on this class/method are enabled.
expectedExceptions	The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.
groups	The list of groups this class/method belongs to.
invocationCount	The number of times this method should be invoked.
invocationTimeOut	The maximum number of milliseconds this test should take for the cumulated time of all the invocation counts. This attribute will be ignored if invocationCount is not specified.
priority	The priority for this test method. Lower priorities will be scheduled first.
successPercentage	The percentage of success expected from this method.
singleThreaded	If set to true, all the methods on this test class are guaranteed to run in the same thread, even if the tests are currently being run with parallel=”methods”. This attribute can only be used at the class level and it will be ignored if used at the method level. Note: this attribute used to be called sequen- tial (now deprecated).
timeOut threadPoolSize	The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocation Count. Note: this attribute is ignored if invocation Count is not specified

Important is creating a group.

@Test(priority=1,groups=”smoke”)
public void checkBlankUname() {
    Assert.assertEquals(“blank not allowed”, “blank not allowed”);
}
@Test(priority=2,groups=”critical”) 
public void checkSpacialCharUname() {
    Assert.assertEquals(“sp char not allowed”, “sp char allowed”);
}

Now in testng.xml we can write about this group.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test name="Test">
        <groups>
            <run>
                <include name="smoke"></include>
            </run>
        </groups> 

        <packages>
            <package name="com.javabykiran"></package>
        </package>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Generally people create groups like critical medium as per severity, some may create like smoke, regression etc.

Different flavors of “ testng.xml” as below.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
    <test name="Nopackage">
        <classes>
            <class name="NoPackageTest" />
        </classes>
    </test>

    <test name="Regression1">
        <classes>
            <class name="test.sample.ParameterSample"/>
            <class name="test.sample.ParameterTest"/>
        </classes>
    </test>
</suite>

You can specify package names instead of class names:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
    <test name="Regression1">
        <packages>
            <package name="test.sample"/>
        </packages>
    </test>
</suite>
    <test name="Regression1">
        <groups>
            <run>
                <exclude name="brokenTests"/>
                <include name="checkinTests"/>
            </run>
        </groups>
        <classes>
            <class name="test.IndividualMethodsTest">
                <methods>
                    <include name="testMethod"/>
                </methods>
            </class>
        </classes>
    </test>

In this example, TestNG will look at all the classes in the package “test.sample” and will retain only classes that have TestNG annotations.
You can also specify groups and methods to be included and excluded:
You can also define new groups inside “testng.xml” and specify additional details in attributes, such as whether to run the tests in parallel, how many threads to use, whether you are running Junit tests, etc.
By default, TestNG will run your tests in the order they are found in the XMLfile. If you want, the classes and methods listed in this file to run in an unpredictable order, set the preserve-order attribute to false.

<test name="Regression1" preserve-order="false">
    <classes>
        <class name="test.Test1">
            <methods>
                <include name="m1" />
                <include name="m2" />
            </methods>
        </class>
        <class name="test.Test2" />
    </classes>
</test>

Important is creating a group.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
    <suite name="Parallel test suite" parallel="methods" thread- count="2">
        <test name="Regression 1">
            <classes>
                <class name="com.parallel.TestParallelOne"/>
            </classes>
        </test>
    </suite>

package com.javabykiran;
import org.testng.annotations.Test; 
public class TestParallelOne {
    @Test
    public void testCaseOne() {
        // Printing Id of the thread on using which test method 
        // got executed
        System.out.println("Test Case One with Thread Id:- "
        + Thread.currentThread().getId());
    }
    @Test
    public void testCaseTwo() {
        // Printing Id of the thread on using which test method 
        // got executed
        System.out.println("Test Case two with Thread Id:- "
        + Thread.currentThread().getId());
    }
}

Now run above example and try observing output, it should look like below:

The above result shows that two methods are executed using different threads. When we run the same “testng.xml” again, the result may vary. The assigning of the thread is taken care by the processor. So we can’t say which thread is going to execute which method.

Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the test-output folder of TestNg project. This XML file contains the necessary information to rerun only these methods that failed, utilizing this we can easily rerun the failed tests without having to run the entire test suite.

For that we have to follow below steps:

Create one class with two to three TCs as below:

package com.jbk;
import org.testng.Assert;
import org.testng.annotations.Test;
public class TestNgTest { 
    @Test(priority=1)
    public void FirstTest(){ 
        Assert.assertEquals(“test”, “test1”);
    }
    @Test(priority=2)
    public void SecondTest(){ 
        Assert.assertEquals(“test”, “test”);
    }
    @Test(priority=3)  
    public void ThirdTest(){
        Assert.assertEquals(“TestngTest”, “TestngTest1”);
    }
}

As seen above we are going to fail TC intentionally as expected and actual result differ.

Assert.assertEquals(“test”, “test1”)

Now Run the TestNG class and you will able to see Test-Output folder as seen below:

Locate the testng - failed.xml file
Project -> test-output -> testng-failed.xml

<?xml version=”1.0” encoding=”UTF-8”?>
<!DOCTYPE suite SYSTEM “http://testng.org/testng-1.0.dtd”>
    <suite name=”Failed suite [Failed suite [Default suite]]”>
        <test name=”Default test(failed)(failed)”>
            <classes>
                <class name=”com.jbk.TestNgTest”>
                    <methods>
                        <include name=”ThirdTest”/>
                        <include name=”FirstTest”/>
                    </methods>
                </class> <!-- com.jbk.TestNgTest -->
            </classes>
        </test> <!-- Default test(failed)(failed) -->
    </suite> <!-- Failed suite [Failed suite [Default suite]] -->

Open the file and as above we can see that the only failed method are included in include tag in xml file. Now if we rerun this file only Failed TC will execute.

Selenium

Exploring TestNG .xml File

How to generate testng.xml?

Attributes

Creating a Group

How to run Parallel Tests using TestNG?

Re-Run Failed Test Cases in TestNG