Issue with dependsOnMethods Not Working When Using a Factory with Parallelized Methods Grouped by Instances

[jmezquita-denodo]

TestNG Version

7.10.2

Expected behavior

After each instance completes, the dependencies should be reset, ensuring that dependsOnMethods is respected for each configuration

Setup for config: config1
Running testA with config: config1
Running testB with config: config1
Running testC with config: config1
Running testD with config: config1
Teardown for config: config1
Setup for config: config3
Running testA with config: config3
Running testB with config: config3
Running testC with config: config3
Running testD with config: config3
Teardown for config: config3
Setup for config: config4
Running testA with config: config4
Running testB with config: config4
Running testC with config: config4
Running testD with config: config4
Teardown for config: config4
Setup for config: config2
Running testA with config: config2
Running testB with config: config2
Running testC with config: config2
Running testD with config: config2
Teardown for config: config2

Actual behavior

When running a test class that uses a factory with parallel execution set to methods and instance grouping enabled, dependsOnMethods is not respected after the first configuration is executed.

Setup for config: config1
Running testA with config: config1
Running testB with config: config1
Running testC with config: config1
Running testD with config: config1
Teardown for config: config1
Setup for config: config3
Running testC with config: config3
Running testB with config: config3
Running testD with config: config3
Running testA with config: config3
Teardown for config: config3
Setup for config: config4
Running testA with config: config4
Running testC with config: config4
Running testB with config: config4
Running testD with config: config4
Teardown for config: config4
Setup for config: config2
Running testA with config: config2
Running testB with config: config2
Running testD with config: config2
Running testC with config: config2
Teardown for config: config2

Is the issue reproducible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

import org.testng.annotations.Factory;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.AfterClass;
import org.testng.annotations.DataProvider;

public class ParallelTestWithFactory {

    private String config;

    public ParallelTestWithFactory(String config) {
        this.config = config;
    }

    @BeforeClass
    public void setUp() {
        System.out.println("Setup for config: " + config);
    }

    @AfterClass
    public void tearDown() {
        System.out.println("Teardown for config: " + config);
    }

    @Test
    public void testA() {
        System.out.println("Running testA with config: " + config);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Test(dependsOnMethods = {"testA"})
    public void testB() {
        System.out.println("Running testB with config: " + config);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Test(dependsOnMethods = {"testB"})
    public void testC() {
        System.out.println("Running testC with config: " + config);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Test(dependsOnMethods = {"testC"})
    public void testD() {
        System.out.println("Running testD with config: " + config);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Factory
    public static Object[] factoryMethod() {
        return new Object[] {
            new ParallelTestWithFactory("config1"),
            new ParallelTestWithFactory("config2"),
            new ParallelTestWithFactory("config3"),
            new ParallelTestWithFactory("config4")
        };
    }

    @DataProvider(name = "configProvider", parallel = true)
    public static Object[][] configProvider() {
        return new Object[][] {
            {"config1"},
            {"config2"},
            {"config3"},
            {"config4"}
        };
    }
}

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test examples suite" parallel="methods" thread-count="5" time-out="600000">
  
  <test name="ParallelTestWithFactory" parallel="methods" group-by-instances="true">

    <classes>
      <class name="ParallelTestWithFactory" />
    </classes>
  </test>

</suite>

Contribution guidelines

Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.

[krmahadevan]

@jmezquita-denodo - quick question. Do you know if this combo worked in the past. If yes can you please let us know in which version it worked fine? Just trying to see if this was broken in some recent release.

[jmezquita-denodo]

@krmahadevan - As far as I know it does not work in version 7.4.0 neither. It is not something recent, it happened before.