Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cleanup of End-To-End testing part #2989

Merged
merged 5 commits into from
Dec 5, 2023
Merged

cleanup of End-To-End testing part #2989

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e8081b7
cleanup of End-To-End testing part
mstahv Nov 27, 2023
c88fc64
Merge branch 'latest' into bugfix/tutorial-e-2-e-fixes
russelljtdyer Nov 30, 2023
a8e2726
Vale fixes to start.
russelljtdyer Nov 30, 2023
f084059
Initial edits.
russelljtdyer Nov 30, 2023
4d536c3
Second pass at editing.
russelljtdyer Nov 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 22 additions & 91 deletions articles/tutorial/e2e-testing-with-testbench.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,91 +2,26 @@
title: End-to-End Testing
order: 130
page-title: Run End-to-End Browser Tests with Vaadin TestBench
description: Explains how to do full-stack tests on a Flow application using Vaadin TestBench.
description: Explains how to do full-stack tests on a Vaadin Flow application using Vaadin TestBench.
---


= End-To-End Test Applications in Browser

End-to-end (e2e) tests are used to test an entire application. They're much more coarse-grained than unit or integration tests. This makes them well suited to check that the application works as a whole, and catch any regressions that may be missed by more specific tests.

End-to-end tests are executed in a browser window. Vaadin TestBench controls the browser window using Selenium WebDriver.
End-to-end tests are executed in a browser window, simulating user interactions. Vaadin TestBench controls the browser window using Selenium WebDriver.

.Vaadin TestBench is a Commercial Product
[NOTE]
====
The end-to-end tests use https://vaadin.com/testbench[Vaadin TestBench], which is a commercial tool that's part of the Vaadin Pro Subscription. You can get a free trial at https://vaadin.com/trial. All Vaadin Pro tools and components are free for students through the https://education.github.com/pack[GitHub Student Developer Pack]. For an open source alternative for TestBench, you can get similar results with plain https://www.selenium.dev[Selenium WebDriver] or https://playwright.dev[Playwright].
====


== The Base Test Class

Vaadin TestBench contains handy base classes that you can use as a basis for your e2e tests. The JUnit5 version is called `BrowserTestBase`. It can be used alone, if you orchestrate starting and stopping your server. For example, it can be used with Maven executions in `pre-integration-test` and `post-integration-test`, and to execute the actual test in integration test phase.

In Spring Boot applications, it's easier to use the same @SpringBootTest annotation that you already used in the previous phase to ensure a running server during the browser test execution.

First, create a new class, [classname]`LoginE2ETest` in the `com.example.application.it` package. Be sure to place it in `src/test/java` and not `src/main/java`.

.`LoginE2ETest.java`
[source,java]
----
package com.example.application.it;

import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import com.vaadin.testbench.IPAddress;
import com.vaadin.testbench.ScreenshotOnFailureRule;
import com.vaadin.testbench.parallel.ParallelTest;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Rule;
import org.slf4j.LoggerFactory;

public abstract class LoginE2ETest extends ParallelTest {
private static final String SERVER_HOST = IPAddress.findSiteLocalAddress();
private static final int SERVER_PORT = 8080;
private final String route;

static {
// Prevent debug logging from Apache HTTP client
Logger root = (Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);
root.setLevel(Level.INFO);
}

@BeforeClass
public static void setupClass() {
WebDriverManager.chromedriver().setup(); // <1>
}

@Rule // <2>
public ScreenshotOnFailureRule rule = new ScreenshotOnFailureRule(this, true);

@Before
public void setup() throws Exception {
super.setup();
getDriver().get(getURL(route)); // <3>
}

protected LoginE2ETest(String route) {
this.route = route;
}

private static String getURL(String route) {
return String.format("http://%s:%d/%s", SERVER_HOST, SERVER_PORT, route);
}
}
----
<1> Start by invoking the *Chrome* `WebDriverManager` before any test method is invoked. TestBench doesn't invoke the WebDriver manager.
<2> `ScreenshotOnFailureRule` tells TestBench to grab a screenshot before exiting, if a test fails. This can help you understand what went wrong when tests don't pass.
<3> Open the browser to the correct URL before each test. For this, you need the host name where the application runs (i.e., "localhost" in development), the port the server uses, which is set to 8080 in application.properties, and information about the route from which to start.
The end-to-end tests use https://vaadin.com/testbench[Vaadin TestBench], which is a commercial tool that's part of the Vaadin Pro Subscription. You can get a free trial at https://vaadin.com/trial[Vaadin Commercial Trial]. All Vaadin Pro tools and components are free for students through the https://education.github.com/pack[GitHub Student Developer Pack]. For an open source alternative to TestBench, you can get similar results with https://www.selenium.dev[Selenium WebDriver] or https://playwright.dev[Playwright].


== Test the Login View

Now that your setup is complete, you can start developing your first test: ensuring that a user can log in. For this test, you need to open the base URL.
In this tutorial, you'll build a test that ensures a user can log in. For this test, you'll need to open the base URL, fill in the user name and password, then click the login button, and make certain the actual application view opens.

Create a new class, [classname]`LoginIT`, in the same package as [classname]`LoginE2ETest`. The test validates that logging in with the correct user and password succeeds.
First, create a new class, [classname]`LoginE2ETest` in the `src/test/java/com/example/application/it` directory. The test validates that logging in with the correct user and password succeeds.

.`LoginE2ETest.java`
[source,java]
Expand All @@ -107,8 +42,8 @@ import static org.junit.jupiter.api.Assertions.assertFalse;

// <1>
// @RunLocally(Browser.FIREFOX)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class LoginE2ETest extends BrowserTestBase { // <2>
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) // <2>
public class LoginE2ETest extends BrowserTestBase { // <3>

@Autowired
Environment env;
Expand All @@ -121,10 +56,10 @@ public class LoginE2ETest extends BrowserTestBase { // <2>
@BeforeEach
void openBrowser() {
getDriver().get("http://localhost:" +
env.getProperty("local.server.port") + "/"); // <3>
env.getProperty("local.server.port") + "/"); // <4>
}

@BrowserTest // <4>
@BrowserTest // <5>
public void loginAsValidUserSucceeds() {
// Find the LoginForm used on the page, using a
// typed selector API provided by TestBench
Expand All @@ -147,22 +82,20 @@ public class LoginE2ETest extends BrowserTestBase { // <2>

}
----
<1> This optional annotation specifies the test to be run on the local machine and using Firefox.
The default is Chrome.
<2> The super class `BrowserTestBase` provides handy helper methods and configures TestBench.
<3> The `openBrowser` method is annotated to be executed before each actual tests.
The URL points to local test server with the random port SpringBootTest has selected.
The browser should be automatically redirected to the login screen.
<4> BrowserTest annotation is a TestBench extension of the better known `Test` annotation, that is handy if you decide to extend your end-to-end tests to cover multiple browsers at some point.
<1> This optional annotation specifies the test to be run on the local machine and using Firefox. The default is Chrome.
<2> This annotation instructs Spring Boot test helpers to start an actual web server for this test. A random port is assigned so that running this test doesn't potentially conflict with a running development server. Alternatively, you can package the whole application before the end-to-end tests and use Maven integration test phases together with Maven Failsafe plugin to ensure the application server is running during the test.
<3> The super class `BrowserTestBase` provides handy helper methods and configures TestBench.
<4> The `openBrowser` method is annotated to be executed before each actual test. The URL points to a local test server with the random port SpringBootTest selected. The browser should be redirected automatically to the login screen.
<5> `BrowserTest` annotation is a TestBench extension of the better known `Test` annotation. This is useful if you decide to extend your end-to-end tests to cover multiple browsers at some point.

Right-click [classname]`LoginE2ETest.java` and select *Run 'LoginE2ETest'*.
Right-click [classname]`LoginE2ETest.java` and select _Run 'LoginE2ETest'_.


== Create a View Object

You can now add a second test, one to validate that you can't log in with an invalid password.

For this test, you need to write the same code to access the components in the view as you did for the first test. To make your tests more maintainable, you can create for each view a view object -- otherwise known as a call page object or element class. A view object provides a high-level API to interact with the view and hides the implementation details.
For this test, you'll need to use the same code for accessing the components in the view as you did for the first test. To make your tests more maintainable, you can create a view object for each view -- otherwise known as a call page object or element class. A view object provides a high-level API to interact with the view and hides the implementation details.

For the login view, create the [classname]`LoginViewElement` class in a new package, `com.example.application.it.elements`:

Expand Down Expand Up @@ -203,19 +136,17 @@ public class LoginViewElement extends VerticalLayoutElement {

.Class Hierarchies Must Match
[CAUTION]
====
To make the correct functionality available from superclasses, the hierarchy of the view object should match the hierarchy of the view (i.e., `public class LoginView extends VerticalLayout` vs `public class LoginViewElement extends VerticalLayoutElement`).
====
To make the correct functionality available from superclasses, the hierarchy of the view object should match the hierarchy of the view (i.e., `public class LoginView extends VerticalLayout` vs. `public class LoginViewElement extends VerticalLayoutElement`).

Adding the `@Attribute(name = "class", contains = "login-view")` annotation allows you to find the [classname]`LoginViewElement` using the TestBench query API. The following is an example of this:
Adding the `@Attribute(name = "class", contains = "login-view")` annotation allows you to find a specific [classname]`LoginViewElement` using the TestBench query API, in this case by CSS class name. The following is an example of this:

.Finding a LoginViewElement using the TestBench query API
.Finding a LoginViewElement Using the TestBench Query API
[source,java]
----
LoginViewElement loginView = $(LoginViewElement.class).onPage().first();
----

The annotation searches for the `login-view` class name, which is set for the login view in the constructor. The [methodname]`onPage()` call ensures that the whole page is searched. By default, a `$` query starts from the active element.
The annotation searches for the `login-view` CSS class name, which is set for the [classname]`LoginLiew` in the constructor. The [methodname]`onPage()` call ensures that the whole page is searched. By default, a `$` query starts from the active element.

Now that you have the [classname]`LoginViewElement` class, you can simplify your [methodname]`loginAsValidUserSucceeds()` test to be this:

Expand All @@ -229,7 +160,7 @@ public void loginAsValidUserSucceeds() {
}
----

Add a test to use an invalid password as follows:
You might also add a test to use an invalid password as follows:

.`LoginE2ETest.java`
[source,java]
Expand All @@ -243,7 +174,7 @@ public void loginAsInvalidUserFails() {

You can continue testing the other views by creating similar view objects and IT classes.

If you're building a large application, it's probably better to make slower end-to-end tests executed only when requested separately. You can do this by using https://maven.apache.org/surefire/maven-failsafe-plugin/[Maven Failsafe plugin] or using the https://junit.org/junit5/docs/current/user-guide/#writing-tests-tagging-and-filtering[tagging feature in JUnit 5].
If you're building a large application, it's probably better to make slower end-to-end tests execute only when requested separately. You can do this by using https://maven.apache.org/surefire/maven-failsafe-plugin/[Maven Failsafe plugin] or using the https://junit.org/junit5/docs/current/user-guide/#writing-tests-tagging-and-filtering[tagging feature in JUnit 5].

The next part covers how to make a production build of the application and deploy it to a cloud platform.

Expand Down
Loading