Merge pull request #1791 from dedis/work-karate-4xiom5-minimal-cross-…

…platform-ui-testing

Minimal cross platform UI testing: First test

fe1-web/src/features/wallet/screens/WalletCreateSeed.tsx

@@ -116,7 +116,9 @@ const WalletCreateSeed = () => {
           <Text>{STRINGS.wallet_welcome_text_wallet_explanation_4}</Text>
         </Text>
         <View style={styles.words}>
-          <Text style={styles.seed}>{seed}</Text>
+          <Text style={styles.seed} testID="seed_wallet_text">
+            {seed}
+          </Text>
         </View>
         <Text style={Typography.paragraph}>
           <Text>{STRINGS.wallet_welcome_text_wallet_explanation_5}</Text>

fe1-web/src/features/wallet/screens/__tests__/__snapshots__/WalletCreateSeed.test.tsx.snap

@@ -475,6 +475,7 @@ exports[`Wallet create seed screen renders correctly 1`] = `
                                       "textAlign": "left",
                                     }
                                   }
+                                  testID="seed_wallet_text"
                                 >
                                   one two three
                                 </Text>

tests/karate/docs/README.md

@@ -14,18 +14,18 @@ We then check that the responses the mock components receive are as expected.
 ## Architecture
 ### Features and Scenarios
 Karate test cases are called scenarios and they are grouped within different feature files.
-Each feature file tests a different message type (i.e. electionOpen, createRollCall etc.). 
+Each feature file tests a different message type (i.e. electionOpen, createRollCall etc.).
 
 Scenarios are written in the Gerkhin syntax using the following keywords:
 
-- **Given**: Prepare the JSON payload to be sent to the component being tested. 
+- **Given**: Prepare the JSON payload to be sent to the component being tested.
 - **When**: Defines the action that is to be performed with the payload.
-For instance, `publish` creates a message of type publish that contains some high-level message data, or `send` to send raw JSON data. 
+For instance, `publish` creates a message of type publish that contains some high-level message data, or `send` to send raw JSON data.
 - **Then**: Asserts that the action taken in the 'When' step has the expected outcome.
 - **And**: Connector that can be used after any of the other keywords.
 
 ### Background section
-Code defined in the background section of a feature file runs before each scenario. This is especially useful for: 
+Code defined in the background section of a feature file runs before each scenario. This is especially useful for:
 
 - **Sharing scopes with other features**: The call to `read(classpath: "path/to/feature")` is used to make the current feature share the same scope as the feature that is called.
 This means they share definitions (def variables) and configurations.
@@ -36,15 +36,15 @@ For instance, reading `mockClient.feature` exposes functions like `createMockFro
 `simpleScenarios.feature` contains many such useful setup steps.
 
 ### Data model
-To generate valid message data for JSON payloads dynamically, a simplified version of the model is implemented in Java code. 
+To generate valid message data for JSON payloads dynamically, a simplified version of the model is implemented in Java code.
 Mock components can create valid objects (for instance LAO, RollCall, Elections etc.), that can be used to handcraft messages.
-These objects also provide functions to override their own data to some invalid values, to craft invalid messages. 
+These objects also provide functions to override their own data to some invalid values, to craft invalid messages.
 
 Some care needs to be taken if more of these functions to override valid data are implemented in the future.
 For example, when setting an invalid LAO name to test if the server rejects this, the LAO id also needs to be recomputed.
 The LAO id depends on the LAO name, and if the id is not recomputed the test might fail due to invalid LAO id, and not because the name was invalid.
 As of February 2024, there is no way to distinguish this as only the error code is asserted and not the error message.
-This is a possible improvement that could be done in the future. 
+This is a possible improvement that could be done in the future.
 
 <div align="center">
   <img src="images/architecture_diagram.png" alt="Architecture"/>
@@ -105,31 +105,56 @@ The Karate plugin for IntelliJ is also recommended.
 To test a backend, you don't need more setup that what is needed to build that backend.
 Use the resources provided by those projects.
 
-### Android Front-end
+### Frontend
+
+#### Appium
+Appium provides the API that will allow us to test both frontends on multiple platforms. You can install the CLI version with the command `npm install -g appium` or install the [Desktop App](https://github.com/appium/appium-desktop/releases/) instead.
 
-To test the Android Frontend, you need to have an Android emulator installed.
-The easiest way to achieve it is to install it through [Android Studio](https://developer.android.com/studio) :
-Go to `Tools -> AVD Manager` and create an emulator.
+#### Android
+You need to add the Android driver to Appium with the following command.
+```shell
+appium driver install uiautomator2
+```
 
-Then you need to install Appium.
-You can install either the command line app with `npm install -g appium`, or the [Desktop App](https://github.com/appium/appium-desktop/releases/).
+Then, you need to create an Android emulator in Android studio if you don't have one yet. The easiest way to achieve it is to install it through [Android Studio](https://developer.android.com/studio): Go to `Tools -> Device Manager` and create a virtual device.
 
 Finally, you need to set the environment variable `ANDROID_HOME` (The previous name was`ANDROID_SDK_ROOT` and it still works) to your Android SDK installation.
 Find it by opening Android Studio and going to `Tools -> SDK Manager`.
 It stands next to `Android SDK Location`.
 
-If your Computer runs on Windows : we strongly advise that you do not use a VM or WSL.
-You will encounter problems you would not have otherwise, some of which might even be technically impossible to solve.
+If your Computer runs on Windows: we strongly advise that you do not use a VM or WSL. You will encounter problems you would not have otherwise, some of which might even be technically impossible to solve.
 
-### Web Front-end
+### Google Chrome & Microsoft Edge
+Make sure you have Google Chrome and/or Microsoft Edge installed.
+
+Then, install the Appium driver.
+```
+appium driver install chromium
+```
+### Firefox
+Make sure you have Firefox installed. Then download the latest release of [geckodriver](https://github.com/mozilla/geckodriver/releases) and add it to your path.
+
+Then, install the Appium driver.
+```
+appium driver install gecko
+```
+
+### Safari
+Install the Appium driver.
+```
+appium driver install safari
+```
 
-Make sure you have [Google Chrome](https://www.google.com/intl/en/chrome/) and [npm](https://nodejs.org/en/download/) installed.
+Then, allow remote automation of Safari.
+```
+safaridriver --enable
+```
 
 ## Running the Tests
 
 ### Backend
 
-Build the backend you want to test. 
+Build the backend you want to test.
 Follow the steps described in the corresponding subproject.
 Keep the executables in their default build location, Karate will find them there.
 
@@ -150,51 +175,39 @@ With the Karate plugin for IntelliJ, the full tests can also be run directly fro
 
 
 ### Android Front-end
-
 Build the application by running `./gradlew assembleDebug` in the corresponding directory.
 
-Start the Android Emulator.
-Start Appium : if you use the GUI, delete the text in Host and Port and click on the start server button.
-If you use the terminal, run `appium`.
-
-With Android Hedgehog the emulator can either run in a tool window or a standalone window.
-(To have it in a standalone window, go to `File -> Settings -> Tools -> Emulator` and unselect `Launch in a tool window`).
-- Standalone window : \
-The emulator window name should match : `Android Emulator - avd:id` \
-Ex: `Android Emulator - Pixel_4_API_30:5554`
-- Tool window : open the `Extended Controls` (the 3 points above the emulator)
-  - The Extended Controls window name is the `avd` but with ' ' instead of '_' \
-    Ex: `Pixel 4 API 30` for an `avd` of `Pixel_4_API_30`
-  - Go to `Help`, under `Emulator ADB serial number` it should match `emulator-id`\
-    Ex: `emulator-5554`
-
-<img src="images/emulator_extended_controls.png" alt="Emulator" width="50%">
-<em>Emulator standalone window with Extended Controls.</em><br><br>
+Then, start an emulator from Android Studio and launch the Appium server (using the command `appium`).
 
+Finally run the tests.
+```
+mvn test -Dkarate.env=android -Dtest=FrontEndTest#fullTest
+```
 
-Make sure the [karate-config](src/test/java/karate-config.js) is correct.
-More precisely :
+In case you have multiple emulators running, you may specify one by avd id. To find the avd id of some emulator, go to the Device Manager (`Tools -> Device Manager`) and follow the steps in the image below.
 
-- `deviceName` is set to `emulator-<id>`, here it would be `emulator-5554`.
-- `avd` is set to the avd name indicated on the emulator window, here it would be `Pixel_4_API_30`.
+![Find avd id Android Studio](./images/android_studio_find_avd_id.png)
 
-Run the test with :
-`mvn test -DargLine="-Dkarate.env=android"`
+Once you have the avd id of your emulator, you can use the command below to run the tests on this specific emulator.
+```shell
+mvn test -Dkarate.env=android -Davd=<avd_id> -Dtest=FrontEndTest#fullTest
+#e.g. mvn test -Dkarate.env=android -Davd=Galaxy_Note_9_API_29 -Dtest=FrontEndTest#fullTest
+```
 
 ### Web Front-end
-
 Build the app with `npm run build-web` in the corresponding directory.
 
-If your Chrome installation is not one of these :
-
-- mac: `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`
-- win: `C:/Program Files (x86)/Google/Chrome/Application/chrome.exe` \
-  (You should check, it is also common for Chrome to installed in `/Programm Files/` rather than `/Program Files (x86)/`)
+Launch the Appium server (with `appium`).
 
-You need to set the executable manually in the driver definition in the [Web page object](src/test/java/fe/utils/web.feature) (line 6).
-
-Change the line from `* configure driver = { type: 'chrome' }`to `* configure driver = { type: 'chrome', executable: 'PATH' }`\
-where PATH is the path to your Chrome installation.\
+Run the tests.
+```
+mvn test -Dkarate.env=web -Dtest=FrontEndTest#fullTest
+```
 
-Run the test with :
-`mvn test -DargLine="-Dkarate.env=web"`
+The following options are available (option names must be prefixed by `-D`).
+| Name         | Description                                    | Default                                   |
+|--------------|------------------------------------------------|-------------------------------------------|
+| browser      | One of 'chrome', 'safari', 'edge' or 'firefox' | 'chrome'                                  |
+| url          | URL of the web app                             | 'file:../../fe1-web/web-build/index.html' |
+| screenWidth  | Width of the browser                           | 1920                                      |
+| screenHeight | Height of the browser                          | 1080                                      |