This directory contains the sources for the automatic testing of the e3-opcua module.
(The following notes assume this directory is named test
.)
In order to run the test suite, you must install the following system packages:
- python3
- libfaketime
E.g., on CentOS 7 run
sudo yum install -y python3 libfaketime
The following python modules are also needed:
- pytest
- pyepics
- opcua
- run-iocsh
You can use the following pip3 commands (as root) to install them
pip3 install pytest opcua pyepics
pip3 install run-iocsh -i https://artifactory.esss.lu.se/artifactory/api/pypi/pypi-virtual/simple
You must configure the EPICS environment before running the test suite.
For the E3 environment, this requires you to source setE3Env.bash
.
At least EPICS_BASE
and EPICS_HOST_ARCH
need to be set.
Finally, compile the test server that is used by the test suite:
make -C test/server
The test setup consists of three main components:
A simple test opcua server, created using open62541 [1]. The server configuration currently consists of a number of variables provided for testing purposes.
The server listens for connections on opc.tcp://localhost:4840
and the simulated
signals are available in OPC UA namespace 2.
For further information on the server configuration, see simulation server.
A test IOC is provided that translates the OPC UA variables from the test server. The following records are defined:
Record | EPICS Type | OPC-UA Type | Record | EPICS Type | OPC-UA Type |
---|---|---|---|---|---|
TstRamp | ai | Double | |||
VarCheckBool | bi | Bool | VarCheckBoolOut | bo | Bool |
VarCheckSByte | ai | SByte | VarCheckSByteOut | ao | SByte |
VarCheckByte | ai | Byte | VarCheckByteOut | ao | Byte |
VarCheckUInt16 | ai | Uint16 | VarCheckUInt16Out | ao | Uint16 |
VarCheckInt16 | ai | Int16 | VarCheckInt16Out | ao | Int16 |
VarCheckUInt32 | ai | Uint32 | VarCheckUInt32Out | ao | Uint32 |
VarCheckInt32 | ai | Int32 | VarCheckInt32Out | ao | Int32 |
VarCheckUInt64 | ai | Uint64 | VarCheckUInt64Out | ao | Uint64 |
VarCheckInt64 | ai | Int64 | VarCheckInt64Out | ao | Int64 |
VarCheckString | stringin | String | VarCheckStringOut | stringout | String |
VarCheckFloat | ai | Float | VarCheckFloatOut | ao | Float |
VarCheckDouble | ai | Double | VarCheckDoubleOut | ao | Double |
A secondary IOC is provided for use with the negative tests, consisting of two records:
- BadVarName - an analog input that specifies an incorrect OPC-UA variable name
- VarNotBoolean - a binary input that specifies an OPC-UA variable of float datatype
Startup scripts and database files are provided in the cmd/ and db/ subdirectories.
The pytest framework [2] is used to implement the test cases. Individual test cases are provided as python functions (defs) in (opcua_test_cases.py). Under the hood, run_iocsh [3] and pyepics [4] are used for communication with the test IOC.
To add a new test case, simply add a new funtion (def) to (opcua_test_cases.py),
ensuring that the function name begins with the prefix test_
The test points are split into four classes:
-
test_start_server_then_ioc: With the server running, start and stop the test IOC. Parse the IOC output and check it connects and disconnects to the OPC UA server successfully.
-
test_stop_and_restart_server: Start the server, start the IOC. Stop the server, check for appropriate messaging. Start the server, check that the IOC reconnects.
-
test_start_ioc_then_server: Start an IOC with no server running. Then start the server. Check the module reports appropriately.
-
test_disconnect_on_ioc_exit: Start the server. Start an IOC and ensure connection is made to the server. Shutdown the IOC. Check that the subscriptions and sessions are cleanly disconnected.
-
test_server_status: Check the informational values provided by the server are being translated via the module.
-
test_variable_pvget: Start the test IOC and use pvget to read the
TstRamp
PV value multiple times (every second). Check that it is incrementing as a ramp. -
test_read_variable: Read the deafult value of a variable from the opcua server and check it matches the expected value. Parametrised for all supported datatypes (boolean, sbyte, byte, int16, uint16, int32, uint32, int64, uint64, float, double, string.)
-
test_write_variable: Write a known value to the opcua server via the output PV linked to the variable. Read back via the input PV and check the values match. Parametrised for all supported datatypes. (boolean, sbyte, byte, int16, uint16, int32, uint32, int64, uint64, float, double, string.)
-
test_timestamps: Start the test server in a shell session with with a fake time in the past, using libfaketime [5]. Check that the timestamp for the PV read matches the known fake time given to the server. If they match, the OPCUA EPICS module is correctly pulling the timestamps from the OPCUA server (and not using a local timestamp).
-
test_write_performance: Write 5000 variable values and measure time and memory consumption before and after. Repeat 10 times
-
test_read_performance: Read 5000 variable values and measure time and memory consumption before and after. Repeat 10 times
-
test_no_server: Start an OPC-UA IOC with no server running. Check the module reports this correctly.
-
test_bad_var_name: Specify an incorrect OPC UA variable name in an EPICS record. Start the IOC and verify a sensible error is displayed.
-
test_wrong_datatype: Specify an incorrect record type for an OPC UA variable. Binary input record for a float datatype. Check that appripriate errors are displayed.
You can run the test suite from the root of the repository wuth the following command:
pytest -v end2endTest
To view the stdout output from the tests in real-time, you can provide the -s
flag:
pytest -v -s end2endTest
To run all tests in a class:
pytest -v end2endTest -k TestConnectionTests
To run an individual test point:
pytest -v end2endTest -k test_stop_and_restart_server
[2] https://docs.pytest.org/en/stable/
[3] https://gitlab.esss.lu.se/ics-infrastructure/run-iocsh