From 75bb0d297b0624120fd62bea2ee9d5dfc400a18f Mon Sep 17 00:00:00 2001 From: Dale Emery Date: Mon, 12 Apr 2021 10:10:13 -0700 Subject: [PATCH] GEODE-8899: Upgrade to Gradle v6.8.3 (#6280) * GEODE-8899: Upgrade to Gradle v6.8.3 Updated Geode's build system to use Gradle 6.8.3: - Updated the gradle wrapper to use Gradle 6.8.3. - Updated geode-assembly/build.gradle to make the ivy "repository" compatible with Gradle 6.8.3. - Changed ./gradle.properties to specify 'minimumGradleVersion=6.8'. - Updated the expected-pom.xml file to the format that Gradle 6.8.3 produces. - Updated all Gradle plugins to the latest versions. - In buildSrc/build.gradle, added groovy source sets for testing, to allow running tests of plugin code. - Updated common "test isolation" code to be compatible with Gradle 6.8.3. See below for details. - Updated the RepeatTest task to be compatible with Gradle 6.8.3. See below for details. - Rewrote the Dockerized test plugin to be compatible with Gradle 6.8.3. See below for details. - Added a new 'dunitDockerJVM' project property to allow developers run tests in Docker on macOS. To do this, use -PtestJVM to identify the mac's JVM, and use -PdunitDockerJVM to specify the JVM for the test worker process running in Docker. - Updated the BUILDING.md file for Gradle 6.8.3. As part of this I added two versions of the Apache copyright notice to the etc dir: A plain text version and an xml version that IntelliJ IDEA can import into its copyright settings. ===== Test isolation code ===== Some code for isolating tests is used by the RepeatTest task and the Dockerized test plugin. This code was updated to be compatible with Gradle v6.8.3, and to somewhat reduce its dependence on Gradle internal implementation details: - WorkingDirectoryIsolator adjusts a given ProcessBuilder to give it a unique working directory. - AdjustableProcessLauncher applies a given Consumer (such as the isolator, above) to adjust each ProcessBuilder before launching the process. - LauncherProxyWorkerProcessBuilder and LauncherProcessWorkerProcessFactory forcefully update each Gradle exec handle to use a given ProcessLauncher to launch test worker processes. - Executers and Workers offer convenience methods for constructing TestExecuters and ProcessWorker builders and factories. ===== RepeatTest task update ===== Geode's RepeatTest task uses a custom TestExecuter that was copied from Gradle 5.5 (or earlier) and modified. This custom class is now updated to be compatible with Gradle 6.8.3, and renamed from OverriddenTestExecutor to RepeatableTestExecuter. ===== Dockerized test plugin rewrite ===== Removed the modified copy of pedjak's Dockerized test plugin. Configuration ------------- - Rewrote gradle/docker.gradle and renamed it as gradle/multi-process-test.gradle. - Moved the code that translates project properties into configuration settings from the plugin to the config class. Renamed the config class as DockerTestWorkerConfig. - Moved code that configures command lines and environments for Docker test workers based on project properties. Thise code was defiend as a closure in a build script. It is now a method in DockerTestWorkerConfig. - Changed command line options that CI scripts pass to Gradle to run tests in Docker: - Specify the --max-workers Gradle option to configure the maximum number of workers. - Specify the 'testMaxParallelForks' project property to configure the maximum number of forks for each test task. - Remove the 'dunitParallelForks' option. Implementation -------------- The Dockerized test plugin uses these classes to launches processes in Docker containers: - DockerProcess overrides java.lang.Process to represent a test worker process running in Docker. - DockerProcessLauncher launches each test worker process in a Docker container, and creates a DockerProcess to represent and manage it. - Other classes described in the "Test isolation code" section, above. The plugin overrides several of Gradle's internal classes to allow test worker processes running in Docker containers to communicate with Gradle: - DockerConnectionAcceptor implements Gradle's ConnectionAcceptor, which (among other responsibilities), produces a "multi-choice address" that a test worker process can use to connect to Gradle's messaging server. Gradle's default implementation produces multi-choice addresses that processes in Docker containers cannot use. - DockerMessagingServer implements Gradle's MessagingServer using DockerConnectionAcceotpr. - WildcardBindingInetAddressFactory overrides Gradle's InetAddressFactory to instruct messaging servers to listen on an address that processes in Dockerized containers can connect to. Timeouts -------- Due to a recent surge of CI problems due to delays in Docker operations, I rewrote the plugins's timeout mechanism to make it configurable and much more robust: - Added a 'dockerTimeout' project property, defaulting to 5 minutes. Each Docker operation will throw an exception if its duration exceeds this value. - Log a warning if any Docker operation takes longer than 1 minute. This can help us to see how often significant delays occur in CI, while still allowing tests to proceed. - These exceptions and warnings identify the project for which test worker process is being launched. This can help us to see whether there are project-specific factors that affect the delays. * Fix problems identified by rhoughton-pivot --- BUILDING.md | 139 ++-- .../src/test/resources/expected-pom.xml | 154 +--- build.gradle | 10 +- .../DefaultWorkerSemaphore.groovy | 71 -- .../DockerizedJavaExecHandleBuilder.groovy | 108 --- .../DockerizedTestExtension.groovy | 58 -- .../DockerizedTestPlugin.groovy | 188 ----- .../ExitCodeTolerantExecHandle.groovy | 92 --- .../dockerizedtest/WorkerSemaphore.groovy | 28 - .../org/apache/geode/gradle/RepeatTest.groovy | 66 -- .../geode/gradle/testing/Executers.groovy | 59 ++ .../geode/gradle/testing/Workers.groovy | 47 ++ .../dockerized/DockerTestWorkerConfig.groovy | 92 +++ .../dockerized/DockerizedTestPlugin.groovy | 137 ++++ .../gradle/testing/repeat/RepeatTest.groovy | 59 ++ .../dockerizedtest/DockerizedExecHandle.java | 689 ------------------ .../DockerizedExecHandleRunner.java | 102 --- .../ForciblyStoppableTestWorker.java | 45 -- .../ForkingTestClassProcessor.java | 153 ---- .../dockerizedtest/NoMemoryManager.java | 59 -- .../plugins/dockerizedtest/TestExecuter.java | 119 --- .../RunInSubdirectoryTestFramework.java | 102 --- .../dockerized/DockerConnectionAcceptor.java | 114 +++ .../dockerized/DockerMessagingServer.java | 69 ++ .../testing/dockerized/DockerProcess.java | 287 ++++++++ .../dockerized/DockerProcessLauncher.java | 227 ++++++ .../WildcardBindingInetAddressFactory.java | 32 + .../isolation/WorkingDirectoryIsolator.java | 95 +++ .../process/AdjustableProcessLauncher.java | 44 ++ .../LauncherProxyWorkerProcessBuilder.java | 160 ++++ .../LauncherProxyWorkerProcessFactory.java | 60 ++ .../testing/process/ProcessLauncher.java | 21 + .../gradle/testing/process/Reflection.java | 99 +++ .../repeat/RepeatableTestExecuter.java} | 77 +- ...m.github.pedjak.dockerized-test.properties | 1 - .../geode-dockerized-test.properties | 1 + ci/scripts/execute_build.sh | 2 +- ci/scripts/execute_tests.sh | 2 +- etc/apache-copyright-notice.txt | 12 + etc/intellij-apache-copyright-notice.xml | 6 + geode-assembly/build.gradle | 6 +- gradle.properties | 4 +- gradle/docker.gradle | 149 ---- gradle/multi-process-test.gradle | 83 +++ .../standard-subproject-configuration.gradle | 2 +- gradle/test.gradle | 20 +- gradle/wrapper/gradle-wrapper.jar | Bin 55190 -> 59203 bytes gradle/wrapper/gradle-wrapper.properties | 2 +- gradlew | 53 +- gradlew.bat | 43 +- 50 files changed, 1916 insertions(+), 2332 deletions(-) delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DefaultWorkerSemaphore.groovy delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedJavaExecHandleBuilder.groovy delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestExtension.groovy delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestPlugin.groovy delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/ExitCodeTolerantExecHandle.groovy delete mode 100644 buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/WorkerSemaphore.groovy delete mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/RepeatTest.groovy create mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Executers.groovy create mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Workers.groovy create mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerTestWorkerConfig.groovy create mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerizedTestPlugin.groovy create mode 100644 buildSrc/src/main/groovy/org/apache/geode/gradle/testing/repeat/RepeatTest.groovy delete mode 100755 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandle.java delete mode 100644 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandleRunner.java delete mode 100644 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForciblyStoppableTestWorker.java delete mode 100644 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForkingTestClassProcessor.java delete mode 100644 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/NoMemoryManager.java delete mode 100644 buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/TestExecuter.java delete mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/RunInSubdirectoryTestFramework.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerConnectionAcceptor.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerMessagingServer.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcess.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcessLauncher.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/WildcardBindingInetAddressFactory.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/isolation/WorkingDirectoryIsolator.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/process/AdjustableProcessLauncher.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessBuilder.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessFactory.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/process/ProcessLauncher.java create mode 100644 buildSrc/src/main/java/org/apache/geode/gradle/testing/process/Reflection.java rename buildSrc/src/main/java/org/apache/geode/gradle/{OverriddenTestExecutor.java => testing/repeat/RepeatableTestExecuter.java} (63%) delete mode 100644 buildSrc/src/main/resources/META-INF/gradle-plugins/com.github.pedjak.dockerized-test.properties create mode 100644 buildSrc/src/main/resources/META-INF/gradle-plugins/geode-dockerized-test.properties create mode 100644 etc/apache-copyright-notice.txt create mode 100644 etc/intellij-apache-copyright-notice.xml delete mode 100644 gradle/docker.gradle create mode 100644 gradle/multi-process-test.gradle diff --git a/BUILDING.md b/BUILDING.md index ca7b7ff30333..ebcb84204bd4 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -2,7 +2,7 @@ All platforms require a Java installation, with JDK 1.8 or more recent version. -Set the JAVA\_HOME environment variable. For example: +Set the JAVA\_HOME environment variable. For example: | Platform | Command | | :---: | --- | @@ -14,16 +14,17 @@ Download the project source from the Releases page at [Apache Geode](http://geode.apache.org/releases/), and unpack the source code. Within the directory containing the unpacked source code, run the gradle build: + ```console $ ./gradlew build ``` Once the build completes, the project files will be installed at -`geode-assembly/build/install/apache-geode`. The distribution archives will be -created in `geode-assembly/build/distributions/`. +`geode-assembly/build/install/apache-geode`. The distribution archives will be created +in `geode-assembly/build/distributions/`. + +Verify the installation by invoking the `gfsh` shell command to print version information: -Verify the installation by invoking the `gfsh` shell command to print version -information: ```console $ ./geode-assembly/build/install/apache-geode/bin/gfsh version v1.1.0 @@ -33,64 +34,74 @@ Note: on Windows invoke the `gfsh.bat` script to print the version string. ## Setting up IntelliJ -The following steps have been tested with: - -* **IntelliJ IDEA 2018.3.5** - -1. Run `./gradlew --parallel generate` from Geode repository root to create compiler generated source. - 1. Alternatively (and to ensure these sources stay up-to-date): - - Perform step 2 - - In the **Gradle** dockable, expand the **geode** (the root project) -> **Tasks** -> **build** - - Right-click **generate** task, select **Execute Before Sync** and **Execute Before Build** - - Click **Refresh All Gradle Projects** at the top of the Gradle dockable - - ![After](https://cwiki.apache.org/confluence/download/attachments/103096184/geode-generate-before-sync.png?api=v2) - -2. Import project into IntelliJ IDEA. - - From the **Welcome to IntelliJ IDEA** window: - - 1. **Import Project ->** select *build.gradle* file from Geode repository root and press **Open**. - 2. Optionally, enable **Use auto-import** - 3. Enable **Create separate module per source set** - 4. Select **Use Project JDK 1.8.0_*nnn*** where *nnn* is latest build required for Geode - -3. Change Code Style Scheme to GeodeStyle. - - Navigate to **IntelliJ IDEA -> Preferences... -> Editor -> Code Style**. Select *GeodeStyle* in Scheme drop-down box if it already exists. - - To define the *GeodeStyle* in **Scheme**, select the gear icon next to the drop-down box, click **Import Scheme ->** and select **IntelliJ IDEA code style XML**. Select *etc/intellij-java-modified-google-style.xml* from Geode repository root, enter **To:** *GeodeStyle*, check **Current scheme** and press **OK**. - -4. Make Apache the default Copyright. - - Navigate to **IntelliJ IDEA -> Preferences... -> Editor -> Copyright**. Select *Apache* in drop-down box **Default project copyright**. - - To define *Apache* in **Copyright**, navigate to **IntelliJ IDEA -> Preferences... -> Editor -> Copyright -> Copyright Profiles**. Click **+** to add a new project. Enter *Apache* as the **Name** and enter the following block without asterisks or leading spaces: - - ```text - Licensed to the Apache Software Foundation (ASF) under one or more contributor license - agreements. See the NOTICE file distributed with this work for additional information regarding - copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance with the License. You may obtain a - copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software distributed under the License - is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express - or implied. See the License for the specific language governing permissions and limitations under - the License. - ``` - - ...then return to **Copyright** and select *Apache* in drop-down box **Default project copyright**. - - Navigate to **IntelliJ IDEA -> Preferences... -> Editor -> Copyright -> Formatting**. Uncheck **Add blank line after** and press **OK**. - -5. Rebuild the Project. - - Navigate to **Build -> Rebuild Project** and the full project should compile without errors. +The following steps have been tested with **IntelliJ IDEA 2020.3.3** + +1. Run `./gradlew --parallel generate` from Geode repository root to create compiler generated + source. + +1. Import the project into IntelliJ IDEA. + + 1. Select **File -> Open...** from the menu. + 1. Select the `build.gradle` file in the Geode repository root and select **Open**. + 1. In the **Open Project?** popup, select **Open Project**. + 1. In the **Trust and Open Gradle Project?** popup, select **Trust Project**. + 1. Wait for IntelliJ to import the project and complete its background tasks. + +1. Configure IntelliJ IDEA to build and run the project and tests. + * Set the Java SDK for the project. + 1. Select **File -> Project Structure...** from the menu. + 1. Open the **Project Settings -> Project** section. + 1. Set **Project SDK** to your most recent Java 1.8 JDK. + + * To automatically re-generate sources when needed (recommended). + 1. Select **View -> Tool Windows -> Gradle** from the menu. + 1. In the Gradle dockable, open **geode -> Tasks -> build**. + 1. Right click the **generate** task and select **Execute Before Sync**. + 1. Right click the **generate** task and select **Execute Before Build**. + + * To reload the project when build scripts change (recommended). + 1. Select **IntelliJ IDEA -> Preferences...** from the menu. + 1. Open the **Build, Execution, Deployment -> Build Tools** section. + 1. Set **Reload project after changes in the build scripts:** to **Any changes**. + + * To build and run with Gradle (recommended). + 1. Select **IntelliJ IDEA -> Preferences...** from the menu. + 1. Open the **Build, Execution, Deployment -> Build Tools -> Gradle** section. + 1. Set **Build and run using:** to **Gradle**. + 1. Set **Run tests using:** to **Gradle**. + +1. Set the Code Style Scheme to GeodeStyle. + + 1. Select **IntelliJ IDEA -> Preferences...** + 1. Open the **Editor -> Code Style** section. + 1. If *GeodeStyle* style does not appear in the **Scheme** drop-down box + 1. Select the gear icon next to the drop-down. + 1. Select **Import Scheme -> IntelliJ IDEA code style XML**. + 1. Select `etc/intellij-java-modified-google-style.xml` from the Geode repository root. + 1. Enter **To:** *GeodeStyle*, check **Current scheme**, and press **OK**. + 1. Select *GeodeStyle* in **Scheme** drop-down box. + +1. Make Apache the default Copyright. + + 1. Select **IntelliJ IDEA -> Preferences...** from the menu. + 1. Open the **Editor -> Copyright** section. + 1. If *Apache* does not appear in the **Default project copyright** drop-down box: + 1. Open the **Copyright Profiles** subsection. + 1. Select the "import" icon (the small arrow pointing down and to the left) from the + Copyright Profiles section's toolbar. + 1. Select `etc/intellij-apache-copyright-notice.xml` from the Geode repository root. + 1. Return to the **Copyright** section. + 1. Select *Apache* in the **Default project copyright** drop-down box. + 1. Open the **Formatting** subsection. + 1. Uncheck **Add blank line after** and select **OK**. + +1. Rebuild the Project. + + 1. Select **Build -> Rebuild Project** from the menu. The full project should compile without + errors. Some optional sanity tests to make sure things are working properly: - * Try looking up classes using **Navigate -> Class...** - * Open and run a distributed test such as BasicDistributedTest in geode-core. - * Create a new java class and ensure the Apache license is automatically added to the top of the file with no blank line before the package line. + * Try looking up classes using **Navigate -> Class...** + * Open and run a distributed test such as BasicDistributedTest in geode-core. + * Create a new java class and ensure the Apache license is automatically added to the top of the + file with no blank line before the package line. diff --git a/boms/geode-all-bom/src/test/resources/expected-pom.xml b/boms/geode-all-bom/src/test/resources/expected-pom.xml index c7d889068c90..93e322913458 100644 --- a/boms/geode-all-bom/src/test/resources/expected-pom.xml +++ b/boms/geode-all-bom/src/test/resources/expected-pom.xml @@ -16,6 +16,11 @@ See the License for the specific language governing permissions and limitations under the License. --> + + + + + 4.0.0 org.apache.geode geode-all-bom @@ -41,823 +46,686 @@ antlr antlr 2.7.7 - compile cglib cglib 3.3.0 - compile com.arakelian java-jq 1.1.0 - compile com.carrotsearch.randomizedtesting randomizedtesting-runner 2.7.8 - compile com.github.davidmoten geo 0.7.7 - compile com.github.stefanbirkner system-rules 1.19.0 - compile com.github.stephenc.findbugs findbugs-annotations 1.3.9-1 - compile com.google.code.findbugs jsr305 3.0.2 - compile com.google.guava guava 30.1-jre - compile com.healthmarketscience.rmiio rmiio 2.1.2 - compile com.mockrunner mockrunner-servlet 2.0.6 - compile com.nimbusds nimbus-jose-jwt 8.11 - compile com.nimbusds oauth2-oidc-sdk 8.9 - compile com.sun.istack istack-commons-runtime 4.0.0 - compile com.sun.mail javax.mail 1.6.2 - compile com.sun.xml.bind jaxb-impl 2.3.2 - compile com.tngtech.archunit archunit-junit4 0.15.0 - compile com.zaxxer HikariCP 4.0.3 - compile commons-beanutils commons-beanutils 1.9.4 - compile commons-codec commons-codec 1.15 - compile commons-collections commons-collections 3.2.2 - compile commons-configuration commons-configuration 1.10 - compile commons-digester commons-digester 2.1 - compile commons-fileupload commons-fileupload 1.4 - compile commons-io commons-io 2.8.0 - compile commons-logging commons-logging 1.2 - compile commons-modeler commons-modeler 2.0.1 - compile commons-validator commons-validator 1.7 - compile io.github.classgraph classgraph 4.8.52 - compile io.micrometer micrometer-core 1.6.5 - compile io.netty netty-all 4.1.59.Final - compile io.swagger swagger-annotations 1.6.2 - compile it.unimi.dsi fastutil 8.5.2 - compile javax.annotation javax.annotation-api 1.3.2 - compile javax.annotation jsr250-api 1.0 - compile javax.ejb ejb-api 3.0 - compile javax.mail javax.mail-api 1.6.2 - compile javax.resource javax.resource-api 1.7.1 - compile javax.servlet javax.servlet-api 3.1.0 - compile javax.xml.bind jaxb-api 2.3.1 - compile joda-time joda-time 2.10.9 - compile junit junit 4.13.2 - compile mx4j mx4j-tools 3.0.1 - compile mysql mysql-connector-java 5.1.46 - compile net.java.dev.jna jna 5.7.0 - compile net.java.dev.jna jna-platform 5.7.0 - compile net.openhft compiler 2.4.1 - compile net.sf.jopt-simple jopt-simple 5.0.4 - compile net.sourceforge.pmd pmd-java 6.32.0 - compile net.sourceforge.pmd pmd-test 6.32.0 - compile net.spy spymemcached 2.12.3 - compile org.apache.bcel bcel 6.5.0 - compile org.apache.commons commons-lang3 3.12.0 - compile org.apache.commons commons-text 1.9 - compile org.apache.derby derby 10.14.2.0 - compile org.apache.httpcomponents httpclient 4.5.13 - compile org.apache.httpcomponents httpcore 4.4.14 - compile org.apache.shiro shiro-core 1.7.1 - compile org.assertj assertj-core 3.19.0 - compile org.awaitility awaitility 4.0.3 - compile org.bouncycastle bcpkix-jdk15on 1.68 - compile org.codehaus.cargo cargo-core-uberjar 1.9.2 - compile org.eclipse.jetty jetty-server 9.4.39.v20210325 - compile org.eclipse.jetty jetty-webapp 9.4.39.v20210325 - compile org.eclipse.persistence javax.persistence 2.2.1 - compile org.httpunit httpunit 1.7.3 - compile org.iq80.snappy snappy 0.4 - compile org.jboss.modules jboss-modules 1.11.0.Final - compile org.jgroups jgroups 3.6.14.Final - compile org.mockito mockito-core 3.8.0 - compile org.mortbay.jetty servlet-api 3.0.20100224 - compile org.openjdk.jmh jmh-core 1.26 - compile org.postgresql postgresql 42.2.8 - compile org.skyscreamer jsonassert 1.5.0 - compile org.slf4j slf4j-api 1.7.30 - compile org.springframework.hateoas spring-hateoas 1.2.5 - compile org.springframework.ldap spring-ldap-core 2.3.2.RELEASE - compile org.springframework.shell spring-shell 1.2.0.RELEASE - compile org.testcontainers testcontainers 1.14.3 - compile pl.pragmatists JUnitParams 1.1.0 - compile redis.clients jedis 3.5.2 - compile io.lettuce lettuce-core 6.0.3.RELEASE - compile xerces xercesImpl 2.12.0 - compile com.fasterxml.jackson.core jackson-annotations 2.12.2 - compile com.fasterxml.jackson.core jackson-core 2.12.2 - compile com.fasterxml.jackson.core jackson-databind 2.12.2 - compile com.fasterxml.jackson.datatype jackson-datatype-joda 2.12.2 - compile com.fasterxml.jackson.datatype jackson-datatype-jsr310 2.12.2 - compile com.jayway.jsonpath json-path-assert 2.5.0 - compile com.jayway.jsonpath json-path 2.5.0 - compile com.palantir.docker.compose docker-compose-rule-core 0.31.1 - compile com.palantir.docker.compose docker-compose-rule-junit4 0.31.1 - compile com.pholser junit-quickcheck-core 1.0 - compile com.pholser junit-quickcheck-generators 1.0 - compile io.springfox springfox-swagger-ui 2.9.2 - compile io.springfox springfox-swagger2 2.9.2 - compile mx4j mx4j-remote 3.0.2 - compile mx4j mx4j 3.0.2 - compile org.apache.logging.log4j log4j-api 2.14.1 - compile org.apache.logging.log4j log4j-core 2.14.1 - compile org.apache.logging.log4j log4j-jcl 2.14.1 - compile org.apache.logging.log4j log4j-jul 2.14.1 - compile org.apache.logging.log4j log4j-slf4j-impl 2.14.1 - compile org.apache.lucene lucene-analyzers-common 6.6.6 - compile org.apache.lucene lucene-analyzers-phonetic 6.6.6 - compile org.apache.lucene lucene-core 6.6.6 - compile org.apache.lucene lucene-queryparser 6.6.6 - compile org.apache.lucene lucene-test-framework 6.6.6 - compile org.hamcrest hamcrest 2.2 - compile org.seleniumhq.selenium selenium-api 3.141.59 - compile org.seleniumhq.selenium selenium-chrome-driver 3.141.59 - compile org.seleniumhq.selenium selenium-remote-driver 3.141.59 - compile org.seleniumhq.selenium selenium-support 3.141.59 - compile org.springframework.security spring-security-config 5.4.5 - compile org.springframework.security spring-security-core 5.4.5 - compile org.springframework.security spring-security-ldap 5.4.5 - compile org.springframework.security spring-security-test 5.4.5 - compile org.springframework.security spring-security-web 5.4.5 - compile org.springframework.security spring-security-oauth2-core 5.4.5 - compile org.springframework.security spring-security-oauth2-client 5.4.5 - compile org.springframework.security spring-security-oauth2-jose 5.4.5 - compile org.springframework spring-aspects 5.3.5 - compile org.springframework spring-beans 5.3.5 - compile org.springframework spring-context 5.3.5 - compile org.springframework spring-core 5.3.5 - compile org.springframework spring-expression 5.3.5 - compile org.springframework spring-oxm 5.3.5 - compile org.springframework spring-test 5.3.5 - compile org.springframework spring-tx 5.3.5 - compile org.springframework spring-web 5.3.5 - compile org.springframework spring-webmvc 5.3.5 - compile org.springframework.boot spring-boot-starter 2.4.3 - compile org.springframework.boot spring-boot-starter-jetty 2.4.3 - compile org.springframework.boot spring-boot-starter-web 2.4.3 - compile org.springframework.boot spring-boot-starter-data-redis 2.4.3 - compile org.springframework.session spring-session-data-redis 2.4.2 - compile org.jetbrains annotations 20.1.0 - compile org.apache.geode @@ -895,12 +763,6 @@ ${version} compile - - org.apache.geode - geode-deployment-legacy - ${version} - compile - org.apache.geode geode-dunit @@ -1045,6 +907,12 @@ ${version} compile + + org.apache.geode + geode-deployment-legacy + ${version} + compile + org.apache.geode geode-lucene-test diff --git a/build.gradle b/build.gradle index 66e8644ca2b0..349f664fc21e 100755 --- a/build.gradle +++ b/build.gradle @@ -21,16 +21,16 @@ plugins { id "base" id "idea" id "eclipse" - id "com.diffplug.spotless" version "5.10.2" apply false - id "com.github.ben-manes.versions" version "0.36.0" apply false - id "nebula.lint" version "16.17.0" apply false + id "com.diffplug.spotless" version "5.11.1" apply false + id "com.github.ben-manes.versions" version "0.38.0" apply false + id "nebula.lint" version "16.17.1" apply false id "com.palantir.docker" version "0.26.0" apply false id "io.spring.dependency-management" version "1.0.11.RELEASE" apply false id "org.ajoberstar.grgit" version "4.1.0" apply false - id "org.nosphere.apache.rat" version "0.6.0" apply false + id "org.nosphere.apache.rat" version "0.7.0" apply false id "org.sonarqube" version "3.1.1" apply false id "me.champeau.gradle.japicmp" apply false // Version defined in buildSrc/build.gradle - id 'me.champeau.gradle.jmh' version '0.5.2' apply false + id 'me.champeau.gradle.jmh' version '0.5.3' apply false } diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DefaultWorkerSemaphore.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DefaultWorkerSemaphore.groovy deleted file mode 100644 index a42c4a1d74df..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DefaultWorkerSemaphore.groovy +++ /dev/null @@ -1,71 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import org.gradle.api.Project -import org.gradle.api.tasks.testing.Test - -import java.util.concurrent.Semaphore - -class DefaultWorkerSemaphore implements WorkerSemaphore { - private int maxWorkers = Integer.MAX_VALUE - private Semaphore semaphore - private logger - - @Override - void acquire() { - semaphore().acquire() - logger.debug("Semaphore acquired, available: {}/{}", semaphore().availablePermits(), maxWorkers) - } - - @Override - void release() { - semaphore().release() - logger.debug("Semaphore released, available: {}/{}", semaphore().availablePermits(), maxWorkers) - } - - @Override - synchronized void applyTo(Project project) { - if (semaphore) return - if (!logger) { - logger = project.logger - } - - maxWorkers = project.tasks.withType(Test).findAll { - it.extensions.docker?.image != null - }.collect { - def v = it.maxParallelForks - it.maxParallelForks = 10000 - v - }.min() ?: 1 - semaphore() - } - - private synchronized setMaxWorkers(int num) { - if (this.@maxWorkers > num) { - this.@maxWorkers = num - } - } - - private synchronized Semaphore semaphore() { - if (semaphore == null) { - semaphore = new Semaphore(maxWorkers) - logger.lifecycle("Do not allow more than {} test workers", maxWorkers) - } - semaphore - } -} diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedJavaExecHandleBuilder.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedJavaExecHandleBuilder.groovy deleted file mode 100644 index 8f4e04c433d1..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedJavaExecHandleBuilder.groovy +++ /dev/null @@ -1,108 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import com.pedjak.gradle.plugins.dockerizedtest.DockerizedTestExtension -import com.pedjak.gradle.plugins.dockerizedtest.ExitCodeTolerantExecHandle -import com.pedjak.gradle.plugins.dockerizedtest.WorkerSemaphore -import org.gradle.api.internal.file.FileCollectionFactory -import org.gradle.api.internal.file.FileResolver -import org.gradle.initialization.BuildCancellationToken -import org.gradle.process.internal.* -import org.gradle.process.internal.streams.OutputStreamsForwarder - -import java.util.concurrent.Executor - -class DockerizedJavaExecHandleBuilder extends JavaExecHandleBuilder { - protected final FileCollectionFactory fileCollectionFactory - - def streamsHandler - def executor - def buildCancellationToken - private final DockerizedTestExtension extension - - private final WorkerSemaphore workersSemaphore - - DockerizedJavaExecHandleBuilder(DockerizedTestExtension extension, - FileResolver fileResolver, - FileCollectionFactory fileCollectionFactory, - Executor executor, - BuildCancellationToken buildCancellationToken, - WorkerSemaphore workersSemaphore) { - super(fileResolver, fileCollectionFactory, executor, buildCancellationToken) - this.fileCollectionFactory = fileCollectionFactory - this.extension = extension - this.executor = executor - this.buildCancellationToken = buildCancellationToken - this.workersSemaphore = workersSemaphore - } - - def StreamsHandler getStreamsHandler() { - StreamsHandler effectiveHandler; - if (this.streamsHandler != null) { - effectiveHandler = this.streamsHandler; - } else { - boolean shouldReadErrorStream = !redirectErrorStream; - effectiveHandler = new OutputStreamsForwarder(standardOutput, errorOutput, shouldReadErrorStream); - } - return effectiveHandler; - } - - ExecHandle build() { - - return new ExitCodeTolerantExecHandle(new DockerizedExecHandle(extension, getDisplayName(), - getWorkingDir(), - 'java', - allArguments, - getActualEnvironment(), - getStreamsHandler(), - inputHandler, - listeners, - redirectErrorStream, - timeoutMillis, - daemon, - executor, - buildCancellationToken), - workersSemaphore) - - } - - def timeoutMillis = Integer.MAX_VALUE - - @Override - AbstractExecHandleBuilder setTimeout(int timeoutMillis) { - this.timeoutMillis = timeoutMillis - return super.setTimeout(timeoutMillis) - } - - boolean redirectErrorStream - - @Override - AbstractExecHandleBuilder redirectErrorStream() { - redirectErrorStream = true - return super.redirectErrorStream() - } - - def listeners = [] - - @Override - AbstractExecHandleBuilder listener(ExecHandleListener listener) { - listeners << listener - return super.listener(listener) - } - -} diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestExtension.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestExtension.groovy deleted file mode 100644 index 8a007f117fae..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestExtension.groovy +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import com.github.dockerjava.api.DockerClient - -class DockerizedTestExtension { - - String image - Map volumes - String user - - Closure beforeContainerCreate - - Closure afterContainerCreate - - Closure beforeContainerStart - - Closure afterContainerStart - - Closure afterContainerStop = { containerId, client -> - try { - client.removeContainerCmd(containerId).exec(); - } catch (Exception e) { - // ignore any error - } - } - - // could be a DockerClient instance or a closure that returns a DockerClient instance - private def clientOrClosure - - void setClient(clientOrClosure) { - this.clientOrClosure = clientOrClosure - } - - DockerClient getClient() { - if (clientOrClosure == null) return null - if (DockerClient.class.isAssignableFrom(clientOrClosure.getClass())) { - return (DockerClient) clientOrClosure; - } else { - return (DockerClient) ((Closure) clientOrClosure).call(); - } - } -} diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestPlugin.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestPlugin.groovy deleted file mode 100644 index 176fa58cf930..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/DockerizedTestPlugin.groovy +++ /dev/null @@ -1,188 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import com.github.dockerjava.api.DockerClient -import com.github.dockerjava.core.DefaultDockerClientConfig -import com.github.dockerjava.core.DockerClientBuilder -import com.github.dockerjava.netty.NettyDockerCmdExecFactory -import org.apache.commons.lang3.SystemUtils -import org.apache.maven.artifact.versioning.ComparableVersion -import org.gradle.api.Action -import org.gradle.api.GradleException -import org.gradle.api.Plugin -import org.gradle.api.Project -import org.gradle.api.internal.file.DefaultFileCollectionFactory -import org.gradle.api.tasks.testing.Test -import org.gradle.initialization.DefaultBuildCancellationToken -import org.gradle.internal.concurrent.DefaultExecutorFactory -import org.gradle.internal.concurrent.ExecutorFactory -import org.gradle.internal.operations.BuildOperationExecutor -import org.gradle.internal.remote.Address -import org.gradle.internal.remote.ConnectionAcceptor -import org.gradle.internal.remote.MessagingServer -import org.gradle.internal.remote.ObjectConnection -import org.gradle.internal.remote.internal.ConnectCompletion -import org.gradle.internal.remote.internal.IncomingConnector -import org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection -import org.gradle.internal.remote.internal.inet.MultiChoiceAddress -import org.gradle.internal.time.Clock -import org.gradle.process.internal.JavaExecHandleFactory -import org.gradle.process.internal.worker.DefaultWorkerProcessFactory - -import javax.inject.Inject - -class DockerizedTestPlugin implements Plugin { - - def supportedVersion = '4.8' - def currentUser - def messagingServer - def static workerSemaphore = new DefaultWorkerSemaphore() - def memoryManager = new com.pedjak.gradle.plugins.dockerizedtest.NoMemoryManager() - - @Inject - DockerizedTestPlugin(MessagingServer messagingServer) { - this.currentUser = SystemUtils.IS_OS_WINDOWS ? "0" : "id -u".execute().text.trim() - this.messagingServer = new MessageServer(messagingServer.connector, messagingServer.executorFactory) - } - - void configureTest(project, test) { - def ext = test.extensions.create("docker", DockerizedTestExtension, [] as Object[]) - def startParameter = project.gradle.startParameter - ext.volumes = ["$startParameter.gradleUserHomeDir": "$startParameter.gradleUserHomeDir", - "$project.projectDir" : "$project.projectDir"] - ext.user = currentUser - test.doFirst { - def extension = test.extensions.docker - - if (extension?.image) { - - workerSemaphore.applyTo(test.project) - test.testExecuter = new com.pedjak.gradle.plugins.dockerizedtest.TestExecuter(newProcessBuilderFactory(project, extension, test.processBuilderFactory), actorFactory, moduleRegistry, services.get(BuildOperationExecutor), services.get(Clock)); - - if (!extension.client) { - extension.client = createDefaultClient() - } - } - - } - } - - DockerClient createDefaultClient() { - DockerClientBuilder.getInstance(DefaultDockerClientConfig.createDefaultConfigBuilder()) - .withDockerCmdExecFactory(new NettyDockerCmdExecFactory()) - .build() - } - - void apply(Project project) { - - boolean unsupportedVersion = new ComparableVersion(project.gradle.gradleVersion).compareTo(new ComparableVersion(supportedVersion)) < 0 - if (unsupportedVersion) throw new GradleException("dockerized-test plugin requires Gradle ${supportedVersion}+") - - project.tasks.withType(Test).each { test -> configureTest(project, test) } - project.tasks.whenTaskAdded { task -> - if (task instanceof Test) configureTest(project, task) - } - } - - def newProcessBuilderFactory(project, extension, defaultProcessBuilderFactory) { - - def executorFactory = new DefaultExecutorFactory() - def executor = executorFactory.create("Docker container link") - def buildCancellationToken = new DefaultBuildCancellationToken() - - def defaultfilecollectionFactory = new DefaultFileCollectionFactory(project.fileResolver, null) - def execHandleFactory = [newJavaExec: { -> - new DockerizedJavaExecHandleBuilder( - extension, project.fileResolver, defaultfilecollectionFactory, - executor, buildCancellationToken, workerSemaphore) - }] as JavaExecHandleFactory - new DefaultWorkerProcessFactory(defaultProcessBuilderFactory.loggingManager, - messagingServer, - defaultProcessBuilderFactory.workerImplementationFactory.classPathRegistry, - defaultProcessBuilderFactory.idGenerator, - defaultProcessBuilderFactory.gradleUserHomeDir, - defaultProcessBuilderFactory.workerImplementationFactory.temporaryFileProvider, - execHandleFactory, - defaultProcessBuilderFactory.workerImplementationFactory.jvmVersionDetector, - defaultProcessBuilderFactory.outputEventListener, - memoryManager - ) - } - - class MessageServer implements MessagingServer { - def IncomingConnector connector; - def ExecutorFactory executorFactory; - - public MessageServer(IncomingConnector connector, ExecutorFactory executorFactory) { - this.connector = connector; - this.executorFactory = executorFactory; - } - - public ConnectionAcceptor accept(Action action) { - return new ConnectionAcceptorDelegate(connector.accept(new ConnectEventAction(action, executorFactory), true)) - } - - - } - - class ConnectEventAction implements Action { - def action; - def executorFactory; - - public ConnectEventAction(Action action, executorFactory) { - this.executorFactory = executorFactory - this.action = action - } - - public void execute(ConnectCompletion completion) { - action.execute(new MessageHubBackedObjectConnection(executorFactory, completion)); - } - } - - class ConnectionAcceptorDelegate implements ConnectionAcceptor { - - MultiChoiceAddress address - - @Delegate - ConnectionAcceptor delegate - - ConnectionAcceptorDelegate(ConnectionAcceptor delegate) { - this.delegate = delegate - } - - Address getAddress() { - synchronized (delegate) - { - if (address == null) { - def remoteAddresses = NetworkInterface.networkInterfaces.findAll { - try { - return it.up && !it.loopback - } catch (SocketException ex) { - logger.warn("Unable to inspect interface " + it) - return false - } - }*.inetAddresses*.collect { it }.flatten() - def original = delegate.address - address = new MultiChoiceAddress(original.canonicalAddress, original.port, remoteAddresses) - } - } - address - } - } - -} diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/ExitCodeTolerantExecHandle.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/ExitCodeTolerantExecHandle.groovy deleted file mode 100644 index ab1c6f119c2f..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/ExitCodeTolerantExecHandle.groovy +++ /dev/null @@ -1,92 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import com.pedjak.gradle.plugins.dockerizedtest.WorkerSemaphore -import org.gradle.process.ExecResult -import org.gradle.process.internal.ExecException -import org.gradle.process.internal.ExecHandle -import org.gradle.process.internal.ExecHandleListener - -/** - * All exit codes are normal - */ -class ExitCodeTolerantExecHandle implements ExecHandle { - - private final WorkerSemaphore testWorkerSemaphore - - @Delegate - private final ExecHandle delegate - - ExitCodeTolerantExecHandle(ExecHandle delegate, WorkerSemaphore testWorkerSemaphore) { - this.delegate = delegate - this.testWorkerSemaphore = testWorkerSemaphore - delegate.addListener(new ExecHandleListener() { - - @Override - void executionStarted(ExecHandle execHandle) { - // do nothing - } - - @Override - void executionFinished(ExecHandle execHandle, ExecResult execResult) { - testWorkerSemaphore.release() - } - }) - } - - ExecHandle start() { - testWorkerSemaphore.acquire() - try { - delegate.start() - } catch (Exception e) { - testWorkerSemaphore.release() - throw e - } - } - - private static class ExitCodeTolerantExecResult implements ExecResult { - - @Delegate - private final ExecResult delegate - - ExitCodeTolerantExecResult(ExecResult delegate) { - this.delegate = delegate - } - - ExecResult assertNormalExitValue() throws ExecException { - // no op because we are perfectly ok if the exit code is anything - // because Docker can complain about not being able to remove the used image - // although the tests completed fine - this - } - } - - private static class ExecHandleListenerFacade implements ExecHandleListener { - - @Delegate - private final ExecHandleListener delegate - - ExecHandleListenerFacade(ExecHandleListener delegate) { - this.delegate = delegate - } - - void executionFinished(ExecHandle execHandle, ExecResult execResult) { - delegate.executionFinished(execHandle, new ExitCodeTolerantExecResult(execResult)) - } - } -} diff --git a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/WorkerSemaphore.groovy b/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/WorkerSemaphore.groovy deleted file mode 100644 index 0268088a434a..000000000000 --- a/buildSrc/src/main/groovy/com/pedjak/gradle/plugins/dockerizedtest/WorkerSemaphore.groovy +++ /dev/null @@ -1,28 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest - -import org.gradle.api.Project - -interface WorkerSemaphore { - - void acquire() - - void release() - - void applyTo(Project project) -} \ No newline at end of file diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/RepeatTest.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/RepeatTest.groovy deleted file mode 100644 index 71231016b400..000000000000 --- a/buildSrc/src/main/groovy/org/apache/geode/gradle/RepeatTest.groovy +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more contributor license - * agreements. See the NOTICE file distributed with this work for additional information regarding - * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance with the License. You may obtain a - * copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software distributed under the License - * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express - * or implied. See the License for the specific language governing permissions and limitations under - * the License. - */ -package org.apache.geode.gradle - -import org.gradle.StartParameter -import org.gradle.api.file.FileTree -import org.gradle.api.internal.DocumentationRegistry -import org.gradle.api.internal.tasks.testing.JvmTestExecutionSpec -import org.gradle.api.internal.tasks.testing.TestExecuter -import org.gradle.api.internal.tasks.testing.detection.DefaultTestExecuter -import org.gradle.api.internal.tasks.testing.filter.DefaultTestFilter -import org.gradle.api.tasks.testing.Test -import org.gradle.internal.operations.BuildOperationExecutor -import org.gradle.internal.time.Clock -import org.gradle.internal.work.WorkerLeaseRegistry - -class RepeatTest extends Test { - int times = 1 - - @Override - FileTree getCandidateClassFiles() { - FileTree candidates = super.getCandidateClassFiles() - int additionalRuns = times - 1 - for (int i = 0; i < additionalRuns; i++) { - candidates = candidates.plus(super.getCandidateClassFiles()) - } - - return candidates - } - - /* - * We have to override gradles default test executor, because that uses {@link RunPreviousFailedFirstTestClassProcessor} - * Which deduplicates the test specs we're passing in - */ - @Override - protected TestExecuter createTestExecuter() { - def oldExecutor = super.createTestExecuter() - - //Use the previously set worker process factory. If the test is - //being run using the parallel docker plugin, this will be a docker - //process factory - def workerProcessFactory = oldExecutor.workerFactory - - return new OverriddenTestExecutor(workerProcessFactory, getActorFactory(), - getModuleRegistry(), - getServices().get(WorkerLeaseRegistry.class), - getServices().get(BuildOperationExecutor.class), - getServices().get(StartParameter.class).getMaxWorkerCount(), - getServices().get(Clock.class), - getServices().get(DocumentationRegistry.class), - (DefaultTestFilter) getFilter()) - } - -} diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Executers.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Executers.groovy new file mode 100644 index 000000000000..01d33ca16aca --- /dev/null +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Executers.groovy @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing + +import org.apache.geode.gradle.testing.process.AdjustableProcessLauncher +import org.gradle.StartParameter +import org.gradle.api.internal.DocumentationRegistry +import org.gradle.api.internal.tasks.testing.JvmTestExecutionSpec +import org.gradle.api.internal.tasks.testing.TestExecuter +import org.gradle.api.internal.tasks.testing.detection.DefaultTestExecuter +import org.gradle.internal.time.Clock +import org.gradle.internal.work.WorkerLeaseRegistry + +class Executers { + /** + * Creates a {@code TestExecuter} that applies an adjustment to each test worker + * {@link ProcessBuilder} just before launching the process. + */ + static TestExecuter withAdjustment(testTask, adjustment) { + def gradleWorkerProcessFactory = testTask.createTestExecuter().workerFactory + def isolatingWorkerProcessFactory = Workers.createWorkerProcessFactory( + gradleWorkerProcessFactory, + new AdjustableProcessLauncher(adjustment), + gradleWorkerProcessFactory.server) + return withFactory(testTask, isolatingWorkerProcessFactory) + } + + /** + * Creates a {@code TestExecuter} that uses the given factory to create test worker processes. + */ + static TestExecuter withFactory(testTask, workerProcessFactory) { + def services = testTask.services + return new DefaultTestExecuter( + workerProcessFactory, + testTask.actorFactory, + testTask.moduleRegistry, + services.get(WorkerLeaseRegistry), + services.get(StartParameter).getMaxWorkerCount(), + services.get(Clock), + services.get(DocumentationRegistry), + testTask.filter + ) + } + +} diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Workers.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Workers.groovy new file mode 100644 index 000000000000..5d0b46fd2091 --- /dev/null +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/Workers.groovy @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing + +import org.apache.geode.gradle.testing.process.LauncherProxyWorkerProcessFactory +import org.apache.geode.gradle.testing.process.ProcessLauncher +import org.gradle.internal.remote.MessagingServer +import org.gradle.process.internal.worker.WorkerProcessFactory + +class Workers { + /** + * Creates a worker process factory that borrows most components from the donor, but uses the + * given process launcher and messaging server to launch worker processes. + */ + static WorkerProcessFactory createWorkerProcessFactory( + WorkerProcessFactory donor, + ProcessLauncher processLauncher, + MessagingServer messagingServer) { + def workerImplementationFactory = donor.workerImplementationFactory + return new LauncherProxyWorkerProcessFactory( + donor.loggingManager, + messagingServer, + workerImplementationFactory.classPathRegistry, + donor.idGenerator, + workerImplementationFactory.gradleUserHomeDir, + workerImplementationFactory.temporaryFileProvider, + donor.execHandleFactory, + workerImplementationFactory.jvmVersionDetector, + donor.outputEventListener, + donor.memoryManager, + processLauncher) + } +} diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerTestWorkerConfig.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerTestWorkerConfig.groovy new file mode 100644 index 000000000000..61cca0615c8e --- /dev/null +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerTestWorkerConfig.groovy @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.dockerized + +import org.gradle.api.Project + +class DockerTestWorkerConfig { + static long durationWarningThreshold = 60_000 + String image + String javaHome + String localUserID + String name + int timeoutMillis = 300_000 + String user + Map volumes = new HashMap<>() + + DockerTestWorkerConfig(Project project) { + name = project.path + image = project.dunitDockerImage + user = project.dunitDockerUser + + if (project.hasProperty('dunitDockerJVM') && !project.dunitDockerJVM.trim().isEmpty()) { + javaHome = project.dunitDockerJVM as String + } else if (project.hasProperty('testJVM') && !project.testJVM.trim().isEmpty()) { + javaHome = project.testJVM as String + } + + // Mount the user's Gradle home dir, the Geode project root directory, and any + // user-specified volumes. + def gradleUserHomeDir = project.gradle.startParameter.gradleUserHomeDir.getAbsolutePath() as String + def geodeDir = new File(System.getenv('PWD')).getCanonicalPath() + volumes = [(geodeDir) : geodeDir, + (gradleUserHomeDir): gradleUserHomeDir] + + if (project.hasProperty('dunitDockerVolumes')) { + volumes.putAll(project.dunitDockerVolumes) + } + + if (project.hasProperty("dunitDockerTimeout")) { + timeoutMillis = Integer.parseUnsignedInt(project.dunitDockerTimeout) + } + + // Unfortunately this snippet of code is here and is required by + // dev-tools/docker/base/entrypoint.sh. This allows preserving the outer user inside the + // running container. Required for Jenkins and other environments. There doesn't seem to be + // a way to pass this environment variable in from a Jenkins Gradle job. + if (System.env['LOCAL_USER_ID'] == null) { + def username = System.getProperty("user.name") + localUserID = ['id', '-u', username].execute().text.trim() as String + } + } + + /** + * Adjust the process builder's command and environment to run in a Docker container. + */ + def dockerize(processBuilder) { + def command = processBuilder.command() + def environment = processBuilder.environment() + + // The JAVA_HOME and PATH environment variables set by Gradle are meaningless inside a + // Docker container. Remove them. + if (environment['JAVA_HOME']) { + environment.remove 'JAVA_HOME' + environment['JAVA_HOME_REMOVED'] = "" + } + if (environment['PATH']) { + environment.remove 'PATH' + environment['PATH_REMOVED'] = "" + } + + if (javaHome) { + environment['JAVA_HOME'] = javaHome + command.set(0, "${javaHome}/bin/java" as String) + } + + if (localUserID) { + environment['LOCAL_USER_ID'] = localUserID + } + } +} diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerizedTestPlugin.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerizedTestPlugin.groovy new file mode 100644 index 000000000000..6f7c570caddd --- /dev/null +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/dockerized/DockerizedTestPlugin.groovy @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.dockerized + + +import org.apache.geode.gradle.testing.Executers +import org.apache.geode.gradle.testing.Workers +import org.apache.geode.gradle.testing.isolation.WorkingDirectoryIsolator +import org.gradle.api.Plugin +import org.gradle.api.Project +import org.gradle.api.tasks.testing.Test +import org.gradle.internal.remote.MessagingServer +import org.gradle.internal.remote.internal.inet.TcpIncomingConnector +import org.gradle.process.internal.worker.WorkerProcessFactory + +import javax.inject.Inject + +/** + * Extends each test task with a configuration that it can use to launch its test workers in Docker + * containers. The /gradle/multi-process-test.gradle file defines which test + * tasks actually apply the configuration to run test workers. + */ +class DockerizedTestPlugin implements Plugin { + /** + * A custom {@link MessagingServer} that supports communication between Gradle and processes + * running in Docker containers. + */ + def static dockerMessagingServer + /** + * The singleton {@link MessagingServer} created by Gradle. This plugin borrows the server's + * internal components to build a custom {@code MessagingServer} that can communicate with + * processes running in Docker containers. + */ + def static gradleMessagingServer + /** + * The singleton {@link WorkerProcessFactory} created by Gradle. This plugin borrows the + * factory's internal components to build a custom {@code WorkerProcessFactory} that launches + * processes in Docker containers. + */ + def static gradleWorkerProcessFactory + + /** + * The injected values are singletons. Gradle injects the same instances into each instance of + * this plugin. + *

+ * CAVEAT: The types of these parameters are declared internal by Gradle v6.8.3. Future + * versions of Gradle may not include these types, or may change their implementation. + *

+ * CAVEAT: The Gradle v6.8.3 documentation does not list these types among the services that + * Gradle will inject into plugins. Future versions of Gradle may not inject these values. + */ + @Inject + DockerizedTestPlugin(MessagingServer gradleMessagingServer, + WorkerProcessFactory gradleWorkerProcessFactory) { + initializeGradleWorkerProcessFactory(gradleWorkerProcessFactory) + initializeGradleMessagingServer(gradleMessagingServer) + } + + @Override + void apply(Project project) { + if (!project.hasProperty('parallelDunit')) { + return + } + + initializeMessagingServer() + + def dockerTestWorkerConfig = new DockerTestWorkerConfig(project) + def dockerProcessLauncher = new DockerProcessLauncher(dockerTestWorkerConfig, new WorkingDirectoryIsolator()) + def dockerWorkerProcessFactory = Workers.createWorkerProcessFactory( + gradleWorkerProcessFactory, + dockerProcessLauncher, + dockerMessagingServer) + + def useDockerTestWorker = { + it.doFirst { + testExecuter = Executers.withFactory(it, dockerWorkerProcessFactory) + } + } + + project.tasks.withType(Test).each(useDockerTestWorker) + project.tasks.whenTaskAdded() { + if (it instanceof Test) { + it.configure(useDockerTestWorker) + } + } + } + + synchronized static initializeGradleMessagingServer(server) { + if (!gradleMessagingServer) { + gradleMessagingServer = server + } + } + + synchronized static void initializeGradleWorkerProcessFactory(factory) { + if (!gradleWorkerProcessFactory) { + gradleWorkerProcessFactory = factory + } + } + + synchronized static void initializeMessagingServer() { + if (dockerMessagingServer) { + return + } + + def gradleConnector = gradleMessagingServer.connector + def gradleExecutorFactory = gradleConnector.executorFactory + def gradleIdGenerator = gradleConnector.idGenerator + + /** + * Use a custom {@link WildcardBindingInetAddressFactory} to allow connections from + * processes in Docker containers. + */ + def wildcardAddressFactory = new WildcardBindingInetAddressFactory() + def dockerConnector = new TcpIncomingConnector( + gradleExecutorFactory, + wildcardAddressFactory, + gradleIdGenerator + ) + /** + * Use a custom {@link DockerMessagingServer} that yields connection addresses usable + * by processes in Docker containers. + */ + dockerMessagingServer = new DockerMessagingServer(dockerConnector, gradleExecutorFactory) + } +} diff --git a/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/repeat/RepeatTest.groovy b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/repeat/RepeatTest.groovy new file mode 100644 index 000000000000..bafeab9f4f52 --- /dev/null +++ b/buildSrc/src/main/groovy/org/apache/geode/gradle/testing/repeat/RepeatTest.groovy @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.repeat + +import org.gradle.StartParameter +import org.gradle.api.file.FileTree +import org.gradle.api.internal.DocumentationRegistry +import org.gradle.api.internal.tasks.testing.JvmTestExecutionSpec +import org.gradle.api.internal.tasks.testing.TestExecuter +import org.gradle.api.internal.tasks.testing.filter.DefaultTestFilter +import org.gradle.api.tasks.testing.Test +import org.gradle.internal.time.Clock +import org.gradle.internal.work.WorkerLeaseRegistry + +class RepeatTest extends Test { + int times = 1 + + /** + * Submit each test class for processing multiple times. + */ + @Override + FileTree getCandidateClassFiles() { + FileTree candidates = super.getCandidateClassFiles() + int additionalRuns = times - 1 + for (int i = 0; i < additionalRuns; i++) { + candidates = candidates.plus(super.getCandidateClassFiles()) + } + + return candidates + } + + /** + * Use a custom {@link TestExecuter} that processes each test class as many times as submitted. + */ + @Override + protected TestExecuter createTestExecuter() { + return new RepeatableTestExecuter( + super.createTestExecuter().workerFactory, + getActorFactory(), + getModuleRegistry(), + getServices().get(WorkerLeaseRegistry.class), + getServices().get(StartParameter.class).getMaxWorkerCount(), + getServices().get(Clock.class), + getServices().get(DocumentationRegistry.class), + (DefaultTestFilter) getFilter()) + } +} diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandle.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandle.java deleted file mode 100755 index a3cf93426ce8..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandle.java +++ /dev/null @@ -1,689 +0,0 @@ -/* - * Copyright 2010 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import static java.lang.String.format; - -import java.io.File; -import java.io.InputStream; -import java.io.OutputStream; -import java.io.PipedInputStream; -import java.io.PipedOutputStream; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collections; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.Executor; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; -import java.util.concurrent.locks.Condition; -import java.util.concurrent.locks.Lock; -import java.util.concurrent.locks.ReentrantLock; - -import javax.annotation.Nullable; - -import com.github.dockerjava.api.DockerClient; -import com.github.dockerjava.api.command.CreateContainerCmd; -import com.github.dockerjava.api.model.Frame; -import com.github.dockerjava.api.model.StreamType; -import com.github.dockerjava.api.model.WaitResponse; -import com.github.dockerjava.core.command.AttachContainerResultCallback; -import com.github.dockerjava.core.command.WaitContainerResultCallback; -import com.github.dockerjava.api.model.Bind; -import com.github.dockerjava.api.model.Volume; -import com.google.common.base.Joiner; -import groovy.lang.Closure; -import org.gradle.api.logging.Logger; -import org.gradle.api.logging.Logging; -import org.gradle.initialization.BuildCancellationToken; -import org.gradle.internal.UncheckedException; -import org.gradle.internal.event.ListenerBroadcast; -import org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable; -import org.gradle.process.ExecResult; -import org.gradle.process.internal.ExecException; -import org.gradle.process.internal.ExecHandle; -import org.gradle.process.internal.ExecHandleListener; -import org.gradle.process.internal.ExecHandleShutdownHookAction; -import org.gradle.process.internal.ExecHandleState; -import org.gradle.process.internal.ProcessSettings; -import org.gradle.process.internal.StreamsHandler; -import org.gradle.process.internal.shutdown.ShutdownHooks; - -/** - * Default implementation for the ExecHandle interface. - * - *

State flows

- * - *
    - *
  • INIT -> STARTED -> [SUCCEEDED|FAILED|ABORTED|DETACHED]
    • - *
  • INIT -> FAILED
    • - *
  • INIT -> STARTED -> DETACHED -> ABORTED
    • - *
- * - * State is controlled on all control methods: - *
    - *
  • {@link #start()} allowed when state is INIT
    • - *
  • {@link #abort()} allowed when state is STARTED or DETACHED
    • - *
- */ -public class DockerizedExecHandle implements ExecHandle, ProcessSettings { - - private static final Logger LOGGER = Logging.getLogger(DockerizedExecHandle.class); - - private final String displayName; - - /** - * The working directory of the process. - */ - private final File directory; - - /** - * The executable to run. - */ - private final String command; - - /** - * Arguments to pass to the executable. - */ - private final List arguments; - - /** - * The variables to set in the environment the executable is run in. - */ - private final Map environment; - private final StreamsHandler outputHandler; - private final StreamsHandler inputHandler; - private final boolean redirectErrorStream; - private int timeoutMillis; - private boolean daemon; - - /** - * Lock to guard all mutable state - */ - private final Lock lock; - private final Condition stateChanged; - - private final Executor executor; - - /** - * State of this ExecHandle. - */ - private ExecHandleState state; - - /** - * When not null, the runnable that is waiting - */ - private DockerizedExecHandleRunner execHandleRunner; - - private ExecResultImpl execResult; - - private final ListenerBroadcast broadcast; - - private final ExecHandleShutdownHookAction shutdownHookAction; - - private final BuildCancellationToken buildCancellationToken; - - private final DockerizedTestExtension testExtension; - - public DockerizedExecHandle(DockerizedTestExtension testExtension, String displayName, - File directory, String command, List arguments, - Map environment, StreamsHandler outputHandler, - StreamsHandler inputHandler, - List listeners, boolean redirectErrorStream, - int timeoutMillis, boolean daemon, - Executor executor, BuildCancellationToken buildCancellationToken) { - this.displayName = displayName; - this.directory = directory; - this.command = command; - this.arguments = arguments; - this.environment = environment; - this.outputHandler = outputHandler; - this.inputHandler = inputHandler; - this.redirectErrorStream = redirectErrorStream; - this.timeoutMillis = timeoutMillis; - this.daemon = daemon; - this.executor = executor; - this.buildCancellationToken = buildCancellationToken; - this.testExtension = testExtension; - lock = new ReentrantLock(); - stateChanged = lock.newCondition(); - state = ExecHandleState.INIT; - shutdownHookAction = new ExecHandleShutdownHookAction(this); - broadcast = new ListenerBroadcast<>(ExecHandleListener.class); - broadcast.addAll(listeners); - } - - @Override - public File getDirectory() { - return directory; - } - - @Override - public String getCommand() { - return command; - } - - public boolean isDaemon() { - return daemon; - } - - @Override - public String toString() { - return displayName; - } - - @Override - public List getArguments() { - return Collections.unmodifiableList(arguments); - } - - @Override - public Map getEnvironment() { - return Collections.unmodifiableMap(environment); - } - - @Override - public ExecHandleState getState() { - lock.lock(); - try { - return state; - } finally { - lock.unlock(); - } - } - - private void setState(ExecHandleState state) { - lock.lock(); - try { - LOGGER.debug("Changing state to: {}", state); - this.state = state; - stateChanged.signalAll(); - } finally { - lock.unlock(); - } - } - - private boolean stateIn(ExecHandleState... states) { - lock.lock(); - try { - return Arrays.asList(states).contains(state); - } finally { - lock.unlock(); - } - } - - private void setEndStateInfo(ExecHandleState newState, int exitValue, Throwable failureCause) { - ShutdownHooks.removeShutdownHook(shutdownHookAction); - buildCancellationToken.removeCallback(shutdownHookAction); - ExecHandleState currentState; - lock.lock(); - try { - currentState = state; - } finally { - lock.unlock(); - } - - ExecResultImpl - newResult = - new ExecResultImpl(exitValue, execExceptionFor(failureCause, currentState), displayName); - if (!currentState.isTerminal() && newState != ExecHandleState.DETACHED) { - try { - broadcast.getSource().executionFinished(this, newResult); - } catch (Exception e) { - newResult = new ExecResultImpl(exitValue, execExceptionFor(e, currentState), displayName); - } - } - - lock.lock(); - try { - setState(newState); - execResult = newResult; - } finally { - lock.unlock(); - } - - LOGGER.debug("Process '{}' finished with exit value {} (state: {})", displayName, exitValue, - newState); - } - - @Nullable - private ExecException execExceptionFor(Throwable failureCause, ExecHandleState currentState) { - return failureCause != null - ? new ExecException(failureMessageFor(currentState), failureCause) - : null; - } - - private String failureMessageFor(ExecHandleState currentState) { - return currentState == ExecHandleState.STARTING - ? format("A problem occurred starting process '%s'", displayName) - : format("A problem occurred waiting for process '%s' to complete.", displayName); - } - - @Override - public ExecHandle start() { - LOGGER.info("Starting process '{}'. Working directory: {} Command: {}", - displayName, directory, command + ' ' + Joiner.on(' ').useForNull("null").join(arguments)); - if (LOGGER.isDebugEnabled()) { - LOGGER.debug("Environment for process '{}': {}", displayName, environment); - } - lock.lock(); - try { - if (!stateIn(ExecHandleState.INIT)) { - throw new IllegalStateException( - format("Cannot start process '%s' because it has already been started", displayName)); - } - setState(ExecHandleState.STARTING); - - execHandleRunner = - new DockerizedExecHandleRunner(this, new CompositeStreamsHandler(), executor); - executor.execute(new CurrentBuildOperationPreservingRunnable(execHandleRunner)); - - while (stateIn(ExecHandleState.STARTING)) { - LOGGER.debug("Waiting until process started: {}.", displayName); - try { - if (!stateChanged.await(10, TimeUnit.MINUTES)) { - execHandleRunner.abortProcess(); - throw new RuntimeException("Giving up on " + execHandleRunner); - } - } catch (InterruptedException e) { - //ok, wrapping up - } - } - - if (execResult != null) { - execResult.rethrowFailure(); - } - - LOGGER.info("Successfully started process '{}'", displayName); - } finally { - lock.unlock(); - } - return this; - } - - @Override - public void abort() { - lock.lock(); - try { - if (stateIn(ExecHandleState.SUCCEEDED, ExecHandleState.FAILED, ExecHandleState.ABORTED)) { - return; - } - if (!stateIn(ExecHandleState.STARTED, ExecHandleState.DETACHED)) { - throw new IllegalStateException( - format("Cannot abort process '%s' because it is not in started or detached state. It is currently in state: '%s'", - displayName,state)); - } - execHandleRunner.abortProcess(); - waitForFinish(); - } finally { - lock.unlock(); - } - } - - @Override - public ExecResult waitForFinish() { - lock.lock(); - try { - while (!state.isTerminal()) { - try { - stateChanged.await(); - } catch (InterruptedException e) { - //ok, wrapping up... - throw UncheckedException.throwAsUncheckedException(e); - } - } - } finally { - lock.unlock(); - } - - // At this point: - // If in daemon mode, the process has started successfully and all streams to the process have been closed - // If in fork mode, the process has completed and all cleanup has been done - // In both cases, all asynchronous work for the process has completed and we're done - - return result(); - } - - private ExecResult result() { - lock.lock(); - try { - return execResult.rethrowFailure(); - } finally { - lock.unlock(); - } - } - - void detached() { - setEndStateInfo(ExecHandleState.DETACHED, 0, null); - } - - void started() { - ShutdownHooks.addShutdownHook(shutdownHookAction); - buildCancellationToken.addCallback(shutdownHookAction); - setState(ExecHandleState.STARTED); - broadcast.getSource().executionStarted(this); - } - - void finished(int exitCode) { - if (exitCode != 0) { - setEndStateInfo(ExecHandleState.FAILED, exitCode, null); - } else { - setEndStateInfo(ExecHandleState.SUCCEEDED, 0, null); - } - } - - void aborted(int exitCode) { - if (exitCode == 0) { - // This can happen on Windows - exitCode = -1; - } - setEndStateInfo(ExecHandleState.ABORTED, exitCode, null); - } - - void failed(Throwable failureCause) { - setEndStateInfo(ExecHandleState.FAILED, -1, failureCause); - } - - @Override - public void addListener(ExecHandleListener listener) { - broadcast.add(listener); - } - - @Override - public void removeListener(ExecHandleListener listener) { - broadcast.remove(listener); - } - - public String getDisplayName() { - return displayName; - } - - @Override - public boolean getRedirectErrorStream() { - return redirectErrorStream; - } - - public int getTimeout() { - return timeoutMillis; - } - - public Process runContainer() { - try { - DockerClient client = testExtension.getClient(); - CreateContainerCmd createCmd = client.createContainerCmd(testExtension.getImage()) - .withTty(false) - .withStdinOpen(true) - .withWorkingDir(directory.getAbsolutePath()); - - createCmd.withEnv(getEnv()); - - String user = testExtension.getUser(); - if (user != null) { - createCmd.withUser(user); - } - bindVolumes(createCmd); - List cmdLine = new ArrayList<>(); - cmdLine.add(command); - cmdLine.addAll(arguments); - createCmd.withCmd(cmdLine); - - invokeIfNotNull(testExtension.getBeforeContainerCreate(), createCmd, client); - String containerId = createCmd.exec().getId(); - invokeIfNotNull(testExtension.getAfterContainerCreate(), containerId, client); - - invokeIfNotNull(testExtension.getBeforeContainerStart(), containerId, client); - client.startContainerCmd(containerId).exec(); - invokeIfNotNull(testExtension.getAfterContainerStart(), containerId, client); - - if (!client.inspectContainerCmd(containerId).exec().getState().getRunning()) { - throw new RuntimeException("Container " + containerId + " not running!"); - } - - Process - proc = - new DockerizedProcess(client, containerId, testExtension.getAfterContainerStop()); - - return proc; - } catch (Exception e) { - e.printStackTrace(); - throw new RuntimeException(e); - } - } - - private void invokeIfNotNull(Closure closure, Object... args) { - if (closure != null) { - int l = closure.getParameterTypes().length; - Object[] nargs; - if (l < args.length) { - nargs = new Object[l]; - System.arraycopy(args, 0, nargs, 0, l); - } else { - nargs = args; - } - closure.call(nargs); - } - } - - private List getEnv() { - List env = new ArrayList<>(); - for (Map.Entry e : environment.entrySet()) { - env.add(e.getKey() + "=" + e.getValue()); - } - return env; - } - - private void bindVolumes(CreateContainerCmd cmd) { - List volumes = new ArrayList<>(); - List binds = new ArrayList<>(); - for (Object o : testExtension.getVolumes().entrySet()) { - @SuppressWarnings("unchecked") - Map.Entry e = (Map.Entry) o; - Volume volume = new Volume(e.getValue().toString()); - Bind bind = new Bind(e.getKey().toString(), volume); - binds.add(bind); - volumes.add(volume); - } - cmd.withVolumes(volumes).withBinds(binds); - } - - private static class ExecResultImpl implements ExecResult { - private final int exitValue; - private final ExecException failure; - private final String displayName; - - ExecResultImpl(int exitValue, ExecException failure, String displayName) { - this.exitValue = exitValue; - this.failure = failure; - this.displayName = displayName; - } - - @Override - public int getExitValue() { - return exitValue; - } - - @Override - public ExecResult assertNormalExitValue() throws ExecException { - return this; - } - - @Override - public ExecResult rethrowFailure() throws ExecException { - if (failure != null) { - throw failure; - } - return this; - } - - @Override - public String toString() { - return "{exitValue=" + exitValue + ", failure=" + failure + "}"; - } - } - - private class CompositeStreamsHandler implements StreamsHandler { - @Override - public void connectStreams(Process process, String processName, Executor executor) { - inputHandler.connectStreams(process, processName, executor); - outputHandler.connectStreams(process, processName, executor); - } - - @Override - public void start() { - inputHandler.start(); - outputHandler.start(); - } - - @Override - public void stop() { - inputHandler.stop(); - outputHandler.stop(); - } - - @Override - public void disconnect() { - inputHandler.disconnect(); - outputHandler.disconnect(); - } - } - - private class DockerizedProcess extends Process { - - private final DockerClient dockerClient; - private final String containerId; - private final Closure afterContainerStop; - - private final PipedOutputStream stdInWriteStream = new PipedOutputStream(); - private final PipedInputStream stdOutReadStream = new PipedInputStream(); - private final PipedInputStream stdErrReadStream = new PipedInputStream(); - private final PipedInputStream stdInReadStream = new PipedInputStream(stdInWriteStream); - private final PipedOutputStream stdOutWriteStream = new PipedOutputStream(stdOutReadStream); - private final PipedOutputStream stdErrWriteStream = new PipedOutputStream(stdErrReadStream); - - private final CountDownLatch finished = new CountDownLatch(1); - private AtomicInteger exitCode = new AtomicInteger(); - private final AttachContainerResultCallback - attachContainerResultCallback = - new AttachContainerResultCallback() { - @Override - public void onNext(Frame frame) { - try { - if (frame.getStreamType().equals(StreamType.STDOUT)) { - stdOutWriteStream.write(frame.getPayload()); - } else if (frame.getStreamType().equals(StreamType.STDERR)) { - stdErrWriteStream.write(frame.getPayload()); - } - } catch (Exception e) { - LOGGER.error("Error while writing to stream:", e); - } - super.onNext(frame); - } - }; - - private final WaitContainerResultCallback - waitContainerResultCallback = - new WaitContainerResultCallback() { - @Override - public void onNext(WaitResponse waitResponse) { - exitCode.set(waitResponse.getStatusCode()); - try { - attachContainerResultCallback.close(); - attachContainerResultCallback.awaitCompletion(); - stdOutWriteStream.close(); - stdErrWriteStream.close(); - } catch (Exception e) { - LOGGER.debug("Error by detaching streams", e); - } finally { - try { - invokeIfNotNull(afterContainerStop, containerId, dockerClient); - } catch (Exception e) { - LOGGER.debug("Exception thrown at invoking afterContainerStop", e); - } finally { - finished.countDown(); - } - - } - - - } - }; - - public DockerizedProcess(final DockerClient dockerClient, final String containerId, - final Closure afterContainerStop) throws Exception { - this.dockerClient = dockerClient; - this.containerId = containerId; - this.afterContainerStop = afterContainerStop; - attachStreams(); - dockerClient.waitContainerCmd(containerId).exec(waitContainerResultCallback); - } - - private void attachStreams() throws Exception { - dockerClient.attachContainerCmd(containerId) - .withFollowStream(true) - .withStdOut(true) - .withStdErr(true) - .withStdIn(stdInReadStream) - .exec(attachContainerResultCallback); - if (!attachContainerResultCallback.awaitStarted(2, TimeUnit.MINUTES)) { - LOGGER.warn("Not attached to container " + containerId + " within 10secs"); - throw new RuntimeException("Not attached to container " + containerId + " within 10secs"); - } - } - - @Override - public OutputStream getOutputStream() { - return stdInWriteStream; - } - - @Override - public InputStream getInputStream() { - return stdOutReadStream; - } - - @Override - public InputStream getErrorStream() { - return stdErrReadStream; - } - - @Override - public int waitFor() throws InterruptedException { - finished.await(); - return exitCode.get(); - } - - @Override - public int exitValue() { - if (finished.getCount() > 0) { - throw new IllegalThreadStateException("docker process still running"); - } - return exitCode.get(); - } - - @Override - public void destroy() { - dockerClient.killContainerCmd(containerId).exec(); - } - - @Override - public String toString() { - return "Container " + containerId + " on " + dockerClient.toString(); - } - } - -} diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandleRunner.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandleRunner.java deleted file mode 100644 index cfc2fb8b0107..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/DockerizedExecHandleRunner.java +++ /dev/null @@ -1,102 +0,0 @@ -/* - * Copyright 2010 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import java.util.concurrent.Executor; -import java.util.concurrent.locks.Lock; -import java.util.concurrent.locks.ReentrantLock; - -import org.gradle.api.logging.Logger; -import org.gradle.api.logging.Logging; -import org.gradle.process.internal.StreamsHandler; - -public class DockerizedExecHandleRunner implements Runnable { - private static final Logger - LOGGER = - Logging.getLogger(org.gradle.process.internal.ExecHandleRunner.class); - - private final DockerizedExecHandle execHandle; - private final Lock lock = new ReentrantLock(); - private final Executor executor; - - private Process process; - private boolean aborted; - private final StreamsHandler streamsHandler; - - public DockerizedExecHandleRunner(DockerizedExecHandle execHandle, StreamsHandler streamsHandler, - Executor executor) { - this.executor = executor; - if (execHandle == null) { - throw new IllegalArgumentException("execHandle == null!"); - } - this.streamsHandler = streamsHandler; - this.execHandle = execHandle; - } - - public void abortProcess() { - lock.lock(); - try { - aborted = true; - if (process != null) { - LOGGER.debug("Abort requested. Destroying process: {}.", execHandle.getDisplayName()); - process.destroy(); - } - } finally { - lock.unlock(); - } - } - - @Override - public void run() { - try { - process = execHandle.runContainer(); - streamsHandler.connectStreams(process, execHandle.getDisplayName(), executor); - - execHandle.started(); - - LOGGER.debug("waiting until streams are handled..."); - streamsHandler.start(); - if (execHandle.isDaemon()) { - streamsHandler.stop(); - detached(); - } else { - int exitValue = process.waitFor(); - streamsHandler.stop(); - completed(exitValue); - } - } catch (Throwable t) { - execHandle.failed(t); - } - } - - private void completed(int exitValue) { - if (aborted) { - execHandle.aborted(exitValue); - } else { - execHandle.finished(exitValue); - } - } - - private void detached() { - execHandle.detached(); - } - - public String toString() { - return "Handler for " + process.toString(); - } -} - diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForciblyStoppableTestWorker.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForciblyStoppableTestWorker.java deleted file mode 100644 index fbbe48e50264..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForciblyStoppableTestWorker.java +++ /dev/null @@ -1,45 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import java.util.Timer; -import java.util.TimerTask; -import java.util.concurrent.TimeUnit; - -import org.gradle.api.internal.tasks.testing.WorkerTestClassProcessorFactory; -import org.gradle.api.internal.tasks.testing.worker.TestWorker; - -public class ForciblyStoppableTestWorker extends TestWorker { - private static final int SHUTDOWN_TIMEOUT = 60; // secs - - public ForciblyStoppableTestWorker(WorkerTestClassProcessorFactory factory) { - super(factory); - } - - @Override - public void stop() { - new Timer(true).schedule(new TimerTask() { - @Override - public void run() { - System.err.println("Worker process did not shutdown gracefully within " + SHUTDOWN_TIMEOUT - + "s, forcing it now"); - Runtime.getRuntime().halt(-100); - } - }, TimeUnit.SECONDS.toMillis(SHUTDOWN_TIMEOUT)); - super.stop(); - } -} diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForkingTestClassProcessor.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForkingTestClassProcessor.java deleted file mode 100644 index d6f409b461ab..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/ForkingTestClassProcessor.java +++ /dev/null @@ -1,153 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import java.io.File; -import java.net.URL; -import java.util.List; - -import org.gradle.api.Action; -import org.gradle.api.internal.classpath.ModuleRegistry; -import org.gradle.api.internal.tasks.testing.TestClassProcessor; -import org.gradle.api.internal.tasks.testing.TestClassRunInfo; -import org.gradle.api.internal.tasks.testing.TestResultProcessor; -import org.gradle.api.internal.tasks.testing.WorkerTestClassProcessorFactory; -import org.gradle.api.internal.tasks.testing.worker.RemoteTestClassProcessor; -import org.gradle.api.internal.tasks.testing.worker.TestEventSerializer; -import org.gradle.internal.remote.ObjectConnection; -import org.gradle.process.JavaForkOptions; -import org.gradle.process.internal.worker.WorkerProcess; -import org.gradle.process.internal.worker.WorkerProcessBuilder; -import org.gradle.process.internal.worker.WorkerProcessFactory; -import org.gradle.util.CollectionUtils; - -public class ForkingTestClassProcessor implements TestClassProcessor { - private final WorkerProcessFactory workerFactory; - private final WorkerTestClassProcessorFactory processorFactory; - private final JavaForkOptions options; - private final Iterable classPath; - private final Action buildConfigAction; - private final ModuleRegistry moduleRegistry; - private RemoteTestClassProcessor remoteProcessor; - private WorkerProcess workerProcess; - private TestResultProcessor resultProcessor; - - public ForkingTestClassProcessor(WorkerProcessFactory workerFactory, - WorkerTestClassProcessorFactory processorFactory, - JavaForkOptions options, Iterable classPath, - Action buildConfigAction, - ModuleRegistry moduleRegistry) { - this.workerFactory = workerFactory; - this.processorFactory = processorFactory; - this.options = options; - this.classPath = classPath; - this.buildConfigAction = buildConfigAction; - this.moduleRegistry = moduleRegistry; - } - - @Override - public void startProcessing(TestResultProcessor resultProcessor) { - this.resultProcessor = resultProcessor; - } - - @Override - public void processTestClass(TestClassRunInfo testClass) { - int i = 0; - RuntimeException exception = null; - while (remoteProcessor == null && i < 10) { - try { - remoteProcessor = forkProcess(); - exception = null; - break; - } catch (RuntimeException e) { - exception = e; - i++; - } - } - - if (exception != null) { - throw exception; - } - remoteProcessor.processTestClass(testClass); - } - - RemoteTestClassProcessor forkProcess() { - WorkerProcessBuilder - builder = - workerFactory.create(new ForciblyStoppableTestWorker(processorFactory)); - builder.setBaseName("Gradle Test Executor"); - builder.setImplementationClasspath(getTestWorkerImplementationClasspath()); - builder.applicationClasspath(classPath); - options.copyTo(builder.getJavaCommand()); - buildConfigAction.execute(builder); - - workerProcess = builder.build(); - workerProcess.start(); - - ObjectConnection connection = workerProcess.getConnection(); - connection.useParameterSerializers(TestEventSerializer.create()); - connection.addIncoming(TestResultProcessor.class, resultProcessor); - RemoteTestClassProcessor - remoteProcessor = - connection.addOutgoing(RemoteTestClassProcessor.class); - connection.connect(); - remoteProcessor.startProcessing(); - return remoteProcessor; - } - - List getTestWorkerImplementationClasspath() { - return CollectionUtils.flattenCollections(URL.class, - moduleRegistry.getModule("gradle-core-api").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-core").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-logging").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-messaging").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-base-services").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-cli").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-native").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-testing-base").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-testing-jvm").getImplementationClasspath().getAsURLs(), - moduleRegistry.getModule("gradle-process-services").getImplementationClasspath() - .getAsURLs(), - moduleRegistry.getExternalModule("slf4j-api").getImplementationClasspath().getAsURLs(), - moduleRegistry.getExternalModule("jul-to-slf4j").getImplementationClasspath().getAsURLs(), - moduleRegistry.getExternalModule("native-platform").getImplementationClasspath() - .getAsURLs(), - moduleRegistry.getExternalModule("kryo").getImplementationClasspath().getAsURLs(), - moduleRegistry.getExternalModule("commons-lang").getImplementationClasspath().getAsURLs(), - moduleRegistry.getExternalModule("junit").getImplementationClasspath().getAsURLs(), - ForkingTestClassProcessor.class.getProtectionDomain().getCodeSource().getLocation() - ); - } - - @Override - public void stop() { - if (remoteProcessor != null) { - try { - remoteProcessor.stop(); - workerProcess.waitForStop(); - } finally { - // do nothing - } - } - } - - @Override - public void stopNow() { - stop(); // TODO need anything else ?? - } - -} diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/NoMemoryManager.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/NoMemoryManager.java deleted file mode 100644 index 3a8be604f006..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/NoMemoryManager.java +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import org.gradle.process.internal.health.memory.JvmMemoryStatusListener; -import org.gradle.process.internal.health.memory.MemoryHolder; -import org.gradle.process.internal.health.memory.MemoryManager; -import org.gradle.process.internal.health.memory.OsMemoryStatusListener; - -public class NoMemoryManager implements MemoryManager { - @Override - public void addListener(JvmMemoryStatusListener jvmMemoryStatusListener) { - - } - - @Override - public void addListener(OsMemoryStatusListener osMemoryStatusListener) { - - } - - @Override - public void removeListener(JvmMemoryStatusListener jvmMemoryStatusListener) { - - } - - @Override - public void removeListener(OsMemoryStatusListener osMemoryStatusListener) { - - } - - @Override - public void addMemoryHolder(MemoryHolder memoryHolder) { - - } - - @Override - public void removeMemoryHolder(MemoryHolder memoryHolder) { - - } - - @Override - public void requestFreeMemory(long l) { - - } -} diff --git a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/TestExecuter.java b/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/TestExecuter.java deleted file mode 100644 index b99e124a7473..000000000000 --- a/buildSrc/src/main/java/com/pedjak/gradle/plugins/dockerizedtest/TestExecuter.java +++ /dev/null @@ -1,119 +0,0 @@ -/* - * Copyright 2015 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.pedjak.gradle.plugins.dockerizedtest; - -import java.io.File; -import java.util.Set; -import java.util.UUID; - -import com.google.common.collect.ImmutableSet; -import org.gradle.api.file.FileTree; -import org.gradle.api.internal.classpath.ModuleRegistry; -import org.gradle.api.internal.tasks.testing.JvmTestExecutionSpec; -import org.gradle.api.internal.tasks.testing.TestClassProcessor; -import org.gradle.api.internal.tasks.testing.TestFramework; -import org.gradle.api.internal.tasks.testing.TestResultProcessor; -import org.gradle.api.internal.tasks.testing.WorkerTestClassProcessorFactory; -import org.gradle.api.internal.tasks.testing.detection.DefaultTestClassScanner; -import org.gradle.api.internal.tasks.testing.detection.TestFrameworkDetector; -import org.gradle.api.internal.tasks.testing.processors.MaxNParallelTestClassProcessor; -import org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor; -import org.gradle.api.internal.tasks.testing.processors.TestMainAction; -import org.gradle.internal.Factory; -import org.gradle.internal.actor.ActorFactory; -import org.gradle.internal.operations.BuildOperationExecutor; -import org.gradle.internal.time.Clock; -import org.gradle.process.internal.worker.WorkerProcessFactory; - -public class TestExecuter - implements org.gradle.api.internal.tasks.testing.TestExecuter { - private final WorkerProcessFactory workerFactory; - private final ActorFactory actorFactory; - private final ModuleRegistry moduleRegistry; - private final BuildOperationExecutor buildOperationExecutor; - private final Clock clock; - private TestClassProcessor processor; - - public TestExecuter(WorkerProcessFactory workerFactory, ActorFactory actorFactory, - ModuleRegistry moduleRegistry, BuildOperationExecutor buildOperationExecutor, - Clock clock) { - this.workerFactory = workerFactory; - this.actorFactory = actorFactory; - this.moduleRegistry = moduleRegistry; - this.buildOperationExecutor = buildOperationExecutor; - this.clock = clock; - } - - @Override - public void execute(final JvmTestExecutionSpec testExecutionSpec, - TestResultProcessor testResultProcessor) { - final TestFramework testFramework = testExecutionSpec.getTestFramework(); - final WorkerTestClassProcessorFactory testInstanceFactory = testFramework.getProcessorFactory(); - final Set classpath = ImmutableSet.copyOf(testExecutionSpec.getClasspath()); - final Factory forkingProcessorFactory = new Factory() { - @Override - public TestClassProcessor create() { - return new ForkingTestClassProcessor(workerFactory, testInstanceFactory, - testExecutionSpec.getJavaForkOptions(), - classpath, testFramework.getWorkerConfigurationAction(), moduleRegistry); - } - }; - Factory reforkingProcessorFactory = new Factory() { - @Override - public TestClassProcessor create() { - return new RestartEveryNTestClassProcessor(forkingProcessorFactory, - testExecutionSpec.getForkEvery()); - } - }; - - processor = new MaxNParallelTestClassProcessor(testExecutionSpec.getMaxParallelForks(), - reforkingProcessorFactory, actorFactory); - - final FileTree testClassFiles = testExecutionSpec.getCandidateClassFiles(); - - Runnable detector; - if (testExecutionSpec.isScanForTestClasses()) { - TestFrameworkDetector - testFrameworkDetector = - testExecutionSpec.getTestFramework().getDetector(); - testFrameworkDetector.setTestClasses(testExecutionSpec.getTestClassesDirs().getFiles()); - testFrameworkDetector.setTestClasspath(classpath); - detector = new DefaultTestClassScanner(testClassFiles, testFrameworkDetector, processor); - } else { - detector = new DefaultTestClassScanner(testClassFiles, null, processor); - } - - Object testTaskOperationId; - - try { - testTaskOperationId = buildOperationExecutor.getCurrentOperation().getParentId(); - } catch (Exception e) { - testTaskOperationId = UUID.randomUUID(); - } - - new TestMainAction(detector, processor, testResultProcessor, clock, testTaskOperationId, - testExecutionSpec.getPath(), "Gradle Test Run " + testExecutionSpec.getIdentityPath()) - .run(); - } - - @Override - public void stopNow() { - if (processor != null) { - processor.stopNow(); - } - } -} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/RunInSubdirectoryTestFramework.java b/buildSrc/src/main/java/org/apache/geode/gradle/RunInSubdirectoryTestFramework.java deleted file mode 100644 index 546ae1e7b5c6..000000000000 --- a/buildSrc/src/main/java/org/apache/geode/gradle/RunInSubdirectoryTestFramework.java +++ /dev/null @@ -1,102 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more contributor license - * agreements. See the NOTICE file distributed with this work for additional information regarding - * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance with the License. You may obtain a - * copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software distributed under the License - * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express - * or implied. See the License for the specific language governing permissions and limitations under - * the License. - */ - -package org.apache.geode.gradle; - -import static java.nio.file.StandardCopyOption.COPY_ATTRIBUTES; - -import java.io.IOException; -import java.io.UncheckedIOException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.concurrent.atomic.AtomicLong; - -import org.gradle.api.Action; -import org.gradle.api.internal.tasks.testing.TestFramework; -import org.gradle.api.internal.tasks.testing.WorkerTestClassProcessorFactory; -import org.gradle.api.internal.tasks.testing.detection.TestFrameworkDetector; -import org.gradle.api.tasks.testing.TestFrameworkOptions; -import org.gradle.process.internal.JavaExecHandleBuilder; -import org.gradle.process.internal.worker.WorkerProcessBuilder; - -/** - * Wraps a test framework to run each test worker in a separate working directory. - */ -public class RunInSubdirectoryTestFramework implements TestFramework { - private static final String GEMFIRE_PROPERTIES = "gemfire.properties"; - private final AtomicLong workerId = new AtomicLong(); - private final TestFramework delegate; - - public RunInSubdirectoryTestFramework(TestFramework delegate) { - this.delegate = delegate; - } - - @Override - public TestFrameworkDetector getDetector() { - return delegate.getDetector(); - } - - @Override - public TestFrameworkOptions getOptions() { - return delegate.getOptions(); - } - - @Override - public WorkerTestClassProcessorFactory getProcessorFactory() { - return delegate.getProcessorFactory(); - } - - /** - * Return an action that configures the test worker builder to run the test worker in a unique - * subdirectory of the task's working directory. - */ - @Override - public Action getWorkerConfigurationAction() { - return workerProcessBuilder -> { - delegate.getWorkerConfigurationAction().execute(workerProcessBuilder); - JavaExecHandleBuilder javaCommand = workerProcessBuilder.getJavaCommand(); - - Path taskWorkingDir = javaCommand.getWorkingDir().toPath(); - String workerWorkingDirName = String.format("test-worker-%06d", workerId.incrementAndGet()); - Path workerWorkingDir = taskWorkingDir.resolve(workerWorkingDirName); - - createWorkingDir(workerWorkingDir); - copyGemFirePropertiesFile(taskWorkingDir, workerWorkingDir); - - javaCommand.setWorkingDir(workerWorkingDir); - }; - } - - private void copyGemFirePropertiesFile(Path taskWorkingDir, Path workerWorkingDir) { - Path taskPropertiesFile = taskWorkingDir.resolve(GEMFIRE_PROPERTIES); - if (!Files.exists(taskPropertiesFile)) { - return; - } - Path workerPropertiesFile = workerWorkingDir.resolve(taskPropertiesFile.getFileName()); - try { - Files.copy(taskPropertiesFile, workerPropertiesFile, COPY_ATTRIBUTES); - } catch (IOException e) { - throw new UncheckedIOException(e); - } - } - - private void createWorkingDir(Path workerWorkingDir) { - try { - Files.createDirectories(workerWorkingDir); - } catch (IOException e) { - throw new UncheckedIOException(e); - } - } -} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerConnectionAcceptor.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerConnectionAcceptor.java new file mode 100644 index 000000000000..8e38d266d363 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerConnectionAcceptor.java @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.dockerized; + +import static java.util.Collections.list; +import static java.util.stream.Collectors.toList; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.NetworkInterface; +import java.net.SocketException; +import java.util.List; + +import org.gradle.api.UncheckedIOException; +import org.gradle.internal.remote.Address; +import org.gradle.internal.remote.ConnectionAcceptor; +import org.gradle.internal.remote.internal.inet.MultiChoiceAddress; + +/** + * Wraps a {@link ConnectionAcceptor} to give it a {@link MultiChoiceAddress} that processes in + * Docker containers can use to connect to Gradle's messaging server. + */ +class DockerConnectionAcceptor implements ConnectionAcceptor { + private static final List DOCKER_ACCEPTABLE_ADDRESSES; + + static { + try { + DOCKER_ACCEPTABLE_ADDRESSES = list(NetworkInterface.getNetworkInterfaces()).stream() + .filter(DockerConnectionAcceptor::isAcceptable) + .flatMap(i -> list(i.getInetAddresses()).stream()) + .filter(DockerConnectionAcceptor::isAcceptable) + .collect(toList()); + } catch (SocketException e) { + throw new UncheckedIOException("Unable to identify usable addresses", e); + } + } + + private final MultiChoiceAddress address; + private final ConnectionAcceptor delegate; + + /** + * Creates a {@code MultiChoiceAddress} whose candidate addresses are acceptable for processes in + * Docker containers to use to attempt to connect to Gradle's messaging server. The messaging + * server will accept connections from one of those candidate addresses. + */ + DockerConnectionAcceptor(ConnectionAcceptor delegate) { + this.delegate = delegate; + // The delegate's candidates are the host's loopback addresses, which processes in Docker + // containers cannot use. + MultiChoiceAddress original = (MultiChoiceAddress) delegate.getAddress(); + // Replace the delegate's unacceptable candidate addresses with acceptable ones. + address = new MultiChoiceAddress(original.getCanonicalAddress(), original.getPort(), + DOCKER_ACCEPTABLE_ADDRESSES); + } + + @Override + public Address getAddress() { + return address; + } + + @Override + public void requestStop() { + delegate.requestStop(); + } + + @Override + public void stop() { + delegate.stop(); + } + + /** + * Reports whether the candidate interface is acceptable for processes in Docker containers to use + * to connect to Gradle's messaging server. An interface is acceptable if it satisfies all of: + *
    + *
  • it is up
    • + *
  • it is not a loopback interface
    • + *
  • it is not a point to point interface (e.g. VPN)
    • + */ + private static boolean isAcceptable(NetworkInterface candidate) { + try { + return !candidate.isLoopback() + && !candidate.isPointToPoint() + && candidate.isUp(); + } catch (SocketException ignored) { + return false; + } + } + + /** + * Reports whether the candidate address is acceptable for processes in Docker containers to use + * to connect to Gradle's messaging server. An address is acceptable if it is reachable. + */ + private static boolean isAcceptable(InetAddress candidate) { + try { + return candidate.isReachable(2000); + } catch (IOException ignored) { + return false; + } + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerMessagingServer.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerMessagingServer.java new file mode 100644 index 000000000000..69a28c7a0d2a --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerMessagingServer.java @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.dockerized; + +import org.gradle.api.Action; +import org.gradle.internal.concurrent.ExecutorFactory; +import org.gradle.internal.remote.ConnectionAcceptor; +import org.gradle.internal.remote.MessagingServer; +import org.gradle.internal.remote.ObjectConnection; +import org.gradle.internal.remote.internal.ConnectCompletion; +import org.gradle.internal.remote.internal.IncomingConnector; +import org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection; + +/** + * A copy of MessageHubBackedServer from Gradle v6.8.3, modified to accept connections from + * processes running in Docker containers. + */ +public class DockerMessagingServer implements MessagingServer { + private final IncomingConnector connector; + private final ExecutorFactory executorFactory; + + public DockerMessagingServer(IncomingConnector connector, ExecutorFactory executorFactory) { + this.connector = connector; + this.executorFactory = executorFactory; + } + + /** + * Transforms Gradle's standard connection acceptor into one that will accept connections from + * worker processes in Docker containers. A connection acceptor reports a list of candidate + * addresses for worker processes to try to connect to. Gradle's standard acceptor reports the + * host's loopback addresses, which processes in Docker containers cannot use. The transformed + * acceptor instead reports a list of non-loopback addresses, and the Dockerized process will be + * able to use at least one of those to connect to this server. + */ + @Override + public ConnectionAcceptor accept(Action action) { + ConnectEventAction connectEventAction = new ConnectEventAction(action); + ConnectionAcceptor originalConnectionAcceptor = connector.accept(connectEventAction, true); + return new DockerConnectionAcceptor(originalConnectionAcceptor); + } + + /** + * An unmodified copy of MessageHubBackedServer.ConnectionEvent from Gradle v6.8.3. + */ + private class ConnectEventAction implements Action { + private final Action action; + + public ConnectEventAction(Action action) { + this.action = action; + } + + @Override + public void execute(ConnectCompletion completion) { + action.execute(new MessageHubBackedObjectConnection(executorFactory, completion)); + } + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcess.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcess.java new file mode 100644 index 000000000000..d799b29d5f67 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcess.java @@ -0,0 +1,287 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.dockerized; + +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.apache.geode.gradle.testing.dockerized.DockerTestWorkerConfig.getDurationWarningThreshold; + +import java.io.Closeable; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.PipedInputStream; +import java.io.PipedOutputStream; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicInteger; + +import com.github.dockerjava.api.DockerClient; +import com.github.dockerjava.api.async.ResultCallback; +import com.github.dockerjava.api.async.ResultCallbackTemplate; +import com.github.dockerjava.api.model.Frame; +import com.github.dockerjava.api.model.StreamType; +import com.github.dockerjava.api.model.WaitResponse; +import org.gradle.api.logging.Logger; +import org.gradle.api.logging.Logging; +import org.gradle.internal.UncheckedException; + +/** + * Represents a process running in a Docker container. + */ +public class DockerProcess extends Process { + private static final Logger LOGGER = Logging.getLogger(DockerProcess.class); + + private final String owner; + private final DockerClient client; + private final String containerId; + private final int timeoutMillis; + private final Runnable onCompletion; + private final PipedOutputStream stdIn = new PipedOutputStream(); + private final PipedInputStream stdOut = new PipedInputStream(); + private final PipedInputStream stdErr = new PipedInputStream(); + private final PipedInputStream stdInToContainer = new PipedInputStream(); + private final PipedOutputStream stdOutFromContainer = new PipedOutputStream(); + private final PipedOutputStream stdErrFromContainer = new PipedOutputStream(); + private final AtomicInteger exitCode = new AtomicInteger(); + private final CountDownLatch finished = new CountDownLatch(1); + private final OutputListener outputListener = new OutputListener(); + private final TerminationListener terminationListener = new TerminationListener(); + + /** + * Creates a {@link Process} that represents a process running in a Docker container. + * + * @param owner the name of this process's owner (used for diagnostics) + * @param client a Docker client to use to listen for process output and termination + * @param containerId the ID of the container in which the process is running + * @param timeoutMillis duration to wait for each listener to start + * @param onCompletion a runnable to run when this process completes + * @return a Process that represents the process in the container + */ + public static Process attachedTo(String owner, DockerClient client, String containerId, + int timeoutMillis, Runnable onCompletion) { + DockerProcess process = new DockerProcess( + owner, client, containerId, timeoutMillis, onCompletion); + try { + process.attach(); + } catch (Exception e) { + UncheckedException.throwAsUncheckedException(e); + } + return process; + } + + private DockerProcess(String owner, DockerClient client, String containerId, int timeoutMillis, + Runnable onCompletion) { + this.owner = owner; + this.client = client; + this.containerId = containerId; + this.timeoutMillis = timeoutMillis; + this.onCompletion = onCompletion; + } + + @Override + public OutputStream getOutputStream() { + return stdIn; + } + + @Override + public InputStream getInputStream() { + return stdOut; + } + + @Override + public InputStream getErrorStream() { + return stdErr; + } + + @Override + public int waitFor() throws InterruptedException { + finished.await(); + return exitValue(); + } + + @Override + public int exitValue() { + if (finished.getCount() != 0) { + throw new IllegalThreadStateException(toString() + " is still running"); + } + return exitCode.get(); + } + + @Override + public void destroy() { + finish(); + } + + @Override + public String toString() { + return String.format("DockerProcess{%s:%s}", owner, containerId); + } + + /** + * Attach this {@code DockerProcess}'s input and output streams to the container's, and set a + * callback for when the containerized process finishes. + * + * @throws Exception if an error occurs while attaching to the container + */ + private void attach() throws Exception { + listenForTermination(); + connectStreams(); + listenForOutput(); + } + + private void listenForOutput() throws InterruptedException { + LOGGER.debug("{} installing {}", this, outputListener); + try { + long startTime = System.currentTimeMillis(); + client.attachContainerCmd(containerId) + .withFollowStream(true) + .withStdOut(true) + .withStdErr(true) + .withStdIn(stdInToContainer) + .exec(outputListener); + LOGGER.debug("{} installed {}", this, outputListener); + long duration = System.currentTimeMillis() - startTime; + if(duration > getDurationWarningThreshold()) { + LOGGER.warn("{} {} installation took {}ms", this, outputListener, duration); + } + } catch (RuntimeException e) { + String message = String.format("%s error while installing %s", this, outputListener); + throw new RuntimeException(message, e); + } + waitUntilStarted(outputListener); + } + + private void connectStreams() throws IOException { + stdInToContainer.connect(stdIn); + stdOutFromContainer.connect(stdOut); + stdErrFromContainer.connect(stdErr); + } + + private void listenForTermination() throws InterruptedException { + LOGGER.debug("{} installing {}", this, terminationListener); + try { + long startTime = System.currentTimeMillis(); + client.waitContainerCmd(containerId) + .exec(terminationListener); + LOGGER.debug("{} installed {}", this, terminationListener); + long duration = System.currentTimeMillis() - startTime; + if(duration > getDurationWarningThreshold()) { + LOGGER.warn("{} {} installation took {}ms", this, terminationListener, duration); + } + } catch (RuntimeException e) { + String message = String.format("%s error while installing %s", this, terminationListener); + throw new RuntimeException(message, e); + } + waitUntilStarted(terminationListener); + } + + private void waitUntilStarted(ResultCallbackTemplate listener) + throws InterruptedException { + if (timeoutMillis > 0) { + LOGGER.debug("{} waiting {}ms for {} to start", this, timeoutMillis, listener); + if (!listener.awaitStarted(timeoutMillis, MILLISECONDS)) { + String message = String.format( + "%s timed out after %dms waiting for %s to start", this, timeoutMillis, listener); + throw new RuntimeException(message); + } + } else { + LOGGER.debug("{} waiting for {} to start", this, listener); + listener.awaitStarted(); + } + LOGGER.debug("{} {} started", this, listener); + } + + private void finish() { + close("stdin", stdIn); + close("stdout", stdOut); + close("stderr", stdErr); + close("stdin to container", stdInToContainer); + close("stdout from container", stdOutFromContainer); + close("stderr from container", stdErrFromContainer); + close("client", client); + finished.countDown(); + onCompletion.run(); + } + + private void close(String name, Closeable closeable) { + try { + closeable.close(); + LOGGER.debug("{} closed {}", this, name); + } catch (IOException e) { + String message = String.format("%s error while closing %s", this, name); + LOGGER.warn(message, e); + } + } + + /** + * A listener for Docker to notify whenever the containerized process writes new output. The + * listener copies each frame of the process's output to this DockerProcess's stdout or stderr. + */ + private class OutputListener extends ResultCallback.Adapter { + @Override + public void onNext(Frame frame) { + byte[] payload = frame.getPayload(); + StreamType streamType = frame.getStreamType(); + try { + switch (streamType) { + case STDOUT: + stdOutFromContainer.write(payload); + break; + case STDERR: + stdErrFromContainer.write(payload); + break; + default: + } + } catch (IOException e) { + String message = String.format("%s %s error while writing to %s", + DockerProcess.this.toString(), this, streamType); + LOGGER.error(message, e); + } + } + @Override + public String toString() { + return "output listener"; + } + + } + + /** + * A listener for Docker to notify when the containerized process terminates. The listener records + * the process's exit code, stops watching its streams, closes this {@code DockerProcess}'s output + * streams, and removes the Docker container. + */ + private class TerminationListener extends ResultCallback.Adapter { + @Override + public void onNext(WaitResponse response) { + Integer statusCode = response.getStatusCode(); + LOGGER.debug("{} {} called: process exited with status code {}", + DockerProcess.this, this, statusCode); + exitCode.set(statusCode); + try { + outputListener.close(); + outputListener.awaitCompletion(); + } catch (Exception e) { + String message = String.format("%s error while removing %s", DockerProcess.this, this); + LOGGER.warn(message, e); + } finally { + finish(); + } + } + + @Override + public String toString() { + return "termination listener"; + } + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcessLauncher.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcessLauncher.java new file mode 100644 index 000000000000..eb190d4fa3f4 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/DockerProcessLauncher.java @@ -0,0 +1,227 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.geode.gradle.testing.dockerized; + +import static java.util.stream.Collectors.toList; +import static org.apache.geode.gradle.testing.dockerized.DockerTestWorkerConfig.getDurationWarningThreshold; + +import java.io.IOException; +import java.util.List; +import java.util.Map; +import java.util.function.Consumer; + +import com.github.dockerjava.api.DockerClient; +import com.github.dockerjava.api.command.CreateContainerCmd; +import com.github.dockerjava.api.command.InspectContainerResponse; +import com.github.dockerjava.api.model.Bind; +import com.github.dockerjava.api.model.Volume; +import com.github.dockerjava.core.AbstractDockerCmdExecFactory; +import com.github.dockerjava.core.DockerClientBuilder; +import com.github.dockerjava.netty.NettyDockerCmdExecFactory; +import org.gradle.api.logging.Logger; +import org.gradle.api.logging.Logging; +import org.gradle.internal.UncheckedException; + +import org.apache.geode.gradle.testing.process.ProcessLauncher; + +/** + * A process launcher that launches each process in a Docker container. + */ +public class DockerProcessLauncher implements ProcessLauncher { + private static final Logger LOGGER = Logging.getLogger(DockerProcessLauncher.class); + private final DockerTestWorkerConfig config; + private final Consumer adjustment; + + /** + * @param config the configuration of the docker containers + * @param adjustment configures the process builder before it is dockerized + */ + public DockerProcessLauncher(DockerTestWorkerConfig config, Consumer adjustment) { + this.config = config; + this.adjustment = adjustment; + } + + /** + * Launches the specified process in a Docker container. + * + * @param processBuilder a builder that specifies the process to launch + * @return a Process that represents the process running in the Docker container + */ + @Override + public Process start(ProcessBuilder processBuilder) { + adjustment.accept(processBuilder); + config.dockerize(processBuilder); + int timeout = config.getTimeoutMillis(); + + // For synchronous Docker operations (create, start, inspect), time out if connecting or + // reading takes too long. + DockerClient clientForSynchronousOperations = dockerClient(timeout, timeout); + + String containerId = createContainer(processBuilder, clientForSynchronousOperations); + + try { + startContainer(clientForSynchronousOperations, containerId); + // For asynchronous Docker operations, time out only on connects. The DockerProcess uses the + // async client to listen for process output and process termination. Because the client must + // listen indefinitely for these events, do not time out on reads. + DockerClient clientForAsynchronousOperations = dockerClient(timeout, 0); + return DockerProcess.attachedTo( + config.getName(), clientForAsynchronousOperations, containerId, timeout, + () -> removeContainer(clientForSynchronousOperations, containerId)); + } catch (Exception e) { + removeContainer(clientForSynchronousOperations, containerId); + UncheckedException.throwAsUncheckedException(e); + return null; // Unreachable + } + } + + /** + * Creates a docker client with the given timeouts. + * + * @param connectTimeout timeout for connecting, or 0 to disable connect timeouts + * @param readTimeout timeout for reading, or 0 to disable read timeouts + */ + private static DockerClient dockerClient(int connectTimeout, int readTimeout) { + AbstractDockerCmdExecFactory cmdExecFactory = new NettyDockerCmdExecFactory(); + if (connectTimeout > 0) { + cmdExecFactory.withConnectTimeout(connectTimeout); + } + if (readTimeout > 0) { + cmdExecFactory.withReadTimeout(readTimeout); + } + // Must use the deprecated withDockerCmdExecFactory() because it is currently the only way to + // use Netty, and Netty is currently the only transport that supports timeouts. + @SuppressWarnings("deprecation") + DockerClient client = DockerClientBuilder.getInstance() + .withDockerCmdExecFactory(cmdExecFactory) + .build(); + return client; + } + + private String createContainer(ProcessBuilder processBuilder, DockerClient client) { + CreateContainerCmd createContainerCommand = client.createContainerCmd(config.getImage()) + .withTty(false) + .withStdinOpen(true) + .withWorkingDir(processBuilder.directory().getAbsolutePath()) + .withEnv(asStrings(processBuilder.environment())) + .withCmd(processBuilder.command()); + setUser(createContainerCommand); + setVolumes(createContainerCommand); + LOGGER.debug("{} creating container", this); + try { + long startTime = System.currentTimeMillis(); + String containerId = createContainerCommand.exec().getId(); + long duration = System.currentTimeMillis() - startTime; + if (duration > getDurationWarningThreshold()) { + LOGGER.warn("{} create took {}ms", this, duration); + } + LOGGER.debug("{} created container {}", this, containerId); + return containerId; + } catch (RuntimeException e) { + String message = String.format("%s error while creating container", this); + throw new RuntimeException(message, e); + } + } + + private void startContainer(DockerClient client, String containerId) { + LOGGER.debug("{} starting container {}", this, containerId); + try { + long startTime = System.currentTimeMillis(); + client.startContainerCmd(containerId).exec(); + LOGGER.debug("{} started container {}", this, containerId); + long duration = System.currentTimeMillis() - startTime; + if (duration > getDurationWarningThreshold()) { + LOGGER.warn("{} start {} took {}ms", this, containerId, duration); + } + } catch (RuntimeException e) { + String message = String.format("%s error while starting container %s", this, containerId); + throw new RuntimeException(message, e); + } + InspectContainerResponse report; + try { + long startTime = System.currentTimeMillis(); + report = client.inspectContainerCmd(containerId).exec(); + long duration = System.currentTimeMillis() - startTime; + if (duration > getDurationWarningThreshold()) { + LOGGER.warn("{} inspect {} took {}ms", this, containerId, duration); + } + } catch (RuntimeException e) { + String message = String.format("%s error while inspecting container %s", this, containerId); + throw new RuntimeException(message, e); + } + InspectContainerResponse.ContainerState state = report.getState(); + LOGGER.debug("{} container {} state is {}", this, containerId, state); + Boolean isRunning = state.getRunning(); + if (isRunning == null || !isRunning) { + String message = String.format("%s cannot attach to container %s because it is %s", + this, containerId, state.getStatus()); + throw new RuntimeException(message); + } + } + + private void removeContainer(DockerClient client, String containerId) { + LOGGER.debug("{} removing container {}", this, containerId); + try { + long startTime = System.currentTimeMillis(); + client.removeContainerCmd(containerId) + .withForce(true) + .exec(); + long duration = System.currentTimeMillis() - startTime; + if (duration > getDurationWarningThreshold()) { + LOGGER.warn("{} remove {} took {}ms", this, containerId, duration); + } + LOGGER.debug("{} removed container {}", this, containerId); + } catch (Exception e) { + String message = String.format("%s error while removing container %s", this, containerId); + LOGGER.warn(message, e); + } + try { + client.close(); + LOGGER.debug("{} closed client", this); + } catch (IOException e) { + String message = String.format("%s error while closing client", this); + LOGGER.warn(message, e); + } + } + + private void setUser(CreateContainerCmd command) { + String user = config.getUser(); + if (user != null) { + command.withUser(user); + } + } + + private void setVolumes(CreateContainerCmd command) { + List binds = config.getVolumes().entrySet().stream() + .map(e -> new Bind(e.getKey(), new Volume(e.getValue()))) + .collect(toList()); + List volumes = binds.stream() + .map(Bind::getVolume) + .collect(toList()); + command.withVolumes(volumes); + command.getHostConfig().withBinds(binds); + } + + @Override + public String toString() { + return "DockerProcessLauncher{" + config.getName() + "}"; + } + + private static List asStrings(Map map) { + return map.entrySet().stream() + .map(e -> e.getKey() + "=" + e.getValue()) + .collect(toList()); + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/WildcardBindingInetAddressFactory.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/WildcardBindingInetAddressFactory.java new file mode 100644 index 000000000000..b1fdad9c24d6 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/dockerized/WildcardBindingInetAddressFactory.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.dockerized; + +import java.net.InetAddress; + +import org.gradle.internal.remote.internal.inet.InetAddressFactory; + +/** + * Overrides Gradle's standard {@link InetAddressFactory} to report the host's wildcard address as + * the factory's local binding address. + */ +class WildcardBindingInetAddressFactory extends InetAddressFactory { + @Override + public InetAddress getLocalBindingAddress() { + return super.getWildcardBindingAddress(); + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/isolation/WorkingDirectoryIsolator.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/isolation/WorkingDirectoryIsolator.java new file mode 100644 index 000000000000..0a892fb872c2 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/isolation/WorkingDirectoryIsolator.java @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.isolation; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.List; +import java.util.OptionalInt; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Consumer; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.IntStream; + +import org.gradle.api.UncheckedIOException; + +public class WorkingDirectoryIsolator implements Consumer { + private static final AtomicInteger WORKER_ID = new AtomicInteger(); + private static final Pattern GRADLE_WORKER_CLASSPATH_FILE_PATTERN = + Pattern.compile("^@.*gradle-worker-classpath.*txt$"); + private static final String PROPERTIES_FILE_NAME = "gemfire.properties"; + + /** + * Each test task gives all of its test workers the same working directory. Because + * Geode tests cannot tolerate this when run in parallel, we give each test worker its + * own unique working directory. + */ + @Override + public void accept(ProcessBuilder processBuilder) { + String subdirectory = String.format("test-worker-%06d", WORKER_ID.getAndIncrement()); + Path originalWorkingDirectory = processBuilder.directory().toPath(); + Path newWorkingDirectory = originalWorkingDirectory.resolve(subdirectory); + + try { + Files.createDirectories(newWorkingDirectory); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + processBuilder.directory(newWorkingDirectory.toFile()); + + Path originalPropertiesFile = originalWorkingDirectory.resolve(PROPERTIES_FILE_NAME); + if (Files.exists(originalPropertiesFile)) { + Path newPropertiesFile = newWorkingDirectory.resolve(PROPERTIES_FILE_NAME); + copy(originalPropertiesFile, newPropertiesFile); + } + + // If the command specifies a gradle worker classpath file that exists, copy the file to the + // unique working directory and update the command line argument to refer to the new location. + List command = processBuilder.command(); + findGradleWorkerClasspathArg(command) + .ifPresent(i -> updateGradleWorkerClasspathFile(command, i, newWorkingDirectory)); + } + + private void updateGradleWorkerClasspathFile(List command, int argIndex, Path directory) { + String originalClasspathFileArg = command.get(argIndex); + Matcher matcher = GRADLE_WORKER_CLASSPATH_FILE_PATTERN + .matcher(originalClasspathFileArg); + matcher.matches(); + Path originalClasspathFile = Paths.get(matcher.group().substring(1)); + Path newClasspathFile = directory.resolve("gradle-worker-classpath.txt"); + copy(originalClasspathFile, newClasspathFile); + String newClasspathFileArg = "@" + newClasspathFile.toString(); + command.set(argIndex, newClasspathFileArg); + } + + private static void copy(Path source, Path dest) { + try { + Files.copy(source, dest); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } + + private static OptionalInt findGradleWorkerClasspathArg(List command) { + return IntStream.range(0, command.size()) + .filter(i -> GRADLE_WORKER_CLASSPATH_FILE_PATTERN.matcher(command.get(i)).matches()) + .findFirst(); + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/AdjustableProcessLauncher.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/AdjustableProcessLauncher.java new file mode 100644 index 000000000000..4c6753d7f547 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/AdjustableProcessLauncher.java @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.process; + +import java.io.IOException; +import java.util.function.Consumer; + +import org.gradle.api.UncheckedIOException; + +/** + * A process launcher that applies an adjustment to the {@link ProcessBuilder} before launching the + * process. + */ +public class AdjustableProcessLauncher implements ProcessLauncher { + private final Consumer adjustment; + + public AdjustableProcessLauncher(Consumer adjustment) { + this.adjustment = adjustment; + } + + @Override + public Process start(ProcessBuilder processBuilder) { + adjustment.accept(processBuilder); + try { + return processBuilder.start(); + } catch (IOException e) { + throw new UncheckedIOException("Cannot launch process", e); + } + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessBuilder.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessBuilder.java new file mode 100644 index 000000000000..a18f087b445b --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessBuilder.java @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.process; + +import static org.apache.geode.gradle.testing.process.Reflection.getField; +import static org.apache.geode.gradle.testing.process.Reflection.getFieldValue; +import static org.apache.geode.gradle.testing.process.Reflection.setFieldValue; + +import java.io.File; +import java.lang.reflect.InvocationHandler; +import java.lang.reflect.Proxy; +import java.net.URL; +import java.util.List; +import java.util.Set; + +import org.gradle.api.Action; +import org.gradle.api.logging.LogLevel; +import org.gradle.process.internal.JavaExecHandleBuilder; +import org.gradle.process.internal.worker.WorkerProcess; +import org.gradle.process.internal.worker.WorkerProcessBuilder; +import org.gradle.process.internal.worker.WorkerProcessContext; + +/** + * Wraps a worker process builder to make it use the given process launcher. + */ +public class LauncherProxyWorkerProcessBuilder implements WorkerProcessBuilder { + private final WorkerProcessBuilder delegate; + private final ProcessLauncher processLauncher; + private static Object processLauncherProxy; + + public LauncherProxyWorkerProcessBuilder(WorkerProcessBuilder delegate, + ProcessLauncher processLauncher) { + this.delegate = delegate; + this.processLauncher = processLauncher; + } + + @Override + public WorkerProcessBuilder applicationClasspath(Iterable files) { + return delegate.applicationClasspath(files); + } + + @Override + public Set getApplicationClasspath() { + return delegate.getApplicationClasspath(); + } + + @Override + public WorkerProcessBuilder applicationModulePath(Iterable files) { + return delegate.applicationModulePath(files); + } + + @Override + public Set getApplicationModulePath() { + return delegate.getApplicationModulePath(); + } + + @Override + public WorkerProcessBuilder setBaseName(String baseName) { + return delegate.setBaseName(baseName); + } + + @Override + public String getBaseName() { + return delegate.getBaseName(); + } + + @Override + public WorkerProcessBuilder setLogLevel(LogLevel logLevel) { + return delegate.setLogLevel(logLevel); + } + + @Override + public WorkerProcessBuilder sharedPackages(Iterable packages) { + return delegate.sharedPackages(packages); + } + + @Override + public Set getSharedPackages() { + return delegate.getSharedPackages(); + } + + @Override + public JavaExecHandleBuilder getJavaCommand() { + return delegate.getJavaCommand(); + } + + @Override + public LogLevel getLogLevel() { + return delegate.getLogLevel(); + } + + @Override + public WorkerProcessBuilder sharedPackages(String... packages) { + return delegate.sharedPackages(packages); + } + + @Override + public Action getWorker() { + return delegate.getWorker(); + } + + @Override + public void setImplementationClasspath(List implementationClasspath) { + delegate.setImplementationClasspath(implementationClasspath); + } + + @Override + public void setImplementationModulePath(List implementationModulePath) { + delegate.setImplementationModulePath(implementationModulePath); + } + + @Override + public void enableJvmMemoryInfoPublishing(boolean shouldPublish) { + delegate.enableJvmMemoryInfoPublishing(shouldPublish); + } + + /** + * Replaces the standard worker process's process launcher with this builder's launcher. + */ + @Override + public WorkerProcess build() { + WorkerProcess workerProcess = delegate.build(); + Object workerProcessDelegate = getFieldValue(workerProcess, "delegate"); + Object execHandle = getFieldValue(workerProcessDelegate, "execHandle"); + Class processLauncherType = getField(execHandle, "processLauncher").getType(); + setFieldValue(execHandle, "processLauncher", assignableProcessLauncher(processLauncherType)); + return workerProcess; + } + + /** + * Because the exec handle created by Gradle uses a classloader different from ours, we can't + * simply construct a Gradle {@code ProcessLauncher} to assign. Instead we create proxy, using the + * exec handle's classloader. + */ + private synchronized Object assignableProcessLauncher(Class requiredType) { + if (processLauncherProxy == null) { + // Assume that only start() will be called, and simply delegate to this builder's launcher + InvocationHandler handler = + (proxy, method, args) -> processLauncher.start((ProcessBuilder) args[0]); + ClassLoader classLoader = requiredType.getClassLoader(); + Class[] interfaces = {requiredType}; + processLauncherProxy = Proxy.newProxyInstance(classLoader, interfaces, handler); + } + return processLauncherProxy; + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessFactory.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessFactory.java new file mode 100644 index 000000000000..fcee360ad19f --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/LauncherProxyWorkerProcessFactory.java @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.process; + +import java.io.File; + +import org.gradle.api.Action; +import org.gradle.api.internal.ClassPathRegistry; +import org.gradle.api.internal.file.TemporaryFileProvider; +import org.gradle.api.logging.LoggingManager; +import org.gradle.internal.id.IdGenerator; +import org.gradle.internal.jvm.inspection.JvmVersionDetector; +import org.gradle.internal.logging.events.OutputEventListener; +import org.gradle.internal.remote.MessagingServer; +import org.gradle.process.internal.JavaExecHandleFactory; +import org.gradle.process.internal.health.memory.MemoryManager; +import org.gradle.process.internal.worker.DefaultWorkerProcessFactory; +import org.gradle.process.internal.worker.WorkerProcessBuilder; +import org.gradle.process.internal.worker.WorkerProcessContext; + +/** + * Overrides Gradle's {@link DefaultWorkerProcessFactory} to return an {@link WorkerProcessBuilder} + * that uses the given {@link ProcessLauncher} to launch worker processes. + */ +public class LauncherProxyWorkerProcessFactory extends DefaultWorkerProcessFactory { + private final ProcessLauncher processLauncher; + + public LauncherProxyWorkerProcessFactory(LoggingManager loggingManager, + MessagingServer server, ClassPathRegistry classPathRegistry, + IdGenerator idGenerator, File gradleUserHomeDir, + TemporaryFileProvider temporaryFileProvider, + JavaExecHandleFactory execHandleFactory, + JvmVersionDetector jvmVersionDetector, + OutputEventListener outputEventListener, + MemoryManager memoryManager, ProcessLauncher processLauncher) { + super(loggingManager, server, classPathRegistry, idGenerator, gradleUserHomeDir, + temporaryFileProvider, execHandleFactory, jvmVersionDetector, outputEventListener, + memoryManager); + this.processLauncher = processLauncher; + } + + @Override + public WorkerProcessBuilder create(Action workerAction) { + return new LauncherProxyWorkerProcessBuilder(super.create(workerAction), processLauncher); + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/ProcessLauncher.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/ProcessLauncher.java new file mode 100644 index 000000000000..a5adbc5ad873 --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/ProcessLauncher.java @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.process; + +public interface ProcessLauncher { + Process start(ProcessBuilder processBuilder); +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/Reflection.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/Reflection.java new file mode 100644 index 000000000000..18af487dc1dc --- /dev/null +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/process/Reflection.java @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ + +package org.apache.geode.gradle.testing.process; + +import java.lang.reflect.Field; +import java.util.Objects; +import java.util.function.BiFunction; + +/** + * Utility methods to retrieve and set otherwise inaccessible fields via reflection. + */ +public class Reflection { + /** + * Returns a {@link Field} that describes the named field in the given owner. + */ + public static Field getField(Object owner, String fieldName) { + Objects.requireNonNull(owner); + try { + return owner.getClass().getDeclaredField(fieldName); + } catch (NoSuchFieldException e) { + String message = String.format("Getting %s declaration for %s", fieldName, owner); + throw new RuntimeException(message, e); + } + } + + /** + * Returns the value of the named field from the given owner. + */ + public static Object getFieldValue(Object owner, String fieldName) { + return withAccessibleField(owner, fieldName, getValue()); + } + + /** + * Sets the value of the named field in the given owner. + */ + public static void setFieldValue(Object owner, String fieldName, Object value) { + withAccessibleField(owner, fieldName, setValue(value)); + } + + /** + * Makes a field temporarily accessible and applies the operation to it. + */ + private static Object withAccessibleField(Object owner, String fieldName, + BiFunction operation) { + Field field = getField(owner, fieldName); + boolean accessible = field.isAccessible(); + try { + field.setAccessible(true); + return operation.apply(owner, field); + } finally { + field.setAccessible(accessible); + } + } + + /** + * Creates a function that extracts the value of a field from an owner. + */ + private static BiFunction getValue() { + return (owner, field) -> { + try { + return field.get(owner); + } catch (IllegalAccessException | IllegalArgumentException e) { + String message = String.format("Getting %s (%s) value for %s", + field.getName(), field.isAccessible(), owner); + throw new RuntimeException(message, e); + } + }; + } + + /** + * Creates a function that sets a field of an owner to the given value. + */ + private static BiFunction setValue(Object value) { + return (owner, field) -> { + try { + field.set(owner, value); + } catch (IllegalAccessException | IllegalArgumentException e) { + String message = String.format("Setting %s (%s) value for %s", + field.getName(), field.isAccessible(), owner); + throw new RuntimeException(message, e); + } + return null; + }; + } +} diff --git a/buildSrc/src/main/java/org/apache/geode/gradle/OverriddenTestExecutor.java b/buildSrc/src/main/java/org/apache/geode/gradle/testing/repeat/RepeatableTestExecuter.java similarity index 63% rename from buildSrc/src/main/java/org/apache/geode/gradle/OverriddenTestExecutor.java rename to buildSrc/src/main/java/org/apache/geode/gradle/testing/repeat/RepeatableTestExecuter.java index 63434b2983df..1536ea57a276 100644 --- a/buildSrc/src/main/java/org/apache/geode/gradle/OverriddenTestExecutor.java +++ b/buildSrc/src/main/java/org/apache/geode/gradle/testing/repeat/RepeatableTestExecuter.java @@ -12,9 +12,10 @@ * or implied. See the License for the specific language governing permissions and limitations under * the License. */ -package org.apache.geode.gradle; +package org.apache.geode.gradle.testing.repeat; import java.io.File; +import java.util.List; import java.util.Set; import com.google.common.collect.ImmutableSet; @@ -28,6 +29,7 @@ import org.gradle.api.internal.tasks.testing.TestResultProcessor; import org.gradle.api.internal.tasks.testing.WorkerTestClassProcessorFactory; import org.gradle.api.internal.tasks.testing.detection.DefaultTestClassScanner; +import org.gradle.api.internal.tasks.testing.detection.DefaultTestExecuter; import org.gradle.api.internal.tasks.testing.detection.TestFrameworkDetector; import org.gradle.api.internal.tasks.testing.filter.DefaultTestFilter; import org.gradle.api.internal.tasks.testing.processors.MaxNParallelTestClassProcessor; @@ -40,38 +42,44 @@ import org.gradle.api.logging.Logging; import org.gradle.internal.Factory; import org.gradle.internal.actor.ActorFactory; -import org.gradle.internal.operations.BuildOperationExecutor; import org.gradle.internal.time.Clock; import org.gradle.internal.work.WorkerLeaseRegistry; import org.gradle.process.internal.worker.WorkerProcessFactory; /** - * Test executor that is used to replace gradles DefaultTestExecutor and does - * not include a {@link RunPreviousFailedFirstTestClassProcessor} in the processor - * chain. This is used by the RepeatTest task. + * A copy of {@link DefaultTestExecuter} from Gradle v6.8.3, modified to process each test class + * as many times as it was submitted. This is required by our {@link RepeatTest} task, because: + *
      + *
    • Geode's {@code RepeatTest} task operates by submitting each test class for processing + * multiple times.
      • + *
    • Gradle's {@code DefaultTestExecuter} includes a + * {@link RunPreviousFailedFirstTestClassProcessor}, which de-duplicates the submitted test + * classes, preventing the {@code RepeatTest} from repeating the tests.
      • + *
    + *

    + * This class omits the {@code RunPreviousFailedFirstTestClassProcessor}, and so each test class + * is processed as many times as {@code RepeatTest} submits it. See the comment in {@link #execute}. */ -class OverriddenTestExecutor implements TestExecuter { - private static final Logger LOGGER = Logging.getLogger(OverriddenTestExecutor.class); +public class RepeatableTestExecuter implements TestExecuter { + private static final Logger LOGGER = Logging.getLogger(RepeatableTestExecuter.class); private final WorkerProcessFactory workerFactory; private final ActorFactory actorFactory; private final ModuleRegistry moduleRegistry; private final WorkerLeaseRegistry workerLeaseRegistry; - private final BuildOperationExecutor buildOperationExecutor; private final int maxWorkerCount; private final Clock clock; private final DocumentationRegistry documentationRegistry; private final DefaultTestFilter testFilter; private TestClassProcessor processor; - public OverriddenTestExecutor(WorkerProcessFactory workerFactory, ActorFactory actorFactory, ModuleRegistry moduleRegistry, - WorkerLeaseRegistry workerLeaseRegistry, BuildOperationExecutor buildOperationExecutor, int maxWorkerCount, - Clock clock, DocumentationRegistry documentationRegistry, DefaultTestFilter testFilter) { + public RepeatableTestExecuter(WorkerProcessFactory workerFactory, ActorFactory actorFactory, + ModuleRegistry moduleRegistry, WorkerLeaseRegistry workerLeaseRegistry, int maxWorkerCount, + Clock clock, DocumentationRegistry documentationRegistry, DefaultTestFilter testFilter) { this.workerFactory = workerFactory; this.actorFactory = actorFactory; this.moduleRegistry = moduleRegistry; this.workerLeaseRegistry = workerLeaseRegistry; - this.buildOperationExecutor = buildOperationExecutor; this.maxWorkerCount = maxWorkerCount; this.clock = clock; this.documentationRegistry = documentationRegistry; @@ -79,27 +87,42 @@ public OverriddenTestExecutor(WorkerProcessFactory workerFactory, ActorFactory a } @Override - public void execute(final JvmTestExecutionSpec testExecutionSpec, TestResultProcessor testResultProcessor) { + public void execute(final JvmTestExecutionSpec testExecutionSpec, + TestResultProcessor testResultProcessor) { final TestFramework testFramework = testExecutionSpec.getTestFramework(); final WorkerTestClassProcessorFactory testInstanceFactory = testFramework.getProcessorFactory(); - final WorkerLeaseRegistry.WorkerLease currentWorkerLease = workerLeaseRegistry.getCurrentWorkerLease(); + final WorkerLeaseRegistry.WorkerLease + currentWorkerLease = + workerLeaseRegistry.getCurrentWorkerLease(); final Set classpath = ImmutableSet.copyOf(testExecutionSpec.getClasspath()); + final Set modulePath = ImmutableSet.copyOf(testExecutionSpec.getModulePath()); + final List + testWorkerImplementationModules = + testFramework.getTestWorkerImplementationModules(); final Factory forkingProcessorFactory = new Factory() { @Override public TestClassProcessor create() { - return new ForkingTestClassProcessor(currentWorkerLease, workerFactory, testInstanceFactory, testExecutionSpec.getJavaForkOptions(), - classpath, testFramework.getWorkerConfigurationAction(), moduleRegistry, documentationRegistry); - } - }; - final Factory reforkingProcessorFactory = new Factory() { - @Override - public TestClassProcessor create() { - return new RestartEveryNTestClassProcessor(forkingProcessorFactory, testExecutionSpec.getForkEvery()); + return new ForkingTestClassProcessor(currentWorkerLease, workerFactory, testInstanceFactory, + testExecutionSpec.getJavaForkOptions(), + classpath, modulePath, testWorkerImplementationModules, + testFramework.getWorkerConfigurationAction(), moduleRegistry, documentationRegistry); } }; + final Factory + reforkingProcessorFactory = + new Factory() { + @Override + public TestClassProcessor create() { + return new RestartEveryNTestClassProcessor(forkingProcessorFactory, + testExecutionSpec.getForkEvery()); + } + }; + // Create the chain of test class processors, omitting the + // RunPreviousFailedFirstTestClassProcessor that Gradle's DefaultTestExecuter creates. processor = new PatternMatchTestClassProcessor(testFilter, - new MaxNParallelTestClassProcessor(getMaxParallelForks(testExecutionSpec), reforkingProcessorFactory, actorFactory)); + new MaxNParallelTestClassProcessor(getMaxParallelForks(testExecutionSpec), + reforkingProcessorFactory, actorFactory)); final FileTree testClassFiles = testExecutionSpec.getCandidateClassFiles(); @@ -113,9 +136,8 @@ public TestClassProcessor create() { detector = new DefaultTestClassScanner(testClassFiles, null, processor); } - final Object testTaskOperationId = buildOperationExecutor.getCurrentOperation().getParentId(); - - new TestMainAction(detector, processor, testResultProcessor, clock, testTaskOperationId, testExecutionSpec.getPath(), "Gradle Test Run " + testExecutionSpec.getIdentityPath()).run(); + new TestMainAction(detector, processor, testResultProcessor, clock, testExecutionSpec.getPath(), + "Gradle Test Run " + testExecutionSpec.getIdentityPath()).run(); } @Override @@ -128,7 +150,8 @@ public void stopNow() { private int getMaxParallelForks(JvmTestExecutionSpec testExecutionSpec) { int maxParallelForks = testExecutionSpec.getMaxParallelForks(); if (maxParallelForks > maxWorkerCount) { - LOGGER.info("{}.maxParallelForks ({}) is larger than max-workers ({}), forcing it to {}", testExecutionSpec.getPath(), maxParallelForks, maxWorkerCount, maxWorkerCount); + LOGGER.info("{}.maxParallelForks ({}) is larger than max-workers ({}), forcing it to {}", + testExecutionSpec.getPath(), maxParallelForks, maxWorkerCount, maxWorkerCount); maxParallelForks = maxWorkerCount; } return maxParallelForks; diff --git a/buildSrc/src/main/resources/META-INF/gradle-plugins/com.github.pedjak.dockerized-test.properties b/buildSrc/src/main/resources/META-INF/gradle-plugins/com.github.pedjak.dockerized-test.properties deleted file mode 100644 index 1cfe2cb87aab..000000000000 --- a/buildSrc/src/main/resources/META-INF/gradle-plugins/com.github.pedjak.dockerized-test.properties +++ /dev/null @@ -1 +0,0 @@ -implementation-class = com.pedjak.gradle.plugins.dockerizedtest.DockerizedTestPlugin \ No newline at end of file diff --git a/buildSrc/src/main/resources/META-INF/gradle-plugins/geode-dockerized-test.properties b/buildSrc/src/main/resources/META-INF/gradle-plugins/geode-dockerized-test.properties new file mode 100644 index 000000000000..1ea705cb1680 --- /dev/null +++ b/buildSrc/src/main/resources/META-INF/gradle-plugins/geode-dockerized-test.properties @@ -0,0 +1 @@ +implementation-class = org.apache.geode.gradle.testing.dockerized.DockerizedTestPlugin \ No newline at end of file diff --git a/ci/scripts/execute_build.sh b/ci/scripts/execute_build.sh index ebb7ed92687d..cd91a9a42f16 100755 --- a/ci/scripts/execute_build.sh +++ b/ci/scripts/execute_build.sh @@ -77,7 +77,7 @@ scp ${SSH_OPTIONS} ${SCRIPTDIR}/capture-call-stacks.sh geode@${INSTANCE_IP_ADDRE if [[ -n "${PARALLEL_DUNIT}" && "${PARALLEL_DUNIT}" == "true" ]]; then PARALLEL_DUNIT="-PparallelDunit -PdunitDockerUser=geode" if [ -n "${DUNIT_PARALLEL_FORKS}" ]; then - DUNIT_PARALLEL_FORKS="-PdunitParallelForks=${DUNIT_PARALLEL_FORKS} --max-workers=${DUNIT_PARALLEL_FORKS} -PtestMaxParallelForks=${DUNIT_PARALLEL_FORKS}" + DUNIT_PARALLEL_FORKS="--max-workers=${DUNIT_PARALLEL_FORKS} -PtestMaxParallelForks=${DUNIT_PARALLEL_FORKS}" fi else PARALLEL_DUNIT="" diff --git a/ci/scripts/execute_tests.sh b/ci/scripts/execute_tests.sh index 52148c79e562..916cc305f672 100755 --- a/ci/scripts/execute_tests.sh +++ b/ci/scripts/execute_tests.sh @@ -64,7 +64,7 @@ scp ${SSH_OPTIONS} ${SCRIPTDIR}/capture-call-stacks.sh geode@${INSTANCE_IP_ADDRE if [[ -n "${PARALLEL_DUNIT}" && "${PARALLEL_DUNIT}" == "true" ]]; then PARALLEL_DUNIT="-PparallelDunit -PdunitDockerUser=geode -PdunitDockerImage=\$(docker images --format '{{.Repository}}:{{.Tag}}')" if [ -n "${DUNIT_PARALLEL_FORKS}" ]; then - DUNIT_PARALLEL_FORKS="-PdunitParallelForks=${DUNIT_PARALLEL_FORKS} --max-workers=${DUNIT_PARALLEL_FORKS} -PtestMaxParallelForks=${DUNIT_PARALLEL_FORKS}" + DUNIT_PARALLEL_FORKS="--max-workers=${DUNIT_PARALLEL_FORKS} -PtestMaxParallelForks=${DUNIT_PARALLEL_FORKS}" fi else PARALLEL_DUNIT="" diff --git a/etc/apache-copyright-notice.txt b/etc/apache-copyright-notice.txt new file mode 100644 index 000000000000..58fd5efd3dac --- /dev/null +++ b/etc/apache-copyright-notice.txt @@ -0,0 +1,12 @@ +Licensed to the Apache Software Foundation (ASF) under one or more contributor license +agreements. See the NOTICE file distributed with this work for additional information regarding +copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance with the License. You may obtain a +copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software distributed under the License +is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +or implied. See the License for the specific language governing permissions and limitations under +the License. \ No newline at end of file diff --git a/etc/intellij-apache-copyright-notice.xml b/etc/intellij-apache-copyright-notice.xml new file mode 100644 index 000000000000..940ad4fd6ec0 --- /dev/null +++ b/etc/intellij-apache-copyright-notice.xml @@ -0,0 +1,6 @@ + + + + \ No newline at end of file diff --git a/geode-assembly/build.gradle b/geode-assembly/build.gradle index 3919cbb7a317..a92d8735cd4b 100755 --- a/geode-assembly/build.gradle +++ b/geode-assembly/build.gradle @@ -125,14 +125,16 @@ artifacts { compositeTarget distTar } -//This "repository" only exists to download tomcat-6, because the zip for tomcat 6 is -//not in a maven repo. Later versions of tomcat are. repositories { + //This "repository" only exists to download tomcat-6, because the zip for tomcat 6 is + //not in a maven repo. Later versions of tomcat are. ivy { url 'https://archive.apache.org/' patternLayout { artifact '/dist/tomcat/tomcat-6/v6.0.37/bin/[organisation]-[module]-[revision].[ext]' } + // Infer the metadata from the presence of the artifact + metadataSources { artifact() } } // For gradle tooling dependencies maven { diff --git a/gradle.properties b/gradle.properties index a6c23cd904b9..72924f6b682c 100755 --- a/gradle.properties +++ b/gradle.properties @@ -47,7 +47,7 @@ buildId = 0 productName = Apache Geode productOrg = Apache Software Foundation (ASF) -minimumGradleVersion = 5.5 +minimumGradleVersion = 6.8 # Set this on the command line with -P or in ~/.gradle/gradle.properties # to change the buildDir location. Use an absolute path. buildRoot= @@ -55,8 +55,6 @@ buildRoot= # We want signing to be on by default. Signing requires GPG to be set up. nexusSignArchives = true -# Control how many concurrent dunit (using docker) tests will be run -dunitParallelForks = 8 # This is the name of the Docker image for running parallel dunits dunitDockerImage = apachegeode/geode-build # Docker user for parallel dunit tests diff --git a/gradle/docker.gradle b/gradle/docker.gradle deleted file mode 100644 index 77541fd80a19..000000000000 --- a/gradle/docker.gradle +++ /dev/null @@ -1,149 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* - * Configuration for running (dunit) tests in parallel in Docker containers. - * The container used must hava JAVA_HOME set in it's environment and must - * have 'java' defined on the path. For example, the relevant Dockerfile - * content could be: - * - * ENV JAVA_HOME=/opt/jdk1.8.0_u101 - * ENV PATH=$PATH:$JAVA_HOME/bin - * - * In addition, the container must have docker installed. - * - * The plugin can be activated with the Gradle property 'parallelDunit'. - * Additional properties that can be set are: - * - * dunitDockerImage - The docker image used for running parallel dunits. The - * default image is 'bellsoft/liberica-openjdk-debian:8'. The image is required to - * have 'JAVA_HOME' set as an environment variable. - * dunitParallelForks - The number of parallel containers that will be - * launched. The default is 8. - * dunitDockerUser - The user used within the docker container to run tests. - * The default is 'root'. - */ - -apply plugin: 'com.github.pedjak.dockerized-test' - - -if (project.hasProperty('parallelDunit')) { - def pwd = System.getenv('PWD') - def geodeDir = new File(pwd).getCanonicalPath() - project.ext.dunitDockerVolumes = ["${geodeDir}":geodeDir] -} - -project.ext.dockerConfig = { - maxParallelForks = dunitParallelForks.toInteger() - - docker { - // base image for creating docker containers that execute the tests - image = dunitDockerImage - - // volumes mounted to the containers - // in a form: host_dir : container_dir - def gradleHome = System.getenv('GRADLE_USER_HOME') ?: "${System.getenv('HOME')}/.gradle" - volumes = ["${gradleHome}":gradleHome] - - // Add volumes configured by top-level build script - volumes << project.dunitDockerVolumes - - // specify the user for starting Gradle test worker within the container. - user = dunitDockerUser - - beforeContainerCreate = { cmd, client -> - def javaHomeIdx = -1 - def pathIdx = -1 - def tmpEnv = [] - cmd.getEnv().each { tmpEnv << it } - - tmpEnv.eachWithIndex { x, j -> - if (x.startsWith('JAVA_HOME')) { - javaHomeIdx = j - } - if (x.startsWith('PATH')) { - pathIdx = j - } - } - - // Remove JAVA_HOME and PATH env variables - they might not be the same as the container needs - if (javaHomeIdx >= 0) { - tmpEnv[javaHomeIdx] = 'JAVA_HOME_REMOVED=' - } - if (pathIdx >= 0) { - tmpEnv[pathIdx] = 'PATH_REMOVED=' - } - - if (project.hasProperty('testJVM') && !testJVM.trim().isEmpty()) { - // Docker command is just 'java' so set to full path - tmpEnv << ("JAVA_HOME=${project.testJVM}" as String) - } - - - // Unfortunately this snippet of code is here and is required by dev-tools/docker/base/entrypoint.sh. - // This allows preserving the outer user inside the running container. Required for Jenkins - // and other environments. There doesn't seem to be a way to pass this environment variable - // in from a Jenkins Gradle job. - if (System.env['LOCAL_USER_ID'] == null) { - def username = System.getProperty("user.name") - def uid = ['id', '-u', username].execute().text.trim() - tmpEnv << ("LOCAL_USER_ID=${uid}" as String) - } - - cmd.withEnv(tmpEnv) - - // Infer the index of this invocation - def cmdList = cmd.getCmd() - - if (project.hasProperty('testJVM') && !testJVM.trim().isEmpty()) { - // Docker command is just 'java' so set to full path - cmdList[0] = ("${project.testJVM}/bin/java" as String) - } - - // copy the classpath file to the working dir - def classPathFileIndex = cmdList.findIndexOf { it =~ /^@.*gradle-worker-classpath.*txt$/ } - if (classPathFileIndex > 0) { - def dst = new File(cmd.getWorkingDir(), "gradle-worker-classpath.txt") - if (!dst.exists()) { - def src = new File(cmdList[classPathFileIndex].substring(1)) - dst.write(src.text) - } - cmdList[classPathFileIndex] = '@'+dst.toString() - } - - //println cmd - } - } -} - - -if (project.hasProperty('parallelDunit')) { - uiTest.configure(project.ext.dockerConfig) - repeatUnitTest.configure(project.ext.dockerConfig) - - integrationTest.configure(project.ext.dockerConfig) - repeatIntegrationTest.configure(project.ext.dockerConfig) - - distributedTest.configure(project.ext.dockerConfig) - repeatDistributedTest.configure(project.ext.dockerConfig) - - upgradeTest.configure(project.ext.dockerConfig) - repeatUpgradeTest.configure(project.ext.dockerConfig) - - acceptanceTest.configure(project.ext.dockerConfig) - repeatAcceptanceTest.configure(project.ext.dockerConfig) -} diff --git a/gradle/multi-process-test.gradle b/gradle/multi-process-test.gradle new file mode 100644 index 000000000000..90144a22216c --- /dev/null +++ b/gradle/multi-process-test.gradle @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * Configuration for running multi-process tests in parallel. See the 'multiProcessTestTasks' + * variable for the list of multi-process test tasks. + * + * To run multi-process tests in Docker, set the Gradle property 'parallelDunit'. + * The container used must have JAVA_HOME set in its environment and must + * have 'java' defined on the path. For example, the relevant Dockerfile + * content could be: + * + * ENV JAVA_HOME=/opt/jdk1.8.0_u101 + * ENV PATH=$PATH:$JAVA_HOME/bin + * + * In addition, the container must have docker installed. + * + * Additional configuration properties for running multi-process tests in Docker are: + * + * dunitDockerImage - The docker image used for running parallel dunits. The default image is + * 'bellsoft/liberica-openjdk-debian:8'. The image is required to have + * 'JAVA_HOME' set as an environment variable. + * dunitDockerUser - The user used within the docker container to run tests. + * The default is 'root'. + * dunitDockerVolumes - Docker volumes to mount in the docker container in addition to the ones + * mounted by the plugin. The plugin always mounts the directory of the + * top-level Geode project and the current user's Gradle home directory. + * dunitDockerJVM - The JVM to use to launch test worker processes inside Docker. The default + * is the value of the testJVM property if that property is set. + * + * The following properties apply to multi-process tests, whether running in Docker or not: + * + * --max-workers The maximum number of workers processes for Gradle to run in parallel. + * (Note that Gradle applies this value to all tasks, not just test tasks.) + * + * testMaxParallelForks The maximum number of tests for each multi-process test task to process in + * parallel. If 'parallelDunit' us defined, `testMaxParallelForks` defaults to + * 1/4 of the machine's available processors. If 'parallelDunit' is not + * defined, the default 'testMaxParallelForks' is 1. (Note that test.gradle + * also applies this property to unit test tasks, with different defaults.) + */ + +import org.apache.geode.gradle.testing.Executers +import org.apache.geode.gradle.testing.isolation.WorkingDirectoryIsolator + +def multiProcessTestTasks = [acceptanceTest, repeatAcceptanceTest, + distributedTest, repeatDistributedTest, + integrationTest, repeatIntegrationTest, + upgradeTest, repeatUpgradeTest, + uiTest, repeatUnitTest] + +if (project.hasProperty('parallelDunit')) { + def parallelForks = project.hasProperty('testMaxParallelForks') + ? Integer.parseUnsignedInt(project.testMaxParallelForks) + : Runtime.runtime.availableProcessors().intdiv(4) ?: 1 + for (task in multiProcessTestTasks) { + task.maxParallelForks = parallelForks + } + apply plugin: 'geode-dockerized-test' +} else { + for (task in multiProcessTestTasks) { + if (project.hasProperty('testMaxParallelForks')) { + task.maxParallelForks = Integer.parseUnsignedInt(project.testMaxParallelForks) + } + task.doFirst { + testExecuter = Executers.withAdjustment(it, new WorkingDirectoryIsolator()) + } + } +} diff --git a/gradle/standard-subproject-configuration.gradle b/gradle/standard-subproject-configuration.gradle index 49916eb65992..e3e944a87ca8 100644 --- a/gradle/standard-subproject-configuration.gradle +++ b/gradle/standard-subproject-configuration.gradle @@ -19,7 +19,7 @@ apply from: "${rootDir}/${scriptDir}/java.gradle" apply from: "${rootDir}/${scriptDir}/dependency-resolution.gradle" apply from: "${rootDir}/${scriptDir}/test.gradle" apply from: "${rootDir}/${scriptDir}/code-analysis.gradle" -apply from: "${rootDir}/${scriptDir}/docker.gradle" +apply from: "${rootDir}/${scriptDir}/multi-process-test.gradle" apply from: "${rootDir}/${scriptDir}/spotless.gradle" apply from: "${rootDir}/${scriptDir}/ide.gradle" apply plugin: 'com.github.ben-manes.versions' diff --git a/gradle/test.gradle b/gradle/test.gradle index e601f5c28872..7c0fb78cac98 100644 --- a/gradle/test.gradle +++ b/gradle/test.gradle @@ -1,7 +1,7 @@ import org.apache.geode.gradle.TestPropertiesWriter -import org.apache.geode.gradle.RepeatTest + import org.apache.geode.gradle.plugins.DependencyConstraints -import org.apache.geode.gradle.RunInSubdirectoryTestFramework +import org.apache.geode.gradle.testing.repeat.RepeatTest /* * Licensed to the Apache Software Foundation (ASF) under one or more @@ -41,6 +41,7 @@ test { } } + apply plugin: 'nebula.facet' facets { integrationTest { @@ -139,17 +140,6 @@ configure([integrationTest, distributedTest, performanceTest, acceptanceTest, ui } } -configure([acceptanceTest, distributedTest, integrationTest, uiTest, upgradeTest, repeatAcceptanceTest, repeatDistributedTest, repeatIntegrationTest, repeatUnitTest, repeatUpgradeTest]) { - doFirst { - // Wrap the task's test framework in a wrapper that runs each test worker JVM in a unique - // subdirectory. - def subdirFramework = new RunInSubdirectoryTestFramework(testFramework) - // This call works for now, but the Test class declares useTestFramework() as protected, so - // this could become troublesome in a future version of Gradle/Groovy. - useTestFramework subdirFramework - } -} - configure([repeatDistributedTest, repeatIntegrationTest, repeatUpgradeTest, repeatUnitTest, repeatAcceptanceTest]) { times = Integer.parseInt(repeat) useJUnit {} @@ -212,7 +202,7 @@ gradle.taskGraph.whenReady({ graph -> // The distributed tests seem to need to use /tmp directly, // so exclude them from using the supplied temp directory. - if (!test.name.contains("distributed")) { + if (!test.name.contains("distributed") && !test.name.contains("repeatDistributed")) { systemProperty 'java.io.tmpdir', System.getProperty('java.io.tmpdir') } @@ -238,7 +228,7 @@ gradle.taskGraph.whenReady({ graph -> acceptanceTest { // Acceptance tests may reach out and run a gfsh command from the assembled gfsh. // If the test JVM version is specified, we must have the correct JAVA_HOME set for gfsh to use. - // See also environment configuration for parallel testing in docker.gradle + // See also environment configuration for parallel testing in multi-process-test.gradle if (project.hasProperty('testJVM') && !testJVM.trim().isEmpty()) { environment "JAVA_HOME", "${project.testJVM}" } diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar index 87b738cbd051603d91cc39de6cb000dd98fe6b02..e708b1c023ec8b20f512888fe07c5bd3ff77bb8f 100644 GIT binary patch delta 26193 zcmZ6yQ+S|Vu&o>0wrv|7+qP{xU&pp>+fK)}ZKK1EI!T}Z?6uCtKF>3+>b7Q$8a3;k z=?&n+bKs5iponRd%jI35ARxHlARx>siR4%*il7((1uK)8y@{J!oa(gW@(&EbVLvhOxk#E0z~5*_n$Tfk4BT1W zh~4H^yI$w!jrIW$@8~{|r_Pqh9?;*1{Rs-h$o?FVSot<3yKX_cH33Wqgy&Ugow#-- zd$AFKpvAm7vspRndDP5Y*{X$rg0EvCe9(Ow>lBeyGY!V@xQpXomHjCWwMA-rDRPSv zY@gq7;|J=t%N|R7799odG(V`K@OXpjH2q11r=-spt+w zmng+yzxXW&2lk&G)N z-h)16N@V9eFCNeZrbSiuE44@1#kS@h5wqX)wgpxfDiThLBx=5wyP_q&uT8OO)g!i} z`ony65!)LBh`yjIhNFW{%5vZka3CNsFd!fxA|N)P3^l}%ARrX~g&6-g&ji4>8oCzF zKSH<7MutdMx~SkLQ5g_)<~Gen%{ZC`NJdbH)-9$<(ppE)OUsf4+q=3xf!CmpZ`c>g z4Ys!B49{{P<@lMuM@Gi9cVK3-W&h8s0rx+luP@f0C2um4An0s{!;rApVwtHdlxBE$ zQ}-fiEaWDdk_Z{*`eS})9~03LOH#dsT^&c)G^Img%Th&3)4?O4Wq0Y&WfVC|VC zaAQwCL*mCoU593J>8NRWao@R|t#I*_etEAkj;Pnx3Px?Zvdq$zVwkChuPxdoUTJ_f z91pe6B-4rYiTj&D4-yF14*5BofspUDM3g|`#ZB$#P?p~-U9+h+Tp`mRxh*=&=&T*z2q>oXsGC%dWpshF`TNHT@_F>BTVa9G zFHYaGE;{c#SSHF=YdV#s&=@2HPVty?i1KRKnHMZOfiniPe%}^B8HFW z*bJMvi!j9I(^KSPI$-KTx8V>{A^SDH?XXHk2r zl3KMvKg}3t6q5GEn_a9c#&WLtG*Z+HlGMW<(x+DV$pO_9MbEmMd(|gjX$n1RcRRtv zDT~=Ns?nTUr{?a8bL%J`yhpBI_HJ=#J82CYL&{%*ODAh3aQ7C!&9@kuw2vTv^C5Jv zw{IBtth8_1lePLj%=eb2<|{r*Tr*eZc-3zAmND82-Y2(LCRJAYureb+>#gbkwS;}t z*dnlaO#wXPiqx378QPITyhU~VJF;i9bY&O>9`a<{Z=?tck-b5V=Ao(5iSinj^|JG0 z^Mei%8;vfT5yjl03`{X#`7o70WDu59Odv`qB&Kd4C_bQnCkPF-Z;JgZ^g{2~-?;k+ z9+d0E@fT(gQ5tY0OC?v8n)9lJ>xx!1rO|MACzpKq&HkF1K{ zFMRybn|7SN@1S0)$Tgnp%zhGmU0>jDj(qPx9cd6`;(p;9lq7up`hoX{FkphHyA0DN z5*0_8Cr`bKX4N+FM+NK)D;S+PQe|0pN`4GS%9VQJ!F@1AmvLj;s* z?5g2wVEo&)=YpSxQkAAjZU5QM2_ajp*;-oX5M*sllctPP$Cq)!W#4miWC{L-|8byZ z^iiy&Xyktx3$vQ_qG0ub{r0Drov-9Lg!p(omOcL5a7e1+=Q3+nuHS2}-`t&-(97AN zF!4V4J;ELv>Nxx#>p@srsIrMn3PoV;Fg0q~y9pFFHz~V?YR;q*bQp&>(1D%FPs zC*IM^c|*^YFd>8K!*CNjO?IyyVo0Oj58J=V+HZAg4EGQ_44PdzlAt7maXilOhlg5a01 zY;8K$cR z`?@05v}ykdXcpLFzwCu_^1Ix5oA^_#4P!q~iRGZaX}VfO>ocM-TIR_OVK|WIk=>*a zeNESLpfO2pCzrnXhvy4zVJJ|)bs@SBYiq&LgqV+kKtj0biu*n~3>Ls(AQ(f9*~_s) zl-KPHr3GJJdZbJ9z~4gno4QuCR6$+sXgnW-SR$UwdY>J(?y@VzOw-!z29BZMAa@*>cOMvu0j;NRm&a zA?dwChkQuKL|!${FHs;*>8zcvo{*&C&2uo)%+*>dx4mwK>fGN+2G`s+E+?FoDMB>o zeEk&R)YUb-)hB0btx0ZnL88NiU<4a4`*dXC&pJay_*!BvzV5L3egZfJ^3pQHC8zFj z6=tLQ9h+!XzlAle1MVUJR85LGy?b&&${lv)c!u?eR_HV5VGs~FMZC*XTEE>nWMNC4 zB-UE7S$o)*022^2SpSi8XR351W?e8S@6i^nRhZK21FxbQo=r|Xn3;a2gqkA9jC)ti z1U4zFR*N)5@{VZmvX8drmAd;H`V9VYPp)`CJ<24J3+oWFOOh|D+JrBTtRrT-UNiS+ z(J%rlp)E0cuK~b2h31^v%*?vFF%$`yaX*844h@8tiqzpL`V|45>?UdyCevybaaT+l zdZ{j2LANKlT=)%J%zpa;hj+KHasV#AP+j%_aV7mNFYy(I_e39m$ld%;hB{CR3NVHM zs?_7ryClg9%EdjJGu)cCVf%af}^xD1`=ey;xTSwvKswhL}f-h0rN zx$a-|8ICEr+~Tj|Q4jJ&D%Al!W{4)^8UYCecR(m4n1Xox7yTqZ!IweY}ynJI3 zMz}L%pqst|R9anw=H*@%l{5QYAlsH-?qK4IOT|U_%?X}+&3)yyoEsv22Yx~r!!S#D zFHjffGZQ^)k^dxFhZ09Hl^#$)1$%|>4LpawxEh+VJLRz_6n%#;LPdOiZ?-?QopPOR@$pf0U=M zotnQ+LX;pV(2mBcz5(Tq6mtI6^{nnb0ZCeqdc2jimiO)`7S5L5d*s~AL(wwRx^y_) zh#GRJ6CxyPN*6Zaw(!3)H6nkpJ5EKdx44c~YYcFRRlLrSK}q>QJLrKU+@6Erj&$a; zqseNY2DlS-f#nv2LUG7?M@oSa$z=|r!z!Vove26#Jt4%MY5?-5EAFbS6yguDs?gCE z(vc>HKlF#duyAd`Ef7JyP%dBGd}$B5LM>{YZJ8-rOT-4~#18)T(~5~bG7eBFL@D*sXp~E2b)#Kj-6#_;rKFTcZ5;~q>XBa%9qtn zh|*NK&npj)_X0e@`Q)Duq;*G(ix)9}{FJ(tBr%|Zl{%`3vm<3;ZTao!@JCy!w$Hd$ zeTsHc-QO?{AjJ$$xyVeeiaa(mIS{g>H~RWDd^7NrcuhG+n6;;IOM;Pnv9AXK%*F>l zh;aH=_~cR-sWJe&{k3#sLEJ9Q;xpFrzQ!1IA($)K6A(HHJ78{YNzs2fS9t(^@rq2; z%#zYDCmxz&BIxr`?~%}btlSLY*nUOqSIu}@IIZ6m+uae;rw{n@(ccR5i+I!DYr1e1 za(MOzHzGaa-+2Qi4lE{yhB?MQdUJSqM$cf$?F?6pb~QsYC}kb`SX9W4V`o%%*{zQJ z64`;gk{^o`6(Of^2y<*)?pbt(p*cCs&QLrs354H|(M+Bl84yFLM>{4$a^t<&uOd2( zDhK}WqM`tq3_L~#0nsJ_0U`b0qJjbbXWp&Th4scC_XtdYXp(dG5kaH82(=)@Kwe1p zNKUs;DyER`6;Dj1)k)SGNDhTGJscIq$m5B>ort=n@wBIQ$t`!x`S0)~<-(*&Y|AE0 z)a`OzqP|LRKT9XHDk!b@B{%*6j6c1Qx`5psOT1Mo>jGwg; zn#*@S7dQbm1brRiPk)Qw!Na~6#2i1!S>M_ZKFrd-N5lZxeU;03j1M7M?Pf4N8dDV?~ChR zl~dm^ZdlcjsW>`r+GpIb96^EutTZQ7HZJ;Ji9;!9fm+SDzseGRa8}V!>`vLfEA5< z^)ZQgMs(z0cl=&{{#-n$H4i7s&F>pR0-jaEn=80*K7h4_Idl}?F4O-j&+iq}!q|0u zW>KXn0n?zIbBoRPGAVPB&EzsF)TGUQQbxPlc&8)*U!LeW9DyE}^H`oU%IDjfiRxVP zcZm6`!=m@f-tdayub+@lecT1jHj$I7CX&YH9$FlZ&!uBZ_-j7{`7B}n<(LR^mFlUm zdPXw&F#ypd4Z0KNnbu6!n+YwuuibHJChS6JgbF%iJ2%OW%yroEX{36{1($2+bB-&K zC0J1^1xfhaH|c}lg(Z}>?KcTy2vs7B`wT%jT2_Co)Dt-(~ z$*7abZzyD0vyDjXlMsm7MuDfEld>A6c;^VEn_#?$SSettv}`*0^ICen24YL)KM7sp>Adr36Z)i zLZ)C~O4l-Gf7j-`wbz!yKgf7>5oT>A5&2stuDEVk(Mb*S$E|S5S*`6RXM&8_DcsVv zCK2jHYi|StkgaGR5AN&9NqFZaeIZ_hgv4iEeNG!)WAE2;tZty`kUh#rx0T{i<$L(pl}iOmk&Jn&`s#jtF66@s#y-I zrs`Ym7<%r)TW;h`wil?x%EZp6y^@4kv#)JSw#ajW_F1RVu}r@b$DP>2UqH&8hkfa) zgPn~tP`HoQzMD8L-$()v073u!X)Q$4`&f7(NRb`EPA>$p&-63+XPOTAUl+O`CVMBv zUrQ*z96r#y(>OjwsSPUDv!=|yZT$0zP51zJCqSw3{3pOd>*?-nTsY6nx@rVd9r#ph z^8SY>H;-e8C;6+TlQz|S9$*y4dPkmczn4KLYdPZkMB5YQKCyx1k zJU4XSsE4Vem=3%L+>&Z1KHvXd;|(^h;3Q!f2m$|_<7NAaA#5;^0T^^w!jQ4QL;le} z*zsm?=EF;Xc)4tMRH8y4kh-rGShhn)F}9Mo8($sHHs?z!aFY1UepV8{ZGt?)QusJ7 zzf~$ngGIL`3tXTAjsF%C+r97MR_g|>aA)^c*OI<*yVs|9XU79XL4qJI26Uk7Ijv3x zoINaggyd9a%t_q%0Ph8Ql54a^ty-n_TBVQcb?WVteFYzbIN`|xuv|=$?1TOt2d7n0 zl5X7BBE4qHiyuUa#im|F_P^ddNQq;Qd~V)7b~RAGgIK3?C-mhqFa+JRkpBJJtsb?Y z62_+%!ETHm0%1mb%*fmFd z-(YSD22!E5B|nk@2o@GOO91cOjR9JiitxXy-!;sEW|(do8;%0P&eP`;44pK7O*MhX zdlfBHfc!vIi~q{+{~Yhf~URiRcY@Pkc?o3SCkQQg}E+y{>J{{cN^5EY^KdbZj0eL(u#c~39>G3r3`=a|y(g9VG ze}!4C?0rHYoy*G-LX{VUGPp;XC>s6yOH$*fi2M)v$zDQ~Z-5?Et!K&4XKhzSA@f~# z2D4gmA=DNBUoxh8%ui`)gXJ{mzO;`Ka7Yro`3THr#>S{ih`Fm&Af_p<$IpA2`> zcde1;fXK1=Y%7#s1`@e0{)Ze>eOIs%ZAx25fAtKS2&3m==&$*)WJQuNTkc$bqb}pE z-|}i=Zhj)B^u#$9ue!%(%F|I3Z>Ea=V?a$?fMBd} ze1q9wcHuXQpXD!fBk-UaRsr3cun@5r2vH(J2asdupCZQXtO!W>%S8LAv!xe0kfu2n zjW5_uuq+)vtGwrR=+l{QqF2kBE78!W(J}GR<%hyOSa^#KW2A9#7?v-EpG}|G-ghyn z=?rspMd0U@OLxiB;Y?ZmX`qtu$BtOjBlV)!Sme@`-#+XmY|ZzS&1tu!IJe(QY_SR1 z0UV=l1SuTQ1Woeb)q8dM)^=&@D0MIY=odw=R}Ix1`s7uYSGi_ZR9?Ypz~{)8DQKuF z;;$|^>WXy8;kVBCj)zk}goNWAo+T`+Eu^`)Rq5;Orj}=OtP(k(_`S?Ia*=lu&i(!Y z6ky!UGfXJmharkF3Z=N zw|spi95hC)Z#|XXFr~?^gLGr^%K#mG*CRRdI!@8olzM+?;<` zmMBkcY<*15Y~<5?on)3RjW#`011xN|mi+U4bC%t&;oTcGP~j?Uw(l_ls= z>}ftPt(>WqU$ooRt&WD#h(kUpTahm$0)a5J z-Z=WZ(i=h)mcpDWFgE-@IdH`}B3S$|R`_jjI00_L3YJ|7a(6Hf0Ikg*&c>H}i8a<* z^1^4vKTOgld+Y*{4;&xunXo&fUk)n(nZ7>(A~0v{k}!HW8=|4a&j(!xARx)

    $|c zhPVR@j{Nk&@@WnWOc#pdglpYbnqRa^G=aT%gEf}qea-{5y=4?s%BNYdE!7oVS$dS-jq9^$`zdUjKX!{5-+8{58s{P6Lviq5&f zzQ9h(E0*q;!DLs~tXi-I!aV7pA;bDB7h>Q7WQv&S-Bq<`n10M5tTIoGCSRX*XE}4X-yKG@ z*&4QX9KS>Qd^uW=HyX#d?b+Z|_#4p&jaPATz^QW2$6y|u52x5h~<_a*4PTBwZbROhRfpidY0V55XOH=QZt*L z?wXbvb}pSCneNmv;npvM38FTjLG&fWle~TLgc1+1AD2J@-r>rMbWHR(3+UZ0jQw8 z5ZgwgvjLFKKD^9oT<5j>3@y0WOxmAzkAX#?$$^3yw3s8prBpU|GV=W*PJEt$__E@r z`Stz2}qfA&J!S5F1Wi7fUqy*vAkIja!>mdAZeJkLWFEIi5V(C}TMXN&@V zj#{qpaJwA^>(sLa%xI%rHzgljPs6aV0K4sn;+ABIp zuQB63*`WGhyF+hsmojX3a<7Z|dh7vbcsGv!>0JUB#7*nn5*_9p6AkHI6Wmdy%>ep) z92}15`S_M@$U7q1>&W2ode_xEfne`?Ttb+ss&eG-$>$fH5bzVZdcs(H6oyFkfkhJ2 zUwY62^V&sX)S&ZfJmNGw;q5^Mk~pP+I3uP&`9a3N8m?f>3PXU5SD2nu=9@r>IfA+J zjjX@)X!vzOM-bidc2qIeh#Q>abJC@x<${icn`KprM)xE;qpkSS z|JucpB7kj2oOa8B-r7Ruh=d?*aqcTYdc@#rq>#Q8xE5LNkn$PR`-Y6+D$~(X5QkUK z3)eq)aPjpu>WI=wtB;d7loeZdCF`1Is64W#Ofa`qEBek>9wxW(>0(*7!kB?7Gwit4 z(Gg=8?0SZc;x|X>@MS+o^cebx=8&<8x2{&j1suO|vFF`5buf&7R|&Qg33jOwySf?< zazki_QODaxRqIi;T}qOw`^~gsa_&6m>$vu7N(aezO#KJ>Q~BF13Rr) z0$7K!w7&oXenjB`WX?|*Va$7e`O34 zLtF`Fb~J02*(;X*%5<(50Y(ZST0aGf0h!-(n8&AzL*F`sMjW4*!1BEp?hbH}9S$7f zO%LbIz}qGL&=^by4)9Dd=%7S6^z|#^5wHVEB6bNV8dB3(#{y+03YSOj$ z1(#|0&|-U;@#j?3YNOn0BbCI6-El=66+Lt{i8c(al1Q8EO3eS}T^m!rrCJVdp@Zzq zAiL(E_PK2Ojap$H%gcM8Dvq220xmznRkY$W41^Al2d-gwOQ{Sffi;Tw#nfQ z!8Ky6Fnp@zRJs|43%>a^AMi-vw|-SlWCSf+Wrc2Sko%DI7J60saN5_Sw>^miD=&7& zN~fxsIWvtH-=lfmgMF6IcJ9%DsG#x2`>pAVb`On1v26P|ynUXy#+;Z5sh_4inhVcjhBn8vLv@?^c7~MeNp3Oc8s!_1aYTuu4%I2R7#UhE zeC1~&Ugb+8uBy5n-UPduT4!hS-A&I-y_EFkn`P7d>6W-+E(UwpwHWW~5!9Fl%rhU8 z4rN-lV^xZE9tstUT+j$FGG_R`{nZSVEdIRh#XZ{{CF~w9=z~HC3Za?3cVIqLV7t-KNDDA(PuVk1EQP z<;!`;CMg(tjBc_oR_9o}$LsR%*(lK3;qJ$NpY> zz2Q%SK2J9HPdVIxM|@u?$Ai{=N3RQx8WS(WUmi`K5tJ9V6@4gz0rWS$C?Nt0Z0;Dp zN_9*dEh%Lz9X_yqMWoXnNtH!zgtATSXNv=2l;<=fNed&L!s?27>;+%8%xsZAJYAB> z6*7-ODl4v5g{=50{n>@`0v& zL!{*eD0MShOB38PGu}00NWP}zz23BV?b-8LU7Sutt0p2PG4|hMlDSr~Ovdo_g{v;R ziPuB~LwSn8jpVxkpS+hQSqS-OwkT{yq&tp9>Jy1O0r@%+1*(LwL13yrR6IKYRG!dJ z5t>vF=!WguGx>zusRaBza|PX_wgeTrm#&$9@d6;7K*6 z3e6;^pCyeEsw#~Bbf^@APG9i1S&_$v!%F-4drg!~g!lGP{ z?>(t@{R9);;EhaY*k_PBL`Z*fk|?BYWDC)YXsX7k=SOzbM2c@%WY{Tj&s*u?F0^dq z3*p4AojqtLL^ifH$GHBRj&%G|gRw}kkvKT^mqiZ^JjXwysjH#3$qGU*++|#Yukw`C z902-@ySHLM;e0%y&tyK0oqj2Qj>Fl+q~(%^o>dC+21Pf1y= z9SBh)|AdSSszzwvjk$C@;k}$FRI3&YX1W~XYVK8jRjUo+ooRFNVsds}z|@6ejWB&H zo{iQiz*G~Az7=mMsx-NwiJzMk=)7XcHKvoRulV<0mw!j(wZ}Np{EvkTM*{&N`#)LVofk!b)(#1unSe7mChznn;^*XGaL302Sf!K%NvK1Sp(SkQr9Q5)QXLWZWWSAO z=$Q}C%616O&ya9od*vm$4d-U_o}94_2TOV^deIt8leMP35r4xTw#k2VqZrON?~xqG zd80SRHXo|9Lp+81ZSG7&FRaS8pQn)X}sTvE9 z8-2y&E;K{W@gb+OsLCsHxpuL%Q*$;yjJFmUTh_VCGv#Nao;^Fzmo4P6+GfOj1srPR zsg<`)HXo#SG|j@XaGO@mRa?jd4EM7ECE3qIQ%*s#tLnCE-zC@}S*5I^><4LN)Jzvs z7(Ovy+fmt|)3Vmq99k((g!1juobDEhgZk`E82CJ8+jRbGN@Cmc)iyuK&pOT6-u^~0 z?zVb&Q{|S|$`FExDI}z6!__sPe1qpFTF6w-Wr~OJY*`zQKHKz^TI+JLzcOfmL{)Kl zJ5Oiy@1!;)n?)1Y0(T3g27L-^-p%(?3VBboNm_z;Qv__0l2J9;Z7)Y*yw&7QoN3rq zqBZ*j|NbhQCiNUnQ@nIM@-fgMU0K1>HcT@Gms;}(PjPmb1ot;MCB9qyB&1@>L5;9_ z{`8ryhVDpDwIcp@l)FzNsW>FSt6C;QVyJ>5H~Ah!hu^|icE~1Z+HKCAyxZ-*5zAut z$@jUliq9rz#UW27(F(*uio;<$`%+wYGOD&dRM0gct-U1syj0&lJ2Uo2%Wf>5W_007 z6|b14{7?m^KqM>Vwo6K|!bYtzJoo%?99+9;POxSx?M4v74^6m#O(70!z!t{^NnSa{CzL$VB8p^=*hcrsN=Y%vG=Y`xK z;HDHPKG5@4AM9YIJ>-Y$kGX?|$WE@lrFjzy{2_S?@}s*(=Mb6lQ+hBV>zewlDzt$1 zjW@99Kp?Q{K+9Wx@c0dA3*K-1-X~Mcv{^=&HSruG_StDpNGhbh=ZF2Jbr1ciGMMs~ z5-fboEUpih8Ct1H?=VuWFkPjX({Vj%D_dwgUaZIg8`{muX_3W9@u$K6;Md_DT>en$ zz>>|KM}>lvlC_%Xrj~nX-p$Er3nje`^H}UaT1&9x!H1muqW%KVQWi7iX~L(xm}O|~ z#X+Z^p7)e&r;?wMZm3UN#ARz&eSB6w5T2aYZXTDbQsB?2PjNoZX_YZm@$&ei*e4@< zA-cu+gUfi6DlKU&8_mLNOcm>F1jm)ZN#O zzmg?7)MT_mL2tU;A-l)smN&w_S2{1pQ=5j1v$uP~ENtMAB$VcrcK=oTxtT$B+CQ@1 z&$A^a(@Ka)^h5oyrh0OJ9#gzHDSSme!!y-3XWzXi7nHjZG|r9HxX}HbNrSM@H~AmY zl;6^9=(;b^Zt+UFk(}@*N9)52iLX64_U|w5ZFDiXM=}GaBCMv6LFw6*L_c{ieT5wU zA*K3qh9dlg5ClEl8!d$LFUsxZkcrz%g_DI7%&k#4`w>l747u8#01iJHoi4xrit81G*c; zYb5(ZWKaogi(If@57k)E;!!8`@909<{p z=s$mO)#Hn^uppRy@nfi*eXNEduoB3`4OXj`wMD-$aYLg5Z3?JcABCj9(eM9qE$;{i zSr|+MQmg_su+tUt)(;V1Ix41MT9z{O4P?P%2=!FO2-SVFcG~_M0Mz^g>CJ1Y!FEQA zHyFL{&h9x}E5uL`d#9o?MvjkrU*Wk$C8c(0X?SXAh`aoJc5Sh92J=H8jlQpnncsm)Idi-{nvpE5rUGUtD3?&^ zQA!Xp6!|>dyvmL-u7;{H{HUpYJ};Lq#^Xz%+mwJ?UNka*xdxQeq!<&L7B=xyjxRPN33hW zC1xWWsfp0wh{Q7n;w8D<(WFFbCqeLtz;~2_rAf<{}UZ{%l{`lUH=;0FogcB@M+7 zsxE?(;|m`8I}S!ggq%wlmQTGYZHei386lY?Ua%`q$e#*X%Z4LVb0rqa@Ga5|!*P=i z;_$=Yl*Yv=l4-5&pnFE#buT}@iT`i@WCJ0Y!XNCv9~b{YU7-Ji;v+M`05Xnl?k4v3 zV%8RBcK-vTq@@}tp^IRI@7r`3bnl8X29gx}%jwbS!DXY2;>g5ONief0+&gNAH#dGw zIM#fVJ9RFI7cY*;F@LIzvA4+S$s%$n%+GA*z4G2|X6*_Cz$cjU5IMNZiG{YJGR?&O zk8*mxXjgsC#2+%_ctD8CpSON`LoVB3lUDzceYa^FZDs;3fpU209hdF=4Xpn8npQIO zT4$d=+uK%w3d1rD-_Gbke~nkY9ghyAuz=d7?)!HA-+za!Hf9Xf&!-R@Y$2&?k%^qR z!mPql!wm6O7u)gvs+-r|tc+fJIw*NNz312HbK3vb>^z?k=mjd*=j5*gx7%q=HYW1# zoHMT;s1EH>@Hp-R^Ky`57IFot`W=QU^7t z;cG_8#7O-tI?W}7uzG$bL)asLUI^n^6>`FE5#YIhLr~6dsRnqkSuAbXvo%X4J)3@#~v?(nVQ@fLH9eXs5+tESnmjDP!C+z7& zgB%)&-kK9JNw*s^M)Svlxj2@T5hHGyzeuRqzmV-4y+LEZM;2?M?z}vE>A&MAERIsq zm_4|9w3ud?(4qh4YOqGMZUuapqlR)_{r~Rmi6@wI2?huV6C(%+$^RN?=>LzK(t-6? zUtJZZZs|4gW{3)9u}6|7p*N8NGfhFEzyYIVKwPSfqE*wyvyV)q1W1qPNW{5$W@nxyc7dHeeo_II!6b;oV~QTROH z?>ypP*BuTjgn=%_#dr<7zqu_bNG;nr@%(;=gz%uHG^@hJC9(Y8zsRW2Mdqhmh&y(MjXqK7r9NtDPnW7M)|6ga zFXNK8OX3g#+#{MHUMZg;Rl#(b4(aUFgL~{e(mh>VjfJ;IBCt1YSNfC-(vWuEB@zg| z4$-M8c_segA*xMWN@hrW@O088avV-lcM^wJKwPS!jg%L+WS?9v^Jbr3r3dz@c)3~a z(lWLU)+;!`UutHRX!~xk)CzIwvsd=u7`M;ZfDLi!AE=bb^%#a{gi$&>6hMv->X??m zWus}kLsWWe57_RYo+$oKra~*tTXJ(r^mL;c@H@dxv$RMwBS3a0o7nZAj8U+-Qk6^n2LK4ApdP$ z+_K!7X~{uVJ6EkZZJm&!7OzA28Ljb&yCQdq;A^e z?`VDC`}TBGQeq6cLnieythuwOI3Dd$M}#X+kSHK?}`= zO2TQqq_tFkr;cKIbix;x<@&AuHc*PNE{a)$vE)0$*F5HP0dP^Jdd`wdg%`T;q-y1O~&P9JR`&o<3>izFdsLFd_A^gxhlP9 zfMw(q%D9Geb&<4jX4_!;~84j7z`uPWvp#QwOuRSact(~ zHxk^QLiml7pvftjiW`h{jTnW4!0ll^@a$m~OeJ7@l$~VSU{Lok9>fEvUa%_X3ss?0O>_?HpgyOLDgdTWC@F|kX z{Nyh7W(%uos@?Fml*tO=AWr%{xSW)`x=b&iMd|WYw~(qw^b2c{9t`LMLS`L$`S&2d z@xJhw6h~hr3m>xTUDB^X3vfH?`fJfsgIL?(bZQMHDe4BnHwj1wlTDfd-U=hOlE!G5 z=4jW<)p0de_M9z2Sq7&b+P^XF+$nT1WA(p(BnuZkP=l_vi+dR6Vj?7}Mf#uZY~_Vb^qYp}i*VUR0DkWWVx+yKgMf`lZ*iUmj5*w1;n&;o_y=bEBfxf|UCQ zN`Q7nAl||yUOy6if#xb*O8xpJUAJxpy-mRw_r)jeUc*zxyJQ?7Oim2#-XG<(_TJ*p zZadNOQiBh9!HE&9U0oLrYu24nLj^@!Qc4k<1?(H)9-lx3CR!%dYhz>CCh{ zXX~&_^v-+UiSiPhf6amWLoY*8W03WEqRCMVHFZ!2%}Ko*#@#LZB>oz0eK^63xkix7 zwDdsknE^Zh`<|&-P_gY1_>*|$eVc{W>o&2Oda2yik7p4$s(_qHr$SSolZuwgWlAGg*CouZ_Z(2OY%Jrtj<9IV?M3OuQ{iIf|~x z+OCWNe~txonygr@9v+#y2&5>OFY2d4Je)llU2N>-U1%Y2eMnxIhQRK&)ncRb` zgHl%)N*40F(cQOsjH1!_2+(IRufIn`ayt8{pN|7Lhkrvs0;~Nj#xYzsd-GWHfDt9Q zl8Vyz2bi+vw1PRM7>Qm}rKsit^9e~!XTE>x6PCX`P%9qh-2=mc`lps;{+O||%vRW` zE3_)>o@j#9uE;_8JpA88Ozy*kur+)P@8{Tf!WBvd0`Z!Xf6%X)yzb2S%KAYzn z?rQ+Pf5|}DAyI|U%KQ2!ejYIWOHAfU7xnVl>bq|aK#}@R`j^^tpV_Q%zE1JyOJ_&G ztreXsq4K=^wES>~wCSJG2qCa~N&Kvo_ixMy3{ENcGYc!%lrXfJmz`lSHrZwJl6Z_= zNUMIzH^pX5kKm`9OR}Egne*0~1*LE9@jft;w2EN+%=*AOq{7?Aw2aZAs-6c@dc$O6 zomi1GxZ^0H4Ca;nB7IApks9BRmBkfJb6#vB8+E-Z`eyEF)qU??AGEiYCp$>fwCY1s zvv&_EhvL3@okqXsS;VzuTTz?q2YJc8MBR$F-jY59>8bS-6`H%J8fgf&Q&Rt#=M(_L zAV7vpgL>ey?%X*2D14?^=Qk{@HMnovL&kHwCQ-v>A(8p8F)8y%R0?MD1 zBo;$)ZMMs$Soe+8MqZh+jAPj|t_y5veA>_UYumoK>5uaLdCKF)-GsmPJ(y(0n*P{i zys<-v#V0j(cM>ICv&TEdeBw$~Ls1r0G?i3-TnPp`B$ zRxBck=XpH(i7~L^4z1umh!eEgtRz|5MoouBMTeJa>yFHNtlZxwu9=D3uMln^sN0{J zKNjaga6V#rWdA)4t@RqaSN=$s^94t1i@Tz;o3$bJnF-4*olwZy9?s21?8%lzKwpZY zc@lKhbN1Dm&eJGYR;}1jg9BD(H1sY)*D+lGBwrBd$h{P0$Ma@6$#ws<$_8ZR>gSwY zJ*_vjPnewj++L(AbAwjtayY4c6W0(YfhR?bW5EmauaUURIxO?jIbEa=xA)5 zrlxc7qeYUm^|nc4@6e1Ao$yykN@x*-8@_p}bUmn)(DCP?ZdBqM!9$!YS;}i#&RQRw z(jW5~!{0^E-Q9$IB+~P7snMnyzdv!?;G~ZzTHH=cwC`$t5MTFWEV&ukiLw=bpbLHs z;w~&?-o?0Q9A!c%smUpNDiYcO!w?lgLYc-F*c$* zp`>mI6a?gQ6@$1t3Vr=8xxF>Le$ve~0~R{P9M>zk>p6Fii_@6eL)P^|pLMZ*cq8ZT zRs@N6m7Nr?AjnqVjL8qbh)4=&>70@lIE_JpCQ5I-HSbWKD9&qFZB*W(g|vq}7n1SG zxt2G^ExxnzcFez|*w~eP;&!werZX52sJ{5wlS^m*<=#_lSk{d<3k7imnI;A!m#?eq}DTrH*>+KD?X?&h@m?QK9Bhz5`T4>^h1F zsFSSbI3%1+i}u`K76l5-HceoXiRtAR-ifN_vGYWZS~?vVHMfMj!f;OPBfh8x`-l6FcwFKm-v^o^~0!Hfa%+63{M_!aGh zVf9RWH;A5WgE-`#Z(^zBOfj9kH|dER6c@UO>GdG-%;`0~r0&heV$3h1hAh{9t7a2f zoOe&3L|XVmfI~H4+?6SmQ5wVhtT4nIHVipER(0W4w80uT9+kOMZG^lt_1(_2AFPiN zQuHmwW)(bUb?yS85C;Al>IeN`%vR0Aa_aG~72-XK3#%&2ycqhdc^?knE}sZ$gZwb_ z4Il6a-7R#8+Q`gI+hlj&D49gZC<-h6H15E%^oHW+v%IHB#+6>sc∨fd|u2@Kuk2 zqV*-j@Nbm2ZIgh~9-xQQZ~nA86An7{JNxsTptKPoK&blwV++LWv{gP zysYMJMFRZ`3aTs5<13*|ci-%Yt>~ZzWECwI8=Grr-Jqd#TI-Gm)l6@@5S*xx&Q+Tj z*llb&#HlwTK9VwRDT{mYIKFI}Q66VYJujH3eh9cv@QN`01Tekjg~Jz{b(MG4ew0e&Vovmhf&Maas#s zJ3xw(y}%24W~f+{tQT=7TGW@*JG4>$P`iGArS?GN>xyQblbe(8C@lC3Rj9oCSs5P& zqqIg!6b1E3ET99k#0SKRa^r&x8(QN)zGL>dV543>^OM+qK2G_7Z$c^uswYEU0y3_U_0z)1gLeFJ{8IW-_RpT`x>}pLXi24&E~Q zejvG+ObS{gx7NU|e9hyS_8ngTV)?}YTf8MGr1A_; zR}3^PWz11xIFL#Jm{eO$L~z{;`mrPzkfbAAQLsTT90tG{{^^)rGUMpIm|>hi*F{h$ zA6#71A~Aa%G!LQ~5bYQJF;Lfc5>HASm_iTZ!9i!ajZh+!SRNM(JAYj45ZpT#4FIr@|7_KAVJmk8q$GUYZ@m-4kAw9e2~+llyKYuVa=-*)bRDM{Npg_oSmJ zf27?tYA>Ct!dV|{1Kayddiaajlw^n=Nma?4{awL?4(=IAho0=3cPy+o0MOat?UBd^v5Sgw0rF zeV44FKMFnZbj%rG5W!w8ZH?ukP?Eo(m$r}M!P~qPjSzRl@Chhs5D$dLX*L2g#}He2 ze1epjf!-kS;CHm1<4)#qWECRfzyjsH9D3&oep@QTzTu%rY;Oi}~a&^-be)0)hO|ebt<`Ld> zitFSNYD~Vjo$|D0#vs>|cqu-r>SG-I5hHK!HsuUr+%?IiL)fg`vB^E=kY}z;N-}fA zK-F6GYwSm*5nHAR2qMMeet7qlh#1IXA?;z61vCzv%Yv$+)Kq^vGIdtr{;As}dc^z0 zIqgKaal}w$EKNU4N!w3j{L7RFc7kwu(#8z3@8d-W3fXz?+%C?hvkjMHfIBq{ys>4b zncycCgEw!A%S4jZpfWS-XrTlIEEN5N80YqNeFcqo_9g{yE!Mwg<%~Jg)%4|4_jO77qqKcaUwRz9$#*tI zLQRXc7-a|I)|28DZU~cIpA?_0pE7$G20=VM?5fG}e6}0v!TdW(ah!f{s(xH#GL<^B z9a3x64X|6<(&!eO7iJCfnO>mh%aoW7>#>NktbW&C2x{GKREG78c%2~zM<@8@Yw-gb zn%aBe&}lS318O6+z08?!54U>t^U~Fyu>$5=D$DXv9y@9D()THXBr#x|gbKD=wM{|v zk4CSMI2grVEd=eziL2wl3ltkUf3LRDE_|mjGo^dc2p{3g64s6;^-yPyfMlTjNzsq` z1vPo&pnKHO!&sQO%*q;FN480{;XNu_=CV!cP>LC;iOZbf-E{_z^N=<5 zvl7`7=^XXc9yaneb7Hs2RW^rWTCI`z1-Q)xW6jTsEM4mYT*-D61b3fr&px!Ce{)Uo z+<*FY`g#N4g#41=B)llVj;z?Lp%0c}yj0#3B`~?Tfd{c=W0}as)l3TD%X<)_Pc6}e zwe+lJ2-=0;wFC!wY4*}x$Rg#KhNuV<3>MP}#!kr$Z`Cue;a5>#_W8gNmzmmO^Y!Kt_f(45W zg^%5AHwmPwTW_Z}_NK}50I(ZXRlcV#Tg$wF?R3}Mdw(^4_4UX0Rqxk?$2GpUCNMJia5Tz5`CL}`S22h;+etQSB_`w}A7mS)esJ@#E%hsFvt|rQXtgWd zlgu9{eida|@M=bmQ8bvA+n|05$vQT=3K3C(2u>AroHpa;zR0kz?kYQyq~0{OtA(q! z)7V8+v&3U6#A%i~kZ8r|-lxvXtIow!y?nesQqF0UZhjzQS8da@sHT3}aj6~I*z5Bx zV`P;{cZLIRQApsSN;9^LY73axDXreG?y_r1ez!26mH*_1Hl^oEe3pYKGy=|48=tK_AuzjzJ?xZ44#tqgeNr0k8aZ-uevf1sWp^mM|_k z))_U;gyQh>0`yKN`X>?*j+e|JQ6}Cc6_L%O_8{0xZjf=$ zLNJ1P?R0IJ(=SlVd{^bO-uk%d?2H`TLgJ;`;ysa{lBn#+9b7f-g*;K0>7hnD<>ox- z7n{%EU1QkB@EwaE-S#c#9{W3jw;6uQ{bc5kSa?L$TkklZqn6;VzMPB(r=`zF5UpuO zDx9KerivulEWiToRG#AIM;hg%W%3FX0zN&Fk%R})%rIq9)oar0zj%O zE+LuWs(4a7Oh*SxZaeUor}lhYDQ#tYr$Ty>ex*5HvA3z{LsA(i*T!lvnQrf^*%s?I zGm=(7lv71+Wo5OuZck6DJC>Ft0&juuk0x0BHQU=HPRDl+*aXpU8w9)B<~G(qlyWu^ zE+fiet=0@$^A>72P6gK!juHy%9kkzG=DxI7B{+A*dFT1Myg;3K-*$`@`Jh!pYd){tTw87j%=(!!MJ_F>T_*Sz}@pEOstYU*deJ)TEK}x?s9F*M@+mAhG zifyCtE7cK96Tg>nDS>y&16y}#j@Gksr=1r%oz~6Uukp;JFFwRuut3ajL!&i=cUWXX zE+UR6#B-I6bawD+IiF-D|5$EhFn5|4y$T%`L5$~0<7gd~Qo;=NQTZ$vicBgnFRq_< zP=AWmC-XRatzcQ7JeL208T90tuI#f(F0Lw&#WL4?#4TwVaYdGo2#X{=h$>4lTX3FA zh7zd>o&K&r-1wLMm6mf!XkR^=k>G>bt0y@%>uGTbc_nY`o15-5mnl5b8}O$6x6X##WlWZIiI7rMzmxLNrh)n zFzL9(Z(y|19A{VR9fSI4meOU2Q!Zh3ew;od7+eYNAcQ&Idh?H@2H_UttWX#4t-JP_ z3+l+CBV<=mF4p*k3Br35Q43o@9zG$YcrH|WLGJKD+I@58RAGMf4jdzH9jWqdV}&ri zAS|qSGybdP%fogFc3_JeC3ZEXGs@Q8v2{5$rL5<|SThx;ha|}kPXS}5P*(*d*=HA* z_s6@FjePFK;deOZkTA(5ZlNvOP`KufU-5DOobquvCF|PL)eBy;Or+N|i`ql?Y_ty) z=?hH#vxXZM(!50^3K@h&kQDsEN($yX04k1{jRB%txtN&SS+IDzm^e9ExUe{xxR}^m zxIrWb$levQKGP9W?Pg=)3InmIU$Y9uLZimB{2&iuwCRH6)Cho`bEv@<)5PE^ZzK?d zPz{T+GUj<0UM@=m99E6LSW+Y|vZ(CEMw7v@*b2?6q%T}fuU5B2keumb@nu?+^Q1$7 zsa_Ky_Dkm2c&20L8v(8le$UT8@Vd!0sky0UWyICRP$;oY39n2MZ}~#soS{sVz{YUI zAOLr;+fx(CwaGbUO-n0}jwte(EVYrhl^iWNKAI-hSb>FEl?|qX;5qMv2Ab=rOd~Lw#z~}D z^XD-q=cB?>Las6ub}i3YNg4Q!_96x;N;U#yWSwX}7gY7$T)vJpvoT~XwO1e{ad1^- zdYws8lcL5FA2w>`%~uaeIdF~P747TYB^PS8_g{v~Y)W)l4OtIeEe%5zfk)<4bgWgV zv7MO?D>$VI)2fmyHXG|rSkMV6<9m7S_8*aBhEOy16W}Im2P+huJHjIhXvWlW&XzL!O12;Bme)#{qo2`Qb9hPrST z*+%r#6QF(Xbj8m~hi$_8o|ZX6A0lSVEvLh;E^czQV$o_=c8{o$R8j-N0Wj`onsQzx zdPJ%Fh>xUxvXYksj?FhK1>ZEOR=@e=!femqh<`4o__12Efo$^j110Dk3@dNTi#x|w z{pY+$zXO)5V=KQdYpsT|AbB^o>38uSt_{`sD+H(?gP91C&-2fOP7SP!YjqBmnU7Y0 z?RKw7sgKD?S9Y+gpcW%Q59lL=Rp8eo*Q6cf-?-nw3U^;4on2VXw_TuR7e1d`3qToR z#1~Nv-^{dlLfJe)tzRprHg!IWOU`9WYEE0@7~5f0+991Xhd}AomcZK6NvqR3p{z-W zRyfR!NL%?yq_BZmW`pGu^b1(gqt2kL?B3G6I^pw)^*7~`p{cryXNq|Xuv%oa z9$?d?aG^VE-smOlwn#4?vF8%1=a+Q}+RJc2@`MZS3Q+222V8bWP@rEfM_{>Maz>hb zJ?#Zd$TnXg)Ia({!=Ob)D&jms#+f?`6qMlaamF@70vgafc3D-&e2%HyZK<2(FOnr8 z--Iug^$mA@pRsHspI{hHLhubf(*=yTP*PhM!#vjsi0#%(Bud5QoPG}4BK5*0ypeG* zT~gX*&)S;$a$F&?{OMYH!|`AW6CEl1l)je0av)j6 z1oBXsGN_GKe9%3HgyP$73(XGi+XN1O_n7u5dR{(cpeNBomSdEUZ>R~g<4TgkfM#>K zk5oBv8c(^V+QezQ$&sf(3C~g#UsW&-MD~~W(^gvxE%kIU?^sntX>6;bRnwQFKJF386 z^Vo*Hw8U|3v;~w;#gwd=QDKsG+|*YY1U*p4cJG2sru9B_9!yi{>4ER1kD6_Z%F>e* zW@^#u6OI!V?#0h*6bS>%46x?im-8L1zC1`IG+&@w>shZ_`nb0{dewxKiOmhEtn3~l z_JR-_n_Q(op7`mQ!T&WFcI#N^oYr`)e`nkn-$#qb^OSkSh z`BsWZ>Uhl)vfhU__{oU$tziTkFya%u=s6$2#qGO%54Sx&^+y*-6?lGMEWMXgB~2Ss z++VY7vtP}^yh+8lx=8406nTbpv)F;&4h0Qd2TFG%To^| ziGY|48+d7mY3GcYv&HMtbv!sqLN8Pa*QTXJRZW%aqExD*7M@(l%0~vAnD;w;b>yOT z_p2|b;ik(U^yQ_iM4ohr(R5w_!mu_#iKWtRgC`*}@50`$*rwNjFt{7xL7STPkjz$zmbUq&l0t?gSa%-KVT3GY6? zk{9Ezm^Y2xR{BH0pR6BD8~stvmcAT(Z3(`$F_aAJG*0eCAy@j@LjADIoG_0*g{T&f zDUs6-KLcmwjF6WzxoyVa&u0CMsG9Hs_dCB;d4{;&Iye`A?0cd*ISpBeNQ(t_j;8~o z%>qFa+J~Mv5Ejc0-id-aX!&?XNoR?J1h;@d0nPW46%CS=_)M&*BXQ^jT<(^$fh1>b zVG%MaPU6l4f~pmpKHo52Lig`pd+{B0aDfZ#0XFx$DYxt2Ja4aQK#xDKo1t_sL!x}X z(d0vW%C|^MG4LkhNbFcpu{j%Jw;x2c%8G$F1EG;Zqa>G^^8tEyi4n#%09s}#;slk* z5BGD)o1-OzPOwy*rpt_GBxgGrzbw8*ArM~nAigpkzCr#L_{rN_qBr07iO@*cFo3Sc zpckz0kQfYcu&F+4i&vSXbyV4>$|6l+nV-TUe)LE$a_}tR9-1KyNM;>VYNEDhiJt}O zZ8PK-_7MZ;$0brsj$Yd|<*!E4%^ERa-q0X2^P`o%6JN%=1lB->(@}B+#L0{TwOrki zrf?do#n@nA(<6`hp>s4y7gcSV>gwLt^Hww#7*H+DTJW*1CEXIss=3bbau^DJ_bGhI znjJTnH})i{*Rx3tU8QyU>=$atbXE%5j!8?qMEeNHMQ0LS%o?BmjnfUe5V?Xj&+5 zp2fY!_j;UE$*@b7wXe#?Yj!aqmTf{0nknj7+PcDKY4zaN4?%mo%nX%|Eqz>|82V4D z{2xlk<=>3zkC9HFHj0+giyQKB<#->02~NqGsN2a+J_QrN`Tcs?*LOa#Ff>fIGZ-D? zG}QIhnH)o|>a%eo|8%QsBT!}J=ww{}(<%K8DXxBaQu_&RYDW2)$LeB}bNJ5%d1TfB z3*||=3{|)42Lshgz3ur@BzaHu zITsDB)x7fbQp<$qG+i}T?K`L+t2kJEh?{V8>c(B(oTM9~=+t7w`^?&>f0V$yC_Ou5& zEqaqVpVm%FsKEWYEDCPMZIAZEZI4_-M&A^IJ9nl2+B(Ou^qF|9&SM^HZLxUbk^HUl z1%=)V*4yk|_bz>0-(K_=+#K0EeGwK-L1gr;n(jiYWgIx&Vx0+a*dDGw&qN6eBKKrL z5u9!DQdtSwep$ubg8f9J9fT9X^pC5xrnw;q>SWFTr0YUOtf0YGE8#M=##f&Svwl}t5r2~SIi8VdlCf%It|-Z| zxckB8_MXe815a15N?>x^f+<-Q<1F=1f#G1JRO-70qEOSYO`71BHjw{LN*G&m`0^xyhZu75<@m61JWeI;?*Yp>oTvkmZfRGUxMAkX#oZ@xhP+ zQJGi&Old@MX7k+VpB|ue_jM&#O!aqetUY*$b8l4J@nL(K8ssIgL-`?4Yy=MF&=-au#{RX}R~UJjZ0^u{X^tG4Sg)ln%9;F&bX_eQa8XNKZ_-@pZD4_2;QmXG;ee2+^vWfe=*GC%a0Z2SH+p5TUi-BSBmWqsa*5KB;MeDTDd2|<+SGFH z{i&7cTNnAGDPvndVrTcut5NmIDVzX|)f$-h&T4URD3~!>yL3wvbSkV{$z3Y0*b$z5 zUD)c@&G1_$UnLY6R>-Bzq7{FNOH-h~Lt19a2H3yV(7w%%=J+a;hy~=9OYKiI^s_)8t z!aL~_9jT{ihQUC3S{Ba3gSfvqV3t9?|8<9%X#V34o7vg?pRO>tb^=%*G)VOSetV&G zh!4y@ObE33Z>D}oxBrxa04f-8JQW_~0}aIB(h*FsH#{cle;T6^e>Xxnslgn>1i(Mo zs{cVj_5}e>NdEx;o4oq39)W-G&HiD8jQ+O~1vq^e6Zi**<{!W?tiOSX;G$7{@cl3` zpp4?*s-J}TzYkq|gcy)c`MY|Rh#XSI2*_oCs3hQv5o+LnNH?IM82{+qf*As02J4S9 z0@Qhbn`e!Z0slzt`~&~?^=}XcD+c&+6chO0sTYt{?EjX6fO+A+frzO8ornS68o~$u zvGDz0o4-zFhS=X$2w;gZc7U(eZPh*(C zKVD+|0Y=gP8!Q9}HTx~HX_A7g#~A_1*1y$@aS1^D`@cm5u>p#f;R|w~yQV^iv{{IW;fQu%007>rPwFv=0zdIP+nhUHr$q0Do1rdqB`MuN- z6%#x-NeqDc2zK@cf}bZv0m*(~(J2kU4d{1`z6&b2dx{Yt7z$CTz!=j6z&|2F{~Qiv zUl2g|2?7!VgQgh)V-XOM9xU<^4ZJyx3H-ym{llg+`gg}{Gm-$@gx_M?3=i->Pq|P~ zLVw@~5`TlxW~BfZ8UKrqAptYYVuG#W|9f-8<0no+2tIE=_!Hn~afTz;GRa9`uJU`&10^Boi2oR|TQ!JA}d0jDW3ih)4||owwr$(IC$^nTY}=UFwr$(k&w1DRviDm38@j8ns_LHt zOQ`{?rTGD0zG}7t1`Yyp1`Ps2mzZaaktp{A4WKu+F?4p0R?}8TRY&`ZNjEXT12+~3 zj0j{$p$~6bQmbv0>iYGA?uU)YI>IPXl$_bz=z#P!ruQdg_fwI)ZiO#&WA)nN@>k?n zB%kGT`ltX(Kn3kmI`jL*`tzml)4{d*KYnlr7=FsIy?}rpGGiXnL!#hat%W;G)s$&{ zsz99#4HQv<0YrmQ+mipe(48CPk%@_E6y-@@XB0TNc^&2cklescOq33!9gMrQQ5#vx zI=+W`ueXPO$dP;?W&b!I`{osy<~#L z(A4=+{uS0<7k$!(YC&Jz#EKlFuFFHfC@}ww-=%XT_ZDE=fu5>o=F&gb-oEc-{+CDx zSv$lyyl4XdGdr8jwIXe*J;o4y*f-p;gaMnJ@cC{wi&%7JT`XPjR#>l2I%)jM%s52u z`=R_w;mLE|=@radcQk(aPy&0r+8NaE0xk(exSC5sF?yKZ}1h)9ZZRu!xv#fgQ4E z_)#~yCN`;+Cd{Maur-nTTQ8Ytc1-W`>h#t8bfU@Z6esXY6dEV?_u9QXrF&60o<=Q? zmb8v#*HCR1=7in}C}LY$$*>9B0AEWXn|*zS*0xTv+hRrp?mXK!wf)VefZOcfg0pAM;PanH;dE9DpG$QXMLg}e4N&ZKt{h<%SSkSY`TzlIO6#YZ;rk(H;MhujYyG!8qm(zNu zeEHGmh}8cu%OCBh(`QI4&~MwPm5UnxfzIFSb!WsLg^Y+aOsVvfROlb>TqA7!Ao$+_ zX|zL@s*)Qfe+)kPee^jl<&r*r@8l^x5=s6QXBhHvthiY^pkVnOar7kN^cnW|k?G`b zx%iVzW%Ez-F>0adQ;L&Gc=6{dLje9ZJ-!4rk69_!G8BlAv^V$5>=< zSD!%gqU89Sr#DZhk$=YuKXf_w^E6p)W?s4sI)Me0SE^Qi?hCFL!^NVoAP*Pyr-p^B zJreWfY%h-jY6r4cNLmy-7WWOq=`8CL*Zv2P7!7p`KYVyqh6@S;BJ&djgynzRvKtCI zk$M>mNLJRBM-@clZJ@K!ZPrk?6+I9Z*vNN_)Sq2Qi^D-lw31Lj>7SpGTVoxUW6sig zip+ z6g-HQGVzoWXu9DR_s8CNa0ox-*4z;9>=+Ij;Qu!mM?0Qj(5eA#eB0jDL9&3`jjA|M zF+v^N+zK< z)=ar{C-`C@{-CL-0hBDHtbGPoUX@_5lxy0B65E^$1tY9tTNSAnfy<`%$lL)`8PrrExV`G%B z847+ysh#*G$nZz*M->ZLR25#&n-nfA_FL?80HGta86PhQk4_P5IIWZK9Zf`r&J`re zb>{)mP^aTGpU4?5JzmeN6Y()J{0haYyvvsi$~)Lx8;K=oG2>#YpH#)-^I*t3`xwEG zx9*86u43;-T;313>IW(91uufs-JJ}7QzZaIK^p8Q3B?lu`us7yW>bKB#Ql%HFx z&^^4=UYC**mVz%L3o2lloGlbs7ogf{d>o5jBma3O;P0fIdq}A;rErB|q4mt)5b9P` zC2_9LVt(C+lwM*V6tn5OC6KUa``Vm$9sJe_s9y&-%`Z`VV`+ zezyhQhcE1xA4S;q;5GOyoS!}`kn8cKzzU!ZAbiyMGhn?W{z}=G4}{vC4)7q*SDDvW zn#{?3iTt${`%h(3AN!^L?aLKude=VSMcYS|6;Kzo1nl<+175^G68!jCC>U?k@2lO$*yuAV%1%i zVNaE1mm$>fCMr|*yJ!wcAL;#8Q~BdV>JP0^X|^Q0q$-jw3QG}Jn6**5lCeY;@nX7? z{TLINJQfPfn3JaAm{ij_wXhm){ZJB6OJxnOX+HM$E5sy_3RkvG^b$9ZynZxETefX7 zX|mXc@z71VIb$xDZN*HYfJhd!f+pT!W2!YvijzvPbXry~n=|f-JSzwjx2da;rWE&r zb8Fta{CHy~@5@5n9&xpOdL8q!xiu@zBGSu_ma$G=VLn}^kcWF0e)YD|SgHZYZLazE zEb*OZ@t||8Xh$4ZEp=!u=sLW!+aLNiE!rVRSpE}70%jl)6&2yI>RhfJ)fTmy7sGvx zzU}FualhI1Trz@@lo>WJX(lg5v!sIJ9hlFQe&tT+ebkUO|HEW?j&HVm6NB#t(1OE6QUs@ynOE z%b^7h@dwXD#y6l}iCTH^AR)t-{is`$c`240b-ymW@tjr${_<%IVo(M3@nRO1;^Oo# z>yd_BPW?Op3vZv9T0uu{0G;gUN)XXPpQjr)GPqMc$J) zUNF^|aj^H+IZ+*S+zQPtcQ|L~(_6i%qn0VhdF2mHky-}#A>rktV=6P4j&PPJA^*rN z^^CJ7XYaWwXVjNzLVM4-lrBR?WZ^+-3G-1nv-qA##ZfC%=Fio_;flIHTG*3D zE1f`;_;)>4MX#WTGiBydts`G_%z5N2ZC5$(v}4Ss?r~Q0v|#}GE|TLm2~pkIs|#1e zRXd*IRy7PqbQG%T){-#BT$1srvwUgJG~8aizJs=43^Wwd|P$dm^Mz^*70 zd9Y-QHxX8yZKv3amK*xc{yzDAbJ{lbD-waj^Fm?F zI%YK;S=xT13Ce7xcR~f(P_itinZD2)Ls$mG777HLp+erdJ+eswmPDiT`M7sST zjcmARJq|VHL86AtS8AQ&kklAO*6eCH>CKkV7j)k%9ZBkiN}+{e zg;L76$|s^06KM&YXCzs_(^>+vlNyNhRq79E0tLg`2-N8W=d^|c!NnD+Y2-y*)k!Se zt8P+2s*09IiOcBEA3&%ojNtCthhKjFf@PZa_|9J&Ad& zfS4&34w(Ual*BnGl32VEjKr8HY-2@2>$XwsZ4ZG_m`-|7Lowb((4a9{vvh!LbETg% zPOGaHf@`j>wR%QCjrTB15|%$URs%&@Ejtx9WdI0%Rj65NRcxi{S&2pc(As6hSnCw49Js8}S(k7OI`LY}uyKm#W|12}ci2 z9O2dWc&F0oN}7UY&1>P^%+7p?lyvIri$j-r1fi$agg}sa1&8e7*WgE>RYz4~G5vnu zRt$9NlK(CnDN18+6WK8?lOR47RXROAC3VG$jAIl22*bnW%DaC=iKo?Os%lG6Y#IbB zO*Z_o_sbBZ$iR!ucAs`4`UaNWF*k)FfpCo zB?I7O+DD~VrI*|HB4VdVdET8AOVv|&+okU9L)?X>D$J;mVlA`S3#MzyCYRLxJ-snH z@dK!axlf|jPdQKl-C0MCzLMusMI+nPfy(o<`_RpgV20nWhO|e;4^8(hu%pse0D#7E zeC;uggQ3sT4}WN{V^y7UL>>WF=hmA|hS`c#b>>bWb??S?U0P*&#d&zW(A3yS+FcPT zL)U#}dM%Pw)S|GqHK<)&QB(RA!i;aFp`U=#wr;(qx-}6i&48#`?$tb~q7pcxXD3R6`V6F{XQ$H2rVi83>+L>HjHe$Dp^tY1yH8xIR#->}1H5+ooEroLPLm zS0#LTM}f3WrNCU_6{O^(AdLVGN4%V5^>*uX1N8^=847uCz=(M9Rf= zN?FOc!k;1IxvEDf_DyKc6@Ux+P=+s0 z2c>%DRastkD7!iDEMC>#;uYL4w_2ts+UBy}r+wFLGnCpR=fS9b#U%S(E^*kre)mn8 zjG1ec06=Z1n@y&RT1U8RJ~V3FXC0jkYwQ`_feQa?i6Wd%ake^zq7AoMUvM*PhN|iqvZC^ANS_ z@Yb7bXgcUA-Y+xMft17-Fb_yklP;|?%q3QHg+#&2vZ3N5&4tPJw9RLPtB%M|?#J%< zi-Z7K*VX{Sh5EA_;k8KqiSF1W%J=Ir*ouvyKVPdgA&3QTz`W~%d2J={@6eeWyc9&` zn+VOy3us6tjm0Z#8<=;qx(s^#dAlIrJcpGL1KX9<-gdWdpdP|!%!Yl@qW{WZG5f5X zm(>thnK0PDxhju1W^~1GF+uW#q2Uja&C_OXwMEDb71lO$d>^l{Qd85ddtRt4U)nxX zU8JdXEkEO(&MUh`Erqik&%I3|=khL>gx(O4@Xb(zsHrS}CCvb}fX+;&sbI43+db-d z%t(RL>i@z8e)oUDt!dfi#wT4e>@+{M-`$5~ATD=(BdcTC8M1#3 z&vS$$<`WW<9S<}9QFn}}m*0ow2xQo$?T&)HLH}C-V3gdt&TaK*BJ=$6`% z(i5wcsnr?azAuiokmZ39%j^g*)!o-dkw%YLzXU`9s!4Bb9~|C-V?ICm{c}WET|1+m zYt@CV_4?&!)pDewe5C49rKz6ksZ@y;Jy5QNi%J`(s&690ODq!gDV{4z+ef3V;VRi{ z`+JxHh-}syKd@=eQ@WlCoft*wBjXma?O8Xu5DdJLIs{ zezP)PgxD(^g}kQEbkumtpRMr~bNdhZ)HSlsD~uayy>f=mkl${%*K%K%N88z&8?H?X zEkC+;gpd(z`ljcK?uq#%rVw}gIg$UR1ulOA(XtII>+^RwJ`5pinUpkvzViKZl0NK- zt(-2?cu$Dads{QU*mxGCXzn+ef)o|b1hMriX@vyyE{no0CNLG@&dB*&yaQdHf4B5| z;y=OhR%0=-47i}(caK@vx^<1+260QxqSVl$R%FP#uNxGo9AV%cjvU+f4Io10SV9p1 za|cqF#=H5Gu^kP0=aW_(_jeA0YPhVhgq166V z9#n&Q5ZJ6?0~MDJp@o@oqezXeu}wEjgX#^>;|It>O)^o6OIo|QhOf4!?^tuee~P0S z9BL$IF1`*cOx}83=;xnli}k*sx70^?qCRS`U|#;`)o6V~27xK{rUU)Kpl&f$1D$Vu zS=6&{Q90rl+lF$YQ5g6hBAZ?I{}3}VItU2$|G>%R3MP=H0jYzgj`JN$JHZt7C(<^W zn2{P5gJ-T{oM7v{O>s%k=N-E=TDVVQ4|H$N+bo1fNn^2(^ zRvk^oU@Q_b0$#nVm3+VbXjOt@2+>)v<>7b8oZpX6PGNM+fOkKuKeRmas@$>pPj%+q?uPrG-RU+aXN%V)6!HYkHI4a}f&QFH8d2{Vd zAXx2UMc;|M(imDAl+KuFx0}Tv)r^^H1^@|Se7Wf;WB4((v$gzqvu#>@W;i;HeO@{W zq$9*BqHoU^|4#JVh(N=Yah8IMcp#nujByK4JM*T4#@Lk7yDFnPp z$jRU{OkL4f^^9i0*!-CB&3|_*eNAejj(g`sV|<}$@GR5vV7c)t{ixRsMJ&xF2LLQ= z{Q-{XgT_1J{4g3*!i7S}_ zHAhlTDfQ#J<-<^CMXfv=>|}pXHv1q=DwlB+5t$a8H*c;sa>c1~JfrKjv?5uP3uM(L zYsh99|gQGYX)V2^A8TwMgio-|W; z#4 z@$iWOi+@=rG8*NrRtBu)GQ=9m2kwu8FqW;YmItlbj}Hu=!B}u~oNz?km@;MU6Aj+MZG4a4V0{_r!#cq z=0F&?P@feBd=KVGXfEQ?gs;*Z#L=_BSJ~mh<5R%Cou0UTb+4zmnBnh@Yn8}5(?qxD z${xE<$G12~-O1CCW)RJms@zjyJIj6QKFi)}Ydidf6z9lupowG~{E_g(z_yA!nTklC(Y%Xk(CK$o|M9=P0U^eOcM9IFE(7EU?Z^?9-K} zxY=!VZjPcAgrJo$d5tG8>nc9lGZ$+Gw_%5Qjo5bK7`cIx+T}*K8g3GeK>!PW8EbW~ zk;X@?^P}vNfc@=n6{~YtWey<)!?6m_rlqt7u<<3KOpV+6YOBm?39mMRc!515)GVPrq4M}*l`(63PL#!m)B1oW$VYkk{4Vv~SL6q?82B|uC4uK#;mb<3!4u3H=w}bp zQdsK-a}L$W7rutwfY>wa$lIkOEeu8}&YNNgfT|BLsj5)50#(Y)G*Uhyv{AmpuE@7Eu8YyR<{BtKujsq4ic#(P zCS$acWT}}(ZHpam7j+(R#U<;QHtP$7F=K}vv1~JG?~=0J+BeknwvdGn7uK>8R@1=R zX+p%<|6|!eZ;9rJ>C{VdmLGL{VioGvoM3uy)owox3*hU>!q(Li$aH4 zw*pN`Q6ZY6EIA9!lCv%-K^>Spf`C9H{inJ9r?&*w zAQS(tLIGLouK=DYzTegbVB#RU#&4!)F^y;4EE z9aS2dzB5RyVN8?ZRUuN*Jq9MoJB!xd*K@5gx_9p(XqRA)!US(^j)}IQ-FnQ^laW@Q zD)(`nZBpAS&%nX!5`{vuBR&*i9;IL*F34~7usAP#|JP@Y!%m|R>9!eSsm;DfYEv z|8Qe5F(C~$n5)>dJn>P=zD^`1A5+dc-ocCBhZ+iEA_I_rZ$Xh_ydks)HYymnJH%Lm9O zsQPl^`BJ{6DZF|=;w=fh{4PuX`lXDeht|~d&M=N&K#rA%7#n}0HUDq}05fIbfiMj# z(th85Zut_t!569NRrNDth`VqwA;1~u$U(0kwl#Mr6~`|aleIG_mGuMX-|P*HU&XHb zTwg_N=Eey#ZGYm~0d~!Tqx|g5^(`bguyBL(Rk-W=q1ms0qYlIV>sR{xIa-xX=%oj~ z=(`^Cw&0PA2LksjT`mGP;MH5_ZSe^2na_U94iBV$*(bU;dX>@UUB1P^R%$XGB}*jH z{HWLwieU;p#(d)qbnS7Zp}7BC_lefKx!moK7qM2 zu$@O*KrZ-R(08p=KKgZ!+IC&$wpt{Ry4j*^f|t5H8 z&p=^?{V{bo(B_`MqaixYQRF0{{l_(L`)}f zk-hFl@T3xeEzlhGHYT_f3vYJBqOrj3PPU{-y{~Z<#2S0FEc-pcU=`(V!x=R-CgzWy z71}8&wB`94_fEqOH189?vI7NH&0L+;S5lZEVtY!XdQ4Ao{qoUVMv`MH2KHauwdYBg z{IcbmvYSsLmwseLR0T3;!Gimw4a_$0y5K#NC<4oeAiv&RQ`5Eu#TL4oS?xb~zNs~u zC1~AsKvT3(bfo6g@NAM1+&;FqRlBJOEdDe<=*V{4bX-USNuu48JgZ9xcruXZ6l zlVQ!xV~`jI|Ef0b(;vON8~vt-m#$xN?imEb{_(CzUutvWh@+xCv6C-P3z6Dqh$`1x zW8|6e@I*Q=5u}TfV#iJqXYfECa!Rq~rP?OF^*}t-*&gH284&V;ypWuR*QEKBwTko0 z+LI7}hPBWG0=$2DB8yQ{bOkPMI9VW>tAbLRX;zYIKk)d=TXCoj2jP|~8S-o)E_INc z7*|@I5Y)4vS=I^sQTU=AEHICEIVn*hZg6fj# z2*XU`z^qgKfJ>v@A`L1%+ zC5awM=Fl@q_=5$`8r*Jg{0zx-;!>@711~d#z=n?cmIcfPx!itswnWw`sVoFk(QWf% zFN;d{R%4H0v{tH!Na&`jzeiyJ$-pG8=ed84hY12z#^!H~*uVZrT}Bd7H`kgg=AqH5 zUpdpjPtEo$TG_-*@sV*_PEJWRJ_$KK5w&bGWj!+WtzY6_f|Kqu6iWs<8lj)W;p-v{ zc9_B(6662_7anD!~FRp%I0)YkEY6olZ8@Nd(^p51Y%RSY3 z`i_6R>wW_h5)pY8?`uSM-LsX2q!62GlG0Ml94Hu#H;eL#aaATdrskOv`#3YUb8^m( zicTHcKZ52_%#gIpo>1O!0h3Q2s<@G~!x^79`5J|hcX#a0B-p`*nRc1~N$)UEK%bON zY#D)0?>MhAs88?NqU;?Jq2z^y5cTdRIr()o%M$V!P;eT47zk-P&1biptPo%%Osrb#*9sA7R)j zJI4kqbbV4>DjnyU?2^o?GWGUdVQSPp0z0!Wi?!^W=B4<=ccbmfG0$jTt?9Q&36G_i z#djy|tPbd4tnKW~a#;M>s}cz|Wo7@K1A7>r%~H(6leTycx1RJ{)HaX}oCAi)^m}`5 zp4kjMqUbSgWvXavV?)2+()Q7z>h}?gUFGHA)P&)`~_A_6+FS z&fP)HwZR#;h7NhVZ0d9uM7oz~ZTM ztwq4rDeRE3zwKLufI`i~>w`umpa*MBZhx3gxq#>TV}vvHXWtKFs`)l>ed6U*X(yXz z*k$I`3Sz%3OB3B+mFxq;lU^=WA%b-R10~?g*0X6a8uJXWa8c%~m%vkh(77zbUzugN%h1Ua*28eq z)H}QW$}lR|H(T8J_$$!iBb?e3zC}N71atjI^Y=+4WZo7_NC`Szp;FqYw~ehA1uEo? zI;?ba+ChVmL-^>@L)ulOH~=Q5?#WvV^-B#>>oJXpMtk6-t&G>I|I!i6#g#5dF$CxD zgSC5EHieib61EWM8GknHTY2Aq3aLj zANvq*Lh9aggq?v$^w_QIbAvGYA&2X(WZ@;@5bf2-?xA7do=~@N3?9ECaq$_R?e=TBJz*~#)lp`+Q?w|{s zgcKRKE6zi6VCNSvdI2nGVS_*W9P)b?e#xcuSo@T33k6^47g+nz;0(b5q;`eBEbO|7 zQQ$2@4CQJek-G#+y0GIpWKAmyki#p+_x*E4!`IV?;Ie*^ynzJONVRS5B_&UyHyPSs zN4E$!ohomrL1|z^`%gKX7f_?XSQ@^hMxe)A^i`mJAw1xQ*#b~O@kzg0t8#DfMzF$J zE$TVeYam^rvuEMP`b5Yu*b~7uhEm6uAy)IbcDT)q#cCi+(Y+!^krC|?1wP8Kb;&pJ z5O{4wI+QRNo1%*j!z5Xmhy`JeBnCllV@cyxEk99?gH!|(eD!eZG8}jcOF)|@O~$X5 z$+Rig6a5+NPk<}(P#FEZ8+Y_>6rwFm%WElEF&R^E>_8Q>C)yK1XEs!w=3I-}s`Rxi zKIbCh~3-*F;hlaVR=YN1@Y2`Glnn z<9RNzU#BIk@)pV2P*u)vnA53)=J>;VsPOrkSC~qlJnDgwaUU39Ur%v54xH$@C zRFT~!b1kP#$!;N52%?K;eL&E#Xo6Cn;7IAR!k+GJWD(dZ_mI@!p@1L_|&a1gMS=8Hu?iHPp z*NU%tUu7)MWu2Kds;}k`5Ke;MW30Ee$WW(cX_JkIF3Je@Uc5V5>4cf5kKzw$?0Afw zl>G2?NKaO~^fHmed19m)$)46ItBckmopewKdO#OyILFFiR#wKcYKT%k@U1#|oTq)5 zbN`uN0;#gq|NPsQiyAe&%Xo!&58js*L1k+kvD}4bv-nQ9Q~uD-B6xkmih73`td@Ol zZ*o955@P;Eehl$AG!tYp%2`M&wMBNl*gMme_ky)ip`*~&UcASGW*jjS+>_;ib&Ulq z8o0_zuiMBzwT%_4ojJJ^n%AvQ1+Sg^*)xW+kF-uBTEUJK=#)=P0=J(^rbh>}gdzwB z!q_5B``j!-LR53antz*iGESW^Ci2_fOYl^5Q)PED_AoSYc7{eZ2TpU7D*7GeQ9Ia@ zGVWF_Lv$tXN3!=o^ize;A4KYnio1TJ7eM?4sooW6tc((CqK8hW=9jWB{lMa{zC*ox zBDE2#flb()^mpjH{mCa!#7$T=feq4Oa(Q{v6iiGhY6E*bt^SI6gQVc8b!h&gnGF2L zfKuB&`WShI#hQjx{9*vIAOz|A50E^i!}KwRo2HTLd?Eku$WcqMaoS}41WV;^pmR>? z#WWe1tSS8{|Jl)*7f+NIkY6srbHbj5)kOK1L^H~3{k=lbUYb;EH@Wl{HyEh6v;F`< zxjoCR9{ee%$9WNP-rMX@DG&+Wgg|E#Hb2NHZjj;-<=S-PPKH7QTM(iA>gp%FyIuC- z{S`O{9Pk@f_EW#8_$6*Dy-t4sY%o(%WHD0L_uSO~XrNwTzgaCah+Nagt|bfKGk1*V z49=@?6mp~G5nE4%jar`vV*vHjoCk(v{+$YyuQ+@08y>Gp@$k$GPl&#l1imdq{j}l zN${qoyT6`)+0hJ;#n=7cuv8T|ULuS>awRQ8zv{rTV_&z7H3KZ~!*xb|ir%CD3fg3r zq$hcNia8qVos}C8tJD93u)1sw9@oL-Y8rSgXWEw>6@g7)-U&BORn!AN`UwR_uUlz@ z_#;$MKslfm5oK!1VT*wW7&?r^U%7|vW}|1Q1b4r6`)4L08H@nyBnR}dUw{WYeKL6& zz_Hf+3dN33{$yut18>5v+aEwKU9@0kr1K7+IsoCW%e6ZBSU8j&oh=$;*)$bN@S*hA zkS*Qp(VSm3I)c5&V+mRr%FCD>@v|crJBVS%SfP)B(T0oXHKi&6{P^~KY><)<+?<43 z5ROvz)!2rKdt37&Y2aj9bxWCcve*u)A8>;gcZ$Dtc*up>7iQ`81=}L#xWS4A>A?2K zqv}_kVo5Yu7?fZ_2U-pNHC*doz+f{7o4H?C)J;~5&`iqb`r?0&_2rT+vlJi{viyvUGgodjkfY#4RY1kiN_Q=ELd~M@?#&4l0>OynpDE~^CuYFVg zBAJD35J)1G%j=aErK2!zxpeJ9tIPW#!gZ=$8@0_#V$dGlUbpHHdomnyp55wQkr7v^ z{nE^$OiDMy!~7hSbgUepbkwpNU1BZv)0f26uz?6~mw?O%NY8S&o*-h3J4KmyC1mRG(XTxV{7+&n6UrDQcG_T5zMI#0T=Y{O}D-qaRSm&=uiy z5*qf7!CRgLC=k|;P)qE1h7GO`AMhL1cg~FzYWF$2N<-@-y+${am~B{i*M-mcaP(9< zW0Ud|d=F@;H^oVw(ztyvFMm4aXN4r|;Yd>ihMuwn-`%nWRu4$TizZg*Un8g6_n~Im z_o=45lKZUAy!*cW@CwTOJRk?3E*jiqsVA5$4apA$EPm>Vsju)p*^QMsj2O!Vqki`D zPU+#Ss~k9b;gJm%@g}Xy?h_PrLPFeXLSaV=wEgT%nG>M~J8qESQ`hP!deMDFV+VDVg;K&o2QH78?k3rKvAt&k~1vHUvnw> zvxKS?#Q(^Z+*`og4o3az7INVeErs>PR9sLog#DSc*di>s>H;xY!hJ$NCm@fq4^=tLWea z=w~SrT1OvKKKT*+R4e2iUhE@SaGO7)!Y9OxR{jwcf>=8m_@Dr4{122-&|DZ zk7IqvJ^EZ~4~u{HEQLw)21enNExrmy0ESw1q1~Ck(^G*uD>?-)T7Vxs_B-ae00M0d-AnYQ*+Ld;AfoQUm+JgrbaO1a9WVN!t3#0i~^t zlh*~YX0S2CB&x_%%XJC3w^A(?Q=Jbq+lFAj!9raaCBZ-v_RV&iM>goTCd9uoAS6e} z(kl_wmzo!}3{Rsj3}IKcutri+UpO{nR(fk7v3{auSSGe(JIemU9!Vzk-qDMg=(xfo z0!;mBPKaZK3WX8lvaroVg(ZnV%bX*G`wXt0A{QgDpAh#slEeGOE0_|FD z|Bq*nhQe(xWtjl(A3L~)U74jIfX-y!Q{;^wjzD&!!4I7ix%vA#U#X0-AUfSN28;N2O+{&MCUKN^qsayBNY*z}-iw@fkWrDxP_3xw_K*Z1wqx~Ph6^A$?g@oXe#b&N$YS2=2 zgW|aKI7Sk3Kv6obLIGkt4|-ZQ-g3Cu6tDHUQL+(P3iR|7+H@Ysn+u|jgC^Wj!}YFy z@~x1vhIh>jvm=<<4?~*|o`5mf+QCSN`qa44>=MqI>{<@Q^+V$&!c%!9qn~}{2iUkv z6S?fCbjp8%skmti`7_;+euN}mI^*Qctn<7wBaJ5Hu%W!ZngTZTcCNAvO;Tod4F>pe zl@8e^!616gOutYY@hBS;??hTrUZI3L?IL;@^iGT1I2K=J!G^pj*Mu39s0lh;unEi& z2M2qbFY|X(Ds1m5W3SFS#H20AD#W;U{+~sNamjolVsB6nd}6fUKNt8y z_V@%J?-7+(t#eaPUD!T@zhI(mg$!>9KH)J4gFB!2KJmGQ&8bZF!6MWkIv`xYAJq59 zGmb-){Y1XOEeFtWsB-2hgZb1-AEAlcrNHtbf4E~g9AtG6(Y2$iz_72r^ zAym4ylyh6D^R;d9jd=T|tm6GGHW+yzol_zJolhluV0^_}On!F?K*Stb)8`BSzm~L+@y)N)-;l;`MMh6b+faW<%OPWRfPE+xWIWz{^ z+jkHn+Haae5o-5<7v$e|oDQ7XOyfki9L$!B83yq~dkm!v{YJ-*bVELJwAB(nX-hQ! znCjVrlbTCWmyn^KnAFjs;=f;^78;w23r$w!so+CL(wAa~Lak>Cg&f3l2tf3 z3Qnu-?irO}pU$&P!E`WXFw@+#>Dx_Z&P_m9I7d^dQjcYwm!+hHR&{{R6o0813m;s=zprEvw(zNX9?Do)k*%ez*k^Z5oI z>qXy|l%OIMpqgj)gty5@<79N)`%Ma7YS7?>2;aco3Z$lGgc#MJeAAeZpR%qq`#;`3 z_bLAP+DsILVUl@hFmFj4>%DsH7YN;~Tg3tDzI{Rk6jP#=^)C`+cb|nWr7XSU9agok zDM0PkhzrG)#jKo(PoC&M2GKf7bayjROY6?n#ye0LcAaPoT_SR%l z^XoK{pY}@p>t>R!d|R_C;7)e6UzHIS2-K`9v>b3il}J3n(F;Jwg8nRpNE_AxA#~q1-s;WIGW0@|KVv*Y8<$Wxl3^J#3P-Fo=NRz{QC_Ue zo;B1^zcuN@xwmbPn1zQ?2P}Cq7u;Na+oeU3^Y|iNY(eVtfwrm7Wo(Q5 zzXdXUsg*s9|Lk4u|4L-!|AWXpI_Q9%Qd>dhH!|uO3 zy@;YSP;qeSYRum!nvK>?F4hCa7_{J8YxQv6#5Ji>Vv@b#kBH6GCT(%=R-PCxY`#S+Fn}`+?Fhps2yLi)$sa<7ohueI?G*f7FKk zr~U}TX)~2Iz`RqLX|l<$TkA}PNbNel6e-#-r@NH($mrE1D4QuDzW<&FUd3z9@xEV& zVsCQJXkXm2JfX9?BDwA#JSEWp!zGOdl&QeGgS6y6n{DlZRo;0%7beJRje1ym_1EB6>y4^?$Fh7MX zTcV@;WR0!n87mWTpZ*+*NgKK(u^6AI2VnO5ZKIAmC`JCCF0KMBsxD~bQo<5TFU!&) zEiEnGAt*`+(hbrLE7Bo#>5>-dloX@|q?BA(8U>LQ5vBf>-|zc={=Ivjotg8_oco-+ z=RSANnHl#jd4wFpigCmD#)8GEk^`n;nLc821zYcIRd=$F2%l$PshAkVK6m|+n==u# z1o}FwE0oql!*wJ}S9(ONS0i9-`*q<12chBd77yiNpW5gBc^pniOeSigg%=M)xci5# zr5$ErLEPeZyR@m3lfz#Q+LPa%*#{>EMLiiY%f{X5Bb2g`Fd|Og=b)a@Azy^ zLWliJJ%9Of@3cWF|0mAWob*SW%;u+K9%MvhdY8v&?z`R&w3r}c<^*Gl8VyCDC)obfKdpH-Emf~6k>&s_e2-_@8 zw?A3^+1YbC+;h+sg6;QBW{t+%p-OPEW}KKa=91<1y@wCG_9a@6gQ8T)-0|C_v!%xY z8mQsnE@VD^VB(1nydUPZeV)@v(!F>`W}XIyed`jT&2@-WKn5O8zu)+R+9&(#2BFRTTIWsHuK?I1vcE1+}ZKy`$Q-R*iO!DOMth%R#JszW+om=AF{j@ z7O|<08pK)HLa4uPZ=F|+ANcgbwfM`ao~fkq^!H-S`Hk=B=-t=XVU0lOa(Q{2jffLa>!$0vxIHp9fMoHEXa!BpXvOW1|pjM?%IU8^XY^g zW|sQ}TZ9UZM_}ysDYM(P1J?FccZ9$ zY)8?c81J%4D@+t^T*^Y z1;U>UYix7jY;sta zU24u8a+I7&H9U$(K?htLK7Ju$YkT<~@hXbvOj_nrezVUiAoip(b`1X6_gH8Ho4a)} zc5*h}bkOP&x&cYnOA*yPEIO^I{$7-9Vk6+!`aC$o?}-v$OOE4i!U1bd=t1h&Ypw?_HCu*UY$ZvyInOhpweM!?3>T`2* z*bZzxGd~P{GY)2Ah@?!8A;*_h-CwG!@XY^~w8PZ#d~5}5S|H2He<66F6M$op zU2l}9Zwh8S7!z~1{v6;wmofMK``FA=jF0JQW0i-xme@iEDEGsmF=kHZbO8z089AgGb`;ThWNn!d$(Y{}14p@P#hR)MVLTVJys zDfQeV8zO{_>y@Bv;;5<1shT-1D?fST7=jLd3q>ni^4PhNA68QVytk<$$wpz(kM)z$ z+nwqfFJ01B%s;0TXRc)$G*F79sV5M0aTq@useS&KyR2wx@2kZ+J3Lx24{l4Bh{zmL zfqo?n>`Kz)YBIA6ALZ{KUmuU-K*wVKs1t1C>`WxXHW<4_$$kIRclnfTUhgrknnB!Z z$IptZoco)@)NiE4^e7FhdL_2zCB&Iz>Iy1G-qRevTK4I#@#586rgIpoA|IPC($_Ut z6;vo|8DQXnW|b7gITQpKa(ehFR3V<A+9-cG@0IL@6$_>*NeZ7f{9|2D9kdJ*9XtvC*(qQcOWQsfQ@&A>uF1 z`CisHt|QIwHe6n2zJubias4o{AWn{KE3_UFYq0n={`!sYtMW|$se*Iyo@KF?Zf)J$ zFWz0Iw4T`An^5XJQ}Tn?+*2VCDrn!2{Dt$|>7!lJs0lJ;V)m9#LG1}qJN@Q2S;0y| zxqh~LJ!~YV_iY(Xz=JCEX)f64^z=-PUW;a}Y53&N{T^k1w%;VVWBYxbdHo@Kr8zqn z!TY=?y0I)5?DMjyW8Nax{HB<9y4+e7rayCQg66&Sn@K63IpSzKQjTy(+{|i1MTv(W zqXw5Lk%NS?D7GPa*;^!CwB1__RrKHPFu@xcgXav~M3r@edv*wFLI@_+@!3U8tfTj` zRyUbktncec;69ILbpFyj-@n;qg;XAN#FXkcjB*b-&XUIS##VT*eMfgQ!Y8blPvH<_ zhnR8kDM##Y$@CJbZQue_KOB7nldx!DVEG2Rf<_h?149z14Wb21S8##z7jy{Sc_s3T zY03)bnBL~}&^JXy!Z!lC7LrV}Sg zx)lG^w7JwFFCoPxW@;Xy!6tZKh>|+p%6@t`M@k%Vak>r0aQ9)bxEXV6Gcl2*i%D54 zJd~B%;4#rEewAAQ3*|7J3v`UAC7_-Wt-^H)Su2S(I#KAym=--0=+^(j?y_LE+0-cr ztECRF4M3F@aizBEO?@()QmLMH5T05XqA3YdZiKa_FdrqE!Hy}yX@JH38Im~Pa>BGM zq-ny#Z6om>4m16y(?hMEk9Zypl8(*@H@4;*hU+)QCglx73?}F&dCn27X2Rdh>lY3O z^A-d&QHGUeFT%BI6t$CTtKv3`WS!`dirJ&}d<2st1(k+gD~V2%nsH-L98?QUjEK`= zqgV!_G^o0yyv#tA%4wL7t3_)#`OXibrQWV;LkdRYh`YbNJaM&|TFAUZQo}HZJ*tC$ z!inFg>e=>*a%Me_g~THC9p;;+lt%|P*aaLrQ`!%uyDI?yj$4AO{9JVI1hpoM76~yw zN{V$xBSYc4?=-+W6xm&o&&{$wHxDO>rg}CnDNf~p~cikVp4HO zi`|o^9Nr9cIDUpGZ=_k}tLVw?;v_xIPc$!PR|$-(e$nlIPo^oiqnQ$-z|?^fK}*Z@ zJhD~-BsA~bH+egca7NlBkP!6yo7D`;`r#0lp32|RTa??fyFL5X9_E*Wm*TEAlBrf_ z*}C73Xwda9miV|~OtQCkhrNgH5^s;b`;yi9P}GE^c$ZU7lNubR;QK0rURr0TYoE8$ z`6E)hkW~8&^h7doA^7r^;psk|lyE`lsIHQ?!>0mcX1N&dp#fGIAwNl{#P2{a|xs z`g1H1um}`(&{ZShj3XlGYAu4-F>mO!eX3)R5loqEZ-?zbs+*@5-&h%eDExACq`kRP zy0rJYtTZ2wL9cwDoTp70)p*AN)r&^$k0gq(iZtq>p1P)bSK zZb6?^?Od?gouHx6yWrI?vA!K};$^D#c7_#2=?hl+%QARp|94Uo^XW9!cBU1*8*^}1 zk58N4+$a?lpBz338#CK^2Ph(Q3V7gTRF92hE=jS~*0+~@-8his=N~U7SphqUbhd4K zFg#t+I(Oy`PWFRgIkym%56Dk$)N%_D$CNWk$0X{jIVyV!;XQ>k*~r~x#zNxHi^t?@ z(&_;m9af6#&&12#yn;*3(M>o-j1K+&5t=W~5rZ`|BC+^w2PjLl{fXE^^{r+zqN=u= zFGi%m>3;Rjn_6k_(q4oOQc;I>=idId*(X+SC&Msq(LZQ3oX(O7TdX!{n>ttRexURr zJrjP9C+DZMz`I4#(nYg$i*TE%Y+GkeN1339hfeQ>S}PUo^#$R<{xt4AQw_5Z@XDxt zQ(9KBQXyvFOkawxE;%DO2ILyEOGS1QI6$;7Vy0V!^2O#KNcaP*{Ux}lVUjN@x)!%n znmII5OvU9h8$aEa^6e$G@yh~C#U_sH<+`&-7nAniyu0imHZVx{dz;mwxJKJv@ue6@ zBW!nE-w@%w#E#5wKFZ?YDsh?;yO^DKXx+!8&T=J|?<7I!C+={gQj6(UkGqv4 ztQMj)1u`di3o*B?V3IVJj=i)3zabR19d9(ZLgxhNKAjQFF4H%$Z9P5^Z}@oc=^b|0 z`}Gt(9^}eWr5t7XMPzSZAk9T4ym#;B^}VlKTBQ|vD%Vc}rgf+HPcObYF@ivWX+Pz_~_B?k%*RRQn~Vj6Up5MSSC%1kX9?8>dq~wV|%97`ZjF) zXRF-?>b~h7C>xO|5l8j)otrS_C&xDD^M0bQ{ZioR6zVNsNwG^^F={z<7O0`PHr?Rb zN%ws&o0=yM?H>yVreB)okVoIR+%cG&O+NiyG1AM4dTiatU25aj>__A--lY0Ws0ev> z%#ad(`UjFl`R&L@^7LJT>P}#425hd-J(K4p^V=sD;RsfDNUynF&u80`k6){)-ZXUL zG-F8Mv%UdkCj1 zdE+W{IvU}vb33lkAp6~0g?bWJZhEO%P%~uScwNrXw-#KxbA!r!Tm5}&W%F3m0gIj4 zZ(ASy4dt5OOzT}ktii+j;`;ClEZs|N?kuIgrL0#ql7Wvi{54$cz7sqoHzUR~^o?Vo zY-=jDB96SD&}&zB5m-THGtMe4Q++c2R<`k#lzRw*{Y`$FBrk z>h(l1BNX={9$B7Xyj+6Mwzw8CTu`F|PjGQpchJl$v4pLjwI{sY$gCMD9A?RheR=N} zJ{9Z;tkNISVIXk)8=GZ~vFRDq4;tURaZy@sMvr`)maLojqL6}dL1}3o^9*-LG)RG* z6tkj~DN?Ti0Xm}}5j6pac3{`&0SZPq{pH_wlF#wrJ2w*k~1JyF1MKnPAEDQWvmiaZSp3GwRK1%+P=+`a}37XAyo1Nx?VMaCiF@c%Ym zFfc@~;nOv)K$4FMflVYk{MusvHDHm}UmzI(U4a0W^Dxkb-j%4yg7!*OWD6nqN2Ms3 z$mpt3YXKJ+wIu@x7dSz9CVxe&@axMS|7iUgb4wfaB8V(-3q=I#u>tO&BtVBYz_OqcF1K&_uAl|3IN-`YKUz7#)x?GW?LBvReI)odx_Qu>s;3LO^^OLUYYgyr#DY94!9hHqrod zOAw%883s}dy)yBASr~rpZSk7wX*e2m7tmgZ0Kf_iWEgj)b32;`*jeEO86;k@GOus| zp_i0bDBD#S$Rzn+apkHw$T{s_R<)Sq3MIG(136^;6_Z~5)4*l__vYAS|7|1yme#~U z4_{uf((4kSvivI&QBC*9($&-j#@8u9>qTfe6M(k|AcLE6; z`k<%X0OPpi)!=`b_`iM0_X0mAB>ptOug~%SMxrQ~vL8L;naQzn{s{Te2Q~W5+NS>x F_djzcFxCJ7 diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties index e0c4de36dddb..8cf6eb5ad222 100644 --- a/gradle/wrapper/gradle-wrapper.properties +++ b/gradle/wrapper/gradle-wrapper.properties @@ -1,5 +1,5 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-5.5-all.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-6.8.3-all.zip zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists diff --git a/gradlew b/gradlew index cccdd3d517fc..4f906e0c811f 100755 --- a/gradlew +++ b/gradlew @@ -1,5 +1,21 @@ #!/usr/bin/env sh +# +# Copyright 2015 the original author or authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + ############################################################################## ## ## Gradle start up script for UN*X @@ -28,7 +44,7 @@ APP_NAME="Gradle" APP_BASE_NAME=`basename "$0"` # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. -DEFAULT_JVM_OPTS="" +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' # Use the maximum available, or set MAX_FD != -1 to use that value. MAX_FD="maximum" @@ -66,6 +82,7 @@ esac CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + # Determine the Java command to use to start the JVM. if [ -n "$JAVA_HOME" ] ; then if [ -x "$JAVA_HOME/jre/sh/java" ] ; then @@ -109,10 +126,11 @@ if $darwin; then GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" fi -# For Cygwin, switch paths to Windows format before running java -if $cygwin ; then +# For Cygwin or MSYS, switch paths to Windows format before running java +if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then APP_HOME=`cygpath --path --mixed "$APP_HOME"` CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + JAVACMD=`cygpath --unix "$JAVACMD"` # We build the pattern for arguments to be converted via cygpath @@ -138,19 +156,19 @@ if $cygwin ; then else eval `echo args$i`="\"$arg\"" fi - i=$((i+1)) + i=`expr $i + 1` done case $i in - (0) set -- ;; - (1) set -- "$args0" ;; - (2) set -- "$args0" "$args1" ;; - (3) set -- "$args0" "$args1" "$args2" ;; - (4) set -- "$args0" "$args1" "$args2" "$args3" ;; - (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; - (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; - (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; - (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; - (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + 0) set -- ;; + 1) set -- "$args0" ;; + 2) set -- "$args0" "$args1" ;; + 3) set -- "$args0" "$args1" "$args2" ;; + 4) set -- "$args0" "$args1" "$args2" "$args3" ;; + 5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + 6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + 7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + 8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + 9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; esac fi @@ -159,14 +177,9 @@ save () { for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done echo " " } -APP_ARGS=$(save "$@") +APP_ARGS=`save "$@"` # Collect all arguments for the java command, following the shell quoting and substitution rules eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" -# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong -if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then - cd "$(dirname "$0")" -fi - exec "$JAVACMD" "$@" diff --git a/gradlew.bat b/gradlew.bat index e95643d6a2ca..ac1b06f93825 100644 --- a/gradlew.bat +++ b/gradlew.bat @@ -1,3 +1,19 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + @if "%DEBUG%" == "" @echo off @rem ########################################################################## @rem @@ -13,15 +29,18 @@ if "%DIRNAME%" == "" set DIRNAME=. set APP_BASE_NAME=%~n0 set APP_HOME=%DIRNAME% +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. -set DEFAULT_JVM_OPTS= +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" @rem Find java.exe if defined JAVA_HOME goto findJavaFromJavaHome set JAVA_EXE=java.exe %JAVA_EXE% -version >NUL 2>&1 -if "%ERRORLEVEL%" == "0" goto init +if "%ERRORLEVEL%" == "0" goto execute echo. echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. @@ -35,7 +54,7 @@ goto fail set JAVA_HOME=%JAVA_HOME:"=% set JAVA_EXE=%JAVA_HOME%/bin/java.exe -if exist "%JAVA_EXE%" goto init +if exist "%JAVA_EXE%" goto execute echo. echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% @@ -45,28 +64,14 @@ echo location of your Java installation. goto fail -:init -@rem Get command-line arguments, handling Windows variants - -if not "%OS%" == "Windows_NT" goto win9xME_args - -:win9xME_args -@rem Slurp the command line arguments. -set CMD_LINE_ARGS= -set _SKIP=2 - -:win9xME_args_slurp -if "x%~1" == "x" goto execute - -set CMD_LINE_ARGS=%* - :execute @rem Setup the command line set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + @rem Execute Gradle -"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* :end @rem End local scope for the variables with windows NT shell