diff --git a/DRAFT_RELEASE_NOTES.md b/DRAFT_RELEASE_NOTES.md index 0018b04a3..ef16ee5da 100644 --- a/DRAFT_RELEASE_NOTES.md +++ b/DRAFT_RELEASE_NOTES.md @@ -13,7 +13,7 @@ The baseline Kafka Docker image has moved from using the _wurstmeister/kafka_ im * Python modules were renamed to reflect aiSSEMBLE. These include the following. | Old Python Module | New Python Module | -| --------------------------------------- |---------------------------------------------------| +|-----------------------------------------|---------------------------------------------------| | foundation-core-python | aissemble-core-python | | foundation-model-training-api | aissemble-foundation-model-training-api | | foundation-versioning-service | aissemble-foundation-versioning-service | @@ -26,7 +26,7 @@ The baseline Kafka Docker image has moved from using the _wurstmeister/kafka_ im | foundation-transform-core-python | aissemble-Foundation-transform-core-python | | extensions-model-training-api-sagemaker | aissemble-extensions-model-training-api-sagemaker | | extensions-data-delivery-spark-py | aissemble-extensions-data-delivery-spark-py | -| extensions-encryption-valut-python | aissemble-extensions-encryption-vault-python | +| extensions-encryption-vault-python | aissemble-extensions-encryption-vault-python | | extensions-transform-spark-python | aissemble-extensions-transform-spark-python | | test-data-delivery-pyspark-model | aissemble-test-data-delivery-pyspark-model | | test-data-delivery-pyspark-model-basic | aissemble-test-data-delivery-pyspark-model-basic | @@ -89,18 +89,20 @@ The following steps will upgrade your project to 1.7. These instructions consist ## Automatic Upgrades To reduce burden of upgrading aiSSEMBLE, the Baton project is used to automate the migration of some files to the new version. These migrations run automatically when you build your project, and are included by default when you update the `build-parent` version in your root POM. Below is a description of all of the Baton migrations that are included with this version of aiSSEMBLE. -| Migration Name | Description | -|-------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| upgrade-tiltfile-aissemble-version-migration | Updates the aiSSEMBLE version within your project's Tiltfile | -| upgrade-v2-chart-files-aissemble-version-migration | Updates the helm chart dependencies within your project's deployment resources (-deploy/src/main/resources/apps/) to use the latest version of the aiSSEMBLE | -| upgrade-v1-chart-files-aissemble-version-migration | Updates the docker image tags within your project's deployment resources (-deploy/src/main/resources/apps/) to use the latest version of the aiSSEMBLE | -| upgrade-mlflow-v2-external-s3-migration | Update the mlflow V2 deployment (if present) in your project to utilize Localstack for local development and SealedSecrets for remote deployments | -| upgrade-spark-application-s3-migration | Update the pipeline SparkApplication(s) (if present) in your project to utilize Localstack for local development and SealedSecrets for remote deployments | -| upgrade-foundation-extension-python-package-migration | Updates the pyproject.toml files within your projects pipelines folder (-pipelines) to use the updated aiSSEMBLE foundation and extension Python packages with the latest naming convention | -| upgrade-helm-chart-files-names-migration | Updates the Chart.yaml and values*.yaml files within your project's deploy folder (-deploy) to use the new Helm chart naming convention (aissemble-\-chart) | -| upgrade-dockerfile-pip-install-migration | Updates dockerfiles such that python dependency installations fail during the build, rather than at runtime | -| enable-habushu-build-cache-migration | Updates the `pom.xml` file for any Habushu-managed modules to ensure that the build directory is specified. | -| data-lineage-package-import-migration | Updates the package imports for all java files that are referencing `com.boozallen.aissemble.data.lineage`. +| Migration Name | Description | +|-------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| upgrade-tiltfile-aissemble-version-migration | Updates the aiSSEMBLE version within your project's Tiltfile | +| upgrade-v2-chart-files-aissemble-version-migration | Updates the helm chart dependencies within your project's deployment resources (-deploy/src/main/resources/apps/) to use the latest version of the aiSSEMBLE | +| upgrade-v1-chart-files-aissemble-version-migration | Updates the docker image tags within your project's deployment resources (-deploy/src/main/resources/apps/) to use the latest version of the aiSSEMBLE | +| upgrade-mlflow-v2-external-s3-migration | Update the mlflow V2 deployment (if present) in your project to utilize Localstack for local development and SealedSecrets for remote deployments | +| upgrade-spark-application-s3-migration | Update the pipeline SparkApplication(s) (if present) in your project to utilize Localstack for local development and SealedSecrets for remote deployments | +| upgrade-foundation-extension-python-package-migration | Updates the pyproject.toml files within your projects pipelines folder (-pipelines) to use the updated aiSSEMBLE foundation and extension Python packages with the latest naming convention | +| upgrade-helm-chart-names-migration | Updates the Chart.yaml and values*.yaml files within your project's deploy folder (-deploy) to use the new Helm chart naming convention (`aissemble--chart`). | +| upgrade-helm-module-names-migration | Updates the Chart.yaml and values*.yaml files within your project's deploy folder (-deploy) to use the new Helm module naming convention (`aissemble--chart`) | +| upgrade-helm-chart-repository-url-migration | Updates the Helm repository URL within your project's deploy Chart.yaml file to point to ghcr.io. Only runs if the previous Helm chart repository URL is passed in through the `oldHelmRepositoryUrl` system property (using `-DoldHelmRepositoryUrl`) | +| upgrade-dockerfile-pip-install-migration | Updates dockerfiles such that python dependency installations fail during the build, rather than at runtime | +| enable-habushu-build-cache-migration | Updates the `pom.xml` file for any Habushu-managed modules to ensure that the build directory is specified. | +| data-lineage-package-import-migration | Updates the package imports for all java files that are referencing `com.boozallen.aissemble.data.lineage`. | To deactivate any of these migrations, add the following configuration to the `baton-maven-plugin` within your root `pom.xml`: diff --git a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmNameMigration.java b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmNameMigration.java new file mode 100644 index 000000000..bc0bd53c4 --- /dev/null +++ b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmNameMigration.java @@ -0,0 +1,61 @@ +package com.boozallen.aissemble.upgrade.migration; + +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.util.FileUtils; +import org.apache.commons.collections4.CollectionUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.technologybrewery.baton.BatonException; + +import java.io.File; +import java.io.IOException; +import java.util.List; + +import static com.boozallen.aissemble.upgrade.util.FileUtils.getRegExCaptureGroups; + +/** + * Baton migration used to migrate the Helm Chart names and/or Helm module names in the Chart.yaml and values.yaml files within a project's deploy folder + */ +public abstract class AbstractHelmNameMigration extends AbstractAissembleMigration{ + private static final Logger logger = LoggerFactory.getLogger(AbstractHelmNameMigration.class); + + @Override + protected boolean shouldExecuteOnFile(File file) { + boolean shouldExecute = false; + try { + List namesToUpdate = getRegExCaptureGroups(getReplacementRegex(file), file); + shouldExecute = CollectionUtils.isNotEmpty(namesToUpdate); + } catch (IOException e){ + throw new BatonException("Unable to determine if Helm chart migration must be executed", e); + } + return shouldExecute; + } + + @Override + protected boolean performMigration(File file) { + boolean performedSuccessfully = false; + try { + performedSuccessfully = FileUtils.replaceInFile( + file, + getReplacementRegex(file), + getReplacementText(file) + ); + } catch (Exception e) { + logger.error("Unable to perform Helm chart migration due to exception", e); + } + return performedSuccessfully; + } + + public abstract String getReplacementRegex(File file); + public abstract String getReplacementText(File file); + +} \ No newline at end of file diff --git a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmRepositoryMigration.java b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmRepositoryMigration.java new file mode 100644 index 000000000..199283df6 --- /dev/null +++ b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/AbstractHelmRepositoryMigration.java @@ -0,0 +1,67 @@ +package com.boozallen.aissemble.upgrade.migration; + +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.util.FileUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import java.io.File; + +/** + * Baton migration used to migrate the Helm Chart repository URL in the Chart.yaml within a project's deploy folder. + * Migration is triggered by setting the oldHelmRepositoryUrl system property. + */ +public abstract class AbstractHelmRepositoryMigration extends AbstractAissembleMigration{ + public static final Logger logger = LoggerFactory.getLogger(AbstractHelmRepositoryMigration.class); + private static final String OLD_HELM_REPO_PROPERTY = "oldHelmRepositoryUrl"; + + protected String getOldHelmRepositoryUrl(){ + String oldHelmRepositoryUrl = System.getProperty(OLD_HELM_REPO_PROPERTY); + if (oldHelmRepositoryUrl!= null){ + return oldHelmRepositoryUrl; + } else { + logger.info("oldHelmRepositoryUrl system property not set. Skipping helm chart repository URL migration."); + return null; + } + } + + /** + * Function that checks if user has provided system property containing old Helm repository URL + * If yes, then Helm Charts repository URL migration will be run. + * Otherwise, migration is skipped. + */ + @Override + protected boolean shouldExecuteOnFile(File file) { + boolean shouldExecute = false; + if (getOldHelmRepositoryUrl() != null){ + shouldExecute = true; + } + return shouldExecute; + } + + @Override + protected boolean performMigration(File file) { + boolean performedSuccessfully = false; + try { + performedSuccessfully = FileUtils.replaceInFile( + file, + getReplacementRegex(), + getReplacementText() + ); + } catch (Exception e) { + logger.error("Unable to perform helm chart repository URL migration due to exception", e); + } + return performedSuccessfully; + } + + public abstract String getReplacementRegex(); + public abstract String getReplacementText(); +} diff --git a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsModuleMigration.java b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsModuleMigration.java new file mode 100644 index 000000000..dc4a1dc4a --- /dev/null +++ b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsModuleMigration.java @@ -0,0 +1,36 @@ +package com.boozallen.aissemble.upgrade.migration.v1_7_0; + +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.migration.AbstractHelmNameMigration; + +import java.io.File; + +/** + * Baton migration used to migrate the Chart.yaml and values*.yaml files within a project's deploy folder + * to use new Helm module names + * Example: extensions-helm-kafka to aissemble-kafka-chart + */ +public class HelmChartsModuleMigration extends AbstractHelmNameMigration { + private static final String moduleNameReplaceRegex = "extensions-helm-([^\\/\\n:]+)"; + private static final String AISSEMBLE_PREFIX = "aissemble-"; + private static final String CHART_POSTFIX = "-chart"; + + @Override + public String getReplacementRegex(File file) { + return moduleNameReplaceRegex; + } + + @Override + public String getReplacementText(File file) { + return AISSEMBLE_PREFIX.concat(FIRST_REGEX_GROUPING + CHART_POSTFIX); + } +} diff --git a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsNameMigration.java b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsNameMigration.java index d49d8dcfe..5de18eca8 100644 --- a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsNameMigration.java +++ b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsNameMigration.java @@ -9,25 +9,18 @@ * This software package is licensed under the Booz Allen Public License. All Rights Reserved. * #L% */ - -import com.boozallen.aissemble.upgrade.migration.AbstractAissembleMigration; -import com.boozallen.aissemble.upgrade.util.FileUtils; -import org.apache.commons.collections4.CollectionUtils; +import com.boozallen.aissemble.upgrade.migration.AbstractHelmNameMigration; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.technologybrewery.baton.BatonException; - import java.io.File; -import java.io.IOException; -import java.util.List; -import static com.boozallen.aissemble.upgrade.util.FileUtils.getRegExCaptureGroups; /** * Baton migration used to migrate the Chart.yaml and values*.yaml files within a project's deploy folder * to use the new Helm chart naming convention (aissemble--chart) + * Example: aissemble-kafka to aissemble-kafka-chart */ -public class HelmChartsNameMigration extends AbstractAissembleMigration { +public class HelmChartsNameMigration extends AbstractHelmNameMigration { public static final Logger logger = LoggerFactory.getLogger(HelmChartsNameMigration.class); private static final String dependencyNameReplaceRegex = "(aissemble-[^\\/\\n:]+)(? dependencyNames = getRegExCaptureGroups(dependencyNameReplaceRegex, file); - shouldExecute = CollectionUtils.isNotEmpty(dependencyNames); - } else { - List appNames = getRegExCaptureGroups(appNameReplaceRegex, file); - shouldExecute = CollectionUtils.isNotEmpty(appNames); - } - } catch (IOException e) { - throw new BatonException("Unable to determine if Helm charts should be migrated!", e); - } + public String getDependencyNameReplacementText(){ + return FIRST_REGEX_GROUPING.concat(CHART_POSTFIX); + } - if (shouldExecute) { - logger.info("Found Helm chart file using old naming convention. Migrating file: {}", file.getName()); - } - return shouldExecute; + public String getAppNameReplacementText(){ + return FIRST_REGEX_GROUPING.concat(SECOND_REGEX_GROUPING + CHART_POSTFIX + COLON); } - /** - * Performs the migration if the shouldExecuteOnFile() returns true. - * @param file file to migrate - * @return performedSuccessfully - Whether the file was migrated successfully. - */ @Override - protected boolean performMigration(File file) { - boolean performedSuccessfully = false; - - try { - - String appNameReplacementText = FIRST_REGEX_GROUPING.concat(SECOND_REGEX_GROUPING + CHART_POSTFIX + COLON); - String dependencyNameReplacementText = FIRST_REGEX_GROUPING.concat(CHART_POSTFIX); - - if (file.getName().matches("Chart.yaml")){ - performedSuccessfully = FileUtils.replaceInFile( - file, - dependencyNameReplaceRegex, - dependencyNameReplacementText - ); - } else { - performedSuccessfully = FileUtils.replaceInFile( - file, - appNameReplaceRegex, - appNameReplacementText - ); - } - } catch (Exception e) { - logger.error("Unable to perform helm chart name migration due to exception", e); + public String getReplacementRegex(File file) { + if (file.getName().matches("Chart.yaml")){ + return dependencyNameReplaceRegex; + } else { + return appNameReplaceRegex; } - - return performedSuccessfully; } + @Override + public String getReplacementText(File file){ + if (file.getName().matches("Chart.yaml")){ + return getDependencyNameReplacementText(); + } else { + return getAppNameReplacementText(); + } + } } + + diff --git a/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsRepositoryUrlMigration.java b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsRepositoryUrlMigration.java new file mode 100644 index 000000000..46f8da278 --- /dev/null +++ b/foundation/foundation-upgrade/src/main/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/HelmChartsRepositoryUrlMigration.java @@ -0,0 +1,32 @@ +package com.boozallen.aissemble.upgrade.migration.v1_7_0; + +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.migration.AbstractHelmRepositoryMigration; + +/** + * Baton migration used to migrate the Chart.yaml within a project's deploy folder + * to use the new Helm chart repository URL + * Example: https://old-helm-repository-url.com/ to oci://ghcr.io/boozallen + */ + +public class HelmChartsRepositoryUrlMigration extends AbstractHelmRepositoryMigration { + private static final String NEW_HELM_REPO_URL = "oci://ghcr.io/boozallen"; + @Override + public String getReplacementRegex() { + return getOldHelmRepositoryUrl(); + } + + @Override + public String getReplacementText() { + return NEW_HELM_REPO_URL; + } +} diff --git a/foundation/foundation-upgrade/src/main/resources/migrations.json b/foundation/foundation-upgrade/src/main/resources/migrations.json index 7d24d238d..48115a5c4 100644 --- a/foundation/foundation-upgrade/src/main/resources/migrations.json +++ b/foundation/foundation-upgrade/src/main/resources/migrations.json @@ -45,7 +45,7 @@ ] }, { - "name": "upgrade-helm-chart-files-names-migration", + "name": "upgrade-helm-chart-names-migration", "implementation": "com.boozallen.aissemble.upgrade.migration.v1_7_0.HelmChartsNameMigration", "fileSets": [ { @@ -56,6 +56,29 @@ } ] }, + { + "name": "upgrade-helm-module-names-migration", + "implementation": "com.boozallen.aissemble.upgrade.migration.v1_7_0.HelmChartsModuleMigration", + "fileSets": [ + { + "includes": [ + "*-deploy/src/main/resources/apps/*/Chart.yaml", + "*-deploy/src/main/resources/apps/*/values*.yaml" + ] + } + ] + }, + { + "name": "upgrade-helm-chart-repository-url-migration", + "implementation": "com.boozallen.aissemble.upgrade.migration.v1_7_0.HelmChartsRepositoryUrlMigration", + "fileSets": [ + { + "includes": [ + "*-deploy/src/main/resources/apps/*/Chart.yaml" + ] + } + ] + }, { "name": "enable-habushu-build-cache-migration", "implementation": "com.boozallen.aissemble.upgrade.migration.v1_7_0.EnableDistOutputFolderForHabushuBuildCacheMigration", diff --git a/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartModuleMigrationSteps.java b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartModuleMigrationSteps.java new file mode 100644 index 000000000..d04318243 --- /dev/null +++ b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartModuleMigrationSteps.java @@ -0,0 +1,90 @@ +package com.boozallen.aissemble.upgrade.migration.v1_7_0; +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.migration.AbstractMigrationTest; +import io.cucumber.java.en.Given; +import io.cucumber.java.en.Then; +import io.cucumber.java.en.When; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; + +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +public class ChartModuleMigrationSteps extends AbstractMigrationTest { + @Given("a value file using the older Helm chart module name") + public void a_value_file_using_the_older_helm_chart_module_name() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/migration/kafka-cluster/values.yaml"); + + } + + @When("the Helm module name change migration executes") + public void the_helm_module_name_change_migration_executes() { + performMigration(new HelmChartsModuleMigration()); + } + + + @Then("the value file is migrated to use new module name") + public void the_value_file_is_migrated_to_use_new_module_name(){ + assertTrue(doesNotContainOldModuleName(testFile)); + } + + @Given("a Chart file using the older Helm chart module name") + public void a_chart_file_using_the_older_helm_chart_module_name() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml"); + } + + @Then("the Chart file is migrated to use new module name") + public void the_chart_file_is_migrated_to_use_new_module_name() { + assertTrue(doesNotContainOldModuleName(testFile)); + } + + @Given("a value file using the new Helm chart module naming convention") + public void a_value_file_using_the_new_helm_chart_module_naming_convention() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/values.yaml"); + } + + @Given("a Chart file using the new Helm chart module naming convention") + public void a_chart_file_using_the_new_helm_chart_module_naming_convention() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/values.yaml"); + + } + + @Then("the value file module name migration is skipped") + public void the_value_file_module_name_migration_is_skipped() { + assertFalse("Value file is already using new Helm module name", shouldExecute); + + } + + @Then("the Chart file module name migration is skipped") + public void the_chart_file_module_name_migration_is_skipped() { + assertFalse("Chart file is already using new Helm module name", shouldExecute); + } + + private static boolean doesNotContainOldModuleName(File file) { + try (BufferedReader reader = new BufferedReader(new FileReader(String.valueOf(file)))) { + String line; + while ((line = reader.readLine()) != null) { + if (line.contains("extensions-helm-kafka")) { + return false; + } + } + } catch (IOException e){ + fail(String.format("unable to parse yaml file due to exception: %s", e)); + } + return true; + } +} + diff --git a/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartNameMigrationSteps.java b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartNameMigrationSteps.java index 930875fb5..225a06a6d 100644 --- a/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartNameMigrationSteps.java +++ b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartNameMigrationSteps.java @@ -11,13 +11,14 @@ import com.boozallen.aissemble.upgrade.migration.AbstractMigrationTest; import com.boozallen.aissemble.upgrade.migration.utils.MigrationTestUtils; -import com.boozallen.aissemble.upgrade.migration.v1_7_0.HelmChartsNameMigration; import io.cucumber.java.en.Given; import io.cucumber.java.en.Then; import io.cucumber.java.en.When; + import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; + import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; diff --git a/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartUrlMigrationSteps.java b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartUrlMigrationSteps.java new file mode 100644 index 000000000..4f8903876 --- /dev/null +++ b/foundation/foundation-upgrade/src/test/java/com/boozallen/aissemble/upgrade/migration/v1_7_0/ChartUrlMigrationSteps.java @@ -0,0 +1,87 @@ +package com.boozallen.aissemble.upgrade.migration.v1_7_0; +/*- + * #%L + * aiSSEMBLE::Foundation::Upgrade + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +import com.boozallen.aissemble.upgrade.migration.AbstractMigrationTest; +import com.boozallen.aissemble.upgrade.migration.utils.MigrationTestUtils; +import io.cucumber.java.After; +import io.cucumber.java.en.And; +import io.cucumber.java.en.Given; +import io.cucumber.java.en.Then; +import io.cucumber.java.en.When; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; + +import static org.junit.Assert.*; + + +public class ChartUrlMigrationSteps extends AbstractMigrationTest { + private static final String NEW_HELM_REPO_URL = "oci://ghcr.io/boozallen"; + private static final String OLD_HELM_REPO_URL = "https://old-helm-repository-url.com/"; + + @After + public void tearDown(){ + System.clearProperty("oldHelmRepositoryUrl"); + } + + @Given("a Chart file referencing the older Helm repository URL") + public void a_chart_file_referencing_the_older_helm_repository_url() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml"); + } + @Given("a Chart file") + public void a_chart_file() { + testFile = getTestFile("v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/Chart.yaml"); + } + + @Given("the old Helm repository URL is provided through the system property") + public void the_old_helm_repository_url_is_provided_through_the_system_property() { + System.setProperty("oldHelmRepositoryUrl", OLD_HELM_REPO_URL); + } + + @Given("the old Helm repository URL is not provided through the system property") + public void the_old_helm_repository_url_is_not_provided_through_the_system_property() { + System.clearProperty("oldHelmRepositoryUrl"); + assertNull(System.getProperty("oldHelmRepositoryUrl")); + } + + @When("the Helm chart url migration executes") + public void the_helm_chart_name_change_migration_executes() { + performMigration(new HelmChartsRepositoryUrlMigration()); + } + + @Then("the Chart file is updated to use new Helm repository URL") + public void the_chart_file_is_updated_to_use_new_helm_repository_url() { + HashMap chartFileContents; + try { + chartFileContents = MigrationTestUtils.extractYamlContents(testFile); + Object dependenciesObject = chartFileContents.get("dependencies"); + ArrayList> dependencies = (ArrayList>) dependenciesObject; + + for (HashMap dependency : dependencies) { + String repositoryUrl = (String) dependency.get("repository"); + assertTrue(repositoryUrl.matches(NEW_HELM_REPO_URL)); + } + } catch (IOException e) { + fail(String.format("Unable to parse yaml contents due to exception: %s", e)); + } + } + + @Then("the Chart file is not updated to use new Helm repository URL") + public void the_chart_file_is_not_updated_to_use_new_helm_repository_url() { + assertFalse("Old Helm chart repository URL not passed through system property", shouldExecute); + } + +} + diff --git a/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-module-migration.feature b/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-module-migration.feature new file mode 100644 index 000000000..b9aec7118 --- /dev/null +++ b/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-module-migration.feature @@ -0,0 +1,23 @@ +@chart-module-change-migration +Feature: As an aiSSEMBLE user, I want my aiSSEMBLE Helm module names updated to the latest name automatically + + Scenario: Upgrade aiSSEMBLE Helm chart value file to use to latest module name + Given a value file using the older Helm chart module name + When the Helm module name change migration executes + Then the value file is migrated to use new module name + + Scenario: Upgrade aiSSEMBLE Helm chart Chart file to use to latest module name + Given a Chart file using the older Helm chart module name + When the Helm module name change migration executes + Then the Chart file is migrated to use new module name + + Scenario: Skipping Value file migration if values file already uses new module name + Given a value file using the new Helm chart module naming convention + When the Helm module name change migration executes + Then the value file module name migration is skipped + + Scenario: Skipping Chart file migration if Chart file already uses new module name + Given a Chart file using the new Helm chart module naming convention + When the Helm module name change migration executes + Then the Chart file module name migration is skipped + diff --git a/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-url-migration.feature b/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-url-migration.feature new file mode 100644 index 000000000..5c18789cc --- /dev/null +++ b/foundation/foundation-upgrade/src/test/resources/specifications/v1_7_0/chart-url-migration.feature @@ -0,0 +1,14 @@ +@chart-url-change-migration +Feature: As an aiSSEMBLE user, I want my aiSSEMBLE Helm charts updated to the latest Helm chart repository URL + + Scenario: Upgrade aiSSEMBLE Helm chart Chart file to use new helm repository URL + Given a Chart file referencing the older Helm repository URL + And the old Helm repository URL is provided through the system property + When the Helm chart url migration executes + Then the Chart file is updated to use new Helm repository URL + + Scenario: Skipping Helm Repository URL migration if previous URL not provided through system property + Given a Chart file + And the old Helm repository URL is not provided through the system property + When the Helm chart url migration executes + Then the Chart file is not updated to use new Helm repository URL \ No newline at end of file diff --git a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml index 57605cba7..72db525cf 100644 --- a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml +++ b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/Chart.yaml @@ -31,4 +31,4 @@ appVersion: "1.0.0" dependencies: - name: aissemble-kafka version: 1.7.0-SNAPSHOT - repository: oci://ghcr.io/boozallen \ No newline at end of file + repository: https://old-helm-repository-url.com/ \ No newline at end of file diff --git a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/values.yaml b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/values.yaml index db03236b7..f0284676d 100644 --- a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/values.yaml +++ b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/migration/kafka-cluster/values.yaml @@ -1,5 +1,6 @@ ######################################## ## CONFIG | Kafka Configs +## See aiSSEMBLE extensions-helm-kafka: README for more details. ######################################## aissemble-kafka: kafka: diff --git a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/Chart.yaml b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/Chart.yaml index 4a15e27ca..304f8a34c 100644 --- a/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/Chart.yaml +++ b/foundation/foundation-upgrade/src/test/resources/test-files/v1_7_0/HelmChartNameMigration/skip-migration/kafka-cluster/Chart.yaml @@ -31,4 +31,4 @@ appVersion: "1.0.0" dependencies: - name: aissemble-kafka-chart version: 1.7.0-SNAPSHOT - repository: oci://ghcr.io/boozallen \ No newline at end of file + repository: https://old-helm-repository-url.com/ \ No newline at end of file