fix env var read default values from config (#12)

# Pull Request

## Checklist
- [x] Have you read Digital Catapult's [Code of
Conduct](https://github.com/digicatapult/.github/blob/main/CODE_OF_CONDUCT.md)?
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have made corresponding changes to the documentation.
- [x] My changes generate no new warnings.
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [x] New and existing unit tests pass locally with my changes.

## PR Type


- [x] Bug Fix


## Linked tickets

<!-- If this PR is linked to a JIRA ticket or Github Issue, please
provide the ticket URL here. -->

## High level description

Fix default env var read from config file

## Detailed description

<!-- A detailed description of the feature and if possible describe how
has been implemented. -->


## Describe alternatives you've considered

<!-- A clear and concise description of the alternative solutions you've
considered. Be sure to explain why the existing customisability isn't
suitable for this feature. -->

## Operational impact

<!--- A description of any operational considerations associated with
the change. Is there anything in particular we should be looking at when
deploying the change to make sure it is working as intended. If
something goes wrong will any special actions be needed to revert the
change. -->

## Additional context

<!-- Add any other context or screenshots about the feature request
here. -->

config.yaml

@@ -33,4 +33,4 @@ dvc:
   dvc_remote: "s3://artifacts"   # remote s3 bucket path for dvc to push and store data
   dvc_remote_name: "regression-model-remote"    # a name assigned to the remote
   dvc_endpoint_url: "http://minio"  # dvc endpoint url
-  dvc_region: "us-east-1"    # dvc region - used for s3, just a placeholder for minio
+  dvc_region: "eu-west-2"    # dvc region - used for s3, just a placeholder for minio

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "bridgeai-regression-model-training"
-version = "0.6.0"
+version = "0.6.1"
 description = "A basic containerised regression model training setup"
 authors = ["DC AI Team <[email protected]>"]
 readme = "README.md"

src/fetch_data.py

@@ -60,9 +60,10 @@ def dvc_pull(config):
     delete_file_if_exists(config["dvc"]["train_data_path"])
     delete_file_if_exists(config["dvc"]["test_data_path"])
     delete_file_if_exists(config["dvc"]["val_data_path"])
+    dvc_remote = os.getenv("DVC_REMOTE", config["dvc"]["dvc_remote"])
     try:
         dvc_remote_add(config)
-        dvc_main(["pull", "-r", config["dvc"]["dvc_remote_name"]])
+        dvc_main(["pull", "-r", dvc_remote])
     except Exception as e:
         logger.error(f"DVC pull failed with error: {e}")
         raise e
@@ -75,11 +76,11 @@ def dvc_remote_add(config):
     region = os.getenv("AWS_DEFAULT_REGION")
     try:
         dvc_remote_name = os.getenv(
-            "DVC_REMOTE_NAME", config["dvc_remote_name"]
+            "DVC_REMOTE_NAME", config["dvc"]["dvc_remote_name"]
         )
-        dvc_remote = os.getenv("DVC_REMOTE", config["dvc_remote"])
+        dvc_remote = os.getenv("DVC_REMOTE", config["dvc"]["dvc_remote"])
         dvc_endpoint_url = os.getenv(
-            "DVC_ENDPOINT_URL", config["dvc_endpoint_url"]
+            "DVC_ENDPOINT_URL", config["dvc"]["dvc_endpoint_url"]
         )
 
         dvc_main(["remote", "add", "-f", dvc_remote_name, dvc_remote])