Skip to content

Commit

Permalink
Added docs
Browse files Browse the repository at this point in the history
Signed-off-by: Stefano Savare <[email protected]>
  • Loading branch information
deatinor committed Jul 14, 2021
1 parent 9aaeb26 commit f71ed0b
Show file tree
Hide file tree
Showing 4 changed files with 7 additions and 36 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
- main
paths:
- 'README.rst'
- 'docs/*'
- 'docs/source/*'
- '.github/workflows/documentation.yml'


Expand Down
4 changes: 3 additions & 1 deletion docs/source/aws/terraform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@ Terraform
==========

We currently use `Terraform <https://www.terraform.io>`_ to configure the infrastructure.
`This <https://github.com/the-deep/dfs-deepl-terraform>`_ is the repository.

What is on terraform?
---------------------

- All S3 buckets, except the Terraform state
- The Sagemaker IAM. We do not store on Terraform the rest of the IAM
- The Sagemaker Notebook Instances and Github repository
- The Streamlit ECR repository
- The test environments ECR repository
- The MLFlow server and ECR repository

What is not on Terraform?
-------------------------
Expand Down
35 changes: 2 additions & 33 deletions docs/source/modeling/testing_environment.rst
Original file line number Diff line number Diff line change
@@ -1,37 +1,6 @@
Testing Environment
===================

We have built a simple online test environment to check the predictions of our models on the subpillars.
You can find it `here <http://test-env.eba-crsiq2wb.us-east-1.elasticbeanstalk.com>`_.
We have a dedicated repository dedicated to our offline and online test environment.
You can find it `here <https://github.com/the-deep/deep-test-environment>`_

It is useful to show to externals our results. We plan to add more features to it.

We have used Streamlit as Python library, and Docker + ECR + Beanstalk as deployment option.

Deployment
-----------------------

We use the Github CI to build the test environment image and push it to ECR.
We then use Beanstalk to serve it.

Building the image
~~~~~~~~~~~~~~~~~~

The image is automatically built via the CI.
If you want to change the code, check ``docker/Dockerfile`` and change the variables
``ENTRYPOINT`` and ``CMD``.
Currently the CI downloads the data from
``s3://<DEV_BUCKET>/test-environment/data`` to ``data/streamlit``. If you want to change
the input data, look at that.

Pushing to ECR
~~~~~~~~~~~~~~

It is fully automated and a priori you should not do anything

Update the image on Beanstalk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This step is not automated, and you should do it **every time you want to serve a new image**.
Go in Beanstalk -> Environments -> Test env -> Upload and deploy.
Upload the file ``docker/Dockerrun.aws.json``.
2 changes: 1 addition & 1 deletion requirements/test-env-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
streamlit==0.82.0
streamlit==0.84.1
pdfminer==20191125
nltk==3.6.2
pandas==1.2.4
Expand Down

0 comments on commit f71ed0b

Please sign in to comment.