Replace Lambda Layers with container images

[shawncrawley]

AWS allows Lambda functions to be configured in one of two ways:

1. On top of an AWS-provided python environment that can be customized with unique package requirements via a clunky, self-maintained "AWS Layers" mechanism
2. As a fully self-maintained Docker image

In our current infrastructure, we use a combination of both methods extensively. In the case of number 1 above, there's a lot of undesirable maintenance overhead with managing the unique package requirements and difficulty in creating a reproducible execution environment.

[groutr]

The AWS layers system is nice for quickly deploying python code into production. However, beyond providing a basic execution environment, it has several severe shortcomings.

1. Dependency management becomes unnecessarily complex. The layers system forces you to manually package and manage all of your dependencies. For complicated dependencies like GDAL, the maintenance burden is multiplied. It becomes very difficult to have a consistent execution environment where every dependency is correctly satisfied. Adding a new dependency becomes increasingly difficult and prone to mistakes.
2. Size limits. I believe layers are limited to 50mb in size. Docker images have a 10gb limit.