Add support for VPCOptions to Elasticsearch cluster deployment configuration

[dave-malone]

In order to support Private deployments of the Data lake solution, allow for the configuration of a VPC deployed Elasticsearch cluster. I'm willing to collaborate and contribute on this change request.

[hvital]

Thanks for your feedback!

We’re currently working to publish Active Directory integration and ES authentication via Cognito (https://aws.amazon.com/blogs/database/get-started-with-amazon-elasticsearch-service-use-amazon-cognito-for-kibana-access-control/).

As VPC support also requires to review all other components (ex: lambda, ES, dynamoDB ... ) and also give the option to create a new VPC or reuse an existing one, I’ll put this item in the solution’s backlog.

If you already have something that it’s ok (in terms of the solution’s license) to share/include to the repo, please send a PR.

[jgc234]

If you adjust the CFN templates to put all the lambda functions (including the helper) into a VPC, and the ES cluster, it seems to work fine.

[knihit]

Hi @jgc234, unfortunately even if you put Lambda functions and ES into a VPC, it would still require a NAT and IGW to communicate with S3 and DynamoDB.

@dave-malone I will initiate this a feature request and try to plan this for any future release. At the same time you are welcome to submit any PR on this feature.