How to use the persistence layer (EFS) in AWS SaaS Boost?

[yairregev]

We have been trying to deploy our application on SaaS boost and have been successful.
We didn't want our configuration files to change if we redeploy our application using new images so we want to use the Persistence via File system(EFS)
and this EFS we are mounting as /mnt. but we don't see this as mounted, in order to test we added a script in our image container to read /mnt and create a file every 30 seconds, but when we check logs we see /mnt exist but it's empty, and this /mnt is not mounted to EFS.
It will be helpful if you can share a persistence sample

[brtrvn]

Does your Dockerfile have a corresponding VOLUME /mnt line in it?

[brtrvn]

Also note that the shared file system feature is designed per-tenant. Meaning there is a different, isolated EFS file system for each tenant you onboard into your SaaS Boost environment. The common use case is if your application needs to process file uploads.

In contrast, if you have configuration data per tenant that isn't really part of the functional processing of your application, you may prefer to use the S3 bucket feature instead.

When you onboard a new tenant you can upload a file (or files inside a ZIP archive) to a secure S3 bucket folder for that tenant. That S3 location will be provided to your container as an environment variable and you can use the AWS S3 SDK library in the language of your choice to download the file from S3 for that tenant when your application workload starts up. An example use case for this feature may be CSS, images or branding assets for each of your tenant environments.

[yairregev]

Our plan is to use this persistence layer for several functions:

• XML / config file settings
• Logs
• File uploads
  In all cases we need the data to be persisted between tenant reloads (due to fault cases). In all cases the data is indeed specific per tenant. Our thought was to have softlinks to the persisted file layer (EFS) from our file system.

[brtrvn]

You don't actually need a VOLUME line in your Dockerfile unless you want to test a bind mount locally.

I just created a small test app and installed a fresh SaaS Boost environment from the HEAD of main and confirmed that EFS is working properly and being mounted properly to the Fargate task instances for each tenant. I can start writing files with a single task, stop that task and when ECS launches a new task to replace it all of the files written by the original task are still on the share and available. I also disabled the ECS service for that tenant (set desired count to zero) so that no copies of the application are running and then launched an EC2 instance into the proper network and mounted the EFS file system and all files existed as expected.

There must be some other configuration or action your application is taking that is causing you issues. Let's get some more details about your workload so we can resolve this for you! I can also post the small sample app and corresponding SaaS Boost app config if that is helpful.

[mahara13]

@brtrvn could you verify.
We have next docker file:

FROM centos:7

#initializing all needed services

VOLUME /mnt

CMD ["rundocker.sh"]

the rundocker.sh has next content

#launch needed services

while sleep 30; do

   #some business stuff to check
   
  echo " >>>>> MOUNT_POINT = $MOUNT_POINT "

  echo `ls -la /mnt`
  if [[ -d "/mnt" ]]; then
    echo " >>>>>>>>>>>>>>  inside in if"
    echo "Write from container " >> /mnt/test.txt
  else
    echo "No volume under /mnt"
  fi
done

We check presence by logs.
We try to read the content of mounted EFS files but the /mnt folder didn't contain files from EFS.
We write into /mnt folder(test.txt file) but we didn't get that file on EFS files
The $MOUNT_POINT environment variable is also empty

Here are the steps we do to connect to EFS:

1. Launch centos instance on same tenant VPC
2. Attach the same security group that is used by tenant ECS
3. take the attach command of the tenant EFS and attach in the instance /mnt created on step1

[brtrvn]

I’m confused. Is this Dockerfile your SaaS Boost application? If so, I’m guessing SaaS Boost won’t properly deploy it because it’s not a web application and the load balancer will never return a successful health check.

Do you have a SaaS Boost compatible workload that is running (you’ve onboarded at least 1 tenant) and you can properly access that tenant’s deployment of your application through the load balancer URL?

If this Dockerfile is a tester to see what’s in the EFS file system, how are you launching this Docker image? Did you create a new ECS task definition to match this Dockerfile? Are you launching this by hand on an EC2 instance that you’ve ssh’ed into? It’s not clear to me what steps you’ve taken.

Assuming you’re launching this CentOS image in order to view the contents of the EFS file system, you’ll need to first mount the EFS file system on the host that’s running this docker image. ECS will take care of this step for you if you configure the task definition properly (this is what Boost does for you). Once you’ve mounted the file system on the docker host, you can run the docker image and tell docker to create a bind mount between the host and the container. You would need to add your EC2 instance to the security group SaaS Boost creates for EFS access. Note that there is no MOUNT_POINT environment variable set by the SaaS Boost task definitions. The assumption is that your application is reading/writing from a known directory (what you told SaaS Boost was the mount point).

I would argue that it’s easier to test/view the contents of the EFS file system by simply launching an EC2 instance with the proper networking and security settings and mounting the EFS file system. You can then view the shared file system contents. To get the existing EFS mount targets for the Boost provisioned file system, use the AWS CLI.
aws efs describe-mount-targets --file-system-id $filesystemid

For example:

ssh -l ec2-user -i /path/to/your/ec2/keypair publicIPv4ec2address.region.compute.amazonaws.com
yum install -y nfs-utils
sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport ipAddressOfEfsMountTarget:/ /mnt
ls -la /mnt 

[brtrvn]

Checkout the latest episode of the SaaS Boost Office Hours for a deep dive into shared file systems.
https://www.twitch.tv/videos/1149684609