Docker Compose for Amazon ECS Now Available

Docker is pleased to announce that as of today the integration with Docker Compose and Amazon ECS has reached V1 and is now GA! 🎉

We started this work way back at the beginning of the year with our first step – moving the Compose specification into a community run project. Then in July we announced how we were working together with AWS to make it easier to deploy Compose Applications to ECS using the Docker command line. As of today all Docker Desktop users will now have the stable ECS experience available to them, allowing developers to use docker compose commands with an ECS context to run their containers against ECS.

As part of this we want to thank the AWS team who have helped us make this happen: Carmen Puccio, David Killmon, Sravan Rengarajan, Uttara Sridhar, Massimo Re Ferre, Jonah Jones and David Duffey.

Getting started with Docker Compose & ECS

As an existing ECS user or a new starter all you will need to do is update to the latest Docker Desktop Community version (2.5.0.1 or greater) store your image on Docker Hub so you can deploy it (you can get started with Hub here), then you will need to get yourself setup on AWS and then lastly you will need to create an ECS context using that account. You are then read to use your Compose file to start running your applications in ECS.

We have done a couple of blog posts and videos along with AWS to give you an idea of how to get started or use the ECS experience. 

• Amazon’s GA announcement of the experience
• Docker Announcement / AWS announcement 
• Deploying WordPress to ECS 
• Amazon unboxing of ECS experience 
• Docker Docs 

If you have other questions about the experience or would like to give us feedback then drop us a message in the Compose CLI repo or in the #docker-ecs channel in our community Slack.

New in the Docker Compose ECS integration 

We have been adding new features to the ECS integration over the last few months and we wanted to run you through some of the ones that we are more excited about:

GPU support 

As part of the more recent versions of ECS we have provided the ability to deploy to EC2 (rather than the default fargate) to allow developers to make use of unique instance types/features like GPU within EC2. 

To do this all you need to do is specify that you need a GPU instance type as part of your Compose file and the Compose CLI will take care of the rest! 

services:
   learn:
       image: itamarost/object-detection-app:latest-gpu
       command: python app.py
       ports:
           - target: 8000
             protocol: tcp
             x-aws-protocol: http
       deploy:
           resources:
               # devices:
               #   - capabilities: ["gpu"]
               reservations:
                  memory: 30Gb
                  generic_resources:
                    - discrete_resource_spec:
                        kind: gpus
                        value: 1

EFS support

We heard early feedback from developers that when you are trying to move to the cloud you may not be ready to move to managed service to persist your data and may still want to use volumes with your application. To solve  this we have added Elastic File System (EFS) volume support to the Compose CLI allowing users to create volumes and use them as part of their Compose applications. This is created with a Retain policy so data won’t be deleted on application shut-down. If the same application (same project name) is deployed again, the file system will be re-attached to offer the same user experience developers are used to locally with docker-compose.

To do this I can either specify an existing file system that I have already created:

volumes:
   my-data:
     external: true
     name: fs-123abcd

Or I can create a new one from scratch by providing information about how I want it configured:

volumes:
   my-data:
     driver_opts:
       # Filesystem configuration
       backup_policy: ENABLED
       lifecycle_policy: AFTER_14_DAYS
       performance_mode: maxIO
       throughput_mode: provisioned
       provisioned_throughput: 1

I can also manage these through the docker volume command which lets me list and manage my resources allowing me to remove them when I no longer need them.

Context creation improvements 

We have also been looking at how we improve the context creation flow to make this simpler and more interactive – while also allowing power users to specify things more up front if they know how they want to configure your context. 

When you get started we now have 3 options for creating a new context: 

? Create a Docker context using:  [Use arrows to move, type to filter]
> An existing AWS profile
 A new AWS profile
 AWS environment variables

If you select an existing profile, we will list your available profiles to choose from and allow you to simply select the profile you want to have associated with this context. 

$ docker context create ecs test2
? Create a Docker context using: An existing AWS profile
? Select AWS Profile nondefault
Successfully created ecs context "test2"
 
$ docker context inspect test2
[
   {
       "Name": "test2",
       "Metadata": {
           "Description": "(eu-west-3)",
           "Type": "ecs"
       },
       "Endpoints": {
           "ecs": {
               "Profile": "nondefault",
           }
       ...
   }
]

If you want to create a new profile, we will ask you for the credentials needed to do this as part of the creation flow and will save this profile for you:

? Create a Docker context using: A new AWS profile
? AWS Access Key ID fiasdsdkngjgwka
? Enter AWS Secret Access Key *******************
? Region eu-west-3
Saving to profile "test3"
Successfully created ecs context "test3"
 
$ docker context inspect test3
[
   {
       "Name": "test3",
       "Metadata": {
           "Description": "(eu-west-3)",
           "Type": "ecs"
       },
       "Endpoints": {
           "ecs": {
               "Profile": "test3",
           }
       },
       ...
   }
]

If you want to do this using your existing AWS environment variables, then you can choose this option we will create the context with a reference to these env vars so we continue to respect them as you work with them:

$ docker context create ecs test1
? Create a Docker context using: AWS environment variables
Successfully created ecs context "test1"
$ docker context inspect test1
[
   {
       "Name": "test1",
       "Metadata": {
           "Description": "credentials read from environment",
           "Type": "ecs"
       },
       "Endpoints": {
           "ecs": {
               "CredentialsFromEnv": true
           }
       },
       ...
   }
]

We hope this new simplified way of getting started and the flags we have added in here to allow you to override parts of this will help you get started with ECS even faster than before. 

We are really excited about the new experience we have built with ECS, if you have any feedback on the experience or have ideas for other backends for the Compose CLI please let us know via our Public Roadmap.

Join our workshop “I Didn’t Know I Could Do That with Docker – AWS ECS Integration” with Docker’s Peter McKee and AWS’ Jonah Jones Tuesday, November 24, 2020 – 10:00am PT / 1:00pm ET. Register here.