An EC2 Container Service Reference Architecture for providing Service Discovery to containers using CloudWatch Events, Lambda and Route 53 private hosted zones.
Service discovery is a key component of most distributed systems and service-oriented architectures. With service discovery, services are automatically discovered as they get created and terminated on a given infrastructure. This reference architecture illustrates how service discovery can be built on AWS.
Many AWS customers build service-oriented, distributed applications using services such as Amazon EC2 Container Service (Amazon ECS) or Amazon EC2. The distributed nature of this type of architecture requires a fair amount of integration and synchronization, and the answer to that problem is not trivial. Quite often, our customers build such a functionality themselves and this can be time-consuming. Or they use a third-party solution and this often comes with a financial cost.
In this reference architecture, we propose that by leveraging Amazon ECS, Amazon Route 53 and AWS Lambda, we can eliminate a lot of the work required to install, operate, and scale service discovery at the cluster level.
In this example, a web portal application (PortalApp) presents information from a Twitch application (TwitchApp) and a GoodReads application (GoodreadsApp). As instances of these applications are created within ECS, they are placed behind Elastic Load Balancing load balancers. When they come up, they generate an event in AWS CloudTrail which is picked up by Amazon CloudWatch Events. This in turn triggers a Lambda function, which essentially "registers" the service into an Amazon Route 53 private hosted zone. That CNAME mapping then points to the appropriate load balancers. It is then what the web portal (PortalApp) uses to access both TwitchApp and GoodreadsApp.
Specifically, the architecture described in this diagram can be created with an AWS CloudFormation template. That template does the following: - Creates a VPC with two subnets and their route tables, as well as an Internet gateway - Creates appropriate IAM roles (for the EC2 instances, ECS and Lambda) - Deploys an ECS cluster onto which will be launched a web portal application, a Twitch application, and a GoodReads application - Creates load balancers and security groups for the three applications - Creates an Auto Scaling group for your ECS cluster, with its accompanying launch configuration - Creates a private Amazon Route 53 hosted zone (i.e., internal DNS)
Note: The names of the ECS services have to be DNS-compliant as they are re-used as CNAME values.
Here are the steps that you must take to deploy the architecture.
AWS CloudTrail must be enabled in the AWS Region used to launch this reference architecture, as CloudTrail events are used to trigger the service discovery process. AWS CloudTrail is a web service that records API calls made on your account and delivers log files to your Amazon S3 bucket. The quickest way to get started with CloudTrail is to use the AWS Management Console. You can turn on CloudTrail in a few clicks.
We expect that you have the following available:
Clone this repository. You should see a microservices directory with three sub-directories, each containing the information needed to build three Docker containers. ```python
git clone https://github.com/awslabs/ecs-refarch-service-discovery ```
Get the login credentials to ECR registry by typing below command ```python
aws ecr get-login | sh ```
Navigate to the ECS Console and click on Repositories on the left. Create a new repository and specify the name 'twitchapp'
Repeat the above step for 'goodreadsapp' and 'portalapp'
Build the Docker containers in each of the subdirectories:
> cd microservices > cd twitch > docker build -t twitchapp . > cd ../goodreads > docker build -t goodreadsapp . > cd ../portal > docker build -t portalapp .
Tag and Push the images to your ECR repository by typing these commands, replacing 123456789012 with your Account ID.
> docker tag twitchapp:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/twitchapp:latest > docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/twitchapp:latest > docker tag goodreadsapp:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/goodreadsapp:latest > docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/goodreadsapp:latest > docker tag portalapp:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/portalapp:latest > docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/portalapp:latest
Hint: View the ECR repository in the ECS console and expand the Build, tag, and push Docker image section for more complete instructions.
This takes a few minutes; When CREATE_COMPLETE is displayed, note down the following values from the Outputs tab: - Route53PrivateHostedZoneID - LambdaServiceRole - ECSClusterName
It might be a good idea to keep the Output tab open for the rest of this step-by-step guide.
Repeat this procedure for the other two services, modifying values as follows: - For GoodreadsApp: - Enter "GoodreadsApp" as Service Name. - Use the GoodreadsAppTaskDefinition CloudFormation output as Task Definition. - Use the LoadBalancerGoodreadsApp CloudFormation output as ELB Name. - For PortalApp: - Enter "PortalApp" as Service Name. - Use the PortalAppTaskDefinition CloudFormation output as Task Definition. - Use the LoadBalancerPortalApp CloudFormation output as ELB Name.
The goal of Service Discovery is essentially to allow for the components of a distributed architecture to find each other. This is achieved with two components: - A location to centralize service information - A mechanism to find and register those services in that location
In this example, we used Amazon Route 53 and AWS Lambda, respectively: Amazon Route 53 acts as the repository for service registration, and adding and removing them is achieved via the Lambda function. This is made dynamic by triggering that function upon creation or deletion of services.
You can easily see the records that the Lambda function created for you. Just visit the Route 53 Console, click on "Hosted zones" below DNS management and click on your specified hosted zone (default is "ecs.internal."). You will then see that three CNAME records were added pointing to each of the ELBs fronting the Portal, Goodreads and Twitch services.
In the setup, it is imperative for the web portal application to know the DNS name of the services in advance, which is why the instructions above were telling you to enter a specific service name ("TwitchApp", "GoodreadsApp", "PortalApp").
Note: Services behind load balancers render the infrastructure reliable; they can move around (as part of maintenance, or if the applications fail), with no impact to the service.
Feel free to re-use this example in your application, or take it apart and make it work for your context. For example, another place to store information about services could be an Amazon DynamoDB table, and another way to register services could be to have them self-register.
To delete what you created:
This reference architecture sample is licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.