A complete, serverless starter project
Jump to: Getting Started | Quick Start | CI/CD | Architecture | Contributing
SLIC Starter is a complete starter project for production-grade serverless applications on AWS. SLIC Starter uses an opinionated, pragmatic approach to structuring, developing and deploying a modern, serverless application with one simple, overarching goal:
Get your serverless application into production fast
This project is free to use by enterprise, startups, students, educators, enthusiasts and skeptics alike. We actively encourage contributions, suggestions and questions from anyone.
For the CICD pipeline architecture, see CI/CD
SLIC Starter is a complete, working application. By including all the aspects of a real application, SLIC Starter goes beyond a typical demo project. You are encouraged to:
We chose a monorepo approach. Every serverless module (service) is a folder at the parent level of the repo. The monorepo/multirepo decision is always a tricky one but we think monorepo works best here for these reasons.
SLIC Starter uses:
Authentication is a difficult problem with constantly-evolving security requirements. SLIC Starter uses Cognito and the Amplify SDK to remove the burden. Cognito can still be complex when getting started. SLIC Starter provides a complete setup with user pool and identity pool, sign-up and login. Amplify is used to authenticate in the web client. There is even a Cognito/Amplify simulation that allows you to work in local development mode, avoiding the need for a real Cognito backend in all cases.
SLIC Starter includes useful, working examples of typical CRUD (create, read, update and delete) actions with a REST API. Currently, these APIs are implemented with DynamoDB and the
DocumentClientAPI. For an example of this, look at the implementation of the Checklist service. Upon first deploying SLIC Starter, the API Service will create Record Sets and an API Domain Name for your API. APIGateway and Route53 handles the creation of these resources for us.
A major goal of SLIC Starter is to provide a realistic, working examples for event-driven messaging, including a scalable and real-time message bus.
Following the KISS principle, SLIC Starter only requires the simplest messaging possible for now, so we use CloudWatch Events as inspired by this article.
SLIC Starter has a front-end web application. It uses React, Redux and Material UI. Out of the box, the front end is configured, built, packaged and deployed to S3 and CloudFront with a domain and HTTPS certificate.
sliclists.comis
api.sliclists.com)
Getting continuous integration and deployment (CI/CD) right is one of the most important things in your microservice or serverless project. Having a good foundation here allows you to keep making changes fast. It's also fairly difficult to get right. SLIC Starter has made key choices to help you here.
SLIC Starter covers automated testing with:
All tests can be run in local development mode as well as against a fully-deployed environment. The API and E2E tests are executed against a staging environment before deployment to production.
For details on integration (API) tests, see the README.md in integration-tests
X-Ray is enabled for all services and centralized logging is supported. This is pretty basic in terms of monitoring support so much more is planned and contributions are welcomed.
As a default log centralization solution, SLIC Starter publishes logs to logz.io. The logging module handles this using the logz.io forwarder. This is integrated into each service's Lambda function using the serverless-log-forwarding plugin.
For further details, see the logging README
We use AWS Secrets Manager for storing the GitHub personal access token and AWS Systems Manager Parameter Store for storing other secrets, such as API access tokens.
Coming soon. SLIC Starter will include support for roles and Role-Based Access Control (RBAC).
SLIC Starter is designed to get you up in running with a real-world application as quickly as possible. The fact that we go beyond the average sample application, there is a bit more involved in getting to production. For example:
sliclists.com) and will set up DNS entries for use in production (like
api.sliclists.com) and staging (
stg.sliclists.com,
api.stg.sliclists.com). If you want to get up and running with a system quickly, you can skip domains by following the QUICK_START.md guide.
This section covers a full deployment using multiple accounts with domain names and HTTPS certificates. It takes quite a bit longer than your average sample app since there is DNS and certificate approval involved. If you want to try SLIC Starter out in a single account without domains, go to the QUICK_START.md]
To set up deployment to your own accounts, first run through these steps.
slic-config.json.sampleto
slic-config.jsonand edit it to include the AWS Account IDs of your staging, production and CI/CD accounts. For single account deployment the Account ID's of CICD, Production and Staging would all be the same. This file is
.gitignore'd so your account IDs are not commited to Git. Also specify your GitHub repository details and DNS domain for your deployment. Use a domain you own so you can update DNS entries to point to your deployed environment. When the deployment process runs, the domain owner will be sent an email to verify ownership before the deployment completes.
GitHubPersonalAccessTokenin Secrets Manager in the CICD account. See this post for more detail on this approach.
common-config.json. If desired, you can configure the default regions and domain prefixes by account. If you have no reason to change them, leave the default values as they are.
SecretStringvalues with the following names
test/mailosaur/serverId
test/mailosaur/apiKeyThese are picked up by the integration and end-to-end test CodeBuild projects.
/STAGE/sharing-service/code-secretwhere STAGE is the stage you are deploying to (dev, stg or prod).
npm install -g serverless cd cicd/cross-account AWS_PROFILE=your-staging-account serverless deploy AWS_PROFILE=your-production-account serverless deploy
cd cicd npm install npm run build AWS_PROFILE=your-cicd-account npm run cdk -- bootstrap SLIC_NS_DOMAIN=your_domain AWS_PROFILE=your-cicd-account npm run deploy
The CICD process attempts to build and deploy each service in parallel. This is done so you get quick feedback and to improve the overall deployment speed. It also means that deployment can fail if there are dependencies between services. Out of the box, SLIC Starter has a
certsmodule that sets up a Route 53 Hosted Zone and some certificates. These are required by the
frontendand
checklist-serviceservices, so those builds will fail if the cerificates aren't ready yet. This is just one example. There are other services that depend on common resources so it will require a few retries in both staging and production before everything is deployed.
Note that deployment of some services can take quite some time! In particular,
frontenddeployment will wait until the CloudFront distribution has been created. This can take at least 15 minutes.
You can inspect the failures in the Orchestrator Pipeline view in the CodePipeline console of your CICD account. You can retry the
stgDeployphase by clicking the Retry button in the pipeline.
Once you get all services in staging successfully deployed, you might find that the test stage fails. This is likely to do with the front end being inaccessible. As we already mentioned, your DNS entries will need to be set up. Let's understand how this all works better!
NSand
Arecords for staging in your staging account. When production deployment happens, it will also create
NSand
Arecords in your production account.
sliclists.com)
NSrecords from your production Route53 into your domain registration name server settings. You will then need to copy the
NSrecord (only the
NSrecord!) for staging into your production Route 53 for the apex domain and give the NS record the staging domain (e.g.,
stg.sliclists.com).
NStype records) from your staging and production account into that provider's DNS records.
Once you have set up the required DNS configuration and it has propagated, your front end staging application should be available at
https://stg.YOUR-DOMAIN.TLDand you can click Retry on the test phase of the pipeline!
For the
email-serviceto send emails, you must choose a 'From:' address and set it in parameter store. The paramter name is
//email-service/from-address. should be replaced with
stgor
prodas appropriate.
order for the email service to send requests to SES, either the email address or the domain needs to be verified. This is not automatically done as part of the SLIC Starter deployment.
If you are using SLIC with a domain configuration, the verification process is quite straightforward because we use Route 53 for our DNS records. See here for documentation on how to achieve domain verification throught the AWS Management Console.
By default, SES will require validation of each email address to which emails are being sent. To avoid this, you can request a sending limit increase, which will remove your account/region from the SES Sandbox.
Work on a more seamless local development environment is in progress. In the meantime, here's an introduction to running the
checklist-servicelocally:
In backend services:
sls dynamodb install
Ensure that your AWS credentials are set to some reasonable values. For local development, any dummy value for
AWS_ACCESS_KEY_IDand
AWS_SECRET_ACCESS_KEYis sufficient. You can also opt to use an AWS profile if that's your preferred method for specifying AWS credentials.
SLIC_STAGE=local sls offline start --migrate true
When working in local development, the backend configuration is pulled from .env.local. When building the production frontend for any deployed stage,
npm run buildwill, by default, generate a
.env.productionfile. This file is
.gitignored so it will not be committed. The values for this file are dynamically generated using the CloudFormation outputs retrieved from the stage specified using the
SLIC_STAGEenvironment variable.
SLIC Starter provides a fully-featured application for managing checklists called SLIC Lists. SLIC Starter is self hosting, so SLIC Lists is continuously deployed to sliclists.com from this repository!
SLIC Starter uses Prettier for code formatting and ESLint for syntax checking.
npm run formatin each project)
npm run lintin each project
SLIC Starter is open source and contributions are welcome from everyone. It was started by the team at fourTheorem, also the authors of the book, AI as a Service, a Manning publication on Serverless, AI-enabled applications.
There are many other amazing resources to help you get started, learn and evolve your Serverless practice. Here are a few recommendations!
87/101 | 1:05:21 PM | CREATE_FAILED | AWS::CodeBuild::Project | sourceProject (sourceProjectBCA86C81) Failed to call CreateWebhook, reason: Repository not found or permission denied.
See CONTRIBUTING.md
Copyright fourTheorem Ltd. 2018-2019. Distributed under the MIT License. See LICENCE