serverless-offline

by dherault

Emulate AWS λ and API Gateway locally when developing your Serverless project

3.7K Stars 550 Forks Last release: 28 days ago (v6.8.0) MIT License 2.8K Commits 247 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Serverless Offline

This Serverless plugin emulates AWS λ and API Gateway on your local machine to speed up your development cycles. To do so, it starts an HTTP server that handles the request's lifecycle like APIG does and invokes your handlers.

Features:

  • Node.js, Python, Ruby <!-- and Go --> λ runtimes.
  • Velocity templates support.
  • Lazy loading of your handler files.
  • And more: integrations, authorizers, proxies, timeouts, responseParameters, HTTPS, CORS, etc...

This plugin is updated by its users, I just do maintenance and ensure that PRs are relevant to the community. In other words, if you find a bug or want a new feature, please help us by becoming one of the contributors :v: ! See the contributing section.

Documentation

Installation

First, add Serverless Offline to your project:

npm install serverless-offline --save-dev

Then inside your project's

serverless.yml
file add following entry to the plugins section:
serverless-offline
. If there is no plugin section you will need to add it to the file.

Note that the "plugin" section for serverless-offline must be at root level on serverless.yml.

It should look something like this:

plugins:
  - serverless-offline

You can check wether you have successfully installed the plugin by running the serverless command line:

serverless --verbose

the console should display Offline as one of the plugins now available in your Serverless project.

Usage and command line options

In your project root run:

serverless offline
or
sls offline
.

to list all the options for the plugin run:

sls offline --help

All CLI options are optional:

--apiKey                    Defines the API key value to be used for endpoints marked as private Defaults to a random hash.
--corsAllowHeaders          Used as default Access-Control-Allow-Headers header value for responses. Delimit multiple values with commas. Default: 'accept,content-type,x-api-key'
--corsAllowOrigin           Used as default Access-Control-Allow-Origin header value for responses. Delimit multiple values with commas. Default: '*'
--corsDisallowCredentials   When provided, the default Access-Control-Allow-Credentials header value will be passed as 'false'. Default: true
--corsExposedHeaders        Used as additional Access-Control-Exposed-Headers header value for responses. Delimit multiple values with commas. Default: 'WWW-Authenticate,Server-Authorization'
--disableCookieValidation   Used to disable cookie-validation on hapi.js-server
--enforceSecureCookies      Enforce secure cookies
--hideStackTraces           Hide the stack trace on lambda failure. Default: false
--host                  -o  Host name to listen on. Default: localhost
--httpPort                  Http port to listen on. Default: 3000
--httpsProtocol         -H  To enable HTTPS, specify directory (relative to your cwd, typically your project dir) for both cert.pem and key.pem files
--ignoreJWTSignature        When using HttpApi with a JWT authorizer, don't check the signature of the JWT token. This should only be used for local development.
--lambdaPort                Lambda http port to listen on. Default: 3002
--noPrependStageInUrl       Don't prepend http routes with the stage.
--noAuth                    Turns off all authorizers
--noTimeout             -t  Disables the timeout feature.
--prefix                -p  Adds a prefix to every path, to send your requests to http://localhost:3000/[prefix]/[your_path] instead. Default: ''
--printOutput               Turns on logging of your lambda outputs in the terminal.
--resourceRoutes            Turns on loading of your HTTP proxy settings from serverless.yml
--useChildProcesses         Run handlers in a child process
--useWorkerThreads          Uses worker threads for handlers. Requires node.js v11.7.0 or higher
--websocketPort             WebSocket port to listen on. Default: 3001
--webSocketHardTimeout      Set WebSocket hard timeout in seconds to reproduce AWS limits (https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#apigateway-execution-service-websocket-limits-table). Default: 7200 (2 hours)
--webSocketIdleTimeout      Set WebSocket idle timeout in seconds to reproduce AWS limits (https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#apigateway-execution-service-websocket-limits-table). Default: 600 (10 minutes)
--useDocker                 Run handlers in a docker container.
--layersDir                 The directory layers should be stored in. Default: ${codeDir}/.serverless-offline/layers'
--dockerReadOnly            Marks if the docker code layer should be read only. Default: true
--allowCache                Allows the code of lambda functions to cache if supported.

Any of the CLI options can be added to your

serverless.yml
. For example:
custom:
  serverless-offline:
    httpsProtocol: "dev-certs"
    httpPort: 4000
    stageVariables:
      foo: "bar"

Options passed on the command line override YAML options.

By default you can send your requests to

http://localhost:3000/
. Please note that:
  • You'll need to restart the plugin if you modify your
    serverless.yml
    or any of the default velocity template files.
  • When no Content-Type header is set on a request, API Gateway defaults to
    application/json
    , and so does the plugin. But if you send an
    application/x-www-form-urlencoded
    or a
    multipart/form-data
    body with an
    application/json
    (or no) Content-Type, API Gateway won't parse your data (you'll get the ugly raw as input), whereas the plugin will answer 400 (malformed JSON). Please consider explicitly setting your requests' Content-Type and using separate templates.

Usage with
invoke

To use

Lambda.invoke
you need to set the lambda endpoint to the serverless-offline endpoint:
const { Lambda } = require('aws-sdk')

const lambda = new Lambda({ apiVersion: '2015-03-31', // endpoint needs to be set only if it deviates from the default, e.g. in a dev environment // process.env.SOME_VARIABLE could be set in e.g. serverless.yml for provider.environment or function.environment endpoint: process.env.SOME_VARIABLE ? 'http://localhost:3002' : 'https://lambda.us-east-1.amazonaws.com', })

All your lambdas can then be invoked in a handler using

exports.handler = async function() {
  const params = {
    // FunctionName is composed of: service name - stage - function name, e.g.
    FunctionName: 'myServiceName-dev-invokedHandler',
    InvocationType: 'RequestResponse',
    Payload: JSON.stringify({ data: 'foo' }),
  }

const response = await lambda.invoke(params).promise() }

You can also invoke using the aws cli by specifying

--endpoint-url
aws lambda invoke /dev/null \
  --endpoint-url http://localhost:3002 \
  --function-name myServiceName-dev-invokedHandler

List of available function names and their corresponding serverless.yml function keys are listed after the server starts. This is important if you use a custom naming scheme for your functions as

serverless-offline
will use your custom name. The left side is the function's key in your
serverless.yml
(
invokedHandler
in the example below) and the right side is the function name that is used to call the function externally such as
aws-sdk
(
myServiceName-dev-invokedHandler
in the example below):
serverless offline
...
offline: Starting Offline: local/us-east-1.
offline: Offline [http for lambda] listening on http://localhost:3002
offline: Function names exposed for local invocation by aws-sdk:
           * invokedHandler: myServiceName-dev-invokedHandler

To list the available manual invocation paths exposed for targeting by

aws-sdk
and
aws-cli
, use
SLS_DEBUG=*
with
serverless offline
. After the invoke server starts up, full list of endpoints will be displayed:
SLS_DEBUG=* serverless offline
...
offline: Starting Offline: local/us-east-1.
...
offline: Offline [http for lambda] listening on http://localhost:3002
offline: Function names exposed for local invocation by aws-sdk:
           * invokedHandler: myServiceName-dev-invokedHandler
[offline] Lambda Invocation Routes (for AWS SDK or AWS CLI):
           * POST http://localhost:3002/2015-03-31/functions/myServiceName-dev-invokedHandler/invocations
[offline] Lambda Async Invocation Routes (for AWS SDK or AWS CLI):
           * POST http://localhost:3002/2014-11-13/functions/myServiceName-dev-invokedHandler/invoke-async/

You can manually target these endpoints with a REST client to debug your lambda function if you want to. Your

POST
JSON body will be the
Payload
passed to your function if you were to calling it via
aws-sdk
.

The
process.env.IS_OFFLINE
variable

Will be

"true"
in your handlers and thorough the plugin.

Docker and Layers

To use layers with serverless-offline, you need to have the

useDocker
option set to true. This can either be by using the
--useDocker
command, or in your serverless.yml like this:
yml
custom:
  serverless-offline:
    useDocker: true
This will allow the docker container to look up any information about layers, download and use them. For this to work, you must be using: * AWS as a provider, it won't work with other provider types. * Layers that are compatible with your runtime. * ARNs for layers. Local layers aren't supported as yet. * A local AWS account set-up that can query and download layers.

If you're using least-privilege principals for your AWS roles, this policy should get you by:

json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "lambda:GetLayerVersion",
            "Resource": "arn:aws:lambda:*:*:layer:*:*"
        }
    ]
}
Once you run a function that boots up the Docker container, it'll look through the layers for that function, download them in order to your layers folder, and save a hash of your layers so it can be re-used in future. You'll only need to re-download your layers if they change in the future. If you want your layers to re-download, simply remove your layers folder.

You should then be able to invoke functions as normal, and they're executed against the layers in your docker container.

Additional Options

There are 2 additional options available for Docker and Layer usage. * layersDir * dockerReadOnly

layersDir

By default layers are downloaded on a per-project basis, however, if you want to share them across projects, you can download them to a common place. For example,

layersDir: /tmp/layers
would allow them to be shared across projects. Make sure when using this setting that the directory you are writing layers to can be shared by docker.

dockerReadOnly

For certain programming languages and frameworks, it's desirable to be able to write to the filesystem for things like testing with local SQLite databases, or other testing-only modifications. For this, you can set

dockerReadOnly: false
, and this will allow local filesystem modifications. This does not strictly mimic AWS Lambda, as Lambda has a Read-Only filesystem, so this should be used as a last resort.

Token authorizers

As defined in the Serverless Documentation you can use API Keys as a simple authentication method.

Serverless-offline will emulate the behaviour of APIG and create a random token that's printed on the screen. With this token you can access your private methods adding

x-api-key: generatedToken
to your request header. All api keys will share the same token. To specify a custom token use the
--apiKey
cli option.

Custom authorizers

Only custom authorizers are supported. Custom authorizers are executed before a Lambda function is executed and return an Error or a Policy document.

The Custom authorizer is passed an

event
object as below:
{
  "type": "TOKEN",
  "authorizationToken": "",
  "methodArn": "arn:aws:execute-api:::///"
}

The

methodArn
does not include the Account id or API id.

The plugin only supports retrieving Tokens from headers. You can configure the header as below:

"authorizer": {
  "type": "TOKEN",
  "identitySource": "method.request.header.Authorization", // or method.request.header.SomeOtherHeader
  "authorizerResultTtlInSeconds": "0"
}

Remote authorizers

You are able to mock the response from remote authorizers by setting the environmental variable

AUTHORIZER
before running
sls offline start

Example:

Unix:

export AUTHORIZER='{"principalId": "123"}'

Windows:

SET AUTHORIZER='{"principalId": "123"}'

JWT authorizers

For HTTP APIs, JWT authorizers defined in the

serverless.yml
can be used to validate the token and scopes in the token. However at this time, the signature of the JWT is not validated with the defined issuer. Since this is a security risk, this feature is only enabled with the
--ignoreJWTSignature
flag. Make sure to only set this flag for local development work.

Custom headers

You are able to use some custom headers in your request to gain more control over the requestContext object.

| Header | Event key | | ------------------------------- | ----------------------------------------------------------- | | cognito-identity-id | event.requestContext.identity.cognitoIdentityId | | cognito-authentication-provider | event.requestContext.identity.cognitoAuthenticationProvider |

By doing this you are now able to change those values using a custom header. This can help you with easier authentication or retrieving the userId from a

cognitoAuthenticationProvider
value.

Environment variables

You are able to use environment variables to customize identity params in event context.

| Environment Variable | Event key | | ----------------------------------- | ----------------------------------------------------------- | | SLSCOGNITOIDENTITYPOOLID | event.requestContext.identity.cognitoIdentityPoolId | | SLSACCOUNTID | event.requestContext.identity.accountId | | SLSCOGNITOIDENTITYID | event.requestContext.identity.cognitoIdentityId | | SLSCALLER | event.requestContext.identity.caller | | SLSAPIKEY | event.requestContext.identity.apiKey | | SLSCOGNITOAUTHENTICATIONTYPE | event.requestContext.identity.cognitoAuthenticationType | | SLSCOGNITOAUTHENTICATIONPROVIDER | event.requestContext.identity.cognitoAuthenticationProvider |

You can use serverless-dotenv-plugin to load environment variables from your

.env
file.

AWS API Gateway Features

Velocity Templates

Serverless doc ~ AWS doc

You can supply response and request templates for each function. This is optional. To do so you will have to place function specific template files in the same directory as your function file and add the .req.vm extension to the template filename. For example, if your function is in code-file:

helloworld.js
, your response template should be in file:
helloworld.res.vm
and your request template in file
helloworld.req.vm
.

CORS

Serverless doc

If the endpoint config has CORS set to true, the plugin will use the CLI CORS options for the associated route. Otherwise, no CORS headers will be added.

Catch-all Path Variables

AWS doc

Set greedy paths like

/store/{proxy+}
that will intercept requests made to
/store/list-products
,
/store/add-product
, etc...

ANY method

AWS doc

Works out of the box.

Lambda and Lambda Proxy Integrations

Serverless doc ~ AWS doc

Works out of the box. See examples in the manual_test directory.

HTTP Proxy

Serverless doc ~ AWS doc - AWS::ApiGateway::Method ~ AWS doc - AWS::ApiGateway::Resource

Example of enabling proxy:

custom:
  serverless-offline:
    resourceRoutes: true

or

    YourCloudFormationMethodId:
      Type: AWS::ApiGateway::Method
      Properties:
        ......
        Integration:
          Type: HTTP_PROXY
          Uri: 'https://s3-${self:custom.region}.amazonaws.com/${self:custom.yourBucketName}/{proxy}'
          ......
custom:
  serverless-offline:
    resourceRoutes:
      YourCloudFormationMethodId:
        Uri: 'http://localhost:3001/assets/{proxy}'

Response parameters

AWS doc

You can set your response's headers using ResponseParameters.

May not work properly. Please PR. (Difficulty: hard?)

Example response velocity template:

"responseParameters": {
  "method.response.header.X-Powered-By": "Serverless", // a string
  "method.response.header.Warning": "integration.response.body", // the whole response
  "method.response.header.Location": "integration.response.body.some.key" // a pseudo JSON-path
},

WebSocket

Usage in order to send messages back to clients:

POST http://localhost:3001/@connections/{connectionId}

Or,

const apiGatewayManagementApi = new AWS.ApiGatewayManagementApi({
  apiVersion: '2018-11-29',
  endpoint: 'http://localhost:3001',
});

apiGatewayManagementApi.postToConnection({ ConnectionId: ..., Data: ..., });

Where the

event
is received in the lambda handler function.

There's support for websocketsApiRouteSelectionExpression in it's basic form:

$request.body.x.y.z
, where the default value is
$request.body.action
.

Authorizers and wss:// are currently not supported.

Usage with Webpack

Use serverless-webpack to compile and bundle your ES-next code

Velocity nuances

Consider this requestTemplate for a POST endpoint:

"application/json": {
  "payload": "$input.json('$')",
  "id_json": "$input.json('$.id')",
  "id_path": "$input.path('$').id"
}

Now let's make a request with this body:

{ "id": 1 }

AWS parses the event as such:

{
  "payload": {
    "id": 1
  },
  "id_json": 1,
  "id_path": "1" // Notice the string
}

Whereas Offline parses:

{
  "payload": {
    "id": 1
  },
  "id_json": 1,
  "id_path": 1 // Notice the number
}

Accessing an attribute after using

$input.path
will return a string on AWS (expect strings like
"1"
or
"true"
) but not with Offline (
1
or
true
). You may find other differences.

Debug process

Serverless offline plugin will respond to the overall framework settings and output additional information to the console in debug mode. In order to do this you will have to set the

SLS_DEBUG
environmental variable. You can run the following in the command line to switch to debug mode execution.

Unix:

export SLS_DEBUG=*

Windows:

SET SLS_DEBUG=*

Interactive debugging is also possible for your project if you have installed the node-inspector module and chrome browser. You can then run the following command line inside your project's root.

Initial installation:

npm install -g node-inspector

For each debug run:

node-debug sls offline

The system will start in wait status. This will also automatically start the chrome browser and wait for you to set breakpoints for inspection. Set the breakpoints as needed and, then, click the play button for the debugging to continue.

Depending on the breakpoint, you may need to call the URL path for your function in seperate browser window for your serverless function to be run and made available for debugging.

Resource permissions and AWS profile

Lambda functions assume an IAM role during execution: the framework creates this role and set all the permission provided in the

iamRoleStatements
section of
serverless.yml
.

However, serverless offline makes use of your local AWS profile credentials to run the lambda functions and that might result in a different set of permissions. By default, the aws-sdk would load credentials for you default AWS profile specified in your configuration file.

You can change this profile directly in the code or by setting proper environment variables. Setting the

AWS_PROFILE
environment variable before calling
serverless
offline to a different profile would effectively change the credentials, e.g.

AWS_PROFILE= serverless offline

Scoped execution

Downstream plugins may tie into the

before:offline:start:end
hook to release resources when the server is shutting down.

Simulation quality

This plugin simulates API Gateway for many practical purposes, good enough for development - but is not a perfect simulator. Specifically, Lambda currently runs on Node.js v10.x and v12.x (AWS Docs), whereas Offline runs on your own runtime where no memory limits are enforced.

Usage with serverless-dynamodb-local and serverless-webpack plugin

Run

serverless offline start
. In comparison with
serverless offline
, the
start
command will fire an
init
and a
end
lifecycle hook which is needed for serverless-offline and serverless-dynamodb-local to switch off resources.

Add plugins to your

serverless.yml
file:
plugins:
  - serverless-webpack
  - serverless-dynamodb-local
  - serverless-offline # serverless-offline needs to be last in the list

Credits and inspiration

This plugin was initially a fork of Nopik's Serverless-serve.

License

MIT

Contributing

Yes, thank you! This plugin is community-driven, most of its features are from different authors. Please update the docs and tests and add your name to the package.json file. We try to follow Airbnb's JavaScript Style Guide.

Contributors

dnalborczyk

dherault computerpunc leonardoalifraco daniel-cottone
dnalborczyk dherault computerpunc leonardoalifraco daniel-cottone

mikestaub

Bilal-S dl748 frsechet zoellner
mikestaub Bilal-S dl748 frsechet zoellner

johncmckim

ThisIsNoZaku darthtrevino miltador gertjvr
johncmckim ThisIsNoZaku darthtrevino miltador gertjvr

juanjoDiaz

perkyguy jack-seek hueniverse robbtraister
juanjoDiaz perkyguy jack-seek hueniverse robbtraister

dortega3000

ansraliant joubertredrat andreipopovici Andorbal
dortega3000 ansraliant joubertredrat andreipopovici Andorbal

AyushG3112

franciscocpg kajwiklund ondrowan sulaysumaria
AyushG3112 franciscocpg kajwiklund ondrowan sulaysumaria

jormaechea

awwong1 c24w vmadman encounter
jormaechea awwong1 c24w vmadman encounter

Bob-Thomas

njriordan bebbi trevor-leach emmoistner
Bob-Thomas njriordan bebbi trevor-leach emmoistner

OrKoN

adieuadieu apalumbo anishkny cameroncooper
OrKoN adieuadieu apalumbo anishkny cameroncooper

cmuto09

dschep DimaDK24 dwbelliston eabadjiev
cmuto09 dschep DimaDK24 dwbelliston eabadjiev

Arkfille

garunski james-relyea joewestcott LoganArnett
Arkfille garunski james-relyea joewestcott LoganArnett

ablythe

marccampbell purefan mzmiric5 paulhbarker
ablythe marccampbell purefan mzmiric5 paulhbarker

pmuens

pierreis ramonEmilio rschick selcukcihan
pmuens pierreis ramonEmilio rschick selcukcihan

patrickheeney

rma4ok clschnei djcrabhat adam-nielsen
patrickheeney rma4ok clschnei djcrabhat adam-nielsen

adamelliottsweeting

againer alebianco-doxee koterpillar triptec
adamelliottsweeting againer alebianco-doxee koterpillar triptec

constb

cspotcode aliatsis arnas akaila
constb cspotcode aliatsis arnas akaila

ac360

austin-payne bencooling BorjaMacedo BrandonE
ac360 austin-payne bencooling BorjaMacedo BrandonE

guerrerocarlos

chrismcleod christophgysin Clement134 rlgod
guerrerocarlos chrismcleod christophgysin Clement134 rlgod

dbunker

dobrynin domaslasauskas enolan minibikini
dbunker dobrynin domaslasauskas enolan minibikini

em0ney

erauer gbroques guillaume balassy
em0ney erauer gbroques guillaume balassy

idmontie

jacintoArias jgrigg jsnajdr horyd
idmontie jacintoArias jgrigg jsnajdr horyd

jaydp17

jeremygiberson josephwarrick jlsjonas joostfarla
jaydp17 jeremygiberson josephwarrick jlsjonas joostfarla

kenleytomlin

lalifraco-devspark DynamicSTOP medikoo neverendingqs
kenleytomlin lalifraco-devspark DynamicSTOP medikoo neverendingqs

msjonker

Takeno mjmac ojongerius thepont
msjonker Takeno mjmac ojongerius thepont

WooDzu

PsychicCat Raph22 wwsno gribnoysup
WooDzu PsychicCat Raph22 wwsno gribnoysup

starsprung

shineli-not-used-anymore stesie stevemao ittus
starsprung shineli-not-used-anymore stesie stevemao ittus

tiagogoncalves89

tuanmh Gregoirevda gcphost YaroslavApatiev
tiagogoncalves89 tuanmh Gregoirevda gcphost YaroslavApatiev

zacacollier

allenhartwig demetriusnunes hsz electrikdevelopment
zacacollier allenhartwig demetriusnunes hsz electrikdevelopment

jgilbert01

polaris340 kobanyan leruitga-ss livingmine
jgilbert01 polaris340 kobanyan leruitga-ss livingmine

lteacher

martinmicunda nori3tsu ppasmanik ryanzyy
lteacher martinmicunda nori3tsu ppasmanik ryanzyy

m0ppers

footballencarta bryanvaz
m0ppers footballencarta bryanvaz

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.