Need help with rust-crowbar?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

iliana
197 Stars 16 Forks Other 89 Commits 14 Opened issues

Description

Wrapper to simplify writing AWS Lambda functions in Rust (using the Python execution environment)

Services available

!
?

Need anything else?

Contributors list

# 60,514
federat...
Linux
Ada
Racket
27 commits
# 27,390
Rust
scalajs
serverl...
aws-lam...
14 commits
# 200,600
Rust
C
qemu-kv...
luajit
7 commits
# 83,845
XML
python3
circlec...
sensu
4 commits
# 109,306
elm
Terrafo...
aws-lam...
rpg
3 commits
# 696,912
Rust
aws-lam...
Serverl...
Amazon ...
1 commit
# 22,818
gRPC
sasl
k8s
cobra
1 commit
# 4,904
Rust
webasse...
asmjs
webwork...
1 commit
# 502,122
aws-lam...
Amazon ...
Go
HTML
1 commit

rust-crowbar

Build Status crates.io docs.rs

logo

crowbar makes it easy to write AWS Lambda functions in Rust. It wraps native Rust functions into CPython modules that handle converting Python objects into Rust objects and back again.

Usage

Add both crowbar and cpython to your

Cargo.toml
:
[dependencies]
crowbar = "0.3"
cpython = "0.2"

Use macros from both crates:

#[macro_use(lambda)]
extern crate crowbar;
#[macro_use]
extern crate cpython;

And write your function using the

lambda!
macro:
lambda!(|event, context| {
    println!("hi cloudwatch logs, this is {}", context.function_name());
    // return the event without doing anything with it
    Ok(event)
});

Building Lambda Functions

For your code to be usable in AWS Lambda's Python execution environment, you need to compile to a dynamic library with the necessary functions for CPython to run. The

lambda!
macro does most of this for you, but Cargo still needs to know what to do.

You can configure Cargo to build a dynamic library with the following. If you're using the

lambda!
macro as above, you need to use
lambda
for the library name (see the documentation for
lambda!
if you want to use something else).
[lib]
name = "lambda"
crate-type = ["cdylib"]

cargo build
will now build a
liblambda.so
. Put this in a zip file and upload it to an AWS Lambda function. Use the Python 3.6 execution environment with the handler configured as
liblambda.handler
.

Build Environment

It is notoriously difficult to build a properly-linked shared library against the Lambda execution environment. Using either an Amazon Linux AMI or the Amazon Linux Docker image, even if using the exact version as Lambda, any package installations will likely upgrade OpenSSL from 1.0.1k to 1.0.2k, causing a linker exception at runtime. @naftulikay lost a ridiculous amount of time trying to statically compile, pin package versions, and try linker hacks.

The best solution available is to use the

lambci/lambda:build-python3.6
Docker image which is built from an exact filesystem replica via tarballing the filesystem at runtime from a Python 3.6 runtime Lambda function. The authors went to extensive lengths to pin packages and replicate the environment as accurately as possible, and experience has shown this is the best way to build Python 3.6 shared libraries.

@naftulikay created a sample Rust build environment based on the upstream

lambci/lambda:build-python3.6
image at
naftulikay/crowbar
. Previously,
naftulikay/circleci-amazonlinux-rust
was used and the aforementioned issues were encountered. Despite CircleCI being used in the name, the image is a fairly generic Rust build environment and should be fairly portable and resuable. For Travis CI and CircleCI examples, please look in the
examples/ci
directory.

Because you're building a dynamic library, other libraries that you're dynamically linking against need to also be in the Lambda execution environment. By using the

lambci/lambda:build-python3.6
image, the build environment will be consistent with the runtime environment.

As described here, the Lambda execution environment uses a runtime library path equivalent to:

LD_LIBRARY_PATH=/lib64:/usr/lib64:$LAMBDA_TASK_ROOT:$LAMBDA_TASK_ROOT/lib

@naftulikay wrote a fairly naïve Python script which will recursively copy linked libraries into the deployment package under

lib/
. This ensures that any non-standard libraries will be available on the library path at runtime. See the
examples/ci/{travis,circle}
directories for examples on how to use this, and see naftulikay/docker-crowbar for more information.

Deployment

Serverless framework

Serverless framework is an extemely popular workflow tool for developing and deploying serverless applications and comes with in depth guides for AWS lambda. Serverless framework is surrounded by a strong ecosystem of plugins including a plugin for deploying Rust applications. A serverless rust template for quickly bootstraping, building, and deploying crowbar applications can be found here.

Contributing

crowbar welcomes your contributions:

  • Let us know if you use crowbar in production
  • If you have a bug report or an idea, submit an issue
  • If you want something to work on, check the issues list
  • Please submit non-trivial changes as an issue first; send a pull request when the implementation is agreed on

crowbar follows a code of conduct; please read it.

Alternatives

As AWS Lambda has added more runtimes, more ways to run Rust on Lambda have emerged.

The Rust on AWS Lambda project is kind enough to offer an alternative, and a comparison of itself to crowbar.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.