unFTP

When you need to FTP, but don't want to.

unFTP is a FTP(S) server written in Rust and built on top of libunftp and the Tokio asynchronous run-time. It is unlike your normal FTP server in that it provides:

• Configurable Authentication (e.g. Anonymous, PAM or a JSON file).
• Configurable storage back-ends (e.g. GCS or filesystem)
• An HTTP server with health endpoints for use for example in Kubernetes for readiness and liveness probes.
• Integration with Prometheus for monitoring.
• A proxy protocol mode for use behind proxies like HA Proxy and Nginx.
• Structured logging and the ability to ship logs to a Redis instance.

With unFTP, you can present RFC compliant FTP(S) to the outside world while freeing yourself to use modern APIs and techniques on the inside of your perimeter.

Installation

Binaries

Precompiled binaries for unFTP are available for Linux and macOS. On Linux you can choose between a statically linked image (no PAM integration) or a dynamically linked image with PAM integration:

• unftpx8664-apple-darwin - macOS
• unftpx8664-unknown-linux-musl - Linux statically linked, no PAM support.
• unftpx8664-unknown-linux-gnu - Dynamically linked with PAM support.

To install with Curl:

Linux (static, no PAM):

curl -L https://github.com/bolcom/unFTP/releases/download/v0.12.12/unftp_x86_64-unknown-linux-musl \
  | sudo tee /usr/local/bin/unftp > /dev/null && sudo chmod +x /usr/local/bin/unftp

Linux (dynamic with PAM support):

curl -L https://github.com/bolcom/unFTP/releases/download/v0.12.12/unftp_x86_64-unknown-linux-gnu \
  | sudo tee /usr/local/bin/unftp > /dev/null && sudo chmod +x /usr/local/bin/unftp

macOS:

curl -L https://github.com/bolcom/unFTP/releases/download/v0.12.12/unftp_x86_64-apple-darwin \
  | sudo tee /usr/local/bin/unftp > /dev/null && sudo chmod +x /usr/local/bin/unftp

From Source

You'll need Rust 1.45 (including

cargo

cargo install unftp

and find unftp in

~/.cargo/bin/unftp

~/.cargo/bin

Features

unFTP offers optional features in its Cargo.toml:

• cloud_storage

  : enables the Google Cloud Storage (GCS) storage backend

• jsonfile_auth

  : enables the JSON file authentication module

• pam_auth

  : enables the PAM authentication module

• rest_auth

  : enables the REST authentication module

Usage

Both command line arguments and environment variables are available in unFTP. To show a list of available program arguments:

unftp --help

To run with default settings:

unftp

Example running an instance with a filesystem back-end and custom port:

unftp \
  --root-dir=/home/unftp/data \
  --bind-address=0.0.0.0:2121 \
  --passive-ports=50000-51000 \
  -vv

With FTPS enabled:

# Generate keypair
openssl req \
   -x509 \
   -newkey rsa:2048 \
   -nodes \
   -keyout unftp.key \
   -out unftp.crt \
   -days 3650 \
   -subj '/CN=www.myunftp.domain/O=My Company Name LTD./C=NL'

Run, pointing to cert and key and require TLS on the control channel
unftp 
  --root-dir=/home/unftp/data 
  --ftps-certs-file=/home/unftp/unftp.crt 
  --ftps-key-file=/home/unftp/unftp.key 
  --ftps-required-on-control-channel=all

Enabling the Prometheus exporter on (

http://../metrics

unftp \
  --bind-address=0.0.0.0:2121 \
  --bind-address-http=0.0.0.0:8080 \
  --root-dir=/home/unftp/data

Run with the GCS (Google Cloud Storage) back-end:

unftp \
  --sbe-type=gcs \
  --sbe-gcs-bucket=mybucket \
  --sbe-gcs-key-file=file

Run behind a proxy in proxy protocol mode:

unftp \
    --proxy-external-control-port=2121

Run sending logs to a Redis list:

unftp \
    --log-redis-host=localhost \
    --log-redis-key=logs-key \
    --log-redis-port=6379

Authenticate with credentials stored in a JSON file:

Create a credentials file (e.g. credentials.json):

[
  {
    "username": "alice",
    "password": "12345678"
  },
  {
    "username": "bob",
    "password": "secret"
  }
]

unftp \
    --auth-type=json \
    --auth-json-path=credentials.json

Require Mutual TLS:

# Create Server Root Key and Certificate
openssl genrsa -out unftp_client_ca.key 2048
openssl req -new -x509 -days 365 \
    -key unftp_client_ca.key \
        -subj '/CN=unftp-ca.mysite.com/O=bol.com/C=NL' \
    -out unftp_client_ca.crt

Create a client side key
openssl genrsa -out client.key 2048
Create a client side certificate signing request (CSR)
openssl req -new -sha256 
    -key client.key 
    -subj '/CN=unftp-client.mysite.com/O=bol.com/C=NL' 
    -reqexts SAN 
    -config 
To do per-user settings you can expand the above mentioned JSON file to also include some per user settings:
[
  {
    "username": "alice",
    "password": "12345678",
    "vfs_perms": ["-mkdir","-rmdir","-del","-ren", "-md5"],
    "root": "alice",
    "account_enabled": true
  },
  {
    "username": "bob",
    "password": "secret",
    "client_cert": {
      "allowed_cn": "bob-the-builder"
    }
  },
  {
    "username": "vincent",
    "root": "vincent",
    "vfs_perms": ["none", "+put", "+md5"],
    "client_cert": {}
  }  
]

And let unFTP point to it:
unftp \
    --auth-type=json \
    --auth-json-path=users.json \
    --usr-json-path=users.json \
    ...

In the above configuration we use:

• vfs_perms

  - Specifies what permissions users can have. Alice cannot create directories, remove them, delete files nor calculate the md5 of files. Bob can do everything while Vincent can only do uploads and calculate md5 files. Valid values here are "none", "all", "-mkdir, "-rmdir, "-del","-ren", "-md5", "-get", "-put", "-list", "+mkdir", "+rmdir", "+del", "+ren", "+md5", "+get", "+put" and "+list".

• root

  - Sets the home directory of the user relative to the storage back-end root. Alice can only see files inside

  $SB_ROOT/alice

  , Bob can see all files and Vincent thinks

  $SB_ROOT/vincent

  is the FTP root similar to Alice.

• account_enabled

  - Allows to disable the user's account completely

• client_cert

  - Allows specifying whether a client certificate is required and how to handle it. Alice logs in with normal user/password authentication. No client certificate needed. Bob needs to provide a valid client certificate with common name (CN) containing, 'bob-the-builder' and also needs to provide a password. Vincent can do passwordless login when providing a valid certificate.

Docker image

The project contains templated Dockerfiles . To get a list of available commands, run:

make help

We offer 3 different options for building an unFTP docker image:

• scratch

  : builds the binary in rust:slim and deploys in a

  FROM scratch

  image. The unFTP binary is statically linked using musl libc.

• alpine

  (default): builds in rust:slim and deploy in alpine. This image is built with musl instead of a full-blown libc. The unFTP binary is statically linked using musl libc.

• alpine-debug

  : same images as

  alpine

  but using the debug build of unftp and adds tools like lftp and CurlFtpFS while also running as root.

• alpine-istio

  : same as

  alpine

  but with scuttle installed. For use together with Istio.

• alpine-istio-debug

  : same as alpine-debug but with the additions of

  alpine-istio

  .
  

To build the alpine docker image:

make docker-image-alpine

Alternatively you can download pre-made images from docker hub e.g.:

docker pull bolcom/unftp:v0.12.12-alpine
docker pull bolcom/unftp:v0.12.12-alpine-istio
docker pull bolcom/unftp:v0.12.12-scratch

Example running it:

docker run \
  -e ROOT_DIR=/ \
  -e UNFTP_LOG_LEVEL=info \
  -e UNFTP_FTPS_CERTS_FILE='/unftp.crt' \
  -e UNFTP_FTPS_KEY_FILE='/unftp.key' \
  -e UNFTP_PASSIVE_PORTS=50000-50005 \
  -e UNFTP_SBE_TYPE=gcs \
  -e UNFTP_SBE_GCS_BUCKET=the-bucket-name \
  -e UNFTP_SBE_GCS_KEY_FILE=/key.json \
  -p 2121:2121 \
  -p 50000:50000 \
  -p 50001:50001 \
  -p 50002:50002 \
  -p 50003:50003 \
  -p 50004:50004 \
  -p 50005:50005 \
  -p 8080:8080 \
  -v /Users/xxx/unftp/unftp.key:/unftp.key  \
  -v /Users/xxx/unftp/unftp.crt:/unftp.crt \
  -v /Users/xxx/unftp/the-key.json:/key.json \
  -ti \
  bolcom/unftp:v0.12.12-alpine

Getting help and staying informed

Support is given on a best effort basis. You are welcome to engage us on the discussions page or create a Github issue.

You can also follow news and talk to us on Telegram

License

You're free to use, modify and distribute this software under the terms of the Apache-2.0 license.

See also

• libunftp