Need help with ws-docker-community?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

126 Stars 25 Forks GNU General Public License v3.0 17 Commits 1 Opened issues


Web Sight Docker Deployment

Services available


Need anything else?

Contributors list

# 311,648
2 commits

Web Sight Docker Deployment

Black Hat Arsenal

Web Sight is a software platform that enables red and blue teams to automate the information gathering processes required by their day-to-day jobs. At present, Web Sight performs the following activities:

  • Domain name enumeration
  • DNS record enumeration
  • Network scanning (large scale)
  • Network service fingerprinting
  • SSL support enumeration
  • SSL certificate inspection
  • Application-layer inspection for supported application protocols (currently only HTTP)

These activities are entirely automated, and require only the following information as scanning "seeds":

  • Network ranges
  • Domain names

For web applications that are discovered across an organization's domain names and network ranges, the following activities are conducted:

  • Virtual host enumeration
  • User agent enumeration
  • Crawling
  • Screen shotting

The goal of automating this information gathering process is to provide users with the situational awareness that proper security strategizing (both offensively and defensively) requires. Simply put, how can you hope to attack and/or defend an organization if you don't have a good understanding of the organization's attack surface at a given point in time? Furthermore, given the nature of enterprise attack surface (constant churn, massive scale), any understanding of attack surface is fleeting, and attack surface must be re-evaluated regularly to maintain situational awareness.

Please note that this documentation is very much a work in progress. If you find any part of it confusing, please feel free to submit a question via GitHub and I will do my best to respond in a timely fashion.


This repository contains the software necessary to get Web Sight's front- and back-end components up and running on top of Docker.

Web Sight is comprised of two main components - the front-end and the back-end. The front-end is a single page application written using Angular 2 (Typescript) and the back-end is a fairly complicated distributed N-tier system comprised of RabbitMQ, Python (Django Rest Framework, Celery, SQLAlchemy), PostgreSQL, Redis, and Elasticsearch. As may be apparent from this list of dependencies, getting all of the components up and running and working with one another can be a bit of a chore.

Enter Docker. If you haven't used Docker before, I highly recommend reading the getting started Docker documentation. In a nut shell, Docker enables you to quickly deploy applications and all of their dependencies across various environments at ease. This includes deploying N-tier applications that have inter-application dependencies, which makes Docker a perfect fit for Web Sight deployment.


Dependencies for the front- and back-end components can be found in their respective Git repository README files. Getting things deployed through Docker simplifies things a bit, as many of the back-end dependencies are handled within the Docker deployment. All in all, the Docker deployment relies upon the following:

  • Docker - All software is deployed within Docker containers and connections between application dependencies is handled by Docker as well
  • Elasticsearch - Data collected by task nodes is largely stored in Elasticsearch to enable rapid querying and insertion at scale as well as to reduce database load
  • PostgreSQL - Web Sight makes use of PostgreSQL for storage of all database-related data
  • AWS S3 - File storage

Directory Layout

The contents of the Web Sight Docker project are laid out as follows:

/configs/ - Configuration files for the front- and back-end applications
/nginx/ - Files for the Nginx server that is deployed within Docker
/dockerfiles/ - The .dockerfile files for creating the Docker images that the full Docker deployment relies upon
/scripts/ - Various scripts used for setting up and testing the Docker deployment
/docker-compose.yaml - The Docker compose file for spinning up the N-tier Web Sight application.


Web Sight has been tested and works with both OSX and Ubuntu. The steps here should work on other Linux distributions, but YMMV.

To get started with the Web Sight Docker deployment, first clone the repository and

into the cloned directory:
git clone
cd ws-docker-community

Once the repository is cloned, pull down the front- and back-end repositories that the Docker deployment relies upon by running the following command:

git submodule update --init --recursive

For reference, if you need to build images again due to new code being available in either

, you can run the following command to pull down the latest of both repositories:
git submodule update --recursive --remote

With all of the relevant code now pulled down, we must install and configure the non-Dockerized dependencies that Web Sight relies upon. Note that the installation process for these dependencies can vary greatly depending on what platform you are using, so I'll leave links here to the technologies and their respective installation instructions:

Once all of these dependencies have been installed, you will want to:

  1. Create a database user as well as a database within PostgreSQL
  2. Give full permissions for the given database user on the newly-created database
  3. Ensure that the given PostgreSQL user can access the database server from the IP address(es) where you'll be running Web Sight from
  4. Create a user within Elasticsearch

With all of the Web Sight code pulled down and the software dependencies installed, we can move on to configuring the deployment.


Web Sight makes use of a number of third-party services to provide various parts of functionality to application users. In order for Web Sight to be fully functional, you must sign up for these third-party services:

  • Amazon Web Services (req'd) - Usage of S3 (req'd) and usage of Amazon Elasticsearch (optional)
  • Farsight DNSDB (optional) - Greatly boosts domain name enumeration
  • Stripe (req'd) - Currently used for payment processing and the API currently requires at least dev credentials for orders to be placed (this will be phased out soon)
  • SMTP Mail Server (req'd) - Sending emails, can be any SMTP service
  • reCAPTCHA (req'd) - Protects against automated attacks, will be phased out soon

Once you have signed up for these services and have the relevant API keys, we can move on to configuring the front- and back-end Web Sight components.

First we will want to configure the front-end. To do so, copy the example configuration file over:

cp configs/app.config.ts.example configs/app.config.ts

Once the configuration file has been copied over, we must alter only the following values within it:

stripePublishableKey - The publishable key from your from your [Stripe account](

With the front-end configuration file now set up, let's move on to the back-end. Start by copying over the example configuration file again:

cp configs/tasknode.cfg.example configs/tasknode.cfg

Once the configuration file has been copied over, we must alter the following values:


aws_key_id - Your AWS key ID aws_secret_key - Your AWS secret key


db_host - The IP address or hostname where your PostgreSQL server is running db_port - The port where your PostgreSQL server is running db_name - The name of the database to use for Web Sight db_user - The username to connect to PostgreSQL with db_password - The password to connect to PostgreSQL with


dns_dnsdb_api_key - Your Farsight DNSDB API key


es_username - The username to connect to Elasticsearch with es_password - The password to connect to Elasticsearch with es_host - The hostname or IP address where your Elasticsearch server is running es_port - The port where your Elasticsearch server is running es_use_aws - Whether or not to use AWS Elasticsearch (if you set this value to True, then the credentials in the [AWS] section will be used to connect to AWS and the other connection values within [Elasticsearch] will be ignored.


payments_stripe_publishable_key - Your Stripe API publishable key payments_stripe_secret_key - Your Stripe API secret key


recaptcha_secret - Your reCAPTCHA secret key


rest_domain - The URL where your REST API will be running


smtp_username - The username to connect to your SMTP server with smtp_password - The password to connect to your SMTP server with smtp_host - The hostname or IP address where your SMTP server is running smtp_port - The port where your SMTP server is running

With the

file now properly filled out, we have one configuration file left. Go ahead and copy over the example Django settings file:
cp configs/ configs/

Now let's fill the
file out. The example
file contains blocks surrounded by square brackets (
) for all of the places where you must update the configuration file. The values should be updated as follows:
[[DJANGO_SECRET]] - A large, unguessable random string
[[DB_NAME]] - The name of the database that Web Sight will use
[[DB_USER]] - The user to connect to the database with
[[DB_PASSWORD]] - The password to connect to the database with
[[DB_HOST]] - The hostname or IP address where the database resides
[[DB_PORT]] - The port where the database resides
[[SMTP_HOST]] - The hostname or IP address where your SMTP server resides
[[SMTP_PORT]] - The port where your SMTP server resides
[[SMTP_USER]] - The user to connect to your SMTP server with
[[SMTP_PASSWORD]] - The password to connect to your SMTP server with
[[SMTP_USE_TLS]] - A boolean value depicting whether or not to connect to your SMTP server using SSL/TLS.

Now that all of the configuration files are filled out, we need to create a

file. If you're not planning on using Google Compute Engine, then do the following to create an empty placeholder:
touch secrets/gce.json

If, on the other hand, you are using GCE, then place the service account credential file in


With all of that done, your Dockerized Web Sight deployment should be entirely configured! We can now move on to building the Docker images.

Building the Docker Images

The Web Sight deployment contains all of the Dockerfiles that it relies upon in the

directory. The contents of this directory are explained below:
api.dockerfile - This is the image that runs the Django Rest Framework application for the Web Sight API.
backend-base.dockerfile - This is the base image that both the tasknode and the REST API server are built upon. It installs all of the dependencies that are relied upon by both node types in the deployment.
frontend-base.dockerfile - This is the base image for the web server that hosts the single page application used by Web Sight. It installs and configures Nginx as well as the tools necessary for building the front-end application.
frontend.dockerfile - This is the full image for the front-end that contains the compiled single page application.
tasknode.dockerfile - This is the image that contains the code run by tasknodes in the deployment.

To build all of the images in one fell swoop, go ahead and invoke the Docker building script as follows:


The script quite simply builds all of the images used by the Web Sight Docker deployment:

docker build -f dockerfiles/backend-base.dockerfile -t wsbackend-base:latest .
docker build -f dockerfiles/api.dockerfile -t wsbackend-api:latest .
docker build -f dockerfiles/tasknode.dockerfile -t wsbackend-tasknode:latest .
docker build -f dockerfiles/frontend-base.dockerfile -t wsfrontend-base:latest .
docker build -f dockerfiles/frontend.dockerfile -t wsfrontend:latest .

With the images all successfully built, the only step left is initializing our database and Elasticsearch and then we are good to go!

Initializing Data Stores

Now that all of the images are built, we can use the images to initialize the persistent data stores used by Web Sight. To do so, run the following command from the root project directory:


This script will do the following:

  1. Create all of the database tables
  2. Create the default database objects in all of the relevant database tables
  3. Create the Elasticsearch document models for the data Web Sight stores in Elasticsearch
  4. Prompt you to create a user (this user will be a super user of the application)
  5. Activate the user account that you added

With the data stores now fully populated, we can run some tests to make sure that all of the components have been set up correctly.

Testing The deployment

With the images built and the persistent data stores initialized, we are finally ready to test our deployment to make sure that everything is working as expected.

First, let's run the test suite to make sure that all of our unit tests are passing:


This script will spin up a Web Sight deployment via

, run all of the unit tests on the
instance, and then spin the deployment back down. We can then run all of the same tests on the

If all of the tests on both the

instances passed, then we have one last utility that we can use to ensure that the deployment is configured correctly and is ready to go!


project contains a deployment checking class. To invoke this class and check for the presence of all necessary configuration, run the following:

The output of this script will be a number of lines indicating whether or not a particular required component of the deployment is present and available. If all of the lines display

then we know all of the various requirements of our deployment have been fulfilled!

Running Web Sight

We made it!! We have built all of the images, checked that everything was configured correctly, and can now spin up the Docker deployment. To get Web Sight going, do the following:

docker-compose up -d


can be omitted if you'd like to see the live logs as the system spins up. Once everything is up and running, you should be able to access the front-end web application at the following URL:

The API should be available at the following URL:

Go ahead and start populating organizations, running scans, and enumerating attack surface with your brand-spanking-new Web Sight deployment!

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.