A Cowboy Websocket demo
A Websocket competition. Will your favorite platform come out on top? If you think your platform of choice was poorly represented, "submit a pull request™"
Just create the "Hello, World!" of socket servers, an echo server that speaks WebSocket version 13 (13 is the only version my client supports).
First write your implementation in place it in the
competition/directory.
Added any instructions needed to install dependencies in
configure_ubuntu-12.04.sh. Add any build instructions to
competition/Makefileif your project needs to be compiled
wsdemo_bench utilizes supervisord to start and stop the servers while running the tests. In order to include your server in the test you need to create a supervisord ini file in
competition/servers.
[program:%(server_name)s] command=%(command)s autostart=false autorestart=false startretries=0 stopasgroup=true killasgroup=true
The server name must follow the following format:
{language}-{platform}
For instance if you wrote an echo server using
bashand
ncthe server name would be
bash-nc. You can add additional demarcations if needed. For instance the Python tornado example has a single threaded version and a multiprocessor version which go by the name
python-tornado-1and
python-tornado-N.
Next, you need to add your server to the server list in
wsdemo_bench.app.srcconfiguration file.
Finally, add yourself to the AUTHORS file; you've earned it.
It is best to run the benchmark client on a separate machine than the servers.
Using two brand new Ubuntu 12.04 servers, run the
configure_ubuntu-12.04.shscript on each. This script will destructively alter each server so use caution and review the script before running it.
If you want to use a different OS, use
configure_ubuntu-12.04.shas a template for configuring your system of choice.
Next compile the code on both machines:
make
To run a single benchmark on a server do the following:
./runtest db_path duration hostname port client_count
For instance for a benchmark of 192.168.1.5:8000 of 1000 clients lasting 60 seconds with with the data stored at
data/myserverthe command would be:
./runtest data/myserver 60 192.168.1.5 8000 1000
There are two components in wsdemo_bench. The first component is
supervisord
wsdemo_bench communicates with Supervisord to start and stop each server before each benchmark.
On the machine that you are running the servers on do the following:
sudo bash ulimit -n 999999 cd competition supervisord
You can monitor supervisord by using the
supervisorctlcommand:
competition/ $ supervisorctl status
Next, on the client machine create a
config/small-scale.configfile:
[ {sasl, [ {sasl_error_logger, {file, LogFile}}]},{wsdemo_bench, [ {host, Ip}, {port, 8000}, {db_root, DbRoot}, {clients, 100}, {seconds, 10}, {supervisord, {ServerHost, 9001}}]}].
Replace
LogFile,
Ip,
DbRootand
ServerHostwith the correct values. Ensure that both the path to
LogFileand the path to
DbRootexist or the test will crash.
This configuration file describe a test using 100 clients for 10 seconds each test. We will use this smaller test to ensure that all the servers start and stop properly.
To run this benchmark, do the following:
sudo bash ulimit -n 999999 ./bin/run_all_tests.sh -config config/small-scale.config
If all goes well, you should see the suite running in front of your eyes. Let that run for the entire test. It should run for 12 minutes or so.
If any of the servers crashes you can do the following to diagnose the cause:
competition/ $ supervisorctl tail $SERVER_NAME stderr
After the small scale test, it is time to run the full scale test:
[ {sasl, [ {sasl_error_logger, {file, LogFile}}]},{wsdemo_bench, [ {host, Ip}, {port, 8000}, {db_root, DbRoot}, {supervisord, {ServerHost, 9001}}]}].
Here is the full spec for the wsdemo_bench config:
[ {host, Host :: string()}, {port, Port :: integer()}, {db_root, DbRoot :: directory()}, {supervisord, {Host, 9001}}, {clients, number()}, % defaults to 10000 {seconds, number()}, % defaults to 600 {servers, [ServerName1::string(), ..., ServerNameN::string()]} % defaults to full suite ]
The resulting
leveldbdatabases will be placed in your configured
db_root. The events are stored as binary Erlang terms in the database so you will need to export the events to use them.
There are two scripts to do that.
./bin/compile_all_stats.shand
./bin/convert_all_stats.sh
./bin/compile_all_stats.shcreates three
.csvtables:
./bin/convert_all_stats.shdumps the raw events as a
events.csvtable.
The events data has the following fields:
timestamp, type, client_id, event_key, event_data
The
event_datais used to pair
ws_initand
ws_onopenevents to calculate the elapsed handshake time and the
event_datais used to pair
send_messageevents to
recv_messageevents to calculate the elapsed message time. The
event_datafor
'EXIT'events take the format of
{pid(), Reason :: any()}. These are some of the expected reasons for a client crash:
connection_timeout - The client could not establish a connection with the server in under 2 seconds normal - The server closed the connection. This should never happen for this benchmark
Any reason other than these are an unexpected error and should be reported as an issue. If you see a
{error, Reason}you should probably file a bug report as well unless you can determine it is a server issue.