uwsgi-sloth

uwsgi-sloth is a realtime uwsgi log analyer, designed for helping optimization of uwsgi app's performance.

It can both generates

a static report file 

Image preview:

.. image:: https://github.com/piglei/uwsgi-sloth/raw/master/uwsgi-sloth-screenshot.png

You should consider using uwsgi-sloth if your website are running under uwsgi and have no conception of how slow/fast your website is.

QuickStart

Install ^^^^^^^

uwsgi-sloth is written in python, to install it, simply use pip:

.. code-block:: bash

# Install a stable version
$ pip install uwsgi-sloth

Install the latest version from github
$ pip install -e git+https://github.com/piglei/uwsgi-sloth#egg=uwsgi-sloth

uwsgi-sloth now only supports Python3.5 and above, for Python2 users, please install uwsgi-sloth<3.0.0 instead.

Static report ^^^^^^^^^^^^^

After installation, you can analyzing your uwsgi log using

uwsgi-sloth analyze

.. code-block:: bash

# Generate a report
$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html

Specify threshold for request process time
$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html --min-msecs=400

Check more:

uwsgi-sloth analyze

Realtime reports ^^^^^^^^^^^^^^^^

We do support a more powerful feature: realtime uwsgi log report. It's a little more complicated to configure.

First, create a default config file using

uwsgi-sloth echo_conf

.. code-block:: bash

uwsgi-sloth echo_conf > /data/uwsgi_sloth/myblog.conf

The default config file are like this:

.. code-block:: ini

# A sample uwsgi-sloth config file

uwsgi log path, only support default log format
uwsgi_log_path = '/your_uwsgi_logs/web.log'
All HTML files and data files will store here, must have read/write permissions
data_dir = '/you_data/uwsgi-sloth/'                          
Minimal msecs for detect slow requests, default to 200
min_msecs = 200
Domain for your website, best given
domain = 'http://www.yourwebsite.com/'
Custom url regular expressions file
url_file = '/your_custom_url_file_path'

After modified

uwsgi_log_path

data_dir

uwsgi-sloth start -c /data/uwsgi_sloth/myblog.conf

[2014-06-26 01:32:56,851] uwsgi_sloth INFO: Start from last savepoint, last_log_datetime: 2014-06-26 09:32:04
[2014-06-26 01:32:58,859] uwsgi_sloth INFO: Rendering HTML file /data/uwsgi_sloth/myblog/html/latest_5mins.html...
... ...

This may take several seconds if your log file are big.

Demonize ~~~~~~~~

uwsgi-sloth does not support built-in deamonize option, so you may need tools like

supervisor 

Serve your reports ~~~~~~~~~~~~~~~~~~

Now, HTML files have been generated, we should configure our webserver so we can visit it, this configuration is for nginx: ::

$ cat /etc/nginx/sites-enabled/sloth_myblog.conf
server {

listen   80;
server_name  uwsgi-sloth.zlovezl.cn;

location / {
    root /data/uwsgi_sloth/myblog/html/;
    index "latest_5mins.html";
}
}

After reloading your nginx config, open your browser then you will see the fancy reports waiting for you.

Commands

uwsgi-sloth analyze ^^^^^^^^^^^^^^^^^^^

Available arguments

::

usage: uwsgi-sloth analyze [-h] -f FILEPATH [--output OUTPUT]
                           [--min-msecs MIN_MSECS] [--domain DOMAIN]
                           [--url-file URL_FILE]

optional arguments:
  -h, --help            show this help message and exit
  -f FILEPATH, --filepath FILEPATH
                        Path of uwsgi log file
  --output OUTPUT       HTML report file path
  --min-msecs MIN_MSECS
                        Request serve time lower than this value will not be
                        counted, default: 200
  --domain DOMAIN       Make url in report become a hyper-link by settings a
                        domain
  --url-file URL_FILE   Customized url rules in regular expression
  --limit-url-groups LIMIT_URL_GROUPS
                        Number of url groups considered, default: 200
  --limit-per-url-group LIMIT_PER_URL_GROUP
                        Number of urls per group considered, default: 20

Using a customized url rules ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, get a list of url regular expressions.

::

$ cat url_rules
# A valid url_rules file are seperated regular expressions
^club/(?P\w+)/(?P\d+)/(?P\d+)/signup/$
^club/signup/success/$
^club/checkin/success/$

Using

--url-file

::

$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html --url-file=url_rules

uwsgi-sloth echo_conf ^^^^^^^^^^^^^^^^^^^^^

Print a default config file

uwsgi-sloth start ^^^^^^^^^^^^^^^^^

Start uwsgi-sloth worker to generate realtime report

::

$ uwsgi-sloth start -h
usage: uwsgi-sloth start [-h] -c CONFIG

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        uwsgi-sloth config file, use "uwsgi-sloth echo_conf"
                        for a default one

Notes

• Only default uwsgi log format is supported at present.
• Tested under python 2.6/2.7
• By default, uwsgi-sloth will classify

  url_path

  by replacing sequential digits part by '(\d+)':

  /users/3074/

  ->

  /users/(\d+)

Any feedbacks are greatly welcomed!