uwsgi-sloth

by piglei

piglei / uwsgi-sloth

Realtime uwsgi log file analyzer, slow requests analyzer.

202 Stars 18 Forks Last release: Not found Apache License 2.0 42 Commits 2 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

uwsgi-sloth

uwsgi-sloth is a realtime uwsgi log analyer, designed for helping optimization of uwsgi app's performance.

It can both generates

a static report file 
_ or analyze your log file in realtime.

Image preview:

.. image:: https://github.com/piglei/uwsgi-sloth/raw/master/uwsgi-sloth-screenshot.png

You should consider using uwsgi-sloth if your website are running under uwsgi and have no conception of how slow/fast your website is.

QuickStart

Install ^^^^^^^

uwsgi-sloth is written in python, to install it, simply use pip:

.. code-block:: bash

# Install a stable version
$ pip install uwsgi-sloth

Install the latest version from github

$ pip install -e git+https://github.com/piglei/uwsgi-sloth#egg=uwsgi-sloth

uwsgi-sloth now only supports Python3.5 and above, for Python2 users, please install uwsgi-sloth<3.0.0 instead.

Static report ^^^^^^^^^^^^^

After installation, you can analyzing your uwsgi log using

uwsgi-sloth analyze
command.

.. code-block:: bash

# Generate a report
$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html

Specify threshold for request process time

$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html --min-msecs=400

Check more:

uwsgi-sloth analyze
_

Realtime reports ^^^^^^^^^^^^^^^^

We do support a more powerful feature: realtime uwsgi log report. It's a little more complicated to configure.

First, create a default config file using

uwsgi-sloth echo_conf
:

.. code-block:: bash

uwsgi-sloth echo_conf > /data/uwsgi_sloth/myblog.conf

The default config file are like this:

.. code-block:: ini

# A sample uwsgi-sloth config file

uwsgi log path, only support default log format

uwsgi_log_path = '/your_uwsgi_logs/web.log'

All HTML files and data files will store here, must have read/write permissions

data_dir = '/you_data/uwsgi-sloth/'

Minimal msecs for detect slow requests, default to 200

min_msecs = 200

Domain for your website, best given

domain = 'http://www.yourwebsite.com/'

Custom url regular expressions file

url_file = '/your_custom_url_file_path'

After modified

uwsgi_log_path
and
data_dir
, your can start uwsgi-sloth worker via
uwsgi-sloth start -c /data/uwsgi_sloth/myblog.conf
command, if everything goes fine, you will see some messages like this: ::
[2014-06-26 01:32:56,851] uwsgi_sloth INFO: Start from last savepoint, last_log_datetime: 2014-06-26 09:32:04
[2014-06-26 01:32:58,859] uwsgi_sloth INFO: Rendering HTML file /data/uwsgi_sloth/myblog/html/latest_5mins.html...
... ...

This may take several seconds if your log file are big.

Demonize ~~~~~~~~

uwsgi-sloth does not support built-in deamonize option, so you may need tools like

supervisor 
_ to manage this process.

Serve your reports ~~~~~~~~~~~~~~~~~~

Now, HTML files have been generated, we should configure our webserver so we can visit it, this configuration is for nginx: ::

$ cat /etc/nginx/sites-enabled/sloth_myblog.conf
server {

listen   80;
server_name  uwsgi-sloth.zlovezl.cn;

location / {
    root /data/uwsgi_sloth/myblog/html/;
    index "latest_5mins.html";
}

}

After reloading your nginx config, open your browser then you will see the fancy reports waiting for you.

Commands

uwsgi-sloth analyze ^^^^^^^^^^^^^^^^^^^

Available arguments

::

usage: uwsgi-sloth analyze [-h] -f FILEPATH [--output OUTPUT]
                           [--min-msecs MIN_MSECS] [--domain DOMAIN]
                           [--url-file URL_FILE]

optional arguments: -h, --help show this help message and exit -f FILEPATH, --filepath FILEPATH Path of uwsgi log file --output OUTPUT HTML report file path --min-msecs MIN_MSECS Request serve time lower than this value will not be counted, default: 200 --domain DOMAIN Make url in report become a hyper-link by settings a domain --url-file URL_FILE Customized url rules in regular expression --limit-url-groups LIMIT_URL_GROUPS Number of url groups considered, default: 200 --limit-per-url-group LIMIT_PER_URL_GROUP Number of urls per group considered, default: 20

Using a customized url rules ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, get a list of url regular expressions.

::

$ cat url_rules
# A valid url_rules file are seperated regular expressions
^club/(?P\w+)/(?P\d+)/(?P\d+)/signup/$
^club/signup/success/$
^club/checkin/success/$

Using

--url-file
to specify this url_rules

::

$ uwsgi-sloth analyze -f uwsgi_access.log --output=report.html --url-file=url_rules

uwsgi-sloth echo_conf ^^^^^^^^^^^^^^^^^^^^^

Print a default config file

uwsgi-sloth start ^^^^^^^^^^^^^^^^^

Start uwsgi-sloth worker to generate realtime report

::

$ uwsgi-sloth start -h
usage: uwsgi-sloth start [-h] -c CONFIG

optional arguments: -h, --help show this help message and exit -c CONFIG, --config CONFIG uwsgi-sloth config file, use "uwsgi-sloth echo_conf" for a default one

Notes

  • Only default uwsgi log format is supported at present.
  • Tested under python 2.6/2.7
  • By default, uwsgi-sloth will classify
    url_path
    by replacing sequential digits part by '(\d+)':
    /users/3074/
    ->
    /users/(\d+)

Any feedbacks are greatly welcomed!

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.