OwnTracks Recorder

The OwnTracks Recorder is a lightweight program for storing and accessing location data published via MQTT (or HTTP) by the OwnTracks apps. It is a compiled program which is easy to install and operate even on low-end hardware, and it doesn't require an external database.

There are two main components: the Recorder obtains data via MQTT subscribes or HTTP POST, stores the data in plain files and serve it via its built-in REST API, and the

ocat

We developed the Recorder as a one-stop solution to storing location data published by our OwnTracks apps (iOS and Android) and retrieving this data. Our previous offerings (

m2s

o2s

Pista

See also HOOKS.md, DESIGN.md, and STORE.md for more information on Lua hooks, program design, and storage layout, respectively.

Table of Contents

• recorder

• Installing
  • Packages
  • systemd service
  • Docker
  • Building from source
  • Prerequisites
  • Building
• Getting started

• ot-recorder

  options
• Configuration file
• Reverse proxy
  • nginx
  • Apache
• The HTTP Server
  • Example functionality
  • Last position of a particular user
  • Display map with points starting at a particular date
  • Display a track (a.k.a. linestring)
  • Tabular display
  • Live map

• ocat

  • Environment
  • Examples
  • List users and devices
  • Show the last position reported by a user
  • What were the last 4 positions reported?
• Reverse Geo
  • Precision
  • The geo cache
• Monitoring
• Lua hooks
• Views
  • view JSON
  • Authentication
• HTTP mode
  • Friends in HTTP mode
• Advanced topics
  • Browser API keys
  • The LMDB database

  • topic2tid

  • keys

  • friends

• Encryption
• Tips and Tricks
  • Gatewaying HTTP to MQTT
  • Override reverse-geo precision

recorder

The Recorder serves two purposes:

1. It subscribes to an MQTT broker and reads messages published from the OwnTracks apps, storing these in a particular fashion into what we call the store which is basically a bunch of plain files on the file system. Alternatively the Recorder can listen on HTTP for OwnTracks-type JSON messages POSTed to its HTTP server.
2. It provides a Web server which serves static pages, a REST API you use to request data from the store, and a WebSocket server. The distribution comes with a few examples of how to access the data through its HTTP interface (REST API). In particular a table of last locations has been made available as well as a live map which updates via the Recorder's WebSocket interface when location publishes are received. In addition we provide maps with last points or tracks using the GeoJSON produced by the Recorder.

Installing

We provide ready-to-run packages for a limited number of platforms on our package repository, and we provide a Docker image of the Recorder directly from the Docker hub.

If those don't work for you, you can build from source.

Packages

We create packages for releases for a few distributions. Please note:

• that these packages might depend on libmosquitto1 from the Mosquitto project. In particular, for Debian Stretch (9) this is not the case.
• we typically do not build packages for distributions which have been superceded upstream

Binaries (

ocat

ot-recorder

owntracks

owntracks

If you're on CentOS 7, you can install with:

sudo curl -o /etc/yum.repos.d/mosquitto.repo http://download.opensuse.org/repositories/home:/oojah:/mqtt/CentOS_CentOS-7/home:oojah:mqtt.repo

sudo curl -o /etc/yum.repos.d/owntracks.repo http://repo.owntracks.org/centos/owntracks.repo
sudo yum install ot-recorder

Debian 7 "Wheezy" (including Rasbian):

• we no longer build for this distribution

curl https://raw.githubusercontent.com/owntracks/recorder/master/etc/repo.owntracks.org.gpg.key | sudo apt-key add -
echo "deb  http://repo.owntracks.org/debian wheezy main" | sudo tee /etc/apt/sources.list.d/owntracks.list > /dev/null
sudo apt-get update
sudo apt-get install ot-recorder

Debian 8 "Jessie":

• we no longer build for this distribution

curl https://raw.githubusercontent.com/owntracks/recorder/master/etc/repo.owntracks.org.gpg.key | sudo apt-key add -
echo "deb  http://repo.owntracks.org/debian jessie main" | sudo tee /etc/apt/sources.list.d/owntracks.list > /dev/null
sudo apt-get update
sudo apt-get install libsodium-dev libsodium13
sudo apt-get install ot-recorder

Debian 9 "Stretch":

using Stretch Mosquitto packages:

curl https://raw.githubusercontent.com/owntracks/recorder/master/etc/repo.owntracks.org.gpg.key | sudo apt-key add -
echo "deb  http://repo.owntracks.org/debian stretch main" | sudo tee /etc/apt/sources.list.d/owntracks.list > /dev/null
sudo apt-get update
sudo apt-get install ot-recorder

Debian 10 "Buster":

using Buster Mosquitto packages:

curl https://raw.githubusercontent.com/owntracks/recorder/master/etc/repo.owntracks.org.gpg.key | sudo apt-key add -
echo "deb  http://repo.owntracks.org/debian buster main" | sudo tee /etc/apt/sources.list.d/owntracks.list > /dev/null
sudo apt-get update
sudo apt-get install ot-recorder

systemd service

The packages we provide have a systemd unit file in

/usr/share/doc/ot-recorder/ot-recorder.service

1. Ensure you have a configuration file with the settings you require

2. install -m444 /usr/share/doc/ot-recorder/ot-recorder.service /etc/systemd/system/ot-recorder.service

3. Enable the service to run at startup:

   systemctl enable ot-recorder

4. Launch the service

   systemctl start ot-recorder

Docker

We also have a Docker image to create a container with the Recorder. The Docker image is available from the Docker hub (e.g.

docker pull owntracks/recorder

Building from source

Prerequisites

You will require:

• libmosquitto unless you disable MQTT during building, but see below for platform instructions
• libCurl
• lmdb (included)
• libconfig
• Optionally Lua
• Optionally libsodium for secret-key encryption of payloads

You need a current version of libmosquitto (and you probably require the Mosquitto broker as well for OwnTracks). We strongly recommend installing Mosquitto either from source or from a binary package, both of which are provided by the Mosquitto project. In particular, older or LTS OS versions profit from this.

On Debian, you can install the needed packages with:

apt-get install build-essential linux-headers-$(uname -r) libcurl4-openssl-dev libmosquitto-dev liblua5.2-dev libsodium-dev libconfig-dev

On CentOS 7:

yum groupinstall 'Development Tools'
yum install libmosquitto-devel libcurl-devel lua-devel libsodium-devel libconfig-devel

(libsodium is in epel-stable)

On Ubuntu:

sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
sudo apt-get update
sudo apt-get install libmosquitto-dev libcurl3 libcurl4-openssl-dev libconfig-dev liblmdb-dev

Building

1. Obtain and download the software, via our Homebrew Tap on Mac OS X, directly as a clone of the repository, or as a tar ball which you unpack.
2. Copy the included

   config.mk.in

   file to

   config.mk

   and edit that. You specify the features or tweaks you need. (The file is commented.) Pay particular attention to the installation directory and the value of the store (

   STORAGEDEFAULT

   ): that is where the Recorder will store its files.

   DOCROOT

   is the root of the directory from which the Recorder's HTTP server will serve files.
3. Type

   make

   and watch the fun.

When

make

ot-recorder

ocat

make install

--doc-root ./docroot

Ensure the LMDB databases are initialized by running the following command which is safe to do, also after an upgrade. (This initialization is non-destructive -- it will not delete any data.)

ot-recorder --initialize

Getting started

The Recorder has, like

ocat

-h

--help

If you require authentication or TLS to connect to your MQTT broker, pay attention to the

$OTR_

Launch the Recorder:

$ ./ot-recorder 'owntracks/#'

(In HTTP mode, you do not have to specify a topic.)

Publish a location from your OwnTracks app and you should see the Recorder receive that on the console. If you haven't disabled Geo-lookups, you'll also see the address from which the publish originated.

The location message received by the Recorder will be written to storage. In particular you should verify that your storage directory contains:

1. a directory called

   ghash/

2. a directory called

   rec/

   with several subdirectories and a

   .rec

   file therein.
3. a directory called

   last/

   which contains subdirectories and a

   .json

   file therein.

When the Recorder has received a publish or two, visit it with your favorite Web browser by pointing your browser at

http://127.0.0.1:8083

--http-host

--http-port

Unless already provided by the package you installed, we recommend you create a shell script with which you hence-force launch the Recorder. Note that you can have it subscribe to multiple topics, and you can launch sundry instances of the Recorder (e.g. for distinct brokers) as long as you ensure:

• that each instance uses a distinct

  --storage

• that each instance uses a distinct

  --http-port

  (or

  0

  if you don't wish to provide HTTP support for a particular instance)

You also need to provide API keys for the maps.

Note: all usernames and device names stored by the Recorder are lowercased which means filenames etc. are also in lowercase. As an example, if your device publishes topics for user

Fred

fred

Phone

phone

ot-recorder

options

This section lists the most important options of the Recorder with their long names; check the usage (

recorder -h

--clientid

--host

$OTR_HOST

--port

$OTR_PORT

--identity

--psk

--cafile

--user

$OTR_USER

--qos

--storagedir

$OTR_STORAGEDIR

--useretained

--norec

otr_putrec()

/locations

--norevgeo

ocat

--logfacility

LOCAL0

--quiet

--initialize

--label

--http-host

--http-port

--http-port

--docroot

--viewsdir

/views

--lua-script

--precision

--geokey

"opencage:"

--geokey "opencage:xxxxxxxxxxxxxxxxxxxxxx"      # for OpenCage
--geokey "xxxxxxxxxxxxxxxxxxxxxx"               # for Google
--geokey "revgeod:localhost:8865"               # for Revgeod (host:port)

(The rules of the game for using Google as reverse geocoder changed in May 2018; make sure to check Google's Map Project documentation before using this)

--debug

Configuration file

The Recorder attempts to read its startup configuration from a configuration file; the path to this is compiled into the Recorder (typically

/etc/default/ot-recorder

ocat -v

#

;

OTR_STORAGEDIR="/var/spool/owntracks/recorder/store"

The following configuration settings may be applied (a

Y

$

| Variable | $ | Default | Usage | --------------------- | :--- | :------------ | --------------- |

OTR_STORAGEDIR

OTR_HOST

localhost

OTR_PORT

1883

OTR_USER

OTR_PASS

OTR_QOS

2

OTR_CLIENTID

OTR_HTTPHOST

localhost

OTR_HTTPPORT

8083

OTR_HTTPLOGDIR

OTR_LUASCRIPT

OTR_PRECISION

7

OTR_GEOKEY

OTR_BROWSERAPIKEY

OTR_TOPICS

OTR_CAFILE

OTR_CAPATH

|  Y    |               | Path to PEM-encoded client certificate for MQTT
|

Note that options passed to

ot-recorder

Reverse proxy

Running the Recorder protected by an Nginx or Apache server is possible and is the only recommended method if you want to serve data behind localhost. The snippets below show how to do it, but you would also add authentication to them - or at least, to everything but the views. The snippet for HTTP mode shows an example of how to do this.

nginx

server {
    listen       8080;
    server_name  192.168.1.130;

location / {
    root   html;
    index  index.html index.htm;
}

# Proxy and upgrade WebSocket connection
location /owntracks/ws {
    rewrite ^/owntracks/(.*)    /$1 break;
    proxy_pass      http://127.0.0.1:8083;
    proxy_http_version  1.1;
    proxy_set_header    Upgrade $http_upgrade;
    proxy_set_header    Connection "upgrade";
    proxy_set_header    Host $host;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
}

location /owntracks/ {
    proxy_pass      http://127.0.0.1:8083/;
    proxy_http_version  1.1;
    proxy_set_header    Host $host;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    X-Real-IP $remote_addr;
}

# OwnTracks Recorder Views
location /owntracks/view/ {
     proxy_buffering         off;            # Chrome
     proxy_pass              http://127.0.0.1:8083/view/;
     proxy_http_version      1.1;
     proxy_set_header        Host $host;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header        X-Real-IP $remote_addr;
}
location /owntracks/static/ {
     proxy_pass              http://127.0.0.1:8083/static/;
     proxy_http_version      1.1;
     proxy_set_header        Host $host;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header        X-Real-IP $remote_addr;
}

# HTTP Mode
location /owntracks/pub {
    auth_basic              "OwnTracks pub";
    auth_basic_user_file    /usr/local/etc/nginx/owntracks.htpasswd;
    proxy_pass              http://127.0.0.1:8083/pub;
    proxy_http_version      1.1;
    proxy_set_header        Host $host;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Real-IP $remote_addr;

    # Optionally force Recorder to use username from Basic
    # authentication user. Whether or not client sets
    # X-Limit-U and/or uses ?u= parameter, the user will
    # be set to $remote_user.
    proxy_set_header        X-Limit-U $remote_user;
}
}

Apache

This will hand URIs which begin with

/owntracks/

# WebSocket URL endpoint
# a2enmod proxy_wstunnel
ProxyPass        /owntracks/ws        ws://127.0.0.1:8083/ws keepalive=on retry=60
ProxyPassReverse /owntracks/ws        ws://127.0.0.1:8083/ws keepalive=on

Static files
ProxyPass /owntracks                  http://127.0.0.1:8083/
ProxyPassReverse /owntracks           http://127.0.0.1:8083/
TODO: add views

Caddy

recorder.example.org {
    encode gzip
    reverse_proxy http://localhost:8083
    basicauth * {
        # create new password hashes with `caddy hash-password --plaintext yourpasswordhere`
        john_doe JDJhJDE0JDBNc0FyNGd3b3JzejBTaExUZnkxdnV2THZSUUdHNzJYZmdVTzN3NlZXcjh2YXJLQi5KY04u
    }
}

The HTTP server

The Recorder has a built-in HTTP server with which it servers static files from either the compiled-in default

DOCROOT

--doc-root

/api/0/

The API basically serves the same data as

ocat

/pub

Example functionality

Some examples of what the server can do:

Last position of a particular user

Retrieve the last position of a particular user. In addition to the values obtained in the

location

• username

  contains the name of the user obtained from the publish topic

• device

  contains the user's device name as obtained from the publish topic

• topic

  is the full topic to which the payload was published

• ghash

  is the geohash string which corresponds to

  lat

  and

  lon

• isotst

  is the ISO timestamp of the publish time (

  tst

  )

• disptst

  is the same but designed for displaying

• cc

  is the country code of the location point if available in the cache (see below)

• addr

  is the address of the location point if available in the cache

$ curl http://127.0.0.2:8083/api/0/last -d user=demo -d device=iphone
[
 {
  "tst": 1440405601,
  "acc": 10,
  "_type": "location",
  "alt": 262,
  "lon": 13.60279820860699,
  "vac": 6,
  "vel": 18,
  "lat": 51.06263391678321,
  "cog": 82,
  "tid": "NE",
  "batt": 99,
  "username": "demo",
  "device": "iphone",
  "topic": "owntracks/demo/iphone",
  "ghash": "u31dmx9",
  "isotst": "2015-08-24T08:40:01Z",
  "disptst": "2015-08-24 08:40:01",
  "cc": "DE",
  "addr": "E40, 01156 Dresden, Germany"
 }
]

Display map with points starting at a particular date

By specifying a

format

from

to

http://127.0.0.2:8083/map/index.html?user=demo&device=iphone&format=geojson&from=2014-01-01

In a suitable Web browser, the result is

Display a track (a.k.a linestring)

If we change the

format

linestring

Tabular display

The Recorder's Web server also provides a tabular display which shows the last position of devices, their address, country, etc. Some of the columns are sortable, you can search for users/devices and click on the address to have a map opened at the device's last location.

Live map

The Recorder's built-in WebSocket server updates a map as it receives publishes from the OwnTracks devices. Here's an example:

ocat

ocat

• JSON
• GeoJSON (points)
• GeoJSON (line string)
• CSV
• GPX
• XML
• raw (the lines contained in the REC file with ISO timestamp)
• payload (basically just the payload part from RAW)

The

ocat

ocat

Some example uses we consider useful:

• ocat --list

  show which users are in the store.

• ocat --list --user jjolie

  show devices for the specified user

• ocat --user jjolie --device ipad

  print JSON data for the user's device produced during the last 6 hours.

• ocat --last

  print the LAST position of all users, devices. Can be combined with

  --user

  and

  --device

  .

• ocat ... --format csv

  produces CSV. Limit the fields you want extracted with

  --fields lat,lon,cc

  for example.

• ocat ... --format xml

  produces XML. Limit the fields you want extracted with

  --fields lat,lon,cc

  for example. ```xml <?xml version='1.0' encoding='UTF-8'?> <?xml-stylesheet type='text/xsl' href='owntracks.xsl'?> 1440405601 10.000000 262 13.602798 6.000000 18 51.062634 82 NE 99 demo iphone owntracks/demo/iphone u31dmx9 2015-08-24T08:40:01Z 2015-08-24 08:40:01 DE E40, 01156 Dresden, Germany

``

*

   prints data  for the current month, starting now and going backwards; only 10 locations will be printed. Generally, the

Specifying

--fields lat,tid,lon

lat

The

--from

--to

--from

--to

%Y-%m-%dT%H:%M:%S
%Y-%m-%dT%H:%M
%Y-%m-%dT%H
%Y-%m-%d
%Y-%m

The

--limit

.rec

--limit

Environment

The following environment variables control

ocat

• OCAT_FORMAT

  can be set to the preferred output format. If unset, JSON is used. The

  --format

  option overrides this setting.

• OCAT_USERNAME

  can be set to the preferred username. The

  --user

  option overrides this environment variable.

• OCAT_DEVICE

  can be set to the preferred device name. The

  --device

  option overrides this environment variable.

Examples

The Recorder has been running for a while, and the OwnTracks apps have published data. Let us have a look at some of this data.

List users and devices

We obtain a list of users from the store:

$ ocat --list
{
 "results": [
  "demo"
 ]
}

From which devices has user

demo

$ ocat --list --user demo
{
 "results": [
  "iphone"
 ]
}

Show the last position reported by a user

Where was

demo

--user

--device

$ ocat --last --user demo --device iphone
[
 {
  "tst": 1440405601,
  "acc": 10,
  "_type": "location",
  "alt": 262,
  "lon": 13.60279820860699,
  "vac": 6,
  "vel": 18,
  "lat": 51.06263391678321,
  "cog": 82,
  "tid": "NE",
  "batt": 99,
  "username": "demo",
  "device": "iphone",
  "topic": "owntracks/demo/iphone",
  "ghash": "u31dmx9",
  "isotst": "2015-08-24T08:40:01Z",
  "disptst": "2015-08-24 08:40:01",
  "cc": "DE",
  "addr": "E40, 01156 Dresden, Germany"
 }
]

Several things worth mentioning:

• The returned data structure is an array of JSON objects; had we omitted specifying a particular device or even a particular user we would have obtained the last position of all this user's devices or all users' devices respectively.
• If you are familiar with the JSON data reported by the OwnTracks apps you'll notice that this JSON contains more information: this is provided on the fly by

  ocat

  and the REST API, e.g. from the reverse-geo cache the Recorder maintains.

What were the last 4 positions reported?

We can limit the number of returned elements: Let's do this as CSV, and limit the fields we are given:

$ ocat --user demo --device iphone --limit 4 --format csv --fields isotst,vel,addr
isotst,vel,addr
2015-08-24T08:40:01Z,18,"E40, 01156 Dresden, Germany"
2015-08-24T08:35:01Z,40,"E40, 01723 Wilsdruff, Germany"
2015-08-24T08:30:00Z,50,"A14, 01683 Nossen, Germany"
2015-08-24T08:24:59Z,40,"A14, 04741 Roßwein, Germany"

Reverse Geo

If not disabled with option

--norevgeo

1. obtained via a Lua function you define (see doc/HOOKS.md)
2. obtained via a call to one of the supported reverse geocoders (see

   --geokey

   )

Results of lookups are stored in an LMDB database. If a lookup is not possible, for example because you're over quota, the service isn't available, etc., Recorder keeps tracks of the coordinates which could not be resolved in a file named

missing

$ cat store/ghash/missing
u0tfsr3 48.292223 8.274535
u0m97hc 46.652733 7.868803
...

This can be used to subsequently obtain missed lookups.

We recommend you keep reverse-geo lookups enabled, this data (country code

cc

addr

ocat

Precision

The precision with which reverse-geo lookups are performed is controlled with the

--precision

--precision

ocat

config.mk

precision

As an example, let's assume Jane's device is at position (lat, lon)

48.879840, 2.323522

u09whf7

Every location publish outside that very small blue square would mean another lookup. If, however, we lower the precision to, say, 5, a much larger area is covered

and a precision of 2 would mean that a very large part of France resolves to a single address:

The bottom line: if you run the Recorder with just a few devices and want to know quite exactly where you've been, use a high precision (7 is probably good). If you, on the other hand, run Recorder with many devices and are only interested in where a device was approximately, lower the precision; this also has the effect that fewer reverse-geo lookups will be performed in the geocoding service infrastructure. (Also: respect their quotas!)

The geo cache

As hinted to above, the address data obtained through a reverse-geo lookup is stored in an embedded LMDB database, the content of which we can look at with

$ ocat --dump
u09whf7 {"cc":"FR","addr":"1 Rue de Saint-Pétersbourg, 75008 Paris, France","tst":1445435622,"locality":"Paris"}
u09ey1r {"cc":"FR","addr":"D83, 91590 La Ferté-Alais, France","tst":1445435679,"locality":"La Ferté-Alais"}

The key to this data is the geohash string (here with an example of precision 2).

Lua hooks

You can customize Recorder's behavior with Lua hooks. See HOOKS.md.

Monitoring

In order to monitor the Recorder, whenever an MQTT message is received, a

monitor

1439738692 owntracks/jjolie/ipad

If Recorder is built with

WITH_PING

owntracks/ping/ping

ping

ping

.REC

After sending a pingping, you can query the REST interface to determine the difference in time. The

contrib/

ot-ping.py

OK ot-recorder pingping at http://127.0.0.1:8085: 0 seconds difference

Note that our Docker image has this monitoring built in with recorder-health.sh. What this basically does is it allows Docker to monitor the Recorder's availability in HEALTHCHECK. So, the

ping/ping

Views

A view is a sort of sandboxed look at data provided by the Recorder. Assume you host several devices, be they your own or those of some of your friends, and assume you want to allow somebody else to see where you are or have been during a specific time frame: with the Recorder's default Web server you cannot limit a visitor to see specific data only; once they reach the Recorder's Web interface, they have access to all your data. (We warned you about that earlier.) Using a HTTP proxy, you can provide an insight into certain portions of your data only.

You configure a view by creating a small JSON file of an arbitrary name which defines which user / device combination of data the view should display. Say you are recording data for

owntracks/jjolie/phone

jjolie

phone

vmap.html

The view then provides three URLs:

• views/viewname

  can serve a HTML or text page

• views/viewname?lastpos=1

  serves a JSON array of objects with the last position recorded

• views/viewname?geodata=1

  serves a GeoJSON object containing recorded track data

Suppose Jane wishes to have her acquaintances see where she is while on vacation. Jane knows she'll be en-route between 2015-06-29 and 2015-07-15. She creates a file called, say,

loire.json

views/

{
  "user"  : "jjolie",
  "device": "phone",
  "page"  : "vmap.html",
  "from"  : "2015-06-29",
  "to"    : "2015-07-15"
}

Jane's friends can now visit the URL

/view/loire

.json

It's recommended that you configure your reverse proxy to show views. You can find an example of this for nginx in the reverse proxy section.

view JSON

The JSON in the view file (called

view.json

| element | mandatory | meaning | | ----------------| :---------: | ------------------------------------- | | user | Y | username for data (from topic owntracks/user/device | | device | Y | device for data (from topic owntracks/user/device | | page | Y | HTML page to be loaded from

docroot/views/

from

to

from

to

vmap.html

vmap.html

The

page

views/

• @@@[email protected]@@

  is converted to a URI on which the Recorder will serve the last position data

• @@@[email protected]@@

  is converted to a URI on which the Recorder will serve GeoJSON data from its storage.

The default

page

vmap.html

from

to

A little bit more complex view would look like this:

{
  "config": {
    "port": 9001,
    "pathname": "/tmp/somewhere"
  },
  "zoom": 7,
  "label": "Jane's Loire vacation",
  "to": "2015-07-15",
  "from": "2015-06-29",
  "device": "phone",
  "user": "jjolie",
  "page": "vmap.html"
}

All JSON elements are copied into the

lastpos

http://localhost:8083/view/loire?lastpos=1

{
 "data": [
  {
   "_type": "location",
   "cc": "FR",
   "lon": -1.564214,
   "lat": 47.217871,
   "alt": 35,
   "vel": 0,
   "t": "L",
   "cog": 0,
   "tid": "K2",
   "tst": 1436895718,
   "ghash": "gbqus7u",
   "addr": "Maison d'arrêt, 9 Rue Descartes, 44000 Nantes, France",
   "locality": "Nantes",
   "isorcv": "2015-07-14T19:41:58Z",
   "isotst": "2015-07-14T17:41:58Z",
   "disptst": "2015-07-14 17:41:58",
   "page": "vmap.html",
   "user": "jjolie",
   "device": "phone",
   "from": "2015-06-29",
   "to": "2015-07-15",
   "label": "Jane's Loire vacation",
   "zoom": 7,
   "pathname": "/tmp/somewhere",
   "port": 9001
  }
 ]
}

Note how

pathname

port

page

Recall that while we typically say

htdocs/views/

--viewsdir

Authentication

If

view.json

auth

owntracks-recorder

"auth" : [ "225544f9acf99d18a8880c5ce844f303", "ba69e267302a7ef98e0862b9aae68cab" ]

Each user/password digest combination will be able to access the view.

You create these strings with, say, the

htdigest

contrib/new-view-auth.py

htdigest -c /tmp/dd owntracks-recorder jjolie
Adding password for jjolie in realm owntracks-recorder.
New password:
Re-type new password:

cat /tmp/dd
jjolie:owntracks-recorder:225544f9acf99d18a8880c5ce844f303

In the above example, you copy the 32-character digest into your

view.json

./new-view-auth.py jjolie
Enter password for user jjolie:
Re-enter password:
"auth" : [ "225544f9acf99d18a8880c5ce844f303" ]

HTTP mode

If enabled at compile time (

WITH_HTTP

/pub&u=username&d=device

u

d

X-Limit-U

X-Limit-D

owntracks

phone

curl --data "${payload}" 'http://127.0.0.1:8085/pub?u=jane&d=3s'
curl -H 'X-Limit-U: jane' -H 'X-Limit-D: 3s' --data "${payload}" 'http://127.0.0.1:8085/pub'

The content of the request is used by the Recorder as though it had arrived as an MQTT message; Lua hooks and WebSocket pushes are handled accordingly.

Friends in HTTP mode

When a device posts a location request in HTTP mode, the endpoint may return a JSON array of OwnTracks objects of which

_type

cmd

location

card

$ ocat -S JP --dump=friends
jane-phone ["john/android", "jane/phone"]

when user

jane

phone

[
 {
  "_type": "card",
  "tid": "JA",
  "face": "/9j/4AAQSkZJR...",
  "name": "John Doe"
 },
 {
  "_type": "location",
  "tid": "JA",
  "lat": 48.95833,
  "lon": 2.39523,
  "tst": 1456212791
 }
]

Note, that Jane's user/device tuple should also be returned in order to display Jane on the map or list of friends in the apps.

Advanced topics

Browser API keys

By default, the Recorder uses OpenStreetMap to view points and tracks.

In order to use Google Maps, you have to obtain a Google API "Browser key". You then pass this key to your Recorder by configuring it in the defaults configuration file or by overriding that with the option:

$ ot-recorder --browser-apikey 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' ...

Do not configure

browser-apikey

The LMDB database

ocat --load

ocat --dump

--load=xx

--dump=xx

--load

DELETE

ocat --dump |
    grep xxyz |
    awk '{printf "%s DELETE\n", $1; }' |
    ocat --load

topic2tid

This named lmdb database is keyed on topic name (

owntracks/jane/phone

tid

keys

If the Recorder was built with encryption support (see below), this named database contains the secret decryption keys for users/device pairs. The LMDB key is the username followed by a dash followed by the device name, all lower case, with spaces translated to a single dash. For example, if user Jjolie with device iPhone needs a secret entered, the database key will be

jjolie-iphone

echo "jjolie-iphone s3cr1t" | ocat --load=keys

Beware: these secret keys are stored in plain text so the database must be protected!

friends

For http mode, the

friends

"user:device"

"user/device"

jane

phone

john

android

ocat --load=friends <
The user/device separator in the array's strings may be a slash (
/
), a dash (
-
), or a colon (
:
).

Encryption

If compiled with 
WITH_ENCRYPT
 support (this is the default in our packages), the Recorder will handle messages from OwnTracks devices which support payload encryption. Each user / device requires a secret key which is configured on the device and which must be configured on the Recorder host in order for the Recorder to be able to decrypt the payloads.

Upon successful decryption, the Recorder processes the original (device-transmitted) JSON and stores the result in plain (i.e. un-encrypted) form in the store, adding a 
"_decrypted":true
 tag to the JSON.

Tips and Tricks

Gatewaying HTTP to MQTT

It actually is possible to gateway location publishes arriving via HTTP into MQTT, though you should be careful not to create loops. You can accomplish this with one or more Lua hooks using MQTT from within Lua.

Override reverse-geo precision

If a payload is received with an element called 
_geoprec
 it contains an override for the Recorder's configured reverse-geo precision. So, for example, if Recorder is running with precision 7, say, and the received payload contains 
"_geoprec" : 2
 the 2 will be used for this particular publish. This is not used in the OwnTracks apps, but it can be used with payloads you generate otherwise. If 
_geoprec
 is negative, new reverse geo lookups will not be performed, but cached entries of 
abs(_geoprec)
 will be used.