InfluxDB for TinyCore Linux

Source

Author: akeil
Date: 2017-08-13
Version: 1

This post shows how to set up InfluxDB [1] on a RaspberryPi running TinyCore Linux [2].

This is intended as part of a setup for home automation with where InfluxDB is used as a storage backend for statistics.

Tiny Core Extension

Before we can install InfluxDB, we need to create a TinyCore Extension [3] for it.

Download and extract the prebuilt binary from the InfluxDB Downloads [4] site:

$ wget https://dl.influxdata.com/influxdb/releases/influxdb-1.3.2_linux_armhf.tar.gz
$ tar -xzf influxdb-1.3.2_linux_armhf.tar.gz

Note

The download URL is different for each version.

The extracted archive creates a directory tree like this:

etc/
    influxdb/           # default config
    logrotate.d/
usr/
    bin/                # binaries
    lib/
    share/
var/
    lib/
        influxdb/
    log/
        influxdb/

Only a minimal selection of files goes into the TinyCore extension. Create a temporary directory (/tmp/influxdb/usr/local) and place selected files there. The complete destination directory looks like this:

/tmp/influxdb/
    usr/
        local/
            bin/            # binaries
            etc/            # sample config
            tce.installed/  # post-install script

Binaries

From the source usr/ directory, only the binaries are used. This omits the man pages (under share/) and start scripts (under lib/).

Configuration File

The InfluxDB distribution contains a default configuration file which is the starting point for a custom configuration.

Reduce cache sizes to play nice with the available memory (see InfluxDB docs ):

[data]
cache-max-memory-size = 20971520
cache-snapshot-memory-size = 10485760

Disable usage statistics:

reporting-disabled = true

Place the configuration file in the temporary etc directory, naming it influxdb.sample.conf.

Post-Install Script

TinyCore supports a tce.installed script which is run after the extension was loaded. Our post-install script will

  • if no config file is present, copy the sample config to the default location for the config file.
  • create the influxdb user.
  • create the default data directories with correct permissions.

Create a tce.installed script like this:

#!/bin/sh

CFG=/usr/local/etc/influxdb/influxdb.conf
SAMPLE_CFG=/usr/local/etc/influxdb/influxdb.sample.conf
DATADIR=/var/lib/influxdb
USER=influxdb

# if no config exists, use the sample config
if [[ ! -e $CFG ]]; then
    cp $SAMPLE_CFG $CFG
fi

# create the influxdb user
grep -q $USER /etc/passwd
if [[ $? -ne 0 ]]; then
    adduser -S -H $USER
fi

# create the default data directory
if [[ ! -e $DATADIR ]]; then
    mkdir $DATADIR
    chown $USER:nogroup $DATADIR
fi

The script is added to the temporary installation dir under tce.installed/influxdb.

Package Script

To make the process easily repeatable for new versions of InfluxDB, create a package.sh script. The script includes all steps except downloading the distribution files for InfluxDB.

#!/bin/sh
SRC=influxdb-1.3.2-1
ROOT=/tmp/influxdb/usr/local

# download and install required tools
tce-load -w squashfs-tools
tce-load -i squashfs-tools

# cleanup
sudo rm -rf $ROOT

mkdir -p $ROOT/bin
cp -a $SRC/usr/bin/* $ROOT/bin

mkdir -p $ROOT/etc/influxdb
cp influxdb.conf $ROOT/etc/influxdb/influxdb.sample.conf

# startup script
mkdir -p $ROOT/tce.installed
cp tce.installed $ROOT/tce.installed/influxdb


# default permissions
sudo chown -R root:root $ROOT

# special perms for startup script
sudo chown root:staff $ROOT/tce.installed
sudo chown 775 $ROOT/tce.installed

sudo chown root:staff $ROOT/tce.installed/influxdb
sudo chmod 755 $ROOT/tce.installed/influxdb

# pack and cleanup
mksquashfs /tmp/influxdb /tmp/influxdb.tcz
sudo rm -rf /tmp/influxdb

mv /tmp/influxdb.tcz /etc/sysconfig/tcedir/optional/

Configuration

Persistence

We need to persist two locations, using opt/.filetool.lst:

usr/local/etc/influxdb/influxdb.conf
var/lib/influxdb

This will include the configuration file and the data directories in the filetool.sh backup (you still need to remember to execute this before shutdown). That backup is restored during boot.

Storage

The default storage location is /var/lib/influxdb. on TinyCore, this resides in memory and can be persisted using filetool.sh. Since the data files can become quite large, this is not the best option. Both backup and restore will take a long time and the data will consume available memory.

Whether or not to place any part of the database on volatile storage of course depends on whether or not (partial) data loss is a problem. In this case, it is assumed that this is non-critical data and that it is acceptable if some of the data is lost if the system shuts down unexpectedly.

Note

Deciding whether to store the InfluxDB on "disk" (SD-Card) or in memory (with backup/restore on boot) depends on the InfluxDB Storage Engine [5]. there are two important types of storage:

WAL - Write Ahead Log:
  This is stored under /var/lib/influxdb/wal/<database-name> (the wal-dir= directive from config). New writes go into the WAL, i.e. these files receive many writes and change often.
TSM - Time Structured Merge Tree:
  These are read-only files which contain the bulk of the data. They are stored under /var/lib/influxdb/data/<database-name> (the dir= directive from config). During a process named Compaction, data from the WAL is periodically written to TSM files. Also during compaction, existing TSM files are optimized, i.e. rewritten.

The TSM files will probably become too large to keep in memory and are therefore placed on a separate storage (in this case, additional storage that is mounted under (/mnt/storage).

The WAL is kept at the default location, in memory. This has the benefit of faster access times and also reduces writes to the RaspberryPi's SD-card. If the WAL is lost, we lose data from the most recent writes.

[data]
...
dir= "/mnt/storage/influxdb/data"
wal-dir = "/var/lib/influxdb/wal"
...

Start and Stop Scripts

Create two shell scripts to start and stop InfluxDB.

The start script is included in /opt/bootlocal.sh, the stop script in /opt/shutdown.sh.

The start script does the following:

  • set permissions on pid file
  • set permissions on log file
  • start influxd with parameters for pid file and configuration file and redirect stderr to the logfile

The stop script simply reads the process id from the pid file and kills that process.

Start

CONFIG=/usr/local/etc/influxdb/influxdb.conf
PIDFILE=/var/run/influxd.pid
LOGFILE=/var/log/influxdb.log

touch $PIDFILE
chown influxdb:nogroup $PIDFILE

touch $LOGFILE
chown influxdb:staff $LOGFILE

su influxdb -s /bin/sh -c "influxd run -pidfile $PIDFILE -config $CONFIG 2>$LOGFILE"

Stop

pid=$(cat /var/run/influxd.pid)
if [ -n "$pid" ]; then
    kill "$pid"
fi

Backup

There a several options for backing up InfluxDB. On can backup the full database(s) or only selected retention policies. It is also possible to only back up data since a specific timestamp.

The backup can be run locally or over the network and it can be performed while InfluxDB is running.

For the RaspberryPi, we create full backups, using a local cron job. Backups are created daily and a number of the most recent backups is retained.

The basic steps are as follows: - backup the Metastore and selected Databases to a temporary location - if all went well, compress the result (.tar.gz) - delete the oldest backup

To backup the Metastore:

influxd backup /path/to/backup

Each Database is backed up individually by calling

influxd backup -database <NAME> /path/to/backup

See InfluxDB backup and restore [6] for details.

Monitoring with Monit

Use the following checks with monit [7]:

  • Check whether the process exists using the pidfile
  • Make sure it is run by the influxdb user
  • Call the HTTP interface on the /ping endpoint
  • Check if there is a recent backup

These are the monit checks:

check process influxd pidfile /var/run/influxd.pid
    group influxdb
    if failed uid influxdb then alert
    if failed
        port 8086
        protocol http
        request /ping
        status = 204
    then alert

check file influxdbbackup path /mnt/storage/influxdb/backups/snapshot.0.tar.gz
    group backup
    if does not exist then alert
    if timestamp > 25 hours then alert
[1] https://www.influxdata.com/time-series-platform/influxdb/
[2] http://tinycorelinux.net/
[3] http://wiki.tinycorelinux.net/wiki:creating_extensions
[4] https://portal.influxdata.com/downloads#influxdb
[5] https://docs.influxdata.com/influxdb/v1.3/concepts/storage_engine/
[6] https://docs.influxdata.com/influxdb/v1.3/administration/backup_and_restore/
[7] https://mmonit.com/monit/documentation/monit.html