InfluxDB for TinyCore Linux (ARM)

Source

Author: akeil
Date: 2017-08-12
Version: 1

This post shows how to set up InfluxDB [1] on a RasperryPi running TinyCore Linux [2].

Extension

First, cCreate a TinyCore Extension [3] for InfluxDB.

Download and extract the prebuilt binary from the InfluxDB Downloads [4] site:

$ wget https://dl.influxdata.com/influxdb/releases/influxdb-1.3.2_linux_armhf.tar.gz
$ tar -xzf influxdb-1.3.2_linux_armhf.tar.gz

The download URL is different for each version.

The extracted archive creates a directory tree like this:

etc/
    influxdb/           # default config
    logrotate.d/
usr/
    bin/                # binaries
    lib/
    share/
var/
    lib/
        influxdb/
    log/
        influxdb/

We create a temporary install directory (/tmp/influxdb/usr/local) and place selected files there.

Custom Config

The InfluxDB distribution contains a default configuration file which is the starting point for a custom configuration.

Reduce cache sizes to play nice with the available memory (see InfluxDB docs ):

[data]
cache-max-memory-size = 20971520
cache-snapshot-memory-size = 10485760

Disable usage statistics:

reporting-disabled = true

Place the configuration file in the temporary install, naming it influxdb.sample.conf.

Binaries

From the usr/ directory, only the binaries are used. This omits the man pages (under share/) and start scripts (under lib/).

Post-Install Script

TincYore supports a tce.installed script which is run after the extension was loaded. Our post-install script will

  • if no config file is present, copy the sample config to the default location for the config file.
  • create the influxdb user.
  • create the default data directories with correct permissions.

Create a tce.installed script like this:

#!/bin/sh

CFG=/usr/local/etc/influxdb/influxdb.conf
SAMPLE_CFG=/usr/local/etc/influxdb/influxdb.sample.conf
DATADIR=/var/lib/influxdb
USER=influxdb

# if no config exists, use the sample config
if [[ ! -e $CFG ]]; then
    cp $SAMPLE_CFG $CFG
fi

# create the influxdb user
grep -q $USER /etc/passwd
if [[ $? -ne 0 ]]; then
    adduser -S -H $USER
fi

# create the default data directory
if [[ ! -e $DATADIR ]]; then
    mkdir $DATADIR
    chown $USER:nogroup $DATADIR
fi

The script is added to the temporary installation dir under tce.installed/influxdb.

Package Script

To make the process easily repeatable for new versions of InfluxDB, create a package.sh script. The script includes all steps except downloading the distribution files for InfluxDB.

#!/bin/sh
SRC=influxdb-1.3.2-1
ROOT=/tmp/influxdb/usr/local

# download and install required tools
tce-load -w squashfs-tools
tce-load -w compiletc
tce-load -i squashfs-tools
tce-load -i compiletc

# cleanup
sudo rm -rf $ROOT

mkdir -p $ROOT/bin
cp -a $SRC/usr/bin/* $ROOT/bin

mkdir -p $ROOT/etc/influxdb
cp influxdb.conf $ROOT/etc/influxdb/influxdb.sample.conf

# startup script
mkdir -p $ROOT/tce.installed
cp tce.installed $ROOT/tce.installed/influxdb


# default permissions
sudo chown -R root:root $ROOT

# special perms for startup script
sudo chown root:staff $ROOT/tce.installed
sudo chown 775 $ROOT/tce.installed

sudo chown root:staff $ROOT/tce.installed/influxdb
sudo chmod 755 $ROOT/tce.installed/influxdb


# pack and cleanup
mksquashfs /tmp/influxdb /tmp/influxdb.tcz
sudo rm -rf /tmp/influxdb

mv /tmp/influxdb.tcz /etc/sysconfig/tcedir/optional/

Start and Stop Scripts

Create start and stop scripts (see below) and place them in usr/local/lib/influxdb/scripts

Note

This does NOT seem to work. Scripts are installed, but calling them from bootlocal.sh gives an error

TODO: checkout start-stop-daemon with init scripts?

mkdir -p $ROOT/lib/influxdb/scripts
cp scripts/* $ROOT/lib/influxdb/scripts

The start script does the following:

  • set permissions on pid file
  • set permissions on log file
  • start influxd with parameters for pid file and configuratio file and redirect stderr to the logfile

Since this is not the default location for influxdb, pass the -config option when starting:

influxd -config /usr/local/etc/influxdb/influxdb.conf

The stop script simply reads the process id from the pid file and kills that process.

Configuration

Persistance

We need to persist two locations, using opt/.filetool.lst:

usr/local/etc/influxdb/influxdb.conf
var/lib/influxdb

Storage

The default storage location is /var/lib/influxdb. By default, this directory is not persisted and its contents will be lost on reboot. If we add it to the .filetool.lst it will be backed up and restored. However, since some of the data files become quite large, this is not the best option.

Note

Deciding whether to store the InfluxDB on "disk" (SD-Card) or in memory (with backup/restore on boot) depends on the InfluxDB Storage Engine [5].

Influx stores data in read-only files which are organized into Shards. In the default setup, each shard contains data from 7 days. This means a shard will not change once its completly written.

We can keep these shards in memory and maintain a backup on disk. Since the live data will never change, we can replace it with a backup version any time without further considerations.

The write ahead log (WAL) is constantly written to. Several WAL files exist, named 0001.wal, 0002.wal, ... Only the highest number file is in write-use

The shards are stored in the data directory (default: /var/lib/influxdb/data) and may become quite large. We will place these directly on persistent storage and not in memory. Open the config file. In the [data] section, change the dir= directive to a different location (in this case, additional storage that is mounted under (/mnt/storage).

[data]
...
dir=/mnt/storage/influxdb/data
...

Backup

Backups are always full backups, so we need to:

  • create a snapshot of the current state to a temporary location.
  • when successful, rotate snapshots

We will keep the n most recent snapshots.

Backup is split in two parts:

  • Metastore
  • Databases

For the Metastore:

influxd backup /path/to/backup

Each Database is backed up individually by calling

influxd backup -database <NAME> /path/to/backup

Both can be made while the database is running.

See: https://docs.influxdata.com/influxdb/v1.2/administration/backup_and_restore/

The backup script is scheduled as a cron job running once every day.

Monitoring with Monit

Use the following checks with monit [6]:

  • Check whether the process exists using the pidfile
  • Make sure it is run by the influxdb user
  • Call the HTTP interface on the /ping endpoint
  • Check if there is a recent backup

These are the monit checks:

check process influxd pidfile /var/run/influxd.pid
    group influxdb
    if failed uid influxdb then alert
    if failed
        port 8086
        protocol http
        request /ping
        status = 204
    then alert

check directory influxdbbackup path /mnt/storage/influxdb/backups/snapshot.0
    group backup
    if does not exist then alert
    if timestamp > 25 hours then alert
[1] https://www.influxdata.com/time-series-platform/influxdb/
[2] http://tinycorelinux.net/
[3] http://wiki.tinycorelinux.net/wiki:creating_extensions
[4] https://portal.influxdata.com/downloads#influxdb
[5] https://docs.influxdata.com/influxdb/v1.3/concepts/storage_engine/
[6] https://mmonit.com/monit/documentation/monit.html