Introduction to InfluxDB and TICK Stack

Introduction to
InfluxDB and TICK Stack
Ahmed AbouZaid
@aabouzaid

Ahmed AbouZaid
DevOps @ CrossEngage
Author and Free/Open source geek
who loves the community.
Automation, data, and metrics are my preferred
areas. I have a built-in monitoring chip, and
too lazy to do anything manually :D
Blog | Github | Twitter
Nope, I’m not Mexican, but Egyptian!
So I could actually be anything!

SaaS CDP (Customer Data Platform) that easily combines all customer
data sources and manages cross-channel marketing campaigns
with your existing infrastructure.
crossengage.io

Overview
● What is time series data?
● Why TICK Stack?
● Where could it be used?

Time Series Data
“A time series is a series of data points indexed (or listed or graphed) in time
order. Most commonly, a time series is a sequence taken at successive equally
spaced points in time. Thus it is a sequence of discrete-time data”.
Properties of Time Series Data
● Billions of individual data points.
● High read and write throughput.
● Large deletes (data expiration).
● Mostly an insert/append workload, very few updates.

TICK Stack
TICK = Telegraf (Collect), InfluxDB (Store),
Chrongraf (Visualize), and Kapacitor (Process).
● The Open Source Time Series Stack provides services
and functionality to accumulate, analyze, and act on time series data.
● Has a big community and ecosystem.
● Written entirely in Go. It compiles into a single binary with
no external dependencies.
● TOML configuration,Tom's Obvious, Minimal Language.

Telegraf
The plugin-driven agent for collecting & reporting metrics.
● Minimal memory footprint.
● +100 plugs with a wide number of plugins for many popular services
already exist for well known services and APIs.
● Plugin system allows new inputs and outputs to be easily added.
● Can work with any external scripts.

Telegraf
● Understand Telegraf output:
$ telegraf --config telegraf.conf --input-filter cpu --test
[measurement],[tag1,tag2] [field1,field2,field3] [timestamp]
system,host=tux,env=prod load1=1.25,load5=1.27,load15=1.29 1509997632000000000

InfluxDB
Scalable time series datastore for metrics, events, and real-time analytics.
● High performance datastore written specifically for time series data.
● Simple, high performing write and query HTTP(S) APIs.
● Plugins support for other data sources such as Graphite, and collectd.
● SQL-like query language to easily query aggregated data.
● Tags allow series to be indexed for fast and efficient queries.
● Retention policies efficiently auto-expire stale data.
● Continuous queries automatically compute aggregate data to make
frequent queries more efficient.

InfluxDB
● Query example:
> SELECT "host", "env", "load1" as "load" FROM "cpu" WHERE "host" = 'tux’ LIMIT 1
name: cpu
---------
time host env load
2017-11-01T01:11:00.000000000Z tux prod 1.25

Chronograf
TICK stack web interface for monitoring, visualization, and management.
● Data Visualization.
● Database Management.
● Infrastructure overview.
● Alert Management.

Kapacitor
Framework for processing, monitoring, and alerting on time series data.
● Process both streaming data and batch data.
● Query data from InfluxDB on a schedule, and receive data via the line
protocol and any other method InfluxDB supports.
● Perform any transformation currently possible in InfluxQL.
● Store transformed data back in InfluxDB.
● Support custom user defined functions to detect anomalies.
● Has an easy DSL to define data processing pipelines.
● Integrate with HipChat, OpsGenie, Alerta, Sensu, Slack, and more.

Kapacitor
● TICKscript DSL example:
stream
|from()
.measurement('app')
|eval(lambda: "errors" / "total")
.as('error_percent')
// Write the transformed data to InfluxDB.
|influxDBOut()
.database('app')
.retentionPolicy('15D')
.measurement('errors')
.tag('kapacitor', 'true')
.tag('version', '0.2')

Use cases
● Infrastructure monitoring.
● Work with sensors (i.e. interacting with IoT).
● Anomaly detection.

Resources
● https://docs.influxdata.com/
● https://www.influxdata.com/time-series-database/
● https://docs.influxdata.com/influxdb/v1.3/concepts/storage_engine/#pro
perties-of-time-series-data
● http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc4.htm

Introduction to InfluxDB and TICK Stack

More Related Content

What's hot

Similar to Introduction to InfluxDB and TICK Stack

More from Ahmed AbouZaid

Recently uploaded

Introduction to InfluxDB and TICK Stack