mt-aggs-explain

Usage:

	mt-aggs-explain [flags] [config-file]
           (config file defaults to /etc/metrictank/storage-aggregation.conf)

Flags:
  -metric string
    	specify a metric name to see which aggregation rule it matches
  -version
    	print version string

mt-explain
Explains the execution plan for a given query / set of targets

Usage:

  mt-explain

Example:

  mt-explain -from -24h -to now -mdp 1000 "movingAverage(sumSeries(foo.bar), '2min')" "alias(averageSeries(foo.*), 'foo-avg')"

mt-index-cat

Retrieves a metrictank index and dumps it in the requested format
In particular, the vegeta outputs are handy to pipe requests for given series into the vegeta http benchmark tool

Usage:

  mt-index-cat [global config flags] <idxtype> [idx config flags] output 

global config flags:

  -addr string
    	graphite/metrictank address (default "http://localhost:6060")
  -from string
    	for vegeta outputs, will generate requests for data starting from now minus... eg '30min', '5h', '14d', etc. or a unix timestamp (default "30min")
  -limit int
    	only show this many metrics.  use 0 to disable
  -max-age string
    	max age (last update diff with now) of metricdefs.  use 0 to disable (default "6h30min")
  -prefix string
    	only show metrics that have this prefix
  -substr string
    	only show metrics that have this substring
  -verbose
    	print stats to stderr

idxtype: only 'cass' supported for now

cass config flags:

  -auth
    	enable cassandra user authentication
  -ca-path string
    	cassandra CA certficate path when using SSL (default "/etc/metrictank/ca.pem")
  -consistency string
    	write consistency (any|one|two|three|quorum|all|local_quorum|each_quorum|local_one (default "one")
  -enabled
    	 (default true)
  -host-verification
    	host (hostname and server cert) verification when using SSL (default true)
  -hosts string
    	comma separated list of cassandra addresses in host:port form (default "localhost:9042")
  -keyspace string
    	Cassandra keyspace to store metricDefinitions in. (default "metrictank")
  -max-stale duration
    	clear series from the index if they have not been seen for this much time.
  -num-conns int
    	number of concurrent connections to cassandra (default 10)
  -password string
    	password for authentication (default "cassandra")
  -protocol-version int
    	cql protocol version to use (default 4)
  -prune-interval duration
    	Interval at which the index should be checked for stale series. (default 3h0m0s)
  -ssl
    	enable SSL connection to cassandra
  -timeout duration
    	cassandra request timeout (default 1s)
  -update-cassandra-index
    	synchronize index changes to cassandra. not all your nodes need to do this. (default true)
  -update-interval duration
    	frequency at which we should update the metricDef lastUpdate field, use 0s for instant updates (default 3h0m0s)
  -username string
    	username for authentication (default "cassandra")
  -write-queue-size int
    	Max number of metricDefs allowed to be unwritten to cassandra (default 100000)

output: either presets like dump|list|vegeta-render|vegeta-render-patterns
output: or custom templates like '{{.Id}} {{.OrgId}} {{.Name}} {{.Metric}} {{.Interval}} {{.Unit}} {{.Mtype}} {{.Tags}} {{.LastUpdate}} {{.Partition}}'


You may also use processing functions in templates:
pattern: transforms a graphite.style.metric.name into a pattern with wildcards inserted
EXAMPLES:
mt-index-cat -from 60min cass -hosts cassandra:9042 list
mt-index-cat -from 60min cass -hosts cassandra:9042 'sumSeries({{.Name | pattern}})'
mt-index-cat -from 60min cass -hosts cassandra:9042 'GET http://localhost:6060/render?target=sumSeries({{.Name | pattern}})&from=-6h\nX-Org-Id: 1\n\n'

mt-index-migrate

Migrate metric index from one cassandra keyspace to another.
This tool can be used for moving data to a different keyspace or cassandra cluster
or for resetting partition information when the number of partitions being used has changed.

Flags:

  -dry-run
    	run in dry-run mode. No changes will be made. (default true)
  -dst-cass-addr string
    	Address of cassandra host to migrate to. (default "localhost")
  -dst-keyspace string
    	Cassandra keyspace in use on destination. (default "raintank")
  -log-level int
    	log level. 0=TRACE|1=DEBUG|2=INFO|3=WARN|4=ERROR|5=CRITICAL|6=FATAL (default 2)
  -num-partitions int
    	number of partitions in cluster (default 1)
  -partition-scheme string
    	method used for partitioning metrics. (byOrg|bySeries) (default "byOrg")
  -src-cass-addr string
    	Address of cassandra host to migrate from. (default "localhost")
  -src-keyspace string
    	Cassandra keyspace in use on source. (default "raintank")

mt-index-migrate-050-to-054

Converts a metrictank index to the new 0.5.4 (and beyond) format, using cassandra instead of elasticsearch
differences:
 * schema 0 to schema 1 - proper metrics2.0 - (for 0.5.1, 576d8fcb47888b8a334e9a125d6aadf8e0e4d4d7)
 * store data in cassandra in the metric_def_ix table, as a messagepack encoded blob (cassandra idx new in 0.5.3 - 26be821bd8bead43db120e96d14d0ee88d6b6880)
 * use lastUpdate field (for 0.5.4, a07007ab8d4a22b122bbc5f9fadb51480e1c5b0c)

Flags:

  -cass-addr string
    	Address of cassandra host. (default "localhost")
  -dry-run
    	run in dry-run mode. No changes will be made. (default true)
  -es-addr string
    	address of elasticsearch host. (default "localhost")
  -index string
    	elasticsearch index that contains current metric index values. (default "metric")
  -keyspace string
    	Cassandra keyspace to use. (default "raintank")

mt-index-migrate-06-to-07

Converts a metrictank v0.6 index to the v0.7 format
differences:
 * cassandra table: metric_def_idx -> metric_idx
 * data is stored as individual fields, not as messagepack encoded blob
 * support for partitioning. the partition field in cassandra will be set based on num-partitions and partition-schema

Flags:

  -cass-addr string
    	Address of cassandra host. (default "localhost")
  -dry-run
    	run in dry-run mode. No changes will be made. (default true)
  -keyspace string
    	Cassandra keyspace to use. (default "raintank")
  -log-level int
    	log level. 0=TRACE|1=DEBUG|2=INFO|3=WARN|4=ERROR|5=CRITICAL|6=FATAL (default 2)
  -num-partitions int
    	number of partitions in cluster (default 1)
  -partition-scheme string
    	method used for partitioning metrics. (byOrg|bySeries) (default "byOrg")

mt-kafka-mdm-sniff

Inspects what's flowing through kafka (in mdm format) and reports it to you

Flags:

  -config string
    	configuration file path (default "/etc/metrictank/metrictank.ini")
  -format string
    	template to render the data with (default "{{.Part}} {{.OrgId}} {{.Id}} {{.Name}} {{.Metric}} {{.Interval}} {{.Value}} {{.Time}} {{.Unit}} {{.Mtype}} {{.Tags}}")
  -prefix string
    	only show metrics that have this prefix
  -substr string
    	only show metrics that have this substring

mt-kafka-mdm-sniff-out-of-order

Inspects what's flowing through kafka (in mdm format) and reports it to you

Flags:

  -config string
    	configuration file path (default "/etc/metrictank/metrictank.ini")
  -format string
    	template to render the data with (default "{{.First.Seen}} {{.First.Time}} | {{.Seen}} {{.Time}} {{.Part}} {{.OrgId}} {{.Id}} {{.Name}} {{.Metric}} {{.Interval}} {{.Value}} {{.Unit}} {{.Mtype}} {{.Tags}}")
  -prefix string
    	only show metrics that have this prefix
  -substr string
    	only show metrics that have this substring

mt-replicator

Replicates a kafka mdm topic on a given cluster to a topic on another

Flags:

  -compression string
    	compression: none|gzip|snappy (default "snappy")
  -dst-brokers string
    	tcp address for kafka cluster to consume from (may be be given multiple times as a comma-separated list) (default "localhost:9092")
  -group string
    	Kafka consumer group (default "mt-replicator")
  -initial-offset int
    	initial offset to consume from. (-2=oldest, -1=newest) (default -2)
  -log-level int
    	log level. 0=TRACE|1=DEBUG|2=INFO|3=WARN|4=ERROR|5=CRITICAL|6=FATAL (default 2)
  -metric-dst-topic string
    	metrics topic name on destination cluster (default "mdm")
  -metric-src-topic string
    	metrics topic name on source cluster (default "mdm")
  -metrics
    	replicate metrics
  -partition-scheme string
    	method used for partitioning metrics. (byOrg|bySeries) (default "bySeries")
  -persist
    	replicate persistMetrics
  -persist-dst-topic string
    	metricPersist topic name on destination cluster (default "metricpersist")
  -persist-src-topic string
    	metricPersist topic name on source cluster (default "metricpersist")
  -src-brokers string
    	tcp address of source kafka cluster (may be be given multiple times as a comma-separated list) (default "localhost:9092")
  -version
    	print version string

mt-replicator-via-tsdb

Replicates a kafka mdm topic on a given cluster to a remote tsdb-gw server

Flags:

  -batch-size int
    	number of metrics to send in each batch. (default 10000)
  -client-id string
    	Kafka consumer group client id (default "mt-replicator")
  -consumer-fetch-default int
    	number of bytes to try and fetch from consumer (default 32768)
  -destination-key string
    	admin-key of destination tsdb-gw server (default "admin-key")
  -destination-url string
    	tsdb-gw address to send metrics to (default "http://localhost/metrics")
  -group string
    	Kafka consumer group (default "mt-replicator")
  -initial-offset int
    	initial offset to consume from. (-2=oldest, -1=newest) (default -2)
  -log-level int
    	log level. 0=TRACE|1=DEBUG|2=INFO|3=WARN|4=ERROR|5=CRITICAL|6=FATAL (default 2)
  -src-brokers string
    	tcp address of source kafka cluster (may be be given multiple times as a comma-separated list) (default "localhost:9092")
  -src-topic string
    	metrics topic name on source cluster (default "mdm")
  -version
    	print version string

mt-schemas-explain

Usage:

	mt-schemas-explain [flags] [config-file]
           (config file defaults to /etc/metrictank/storage-schemas.conf)

Flags:
  -int int
    	specify an interval to apply interval-based matching in addition to metric matching (e.g. to simulate kafka-mdm input)
  -metric string
    	specify a metric name to see which schema it matches
  -version
    	print version string
  -window-factor int
    	size of compaction window relative to TTL (default 20)

mt-split-metrics-by-ttl [flags] ttl [ttl...]

Creates schema of metric tables split by TTLs and
assists in migrating the data to new tables.
Flags:
  -cassandra-addrs string
    	cassandra host (may be given multiple times as comma-separated list) (default "localhost")
  -cassandra-auth
    	enable cassandra authentication
  -cassandra-ca-path string
    	cassandra CA certificate path when using SSL (default "/etc/metrictank/ca.pem")
  -cassandra-consistency string
    	write consistency (any|one|two|three|quorum|all|local_quorum|each_quorum|local_one (default "one")
  -cassandra-host-selection-policy string
    	 (default "tokenaware,hostpool-epsilon-greedy")
  -cassandra-host-verification
    	host (hostname and server cert) verification when using SSL (default true)
  -cassandra-keyspace string
    	cassandra keyspace to use for storing the metric data table (default "metrictank")
  -cassandra-password string
    	password for authentication (default "cassandra")
  -cassandra-read-concurrency int
    	max number of concurrent reads to cassandra. (default 20)
  -cassandra-read-queue-size int
    	max number of outstanding reads before blocking. value doesn't matter much (default 100)
  -cassandra-retries int
    	how many times to retry a query before failing it
  -cassandra-ssl
    	enable SSL connection to cassandra
  -cassandra-timeout int
    	cassandra timeout in milliseconds (default 1000)
  -cassandra-username string
    	username for authentication (default "cassandra")
  -cql-protocol-version int
    	cql protocol version to use (default 4)
  -window-factor int
    	the window factor be used when creating the metric table schema (default 20)

mt-store-cat

Retrieves timeseries data from the cassandra store. Either raw or with minimal processing

Usage:

	mt-store-cat [flags] tables

	mt-store-cat [flags] <table-selector> <metric-selector> <format>
	                     table-selector: '*' or name of a table. e.g. 'metric_128'
	                     metric-selector: '*' or an id (of raw or aggregated series) or prefix:<prefix>
	                     format:
	                            - points
	                            - point-summary
	                            - chunk-summary (shows TTL's, optionally bucketed. See groupTTL flag)

EXAMPLES:
mt-store-cat -cassandra-keyspace metrictank -from='-1min' '*' '1.77c8c77afa22b67ef5b700c2a2b88d5f' points
mt-store-cat -cassandra-keyspace metrictank -from='-1month' '*' 'prefix:fake' point-summary
mt-store-cat -cassandra-keyspace metrictank '*' 'prefix:fake' chunk-summary
mt-store-cat -groupTTL h -cassandra-keyspace metrictank 'metric_512' '1.37cf8e3731ee4c79063c1d55280d1bbe' chunk-summary
Flags:
  -cassandra-addrs string
    	cassandra host (may be given multiple times as comma-separated list) (default "localhost")
  -cassandra-auth
    	enable cassandra authentication
  -cassandra-ca-path string
    	cassandra CA certificate path when using SSL (default "/etc/metrictank/ca.pem")
  -cassandra-consistency string
    	write consistency (any|one|two|three|quorum|all|local_quorum|each_quorum|local_one (default "one")
  -cassandra-host-selection-policy string
    	 (default "tokenaware,hostpool-epsilon-greedy")
  -cassandra-host-verification
    	host (hostname and server cert) verification when using SSL (default true)
  -cassandra-keyspace string
    	cassandra keyspace to use for storing the metric data table (default "raintank")
  -cassandra-password string
    	password for authentication (default "cassandra")
  -cassandra-read-concurrency int
    	max number of concurrent reads to cassandra. (default 20)
  -cassandra-read-queue-size int
    	max number of outstanding reads before blocking. value doesn't matter much (default 100)
  -cassandra-retries int
    	how many times to retry a query before failing it
  -cassandra-ssl
    	enable SSL connection to cassandra
  -cassandra-timeout int
    	cassandra timeout in milliseconds (default 1000)
  -cassandra-username string
    	username for authentication (default "cassandra")
  -config string
    	configuration file path (default "/etc/metrictank/metrictank.ini")
  -cql-protocol-version int
    	cql protocol version to use (default 4)
  -fix int
    	fix data to this interval like metrictank does quantization. only for points and points-summary format
  -from string
    	get data from (inclusive). only for points and points-summary format (default "-24h")
  -groupTTL string
    	group chunks in TTL buckets based on s (second. means unbucketed), m (minute), h (hour) or d (day). only for chunk-summary format (default "d")
  -print-ts
    	print time stamps instead of formatted dates. only for points and poins-summary format
  -to string
    	get data until (exclusive). only for points and points-summary format (default "now")
  -version
    	print version string
  -window-factor int
    	the window factor be used when creating the metric table schema (default 20)
Notes:
 * Using `*` as metric-selector may bring down your cassandra. Especially chunk-summary ignores from/to and queries all data.
   With great power comes great responsibility
 * points that are not in the `from <= ts < to` range, are prefixed with `-`. In range has prefix of '>`
 * When using chunk-summary, if there's data that should have been expired by cassandra, but for some reason didn't, we won't see or report it
 * Doesn't automatically return data for aggregated series. It's up to you to query for id_<rollup>_<span> when appropriate
 * (rollup is one of sum, cnt, lst, max, min and span is a number in seconds)

mt-update-ttl [flags] ttl table-in [table-out]

Adjusts the data in Cassandra to use a new TTL value. The TTL is applied counting from the timestamp of the data
If table-out not specified or same as table-in, will update in place. Otherwise will not touch input table and store results in table-out
In that case, it is up to you to assure table-out exists before running this tool
Not supported yet: for the per-ttl tables as of 0.7, automatically putting data in the right table
Flags:
  -cassandra-addrs string
    	cassandra host (may be given multiple times as comma-separated list) (default "localhost")
  -cassandra-auth
    	enable cassandra authentication
  -cassandra-ca-path string
    	cassandra CA certificate path when using SSL (default "/etc/metrictank/ca.pem")
  -cassandra-concurrency int
    	max number of concurrent reads to cassandra. (default 20)
  -cassandra-consistency string
    	write consistency (any|one|two|three|quorum|all|local_quorum|each_quorum|local_one (default "one")
  -cassandra-host-selection-policy string
    	 (default "tokenaware,hostpool-epsilon-greedy")
  -cassandra-host-verification
    	host (hostname and server cert) verification when using SSL (default true)
  -cassandra-keyspace string
    	cassandra keyspace to use for storing the metric data table (default "metrictank")
  -cassandra-password string
    	password for authentication (default "cassandra")
  -cassandra-retries int
    	how many times to retry a query before failing it
  -cassandra-ssl
    	enable SSL connection to cassandra
  -cassandra-timeout int
    	cassandra timeout in milliseconds (default 1000)
  -cassandra-username string
    	username for authentication (default "cassandra")
  -cql-protocol-version int
    	cql protocol version to use (default 4)
  -verbose
    	show every record being processed

mt-view-boundaries

Shows boundaries of rows in cassandra and of spans of specified size.
to see UTC times, just prefix command with TZ=UTC

  -span string
    	see boundaries for chunks of this span
  -version
    	print version string

Usage of ./mt-whisper-importer-reader:
  -chunkspans string
    	List of chunk spans separated by ':'. The 1st whisper archive gets the 1st span, 2nd the 2nd, etc (default "10min")
  -exit-on-error
    	Exit with a message when there's an error (default true)
  -http-endpoint string
    	The http endpoint to send the data to (default "http://127.0.0.1:8080/chunks")
  -name-prefix string
    	Prefix to prepend before every metric name, should include the '.' if necessary
  -orgid int
    	Organization ID the data belongs to  (default 1)
  -read-archives string
    	Comma separated list of positive integers or '*' for all archives (default "*")
  -threads int
    	Number of workers threads to process .wsp files (default 10)
  -verbose
    	Write logs to terminal
  -whisper-directory string
    	The directory that contains the whisper file structure (default "/opt/graphite/storage/whisper")
  -write-unfinished-chunks
    	Defines if chunks that have not completed their chunk span should be written

Usage of ./mt-whisper-importer-writer:
  -cassandra-addrs string
    	cassandra host (may be given multiple times as comma-separated list) (default "localhost")
  -cassandra-keyspace string
    	cassandra keyspace to use for storing the metric data table (default "metrictank")
  -exit-on-error
    	Exit with a message when there's an error (default true)
  -fake-avg-aggregates
    	Generate sum/cnt series out of avg series to accommodate metrictank (default true)
  -http-endpoint string
    	The http endpoint to listen on (default "127.0.0.1:8080")
  -num-partitions int
    	Number of Partitions (default 1)
  -partition-scheme string
    	method used for partitioning metrics. This should match the settings of tsdb-gw. (byOrg|bySeries) (default "bySeries")
  -ttls string
    	list of ttl strings used by MT separated by ',' (default "35d")
  -uri-path string
    	the URI on which we expect chunks to get posted (default "/chunks")
  -verbose
    	Write logs to terminal
  -window-factor int
    	the window factor be used when creating the metric table schema (default 20)