-
Notifications
You must be signed in to change notification settings - Fork 1.5k
docs/oss vs enterprise downsampling #9907
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
docs/oss vs enterprise downsampling #9907
Conversation
Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very important information! Let's clarify a few things.
| via [retention filters](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#retention-filters). | ||
| [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/victoriametrics/enterprise/) supports multiple retention periods natively on both the [cluster](https://docs.victoriametrics.com/victoriametrics/cluster-victoriametrics/#retention-filters) and the [single node](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#multiple-retentions) versions. | ||
| You can filter which metrics a retention filter applies to. Below you can see 3 retention filters. The first one matches any metrics with the `juniors` tag and will keep those for 3 days. The second filter says anything with `dev` or `staging` should be kept for 30 days. And finally, the last filter is the default filter of 1 year. | ||
| ```bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
label
or directly team="juniors"
| This approach requires storing the index for Victoriametrics once per retention period instead of having 1 index for all metrics, which means that this approach requires more disk space than enterprise retention filters. | ||
| The index can be quite large on systems where they have time series that change frequently. In some cases, the index size can be larger than the space you're saving with separate retention periods. See [What is high churn rate](https://docs.victoriametrics.com/victoriametrics/faq/#what-is-high-churn-rate) | ||
|
|
||
| Configuration complexity is also a concern; each retention period would have its own storage nodes and unique configurations. Adding a new retention policy requires a multi-step process. First stop writes going to the cluster, create a new cluster, backup and restore data from an other cluster, change the configuration of all storage nodes to the new retention policy, restart the cluster, than re-add the cluster to the write path. While the cluster is down reads will not contain complete results. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
from other cluster
No description provided.