[processor/cumulativetodelta] Deciding whether to keep the first point based on linear point speculation

[h0cheung]

Component(s)

processor/cumulativetodelta

Is your feature request related to a problem? Please describe.

When there is no accurate time information for the first point, the first two points can be used to determine whether to keep the first point or not, based on the change in value to infer if the first point is growing from 0.

Describe the solution you'd like

VictoriaMetrics' implementation of the increase function may be good for us: https://github.com/VictoriaMetrics/VictoriaMetrics/blob/3d01bc3fbe3d8f4ff4909408fe45ffc1c1acdcf7/app/vmselect/promql/rollup.go#L1828

It does this by taking the first two points (denoted $v0$ and $v1$), and considering that there is a 0 before the first one if and only if $v0 < 10 * (v1 - v0 + 1)$.a 0. (As we require the input to be monotonic, the Abs could be ignored`)

We can use the same rules to decide whether to output the first point directly as a delta metric point.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/cumulativetodelta: @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

@h0cheung this feels possible, but I think it would be good as an option, not a required behavior. Can you propose what the user config would look like?

[h0cheung]

@h0cheung this feels possible, but I think it would be good as an option, not a required behavior. Can you propose what the user config would look like?

Does it feels good to add a config like this?

// KeepLinearInitialValue determines whether to keep the first datapoint when it seems to be a linear increase.
// This only works when InitialValue is set to "auto" and should be dropped according to the starttime based strategy.
// Exactly, let's define the first value as v1 and the second value as v2, then if v1 < (v2 - v1 + 1) * 10, v1 will be kept.
KeepLinearInitialValue bool `mapstructure:"keep_linear_initial_value"`

[TylerHelmuth]

Sure that seems good.

[h0cheung]

When agent restarts, sum metrics with value between 1~9 will be directly reported as delta value.

For error count metrics, this is terrible, which causes alerts.

[TylerHelmuth]

@h0cheung are you saying that is the behavior now or if we implemented this feature?

[h0cheung]

@h0cheung are you saying that is the behavior now or if we implemented this feature?

If we implement this.