PromQL implementation
tsink ships a native PromQL parser and evaluator with no external query layer. The implementation lives entirely insrc/promql/ and is exposed through the
public tsink::promql module.
Architecture
The PromQL pipeline has three stages:| Module | File | Responsibility |
|---|---|---|
| Lexer | src/promql/lexer.rs | Tokenise raw input into a flat Vec<Token> |
| Parser | src/promql/parser.rs | Turn tokens into an Expr AST (Pratt / precedence-climbing) |
| Evaluator | src/promql/eval/ | Walk the AST and resolve values against the storage engine |
| Types | src/promql/types.rs | PromqlValue, Sample, Series, histogram helpers |
| Errors | src/promql/error.rs | PromqlError enum |
Public API
Parsing
Query engine
Engine::new defaults to TimestampPrecision::Nanoseconds.
Both methods parse the expression internally before evaluation.
Value types
PromqlValue mirrors the four PromQL expression result types.
| Variant | Contents | When returned |
|---|---|---|
Scalar(f64, i64) | A single float and its evaluation timestamp | Literals, scalar(), arithmetic on two scalars |
InstantVector(Vec<Sample>) | Zero or more labelled samples at one timestamp | Vector selectors, most functions |
RangeVector(Vec<Series>) | Labelled time series with multiple samples | Matrix selectors, range queries |
String(String, i64) | A string value and its evaluation timestamp | String literals |
Sample
Series
Lexer
The lexer is a single-pass byte scanner. It produces all tokens in one call (Lexer::new(input).tokenize()), returning Vec<Token> or a PromqlError::Parse.
Comments
# starts a line comment; everything until the next newline is discarded.
Identifiers
Identifiers follow the usual[a-zA-Z_][a-zA-Z0-9_]* alphabet. Colons (:)
are also accepted inside identifiers to support recording-rule naming
conventions such as job:http_requests:rate5m.
Keywords (case-insensitive)
by, without, offset, bool, and, or, unless, on, ignoring,
group_left, group_right, atan2, inf, nan
Duration literals
A duration is a sequence of one or more<integer><unit> segments.
| Suffix | Unit |
|---|---|
ms | milliseconds |
s | seconds |
m | minutes |
h | hours |
d | days (24 h) |
w | weeks (7 d) |
y | years (365 d) |
1h30m, 5m30s, 2d12h.
Durations are stored internally as milliseconds (i64).
String literals
Double-quoted strings with\", \\, \n, \r, \t escape sequences.
Number literals
Decimal integers and floats. The special identifiersinf and nan are
recognised as numeric tokens equivalent to f64::INFINITY and f64::NAN.
Parser
The parser implements precedence-climbing (Pratt) parsing for expressions. The entry point isparser::parse(input).
Expression grammar (summary)
Operator precedence
Lower number = binds tighter.| Precedence | Operators |
|---|---|
| 1 (highest) | ^ (right-associative) |
| 2 | *, /, %, atan2 |
| 3 | +, - |
| 4 | ==, !=, <, >, <=, >= |
| 5 | and, unless |
| 6 (lowest) | or |
Label matchers
| Operator | Semantics |
|---|---|
= | Exact equality |
!= | Exact inequality |
=~ | Regex match (anchored) |
!~ | Regex non-match (anchored) |
regex crate.
Vector selector
Matrix selector
Subquery
@modifier and offset
Aggregations
Binary operator modifiers
group_left and group_right cannot be combined with set operators
(and, or, unless).
Evaluator
The evaluator is insrc/promql/eval/ and is split across several files:
| File | Contents |
|---|---|
mod.rs | Engine, instant and range query entry points, prefetch cache, @ resolution |
selector.rs | Instant vector and matrix selector evaluation |
functions.rs | All built-in function implementations |
aggregation.rs | Aggregation operator implementations |
binary.rs | Binary operator evaluation and vector matching |
subquery.rs | Subquery evaluation |
time.rs | Duration/timestamp utilities (duration_to_units, step_times) |
Default parameters
| Parameter | Default |
|---|---|
| Lookback delta | 5 minutes |
| Subquery step | 1 minute |
Range query prefetch
For range queries the engine checks whether any selector uses a dynamic@
modifier or is wrapped in a subquery. When no dynamic time is involved it
pre-fetches all required metric data from storage in a single pass before
iterating over steps. This significantly reduces storage I/O for wide time
ranges. Subqueries and @ modifiers disable prefetch for accuracy.
Aggregation operators
All aggregation operators accept an optionalby (labels) or
without (labels) grouping clause.
| Operator | Parameter | Description |
|---|---|---|
sum | — | Sum of values |
avg | — | Average of values |
min | — | Minimum value |
max | — | Maximum value |
count | — | Number of series |
group | — | 1 for each group (existence aggregation) |
stddev | — | Population standard deviation |
stdvar | — | Population variance |
count_values | label (string) | Count series per distinct value; adds a label dimension |
quantile | φ (scalar) | φ-quantile across the group |
topk | k (scalar) | Top k series by value |
bottomk | k (scalar) | Bottom k series by value |
limitk | k (scalar) | Deterministically select k series (hash-stable) |
limit_ratio | ratio (scalar) | Deterministically select a ratio of series |
sum, avg, min, max, count, group, stddev, and stdvar support
native histograms for sum. count_values, quantile, topk, and
bottomk require float samples.
Functions
Counter and gauge range functions
| Function | Input | Description |
|---|---|---|
rate(v[d]) | range vector | Per-second rate of counter increase (extrapolated to fit d) |
irate(v[d]) | range vector | Per-second instant rate using the last two samples |
increase(v[d]) | range vector | Total counter increase over d (extrapolated) |
delta(v[d]) | range vector | Value change over d (extrapolated, for gauges) |
idelta(v[d]) | range vector | Instant delta between the last two samples |
changes(v[d]) | range vector | Number of value changes within d |
resets(v[d]) | range vector | Number of counter resets within d |
rate and increase support native histogram series and produce a histogram
result. The other range functions require float samples.
Extrapolation: rate, increase, and delta use the same boundary
extrapolation algorithm as Prometheus — the sampled interval is extended toward
the range boundaries when the gap is within 110% of the average sample interval.
Over-time aggregations
All take a range vector and return an instant vector.| Function | Description |
|---|---|
avg_over_time(v[d]) | Average of samples in window |
sum_over_time(v[d]) | Sum |
min_over_time(v[d]) | Minimum |
max_over_time(v[d]) | Maximum |
count_over_time(v[d]) | Count of samples |
last_over_time(v[d]) | Most recent sample |
present_over_time(v[d]) | 1 if any sample exists |
stddev_over_time(v[d]) | Standard deviation |
stdvar_over_time(v[d]) | Variance |
mad_over_time(v[d]) | Median absolute deviation |
quantile_over_time(φ, v[d]) | φ-quantile of samples |
Histogram functions
| Function | Description |
|---|---|
histogram_quantile(φ, v) | φ-quantile from classic (bucket-based) or native histograms |
histogram_avg(v) | Average from native histograms |
histogram_count(v) | Observation count from native histograms |
histogram_sum(v) | Sum of observations from native histograms |
histogram_stddev(v) | Standard deviation from native histograms |
histogram_stdvar(v) | Variance from native histograms |
histogram_fraction(lower, upper, v) | Fraction of observations in (lower, upper] from native histograms |
Regression and prediction
| Function | Description |
|---|---|
deriv(v[d]) | Estimated per-second derivative by linear regression |
predict_linear(v[d], t) | Predicted value t seconds from now using linear regression |
double_exponential_smoothing(v[d], sf, tf) | Double exponential smoothing; sf = smoothing factor, tf = trend factor; also callable as holt_winters |
Math functions
| Function | Description |
|---|---|
abs(v) | Absolute value |
ceil(v) | Ceiling |
floor(v) | Floor |
round(v) | Round to nearest integer |
round(v, to_nearest) | Round to nearest multiple of to_nearest |
sqrt(v) | Square root |
exp(v) | e^v |
ln(v) | Natural logarithm |
log2(v) | Base-2 logarithm |
log10(v) | Base-10 logarithm |
sgn(v) | Sign (−1, 0, or 1) |
clamp(v, min, max) | Clamp value to [min, max] |
clamp_min(v, min) | Lower-clamp |
clamp_max(v, max) | Upper-clamp |
Trigonometry
| Function | Function |
|---|---|
cos(v) | acos(v) |
cosh(v) | acosh(v) |
sin(v) | asin(v) |
sinh(v) | asinh(v) |
tan(v) | atan(v) |
tanh(v) | atanh(v) |
deg(v) — radians to degrees | rad(v) — degrees to radians |
pi() — π as a scalar |
Date and time
When called with no argument these functions use the eval-time timestamp. When called with an instant vector they use each sample’s timestamp.| Function | Description |
|---|---|
time() | Current evaluation time in seconds since epoch (scalar) |
timestamp(v) | Timestamp of each sample in seconds since epoch |
minute(v?) | Minute of the hour (0–59) |
hour(v?) | Hour of the day (0–23) |
day_of_week(v?) | Day of the week (0=Sunday–6=Saturday) |
day_of_month(v?) | Day of the month (1–31) |
day_of_year(v?) | Day of the year (1–366) |
days_in_month(v?) | Number of days in the month (28–31) |
month(v?) | Month (1–12) |
year(v?) | Year |
Label manipulation
| Function | Description |
|---|---|
label_replace(v, dst, repl, src, regex) | Rewrite label src into dst using a capture-aware regex and repl |
label_join(v, dst, sep, src1, src2, ...) | Concatenate source labels into dst with sep as separator |
drop_common_labels(v) | Remove labels that are identical across all series in the vector |
Type coercion
| Function | Description |
|---|---|
scalar(v) | Convert a single-element instant vector to a scalar; NaN if more than one element |
vector(s) | Convert a scalar to a single-element instant vector with no labels |
Sorting
| Function | Description |
|---|---|
sort(v) | Sort by value ascending |
sort_desc(v) | Sort by value descending |
sort_by_label(v, l1, ...) | Sort by the specified label names, ascending |
sort_by_label_desc(v, l1, ...) | Sort by the specified label names, descending |
Absence detection
| Function | Description |
|---|---|
absent(v) | Returns {} 1 when the instant vector is empty; nothing otherwise |
absent_over_time(v[d]) | Returns {} 1 when the range vector is empty; nothing otherwise |
Miscellaneous
| Function | Description |
|---|---|
info(v) | Experimental: fetches info-metric labels and merges them into each series |
count_scalar(v) | Returns the element count of a vector as a scalar |
Supported features vs. standard PromQL
| Feature | Supported |
|---|---|
| Instant and range queries | Yes |
| All arithmetic and set operators | Yes |
bool comparison modifier | Yes |
Vector matching (on / ignoring / group_left / group_right) | Yes |
offset modifier | Yes |
@ modifier with literal timestamp, start(), end() | Yes |
Subqueries expr[range:step] | Yes |
| Native histograms | Yes (float samples only for most functions; rate and increase supported) |
| Stale NaN markers (Prometheus compatibility) | Yes |
limitk / limit_ratio aggregations (VictoriaMetrics extension) | Yes |
mad_over_time | Yes |
double_exponential_smoothing / holt_winters | Yes |
sort_by_label / sort_by_label_desc | Yes |
info | Yes (experimental) |
| UTF-8 / non-ASCII metric names | No — identifiers are ASCII only |
| Backtick string literals | No |
Error types
PromqlError::Storage is constructed automatically from TsinkError via a
From impl, so storage errors surface transparently through the query result.