{"name":"promql","url":"https://skills.sh/grafana/skills/promql","install":"npx skills add grafana/skills --skill promql","sdk":"grafana","key":"grafana/promql","description":"Write, validate, and optimize PromQL for Prometheus / Grafana Mimir / Grafana Cloud Metrics. Covers `rate` vs `irate` vs `increase`, label matchers and regex, `sum / avg / topk / by / without` aggregation, classic + native `histogram_quantile`, ratios with divide-by-zero guards, `absent` / `changes` for staleness, time offsets and `predict_linear`, recording-rule naming, SLO + burn-rate math, and a cardinality-hunting playbook. Use when writing a metric query, fixing wrong p95s, building an error-budget alert, debugging \"query is slow\", finding the noisy label that blew up cardinality, or migrating a dashboard query to a recording rule — even when the user says \"calculate the error rate\", \"p99 latency\", \"sum by service\", \"why is this query slow\", or \"what's filling Mimir\" without naming PromQL.","hasContent":true,"content":"---\nname: promql\nlicense: Apache-2.0\ndescription: Write, validate, and optimize PromQL for Prometheus / Grafana Mimir / Grafana Cloud Metrics. Covers `rate` vs `irate` vs `increase`, label matchers and regex, `sum / avg / topk / by / without` aggregation, classic + native `histogram_quantile`, ratios with divide-by-zero guards, `absent` / `changes` for staleness, time offsets and `predict_linear`, recording-rule naming, SLO + burn-rate math, and a cardinality-hunting playbook. Use when writing a metric query, fixing wrong p95s, building an error-budget alert, debugging \"query is slow\", finding the noisy label that blew up cardinality, or migrating a dashboard query to a recording rule — even when the user says \"calculate the error rate\", \"p99 latency\", \"sum by service\", \"why is this query slow\", or \"what's filling Mimir\" without naming PromQL.\n---\n\n# PromQL Query Patterns\n\n> **Docs**: https://prometheus.io/docs/prometheus/latest/querying/basics/\n\nPromQL returns either an **instant vector**, a **range vector**, or a **scalar**.\n\n**Golden rule:** `rate()` / `increase()` require a range vector ≥ 4× the scrape interval. 60s scrape → use `[5m]` minimum.\n\n## Prerequisites\n\n- A Prometheus / Mimir / Grafana Cloud endpoint to query (`/api/v1/query` or via Grafana Explore)\n- The PromQL pattern library in [`references/patterns.md`](references/patterns.md)\n\n## Common Workflows\n\n### 1. Write + validate a query\n\n```bash\n# 0. Point at your Prometheus/Mimir. For Grafana Cloud, use the metrics endpoint\n#    and add basic auth (-u \"<metrics_user>:<token>\") to each curl below.\nPROM=http://localhost:9090   # or https://prometheus-prod-XX.grafana.net/api/prom\n\n# 1. Sketch the query — for \"5xx error rate per service\":\nEXPR='sum(rate(http_requests_total{status_code=~\"5..\"}[5m])) by (service)'\n\n# 2. Validate syntax + that the metric/labels exist\ncurl -sG --data-urlencode \"query=${EXPR}\" \\\n  \"$PROM/api/v1/query\" | jq '.status, (.data.result|length)'\n# Expect: \"success\" and result count > 0. If 0 — check label spelling and scrape activity:\ncurl -sG --data-urlencode \"match[]=http_requests_total\" \"$PROM/api/v1/series\" | jq '.data | length'\n\n# 3. Sanity-check the magnitude — open Grafana Explore, paste the expr,\n#    confirm the values look right against a known ground truth (k6 run, log count, etc.)\n```\n\n### 2. Common patterns to copy\n\n**Per-status request rate** (aggregate AFTER rate):\n\n```promql\nsum(rate(http_requests_total{job=\"api\"}[5m])) by (status_code)\n```\n\n**p95 latency** (must keep `le` in the inner aggregation):\n\n```promql\nhistogram_quantile(0.95,\n  sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service))\n```\n\n**Error rate with divide-by-zero guard:**\n\n```promql\nsum(rate(http_requests_total{status_code=~\"5..\"}[5m]))\n  / (sum(rate(http_requests_total[5m])) > 0)\n```\n\nFull library (recording rules, SLO burn-rate, offsets, cardinality hunt, native histograms): [`references/patterns.md`](references/patterns.md).\n\n### 3. Convert a slow dashboard query into a recording rule\n\n```yaml\n# 1. Pick the slow expression, give it a recording-rule name\ngroups:\n  - name: http_request_rates\n    interval: 1m\n    rules:\n      - record: job:http_request_duration_p95:rate5m\n        expr: |\n          histogram_quantile(0.95,\n            sum(rate(http_request_duration_seconds_bucket[5m])) by (le, job))\n```\n\n```bash\n# 2. After rules load, verify the new metric exists\ncurl -sG --data-urlencode \"query=job:http_request_duration_p95:rate5m\" \\\n  \"$PROM/api/v1/query\" | jq '.data.result | length'   # → > 0\n\n# 3. Verify it matches the original expression for at least one sample window\n# (Both queries should produce the same value at the same timestamp.)\n\n# 4. Replace the dashboard panel expression with the recording-rule metric.\n```\n\n## Common bugs\n\n- `histogram_quantile` returns NaN → forgot `by (le)` in the inner aggregation\n- \"No data\" → check the metric exists (`/api/v1/series`) and the window ≥ 4× scrape interval\n- Wrong rate magnitude → counter was aggregated before `rate()` (always `rate()` first)\n- Query timeout → series count too high; use `topk(...)` + a recording rule + drop high-cardinality labels (see [`references/patterns.md`](references/patterns.md))\n\n## Resources\n\n- [PromQL basics](https://prometheus.io/docs/prometheus/latest/querying/basics/)\n- [Operators](https://prometheus.io/docs/prometheus/latest/querying/operators/)\n- [Functions](https://prometheus.io/docs/prometheus/latest/querying/functions/)\n- [Grafana Mimir](https://grafana.com/docs/mimir/latest/)\n","contentSource":"skills.sh/api/download/grafana/skills/promql","contentFetchedAt":"2026-07-28T23:15:43.980Z"}