diff --git a/redisdb/tests/README.md b/redisdb/tests/README.md new file mode 100644 index 0000000000000..9500bcc029a31 --- /dev/null +++ b/redisdb/tests/README.md @@ -0,0 +1,118 @@ +# redisdb test fixtures + +This folder holds the pytest suite plus the Compose environments used by the +tests and by the evalya fixtures published from `evalya.yaml`. + +## Compose environments + +| File | What it is | +|---|---| +| `compose/standalone.compose` | Single authenticated Redis instance. | +| `compose/1m-2s.compose` | Master with two slaves (one deliberately unhealthy). | +| `compose/1m-2s-cloud.compose` | Same topology, master loads a mounted `config/redis.conf`. | +| `compose/full-coverage.compose` | Full metric-coverage environment (see below). | + +`evalya.yaml` publishes two of these as reusable fixtures: `redis-standalone` +(the standalone Compose file) and `redis-full` (the full-coverage environment). + +## The `redis-full` fixture and the INFO-rewrite proxy + +`redis-full` exists to make a single fixture emit the **entire** `INFO` surface +that the redisdb check and the OpenTelemetry redisreceiver can parse, including +fields a plain OSS Redis instance can never produce. + +It runs four services: + +- **redis-master**: a real Redis server (AOF on, small `maxmemory` under LRU, + full slowlog). No config file is mounted, so `CONFIG`/`CLIENT`/`CLUSTER` stay + un-renamed and the check's side-channel collections work. +- **redis-replica**: `replicaof` the master, so `INFO replication` reports a + connected slave and the check's with-replica path is exercised. It is not + proxied; the check sees its state through the master's INFO. +- **seed**: a one-shot job that populates typed keys, TTLs, keyspace + hits/misses, an eviction-inducing overfill, benchmark traffic, and forced key + expiry, so the naturally derived counters are non-zero before the first scrape. +- **info-proxy**: the fixture entrypoint, a transparent RESP proxy fronting the + master on 6379. + +### How it sits in front of Redis + +The proxy is the entrypoint; the real Redis runs behind it and is the source of +truth for almost everything. A scraper connects to the proxy on 6379 exactly as +it would to a standalone, and each request flows through: + +``` +scraper (redisdb check / OTel receiver) + │ RESP request + ▼ +info-proxy ──(forwards byte-for-byte)──► redis-master + │ ◄──────────── real reply ────────────┘ + │ + ├─ INFO all → real reply + appended inject.conf lines (length re-framed) + ├─ CLUSTER INFO → canned reply from cluster_info.txt (master not contacted) + └─ everything else → returned unchanged +``` + +This is a transparent forward, not a fallback: the master is always the primary +path, and the proxy only augments one reply and short-circuits one command. + +### What the proxy does + +The proxy speaks Redis (RESP) only. It has no knowledge of Datadog metrics or +metric formats; it fakes raw Redis server output, and the downstream check maps +that to `redis.*` metrics. Because both scrapers read the same `INFO`, one +injection point feeds both. + +Its default path is forward-everything to the master, with two exceptions: + +- `INFO all` (and bare `INFO` / `INFO everything`): the real reply is forwarded, + then the `key:value` lines from `proxy/inject.conf` are appended and the RESP + length is re-framed. These are the fields OSS Redis cannot emit on its own: + managed-service (Azure Cache), cluster, sentinel, and RediSearch fields, each + matching a key the check parses in `constants.py`. +- `CLUSTER INFO`: answered from `proxy/cluster_info.txt` without contacting the + master, since a standalone master would reply that cluster support is disabled. + +Everything else, including section-scoped requests such as `INFO commandstats`, +is forwarded and returned byte-for-byte. The proxy dials one backend connection +per client to preserve per-connection AUTH and the negotiated RESP protocol +version, and it handles both RESP2 (the redisdb check) and RESP3 (the OTel +receiver). + +If you drop the proxy and point the check straight at redis-master, everything +still works; you only lose the faked managed-service/cluster/sentinel/RediSearch +fields. + +### Port and access + +The proxy listens on 6379, the standard Redis port, and presents exactly like a +standalone. The Compose file publishes no host port on purpose (to avoid +clashing with a local Redis or a concurrent run); consumers reach it over the +Compose network. To inspect the fixture by hand, add `--publish 6379:6379` or a +Compose override. + +The proxy source lives in `proxy/`; see `proxy/main.go` for the request loop and +`proxy/resp.go` for the RESP framing. + +### Driving full coverage: the `activity-gen` task + +The `seed` service primes the fixture once, but most rate and counter metrics +report zero on an idle instance, and an assertion that compares 0 to 0 proves +nothing. The `activity-gen` task (`activity-gen.sh`) is an opt-in workload +generator that loops over keyspace hits/misses, command variety, eviction, +expiry, net bytes, blocked clients, forks/RDB, slowlog, and client-side caching, +so both scrapers see the full metric surface with non-zero values across scrape +intervals. + +It is deliberately separate from `redis-full` so the fixture stays quiescent by +default. Add it alongside the fixture when you want sustained activity: + +``` +with: + - redis-full + - activity-gen +``` + +It reads only `DB_HOST`/`DB_PORT`/`REDIS_PASSWORD` (plus optional `ACTIVITY_DB` +and `ACTIVITY_DURATION`), so it has no dependency on any scraper and is reusable +by any consumer of the fixture. diff --git a/redisdb/tests/activity-gen.sh b/redisdb/tests/activity-gen.sh new file mode 100644 index 0000000000000..1e0052640a55a --- /dev/null +++ b/redisdb/tests/activity-gen.sh @@ -0,0 +1,128 @@ +#!/bin/sh +# Drives the Redis fixture through states that make the redisdb check and the +# OTel redis receiver report their full metric surface with non-zero values. +# +# Without this, an idle Redis reports zero for most counters, and any assertion +# that compares 0 to 0 passes without proving anything. Each block below targets +# metrics that stay absent or flat otherwise. +# +# Runs setup once, then loops the workload so both collectors see activity +# across several scrape intervals. Consumes DB_HOST/DB_PORT/REDIS_PASSWORD from +# the redis-full fixture; connects through the info-proxy exactly like any other +# client (CONFIG/CLUSTER pass through to the master). +set -eu + +R="redis-cli --no-auth-warning -h ${DB_HOST} -p ${DB_PORT:-6379} -a ${REDIS_PASSWORD}" +DB="${ACTIVITY_DB:-14}" +DURATION="${ACTIVITY_DURATION:-0}" # 0 = run forever + +log() { echo "activity-gen: $*"; } + +# --- setup ---------------------------------------------------------------- +# Eviction needs a maxmemory ceiling and a policy that actually evicts. Matches +# the full-coverage.compose master (--maxmemory 16mb allkeys-lru); re-set here so +# the script also works when pointed at a plain Redis. Also populates +# redis.mem.maxmemory, which is 0 (unlimited) by default. +log "configuring eviction ceiling and slowlog threshold" +$R config set maxmemory 16mb > /dev/null +$R config set maxmemory-policy allkeys-lru > /dev/null +# 5ms so ordinary commands are not logged but DEBUG SLEEP below is. +$R config set slowlog-log-slower-than 5000 > /dev/null + +# Lua interpreter memory stays at its floor until a script is evaluated. +log "seeding lua interpreter" +$R eval "return redis.status_reply('OK')" 0 > /dev/null + +# tracking_total_keys counts keys in the server's client-side-caching +# invalidation table, which only exists while a RESP3 client holds +# CLIENT TRACKING ON. A one-shot redis-cli closes its connection and the table +# drops back to 0, so hold one session open for the life of this task and keep +# reading through it. Without this the metric reports a constant 0 and its +# assertion compares 0 to 0. +log "opening tracked RESP3 session for client-side caching" +{ + echo "client tracking on" + while :; do + echo "get track:key" + sleep 2 + done +} | $R -3 -n "$DB" > /dev/null 2>&1 & +$R -n "$DB" set "track:key" seed > /dev/null + +end=0 +[ "$DURATION" -gt 0 ] && end=$(( $(date +%s) + DURATION )) +i=0 + +while :; do + i=$(( i + 1 )) + + # --- keyspace: hits, misses, keys, expires, avg_ttl -------------------- + # Spread across two DBs so per-db metrics report more than one series. + for db in "$DB" $(( DB - 1 )); do + $R -n "$db" mset "hit:$i" "v$i" "hit2:$i" "v$i" > /dev/null + $R -n "$db" get "hit:$i" > /dev/null # hit + $R -n "$db" get "absent:$i" > /dev/null # miss + $R -n "$db" set "ttl:$i" "v" ex 2 > /dev/null # expires + avg_ttl, then expired + done + + # --- command variety: redis.cmd.calls / redis.cmd.usec ---------------- + # command_stats is per-command, so exercising several command families + # widens the cmd dimension instead of deepening a single series. + $R -n "$DB" lpush "list:$i" a b c > /dev/null + $R -n "$DB" lrange "list:$i" 0 -1 > /dev/null + $R -n "$DB" sadd "set:$i" x y z > /dev/null + $R -n "$DB" smembers "set:$i" > /dev/null + $R -n "$DB" zadd "zset:$i" 1 a 2 b > /dev/null + $R -n "$DB" hset "hash:$i" f v > /dev/null + $R -n "$DB" xadd "stream:$i" '*' f v > /dev/null + $R -n "$DB" incr "counter" > /dev/null + # CLIENT SETINFO advertises a client lib-name, surfacing it in CLIENT INFO/LIST + # and adding a client|setinfo entry to command_stats. Redis 7.2+ only. + $R -n "$DB" client setinfo lib-name activity-gen > /dev/null 2>&1 || true + + # --- net.input / net.output and client buffer high-water marks -------- + # A large write then a large read moves total_net_input_bytes and + # total_net_output_bytes, and raises client_recent_max_{input,output}_buffer. + $R -n "$DB" set "big:$i" "$(head -c 32768 /dev/zero | tr '\0' 'x')" > /dev/null + $R -n "$DB" get "big:$i" > /dev/null + + # --- eviction: redis.keys.evicted ------------------------------------- + # Write past the 16mb ceiling in bursts so allkeys-lru has to evict. + $R -n "$DB" eval \ + "for j=1,200 do redis.call('SET', KEYS[1]..j, string.rep('x', 4096)) end return 1" \ + 1 "fill:$i:" > /dev/null + + # --- blocked clients: redis.clients.blocked --------------------------- + # BLPOP on an empty key parks a client; it is counted while blocked. + $R -n "$DB" blpop "never:$i" 3 > /dev/null 2>&1 & + + # --- fork + rdb: redis.latest_fork, redis.rdb.changes_since_last_save - + # latest_fork_usec stays 0 until the first background save. + if [ $(( i % 5 )) -eq 1 ]; then + log "iteration $i: triggering BGSAVE" + $R bgsave > /dev/null 2>&1 || true + fi + + # --- slowlog ---------------------------------------------------------- + # DEBUG SLEEP exceeds the 5ms threshold, so the check reports slowlog rates. + if [ $(( i % 5 )) -eq 2 ]; then + $R debug sleep 0.05 > /dev/null 2>&1 || true + fi + + # --- connection churn: redis.net.total_connections_received ----------- + # Each redis-cli invocation opens a fresh connection, but an explicit burst + # keeps the counter climbing between scrapes. + for _ in 1 2 3; do $R ping > /dev/null; done + + if [ $(( i % 10 )) -eq 0 ]; then + stats=$($R info stats) + log "iteration $i: keys=$($R -n "$DB" dbsize)" \ + "evicted=$(echo "$stats" | sed -n 's/^evicted_keys:\([0-9]*\).*/\1/p')" \ + "expired=$(echo "$stats" | sed -n 's/^expired_keys:\([0-9]*\).*/\1/p')" + fi + + [ "$end" -gt 0 ] && [ "$(date +%s)" -ge "$end" ] && break + sleep 1 +done + +log "PASS: activity workload complete after $i iterations" diff --git a/redisdb/tests/compose/full-coverage.compose b/redisdb/tests/compose/full-coverage.compose new file mode 100644 index 0000000000000..f00ae8cde04b9 --- /dev/null +++ b/redisdb/tests/compose/full-coverage.compose @@ -0,0 +1,138 @@ +# Full metric-coverage environment for the redisdb check and the OpenTelemetry +# redisreceiver. +# +# A real master/replica pair produces every INFO field that OSS Redis can emit +# (memory, clients, cpu, stats, persistence, keyspace, replication, AOF, +# eviction, expiry, commandstats, slowlog). The `info-proxy` sits in front of +# the master and augments the `INFO all` reply with the fields OSS Redis never +# emits on its own (managed-service, cluster, sentinel, and RediSearch fields), +# and serves a canned `CLUSTER INFO`. Both scrapers read those fields the same +# way, so a single injection point covers both. See ../proxy for details. +services: + redis-master: + image: "redis:${REDIS_VERSION:-7.4}" + networks: + - network1 + # No config file here on purpose: the managed-service simulation config + # renames CONFIG/CLIENT/CLUSTER, which the check needs for its side-channel + # collections. Enable AOF, a small memory ceiling with LRU eviction (so + # evicted_keys is non-zero after the seed overfills), and full slowlog + # capture so slowlog metrics are populated. + command: > + redis-server + --requirepass devops-best-friend + --appendonly yes + --maxmemory 16mb + --maxmemory-policy allkeys-lru + --slowlog-log-slower-than 0 + --latency-monitor-threshold 1 + healthcheck: + test: ["CMD-SHELL", "redis-cli --no-auth-warning -a devops-best-friend ping"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 5s + + redis-replica: + image: "redis:${REDIS_VERSION:-7.4}" + depends_on: + redis-master: + condition: service_healthy + networks: + - network1 + command: > + redis-server + --requirepass devops-best-friend + --masterauth devops-best-friend + --replicaof redis-master 6379 + healthcheck: + test: ["CMD-SHELL", "redis-cli --no-auth-warning -a devops-best-friend ping"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 5s + + # Populates the master with typed keys, TTL keys, keyspace hits/misses, an + # eviction-inducing overfill, benchmark traffic (commandstats + net.commands), + # and a slow command, so the naturally derived INFO fields are all non-trivial. + seed: + image: "redis:${REDIS_VERSION:-7.4}" + depends_on: + redis-master: + condition: service_healthy + networks: + - network1 + restart: "no" + entrypoint: + - sh + - -c + - | + set -e + R="redis-cli -h redis-master -a devops-best-friend --no-auth-warning" + $$R SET string:1 hello >/dev/null + $$R RPUSH list:1 a b c d e >/dev/null + $$R SADD set:1 x y z >/dev/null + $$R HSET hash:1 f1 v1 f2 v2 f3 v3 >/dev/null + $$R ZADD zset:1 1 a 2 b 3 c >/dev/null + $$R XADD stream:1 '*' field val >/dev/null + i=1; while [ $$i -le 50 ]; do $$R SETEX ttl:$$i 300 v >/dev/null; i=$$((i+1)); done + $$R GET string:1 >/dev/null + $$R GET does:not:exist >/dev/null + echo "seeding traffic (benchmark + overfill for eviction)..." + redis-benchmark -h redis-master -a devops-best-friend -q -n 100000 -r 100000 -d 512 -t set,get,incr,lpush,rpush,sadd,hset,zadd,spop,lpop >/dev/null 2>&1 || true + $$R DEBUG SLEEP 0.05 >/dev/null 2>&1 || true + # Drive expired_keys: the 300s ttl:* keys above are evicted by the LRU + # overfill long before they expire, so create short-TTL keys after the + # overfill, wait past the TTL, then read them to force lazy expiry (the + # active-expiry cycle also counts them). Without this expired_keys stays 0. + i=1; while [ $$i -le 20 ]; do $$R SETEX exp:$$i 1 v >/dev/null; i=$$((i+1)); done + sleep 2 + i=1; while [ $$i -le 20 ]; do $$R GET exp:$$i >/dev/null; i=$$((i+1)); done + echo "evicted_keys=$$($$R INFO stats | grep evicted_keys || true)" + echo "expired_keys=$$($$R INFO stats | grep expired_keys || true)" + echo "seed-done" + + info-proxy: + build: + context: ../proxy + dockerfile: Dockerfile + # info-proxy is the fixture entrypoint (published as redis-full). Gate it on + # the whole environment being ready so a single dependency on the proxy pulls + # up everything a scraper needs: + # - redis-replica healthy, so INFO replication reports a connected slave + # (connected_slaves/slave0) and the with-replica code path is exercised; + # - seed completed, so the naturally derived counters (keyspace hits/misses, + # evicted/expired keys, commands, net bytes, commandstats) are non-zero + # before the first scrape rather than starting at 0. + depends_on: + redis-master: + condition: service_healthy + redis-replica: + condition: service_healthy + seed: + condition: service_completed_successfully + environment: + LISTEN_ADDR: ":6379" + BACKEND_ADDR: "redis-master:6379" + INJECT_FILE: "/etc/info-proxy/inject.conf" + CLUSTER_INFO_FILE: "/etc/info-proxy/cluster_info.txt" + volumes: + - ../proxy/inject.conf:/etc/info-proxy/inject.conf:ro + - ../proxy/cluster_info.txt:/etc/info-proxy/cluster_info.txt:ro + # No host-port publish: the sole consumer is the evalya `redis-full` task, + # whose other services reach the proxy over the managed network via the + # DB_HOST label. Publishing 6379 to the host only invites conflicts with a + # local Redis or a concurrent scenario. A developer inspecting the fixture + # by hand can add `--publish 6379:6379` or a compose override as needed. + networks: + - network1 + healthcheck: + test: ["CMD-SHELL", "redis-cli --no-auth-warning -a devops-best-friend ping"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 5s + +networks: + network1: + name: full-coverage_default diff --git a/redisdb/tests/evalya.yaml b/redisdb/tests/evalya.yaml index a1ef3b296d572..dc18d0f1b86cc 100644 --- a/redisdb/tests/evalya.yaml +++ b/redisdb/tests/evalya.yaml @@ -20,3 +20,39 @@ tasks: timeout: 5s retries: 5 start_period: 10s + + # Full metric-coverage fixture: a real master/replica pair fronted by the + # info-proxy, which augments `INFO all` with the fields OSS Redis cannot emit + # (managed-service, cluster, sentinel, RediSearch) and serves a canned + # `CLUSTER INFO`. Consumers connect to the proxy, which presents on 6379 just + # like a standalone. See ./compose/full-coverage.compose and ./proxy. + - id: redis-full + task: ./compose/full-coverage.compose@info-proxy + labels: + evalya.io/publish: "true" + evalya.io/provides.DB_HOST: "{{ .hostname }}" + evalya.io/provides.DB_PORT: "6379" + evalya.io/provides.REDIS_PASSWORD: devops-best-friend + env: + - name: REDIS_PASSWORD + value: devops-best-friend + healthcheck: + test: ["CMD-SHELL", "redis-cli --no-auth-warning -a \"$$REDIS_PASSWORD\" ping"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 10s + + # Opt-in workload generator for redis-full. An idle Redis reports zero for most + # counters; this drives keyspace, command variety, eviction, expiry, net bytes, + # blocked clients, forks/RDB, slowlog, and client-side caching to non-zero + # values across scrape intervals, so a scraper sees the full metric surface. + # Consumers add it alongside the fixture, e.g. `with: [redis-full, activity-gen]`. + # Not part of redis-full itself, so the fixture stays quiescent by default. + - id: activity-gen + image: "redis:${REDIS_VERSION:-7.4}" + command: ["sh", "/opt/activity-gen.sh"] + volumes: + - ./activity-gen.sh:/opt/activity-gen.sh:ro + with: + - redis-full diff --git a/redisdb/tests/proxy/Dockerfile b/redisdb/tests/proxy/Dockerfile new file mode 100644 index 0000000000000..5b5e9f3598f2b --- /dev/null +++ b/redisdb/tests/proxy/Dockerfile @@ -0,0 +1,12 @@ +# Build the info-proxy binary, then ship it on a redis image so the compose +# healthcheck can use redis-cli against the proxy port. +FROM golang:1.26-alpine AS build +WORKDIR /src +# Stdlib-only module: copying sources is enough, no module download step. +COPY . . +RUN CGO_ENABLED=0 go build -o /usr/local/bin/info-proxy . + +FROM redis:7-alpine +COPY --from=build /usr/local/bin/info-proxy /usr/local/bin/info-proxy +EXPOSE 6379 +ENTRYPOINT ["/usr/local/bin/info-proxy"] diff --git a/redisdb/tests/proxy/cluster_info.txt b/redisdb/tests/proxy/cluster_info.txt new file mode 100644 index 0000000000000..7cff77ca5b5ed --- /dev/null +++ b/redisdb/tests/proxy/cluster_info.txt @@ -0,0 +1,11 @@ +cluster_state:ok +cluster_slots_assigned:16384 +cluster_slots_ok:16384 +cluster_slots_pfail:0 +cluster_slots_fail:0 +cluster_known_nodes:3 +cluster_size:3 +cluster_current_epoch:6 +cluster_my_epoch:2 +cluster_stats_messages_sent:1000 +cluster_stats_messages_received:1000 diff --git a/redisdb/tests/proxy/go.mod b/redisdb/tests/proxy/go.mod new file mode 100644 index 0000000000000..172e3cac0d655 --- /dev/null +++ b/redisdb/tests/proxy/go.mod @@ -0,0 +1,3 @@ +module info-proxy + +go 1.26 diff --git a/redisdb/tests/proxy/inject.conf b/redisdb/tests/proxy/inject.conf new file mode 100644 index 0000000000000..eb8ce4582ced4 --- /dev/null +++ b/redisdb/tests/proxy/inject.conf @@ -0,0 +1,89 @@ +# INFO fields injected by info-proxy into the `INFO all` reply. +# +# These are the redisdb metrics that a plain OSS Redis standalone/replica cannot +# emit no matter how it is loaded or configured, so the environment fakes them +# here to exercise the check's parsing and metric submission. Every key matches +# an entry in redisdb/datadog_checks/redisdb/constants.py (GAUGE_KEYS/RATE_KEYS); +# values are arbitrary small non-zero numbers (the FTF parity check asserts +# co-presence, not value equality). +# +# Naturally generated fields (memory, clients, cpu, stats, persistence, +# keyspace, replication, AOF, eviction, expiry, commandstats, slowlog) are NOT +# listed here; the compose environment produces them for real. + +# --- Managed-service-only (Azure Cache for Redis) ------------------------------ +# Emitted only by Azure's managed Redis, never by OSS redis-server. +bytes_received_per_sec:1024 +bytes_sent_per_sec:2048 + +# --- Cluster ------------------------------------------------------------------- +# Override the standalone's cluster_enabled:0 so the check follows its cluster +# path and calls CLUSTER INFO (served canned by the proxy, see cluster_info.txt). +cluster_enabled:1 +cluster_connections:4 + +# --- Sentinel ------------------------------------------------------------------ +# Present only in the INFO of a redis-server running in --sentinel mode. +sentinel_masters:1 +sentinel_running_scripts:2 +sentinel_scripts_queue_length:3 +sentinel_simulate_failure_flags:4 +sentinel_tilt:5 +sentinel_tilt_since_seconds:1 +sentinel_total_tilt:2 + +# --- RediSearch module (search_*) ---------------------------------------------- +# Present only when the RediSearch/Query-Engine module is loaded (redis-stack). +# Faked here to keep the base image small; these are redisdb-only (the OTel +# receiver has no search metrics). +search_bytes_collected:1 +search_cursors_internal_active:2 +search_cursors_internal_idle:3 +search_cursors_user_active:4 +search_cursors_user_idle:5 +search_errors_for_index_with_max_failures:6 +search_errors_indexing_failures:7 +search_gc_bytes_collected:8 +search_gc_marked_deleted_vectors:9 +search_gc_total_cycles:1 +search_gc_total_docs_not_collected_by_gc:2 +search_gc_total_ms_run:3 +search_global_idle:4 +search_global_total:5 +search_largest_memory_index:6 +search_marked_deleted_vectors:7 +search_number_of_active_indexes:8 +search_number_of_active_indexes_indexing:9 +search_number_of_active_indexes_running_queries:1 +search_number_of_indexes:2 +search_smallest_memory_index:3 +search_total_active_queries:4 +search_total_active_write_threads:5 +search_total_cycles:6 +search_total_docs_not_collected_by_gc:7 +search_total_indexing_time:8 +search_total_ms_run:9 +search_total_queries_processed:1 +search_total_query_commands:2 +search_total_query_execution_time_ms:3 +search_used_memory_indexes:4 +search_used_memory_vector_index:5 +search_fields_geo_Geo:6 +search_fields_geo_NoIndex:7 +search_fields_geo_Sortable:8 +search_fields_geoshape_Geoshape:9 +search_fields_geoshape_NoIndex:1 +search_fields_geoshape_Sortable:2 +search_fields_numeric_NoIndex:3 +search_fields_numeric_Numeric:4 +search_fields_numeric_Sortable:5 +search_fields_tag_CaseSensitive:6 +search_fields_tag_NoIndex:7 +search_fields_tag_Sortable:8 +search_fields_tag_Tag:9 +search_fields_text_NoIndex:1 +search_fields_text_Sortable:2 +search_fields_text_Text:3 +search_fields_vector_Flat:4 +search_fields_vector_HNSW:5 +search_fields_vector_Vector:6 diff --git a/redisdb/tests/proxy/inject.go b/redisdb/tests/proxy/inject.go new file mode 100644 index 0000000000000..8639fcf1092d7 --- /dev/null +++ b/redisdb/tests/proxy/inject.go @@ -0,0 +1,144 @@ +// INFO-reply rewriting: overriding existing fields and appending faked ones. +package main + +import ( + "bufio" + "bytes" + "os" + "strconv" + "strings" +) + +// loadInjectLines reads a file of `key:value` INFO lines. Blank lines and lines +// beginning with '#' are ignored, so the file can be commented and grouped. +func loadInjectLines(path string) ([][]byte, error) { + f, err := os.Open(path) + if err != nil { + return nil, err + } + defer f.Close() + + var lines [][]byte + sc := bufio.NewScanner(f) + sc.Buffer(make([]byte, 1024*1024), 1024*1024) + for sc.Scan() { + ln := bytes.TrimRight(sc.Bytes(), "\r\n") + trimmed := bytes.TrimSpace(ln) + if len(trimmed) == 0 || trimmed[0] == '#' { + continue + } + if bytes.IndexByte(ln, ':') < 0 { + continue // not a key:value line + } + // Copy: Scanner reuses its buffer between iterations. + cp := make([]byte, len(ln)) + copy(cp, ln) + lines = append(lines, cp) + } + return lines, sc.Err() +} + +func lineKey(line []byte) string { + if i := bytes.IndexByte(line, ':'); i >= 0 { + return string(line[:i]) + } + return string(line) +} + +// applyInject overrides any existing INFO field whose key matches an inject +// line and appends the rest under a "# Faked" section. Redis clients parse INFO +// as flat key:value pairs, so the section header is cosmetic; blank lines are +// ignored by both parsers. +func applyInject(payload []byte, inject [][]byte) []byte { + override := make(map[string][]byte, len(inject)) + for _, il := range inject { + override[lineKey(il)] = il + } + + lines := bytes.Split(payload, []byte("\r\n")) + used := make(map[string]bool, len(inject)) + out := make([][]byte, 0, len(lines)+len(inject)+2) + for _, ln := range lines { + if i := bytes.IndexByte(ln, ':'); i >= 0 { + k := string(ln[:i]) + if rep, ok := override[k]; ok { + out = append(out, rep) + used[k] = true + continue + } + } + out = append(out, ln) + } + + appended := false + for _, il := range inject { + if used[lineKey(il)] { + continue + } + if !appended { + out = append(out, []byte("# Faked")) + appended = true + } + out = append(out, il) + } + + // Preserve the trailing CRLF that a real INFO payload ends with. + out = append(out, []byte("")) + return bytes.Join(out, []byte("\r\n")) +} + +// rewriteInfoReply rewrites the payload of a bulk ('$') or verbatim ('=') INFO +// reply and re-frames it with the corrected length. Non-string replies (e.g. an +// error) are returned untouched. +func rewriteInfoReply(reply []byte, inject [][]byte) []byte { + if len(reply) == 0 { + return reply + } + t := reply[0] + if t != '$' && t != '=' { + return reply + } + nl := bytes.IndexByte(reply, '\n') + if nl < 0 { + return reply + } + n := parseLen(reply[1 : nl+1]) + if n < 0 || nl+1+n > len(reply) { + return reply + } + payload := reply[nl+1 : nl+1+n] + + newPayload := applyInject(payload, inject) + + out := make([]byte, 0, len(newPayload)+16) + out = append(out, t) + out = append(out, []byte(strconv.Itoa(len(newPayload)))...) + out = append(out, '\r', '\n') + out = append(out, newPayload...) + out = append(out, '\r', '\n') + return out +} + +// normalizeCRLF rewrites all line endings to CRLF and guarantees a trailing +// CRLF. CLUSTER INFO replies are parsed by splitting on "\r\n" (by both redis-py +// and the receiver), so a canned reply authored with plain "\n" must be +// converted or it parses as a single unusable line. +func normalizeCRLF(s string) string { + s = strings.ReplaceAll(s, "\r\n", "\n") + s = strings.ReplaceAll(s, "\n", "\r\n") + if !strings.HasSuffix(s, "\r\n") { + s += "\r\n" + } + return s +} + +// encodeBulk frames a string as a RESP bulk string reply. +func encodeBulk(s string) []byte { + out := make([]byte, 0, len(s)+16) + out = append(out, '$') + out = append(out, []byte(strconv.Itoa(len(s)))...) + out = append(out, '\r', '\n') + out = append(out, []byte(s)...) + out = append(out, '\r', '\n') + return out +} diff --git a/redisdb/tests/proxy/main.go b/redisdb/tests/proxy/main.go new file mode 100644 index 0000000000000..bcc7246eba694 --- /dev/null +++ b/redisdb/tests/proxy/main.go @@ -0,0 +1,156 @@ +// info-proxy is a transparent Redis (RESP) proxy that forwards every command +// to a backend Redis untouched, except that it augments the reply to `INFO all` +// with additional key:value lines and, optionally, serves a canned `CLUSTER +// INFO` reply. It exists so a test environment can exercise metrics that a +// plain Redis instance cannot emit on its own (managed-service-only fields, +// cluster/sentinel fields, module fields), for any scraper that reads INFO. +// +// Both the Datadog redisdb check and the OpenTelemetry redisreceiver collect +// almost all of their metrics by parsing the INFO reply as flat key:value +// pairs, so a single injection point fakes fields for both. +package main + +import ( + "bufio" + "bytes" + "log" + "net" + "os" + "strings" +) + +type config struct { + listenAddr string + backendAddr string + inject [][]byte + clusterInfoText string +} + +func getenv(key, def string) string { + if v := os.Getenv(key); v != "" { + return v + } + return def +} + +func main() { + cfg := config{ + listenAddr: getenv("LISTEN_ADDR", ":6379"), + backendAddr: getenv("BACKEND_ADDR", "redis-master:6379"), + } + + injectFile := getenv("INJECT_FILE", "/etc/info-proxy/inject.conf") + if lines, err := loadInjectLines(injectFile); err != nil { + log.Printf("info-proxy: no inject file at %s (%v); passing INFO through unchanged", injectFile, err) + } else { + cfg.inject = lines + log.Printf("info-proxy: loaded %d inject line(s) from %s", len(lines), injectFile) + } + + clusterFile := getenv("CLUSTER_INFO_FILE", "/etc/info-proxy/cluster_info.txt") + if b, err := os.ReadFile(clusterFile); err == nil && len(bytes.TrimSpace(b)) > 0 { + cfg.clusterInfoText = normalizeCRLF(string(b)) + log.Printf("info-proxy: will serve canned CLUSTER INFO from %s", clusterFile) + } + + ln, err := net.Listen("tcp", cfg.listenAddr) + if err != nil { + log.Fatalf("info-proxy: listen %s: %v", cfg.listenAddr, err) + } + log.Printf("info-proxy: listening on %s -> backend %s", cfg.listenAddr, cfg.backendAddr) + + for { + conn, err := ln.Accept() + if err != nil { + log.Printf("info-proxy: accept: %v", err) + continue + } + go handle(conn, cfg) + } +} + +// handle proxies a single client connection to a dedicated backend connection. +// Keeping one backend connection per client preserves per-connection state such +// as AUTH and the negotiated RESP protocol version. +// +// It assumes a strict one-reply-per-request exchange, which holds for both +// scrapers. It does not model out-of-band replies (SUBSCRIBE/MONITOR push +// frames); neither the redisdb check nor the OTel receiver uses them. +func handle(client net.Conn, cfg config) { + defer client.Close() + + backend, err := net.Dial("tcp", cfg.backendAddr) + if err != nil { + log.Printf("info-proxy: dial backend %s: %v", cfg.backendAddr, err) + return + } + defer backend.Close() + + cr := bufio.NewReader(client) + br := bufio.NewReader(backend) + + for { + raw, args, err := readRequest(cr) + if len(raw) == 0 || len(args) == 0 { + if err != nil { + return + } + // Nothing actionable (e.g. a stray newline); forward and continue. + if len(raw) > 0 { + if _, werr := backend.Write(raw); werr != nil { + return + } + } + continue + } + + cmd := strings.ToUpper(string(args[0])) + var sub string + if len(args) > 1 { + sub = strings.ToUpper(string(args[1])) + } + + // Serve a canned CLUSTER INFO without touching the backend so a + // standalone instance can present as cluster-enabled. + if cfg.clusterInfoText != "" && cmd == "CLUSTER" && sub == "INFO" { + if _, werr := client.Write(encodeBulk(cfg.clusterInfoText)); werr != nil { + return + } + if err != nil { + return + } + continue + } + + if _, werr := backend.Write(raw); werr != nil { + return + } + + reply, rerr := readReply(br) + if cmd == "INFO" && isFullInfo(args) && len(cfg.inject) > 0 { + reply = rewriteInfoReply(reply, cfg.inject) + } + if _, werr := client.Write(reply); werr != nil { + return + } + + if err != nil || rerr != nil { + return + } + } +} + +// isFullInfo reports whether an INFO request targets the full metric set, which +// is what both scrapers request ("INFO all"). Section-scoped requests such as +// "INFO commandstats" or "INFO keyspace" are left untouched so their narrowly +// parsed replies are not polluted with injected fields. +func isFullInfo(args [][]byte) bool { + if len(args) == 1 { + return true // bare INFO -> default sections + } + switch strings.ToLower(string(args[1])) { + case "all", "everything", "default": + return true + } + return false +} diff --git a/redisdb/tests/proxy/proxy_test.go b/redisdb/tests/proxy/proxy_test.go new file mode 100644 index 0000000000000..d2a3d52bea669 --- /dev/null +++ b/redisdb/tests/proxy/proxy_test.go @@ -0,0 +1,173 @@ +package main + +import ( + "bufio" + "bytes" + "strconv" + "strings" + "testing" +) + +func reader(s string) *bufio.Reader { + return bufio.NewReader(strings.NewReader(s)) +} + +func TestReadReplyBulk(t *testing.T) { + raw, err := readReply(reader("$5\r\nhello\r\n")) + if err != nil { + t.Fatal(err) + } + if string(raw) != "$5\r\nhello\r\n" { + t.Fatalf("got %q", raw) + } +} + +func TestReadReplyNullBulk(t *testing.T) { + raw, _ := readReply(reader("$-1\r\n")) + if string(raw) != "$-1\r\n" { + t.Fatalf("got %q", raw) + } +} + +func TestReadReplyArray(t *testing.T) { + in := "*2\r\n$3\r\nfoo\r\n:42\r\n" + raw, err := readReply(reader(in)) + if err != nil { + t.Fatal(err) + } + if string(raw) != in { + t.Fatalf("got %q", raw) + } +} + +func TestReadReplyRESP3Map(t *testing.T) { + // %1 { "proto" => 3 } as sent by HELLO-style replies + in := "%1\r\n$5\r\nproto\r\n:3\r\n" + raw, err := readReply(reader(in)) + if err != nil { + t.Fatal(err) + } + if string(raw) != in { + t.Fatalf("got %q", raw) + } +} + +func TestReadReplyRESP3Attribute(t *testing.T) { + // |1 attribute prefix followed by the real reply (+OK) + in := "|1\r\n$3\r\nkey\r\n$3\r\nval\r\n+OK\r\n" + raw, err := readReply(reader(in)) + if err != nil { + t.Fatal(err) + } + if string(raw) != in { + t.Fatalf("got %q", raw) + } +} + +func TestReadReplyVerbatim(t *testing.T) { + in := "=15\r\ntxt:hello world\r\n" + raw, err := readReply(reader(in)) + if err != nil { + t.Fatal(err) + } + if string(raw) != in { + t.Fatalf("got %q", raw) + } +} + +func TestReadRequestArray(t *testing.T) { + in := "*2\r\n$4\r\nINFO\r\n$3\r\nall\r\n" + raw, args, err := readRequest(reader(in)) + if err != nil { + t.Fatal(err) + } + if string(raw) != in { + t.Fatalf("raw %q", raw) + } + if len(args) != 2 || string(args[0]) != "INFO" || string(args[1]) != "all" { + t.Fatalf("args %q", args) + } +} + +func TestReadRequestInline(t *testing.T) { + _, args, err := readRequest(reader("PING\r\n")) + if err != nil { + t.Fatal(err) + } + if len(args) != 1 || string(args[0]) != "PING" { + t.Fatalf("args %q", args) + } +} + +func TestIsFullInfo(t *testing.T) { + cases := []struct { + args [][]byte + want bool + }{ + {[][]byte{[]byte("INFO")}, true}, + {[][]byte{[]byte("INFO"), []byte("all")}, true}, + {[][]byte{[]byte("INFO"), []byte("everything")}, true}, + {[][]byte{[]byte("INFO"), []byte("commandstats")}, false}, + {[][]byte{[]byte("INFO"), []byte("keyspace")}, false}, + } + for _, c := range cases { + if got := isFullInfo(c.args); got != c.want { + t.Errorf("isFullInfo(%q)=%v want %v", c.args, got, c.want) + } + } +} + +func TestApplyInjectAppend(t *testing.T) { + payload := []byte("# Server\r\nredis_version:7.2.0\r\n") + inject := [][]byte{[]byte("bytes_received_per_sec:123")} + out := applyInject(payload, inject) + if !bytes.Contains(out, []byte("bytes_received_per_sec:123")) { + t.Fatalf("append missing: %q", out) + } + if !bytes.Contains(out, []byte("redis_version:7.2.0")) { + t.Fatalf("original dropped: %q", out) + } +} + +func TestApplyInjectOverride(t *testing.T) { + payload := []byte("# Cluster\r\ncluster_enabled:0\r\n") + inject := [][]byte{[]byte("cluster_enabled:1")} + out := applyInject(payload, inject) + if bytes.Contains(out, []byte("cluster_enabled:0")) { + t.Fatalf("original value not overridden: %q", out) + } + if !bytes.Contains(out, []byte("cluster_enabled:1")) { + t.Fatalf("override missing: %q", out) + } + // Override must not also append a duplicate under "# Faked". + if bytes.Count(out, []byte("cluster_enabled:")) != 1 { + t.Fatalf("duplicate key after override: %q", out) + } +} + +func TestRewriteInfoReplyReframesLength(t *testing.T) { + payload := "redis_version:7.2.0\r\n" + reply := []byte("$" + strconv.Itoa(len(payload)) + "\r\n" + payload + "\r\n") + inject := [][]byte{[]byte("sentinel_masters:2")} + out := rewriteInfoReply(reply, inject) + + // Parse the reframed reply back and confirm the declared length matches. + raw, err := readReply(reader(string(out))) + if err != nil { + t.Fatalf("reframed reply unreadable: %v (%q)", err, out) + } + if !bytes.Equal(raw, out) { + t.Fatalf("reframed length mismatch: reader consumed %q of %q", raw, out) + } + if !bytes.Contains(out, []byte("sentinel_masters:2")) { + t.Fatalf("inject missing: %q", out) + } +} + +func TestRewriteInfoReplyLeavesErrorsAlone(t *testing.T) { + reply := []byte("-ERR unknown\r\n") + out := rewriteInfoReply(reply, [][]byte{[]byte("x:1")}) + if !bytes.Equal(out, reply) { + t.Fatalf("error reply modified: %q", out) + } +} diff --git a/redisdb/tests/proxy/resp.go b/redisdb/tests/proxy/resp.go new file mode 100644 index 0000000000000..41b5108a2b36f --- /dev/null +++ b/redisdb/tests/proxy/resp.go @@ -0,0 +1,171 @@ +// RESP framing for the INFO-rewrite proxy. +// +// The proxy only needs to (a) split the client->server stream into individual +// requests so it can spot the INFO command, and (b) split the server->client +// stream into individual replies so it can rewrite exactly the reply that +// belongs to an INFO request and forward everything else untouched. It never +// needs to interpret values beyond finding their byte boundaries, so the reader +// captures raw bytes and returns them verbatim. +// +// Both RESP2 and RESP3 are supported because the two scrapers negotiate +// different protocols: the Datadog redisdb check pins RESP2 (protocol=2) while +// the OpenTelemetry redisreceiver's go-redis client negotiates RESP3 via HELLO. +package main + +import ( + "bufio" + "bytes" + "io" + "strconv" +) + +// readLine reads through the next \n and returns the bytes including the +// trailing CRLF. The RESP grammar terminates every framing token with CRLF. +func readLine(r *bufio.Reader) ([]byte, error) { + line, err := r.ReadBytes('\n') + if err != nil { + return line, err + } + return line, nil +} + +// parseLen parses the integer count that follows a RESP type byte (e.g. the +// "5" in "$5\r\n"). It tolerates the trailing CRLF and any RESP3 streaming +// marker ("?"), returning -1 when the count cannot be parsed (treated as null). +func parseLen(b []byte) int { + s := string(bytes.TrimRight(b, "\r\n")) + if s == "" || s == "?" { + return -1 + } + n, err := strconv.Atoi(s) + if err != nil { + return -1 + } + return n +} + +// readReply reads one complete RESP value of any type and returns its raw +// bytes. Aggregate types (arrays, maps, sets, pushes, attributes) are read +// recursively so nested values are captured whole. This is what lets the proxy +// forward every non-INFO reply byte-for-byte. +func readReply(r *bufio.Reader) ([]byte, error) { + line, err := readLine(r) + if err != nil { + return line, err + } + if len(line) == 0 { + return line, nil + } + + switch line[0] { + // Single-line replies: simple string, error, integer, null, boolean, + // double, big number. The whole value is the line itself. + case '+', '-', ':', '_', '#', ',', '(': + return line, nil + + // Length-prefixed blobs: bulk string, blob error, verbatim string. + // A negative length is the RESP2 null bulk string ("$-1\r\n"). + case '$', '!', '=': + n := parseLen(line[1:]) + if n < 0 { + return line, nil + } + buf := make([]byte, n+2) // payload + CRLF + if _, err := io.ReadFull(r, buf); err != nil { + return append(line, buf...), err + } + return append(line, buf...), nil + + // Aggregates with N elements: array, set, push. + case '*', '~', '>': + count := parseLen(line[1:]) + out := line + if count < 0 { + return out, nil + } + for i := 0; i < count; i++ { + child, err := readReply(r) + out = append(out, child...) + if err != nil { + return out, err + } + } + return out, nil + + // Aggregates with N key/value pairs: map, attribute. An attribute block + // ("|") is a metadata prefix that is followed by the actual reply, so we + // read one more value after it to complete the logical reply. + case '%', '|': + count := parseLen(line[1:]) + out := line + if count >= 0 { + for i := 0; i < count*2; i++ { + child, err := readReply(r) + out = append(out, child...) + if err != nil { + return out, err + } + } + } + if line[0] == '|' { + child, err := readReply(r) + out = append(out, child...) + if err != nil { + return out, err + } + } + return out, nil + + default: + // Inline/unknown: forward the single line best-effort. + return line, nil + } +} + +// readRequest reads one client request and returns its raw bytes plus the +// decoded argument list (used only to identify the command). Clients always +// send commands as RESP arrays of bulk strings regardless of the negotiated +// protocol version; inline commands are handled as a fallback for humans using +// a raw socket. +func readRequest(r *bufio.Reader) (raw []byte, args [][]byte, err error) { + line, err := readLine(r) + if err != nil { + return line, nil, err + } + if len(line) == 0 { + return line, nil, nil + } + + if line[0] != '*' { + // Inline command: whitespace-separated tokens on one line. + for _, f := range bytes.Fields(bytes.TrimRight(line, "\r\n")) { + args = append(args, f) + } + return line, args, nil + } + + n := parseLen(line[1:]) + raw = append(raw, line...) + for i := 0; i < n; i++ { + l2, err := readLine(r) + raw = append(raw, l2...) + if err != nil { + return raw, args, err + } + if len(l2) == 0 || l2[0] != '$' { + continue + } + ln := parseLen(l2[1:]) + if ln < 0 { + continue + } + buf := make([]byte, ln+2) // payload + CRLF + if _, err := io.ReadFull(r, buf); err != nil { + raw = append(raw, buf...) + return raw, args, err + } + raw = append(raw, buf...) + args = append(args, buf[:ln]) + } + return raw, args, nil +}