diff --git a/CHANGELOG.md b/CHANGELOG.md index b3f1ef6..c3737c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,8 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] - ### Added + + +- `/api/thread/stacks` (and `getThreadCallStacks()`) print each thread's **name** next to its tid, read from `/proc//comm`. A bare tid is not identifiable when reading the output afterwards; the name is what tells you that thread 1234 is `DrogonIoLoop` and 1235 is a worker parked on a futex. - CMake options documentation and reproducible `CMakePresets.json` presets (`debug`, `release`, `relwithdebinfo`, `coverage`, clang variants) - `find_package` integration test under `cmake/examples/test_find_package/` - Doxygen API documentation support (`-DBUILD_DOCS=ON`) @@ -20,10 +22,72 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `profiler/async_executor.h` — bounded single-worker executor for long-running jobs ### Changed + + +- The Heap Snapshot section is one **output** dropdown (raw profile text / flame graph / call graph) with a single 打开 and 下载 pair, instead of separate buttons per product. Three options, one endpoint: `?format=profile`, `?format=svg&renderer=flamegraph`, `?format=svg&renderer=callgraph`. +- Each chart section owns its renderer selector. The snapshot section previously read the Heap Profiler section's dropdown, so which picture you got from the snapshot depended on a control in a different block. +- Every section offers 打开 (`output=inline`) alongside 下载 (`output=attachment`). The inline mode was reachable only by hand-assembling a URL before, since the buttons hard-coded attachment -- a feature nothing in the UI exercised. +- **Breaking:** the HTTP API was consolidated. Nine endpoints across three + profiler types became four: + + | before | after | + |---|---| + | `/api/{cpu,heap,growth}/analyze` | `/api/pprof/{cpu,heap,growth}` | + | `/api/{cpu,heap,growth}/svg_raw` | `/api/pprof/{cpu,heap,growth}?renderer=callgraph` | + | `/api/{cpu,heap,growth}/flamegraph_raw` | `/api/pprof/{cpu,heap,growth}?renderer=flamegraph` | + | `/api/heap/*?source=state` | `/api/pprof/heap/snapshot` | + + `analyze` and `svg_raw` were the *same operation* — sample for N seconds, then + render — differing only in the renderer and in whether the response forced a + download, so each pair sampled the process twice for one picture. The renderer + is now a parameter (`renderer=flamegraph|callgraph`) and delivery is another + (`output=inline|attachment`). The old paths are gone, not aliased. +- **Breaking:** the heap snapshot is its own endpoint, + `/api/pprof/heap/snapshot`, with `format=profile` (default, raw text for + `go tool pprof`) or `format=svg`. It is state-based ("what is in the heap now") + as opposed to `/api/pprof/heap`, which is window-based ("what was allocated + during the window"). Keeping it on the same resource behind a `source=` flag + would have put two different questions at one URL. +- **Breaking:** `renderer=callgraph` replaces `output_type=pprof`. The old value + named a tool rather than the result: the pprof script draws a node/edge *call + graph*, while FlameGraph draws a *flame graph*. They are different pictures, not + two styles of one, and the parameter should say which picture you get. +- The `duration` default is now 10s everywhere (was 1s for heap, 30s for + `/pprof/profile`). Measured on the bundled example workload, a 3s window failed + to collect enough samples to render 35-45% of the time; 10s never failed. A 1s + default was close to useless. +- Responses are `inline` by default, so a plain link shows the SVG in the browser + instead of downloading it; `output=attachment` restores a forced download. + Inline display is purely the absence of `Content-Disposition` — verified that + the same bytes navigate as a document without it and abort with it. +- The three `/show_*_svg.html` viewer pages are removed, along with the inline-SVG + branch they consumed. Their in-page pan/zoom never worked: the generated SVGs + embed a pan/zoom library that is never initialised (FlameGraph's `zoom()` looks + for a `#viewport` element that its own output does not contain, and the pprof + SVGPan library has no `onload` hook). Downloading the SVG and opening it in a + desktop tool is what actually works, so the panel now offers downloads and a + live log rather than a viewer. +- `ProfilerHttpHandlers` exposes one entry point per profiler — + `handleCpuChart` / `handleHeapChart` / `handleGrowthChart` / `handleHeapSnapshot` — + each taking a `ChartOptions` (renderer, duration, delivery) instead of the + per-endpoint duration/output_type argument lists. Shared rendering moved into + `renderChart`/`renderFlameGraph`/`renderCallGraph`, replacing six copies of the + same shell-out logic. - **Breaking:** `ProfilerManager` is no longer a singleton — construct it directly and manage its lifetime - **Breaking:** removed the spdlog dependency; use `setLogSink()` / `setLogLevel()` instead - **Breaking:** core library decoupled from the web layer — `profiler_core` (no Drogon) and `profiler_web` (optional Drogon adapter) are now separate targets -- **Breaking:** `analyzeHeapProfile()` no longer takes a `duration` argument. gperftools heap profiling is allocation driven (`HeapProfilerStart()` / `HeapProfilerDump()`), so the sample rate is fixed at process start by `TCMALLOC_SAMPLE_PARAMETER` and elapsed time has no effect. `ProfilerHttpHandlers::handleHeapAnalyze()` likewise takes only `output_type`. +- **Breaking:** `analyzeHeapProfile()` takes `(duration, output_type)` again, and `handleHeapAnalyze` / `handleHeapSvgRaw` / `handleHeapFlamegraphRaw` all take a duration. This reverses an earlier removal that was right about the *sampling rate* but wrong to conclude the window is meaningless: gperftools fixes the rate via `TCMALLOC_SAMPLE_PARAMETER` at process start, so `duration` cannot improve precision, but it does decide **how long allocations are collected**. A process that allocates sparsely needs a longer window to produce anything, and a hard-coded 1s gave it no recourse. Default 1s, clamped 1-300s. +- `/api/heap/svg_raw` and `/api/heap/flamegraph_raw` now run their own sampling window (`getRawHeapProfileSample()`) instead of pulling `GetHeapSample()`. Besides honouring the duration, this makes them work in a process started without `TCMALLOC_SAMPLE_PARAMETER`, which previously made `/api/heap/svg_raw` fail with 500 while its sibling `/api/heap/flamegraph_raw` succeeded on the same input. The three window-based endpoints now behave identically. +- Both heap renderers accept `?source=state` to draw the state-based snapshot instead, so the cumulative heap view can be rendered as a chart rather than only exported as text. Previously the graph endpoints could only draw a collection window, which meant the panel had no way to show the live heap visually even though the rendering path existed. +- `handleHeapSvgRaw` renders with `pprof --alloc_space` rather than the default `--inuse_space`. pprof's default plots what is live *now*, so a short collection window -- whose allocations have usually already been freed -- produced a profile with an empty in-use column and the render failed with "No nodes to print" even though the window recorded real allocations. Measured: `?duration=1` went from 500 to a 23KB graph. +- The flame-graph handlers' `pprof --collapsed` step also uses `--alloc_space` now, and no longer counts pprof's own "Using local file ..." chatter as stack data. pprof's default reports what is live *now*, so a short window often produced a collapsed file with zero stack lines -- which still passed the "has data" check because the two chatter lines do not start with `#` -- and flamegraph.pl then rejected it. Measured: `?duration=2` failed 2 times out of 5 before, 0 out of 6 after, and the same sample already rendered fine through the svg path, which is what pointed at the mode rather than the data. +- All four flame-graph handlers share one validator (`flameGraphResponse`). flamegraph.pl answers unusable input with a *valid* SVG whose only content is an "ERROR: ..." message, so the structural `` into the DOM before clicking it; the heap/CPU/growth chart downloads never did, which leaves the click unreliable in browsers that require attachment. - **Breaking:** removed the never-implemented `ProfilerHttpHandlers::dispatch()` declaration; route registration stays with the host framework - **The Drogon adapter no longer runs profiling on the event-loop thread.** Every handler that can block — CPU/heap/growth analysis, the `_raw` renderers, and `/pprof/profile` — is dispatched to a background worker and its response is handed back with `queueInLoop()`. Profiling takes seconds (up to 300s for CPU), so this previously stalled all other requests, including `/api/status`, for the whole sampling window. - Blocking jobs share one worker thread. This is required for correctness, not just throughput: gperftools keeps its profiling session in process-global state, so concurrent analyses would corrupt each other's results. @@ -38,6 +102,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Installation layout made consistent across `lib`/`lib64` hosts so `find_package()` works (GNUInstallDirs is now included before the install rules) ### Fixed +- `stopHeapProfiler()` and `getRawHeapProfileSample()` no longer call `GetHeapProfile()`. They write the window's snapshot with `HeapProfilerDump()` and read that file back, so the profile memory belongs to us: nothing needs freeing, and the `leak:GetHeapProfile` suppression is gone. This replaces the previous workaround of retaining the buffer, which existed only because `GetHeapProfile()`'s return value has no portable deallocator (see ROADMAP). Note the dump lands beside the requested output path under a derived name -- gperftools appends `..heap` to the prefix -- so `stopHeapProfiler()` still writes the caller's path, from the dump's contents. +- Reverted the `free()` added to `stopHeapProfiler()` and `getRawHeapProfileSample()`: it aborted ASan's job with "attempting free on address which was not malloc()-ed". `GetHeapProfile()`'s header says the caller should free the buffer, and upstream's own test does exactly that, but the buffer has no portable deallocator in practice -- with a statically linked tcmalloc it comes from the library's own allocator, which the process's malloc interposer never saw (so `free()` aborts), `tc_free()` only exists from gperftools 2.16.90, and gperftools 2.18 additionally routes its intermediate chunks through an internal arena whose `Free` lives in a private header. The buffer is roughly 11KB per stop, so it is copied out and released to the OS instead; `lsan.supp` carries a matching suppression. The earlier "fix" traded a benign leak for a crash. +- The ASan job now also covers a statically linked tcmalloc build, which is what vcpkg produces and what the crash above required; the shared-library configuration used locally cannot reproduce it. +- Downloaded charts no longer overwrite each other. Filenames were built from a second-precision timestamp, so two downloads completing within the same second produced the same name and the browser silently replaced the first file with the second -- picking 火焰图 then 调用图 for the heap snapshot lost the flame graph, and the panel still reported both as saved. The timestamp now carries milliseconds. Found because the browser regression's download check counted files instead of comparing names, so an overwrite looked like a success; it now tracks the filename set and asserts that each of the seven downloads produced its own file. + + +- The test suite is safe to run in parallel. The profiler's intermediate artifacts + (`pprof_cpu_temp.prof`, `cpu_analyze.prof`, `cpu_collapsed.prof`, + `heap_collapsed.prof`, and the per-endpoint render inputs) had fixed names in a + shared directory, so two test binaries running at once truncated each other's + files. `ctest -j4` reproduced the CI failure locally: FullFlowTest read back a + 0-byte CPU profile because a concurrently running binary had recreated the + shared file. The names now carry the process id. +- `FullFlowTest.GetRawCPUProfile` (and the gperftools profile test) generate real + CPU work across the sampling window instead of sleeping through it. gperftools + samples executing threads, so a window spent idle yields an empty profile by + design -- the test treated that legitimate result as a failure whenever the + machine had nothing to sample. It now fails only for a real fault. +- An out-of-range `duration` now clamps to 1..300 for every analysis endpoint. The two backends disagreed: `getRawCPUProfile()` rejected out-of-range values (surfacing as "Failed to collect a CPU profile in the requested window") while `getRawHeapProfileSample()` clamped them, so the same query succeeded or failed depending on which profiler served it. The clamping happens once at the HTTP boundary. Measured after: `duration=-99` and `duration=0` both run a 1s window, `duration=9999` runs 300s rather than hanging for 9999. +- `stopHeapProfiler()` freed nothing for the profile it generates: + `GetHeapProfile()` returns a malloc'd string the caller must `free()`, and + letting it convert directly into a `std::string` discarded the pointer, leaking + the entire profile on every stop (LeakSanitizer: ~11.6KB per call). The path + only became reachable once `analyzeHeapProfile()` and the new + `getRawHeapProfileSample()` started routing through `stopHeapProfiler()`, but + the leak itself is older: that function previously had no callers at all. + Surfaced by the new HttpHandlersTest cases, which are the first to drive + start/stop directly. - `~ProfilerManager()` did not stop the heap profiler: it called `IsHeapProfilerRunning()`, a query whose result was discarded, leaving the process-global profiler recording after the object was destroyed and still dumping profiles during process exit - `/pprof/symbol` now implements the Go pprof symbolz protocol correctly - CMake package config exports the correct targets and dependency lookup @@ -46,7 +138,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - clang-tidy job could not locate dependency headers - Suppressed a known tcmalloc leak so ASan builds fail only on real errors -### Fixed - A session opened by the host through `startCPUProfiler()` / `startHeapProfiler()` is no longer preempted. `getRawCPUProfile()` and `analyzeCPUProfile()` used to stop whatever session was running and start their own; they now refuse and @@ -58,7 +149,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 while stopping an existing session; the stop and the state update now happen under the lock, as does the corresponding teardown in `getRawCPUProfile()`. +- `/pprof/heap` reported success when heap sampling was off. `GetHeapSample()` does + not return an empty string in that case -- it returns pprof's `%warn` advisory + followed by a valid-looking but empty profile (`@ heap_v2/0`) -- so the + emptiness check never fired and clients received HTTP 200 carrying the warning + text. The zero sampling rate (or a leading `%warn`) is now recognised and + reported as an error. This also removes an inconsistency where + `/api/heap/svg_raw` failed with 500 while `/api/heap/flamegraph_raw` rendered a + 13KB graph from the identical input. +- Analyze endpoints emitted invalid JSON on failure: the core returns a + `{"error": ...}` string and the handler wrapped it again without escaping, + producing `{"error":"{"error": "..."}`. The inner message is now extracted + first; `handleHeapAnalyze` also no longer discards the reason in favour of a + generic string. +- `/pprof/heap` and `/pprof/growth` errors now follow Go's `serveError()` shape + (`text/plain` plus `X-Go-Pprof: 1`), so `go tool pprof` prints the reason + instead of the content type misleading it. +- `flamegraph.pl` answers empty input with a *valid* SVG whose entire content is + an error message, so the structural SVG check passed and callers got HTTP 200 + plus a graph reading "ERROR: No valid input provided to flamegraph.pl." That is + now detected and returned as a 500 explaining that the sample window was too + short or the process idle. + +### Removed + + +- **Breaking:** `ProfilerManager::getThreadStacks()` is gone. It was 122 lines that nothing called: `/api/thread/stacks` has always used `getThreadCallStacks()`, which answers the same question ("where is each thread stuck") with a full call stack instead of a `/proc` field. Its one unique contribution, the thread name, is now part of the endpoint output. +- `logger.h` (unused after the logging rework) +- The synthetic allocation thread inside `analyzeHeapProfile()` + ### Documentation + + - README, the API reference and the troubleshooting guide now spell out that the two heap views are **opposite in kind**, not two renderings of one dataset: `/pprof/heap` is state-based (cumulative snapshot of the current heap, needs @@ -70,13 +192,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 allocations freed inside the window still do, and consecutive sessions do not accumulate. - The heap Web panel says which of the two it performs, and no longer labels a - window-rendered SVG download as a "heap profile". - -### Removed -- `logger.h` (unused after the logging rework) -- The synthetic allocation thread inside `analyzeHeapProfile()` + window-rendered SVG download as a "heap profile". It also gained the missing + entry point for the state-based view: the panel previously offered only the + window-based endpoints, so a process holding a large heap but no longer + allocating could not be inspected from the UI at all. The Heap section is now + two labelled cards, the second driving `GET /pprof/heap` with view and + download actions. ### Notes + + - The repository currently has **no git tags**; `v0.1.0` below refers to commit `a503106`. Tagging is tracked in [ROADMAP.md](ROADMAP.md). ## [0.1.0] - 2026-02-05 diff --git a/CMakeLists.txt b/CMakeLists.txt index 0d8100a..d78cfb4 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -266,6 +266,27 @@ if(REMOTE_PROFILER_BUILD_TESTS) ) add_test(NAME AsyncExecutorTest COMMAND test_async_executor) + # Return-value parsing shared by the HTTP layer (internal, no Drogon needed) + add_executable(test_result_parsing tests/test_result_parsing.cpp) + target_link_libraries(test_result_parsing + profiler_core + GTest::gtest + GTest::gtest_main + pthread + ) + add_test(NAME ResultParsingTest COMMAND test_result_parsing) + + # Thread stack capture (core only -- works without Drogon) + add_executable(test_thread_stacks tests/test_thread_stacks.cpp) + target_link_libraries(test_thread_stacks + PRIVATE + profiler_core + GTest::gtest + GTest::gtest_main + pthread + ) + add_test(NAME ThreadStacksTest COMMAND test_thread_stacks) + message(STATUS "Tests will be built") else() message(STATUS "Tests disabled") diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b1f8b67..884c062 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -285,6 +285,32 @@ cd build ctest -R --output-on-failure ``` +### 提交前的附加检查 + +`ctest` 通过**不足以**覆盖两类问题,仓库里有两个脚本专门补这个缺口: + +```bash +# 1) Sanitizer:本地复刻 CI 的 ASan/UBSan/TSan 配置 +scripts/check-sanitizers.sh asan # 默认只跑 ASan +scripts/check-sanitizers.sh all # ASan + UBSan + TSan + +# 2) Web UI:用无头 Chrome 驱动真实控制面板,断言每个动作都发出正确的请求 +# 并真正落盘(需要先启动 profiler_example,以及 puppeteer-core) +scripts/verify-web-ui.mjs http://localhost:8080 +``` + +为什么需要它们: + +- **Sanitizer**:`ctest` 默认构建不会报内存泄漏与未定义行为。曾经有一个 + `stopHeapProfiler()` 的 LeakSanitizer 报告只被 CI 发现,因为本地从没跑过 + `-fsanitize=address`。 +- **Web UI**:`curl` **无法执行页面 JavaScript**,因此看不到"按钮点击报 + ReferenceError""按钮卡在 disabled""下载其实没开始"这类问题。前端改动必须用 + 真实浏览器验证后才算完成。 + +> 改前端(`src/web_resources.cpp` 里的 HTML/JS 字符串)时尤其注意:那段 JS 是 +> C++ 字符串字面量,**编译器不会检查它**——函数名写错、参数漏传都能编译通过。 + ## 需要帮助? 如果你有任何问题,可以: diff --git a/README.md b/README.md index 843f5c6..6d027f0 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ - **Heap Growth Profiling** — 堆增长栈分析,无需 `TCMALLOC_SAMPLE_PARAMETER` - **线程堆栈捕获** — 采集进程内所有线程的调用栈,支持动态线程数 - **标准 pprof 接口** — `/pprof/*` 兼容 Go pprof 工具链 -- **一键分析** — `/api/*/analyze` 直接返回火焰图 SVG,浏览器可看 +- **一键分析** — `/api/pprof/*` 直接返回渲染好的 SVG(默认火焰图,可选调用图),浏览器可显示 - **框架无关** — `ProfilerHttpHandlers` 返回普通结构体,可接入任意 Web 框架 - **可选 Web 层** — 核心库不依赖任何 Web 框架 - **可配置日志** — 实现 `LogSink` 即可接入宿主应用的日志系统 @@ -39,7 +39,8 @@ 参考 Go pprof 提供两种互补的使用方式: 1. **标准 pprof 模式** — `/pprof/profile`、`/pprof/heap` 返回原始 profile 文件,交给 `go tool pprof` 分析 -2. **一键分析模式** — `/api/cpu/analyze` 等直接返回 SVG,适合浏览器即时查看 +2. **一键分析模式** — `/api/pprof/cpu` 等直接返回 SVG(`renderer=flamegraph` 火焰图或 `renderer=callgraph` 调用图), + 默认 `output=inline`,浏览器可直接显示 架构上分为两层,边界清晰: @@ -48,7 +49,12 @@ - `ProfilerHttpHandlers` — 位于核心库中,返回 `HandlerResponse{status, content_type, body, headers}`,不绑定任何框架 - `ProfilerManager` — 普通类而非单例,生命周期由使用者管理 -接口命名规则:`/pprof/*` 为 Go pprof 标准接口;`/api/*` 为项目自定义的分析/辅助接口;`/show_*.html` 为内置的 SVG 查看页。 +接口命名规则: + +- `/pprof/*` — Go pprof 标准接口,返回**原始 profile** 供 `go tool pprof` 消费 +- `/api/pprof/*` — 项目自定义的**分析接口**,服务端采样/渲染后直接返回 SVG +- `/api/status`、`/api/thread/stacks` — 状态查询与线程栈 +- `/` — Web 控制面板(不再有独立的查看器页) ## 快速开始 @@ -124,6 +130,17 @@ cd build && ./profiler_example 服务监听 `http://localhost:8080`。 +示例程序会起几个**有名字的后台线程**,便于验证 `/api/thread/stacks`: + +| 线程名 | 行为 | 预期栈顶 | +|--------|------|----------| +| `profiler-worker` | 跑 CPU/内存负载后 sleep 5s | `clock_nanosleep` | +| `profiler-sleeper` | 每 1.5s 醒一次 | `clock_nanosleep` | +| `profiler-parked` | 永久等在条件变量上 | futex 等待 | + +> 端点输出里会看到 `profiler-sleepe`(少一个 `r`)——这是**内核把线程名截断到 15 字符**, +> 不是拼写错误。输出读的是 `/proc//comm`,与 `ps`/`top` 显示的值一致。 + ## 运行时依赖 `ProfilerManager` 构造时会**向进程当前工作目录写入两个脚本**,分析接口通过相对路径调用它们: @@ -135,7 +152,7 @@ cd build && ./profiler_example 由此带来三点要求: -1. **工作目录必须可写**。以只读目录(如 `/`)为 CWD 启动会导致脚本写入失败,所有 `/api/*/analyze`、`/api/*/svg_raw`、`/api/*/flamegraph_raw` 接口返回 500。 +1. **工作目录必须可写**。以只读目录(如 `/`)为 CWD 启动会导致脚本写入失败,所有 `/api/pprof/*` 图表接口返回 500。 2. **必须安装 perl**:`sudo apt-get install -y perl`。 3. **不要删除这两个文件**,每个进程实例都会重新生成。 @@ -150,15 +167,15 @@ cd build && ./profiler_example | 端点 | 并发请求的响应 | |------|----------------| | `/pprof/profile` | `500`,`text/plain`,`Could not enable CPU profiling: cpu profiling already in use`,并带 `X-Go-Pprof: 1` 头 | -| `/api/cpu/analyze`、`/api/cpu/svg_raw`、`/api/cpu/flamegraph_raw` | `409`,`{"error":"cpu profiling already in use"}` | +| `/api/pprof/cpu` | `409`,`{"error":"cpu profiling already in use"}` | `X-Go-Pprof: 1` 是给 `go tool pprof` 的信号:告诉它响应体是错误消息而非 profile 数据。 -**宿主自己开的会话不会被抢占。** 如果你的程序用 `startCPUProfiler()` 主动开了一段采样,此期间的 HTTP 请求会被拒绝(同上表),而**不会**停掉你那段采样。同理 `startHeapProfiler()` 开的 heap 会话也不会被 `/api/heap/analyze` 抢占。 +**宿主自己开的会话不会被抢占。** 如果你的程序用 `startCPUProfiler()` 主动开了一段采样,此期间的 HTTP 请求会被拒绝(同上表),而**不会**停掉你那段采样。同理 `startHeapProfiler()` 开的 heap 会话也不会被 `/api/pprof/heap` 抢占。 -**Heap 分析同样独占**。`HeapProfilerStart()` 与 CPU profiler 不同——它**没有失败模式**,重复调用会静默替换输出前缀,所以必须先原子认领再启动,否则并发调用会共用同一份快照。`/api/heap/analyze` 在已有分析进行时返回 `409` + `{"error":"heap profiling already in use"}`。 +**Heap 分析同样独占**。`HeapProfilerStart()` 与 CPU profiler 不同——它**没有失败模式**,重复调用会静默替换输出前缀,所以必须先原子认领再启动,否则并发调用会共用同一份快照。`/api/pprof/heap` 在已有分析进行时返回 `409` + `{"error":"heap profiling already in use"}`。 -`/api/growth/analyze` **不受此限制**:它读取 `GetHeapGrowthStacks()`,不占用任何 profiler 会话,可与 CPU 采样并存。 +`/api/pprof/growth` **不受此限制**:它读取 `GetHeapGrowthStacks()`,不占用任何 profiler 会话,可与 CPU 采样并存。 快速接口(`/api/status`、`/`)不受采样影响,采样期间依然即时响应。 @@ -263,7 +280,7 @@ int main() { int main() { profiler::ProfilerManager profiler; - profiler::registerDrogonHandlers(profiler); // 注册全部 /pprof/* 与 /api/* 路由 + profiler::registerDrogonHandlers(profiler); // 注册 /pprof/* 与 /api/pprof/* 等全部路由 drogon::app().addListener("0.0.0.0", 8080).run(); } ``` @@ -279,7 +296,8 @@ int main() { profiler::ProfilerManager profiler; profiler::ProfilerHttpHandlers handlers(profiler); -auto resp = handlers.handleCpuAnalyze(10, "flamegraph"); +profiler::ChartOptions options; // renderer / duration / inline_display +auto resp = handlers.handleCpuChart(options); // resp.status / resp.content_type / resp.body / resp.headers → 用你的框架包一层 ``` @@ -307,73 +325,81 @@ profiler.setLogLevel(profiler::LogLevel::Debug); 由 `registerDrogonHandlers()` 注册的全部路由。 -> **Heap 的两套机制**(详见[下方说明](#heap-的两个端点语义相反)): -> `/pprof/heap` 是**状态式**(当前堆的累计快照),`/api/heap/*` 是**窗口式**(默认 1 秒内发生的分配)。 -> 二者不可互相替代,且**环境变量依赖相反**。 - | 端点 | 方法 | 说明 | |------|------|------| -| **标准 pprof 接口** | | | -| `/pprof/profile` | GET | CPU profile 原始文件;`?seconds=N`,默认 **30**,范围 1–300 | -| `/pprof/heap` | GET | **状态式**:当前堆的累计采样快照(原始文本,交给 `go tool pprof`);**需** `TCMALLOC_SAMPLE_PARAMETER` | -| `/pprof/growth` | GET | Heap growth 栈原始文本;无需上述环境变量 | -| `/pprof/symbol` | POST | 符号化接口(Go pprof symbolz 协议) | -| **一键分析(返回 SVG)** | | | -| `/api/cpu/analyze` | GET/POST | 采样并返回 CPU 图;`?duration=N` 默认 10,范围 1–300 | -| `/api/heap/analyze` | GET | **窗口式**:采样 1 秒后渲染;**不接受** `duration`;**无需**环境变量 | -| `/api/growth/analyze` | GET | 返回 Heap Growth 图 | -| **原始 SVG 下载** | | | -| `/api/cpu/svg_raw` | GET | pprof 生成的 CPU SVG;`?duration=N` 默认 10 | -| `/api/heap/svg_raw` | GET | **窗口式**:pprof 渲染的 Heap SVG;**需** `TCMALLOC_SAMPLE_PARAMETER` | -| `/api/growth/svg_raw` | GET | pprof 生成的 Growth SVG | -| `/api/cpu/flamegraph_raw` | GET | FlameGraph 渲染的 CPU SVG;`?duration=N` 默认 10 | -| `/api/heap/flamegraph_raw` | GET | **窗口式**:FlameGraph 渲染的 Heap SVG;**无需**环境变量 | -| `/api/growth/flamegraph_raw` | GET | FlameGraph 渲染的 Growth SVG | -| **线程分析** | | | -| `/api/thread/stacks` | GET | 所有线程的调用栈 | -| **辅助与页面** | | | -| `/` | GET | Web 控制面板 | +| **分析接口(主动采样后出图)** | | | +| `/api/pprof/cpu` | GET | 采样 CPU 并返回图 | +| `/api/pprof/heap` | GET | **窗口式**:采集窗口内发生的分配并返回图 | +| `/api/pprof/growth` | GET | 采集 heap growth 栈并返回图 | +| **Heap 快照(状态式)** | | | +| `/api/pprof/heap/snapshot` | GET | **状态式**:当前堆的累计快照(存量,非窗口) | +| **状态与辅助** | | | | `/api/status` | GET | 各 profiler 的运行状态与输出路径(JSON) | -| `/show_svg.html` | GET | CPU SVG 查看页 | -| `/show_heap_svg.html` | GET | Heap SVG 查看页 | -| `/show_growth_svg.html` | GET | Growth SVG 查看页 | +| `/api/thread/stacks` | GET | 所有线程的调用栈(含线程名,可看出各线程阻塞在哪个函数) | +| `/` | GET | Web 控制面板 | +| **标准 pprof 接口(机器可读,`go tool pprof` 用)** | | | +| `/pprof/profile` | GET | CPU profile 原始文件;`?seconds=N`,默认 **30**,范围 1–300 | +| `/pprof/heap` | GET | 当前堆快照原始文本;**需** `TCMALLOC_SAMPLE_PARAMETER` | +| `/pprof/growth` | GET | Heap growth 栈原始文本;无需该环境变量 | +| `/pprof/symbol` | POST | 符号化(Go pprof symbolz 协议) | -所有分析类接口都接受 `?output_type=flamegraph|pprof`: +### 分析接口的参数 -- `pprof` — 由内置 pprof 脚本渲染的图形(**HTTP 层的默认值**) -- `flamegraph` — 由 FlameGraph 渲染的火焰图 +三个 `/api/pprof/{cpu,heap,growth}` 端点共用同一组参数: -> 注意默认值差异:C++ API `analyzeCPUProfile()` 的形参默认是 `"flamegraph"`,而 HTTP 路由在未传 `output_type` 时使用 `"pprof"`。想稳定拿到火焰图请显式写 `?output_type=flamegraph`。 +| 参数 | 取值 | 默认 | 说明 | +|------|------|------|------| +| `renderer` | `flamegraph` \| `callgraph` | `flamegraph` | **两种不同的图形**,见下 | +| `duration` | 1–300 | **10** | 采集窗口(秒)。CPU/Growth 用于采样,heap 用于记录分配 | +| `output` | `inline` \| `attachment` | `inline` | 见下方「两种交付方式」 | -### 使用示例 +`/api/pprof/heap/snapshot` 另用 `format`: -```bash -# CPU:拿原始 profile 交给 pprof 工具(默认 30 秒) -curl 'http://localhost:8080/pprof/profile?seconds=10' > cpu.prof -go tool pprof -http=:8081 cpu.prof +| 参数 | 取值 | 默认 | 说明 | +|------|------|------|------| +| `format` | `profile` \| `svg` | `profile` | `profile` 返回原始 pprof 文本(交给 `go tool pprof`);`svg` 渲染成图(面板的下拉即这三项:文本 / 火焰图 / 调用图) | +| `renderer` | 同上 | `flamegraph` | 仅 `format=svg` 时有意义 | -# CPU 火焰图(显式指定 output_type) -curl 'http://localhost:8080/api/cpu/analyze?duration=10&output_type=flamegraph' -o cpu.svg +### 两种交付方式 -# Heap(需先设置 TCMALLOC_SAMPLE_PARAMETER) -curl 'http://localhost:8080/api/heap/analyze?output_type=flamegraph' -o heap.svg +`output` 决定浏览器是**显示**还是**下载**同一份 SVG。区别**只在响应头**: -# 所有线程调用栈 -curl 'http://localhost:8080/api/thread/stacks' -``` +| 取值 | `Content-Type` | `Content-Disposition` | 浏览器行为 | +|------|----------------|----------------------|-----------| +| `inline`(默认) | `image/svg+xml` | **不发** | 当作 SVG 文档**直接渲染** | +| `attachment` | `image/svg+xml` | `attachment; filename=…` | **下载**为文件 | -也支持直接让 pprof 从 URL 拉取: +所以想看一眼图表,直接把 URL 粘到地址栏即可,无需下载;需要存盘或交给桌面工具时用 +`output=attachment`。Web 面板每个区块都提供「🔗 打开」(inline)与「📥 下载」(attachment)两个按钮。 -```bash -go tool pprof -http=:8081 'http://localhost:8080/pprof/profile?seconds=10' -``` +> ⚠️ `inline` 只是"显示",**不代表可缩放**——产物内嵌的 pan/zoom 脚本依赖的元素/初始化挂钩 +> 缺失(见下)。需要缩放请下载后用桌面工具打开,或用浏览器页面缩放。 + +### 两种图形不是同一种东西 + +`renderer` 的选择会改变**图形表达**,不只是画法: + +| 取值 | 产出 | 特点 | +|------|------|------| +| `flamegraph` | **火焰图**(栈帧堆叠) | 层次直观,适合看调用占比 | +| `callgraph` | **调用图**(graphviz 节点/边) | 能解析内联帧,符号化更完整 | + +> ⚠️ **两者都不可在浏览器里缩放。** 生成的 SVG 内嵌了 pan/zoom 脚本,但依赖的 +> 元素或初始化挂钩并未出现在产物中(实测:FlameGraph 的 `zoom()` 找不到 +> `#viewport`,pprof 的 SVGPan 从未被初始化)。需要缩放时请把 SVG 下载后用桌面 +> 工具打开,或在浏览器里用页面缩放(Ctrl + 滚轮)。 + +### 关于 `duration` 的默认值 + +默认 **10 秒**而非更短,是实测结果:示例负载下 3 秒窗口有 **35–45%** 的请求采不到 +足够样本而渲染失败,10 秒窗口为 **0%**。分配稀疏进程需要更长窗口。 ## 配置说明 ### 环境变量 `TCMALLOC_SAMPLE_PARAMETER` 控制 tcmalloc 的堆采样间隔(单位字节)。**默认值为 `0`,即关闭采样**。它影响的是**状态式**拉取路径 -(`GetHeapSample()`),也就是 `/pprof/heap` 与 `/api/heap/svg_raw`: +(`GetHeapSample()`),也就是 `/pprof/heap` 与 `/api/pprof/heap/snapshot?format=profile`: ```bash export TCMALLOC_SAMPLE_PARAMETER=524288 # 512KB,开发环境常用 @@ -388,38 +414,69 @@ tcmalloc 在**进程初始化时**读取该变量,因此必须在启动前用 | 生产 | `2097152`(2MB)或更大,降低开销 | **不受该变量影响的路径**(它们走 `HeapProfilerStart/Dump`,由 -`HEAP_PROFILE_ALLOCATION_INTERVAL`(默认 1MB)等控制):`/api/heap/analyze`、 -`/api/heap/flamegraph_raw`。Heap Growth(`/pprof/growth`、`/api/growth/*`)走 +`HEAP_PROFILE_ALLOCATION_INTERVAL`(默认 1MB)等控制):`/api/pprof/heap`、 +`/api/pprof/heap/snapshot?format=svg`。Heap Growth(`/pprof/growth`、`/api/pprof/growth`)走 `GetHeapGrowthStacks()`,也不需要该变量。 -> ⚠️ 因此同一进程内可能出现"一个 heap 端点成功、另一个失败": -> 不设该变量时 `/api/heap/analyze` 与 `/api/heap/flamegraph_raw` 正常返回 SVG, -> 而 `/api/heap/svg_raw` 返回 500 —— 它们的数据源不同,不是同一个缺陷。 +> ⚠️ 不设该变量时:**窗口式**端点(`/api/pprof/heap`)仍然正常返回 SVG,因为它自己运行 +> `HeapProfilerStart/HeapProfilerDump/Stop` 窗口;而**状态式**的 `/pprof/heap` 与 +> `/api/pprof/heap/snapshot?format=profile` 返回 500。 ### Heap 的两个端点语义相反 这两个端点容易混淆,因为名字里都有 "heap",但记录的是**完全不同的东西**: -| | `/pprof/heap` | `/api/heap/analyze` | +| | `/pprof/heap`、`/api/pprof/heap/snapshot` | `/api/pprof/heap` | |---|---|---| -| 语义 | **状态式**:当前堆里的累计采样 | **窗口式**:1 秒内**发生的分配** | -| 底层 | `GetHeapSample()` 拉取 | `HeapProfilerStart()` → dump → `Stop()` | +| 语义 | **状态式**:当前堆里的累计采样 | **窗口式**:采集窗口内**发生的分配** | +| 底层 | `GetHeapSample()` 拉取(pull) | `HeapProfilerStart()` → `HeapProfilerDump()` → 读文件 | | 是否建 profiler 会话 | 否 | 是(独占,并发返回 409) | | 输出 | 原始 profile 文本,**客户端**用 `go tool pprof` 渲染 | 服务端渲染好的 SVG | | `TCMALLOC_SAMPLE_PARAMETER` | **必需** | 不需要 | +| 时长参数 | 无 | `?duration=N`,默认 10 秒 | + +> ⚠️ **两条路的取数接口不能互换。** 状态式用的 `GetHeapSample(std::string*)` 写进调用方的 +> 字符串,无分配无释放,看起来更省事——但它是 **pull 统计抽样**,与窗口式的 push 记录是两套机制。 +> 实测同一时刻:窗口 dump 报 `202: 13121057`(1312 万字节,即该窗口实际分配量),而 +> `TCMALLOC_SAMPLE_PARAMETER=524288` 下 `GetHeapSample` 只报 `17: 1114112`(111 万), +> 未设该变量时直接返回 `%warn`。用它替代窗口式取数会**静默少报约 12 倍**。 +> +> 同理,窗口式也不使用 `GetHeapProfile()`:它返回的缓冲区没有可移植的释放方式 +> (静态链接 tcmalloc 时 `free()` 会被 ASan 判为 bad-free),因此改为 dump 到文件再读回, +> 内存归调用方所有。详见 ROADMAP。 + 关键差别在于**"存量"与"流量"**: -- 进程已占 500MB 但停止分配 → `/pprof/heap` 能看到这 500MB;`/api/heap/analyze` 返回**空**并报 +- 进程已占 500MB 但停止分配 → `/pprof/heap` 能看到这 500MB;`/api/pprof/heap` 返回**空**并报 `No heap profile data was produced` -- 进程频繁 malloc/free,dump 时已全部释放 → `/api/heap/analyze` **能看到**这些分配;`/pprof/heap` +- 进程频繁 malloc/free,dump 时已全部释放 → `/api/pprof/heap` **能看到**这些分配;`/pprof/heap` 的当前堆则可能很小 -所以窗口式分析需要**被分析进程在采样窗口内确实有分配**。窗口目前固定为 1 秒;需要更长窗口时, -请用 `startHeapProfiler()` / `stopHeapProfiler()` 自行掌控时机。 +所以窗口式分析需要**被分析进程在采集窗口内确实有分配**。 + +⚠️ **heap 的 `duration` 不是采样率**:采样率由 `TCMALLOC_SAMPLE_PARAMETER` 在进程启动时固定, +`duration` 只决定**采集多久**。分配稀疏的进程需要更长的窗口才能采到东西,所以它是"覆盖度"参数 +而不是"精度"参数。默认 10 秒,范围 1–300。 + +Web 控制面板分成四块,各自对应一个端点: + +| 区块 | 端点 | 说明 | +|------|------|------| +| CPU Profiler | `/api/pprof/cpu` | 采样后出图 | +| Heap Profiler | `/api/pprof/heap` | **窗口式**:采集窗口内的分配(流量) | +| Heap Snapshot | `/api/pprof/heap/snapshot` | **状态式**:当前堆累计快照(存量) | +| Heap Growth | `/api/pprof/growth` | 增长栈分析 | -Web 控制面板里的两个 Heap 按钮走的是**窗口式**(`/api/heap/analyze` 与 -`/api/heap/{svg_raw,flamegraph_raw}`),面板上已标注这一点。 +每块都有「🔗 打开」(inline,浏览器内显示)与「📥 下载」(attachment)两个按钮。 + +Heap Snapshot 那块用一个**快照输出**下拉选择产出,三种选项走同一个端点: + +| 下拉选项 | 实际请求 | +|----------|----------| +| 原始 profile 文本 | `?format=profile`(交给 `go tool pprof`) | +| 火焰图 (FlameGraph) | `?format=svg&renderer=flamegraph` | +| 调用图 (callgraph) | `?format=svg&renderer=callgraph` | ### 依赖版本 @@ -465,7 +522,10 @@ cpp-remote-profiler/ ├── docs/ │ ├── mainpage.dox Doxyfile.in │ └── user_guide/ # 用户文档(安装、API、集成、排错) -├── scripts/check-format.sh # clang-format 检查(CI 使用 18) +├── scripts/ +│ ├── check-format.sh # clang-format 检查(CI 使用 18) +│ ├── check-sanitizers.sh # 本地复刻 CI 的 ASan/UBSan/TSan(CTEST_JOBS=n 可并行) +│ └── verify-web-ui.mjs # 无头 Chrome 驱动真实面板,验证按钮→请求→落盘 └── .github/workflows/ # CI:构建、测试、ASan/UBSan、clang-tidy、格式检查 ``` @@ -480,6 +540,22 @@ ctest --test-dir build/debug --output-on-failure ./scripts/check-format.sh --fix # 自动修复 ``` +`ctest` 通过不足以覆盖两类问题,另外两个脚本专门补这个缺口: + +```bash +# Sanitizer:本地复刻 CI 的 ASan/UBSan/TSan 配置 +scripts/check-sanitizers.sh asan +CTEST_JOBS=6 scripts/check-sanitizers.sh asan # 同时验证并行下的稳定性 + +# Web UI:无头 Chrome 驱动真实控制面板,断言每个动作发出正确请求并真正落盘 +npm install puppeteer-core # 一次性 +scripts/verify-web-ui.mjs http://localhost:8080 +``` + +> ⚠️ **`curl` 不能替代浏览器验证**:它无法执行页面 JavaScript,因此看不到「按钮点击报 +> `ReferenceError`」「按钮卡在 disabled」「下载其实没开始」这类问题。前端改动必须用真实 +> 浏览器验证。另外面板的 JS 存在于 C++ 字符串字面量中,**编译器不检查它**。 + 贡献流程见 [CONTRIBUTING.md](CONTRIBUTING.md);版本历史见 [CHANGELOG.md](CHANGELOG.md);后续计划见 [ROADMAP.md](ROADMAP.md)。 ## 文档索引 diff --git a/ROADMAP.md b/ROADMAP.md index e445fc1..8454770 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -15,7 +15,7 @@ ### 已修复
-1. `/api/heap/analyze` 的 duration 语义(✅ 已修复) +1. `/api/heap/analyze` 的 duration 语义(✅ 已修复;该路径现已并入 `/api/pprof/heap`) 原问题:路由只解析 `output_type`,`handleHeapAnalyze()` 也没有 duration 形参,最终硬编码 `analyzeHeapProfile(1, ...)`。 @@ -68,14 +68,68 @@
+### 已修复(本轮,用 go tool pprof 验证) + +- `/pprof/heap` 在采样关闭时返回 200 + `%warn` 文本:`GetHeapSample()` 不返回空串,导致空值检查失效。现按 `@ heap_v2/0` / 前导 `%warn` 识别,返回 500。 +- `/api/heap/svg_raw` 与 `flamegraph_raw` 的行为不一致:根因同上,两者现在都返回同样的明确错误(后经 API 重构合并为 `/api/pprof/heap`)。 +- 分析类接口的错误响应**双重包裹**成非法 JSON(`{"error":"{"error": "..."}`):现将内部错误消息解出后再包一层。 +- `/pprof/heap` 与 `/pprof/growth` 的错误改用 Go `serveError()` 的形状(`text/plain` + `X-Go-Pprof: 1`),`go tool pprof` 能直接显示原因。 +- `flamegraph.pl` 对空输入返回"合法 SVG + ERROR 文本",被当作 200 成功:现识别并转为带原因的 500。 + +### 已完成(本轮 API 重构) + +- HTTP API 收敛:9 个端点 → 4 个(`/api/pprof/{cpu,heap,growth}` + `/api/pprof/heap/snapshot`), + 渲染器与交付方式改为参数(`renderer` / `output`),消除了 `analyze` 与 `*_raw` 各采样一次的重复。 +- `duration` 默认值 1 秒 → 10 秒(实测 3 秒失败率 35–45%、10 秒 0%,见 CHANGELOG)。 +- 移除 3 个查看器页及其内联 SVG 分支——其页内缩放从未生效(生成的 SVG 内嵌了从未被初始化的 + pan/zoom 库)。面板改为「下载 + 实时日志」。 +- 新增 `scripts/verify-web-ui.mjs`(CDP 驱动真实浏览器回归,23 项检查)与 + `scripts/check-sanitizers.sh`(本地复刻 CI 的 ASan/UBSan/TSan 配置)。 + +### 已完成(CI 稳定性) + +- 测试可在并行下运行(`ctest -j`)。此前 `pprof_cpu_temp.prof`、`cpu_collapsed.prof` 等 + 中间产物用**固定文件名**放在共享目录,两个测试进程会互相截断文件——CI 的 + `FullFlowTest` 读到 0 字节 profile 即源于此。现按 PID 命名。 +- `FullFlowTest.GetRawCPUProfile` 在采样窗口内**真的跑 CPU 负载**,而不是 sleep。 + gperftools 只采样正在执行的线程,窗口内空闲时返回空 profile 是**设计如此**, + 原断言把这种合法结果当成失败。 +- `scripts/check-sanitizers.sh` 支持 `CTEST_JOBS=` 以复现并行场景。 + ### 待修 +- **不要用 `GetHeapProfile()` 取窗口式 profile**(已通过绕开解决,此处记录原因): + 它返回的缓冲区头文件说调用方应 `free()`、上游测试也这么做,但没有可移植的释放方式—— + 静态链接 tcmalloc 时缓冲区来自库自身分配器,进程的 malloc 拦截器(ASan)不认识它, + `free()` 会 abort;`tc_free()` 仅 gperftools ≥ 2.16.90 存在;2.18 的中间 chunk 还走内部 + arena,其 `Free` 在私有头文件里。 + 现在两条窗口式路径(`stopHeapProfiler()`、`getRawHeapProfileSample()`)都改为 + **`HeapProfilerDump()` 落盘 + 读回**,内存归我们自己,无泄漏也无需抑制规则。 + 若将来 gperftools 提供公开释放函数,可再评估直接取缓冲区。 +- **`GetHeapSample(string*)` 不能替代窗口式取数**(实测记录,避免误改):它是 **pull 统计抽样**, + 与 `HeapProfilerStart/Stop` 的窗口记录是两套机制。实测同一时刻:窗口 dump 报 + `202: 13121057`(1312 万字节,即实际分配量),而 `TCMALLOC_SAMPLE_PARAMETER=524288` 下 + `GetHeapSample` 只报 `17: 1114112`(111 万),无该变量时干脆返回 `%warn`。若替换会**静默少报约 12 倍**。 + 状态式快照 `/api/pprof/heap/snapshot` 用它是对的,窗口式不能用。 + +- **生成的 SVG 无法缩放**:FlameGraph 的 `zoom()` 找不到它要操作的 `#viewport`(其产物只有 + `#frames`),pprof 的 SVGPan 库则从未被初始化(产物缺少 `onload` 挂钩)。两条路径的交互脚本 + 都是死代码。可选方案:改为在服务端生成自己的交互式 SVG,或在文档中明确"下载后用桌面工具看"。 + +- **火焰图里的函数名带地址后缀**:内置 pprof 的 `--collapsed` 输出把函数名写成 + `cpuIntensiveTask()<0000000000409360>`,经 `flamegraph.pl` 渲染后直接显示在图上。除了噪音,还有一个 + 理论隐患:同一函数出现在不同地址时(模板实例化)会被当成**不同栈帧**,本该合并的被拆开。 + 可在送入 `flamegraph.pl` 之前剥掉 `<十六进制>` 后缀。 +- **部分符号未 demangle**:实测火焰图里残留 `_ZSt12construct_at...` 之类的 mangled 名,且这类帧常显示为 + 无上下文的 `operator()[inline]`(lambda 的 `operator()`)。可对未解开的符号做二次 `__cxa_demangle`。 +- **两个 pprof 实现的差异未写进文档**:内置 Perl pprof(`renderer=callgraph` 的渲染器)符号化更完整—— + 实测同一份 profile 未解析帧为 0、且能标出 `(inline)`;而 `go tool pprof` 强在对比与交互 + (`-diff_base`、`-peek`、`-traces`)。README 目前只说了"两种渲染方式",没说各自适合什么场景。 - **`pprof --svg` 的失败信息不透出**:内置 pprof 脚本用 `dot`(graphviz) 渲染,采样点过少时 `dot` 失败,代码只回一句 `pprof did not generate valid SVG. Output: `(且 `svg_output` 为空),无法定位原因。应把 `dot` 的 stderr 一并返回。**这是异步改造期间实测复现的既有缺陷**,与请求调度无关。 -- **`stopHeapProfiler()` 的 `output_path` 语义**:它把 `GetHeapProfile()` 的返回值写进 `output_path` 文件,而 `.heap` 是 gperftools 自己按 prefix 写的,两套产物并存容易混淆。 -- **`/pprof/heap` 在采样关闭时返回 200,响应体却不是 profile**:`GetHeapSample()` 在 `TCMALLOC_SAMPLE_PARAMETER` 未设置(默认 0)时**不返回空串**,而是返回 pprof 的 `%warn` 警告文本加一行零样本统计(`heap profile: 0: 0 [0: 0] @ heap_v2/0`)。因此 `if (heap_sample.empty())` 这个错误分支进不去,客户端收到 200 + 看起来像 profile 的东西。应识别 `@ heap_v2/0` 或首行 `%warn` 并返回明确错误。 -- **`/api/heap/svg_raw` 与 `/api/heap/flamegraph_raw` 在无 `TCMALLOC_SAMPLE_PARAMETER` 时表现不一致**:两者**都**先调 `getRawHeapSample()`(源码已核对),拿到的都是 `%warn` + 零样本文本,但因为上面那条缺陷没被拦下,后续命令的容错差异决定成败——`pprof --svg`(`svg_raw`)对零样本 profile 失败并超出 3 秒超时,返回 500 `Failed to generate SVG`;`pprof --collapsed` + `flamegraph.pl`(`flamegraph_raw`)却渲染出 13KB 的 SVG,返回 200。实测复现。修掉上面那条(识别零样本)即可同时解决。 -- **`/tmp/cpp_profiler` 下的 `.heap` 快照不再清理**:每次 `analyzeHeapProfile` 用唯一前缀(时间戳 + 序号)产生一个新文件,长期运行会累积。 -- **`analyzeHeapProfile` 的采样窗口固定 1 秒且不可配置**:分配稀疏的进程在窗口内可能一次分配都没有,只能得到空结果。可考虑加一个可选的窗口参数,或复用 `startHeapProfiler()`/`stopHeapProfiler()` 让调用方掌控。 +- ~~**`stopHeapProfiler()` 的 `output_path` 语义**~~(**已解决**):两者不再并存——`HeapProfilerDump()` 写出的 `.heap` 只是**中间产物**,其内容被读出后写入 `output_path`,调用方拿到的仍是它请求的路径。中间文件保留在磁盘上(可作排查用),语义不再含混。 +- **窗口式 heap 端点缺少"采样率"调节**:`duration` 控制的是**采集多久**(覆盖度),采样率仍由 + `TCMALLOC_SAMPLE_PARAMETER` 在启动时固定。若调用方希望在运行期调整**精度**,目前没有途径 + (`HEAP_PROFILE_ALLOCATION_INTERVAL` 只在 `HeapProfilerStart` 前设环境变量才生效)。 ### 已完成的调度改造(供参考) @@ -84,7 +138,7 @@ - `include/profiler/async_executor.h` — 单工作线程执行器,把阻塞任务搬离事件循环,完成后经 `queueInLoop()` 回送响应 - 阻塞接口全部投递到该工作线程;`/api/status`、`/` 等快速接口保持在事件循环上 - **CPU 采样独占且 fail-fast**:`cpu_profiling_in_progress_` 是所有 CPU 采样入口(`getRawCPUProfile` / `analyzeCPUProfile`)共用的进程级标记。占用期间新请求**立即被拒绝**,不会排队、也不会打断正在进行的采样 -- 拒绝语义与 Go 的 `net/http/pprof` 对齐:`/pprof/profile` 返回 500 + `text/plain` + `X-Go-Pprof: 1`;`/api/cpu/*` 返回 409 JSON +- 拒绝语义与 Go 的 `net/http/pprof` 对齐:`/pprof/profile` 返回 500 + `text/plain` + `X-Go-Pprof: 1`;`/api/pprof/cpu` 返回 409 JSON - 拒绝判定发生在**事件循环线程上、任务入队之前**(`runAsync` 的 precheck)。这是必须的:若放在任务内部判断,只能在前一个任务结束后才执行,永远观察不到它正在运行 实测:20 秒采样期间 `/api/status` 响应 0.0004s;并发请求第二个 0.0005s 内被拒且第一个不受影响。 diff --git a/docs/mainpage.dox b/docs/mainpage.dox index 2b448eb..bf667f9 100644 --- a/docs/mainpage.dox +++ b/docs/mainpage.dox @@ -42,11 +42,20 @@ * The canonical, always-current endpoint list lives in the project README under * "API 端点" (README.md). Summary: * - * - Standard pprof: @c /pprof/profile, @c /pprof/heap, @c /pprof/growth, @c /pprof/symbol - * - One-click analysis (SVG): @c /api/{cpu,heap,growth}/analyze - * - Raw SVG download: @c /api/{cpu,heap,growth}/svg_raw and @c .../flamegraph_raw + * - Standard pprof (machine-readable): @c /pprof/profile, @c /pprof/heap, + * @c /pprof/growth, @c /pprof/symbol + * - Analysis charts: @c /api/pprof/{cpu,heap,growth} with @c renderer=flamegraph + * (stacked frames) or @c renderer=callgraph (graphviz node/edge graph), plus + * @c duration and @c output=inline|attachment + * - Heap snapshot (state-based, "what is in the heap now"): + * @c /api/pprof/heap/snapshot with @c format=profile|svg * - Thread stacks: @c /api/thread/stacks - * - Panel and viewers: @c /, @c /show_svg.html, @c /show_heap_svg.html, @c /show_growth_svg.html + * - Control panel: @c / + * + * There are no separate viewer pages; charts are delivered as SVG by the + * analysis endpoints themselves. Note that the generated SVGs cannot be zoomed + * in the browser (the embedded pan/zoom scripts depend on elements that the + * output does not contain), so scaling requires a desktop tool. * * @section runtime_dependencies Runtime Dependencies * diff --git a/docs/user_guide/01_quick_start.md b/docs/user_guide/01_quick_start.md index 1938df0..630f38f 100644 --- a/docs/user_guide/01_quick_start.md +++ b/docs/user_guide/01_quick_start.md @@ -218,7 +218,8 @@ int main() { profiler::ProfilerHttpHandlers handlers(profiler); // 调用任意 handler,获得框架无关的响应 - profiler::HandlerResponse resp = handlers.handleCpuAnalyze(10, "flamegraph"); + profiler::ChartOptions options; // renderer / duration / inline_display + profiler::HandlerResponse resp = handlers.handleCpuChart(options); // resp.status, resp.content_type, resp.body // 用你自己的 Web 框架包装这些数据 diff --git a/docs/user_guide/02_api_reference.md b/docs/user_guide/02_api_reference.md index 9ab5ef5..3cbb05c 100644 --- a/docs/user_guide/02_api_reference.md +++ b/docs/user_guide/02_api_reference.md @@ -42,7 +42,9 @@ profiler::ProfilerManager profiler; **说明**: 析构时停止正在运行的 **CPU** profiler,并恢复构造函数之外由本对象安装的信号处理器。 -> ⚠️ **已知缺陷**: 析构函数对 heap profiler 只调用了查询函数 `IsHeapProfilerRunning()`,**并未调用 `HeapProfilerStop()`**(`src/profiler_manager.cpp:94-96`)。因此 heap profiler 不会随对象销毁而停止,需要显式调用 `stopHeapProfiler()`。修复计划见 [ROADMAP](../../ROADMAP.md)。 +**heap profiler 也会被停止**:析构函数调用 `HeapProfilerStop()`(`ProfilerManager::~ProfilerManager`),因为 heap profiler 存活在**进程级**的 tcmalloc 状态里——不停止的话,对象销毁后它仍会记录并 dump。 + +> 该问题曾存在(析构里只调用了查询函数 `IsHeapProfilerRunning()`,其返回值被丢弃),现已修正并有回归测试 `ProfilerLifecycleTest.HeapProfilerStopsOnDestruction` 覆盖。 --- @@ -63,21 +65,31 @@ explicit ProfilerHttpHandlers(ProfilerManager& profiler); | 方法 | 签名 | 说明 | |------|------|------| | `handleStatus` | `HandlerResponse handleStatus()` | 返回所有 profiler 状态 (JSON) | -| `handleCpuAnalyze` | `HandlerResponse handleCpuAnalyze(int duration, const std::string& output_type)` | CPU 分析,返回 SVG | -| `handleCpuSvgRaw` | `HandlerResponse handleCpuSvgRaw(int duration)` | CPU 原始 SVG (pprof 生成) | -| `handleCpuFlamegraphRaw` | `HandlerResponse handleCpuFlamegraphRaw(int duration)` | CPU FlameGraph SVG | -| `handleHeapAnalyze` | `HandlerResponse handleHeapAnalyze(const std::string& output_type)` | Heap 分析,返回 SVG | -| `handleHeapSvgRaw` | `HandlerResponse handleHeapSvgRaw()` | Heap 原始 SVG | -| `handleHeapFlamegraphRaw` | `HandlerResponse handleHeapFlamegraphRaw()` | Heap FlameGraph SVG | -| `handleGrowthAnalyze` | `HandlerResponse handleGrowthAnalyze(const std::string& output_type)` | Growth 分析,返回 SVG | -| `handleGrowthSvgRaw` | `HandlerResponse handleGrowthSvgRaw()` | Growth 原始 SVG | -| `handleGrowthFlamegraphRaw` | `HandlerResponse handleGrowthFlamegraphRaw()` | Growth FlameGraph SVG | +| `handleCpuChart` | `HandlerResponse handleCpuChart(const ChartOptions&)` | CPU 采样并出图 | +| `handleHeapChart` | `HandlerResponse handleHeapChart(const ChartOptions&)` | Heap 窗口式采集并出图 | +| `handleGrowthChart` | `HandlerResponse handleGrowthChart(const ChartOptions&)` | Growth 采集并出图 | | `handlePprofProfile` | `HandlerResponse handlePprofProfile(int seconds)` | 标准 pprof CPU profile (二进制) | | `handlePprofHeap` | `HandlerResponse handlePprofHeap()` | 标准 pprof heap profile | | `handlePprofGrowth` | `HandlerResponse handlePprofGrowth()` | 标准 pprof growth profile | | `handlePprofSymbol` | `HandlerResponse handlePprofSymbol(const std::string& body)` | 符号化接口 (POST) | | `handleThreadStacks` | `HandlerResponse handleThreadStacks()` | 线程调用栈 | +### ChartOptions + +`handleCpuChart` / `handleHeapChart` / `handleGrowthChart` 共用同一个选项结构: + +```cpp +struct ChartOptions { + ChartRenderer renderer = ChartRenderer::FlameGraph; // FlameGraph 或 CallGraph + int duration = 10; // 采集窗口(秒),内部钳制到 1..300 + bool inline_display = true; // true 不发 Content-Disposition;false 强制下载 +}; +``` + +HTTP 层的 `renderer` 取值 `flamegraph` / `callgraph` 由此处的 `ChartRenderer` 决定 +(`callgraph` 即 pprof 脚本经 graphviz 画出的调用图)。`handleHeapSnapshot(options, as_profile)` +另有一个 `as_profile` 开关:true 返回原始 profile 文本,false 按 `options.renderer` 出图。 + ### 使用示例 ```cpp @@ -88,7 +100,8 @@ profiler::ProfilerManager profiler; profiler::ProfilerHttpHandlers handlers(profiler); // 调用任意 handler -profiler::HandlerResponse resp = handlers.handleCpuAnalyze(10, "flamegraph"); +profiler::ChartOptions options; // renderer / duration / inline_display +profiler::HandlerResponse resp = handlers.handleCpuChart(options); // resp.status, resp.content_type, resp.body, resp.headers // 用你自己的 Web 框架包装这些数据 @@ -146,7 +159,9 @@ struct ProfilerState { }; ``` -> ⚠️ `start_time` 与 `duration` 的单位都是**毫秒**(`src/profiler_manager.cpp:189-191`、`214`),HTTP 层 `/api/status` 返回的 JSON 键名也相应为 `duration_ms`。`include/profiler_manager.h:40` 的注释写作 "seconds",属过时注释。 +> ⚠️ `start_time` 与 `duration` 的单位都是**毫秒**(见 `ProfilerState` 的字段注释),HTTP 层 `/api/status` 返回的 JSON 键名也相应为 `duration_ms`。 +> +> 注意与**采集窗口**区分:`analyzeCPUProfile(duration)`、`ChartOptions::duration` 里的 `duration` 是**秒**,而 `ProfilerState::duration` 是**毫秒**。两者同名不同单位。 --- @@ -210,7 +225,7 @@ void setLogSink(std::shared_ptr sink); **说明**: - 设置后,profiler 的所有日志将输出到自定义 sink -- 默认 sink 按级别分流:`Trace`/`Debug`/`Info` 写 **stdout**,`Warning`/`Error`/`Fatal` 写 **stderr**(`src/internal/default_log_sink.cpp:70-74`) +- 默认 sink 按级别分流:`Trace`/`Debug`/`Info` 写 **stdout**,`Warning`/`Error`/`Fatal` 写 **stderr**(`LogSink` 默认实现中的级别判断) - 设置 `nullptr` 可恢复默认行为 **示例**: @@ -300,7 +315,10 @@ std::string analyzeCPUProfile(int duration, const std::string& output_type = "fl **参数**: - `duration`: 采样时长(秒) -- `output_type`: 输出类型("flamegraph" 或 "pprof") +- `output_type`: 渲染方式(`"flamegraph"` 或 `"pprof"`) + +> 这是 **C++ 层**的形参。HTTP 层的等价参数叫 `renderer`,取值 `flamegraph` / `callgraph` +> (见 README 的端点说明)——`callgraph` 对应这里的 `"pprof"`。 **返回值**: SVG 字符串 @@ -325,7 +343,7 @@ std::string getRawCPUProfile(int seconds); 宿主用 `startCPUProfiler()` 打开的会话**归调用方所有,不会被本函数停止或接管**。调用前可用 `isProfilerRunning(ProfilerType::CPU)` 判断;HTTP 层则通过 `isCpuProfilerBusy()` 区分并返回 -`500`(`/pprof/profile`,Go 风格)或 `409`(`/api/cpu/*`)。 +`500`(`/pprof/profile`,Go 风格)或 `409`(`/api/pprof/cpu`)。 --- @@ -358,17 +376,22 @@ bool stopHeapProfiler(); 采集 Heap 并生成图表 SVG。 ```cpp -std::string analyzeHeapProfile(const std::string& output_type = "flamegraph"); +std::string analyzeHeapProfile(int duration = 1, const std::string& output_type = "flamegraph"); ``` **参数**: -- `output_type`: 输出类型(`"flamegraph"` 或 `"pprof"`) +- `duration`: 采集窗口(秒),默认 **1**,内部钳制到 1..300 +- `output_type`: 渲染方式(`"flamegraph"` 或 `"pprof"`) **返回值**: SVG 字符串;失败时返回 `{"error":"..."}` JSON 字符串 -**说明**: **没有 duration 参数**。gperftools 的 heap profiling 是按分配驱动的:`HeapProfilerStart()` -开始记录,`HeapProfilerDump()` 写出快照,与经过的时间无关。实现内部只开启一个固定的采样窗口(1 秒), -然后 dump。 +**说明**: `duration` 是**采集窗口**,不是采样率。gperftools 的 heap profiling 按分配驱动: +`HeapProfilerStart()` 开始记录,`HeapProfilerDump()` 写出快照。窗口决定**覆盖多久的分配**, +采样率则由 `TCMALLOC_SAMPLE_PARAMETER` 在进程启动前固定,运行时不可调。 + +> ⚠️ **C++ 层默认 1 秒,HTTP 层默认 10 秒**。HTTP 层用更长默认值是实测结果:短窗口经常 +> 采不到足够的分配而渲染失败。直接调用此函数时,分配稀疏的进程请显式传入更大的 +> `duration`。 **记录的是"窗口内发生的分配",不是"当前堆里的内存"** —— 这是最容易误解的一点: @@ -418,13 +441,36 @@ std::string getRawHeapGrowthStacks(); ### getThreadCallStacks -获取完整线程调用堆栈。 +获取所有线程的调用堆栈,`/api/thread/stacks` 即调用它。 ```cpp std::string getThreadCallStacks(); ``` -**说明**: 逐个地址调用 `symbolizeAddress()` 做符号化,该函数**优先使用 Abseil**(`absl::Symbolize`),失败后再回退到 `resolveSymbolWithBackward()`(`src/profiler_manager.cpp:1112-1121`、`1311-1314`)。 +**输出格式**: + +``` +Thread Call Stacks (via Signal Handler) +========================================= + +Total threads captured: 2 + +Thread 1234 (DrogonIoLoop): + Frames: 13 + #0 profiler::v0_1_0::ProfilerManager::signalHandler() + #1 __restore_rt + ... + #5 epoll_wait ← 该线程阻塞在这里 +``` + +- **线程名**取自 `/proc//comm`(内核截断到 15 字符,与 `ps`/`top` 显示一致)。只有 tid + 时输出 `Thread 1234:`,读日志时几乎无法分辨是哪个线程,因此带上名字。 +- **阻塞点**是跳过信号捕获机制帧后的第一个真实帧(栈顶前几帧恒为 + `signalHandler`/`__restore_rt`/`__syscall_cancel_arch`,它们属于捕获手段而非线程状态)。 +- 正在执行本请求的线程**不会**出现在结果里:信号只能采集其他线程,执行中的线程无法被抓取。 + 这也是可接受的——正在运行的线程本来就没有"卡住"。 + +**说明**: 逐个地址调用 `symbolizeAddress()` 做符号化,该函数**优先使用 Abseil**(`absl::Symbolize`),失败后再回退到 `resolveSymbolWithBackward()`。 --- @@ -438,7 +484,7 @@ std::string getThreadCallStacks(); std::string resolveSymbolWithBackward(void* address); ``` -**说明**: 实际的符号化链及顺序为(`src/symbolize.cpp:30`、`42`、`66`): +**说明**: 实际的符号化链及顺序为(`src/symbolize.cpp` 的 `symbolize()` 内,按顺序尝试): 1. `absl::Symbolize` — 最可靠,命中即返回函数名(源文件记为 `??`、行号为 0) 2. `dladdr` + `abi::__cxa_demangle` — 回退,可拿到所在模块名 @@ -485,7 +531,7 @@ std::string getExecutablePath(); static void setStackCaptureSignal(int signal); ``` -**说明**: 默认使用 `SIGUSR1`。推荐在第一次捕获线程栈之前调用;若处理器已安装后再调用,实现会先恢复旧处理器并打印一条 `[WARN]` 日志,然后切换(`src/profiler_manager.cpp:112-128`)。 +**说明**: 默认使用 `SIGUSR1`。推荐在第一次捕获线程栈之前调用;若处理器已安装后再调用,实现会先恢复旧处理器并打印一条 `[WARN]` 日志,然后切换(`ProfilerManager::setStackCaptureSignal()`)。 ```cpp profiler::ProfilerManager::setStackCaptureSignal(SIGUSR2); @@ -551,7 +597,7 @@ int main() { 它会同时考虑"已有请求在采样"和"宿主占用了会话"两种情况。 Heap 侧同理:`analyzeHeapProfile()` 与 `startHeapProfiler()` 互斥,谁先占谁赢。 -`/api/growth/analyze` 不占用任何会话,不受限制。 +`/api/pprof/growth` 不占用任何会话,不受限制。 --- @@ -559,7 +605,7 @@ Heap 侧同理:`analyzeHeapProfile()` 与 `startHeapProfiler()` 互斥,谁 - 大多数控制类方法(`start*` / `stop*`)返回 `bool` 表示成功/失败 - 原始数据获取方法(`getRawCPUProfile`、`getRawHeapSample`、`getRawHeapGrowthStacks`、`getThreadCallStacks`)在失败时返回**空字符串** -- ⚠️ 但 `analyzeCPUProfile()` / `analyzeHeapProfile()` 在失败时返回的是形如 `{"error":"..."}` 的 **JSON 字符串**,而非空串(`src/profiler_manager.cpp:427`、`469`、`475`、`491`、`520`)。HTTP 层正是靠 `{"` 前缀识别它并转成 500(`src/http_handlers.cpp:69-71`)——自行调用这两个 API 时需要做同样的判断 +- ⚠️ 但 `analyzeCPUProfile()` / `analyzeHeapProfile()` 在失败时返回的是形如 `{"error":"..."}` 的 **JSON 字符串**,而非空串(`analyzeCPUProfile()` / `analyzeHeapProfile()` 的各个失败分支)。HTTP 层正是靠 `{"` 前缀识别它并转成 500(`profiler::internal::isJsonError()`,见 `src/internal/result_parsing.h`)——自行调用这两个 API 时需要做同样的判断 - `HandlerResponse::error()` 返回包含错误信息的 JSON 响应 --- diff --git a/docs/user_guide/03_integration_examples.md b/docs/user_guide/03_integration_examples.md index 4f1b7fe..54397ab 100644 --- a/docs/user_guide/03_integration_examples.md +++ b/docs/user_guide/03_integration_examples.md @@ -181,12 +181,14 @@ int main() { YourWebServer server; // 注册 CPU 分析端点 - server.route("GET", "/api/cpu/analyze", [&](const Request& req) { - int duration = req.get_param_int("duration", 10); - std::string output_type = req.get_param("output_type", "pprof"); + server.route("GET", "/api/pprof/cpu", [&](const Request& req) { + profiler::ChartOptions options; + options.duration = req.get_param_int("duration", 10); + options.renderer = profiler::parseChartRenderer(req.get_param("renderer", "flamegraph")); + options.inline_display = req.get_param("output", "inline") != "attachment"; // 调用 handler 获取框架无关的响应 - profiler::HandlerResponse resp = handlers.handleCpuAnalyze(duration, output_type); + profiler::HandlerResponse resp = handlers.handleCpuChart(options); // 用你的框架包装响应 return YourResponse() @@ -255,10 +257,15 @@ public: return toOat(handlers_.handleStatus()); } - ENDPOINT("GET", "/api/cpu/analyze", cpuAnalyze, + ENDPOINT("GET", "/api/pprof/cpu", cpuChart, QUERY(Int32, duration, "duration", "10"), - QUERY(String, output_type, "output_type", "flamegraph")) { - return toOat(handlers_.handleCpuAnalyze(duration, output_type)); + QUERY(String, renderer, "renderer", "flamegraph"), + QUERY(String, output, "output", "inline")) { + profiler::ChartOptions options; + options.duration = duration; + options.renderer = profiler::parseChartRenderer(renderer); + options.inline_display = output != "attachment"; + return toOat(handlers_.handleCpuChart(options)); } ENDPOINT("GET", "/pprof/profile", pprofProfile, diff --git a/docs/user_guide/04_troubleshooting.md b/docs/user_guide/04_troubleshooting.md index 2e51e50..f0aa501 100644 --- a/docs/user_guide/04_troubleshooting.md +++ b/docs/user_guide/04_troubleshooting.md @@ -707,7 +707,7 @@ int main() { ## 常见问题 FAQ ### Q: 支持 macOS / Windows 吗? -**A**: 不支持。实现依赖 Linux 专有接口:`/proc/self/exe` 与 `/proc/self/task`(`src/profiler_manager.cpp:288-297`、`932-939`)、`sigaction`/`ucontext` 信号栈捕获。WSL2 可用。 +**A**: 不支持。实现依赖 Linux 专有接口:`/proc/self/exe` 与 `/proc/self/task`(前者用于 `getExecutablePath()`,后者用于 `captureAllThreadStacks()`)、`sigaction`/`ucontext` 信号栈捕获。WSL2 可用。 ### Q: 可以在生产环境使用吗? **A**: 当前版本是 v0.x.x(开发阶段),不建议用于生产环境。等待 v1.0.0 稳定版。 @@ -718,16 +718,20 @@ int main() { ### Q: 可以同时进行 CPU 和 Heap profiling 吗? **A**: CPU 与 Heap 是两套独立的 profiling 状态,可以并存。但**同一时刻只能有一个 CPU 采样会话**:`analyzeCPUProfile()` / `getRawCPUProfile()` 会先原子认领会话,认领失败即拒绝,**不会**打断正在采样的另一方(包括宿主用 `startCPUProfiler()` 开的会话)。 -### Q: `/api/heap/analyze` 为什么不接受 `duration` 参数? -**A**: 因为对 heap profiling 而言时长没有意义。gperftools 的 heap profiling 是**按分配驱动**的:`HeapProfilerStart()` 开始记录、`HeapProfilerDump()` 写出快照,采样率由**进程启动时**的 `TCMALLOC_SAMPLE_PARAMETER` 决定,与运行时长无关。 +### Q: Heap 窗口式采集的 `duration` 是什么意思? +**A**: 它决定**采集多久**(覆盖度),不是**采样率**(精度)。gperftools 的 heap profiling 按分配驱动: +`HeapProfilerStart()` 开始记录、`HeapProfilerDump()` 写出快照;采样率由**进程启动时**的 +`TCMALLOC_SAMPLE_PARAMETER` 决定,运行时不可调。 -该接口内部只开一个固定的 1 秒窗口让应用自身的分配被记录,然后 dump。要提高采样精度,请**在启动前**设置 `TCMALLOC_SAMPLE_PARAMETER=524288`(默认 `0` 表示不采样,此时快照会明显偏稀疏)。 +所以 `duration` 越长,覆盖到的分配越多——分配稀疏的进程需要更长窗口才采得到东西。HTTP 层默认 +**10 秒**(实测 3 秒窗口失败率 35–45%)。要提高**采样精度**则必须**在启动前**设置 +`TCMALLOC_SAMPLE_PARAMETER=524288`(默认 `0` 表示不采样,此时快照会明显偏稀疏)。 ### Q: heap 图是空的 / 报 "No heap profile data was produced"? -**A**: 说明这 1 秒窗口内进程**没有发生内存分配**。heap profiler 记录的是**窗口内的分配事件**,不是当前堆的内容,所以不会凭空造出数据。请确保被分析进程正处于负载中,或在窗口内制造真实分配后再试。 +**A**: 说明该采集窗口内进程**没有发生内存分配**。heap profiler 记录的是**窗口内的分配事件**,不是当前堆的内容,所以不会凭空造出数据。请确保被分析进程正处于负载中,或在窗口内制造真实分配后再试。 ### Q: 进程内存很高,为什么 heap 图却是空的? -**A**: 因为 `/api/heap/analyze`(以及 Web 面板上的 Heap 按钮)是**窗口式**的:只记录采样窗口内**新发生**的分配。如果内存是在采样开始前分配的、窗口内已经停止分配,结果就是空的。 +**A**: `/api/pprof/heap` 是**窗口式**的:只记录采集窗口内**新发生**的分配(流量),不反映此刻堆里的存量。`duration` 决定采集多久,默认 10 秒——实测 3 秒窗口有 35–45% 的请求采不到样本而失败。如果内存是在采样开始前分配的、窗口内已停止分配,结果就是空的。 想看**当前堆的累计快照**,请改用**状态式**的 `/pprof/heap`: @@ -741,7 +745,7 @@ go tool pprof -http=:8081 ./your_app heap.prof 两者语义相反、不可互相替代;详见 [README 的说明](../../README.md#heap-的两个端点语义相反)。 ### Q: 为什么长时间采样期间,其它接口还能正常返回? -**A**: 因为 Drogon 适配层不把 profiling 放在事件循环线程上执行。所有会产生图表的接口(`/pprof/profile`、`/api/*/analyze`、`/api/*/svg_raw`、`/api/*/flamegraph_raw`)都被投递到一个后台工作线程,完成后用 `queueInLoop()` 把响应送回事件循环。因此一个 300 秒的采样不会拖住 `/api/status` 或首页。 +**A**: 因为 Drogon 适配层不把 profiling 放在事件循环线程上执行。所有会产生图表的接口(`/pprof/profile`、`/api/pprof/{cpu,heap,growth}`、`/api/pprof/heap/snapshot`)都被投递到一个后台工作线程,完成后用 `queueInLoop()` 把响应送回事件循环。因此一个 300 秒的采样不会拖住 `/api/status` 或首页。 ### Q: 并发请求 CPU 采样时返回 409 / 500 "cpu profiling already in use" **A**: 这是**有意的拒绝**,不是故障。CPU 采样是独占的(gperftools 的采样会话是进程级全局状态),所以已有采样进行时,第二个请求会立即失败而**不会排队等待**——避免用户以为"已经在采样了",实际却要排在别人后面等几十秒到几分钟。 @@ -749,17 +753,41 @@ go tool pprof -http=:8081 ./your_app heap.prof 具体响应与 Go 的 `net/http/pprof` 对齐: - `/pprof/profile` → `500` + `text/plain` + `Could not enable CPU profiling: cpu profiling already in use`,并带 `X-Go-Pprof: 1`(`go tool pprof` 靠它识别这是错误而非 profile 数据) -- `/api/cpu/*` → `409` + `{"error":"cpu profiling already in use"}` +- `/api/pprof/cpu` → `409` + `{"error":"cpu profiling already in use"}` 稍后重试即可。**正在进行的那个采样不受影响**,会正常完成。 -Heap 分析(`/api/heap/analyze`)出于同样的原因也独占,并发时返回 `409` + `{"error":"heap profiling already in use"}`。注意 `HeapProfilerStart()` 与 CPU profiler 不同——它**没有失败模式**,重复调用会静默替换输出前缀,所以必须在调用之前原子认领,而不是"先查询再启动"。`/api/growth/analyze` 不占用会话,不受限制。 +Heap 分析(`/api/pprof/heap`)出于同样的原因也独占,并发时返回 `409` + `{"error":"heap profiling already in use"}`。注意 `HeapProfilerStart()` 与 CPU profiler 不同——它**没有失败模式**,重复调用会静默替换输出前缀,所以必须在调用之前原子认领,而不是"先查询再启动"。`/api/pprof/growth` 不占用会话,不受限制。 + +### Q: 接口返回 500 "Failed to get heap sample" / "heap sampling is off" +**A**: 状态式 heap 端点(`/pprof/heap`、`/api/pprof/heap/snapshot`)依赖 `TCMALLOC_SAMPLE_PARAMETER`,而它**必须在进程启动前**设置(默认 `0` 表示关闭采样): + +```bash +export TCMALLOC_SAMPLE_PARAMETER=524288 +./your_app +``` + +窗口式端点(`/api/pprof/heap`)自己运行采集窗口,不受影响。 + +> 该变量缺失时,所有状态式端点现在都返回带原因的 500(此前是三种不同表现:200 + `%warn` 文本、500、200 + 空图)。 + +### Q: 接口返回 500 "No stack samples to render (profile is empty)" +**A**: 采样窗口内没有收集到足够的栈样本,`flamegraph.pl` 拒绝渲染。常见原因是采样时长太短,而目标进程在那段时间里正好空闲。加长 `duration`(例如 10–30 秒)或在有负载时重试。 + +> 以前这种情况会返回 **200** 和一张只含 `ERROR: No valid input provided to flamegraph.pl.` 文字的 SVG,看起来像成功。现已改为带原因的 500。 ### Q: 接口返回 500 "pprof did not generate valid SVG" **A**: 内置 pprof 脚本用 `dot`(graphviz)渲染,当 profile 采样点太少或数据异常时 `dot` 会失败并输出错误文本,因而被判定为"非 SVG"。请:① 安装 graphviz;② 加长采样时长(`?duration=30`);③ 确认编译带 `-g`。这是已知的**既有缺陷**(错误信息未把 `dot` 的原始输出透出),记录在 [ROADMAP](../../ROADMAP.md)。 -### Q: 为什么 `?output_type=flamegraph` 才是火焰图? -**A**: C++ API 的形参默认值是 `"flamegraph"`,但 HTTP 路由在未显式传参时默认 `"pprof"`,返回的是 pprof 脚本渲染的图形而非 FlameGraph 火焰图。想稳定拿到火焰图请显式指定 `output_type=flamegraph`。 +### Q: `renderer=flamegraph` 和 `renderer=callgraph` 有什么区别? +**A**: 是**两种不同的图形**,不是同一份数据换画法: + +| 取值 | 产出 | +|------|------| +| `flamegraph`(默认) | **火焰图**:栈帧堆叠 | +| `callgraph` | **调用图**:graphviz 节点/边,能解析内联帧,符号化更完整 | + +> ⚠️ 两者都**不能**在浏览器里缩放——产物内嵌的 pan/zoom 脚本依赖的元素/初始化挂钩并未出现(实测)。需要缩放请下载后用桌面工具打开。 ### Q: Profile 文件很大怎么办? **A**: 可以压缩后交给 pprof(`pprof` 支持从 stdin 读取): diff --git a/example/main.cpp b/example/main.cpp index 59c232a..ad23e0c 100644 --- a/example/main.cpp +++ b/example/main.cpp @@ -2,7 +2,11 @@ #include "profiler_manager.h" #include "workload.h" #include +#include #include +#include +#include +#include #include #ifdef REMOTE_PROFILER_ENABLE_WEB @@ -19,16 +23,54 @@ int main([[maybe_unused]] int argc, [[maybe_unused]] char* argv[]) { std::cout << "Starting HTTP server on " << host << ":" << port << "\n"; std::cout << "Open your browser and visit: http://localhost:" << port << "\n\n"; - // Start background workload thread - std::thread worker([]() { - while (true) { - cpuIntensiveTask(); - memoryIntensiveTask(); - std::this_thread::sleep_for(std::chrono::seconds(5)); - } - }); + // Give the sample threads recognisable names so /api/thread/stacks can be + // checked against what the kernel reports (the endpoint reads + // /proc//comm, the same source ps and top use). The names below are + // deliberately unmistakable: if they show up in the endpoint output, the + // naming works; if they do not, it does not. + auto nameCurrentThread = [](const std::string& name) { + // Linux truncates thread names to 15 characters plus the terminator. + pthread_setname_np(pthread_self(), name.substr(0, 15).c_str()); + }; + + // Busy thread: alternately burns CPU and allocates, then idles 5s. + std::thread worker( + [](auto setName) { + setName("profiler-worker"); + while (true) { + cpuIntensiveTask(); + memoryIntensiveTask(); + std::this_thread::sleep_for(std::chrono::seconds(5)); + } + }, + nameCurrentThread); worker.detach(); + // A second thread that mostly sleeps, so the output shows a thread blocked in + // a timer wait next to the busy one. + std::thread sleeper( + [](auto setName) { + setName("profiler-sleeper"); + while (true) { + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + } + }, + nameCurrentThread); + sleeper.detach(); + + // A third thread parked on a condition variable forever: its stack top should + // read as a futex wait, which is the clearest "stuck here" signal. + std::thread parked( + [](auto setName) { + setName("profiler-parked"); + std::mutex mutex; + std::condition_variable never_signalled; + std::unique_lock lock(mutex); + never_signalled.wait(lock, []() { return false; }); + }, + nameCurrentThread); + parked.detach(); + // Create ProfilerManager instance (no longer a singleton) profiler::ProfilerManager profiler; diff --git a/include/profiler/http_handlers.h b/include/profiler/http_handlers.h index d1d730a..2d6dda1 100644 --- a/include/profiler/http_handlers.h +++ b/include/profiler/http_handlers.h @@ -42,6 +42,38 @@ struct HandlerResponse { } }; +/// @brief Which graphical form a chart endpoint should produce +/// +/// These are two genuinely different representations, not two styles of one: +/// a flame graph is a stack of frames, a call graph is a node/edge diagram. +enum class ChartRenderer { + /// Stacked frames (FlameGraph). Self-contained SVG; the classic choice. + FlameGraph, + /// Node/edge call graph produced by the pprof script via graphviz dot. + /// Carries better symbolisation (it resolves inline frames). + CallGraph, +}; + +/// @brief Options shared by the /api/pprof/{cpu,heap,growth} analysis endpoints +struct ChartOptions { + ChartRenderer renderer = ChartRenderer::FlameGraph; + /// Collection window in seconds, clamped to 1..300. Note this is how long + /// allocations/samples are *collected*, not a sampling rate. + int duration = 10; + /// true -> omit Content-Disposition so a browser renders the SVG in place + /// false -> send `Content-Disposition: attachment` to force a download + bool inline_display = true; +}; + +/// @brief Parse the `renderer` query value; unknown values fall back to FlameGraph +ChartRenderer parseChartRenderer(const std::string& value); + +/// @brief Clamp a collection window to the supported 1..300 second range +/// +/// Applied once at the HTTP boundary so every endpoint treats an out-of-range +/// duration the same way, regardless of which profiler backend serves it. +int clampChartDuration(int duration); + /// @brief Framework-agnostic profiler HTTP endpoint handlers /// /// Usage example with any framework: @@ -60,55 +92,47 @@ class ProfilerHttpHandlers { // --- Status --- HandlerResponse handleStatus(); - // --- CPU endpoints --- - /// @brief Whether a CPU profiling session is already claimed - /// - /// CPU profiling is exclusive: gperftools keeps one process-global sampling - /// session, so at most one request may sample at a time. + // --- Exclusivity probes --- + // + // Both are exposed so a web adapter can refuse a request on the event-loop + // thread — before queueing it — using the same wording the handlers produce. + /// @brief Whether a CPU sampling session is claimed (by a request or the host) bool isCpuProfilerBusy() const; - /// @brief The response to use when @ref isCpuProfilerBusy is true - /// - /// Provided so a web adapter can refuse a request on the event-loop thread — - /// before it is queued for execution — using the same wording and status the - /// handlers themselves produce. Calling it only makes sense while - /// @ref isCpuProfilerBusy is true. - /// - /// `pprof_style` selects the shape: - /// - true — the shape Go's net/http/pprof uses on /pprof/profile: HTTP 500, - /// `text/plain`, plus an `X-Go-Pprof: 1` marker so `go tool pprof` - /// reports the message instead of parsing it as profile data - /// - false — HTTP 409 with a JSON error, for the custom /api/* endpoints + /// @brief Whether a heap analysis is running, or the host opened a heap session + bool isHeapAnalyzerBusy() const; + + /// @brief The response for a CPU request that arrived while the profiler is busy + /// @param pprof_style true -> Go's /pprof/profile shape (500 + text/plain + + /// `X-Go-Pprof: 1`); false -> 409 with a JSON error for /api/* routes HandlerResponse cpuProfilerBusyResponse(bool pprof_style) const; - /// @brief Whether a heap analysis is already running + /// @brief The response for a heap analysis that arrived while one is running + HandlerResponse heapAnalyzerBusyResponse() const; + + // --- Analysis endpoints: one entry point per profiler --- + /// @brief Sample CPU for the requested window and render a chart /// - /// Heap analysis reconfigures the single process-global heap profiler, so - /// concurrent calls are exclusive for the same reason CPU profiling is. - bool isHeapAnalyzerBusy() const; + /// Exclusive: refuses while another sampling session is active (409) rather + /// than queueing or preempting it. + HandlerResponse handleCpuChart(const ChartOptions& options); + + /// @brief Sample heap for the requested window and render a chart + /// @note Exclusive in the same way CPU is. + HandlerResponse handleHeapChart(const ChartOptions& options); - /// @brief The response to use when @ref isHeapAnalyzerBusy is true + /// @brief Render the current heap snapshot (state-based, not a window) /// - /// Mirrors @ref cpuProfilerBusyResponse but for heap analysis, which has no - /// Go pprof equivalent to match: HTTP 409 with a JSON error. - HandlerResponse heapAnalyzerBusyResponse() const; + /// Answers "what is in the heap now" rather than "what was allocated during + /// the window", so it needs TCMALLOC_SAMPLE_PARAMETER to have been set before + /// the process started. `renderer` is ignored when @p as_profile is true. + /// + /// @param options Chart options (renderer used only when @p as_profile is false) + /// @param as_profile Return the raw profile text instead of an SVG + HandlerResponse handleHeapSnapshot(const ChartOptions& options, bool as_profile); - HandlerResponse handleCpuAnalyze(int duration, const std::string& output_type); - HandlerResponse handleCpuSvgRaw(int duration); - HandlerResponse handleCpuFlamegraphRaw(int duration); - - // --- Heap endpoints --- - /// @note Heap analysis has no duration parameter: gperftools heap profiling - /// is allocation driven, so the sample rate is fixed at process start - /// by TCMALLOC_SAMPLE_PARAMETER rather than by elapsed time. - HandlerResponse handleHeapAnalyze(const std::string& output_type); - HandlerResponse handleHeapSvgRaw(); - HandlerResponse handleHeapFlamegraphRaw(); - - // --- Growth endpoints --- - HandlerResponse handleGrowthAnalyze(const std::string& output_type); - HandlerResponse handleGrowthSvgRaw(); - HandlerResponse handleGrowthFlamegraphRaw(); + /// @brief Sample heap growth and render a chart + HandlerResponse handleGrowthChart(const ChartOptions& options); // --- Standard pprof endpoints --- HandlerResponse handlePprofProfile(int seconds); diff --git a/include/profiler_manager.h b/include/profiler_manager.h index 7ca88ec..cd3d614 100644 --- a/include/profiler_manager.h +++ b/include/profiler_manager.h @@ -160,9 +160,14 @@ class ProfilerManager { /// driven, so the sample rate is governed by TCMALLOC_SAMPLE_PARAMETER /// (read once at process startup), not by elapsed time. /// + /// @param duration Collection window in seconds (default 1, clamped 1..300). + /// This is NOT a sampling rate: the rate is fixed by + /// TCMALLOC_SAMPLE_PARAMETER at process start. It controls how long + /// allocations are recorded, so a process that allocates sparsely + /// needs a longer window to produce anything. /// @param output_type Output graph type: "flamegraph" (default) or "pprof" /// @return SVG content as string, or a {"error":...} JSON string on failure - std::string analyzeHeapProfile(const std::string& output_type = "flamegraph"); + std::string analyzeHeapProfile(int duration = 1, const std::string& output_type = "flamegraph"); /// @brief Get raw CPU profile data (for /pprof/profile endpoint) /// @param seconds Sampling duration in seconds @@ -170,17 +175,32 @@ class ProfilerManager { std::string getRawCPUProfile(int seconds); /// @brief Get raw heap sample data (for /pprof/heap endpoint) + /// + /// Pulls tcmalloc's cumulative sample of the current heap. Requires + /// TCMALLOC_SAMPLE_PARAMETER to have been set before the process started; + /// returns "" when sampling is off. + /// /// @return Heap sample in text format (compatible with pprof) std::string getRawHeapSample(); + /// @brief Sample allocations for @p duration seconds and return the raw profile text + /// + /// Unlike getRawHeapSample() this needs no startup configuration: it runs its + /// own HeapProfilerStart/HeapProfilerDump/Stop window, so it works in a process + /// launched without TCMALLOC_SAMPLE_PARAMETER. Use this when the caller wants + /// to choose how long to collect (sparse allocators need longer) and then + /// render or analyse the text themselves. + /// + /// @param duration Collection window in seconds (clamped 1..300) + /// @return Profile text (pprof format), or a {"error":...} JSON string on failure + /// @note Exclusive, like analyzeHeapProfile(): rejects with "heap profiling + /// already in use" if another analysis or a host session is active + std::string getRawHeapProfileSample(int duration); + /// @brief Get heap growth stacks data (for /pprof/growth endpoint) /// @return Heap growth stacks in text format (compatible with pprof) std::string getRawHeapGrowthStacks(); - /// @brief Get all thread stacks (for /api/thread/stacks endpoint) - /// @return Thread stacks in text format - std::string getThreadStacks(); - /// @brief Get thread callstack with full backtrace using signal handler /// @return Thread callstack information std::string getThreadCallStacks(); diff --git a/lsan.supp b/lsan.supp index 2d31928..b80a034 100644 --- a/lsan.supp +++ b/lsan.supp @@ -6,3 +6,4 @@ # This is expected behavior and not a real memory leak. # See: https://github.com/gperftools/gperftools leak:MallocExtension::Initialize + diff --git a/scripts/check-sanitizers.sh b/scripts/check-sanitizers.sh new file mode 100755 index 0000000..e5f7215 --- /dev/null +++ b/scripts/check-sanitizers.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# Reproduce the CI sanitizer configurations locally. +# +# CI caught a LeakSanitizer failure in stopHeapProfiler() that local release +# builds never showed, because nothing here ran with -fsanitize=address. This +# script exists so that check is one command instead of a CI round trip. +# +# Usage: +# scripts/check-sanitizers.sh # ASan (the job that failed in CI) +# scripts/check-sanitizers.sh asan ubsan # pick jobs +# scripts/check-sanitizers.sh all +# +# Requires: cmake, a vcpkg checkout with the project's dependencies installed +# (same prerequisites as a normal build). + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +cd "$PROJECT_ROOT" + +VCPKG_TOOLCHAIN="$PROJECT_ROOT/vcpkg/scripts/buildsystems/vcpkg.cmake" +VCPKG_TRIPLET="${VCPKG_TARGET_TRIPLET:-x64-linux-release}" +BUILD_ROOT="${SANITIZER_BUILD_ROOT:-$PROJECT_ROOT/build/sanitizers}" +JOBS="$(nproc)" + +log() { printf '\n=== %s ===\n' "$*"; } + +# Extra ctest flags. Empty by default because CI runs ctest serially; set +# CTEST_JOBS= to also exercise the parallel case, which is where the tests +# used to clobber each other's profile files. +ctest_parallel=() +[[ -n "${CTEST_JOBS:-}" ]] && ctest_parallel=(-j "$CTEST_JOBS") +die() { printf 'error: %s\n' "$*" >&2; exit 1; } + +[[ -f "$VCPKG_TOOLCHAIN" ]] || die "vcpkg toolchain not found at $VCPKG_TOOLCHAIN" + +# Allow the pre-installed vcpkg tree to be reused instead of installing a +# manifest from scratch (matches how the local dev builds are run). +VCPKG_INSTALLED_DIR="$PROJECT_ROOT/vcpkg_installed" +extra_configure_args=() +[[ -d "$VCPKG_INSTALLED_DIR" ]] && extra_configure_args+=( + -DVCPKG_MANIFEST_MODE=OFF + "-DVCPKG_INSTALLED_DIR=$VCPKG_INSTALLED_DIR" +) + +# Mirrors the flag sets in .github/workflows/code-quality.yml +run_asan() { + log "ASan + LSan (mirrors the CI 'Address Sanitizer' job)" + local dir="$BUILD_ROOT/asan" + cmake -S . -B "$dir" \ + -DCMAKE_BUILD_TYPE=Debug \ + -DCMAKE_TOOLCHAIN_FILE="$VCPKG_TOOLCHAIN" \ + "-DVCPKG_TARGET_TRIPLET=$VCPKG_TRIPLET" \ + -DCMAKE_CXX_FLAGS="-fsanitize=address -fno-omit-frame-pointer -g" \ + -DCMAKE_C_FLAGS="-fsanitize=address -fno-omit-frame-pointer -g" \ + -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" \ + -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=address" \ + "${extra_configure_args[@]}" + cmake --build "$dir" -j"$JOBS" + # detect_leaks + halt_on_error: a leak must fail the run, as it does in CI. + ( cd "$dir" && ASAN_OPTIONS=detect_leaks=1:halt_on_error=1 \ + LSAN_OPTIONS="suppressions=$PROJECT_ROOT/lsan.supp" \ + ctest --output-on-failure "${ctest_parallel[@]}" ) +} + +run_ubsan() { + log "UBSan (mirrors the CI 'Undefined Behavior Sanitizer' job)" + local dir="$BUILD_ROOT/ubsan" + cmake -S . -B "$dir" \ + -DCMAKE_BUILD_TYPE=Debug \ + -DCMAKE_TOOLCHAIN_FILE="$VCPKG_TOOLCHAIN" \ + "-DVCPKG_TARGET_TRIPLET=$VCPKG_TRIPLET" \ + -DCMAKE_CXX_FLAGS="-fsanitize=undefined -fno-omit-frame-pointer -g" \ + -DCMAKE_C_FLAGS="-fsanitize=undefined -fno-omit-frame-pointer -g" \ + -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=undefined" \ + -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=undefined" \ + "${extra_configure_args[@]}" + cmake --build "$dir" -j"$JOBS" + # halt_on_error so a single UB report fails the run rather than scrolling by. + ( cd "$dir" && UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1 \ + ctest --output-on-failure "${ctest_parallel[@]}" ) +} + +run_tsan() { + log "TSan (mirrors the CI job; disabled there, kept here for manual runs)" + local dir="$BUILD_ROOT/tsan" + cmake -S . -B "$dir" \ + -DCMAKE_BUILD_TYPE=Debug \ + -DCMAKE_TOOLCHAIN_FILE="$VCPKG_TOOLCHAIN" \ + "-DVCPKG_TARGET_TRIPLET=$VCPKG_TRIPLET" \ + -DCMAKE_CXX_FLAGS="-fsanitize=thread -fno-omit-frame-pointer -g" \ + -DCMAKE_C_FLAGS="-fsanitize=thread -fno-omit-frame-pointer -g" \ + -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=thread" \ + -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=thread" \ + "${extra_configure_args[@]}" + cmake --build "$dir" -j"$JOBS" + ( cd "$dir" && TSAN_OPTIONS=halt_on_error=1 ctest --output-on-failure "${ctest_parallel[@]}" ) +} + +targets=("$@") +[[ ${#targets[@]} -eq 0 ]] && targets=(asan) +[[ "${targets[0]}" == "all" ]] && targets=(asan ubsan tsan) + +for t in "${targets[@]}"; do + case "$t" in + asan) run_asan ;; + ubsan) run_ubsan ;; + tsan) run_tsan ;; + *) die "unknown target '$t' (expected asan, ubsan, tsan or all)" ;; + esac +done + +log "All requested sanitizer runs passed" diff --git a/scripts/verify-web-ui.mjs b/scripts/verify-web-ui.mjs new file mode 100755 index 0000000..aa30fad --- /dev/null +++ b/scripts/verify-web-ui.mjs @@ -0,0 +1,302 @@ +#!/usr/bin/env node +// Drive the real control panel in headless Chrome and assert that each action +// actually does something. +// +// Why this exists: several front-end defects in this project were reported as +// "the button does nothing" *after* having been declared fixed on the strength of +// curl requests against the server. curl cannot execute page JavaScript, so it +// cannot see an undefined function, a button stuck disabled by a synchronous +// throw, or a download that never starts. This script checks the browser-side +// behaviour directly: +// +// - the page loads with no JavaScript errors +// - every action issues the request it should, to the path it should +// - downloads leave a file on disk +// - buttons return to their normal state afterwards +// +// Usage: +// scripts/verify-web-ui.mjs [base-url] +// +// Requires: a running profiler_example server and a local Chrome/Chromium plus +// puppeteer-core. If puppeteer-core is not resolvable, the script explains how to +// get it and exits 2 rather than silently skipping -- a skipped check here is +// what let the bugs through before. + +import fs from 'node:fs'; +import http from 'node:http'; +import os from 'node:os'; +import path from 'node:path'; +import { createRequire } from 'node:module'; + +const BASE = (process.argv[2] || 'http://localhost:8080').replace(/\/$/, ''); +const DOWNLOAD_DIR = fs.mkdtempSync(path.join(os.tmpdir(), 'verify-web-ui-')); + +// Resolve puppeteer-core from the usual places without requiring an install in +// this repository. +function loadPuppeteer() { + const require = createRequire(import.meta.url); + const candidates = ['puppeteer-core', 'puppeteer']; + for (const name of candidates) { + try { + return require(name); + } catch { + /* try the next */ + } + } + console.error( + 'puppeteer-core is required.\n' + + ' npm install puppeteer-core # in this repo, or\n' + + ' npm install --prefix /tmp puppeteer-core\n' + + 'and set NODE_PATH if it lives outside this tree.', + ); + process.exit(2); +} + +function findChrome() { + const candidates = [ + process.env.CHROME_PATH, + '/usr/bin/google-chrome', + '/usr/bin/google-chrome-stable', + '/usr/bin/chromium', + '/usr/bin/chromium-browser', + '/snap/bin/chromium', + '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', + ].filter(Boolean); + for (const c of candidates) { + if (fs.existsSync(c)) return c; + } + console.error('No Chrome/Chromium found. Set CHROME_PATH to its executable.'); + process.exit(2); +} + +async function serverReachable() { + return new Promise(resolve => { + const req = http.get(`${BASE}/api/status`, res => { + res.resume(); + resolve(res.statusCode === 200); + }); + req.on('error', () => resolve(false)); + req.setTimeout(5000, () => { + req.destroy(); + resolve(false); + }); + }); +} + +const puppeteer = loadPuppeteer(); + +if (!(await serverReachable())) { + console.error(`No profiler server answering at ${BASE}/api/status.\n` + + 'Start one first, e.g. TCMALLOC_SAMPLE_PARAMETER=524288 ./build/profiler_example'); + process.exit(2); +} + +const results = []; +function record(name, ok, detail = '') { + results.push({ name, ok, detail }); + console.log(` ${ok ? 'PASS' : 'FAIL'} ${name}${detail ? ` -- ${detail}` : ''}`); +} + +const browser = await puppeteer.launch({ + executablePath: findChrome(), + headless: true, + protocolTimeout: 180000, + args: ['--no-sandbox', '--disable-dev-shm-usage'], +}); + +try { + const page = await browser.newPage(); + const cdp = await page.createCDPSession(); + await cdp.send('Browser.setDownloadBehavior', { + behavior: 'allow', + downloadPath: DOWNLOAD_DIR, + eventsEnabled: true, + }); + + const pageErrors = []; + const requests = []; + page.on('pageerror', e => pageErrors.push(e.message)); + page.on('request', r => { + const u = r.url(); + if (u.includes('/api/')) requests.push(u); + }); + + console.log(`\nLoading ${BASE}/`); + await page.goto(`${BASE}/`, { waitUntil: 'networkidle2', timeout: 30000 }); + + record('page loads without JavaScript errors', pageErrors.length === 0, pageErrors.join(' | ')); + + // The panel must not reference endpoints that no longer exist. + const staleRefs = await page.evaluate(() => { + const html = document.documentElement.innerHTML; + const stale = []; + for (const p of ['/api/cpu/', '/api/heap/', '/api/growth/', 'output_type', + 'show_svg.html', 'show_heap_svg.html', 'show_growth_svg.html']) { + if (html.includes(p)) stale.push(p); + } + return stale; + }); + record('panel references no removed endpoints or parameters', + staleRefs.length === 0, staleRefs.join(', ')); + + const openButtons = await page.evaluate(() => + document.querySelectorAll('button[onclick^="openChart"], button[onclick^="openSnapshot"]').length); + record('panel exposes the inline (open in place) actions', openButtons === 4, `found ${openButtons}`); + + // Every chart-producing section owns its renderer selector. The snapshot + // section previously read the Heap Profiler's dropdown, so which picture you + // got depended on a control in a different section. + const selectors = await page.evaluate(() => ({ + cpu: !!document.getElementById('cpu-renderer'), + heap: !!document.getElementById('heap-renderer'), + growth: !!document.getElementById('growth-renderer'), + snapshot: !!document.getElementById('snapshot-output'), + })); + record('each section has its own renderer selector', Object.values(selectors).every(Boolean), + JSON.stringify(selectors)); + + const controls = await page.evaluate(() => ({ + cpuDuration: !!document.getElementById('cpu-duration'), + cpuRenderer: !!document.getElementById('cpu-renderer'), + heapDuration: !!document.getElementById('heap-duration'), + heapRenderer: !!document.getElementById('heap-renderer'), + growthRenderer: !!document.getElementById('growth-renderer'), + snapshotOutput: !!document.getElementById('snapshot-output'), + snapshotButton: !!document.getElementById('snapshot-btn'), + })); + record('all chart controls present', Object.values(controls).every(Boolean), + JSON.stringify(controls)); + + async function waitForFileCount(before, timeoutMs = 60000) { + const deadline = Date.now() + timeoutMs; + const read = () => fs.readdirSync(DOWNLOAD_DIR).filter(f => !f.endsWith('.crdownload')); + while (Date.now() < deadline) { + const files = read(); + if (files.length > before) return files; + await new Promise(r => setTimeout(r, 250)); + } + return read(); + } + + async function checkDownload(name, clickExpr) { + // Track the set of filenames, not just the count: a download that reuses an + // existing name overwrites it, leaving the count unchanged. That is exactly + // how a second-precision filename timestamp hid a real defect -- the + // call-graph snapshot silently replaced the flame-graph one, and a + // count-based check reported success. + const before = new Set(fs.readdirSync(DOWNLOAD_DIR).filter(f => !f.endsWith('.crdownload'))); + requests.length = 0; + await page.evaluate(clickExpr); + const files = await waitForFileCount(before.size); + const added = files.filter(f => !before.has(f)); + const saved = added.length > 0; + const disabled = await page.$$eval('button', bs => + bs.filter(b => b.disabled).map(b => b.textContent.trim())); + const logText = await page.$eval('#output', el => el.textContent.replace(/\s+/g, ' ').slice(0, 90)); + record(`${name}: request issued`, requests.length > 0, requests.join(', ')); + record(`${name}: file saved`, saved, + saved ? `added ${added.join(', ')}` : `no new filename (files=${files.length})`); + record(`${name}: no button left disabled`, disabled.length === 0, disabled.join(' | ')); + if (!saved) console.log(` log panel: ${logText}`); + } + + console.log('\nCPU chart (flamegraph)'); + await checkDownload('cpu chart', () => downloadChart('cpu')); + + console.log('\nCPU chart (callgraph)'); + await page.select('#cpu-renderer', 'callgraph'); + await checkDownload('cpu callgraph', () => downloadChart('cpu')); + + console.log('\nHeap chart'); + await checkDownload('heap chart', () => downloadChart('heap')); + + console.log('\nHeap growth chart'); + await checkDownload('growth chart', () => downloadChart('growth')); + + // The snapshot section is one dropdown (raw text / flame graph / call graph) + // plus one download button, so each option must produce a distinct file. + for (const output of ['profile', 'flamegraph', 'callgraph']) { + console.log(`\nHeap snapshot (${output})`); + await page.select('#snapshot-output', output); + await checkDownload(`snapshot ${output}`, () => downloadSnapshot()); + } + + // The delivery modes must actually differ in the response, not just in the + // query string: `inline` omits Content-Disposition so the browser renders the + // SVG as a document, `attachment` sets it so the browser downloads. + console.log('\nDelivery modes'); + const modes = await page.evaluate(async () => { + const out = {}; + for (const mode of ['inline', 'attachment']) { + const r = await fetch(`/api/pprof/cpu?duration=10&renderer=flamegraph&output=${mode}`); + out[mode] = { + status: r.status, + contentType: r.headers.get('content-type') || '', + disposition: r.headers.get('content-disposition') || '', + bytes: (await r.blob()).size, + }; + } + return out; + }); + record('inline returns SVG with no Content-Disposition', + modes.inline.status === 200 && + modes.inline.contentType.includes('image/svg+xml') && + modes.inline.disposition === '', + JSON.stringify(modes.inline)); + record('attachment sets Content-Disposition', + modes.attachment.status === 200 && modes.attachment.disposition.startsWith('attachment'), + JSON.stringify(modes.attachment)); + record('both modes return the same kind of payload', + modes.inline.bytes > 1000 && modes.attachment.bytes > 1000, + `inline=${modes.inline.bytes}B attachment=${modes.attachment.bytes}B`); + + // The snapshot endpoint must honour renderer too, for both values. + console.log('\nHeap snapshot renderers'); + const snap = await page.evaluate(async () => { + const out = {}; + for (const renderer of ['flamegraph', 'callgraph']) { + const r = await fetch(`/api/pprof/heap/snapshot?format=svg&renderer=${renderer}`); + const body = await r.text(); + out[renderer] = { + status: r.status, + rects: (body.match(/ 0 && snap.flamegraph.polygons === 0, + JSON.stringify(snap.flamegraph)); + record('snapshot renders a call graph', + snap.callgraph.status === 200 && snap.callgraph.polygons > 0 && snap.callgraph.rects === 0, + JSON.stringify(snap.callgraph)); + + record('no JavaScript errors during the run', pageErrors.length === 0, pageErrors.join(' | ')); + + const savedFiles = fs.readdirSync(DOWNLOAD_DIR); + console.log(`\nFiles saved to ${DOWNLOAD_DIR}:`); + for (const f of savedFiles) { + console.log(` ${f} (${fs.statSync(path.join(DOWNLOAD_DIR, f)).size} bytes)`); + } + const emptyFiles = savedFiles.filter(f => fs.statSync(path.join(DOWNLOAD_DIR, f)).size === 0); + record('every saved file is non-empty', emptyFiles.length === 0, emptyFiles.join(', ')); + + // Seven downloads are exercised -- cpu flamegraph, cpu callgraph, heap, + // growth, and the three snapshot outputs -- so fewer files means two of them + // ended up with the same name and one overwrote the other. + const EXPECTED_DOWNLOADS = 7; + record('each download produced its own file', + savedFiles.length >= EXPECTED_DOWNLOADS, + `${savedFiles.length} files (expected ${EXPECTED_DOWNLOADS})`); +} finally { + await browser.close(); +} + +const failed = results.filter(r => !r.ok); +console.log(`\n${results.length - failed.length}/${results.length} checks passed`); +if (failed.length) { + console.log('Failed checks:'); + for (const f of failed) console.log(` - ${f.name}${f.detail ? `: ${f.detail}` : ''}`); + process.exit(1); +} diff --git a/src/drogon_adapter.cpp b/src/drogon_adapter.cpp index bc6ee3d..269c2d9 100644 --- a/src/drogon_adapter.cpp +++ b/src/drogon_adapter.cpp @@ -22,8 +22,11 @@ static void sendResponse(const HandlerResponse& hr, std::functionsetContentTypeCode(drogon::CT_TEXT_HTML); } else if (hr.content_type == "application/json") { resp->setContentTypeCode(drogon::CT_APPLICATION_JSON); - } else if (hr.content_type == "image/svg+xml" || hr.content_type == "text/xml") { - resp->setContentTypeCode(drogon::CT_TEXT_XML); + } else if (hr.content_type == "image/svg+xml") { + // CT_IMAGE_SVG_XML, not CT_TEXT_XML: the latter is the generic XML type. + // The SVG type is what tells the browser to treat the body as an image + // document (and what makes "save image as" work as an SVG). + resp->setContentTypeCode(drogon::CT_IMAGE_SVG_XML); } else if (hr.content_type == "application/octet-stream") { resp->setContentTypeCode(drogon::CT_APPLICATION_OCTET_STREAM); } else { @@ -99,6 +102,43 @@ void runAsync(const std::shared_ptr& executor, const drogon::Http } } +/// Read the analysis endpoints' query parameters into a ChartOptions. +/// +/// Defaults are deliberate: `duration` 10s because short windows collect too few +/// samples to render (measured: 3s failed 35-45% of the time, 10s never), and +/// `inline` so a plain link shows the SVG instead of downloading it. +ChartOptions chartOptionsFrom(const drogon::HttpRequestPtr& req) { + ChartOptions options; + + options.renderer = parseChartRenderer(req->getParameter("renderer")); + + auto d = req->getParameter("duration"); + if (!d.empty()) { + try { + options.duration = std::stoi(d); + } catch (...) {} + } + + if (req->getParameter("output") == "attachment") + options.inline_display = false; + + return options; +} + +/// Whether a CPU request should be refused, and with what. +std::optional cpuBusyRejection(const ProfilerHttpHandlers& handlers) { + if (handlers.isCpuProfilerBusy()) + return handlers.cpuProfilerBusyResponse(/*pprof_style=*/false); + return std::nullopt; +} + +/// Whether a heap analysis request should be refused, and with what. +std::optional heapBusyRejection(const ProfilerHttpHandlers& handlers) { + if (handlers.isHeapAnalyzerBusy()) + return handlers.heapAnalyzerBusyResponse(); + return std::nullopt; +} + } // namespace void registerDrogonHandlers(profiler::ProfilerManager& profiler) { @@ -109,225 +149,116 @@ void registerDrogonHandlers(profiler::ProfilerManager& profiler) { // profiling keeps its session in process-global state. auto executor = std::make_shared(); - // --- GET routes --- - auto registerGet = [&](const std::string& path, auto fn) { + // --------------------------------------------------------------------- + // Analysis endpoints: /api/pprof/{cpu,heap,growth} + // + // Each takes renderer / duration / output. The exclusivity verdict is + // evaluated here, on the event-loop thread before the job is queued -- a + // check performed inside the job runs only after the previous job finished, + // so it could never observe that job still running. + // --------------------------------------------------------------------- + auto registerChart = [&](const std::string& path, HandlerResponse (ProfilerHttpHandlers::*fn)(const ChartOptions&), + std::optional (*precheck)(const ProfilerHttpHandlers&)) { drogon::app().registerHandler( path, - [handlers, fn = std::move(fn)]([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(((*handlers).*fn)(), std::move(callback)); + [handlers, executor, fn, precheck](const drogon::HttpRequestPtr& req, + std::function&& callback) { + if (precheck) { + if (auto rejection = precheck(*handlers)) { + sendResponse(*rejection, std::move(callback)); + return; + } + } + const ChartOptions options = chartOptionsFrom(req); + runAsync(executor, req, std::move(callback), + [handlers, fn, options]() { return ((*handlers).*fn)(options); }); }, {drogon::Get}); }; - // CPU profiling is exclusive (one process-global gperftools session), so a - // request that arrives while another is sampling is refused on the spot, - // exactly like Go's net/http/pprof. The verdict is produced by the handler - // itself so the wording and status stay in one place. - auto makeCpuBusyPrecheck = [handlers](bool pprof_style) { - return [handlers, pprof_style]() -> std::optional { - if (handlers->isCpuProfilerBusy()) { - return handlers->cpuProfilerBusyResponse(pprof_style); - } - return std::nullopt; - }; - }; - using CpuBusyPrecheck = decltype(makeCpuBusyPrecheck(true)); - - // /pprof/profile must answer the way `go tool pprof` expects; the custom - // /api/* endpoints use an accurate JSON status instead. - const CpuBusyPrecheck cpu_busy_pprof = makeCpuBusyPrecheck(true); - const CpuBusyPrecheck cpu_busy_json = makeCpuBusyPrecheck(false); - - // Heap analysis is exclusive for the same reason: it reconfigures the one - // process-global heap profiler, so a concurrent call would race for it. - auto heap_busy = [handlers]() -> std::optional { - if (handlers->isHeapAnalyzerBusy()) { - return handlers->heapAnalyzerBusyResponse(); - } - return std::nullopt; - }; - - // Same, but offloaded to the executor: used for handlers that block for - // seconds (sampling) or shell out to pprof/flamegraph.pl (rendering). - auto registerGetAsync = [&](const std::string& path, auto fn) { - drogon::app().registerHandler( - path, - [handlers, executor, fn = std::move(fn)](const drogon::HttpRequestPtr& req, - std::function&& callback) { - runAsync(executor, req, std::move(callback), [handlers, fn]() { return ((*handlers).*fn)(); }); - }, - {drogon::Get}); - }; + registerChart("/api/pprof/cpu", &ProfilerHttpHandlers::handleCpuChart, &cpuBusyRejection); + registerChart("/api/pprof/heap", &ProfilerHttpHandlers::handleHeapChart, &heapBusyRejection); + registerChart("/api/pprof/growth", &ProfilerHttpHandlers::handleGrowthChart, nullptr); - // --- Static pages (served directly via WebResources) --- - drogon::app().registerHandler("/", - []([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(HandlerResponse::html(WebResources::getIndexPage()), - std::move(callback)); - }, - {drogon::Get}); - drogon::app().registerHandler("/show_svg.html", - []([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(HandlerResponse::html(WebResources::getCpuSvgViewerPage()), - std::move(callback)); - }, - {drogon::Get}); - drogon::app().registerHandler("/show_heap_svg.html", - []([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(HandlerResponse::html(WebResources::getHeapSvgViewerPage()), - std::move(callback)); - }, - {drogon::Get}); - drogon::app().registerHandler("/show_growth_svg.html", - []([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(HandlerResponse::html(WebResources::getGrowthSvgViewerPage()), - std::move(callback)); + // --------------------------------------------------------------------- + // Heap snapshot: state-based, i.e. "what is in the heap now" rather than + // "what was allocated during a window". Needs TCMALLOC_SAMPLE_PARAMETER. + // --------------------------------------------------------------------- + drogon::app().registerHandler("/api/pprof/heap/snapshot", + [handlers, executor](const drogon::HttpRequestPtr& req, + std::function&& callback) { + const ChartOptions options = chartOptionsFrom(req); + const bool as_profile = req->getParameter("format") != "svg"; + runAsync(executor, req, std::move(callback), [handlers, options, as_profile]() { + return handlers->handleHeapSnapshot(options, as_profile); + }); }, {drogon::Get}); - // --- Status --- - registerGet("/api/status", &ProfilerHttpHandlers::handleStatus); - - // --- Thread stacks --- - registerGet("/api/thread/stacks", &ProfilerHttpHandlers::handleThreadStacks); - - // --- Standard pprof: /pprof/profile (blocking: samples for `seconds`) --- - drogon::app().registerHandler( - "/pprof/profile", - [handlers, executor, cpu_busy_pprof](const drogon::HttpRequestPtr& req, - std::function&& callback) { - int seconds = 30; - auto p = req->getParameter("seconds"); - if (!p.empty()) { - try { - seconds = std::stoi(p); - } catch (...) {} - if (seconds < 1) - seconds = 1; - if (seconds > 300) - seconds = 300; - } - runAsync( - executor, req, std::move(callback), - [handlers, seconds]() { return handlers->handlePprofProfile(seconds); }, cpu_busy_pprof); - }, - {drogon::Get}); - - // --- Standard pprof: /pprof/heap --- - registerGet("/pprof/heap", &ProfilerHttpHandlers::handlePprofHeap); - - // --- Standard pprof: /pprof/growth --- - registerGet("/pprof/growth", &ProfilerHttpHandlers::handlePprofGrowth); - - // --- /pprof/symbol (POST) --- - drogon::app().registerHandler("/pprof/symbol", - [handlers]([[maybe_unused]] const drogon::HttpRequestPtr& req, - std::function&& callback) { - sendResponse(handlers->handlePprofSymbol(std::string(req->body())), - std::move(callback)); - }, - {drogon::Post}); - - // --- CPU analyze (blocking: samples for `duration` seconds) --- + // --- Status and auxiliary endpoints (fast enough to stay on the loop) --- drogon::app().registerHandler( - "/api/cpu/analyze", - [handlers, executor, cpu_busy_json](const drogon::HttpRequestPtr& req, - std::function&& callback) { - int duration = 10; - auto dp = req->getParameter("duration"); - if (!dp.empty()) { - try { - duration = std::stoi(dp); - } catch (...) {} - } - std::string output_type = req->getParameter("output_type"); - if (output_type.empty()) - output_type = "pprof"; - - runAsync( - executor, req, std::move(callback), - [handlers, duration, output_type]() { return handlers->handleCpuAnalyze(duration, output_type); }, - cpu_busy_json); - }, - {drogon::Get, drogon::Post}); - - // --- CPU raw SVG (blocking) --- - drogon::app().registerHandler( - "/api/cpu/svg_raw", - [handlers, executor, cpu_busy_json](const drogon::HttpRequestPtr& req, - std::function&& callback) { - int duration = 10; - auto dp = req->getParameter("duration"); - if (!dp.empty()) { - try { - duration = std::stoi(dp); - } catch (...) {} - } - runAsync( - executor, req, std::move(callback), - [handlers, duration]() { return handlers->handleCpuSvgRaw(duration); }, cpu_busy_json); + "/api/status", + [handlers](const drogon::HttpRequestPtr&, std::function&& cb) { + sendResponse(handlers->handleStatus(), std::move(cb)); }, {drogon::Get}); - - // --- CPU FlameGraph raw (blocking) --- drogon::app().registerHandler( - "/api/cpu/flamegraph_raw", - [handlers, executor, cpu_busy_json](const drogon::HttpRequestPtr& req, - std::function&& callback) { - int duration = 10; - auto dp = req->getParameter("duration"); - if (!dp.empty()) { - try { - duration = std::stoi(dp); - } catch (...) {} - } - runAsync( - executor, req, std::move(callback), - [handlers, duration]() { return handlers->handleCpuFlamegraphRaw(duration); }, cpu_busy_json); + "/api/thread/stacks", + [handlers](const drogon::HttpRequestPtr&, std::function&& cb) { + sendResponse(handlers->handleThreadStacks(), std::move(cb)); }, {drogon::Get}); - // --- Heap analyze (blocking) --- + // --- Web control panel --- drogon::app().registerHandler( - "/api/heap/analyze", - [handlers, executor, heap_busy](const drogon::HttpRequestPtr& req, - std::function&& callback) { - std::string output_type = req->getParameter("output_type"); - if (output_type.empty()) - output_type = "pprof"; - runAsync( - executor, req, std::move(callback), - [handlers, output_type]() { return handlers->handleHeapAnalyze(output_type); }, heap_busy); + "/", + [](const drogon::HttpRequestPtr&, std::function&& cb) { + sendResponse(HandlerResponse::html(WebResources::getIndexPage()), std::move(cb)); }, {drogon::Get}); - // --- Heap raw / FlameGraph (blocking: shells out to pprof/flamegraph.pl) --- - registerGetAsync("/api/heap/svg_raw", &ProfilerHttpHandlers::handleHeapSvgRaw); - registerGetAsync("/api/heap/flamegraph_raw", &ProfilerHttpHandlers::handleHeapFlamegraphRaw); - - // --- Growth analyze (blocking: shells out to pprof/flamegraph.pl) --- - drogon::app().registerHandler("/api/growth/analyze", + // --------------------------------------------------------------------- + // Standard Go pprof interface -- unchanged. These are the machine-readable + // endpoints `go tool pprof` consumes, as opposed to the /api/pprof visuals. + // --------------------------------------------------------------------- + drogon::app().registerHandler("/pprof/profile", [handlers, executor](const drogon::HttpRequestPtr& req, std::function&& callback) { - std::string output_type = req->getParameter("output_type"); - if (output_type.empty()) - output_type = "pprof"; - // No precheck: growth stacks come from GetHeapGrowthStacks() and read - // no profiler session, so this may run alongside CPU sampling. - runAsync(executor, req, std::move(callback), [handlers, output_type]() { - return handlers->handleGrowthAnalyze(output_type); - }); + int seconds = 30; + auto p = req->getParameter("seconds"); + if (!p.empty()) { + try { + seconds = std::stoi(p); + } catch (...) {} + if (seconds < 1) + seconds = 1; + if (seconds > 300) + seconds = 300; + } + runAsync(executor, req, std::move(callback), + [handlers, seconds]() { return handlers->handlePprofProfile(seconds); }); }, {drogon::Get}); - - // --- Growth raw / FlameGraph (blocking: shells out to pprof/flamegraph.pl) --- - registerGetAsync("/api/growth/svg_raw", &ProfilerHttpHandlers::handleGrowthSvgRaw); - registerGetAsync("/api/growth/flamegraph_raw", &ProfilerHttpHandlers::handleGrowthFlamegraphRaw); + drogon::app().registerHandler("/pprof/heap", + [handlers, executor](const drogon::HttpRequestPtr& req, + std::function&& callback) { + runAsync(executor, req, std::move(callback), + [handlers]() { return handlers->handlePprofHeap(); }); + }, + {drogon::Get}); + drogon::app().registerHandler("/pprof/growth", + [handlers, executor](const drogon::HttpRequestPtr& req, + std::function&& callback) { + runAsync(executor, req, std::move(callback), + [handlers]() { return handlers->handlePprofGrowth(); }); + }, + {drogon::Get}); + drogon::app().registerHandler( + "/pprof/symbol", + [handlers](const drogon::HttpRequestPtr& req, std::function&& callback) { + sendResponse(handlers->handlePprofSymbol(std::string(req->body())), std::move(callback)); + }, + {drogon::Post}); } PROFILER_NAMESPACE_END diff --git a/src/http_handlers.cpp b/src/http_handlers.cpp index fffe626..36b1c78 100644 --- a/src/http_handlers.cpp +++ b/src/http_handlers.cpp @@ -2,10 +2,13 @@ /// @brief Framework-agnostic HTTP endpoint handlers implementation #include "profiler/http_handlers.h" +#include "internal/result_parsing.h" #include "profiler_manager.h" #include #include +#include #include +#include PROFILER_NAMESPACE_BEGIN @@ -17,10 +20,6 @@ static HandlerResponse errorResp(int status, const std::string& message) { return HandlerResponse::error(status, message); } -static bool validateOutputType(const std::string& output_type) { - return output_type == "flamegraph" || output_type == "pprof"; -} - static int clampDuration(int duration, int lo, int hi) { if (duration < lo) return lo; @@ -91,382 +90,222 @@ HandlerResponse ProfilerHttpHandlers::handleStatus() { // --- CPU endpoints --- -HandlerResponse ProfilerHttpHandlers::handleCpuAnalyze(int duration, const std::string& output_type) { - duration = clampDuration(duration, 1, 300); - - // Fail fast instead of waiting behind (or disturbing) an in-flight session. - if (isCpuProfilerBusy()) { - return cpuProfilerBusyResponse(false); - } - - if (!validateOutputType(output_type)) { - return errorResp(400, "Invalid output_type. Must be 'flamegraph' or 'pprof'"); - } +namespace { - std::string svg = profiler_.analyzeCPUProfile(duration, output_type); +/// Convert flamegraph.pl output into a response, or an error. +/// +/// flamegraph.pl answers unusable input with a *valid* SVG whose only content is +/// an "ERROR: ..." message, so a structural check for 10 && svg[0] == '{' && svg[1] == '"') { - return errorResp(500, svg); + if (auto pos = svg.find("ERROR:"); pos != std::string::npos) { + auto end = svg.find('<', pos); + std::string message = svg.substr(pos, end == std::string::npos ? std::string::npos : end - pos); + return errorResp(500, "No stack samples to render (profile is empty). The sample window may have " + "been too short or the process idle; increase the duration or sample under " + "load. flamegraph.pl said: " + + message); } - return HandlerResponse::svg(svg); } -HandlerResponse ProfilerHttpHandlers::handleCpuSvgRaw(int duration) { - duration = clampDuration(duration, 1, 300); - - if (isCpuProfilerBusy()) { - return cpuProfilerBusyResponse(false); - } - - std::string profile_data = profiler_.getRawCPUProfile(duration); - if (profile_data.empty()) { - return errorResp(500, "Failed to generate CPU profile"); - } - - std::string temp_file = "/tmp/cpu_raw.prof"; - { - std::ofstream out(temp_file, std::ios::binary); - out.write(profile_data.data(), profile_data.size()); - } +} // namespace + +int clampChartDuration(int duration) { + // Clamp rather than reject: the HTTP layer documents duration as a window + // that is clamped to this range, and the two profiler backends disagreed -- + // getRawCPUProfile() rejected out-of-range values while + // getRawHeapProfileSample() clamped them, so the same query behaved + // differently per endpoint. Clamping here makes one rule for all of them, + // and keeps a stray value from being reported as a sampling failure. + if (duration < 1) + return 1; + if (duration > 300) + return 300; + return duration; +} - std::string exe_path = profiler_.getExecutablePath(); - std::string cmd = "./pprof --svg " + exe_path + " " + temp_file + " 2>/dev/null"; - std::string svg; - profiler_.executeCommand(cmd, svg); +ChartRenderer parseChartRenderer(const std::string& value) { + // "callgraph" is the pprof script's graphviz diagram; "flamegraph" is + // FlameGraph. Both were previously wrapped in one "output_type" whose values + // named a tool rather than the resulting picture. + if (value == "callgraph") + return ChartRenderer::CallGraph; + return ChartRenderer::FlameGraph; +} - size_t pos = svg.find(" 0) - svg = svg.substr(pos); +namespace { - if (svg.empty() || svg.find(" " << collapsed_file << " 2>&1"; - std::string out; - if (!profiler_.executeCommand(cmd.str(), out)) { - return errorResp(500, "Failed to execute pprof --collapsed command"); - } +/// Write @p text to @p path, returning false if that fails. +bool writeTempFile(const std::string& path, const std::string& text) { + std::ofstream out(path); + if (!out.is_open()) + return false; + out << text; + return out.good(); +} - // Verify collapsed data - std::ifstream in(collapsed_file); +/// Whether a collapsed-stacks file contains real stack data. +/// +/// pprof prints "Using local file ..." progress lines; counting those as data +/// would let an empty result through, and flamegraph.pl would then reject it two +/// layers away from the cause. +bool hasCollapsedData(const std::string& path) { + std::ifstream in(path); if (!in.is_open()) - return errorResp(500, "Failed to create collapsed file"); + return false; std::string line; - bool has_data = false; while (std::getline(in, line)) { - if (!line.empty() && line[0] != '#') { - has_data = true; - break; - } - } - in.close(); - if (!has_data) - return errorResp(500, "pprof --collapsed produced no data."); - - std::string fg_cmd = - "perl ./flamegraph.pl --title=\"CPU Flame Graph\" --width=1200 " + collapsed_file + " 2>/dev/null"; - std::string svg; - profiler_.executeCommand(fg_cmd, svg); - - if (svg.find(" " << collapsed << " 2>/dev/null"; - // No duration argument: heap profiling is allocation driven, not time driven. - std::string svg = profiler_.analyzeHeapProfile(output_type); + std::string ignored; + if (!profiler.executeCommand(cmd.str(), ignored)) + return errorResp(500, "Failed to execute pprof --collapsed command"); + if (!hasCollapsedData(collapsed)) + return errorResp(500, "No stack samples to render (profile is empty). The sample window may have " + "been too short or the process idle; increase the duration or sample under " + "load."); - if (svg.size() > 10 && svg[0] == '{' && svg[1] == '"') { - return errorResp(500, "Failed to generate heap flame graph"); - } + std::string svg; + std::string fg = "perl ./flamegraph.pl --title=\"" + title + "\" --width=1200 " + collapsed + " 2>/dev/null"; + profiler.executeCommand(fg, svg); - return HandlerResponse::svg(svg); + return finishChart(flameGraphResponse(svg), diagram_name, inline_display); } -HandlerResponse ProfilerHttpHandlers::handleHeapSvgRaw() { - std::string heap_sample = profiler_.getRawHeapSample(); - if (heap_sample.empty()) { - return errorResp(500, "Failed to get heap sample. Make sure TCMALLOC_SAMPLE_PARAMETER is set."); - } - - std::string temp_file = "/tmp/heap_raw.prof"; - { - std::ofstream out(temp_file); - out << heap_sample; - } - - std::string exe_path = profiler_.getExecutablePath(); - std::string cmd = "./pprof --svg " + exe_path + " " + temp_file + " 2>/dev/null"; +/// Render a gperftools profile as a call-graph SVG via the pprof script. +static HandlerResponse renderCallGraph(ProfilerManager& profiler, const std::string& profile_path, + const std::string& diagram_name, bool inline_display) { + const std::string exe = profiler.getExecutablePath(); std::string svg; - profiler_.executeCommand(cmd, svg); + // --alloc_space: the default --inuse_space plots what is live now, which a + // short collection window rarely has; the window's allocations were already + // freed. stderr is kept so a failure explains itself. + profiler.executeCommand("./pprof --svg --alloc_space " + exe + " " + profile_path + " 2>&1", svg); - size_t pos = svg.find(" 0) svg = svg.substr(pos); if (svg.empty() || svg.find(" " << collapsed_file << " 2>&1"; - std::string out; - if (!profiler_.executeCommand(cmd.str(), out)) { - return errorResp(500, "Failed to execute pprof --collapsed command"); - } - - std::ifstream in(collapsed_file); - if (!in.is_open()) - return errorResp(500, "Failed to create collapsed file"); - std::string line; - bool has_data = false; - while (std::getline(in, line)) { - if (!line.empty() && line[0] != '#') { - has_data = true; - break; - } - } - in.close(); - if (!has_data) - return errorResp(500, "pprof --collapsed produced no data"); - - std::string fg_cmd = - "perl ./flamegraph.pl --title=\"Heap Flame Graph\" --width=1200 " + collapsed_file + " 2>/dev/null"; - std::string svg; - profiler_.executeCommand(fg_cmd, svg); - - if (svg.find("(std::chrono::system_clock::now().time_since_epoch()).count()); - resp.headers["Content-Disposition"] = "attachment; filename=heap_flamegraph_" + ts + ".svg"; - return resp; + if (options.renderer == ChartRenderer::CallGraph) + return renderCallGraph(profiler, tmp_path, diagram_name, options.inline_display); + return renderFlameGraph(profiler, tmp_path, flamegraph_title, diagram_name, options.inline_display); } -// --- Growth endpoints --- - -HandlerResponse ProfilerHttpHandlers::handleGrowthAnalyze(const std::string& output_type) { - if (!validateOutputType(output_type)) { - return errorResp(400, "Invalid output_type. Must be 'flamegraph' or 'pprof'"); - } - - std::string growth = profiler_.getRawHeapGrowthStacks(); - if (growth.empty()) { - return errorResp(500, "Failed to get heap growth stacks. No heap growth data available."); - } - - std::string temp_file = "/tmp/growth_sample.prof"; - { - std::ofstream out(temp_file); - out << growth; - } - - std::string exe_path = profiler_.getExecutablePath(); - std::string svg; - - if (output_type == "flamegraph") { - std::string collapsed_file = "/tmp/growth_collapsed.prof"; - std::ostringstream cmd; - cmd << "./pprof --collapsed " << exe_path << " " << temp_file << " > " << collapsed_file << " 2>/dev/null"; - - std::string out; - if (!profiler_.executeCommand(cmd.str(), out)) { - return errorResp(500, "Failed to generate collapsed format"); - } +/// Turn a profiler-core return value into an error response, or nothing if it is fine. +static std::optional coreError(const std::string& value) { + if (internal::isJsonError(value)) + return errorResp(500, internal::jsonErrorMessage(value)); + return std::nullopt; +} - std::ostringstream fg; - fg << "perl ./flamegraph.pl --title=\"Heap Growth Flame Graph\" --width=1200 " << collapsed_file - << " 2>/dev/null"; - if (!profiler_.executeCommand(fg.str(), svg)) { - return errorResp(500, "Failed to execute flamegraph.pl command"); - } +HandlerResponse ProfilerHttpHandlers::handleCpuChart(const ChartOptions& options) { + if (isCpuProfilerBusy()) + return cpuProfilerBusyResponse(false); - if (svg.find("&1"; - if (!profiler_.executeCommand(cmd.str(), svg)) { - return errorResp(500, "Failed to execute pprof command"); - } + std::string profile = profiler_.getRawCPUProfile(clampChartDuration(options.duration)); + if (auto err = coreError(profile)) + return *err; + if (profile.empty()) + return errorResp(500, "Failed to collect a CPU profile in the requested window."); - size_t svg_start = svg.find("", svg_start); - if (tag_end != std::string::npos) { - std::string tag = svg.substr(svg_start, tag_end - svg_start); - if (tag.find("viewBox") == std::string::npos) { - svg.insert(tag_end, " viewBox=\"0 -1000 2000 1000\""); - } - } - } - } - - return HandlerResponse::svg(svg); + return renderChart(profiler_, "/tmp/cpu_chart_" + std::to_string(::getpid()) + ".prof", profile, options, + "cpu_profile", "CPU Flame Graph"); } -HandlerResponse ProfilerHttpHandlers::handleGrowthSvgRaw() { - std::string growth = profiler_.getRawHeapGrowthStacks(); - if (growth.empty()) { - return errorResp(500, "Failed to get heap growth stacks. No heap growth data available."); - } +HandlerResponse ProfilerHttpHandlers::handleHeapChart(const ChartOptions& options) { + if (isHeapAnalyzerBusy()) + return heapAnalyzerBusyResponse(); - std::string temp_file = "/tmp/growth_raw.prof"; - { - std::ofstream out(temp_file); - out << growth; - } + std::string sample = profiler_.getRawHeapProfileSample(clampChartDuration(options.duration)); + if (auto err = coreError(sample)) + return *err; + if (sample.empty()) + return errorResp(500, "Failed to collect a heap sample in the requested window."); - std::string exe_path = profiler_.getExecutablePath(); - std::string cmd = "./pprof --svg " + exe_path + " " + temp_file + " 2>/dev/null"; - std::string svg; - profiler_.executeCommand(cmd, svg); + return renderChart(profiler_, "/tmp/heap_chart_" + std::to_string(::getpid()) + ".prof", sample, options, + "heap_profile", "Heap Flame Graph"); +} - size_t pos = svg.find(" 0) - svg = svg.substr(pos); +HandlerResponse ProfilerHttpHandlers::handleHeapSnapshot(const ChartOptions& options, bool as_profile) { + std::string sample = profiler_.getRawHeapSample(); + if (sample.empty()) + return errorResp(500, "Failed to get heap sample. Make sure TCMALLOC_SAMPLE_PARAMETER is set " + "before the process was started."); - if (svg.empty() || svg.find(" " << collapsed_file << " 2>&1"; - std::string out; - if (!profiler_.executeCommand(cmd.str(), out)) { - return errorResp(500, "Failed to execute pprof --collapsed command"); - } - - std::ifstream in(collapsed_file); - if (!in.is_open()) - return errorResp(500, "Failed to create collapsed file"); - std::string line; - bool has_data = false; - while (std::getline(in, line)) { - if (!line.empty() && line[0] != '#') { - has_data = true; - break; - } - } - in.close(); - if (!has_data) - return errorResp(500, "pprof --collapsed produced no data"); - - std::string fg_cmd = - "perl ./flamegraph.pl --title=\"Heap Growth Flame Graph\" --width=1200 " + collapsed_file + " 2>/dev/null"; - std::string svg; - profiler_.executeCommand(fg_cmd, svg); - - if (svg.find("(std::chrono::system_clock::now().time_since_epoch()).count()); - resp.headers["Content-Disposition"] = "attachment; filename=growth_flamegraph_" + ts + ".svg"; - return resp; + return renderChart(profiler_, "/tmp/growth_chart_" + std::to_string(::getpid()) + ".prof", growth, options, + "growth_profile", "Heap Growth Flame Graph"); } -// --- Standard pprof --- - HandlerResponse ProfilerHttpHandlers::handlePprofProfile(int seconds) { seconds = clampDuration(seconds, 1, 300); @@ -491,10 +330,28 @@ HandlerResponse ProfilerHttpHandlers::handlePprofProfile(int seconds) { return resp; } +namespace { + +/// Error response for the /pprof/* family, following Go's serveError(): plain +/// text plus the X-Go-Pprof marker, so `go tool pprof` shows the message instead +/// of trying to parse the body as profile data. Using the JSON helper here would +/// both mislabel the content type and hide the message from the tool. +HandlerResponse pprofErrorResponse(const std::string& message) { + HandlerResponse resp; + resp.status = 500; + resp.content_type = "text/plain; charset=utf-8"; + resp.body = message + "\n"; + resp.headers["X-Go-Pprof"] = "1"; + return resp; +} + +} // namespace + HandlerResponse ProfilerHttpHandlers::handlePprofHeap() { std::string data = profiler_.getRawHeapSample(); if (data.empty()) { - return errorResp(500, "Failed to get heap sample. Make sure TCMALLOC_SAMPLE_PARAMETER is set."); + return pprofErrorResponse("Could not read heap sample: heap sampling is off or produced no " + "data. Set TCMALLOC_SAMPLE_PARAMETER before starting the process."); } HandlerResponse resp; @@ -508,7 +365,7 @@ HandlerResponse ProfilerHttpHandlers::handlePprofHeap() { HandlerResponse ProfilerHttpHandlers::handlePprofGrowth() { std::string data = profiler_.getRawHeapGrowthStacks(); if (data.empty()) { - return errorResp(500, "Failed to get heap growth stacks. No heap growth data available."); + return pprofErrorResponse("Could not read heap growth stacks: no heap growth data available."); } HandlerResponse resp; diff --git a/src/internal/result_parsing.h b/src/internal/result_parsing.h new file mode 100644 index 0000000..9d3c648 --- /dev/null +++ b/src/internal/result_parsing.h @@ -0,0 +1,77 @@ +/// @file result_parsing.h +/// @brief Helpers for interpreting profiler-core return values +/// +/// The ProfilerManager methods signal failure inconsistently by design (they are +/// older than the HTTP layer): the analyze*() family returns a `{"error": ...}` +/// JSON string, while the raw getters return an empty string. Web adapters need +/// to tell those apart and surface the actual reason, so the parsing lives here +/// where it can be unit tested in isolation from any real profiler state. + +#pragma once + +#include "profiler_version.h" +#include + +PROFILER_NAMESPACE_BEGIN + +namespace internal { + +/// @brief Whether @p value is one of the core's `{"error": ...}` strings +/// +/// The check is structural rather than a JSON parse -- the core only ever emits +/// this exact shape, and callers sit on a request path. It does require the key, +/// so an unrelated JSON document that merely starts with `{"` is not mistaken +/// for a failure. +inline bool isJsonError(const std::string& value) { + return value.size() > 10 && value[0] == '{' && value.find("\"error\"") != std::string::npos; +} + +/// @brief Extract the message from a core `{"error": ...}` string +/// +/// Needed because wrapping the raw JSON in another error response would both +/// double-wrap it and, since the inner quotes are unescaped, emit invalid JSON +/// (`{"error":"{"error": "..."}`). Substituting a fixed message instead would +/// discard the reason, which is how "cpu profiling already in use" used to +/// arrive at clients as a generic failure. +/// +/// @param value A string previously accepted by isJsonError() +/// @return The inner message, or @p value unchanged if it cannot be parsed +inline std::string jsonErrorMessage(const std::string& value) { + const std::string key = "\"error\""; + auto key_pos = value.find(key); + if (key_pos == std::string::npos) { + return value; + } + auto open_quote = value.find('"', key_pos + key.size()); + if (open_quote == std::string::npos) { + return value; + } + auto close_quote = value.rfind('"'); + if (close_quote == std::string::npos || close_quote <= open_quote) { + return value; + } + return value.substr(open_quote + 1, close_quote - open_quote - 1); +} + +/// @brief Whether a heap sample from GetHeapSample() carries no usable data +/// +/// With heap sampling off (TCMALLOC_SAMPLE_PARAMETER unset, whose default is 0) +/// tcmalloc does not return an empty string. It returns pprof's `%warn` +/// advisory followed by a syntactically valid but empty profile: +/// +/// ``` +/// %warn +/// %warn This heap profile does not have any data in it, because +/// heap profile: 0: 0 [ 0: 0] @ heap_v2/0 +/// MAPPED_LIBRARIES: +/// ``` +/// +/// So testing for emptiness alone never fires, and callers render a graph out of +/// nothing. The zero sampling rate (or the leading `%warn`) marks the case. +inline bool isEmptyHeapSample(const std::string& sample) { + return sample.empty() || sample.find("@ heap_v2/0") != std::string::npos || sample.rfind("%warn", 0) == 0; +} + +} // namespace internal + +PROFILER_NAMESPACE_END diff --git a/src/internal/web_resources.h b/src/internal/web_resources.h index 4eaf8ee..01c7365 100644 --- a/src/internal/web_resources.h +++ b/src/internal/web_resources.h @@ -8,17 +8,9 @@ PROFILER_NAMESPACE_BEGIN // Web 资源管理类 - 提供嵌入的 HTML 页面内容 class WebResources { public: - // 获取主页面 HTML + /// The single control panel page. Charts are delivered as downloadable or + /// inline SVG by the analysis endpoints; there are no separate viewer pages. static std::string getIndexPage(); - - // 获取 CPU 火焰图查看器 HTML - static std::string getCpuSvgViewerPage(); - - // 获取 Heap 火焰图查看器 HTML - static std::string getHeapSvgViewerPage(); - - // 获取 Growth 火焰图查看器 HTML - static std::string getGrowthSvgViewerPage(); }; PROFILER_NAMESPACE_END diff --git a/src/profiler_manager.cpp b/src/profiler_manager.cpp index 67a9dde..9b8d1fe 100644 --- a/src/profiler_manager.cpp +++ b/src/profiler_manager.cpp @@ -5,6 +5,7 @@ #include "internal/embed_pprof.h" #include "internal/log_macros.h" #include "internal/log_manager.h" +#include "internal/result_parsing.h" #include "internal/symbolize.h" #include #include @@ -17,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -37,6 +39,30 @@ PROFILER_NAMESPACE_BEGIN +namespace { + +/// Read an entire file into a string. Returns false if it cannot be opened. +/// +/// Used for profile artifacts written by gperftools: reading them back into our +/// own std::string keeps ownership on our side of the boundary, which +/// GetHeapProfile()'s return value does not allow (see stopHeapProfiler()). +bool readWholeFile(const std::string& path, std::string* out) { + std::ifstream file(path, std::ios::binary); + if (!file.is_open()) + return false; + + file.seekg(0, std::ios::end); + const std::streamoff size = file.tellg(); + file.seekg(0, std::ios::beg); + if (size > 0) { + out->resize(static_cast(size)); + file.read(&(*out)[0], size); + } + return true; +} + +} // namespace + // Static member initialization // Process-global, like the gperftools sampling session it guards. std::atomic ProfilerManager::cpu_profiling_in_progress_{false}; @@ -263,14 +289,48 @@ bool ProfilerManager::stopHeapProfiler() { } if (IsHeapProfilerRunning()) { - // 保存 heap profile 到文件 - std::string heap_profile = GetHeapProfile(); - std::string output_path = profiler_states_[ProfilerType::HEAP].output_path; + const std::string output_path = profiler_states_[ProfilerType::HEAP].output_path; + + // Write the snapshot to disk and read it back, rather than taking it from + // GetHeapProfile(). That function hands back a buffer the caller is told to + // free(), but no deallocator is correct across the supported gperftools + // range: under a statically linked tcmalloc the buffer belongs to the + // library's own allocator, which the process's malloc interposer never saw, + // so free() aborts (this is what broke CI); tc_free() only exists from + // 2.16.90; and 2.18 routes intermediate chunks through an internal arena + // whose Free sits in a private header. Dumping to a file keeps the memory + // on our side, so there is nothing to free and no leak to suppress. + // + // HeapProfilerDump() appends "..heap" to the profiler's prefix, + // which is the path startHeapProfiler() was given -- so the dump lands next + // to output_path under a derived name, not at output_path itself. + const std::filesystem::path output_as_path(output_path); + const std::string dump_dir = output_as_path.has_parent_path() ? output_as_path.parent_path().string() : "."; + const std::string dump_prefix = output_as_path.filename().string(); + + PROFILER_DEBUG("Calling HeapProfilerDump()..."); + HeapProfilerDump("stop"); + + // gperftools writes the dump from a signal-adjacent context; give the + // filesystem a moment to make it visible, as the analysis path does. + usleep(100000); + + std::string heap_profile; + const std::string dump_file = findLatestHeapProfile(dump_dir, dump_prefix); + if (dump_file.empty()) { + PROFILER_WARNING("HeapProfilerDump() produced no .heap file matching '{}' in {}", dump_prefix, dump_dir); + } else { + PROFILER_INFO("Using heap dump: {}", dump_file); + if (!readWholeFile(dump_file, &heap_profile)) + PROFILER_ERROR("Failed to read heap dump: {}", dump_file); + } std::ofstream file(output_path); if (file.is_open()) { file << heap_profile; file.close(); + } else { + PROFILER_WARNING("Failed to write heap profile to {}", output_path); } HeapProfilerStop(); @@ -349,6 +409,19 @@ std::string ProfilerManager::generateFlameGraph(const std::string& collapsed_fil return R"({"error": "flamegraph.pl did not generate valid SVG"})"; } + // flamegraph.pl answers empty or unusable input with a *valid* SVG whose only + // content is an error message, so the structural check above passes and a + // 200 carrying "ERROR: ..." would reach the caller. Treat that as a failure + // and say why, since the usual cause is a profile with no samples. + if (svg_output.find("ERROR:") != std::string::npos) { + auto error_pos = svg_output.find("ERROR:"); + auto error_end = svg_output.find('<', error_pos); + std::string message = svg_output.substr(error_pos, error_end - error_pos); + PROFILER_ERROR("flamegraph.pl reported: {}", message); + return R"({"error": "No stack samples to render (profile is empty). The sampled window may have been too short or the process was idle; try a longer duration or sample under load. flamegraph.pl said: )" + + message + R"("})"; + } + return svg_output; } @@ -424,7 +497,7 @@ bool ProfilerManager::isHeapAnalysisInProgress() const { } std::string ProfilerManager::analyzeCPUProfile(int duration, const std::string& output_type) { - std::string profile_path = profile_dir_ + "/cpu_analyze.prof"; + std::string profile_path = profile_dir_ + "/cpu_analyze_" + std::to_string(::getpid()) + ".prof"; // Step 1: Claim the CPU profiling session. // @@ -497,7 +570,7 @@ std::string ProfilerManager::analyzeCPUProfile(int duration, const std::string& PROFILER_INFO("Generating FlameGraph output..."); // Step 5a: Generate collapsed format using pprof --collapsed - std::string collapsed_file = "/tmp/cpu_collapsed.prof"; + std::string collapsed_file = "/tmp/cpu_collapsed_" + std::to_string(::getpid()) + ".prof"; std::ostringstream collapsed_cmd; collapsed_cmd << "./pprof --collapsed " << exe_path << " " << profile_path << " > " << collapsed_file << " 2>&1"; @@ -596,14 +669,18 @@ std::string ProfilerManager::analyzeCPUProfile(int duration, const std::string& return svg_output; } -std::string ProfilerManager::analyzeHeapProfile(const std::string& output_type) { +std::string ProfilerManager::analyzeHeapProfile(int duration, const std::string& output_type) { // gperftools heap profiling is allocation driven: HeapProfilerStart() begins // recording and HeapProfilerDump() writes a snapshot. Unlike the CPU // profiler there is no sampling period to tune -- the sample rate comes from // TCMALLOC_SAMPLE_PARAMETER, which tcmalloc reads once at process startup. // So there is no user-facing duration here: we simply open a short window // during which the application's own allocations get recorded, then dump. - static constexpr int kSampleWindowSeconds = 1; + // Guard the window the same way the CPU path guards its sampling duration. + if (duration < 1) + duration = 1; + if (duration > 300) + duration = 300; PROFILER_INFO("=== Starting Heap Profile Analysis ==="); @@ -667,8 +744,8 @@ std::string ProfilerManager::analyzeHeapProfile(const std::string& output_type) // allocations here: a profiler must report the host process's real memory // behaviour, and fabricating allocations both pollutes the result and leaks // memory in the process under test. - PROFILER_INFO("Collecting heap samples for {} second(s)...", kSampleWindowSeconds); - sleep(kSampleWindowSeconds); + PROFILER_INFO("Collecting heap samples for {} second(s)...", duration); + sleep(duration); // Step 5: Take the snapshot, then stop if (IsHeapProfilerRunning()) { @@ -715,7 +792,7 @@ std::string ProfilerManager::analyzeHeapProfile(const std::string& output_type) PROFILER_INFO("Generating Heap FlameGraph..."); // Step 9a: Generate collapsed format using pprof --collapsed - std::string collapsed_file = "/tmp/heap_collapsed.prof"; + std::string collapsed_file = "/tmp/heap_collapsed_" + std::to_string(::getpid()) + ".prof"; std::ostringstream collapsed_cmd; collapsed_cmd << "./pprof --collapsed " << exe_path << " " << latest_heap_file << " > " << collapsed_file << " 2>&1"; @@ -865,7 +942,7 @@ std::string ProfilerManager::getRawCPUProfile(int seconds) { FlagGuard guard{cpu_profiling_in_progress_}; // Generate temporary profile file path - std::string profile_path = profile_dir_ + "/pprof_cpu_temp.prof"; + std::string profile_path = profile_dir_ + "/pprof_cpu_temp_" + std::to_string(::getpid()) + ".prof"; // Start CPU profiler. A session opened through startCPUProfiler() belongs to // the caller, so refuse rather than stopping it; the caller of this function @@ -938,8 +1015,9 @@ std::string ProfilerManager::getRawHeapSample() { // GetHeapSample writes the heap profile into the string MallocExtension::instance()->GetHeapSample(&heap_sample); - if (heap_sample.empty()) { - PROFILER_ERROR("Failed to get heap sample. Make sure TCMALLOC_SAMPLE_PARAMETER environment variable is set."); + if (internal::isEmptyHeapSample(heap_sample)) { + PROFILER_ERROR("Heap sampling is off or produced no data. Set TCMALLOC_SAMPLE_PARAMETER " + "(e.g. 524288) before starting the process."); return ""; } @@ -947,143 +1025,98 @@ std::string ProfilerManager::getRawHeapSample() { return heap_sample; } -std::string ProfilerManager::getRawHeapGrowthStacks() { - // Get heap growth stacks from tcmalloc - // MallocExtensionWriter is typedef'd as std::string - std::string heap_growth_stacks; +std::string ProfilerManager::getRawHeapProfileSample(int duration) { + if (duration < 1) + duration = 1; + if (duration > 300) + duration = 300; - // GetHeapGrowthStacks writes the heap growth stacks into the string - MallocExtension::instance()->GetHeapGrowthStacks(&heap_growth_stacks); - - if (heap_growth_stacks.empty()) { - PROFILER_ERROR("Failed to get heap growth stacks. No heap growth data available."); - return ""; + // Same claim as analyzeHeapProfile(): the heap profiler is process-global, so + // a window may only be opened by one caller at a time. + { + bool expected = false; + if (!heap_analysis_in_progress_.compare_exchange_strong(expected, true)) { + PROFILER_ERROR("Heap analysis already in progress; rejecting this request"); + return R"({"error": "heap profiling already in use"})"; + } } - PROFILER_INFO("Heap growth stacks size: {} bytes", heap_growth_stacks.size()); - return heap_growth_stacks; -} - -std::string ProfilerManager::getThreadStacks() { - std::ostringstream result; + struct ClaimGuard { + static void release() { + heap_analysis_in_progress_.store(false); + } + ~ClaimGuard() { + release(); + } + } claim_guard; - // Open /proc/self/task directory to list all threads - DIR* task_dir = opendir("/proc/self/task"); - if (!task_dir) { - PROFILER_ERROR("Failed to open /proc/self/task"); - return ""; + { + std::lock_guard lock(mutex_); + if (profiler_states_[ProfilerType::HEAP].is_running) { + PROFILER_ERROR("A heap profiler session is already open; refusing to take it over"); + return R"({"error": "heap profiling already in use"})"; + } } - result << "Thread Stacks Snapshot\n"; - result << "======================\n\n"; - - // Iterate through all thread directories - struct dirent* entry; - int thread_count = 0; + // Unique prefix: keeps this window's records separate from any other run, and + // it is also where HeapProfilerDump() below writes (it appends "..heap"). + auto now = std::chrono::system_clock::now(); + auto timestamp = std::chrono::duration_cast(now.time_since_epoch()).count(); + std::string prefix_basename = + "heap_raw_" + std::to_string(timestamp) + "_" + std::to_string(heap_prefix_sequence_.fetch_add(1)); + std::string prefix = profile_dir_ + "/" + prefix_basename; - while ((entry = readdir(task_dir)) != nullptr) { - // Skip "." and ".." entries - if (entry->d_name[0] == '.') { - continue; - } + PROFILER_INFO("Collecting heap samples for {} second(s) (raw)...", duration); + HeapProfilerStart(prefix.c_str()); + sleep(duration); - // Get thread ID - pid_t tid = atoi(entry->d_name); - if (tid == 0) { - continue; - } + std::string sample; + if (IsHeapProfilerRunning()) { + // Dump to disk and read it back instead of calling GetHeapProfile(): the + // buffer that function returns has no deallocator that is correct across + // the supported gperftools range (see stopHeapProfiler()). Going through a + // file keeps the memory on our side, so nothing needs freeing. + PROFILER_DEBUG("Calling HeapProfilerDump()..."); + HeapProfilerDump("raw"); + HeapProfilerStop(); - thread_count++; - - result << "Thread " << tid << ":\n"; - - // Read thread stat to get state and name - std::string stat_file = std::string("/proc/self/task/") + entry->d_name + "/stat"; - std::ifstream stat_stream(stat_file); - - if (stat_stream.is_open()) { - std::string line; - if (std::getline(stat_stream, line)) { - // Parse stat file (format: pid (comm) state ...) - size_t open_paren = line.find('('); - size_t close_paren = line.find(')', open_paren); - - if (open_paren != std::string::npos && close_paren != std::string::npos) { - std::string name = line.substr(open_paren + 1, close_paren - open_paren - 1); - - // Get state character (after close_paren + 2) - if (close_paren + 2 < line.length()) { - char state = line[close_paren + 2]; - - // Convert state to readable format - const char* state_str = "Unknown"; - switch (state) { - case 'R': - state_str = "Running"; - break; - case 'S': - state_str = "Sleeping"; - break; - case 'D': - state_str = "Disk sleep"; - break; - case 'Z': - state_str = "Zombie"; - break; - case 'T': - state_str = "Stopped"; - break; - case 't': - state_str = "Tracing stop"; - break; - case 'X': - state_str = "Dead"; - break; - case 'x': - state_str = "Dead"; - break; - case 'K': - state_str = "Wakekill"; - break; - case 'W': - state_str = "Waking"; - break; - case 'P': - state_str = "Parked"; - break; - } - - result << " Name: " << name << "\n"; - result << " State: " << state_str << " (" << state << ")\n"; - } - } - } - stat_stream.close(); + const std::string dump_file = findLatestHeapProfile(profile_dir_, prefix_basename); + if (dump_file.empty()) { + PROFILER_WARNING("HeapProfilerDump() produced no .heap file matching '{}' in {}", prefix_basename, + profile_dir_); + } else { + PROFILER_INFO("Using heap dump: {}", dump_file); + if (!readWholeFile(dump_file, &sample)) + PROFILER_ERROR("Failed to read heap dump: {}", dump_file); } + } - // Try to read wchan (what thread is waiting on) - std::string wchan_file = std::string("/proc/self/task/") + entry->d_name + "/wchan"; - std::ifstream wchan_stream(wchan_file); - - if (wchan_stream.is_open()) { - std::string wchan; - if (std::getline(wchan_stream, wchan) && !wchan.empty()) { - result << " Waiting in: " << wchan << "\n"; - } - wchan_stream.close(); - } + ClaimGuard::release(); - result << "\n"; + if (internal::isEmptyHeapSample(sample)) { + PROFILER_ERROR("Heap profiling produced no data in the {}s window", duration); + return R"({"error": "No heap profile data was produced. The heap profiler only records allocations made while it is running, so the analyzed process must be actively allocating. Increase the duration for a process that allocates sparsely."})"; } - closedir(task_dir); + PROFILER_INFO("Raw heap profile size: {} bytes", sample.size()); + return sample; +} - result << "Total threads: " << thread_count << "\n"; +std::string ProfilerManager::getRawHeapGrowthStacks() { + // Get heap growth stacks from tcmalloc + // MallocExtensionWriter is typedef'd as std::string + std::string heap_growth_stacks; - std::string output = result.str(); - PROFILER_INFO("Thread stacks collected, size: {} bytes", output.size()); + // GetHeapGrowthStacks writes the heap growth stacks into the string + MallocExtension::instance()->GetHeapGrowthStacks(&heap_growth_stacks); - return output; + if (heap_growth_stacks.empty()) { + PROFILER_ERROR("Failed to get heap growth stacks. No heap growth data available."); + return ""; + } + + PROFILER_INFO("Heap growth stacks size: {} bytes", heap_growth_stacks.size()); + return heap_growth_stacks; } // Signal handler for capturing stack traces (signal-safe) @@ -1327,6 +1360,25 @@ std::vector ProfilerManager::captureAllThreadStacks() { return result; } +namespace { + +/// Read a thread's name from /proc//comm (what ps and top display). +/// +/// Returns an empty string when the thread has exited or the file cannot be +/// read, in which case the caller prints a bare tid rather than failing. +std::string readThreadName(pid_t tid) { + std::ifstream comm("/proc/self/task/" + std::to_string(tid) + "/comm"); + std::string name; + if (comm.is_open()) + std::getline(comm, name); + // The kernel truncates to 15 characters and may leave a trailing newline. + while (!name.empty() && (name.back() == '\n' || name.back() == '\r' || name.back() == ' ')) + name.pop_back(); + return name; +} + +} // namespace + std::string ProfilerManager::getThreadCallStacks() { std::ostringstream result; @@ -1340,7 +1392,14 @@ std::string ProfilerManager::getThreadCallStacks() { // Process each thread's stack for (const auto& trace : stacks) { - result << "Thread " << trace.tid << ":\n"; + // Include the thread's name. A bare tid means nothing when reading the + // output afterwards, whereas "DrogonIoLoop" or "worker-3" identifies the + // thread at a glance. Same source ps and top use. + result << "Thread " << trace.tid; + const std::string name = readThreadName(trace.tid); + if (!name.empty()) + result << " (" << name << ")"; + result << ":\n"; result << " Frames: " << trace.depth << "\n"; // Symbolize and print each frame using abseil diff --git a/src/web_resources.cpp b/src/web_resources.cpp index e4e47bd..2a2096e 100644 --- a/src/web_resources.cpp +++ b/src/web_resources.cpp @@ -145,6 +145,18 @@ static const char INDEX_PAGE[] = R"HTML( display: inline-block; margin: 5px; } + .card { + border: 1px solid #e0e0e0; + border-radius: 4px; + padding: 12px; + margin: 10px 0; + background-color: #fff; + } + .card h3 { + margin: 0 0 6px; + font-size: 15px; + color: #333; + } .hint { display: block; margin: 6px 0 10px; @@ -180,41 +192,65 @@ static const char INDEX_PAGE[] = R"HTML( +
+ + +
- 两种图表类型都按上面的时长采样,区别只在渲染方式(pprof 脚本 vs FlameGraph)。 - CPU 采样是独占的:若已有请求或宿主程序正在采样,本次会被拒绝(409 / 500)。 + 两种图形是不同的表达:火焰图是栈帧堆叠,调用图是节点/边的有向图(且能解析内联帧)。 + CPU 采样是独占的:若已有请求或宿主程序正在采样,本次会被拒绝(409)。 + 时长太短会采不到样本,建议 ≥10 秒。 + + + + + +
+

Heap Profiler

+
- - +
+
+ +
- - + + ⚠️ 只记录采集窗口内发生的分配(流量),不反映此刻已经在堆里的内存(存量)。 + 想看当前堆快照请用下面的 Heap Snapshot。分配稀疏时把窗口调长。 + + +
-

Heap Profiler

+

Heap Snapshot

- ⚠️ 这里的两个按钮都是窗口式分析:采样 1 秒并记录这段时间内发生的分配, - 不反映此刻已经在堆里的内存。若进程内存虽高但已停止分配,结果会是空的(返回 - "No heap profile data was produced")。想看当前堆的累计快照,请用 - GET /pprof/heap 导出原始 profile(需在启动前设置 - TCMALLOC_SAMPLE_PARAMETER)。 + 状态式:反映堆里现存的内存(tcmalloc 累计采样),对应 + GET /api/pprof/heap/snapshot。 + 需要在进程启动前设置 TCMALLOC_SAMPLE_PARAMETER,否则返回 500。 + 刚启动的进程可能显示空快照(tcmalloc 在第一次采样事件前不报告数据),稍等再试即可。
- - + + +
- - + +
@@ -222,14 +258,15 @@ static const char INDEX_PAGE[] = R"HTML(

Heap Growth Profiler

- - + +
- - + 堆增长栈分析,不需要 TCMALLOC_SAMPLE_PARAMETER,即时获取。 + +
@@ -247,37 +284,33 @@ static const char INDEX_PAGE[] = R"HTML( - - -)HTML"; - -static const char HEAP_SVG_VIEWER_PAGE[] = R"HTML( - - - - Heap Flame Graph Viewer - - - -

🔥 Heap 内存火焰图

-
- 💡 查看提示: -
    -
  • 滚动: 图表可能很大,请滚动查看所有函数
  • -
  • 缩放: 使用鼠标滚轮可以缩放视图
  • -
  • 拖拽: 按住鼠标左键可以拖动图表
  • -
  • 搜索: 使用 Ctrl+F 搜索特定函数(如 cpuIntensiveTask、FibonacciCalculator)
  • -
  • 点击: 点击节点可以高亮显示相关调用
  • -
-
-
- - - - - - - -
-
加载中...
- - - - -)HTML"; - -static const char GROWTH_SVG_VIEWER_PAGE[] = R"HTML( - - - - Growth Flame Graph Viewer - - - -

🔥 Heap Growth 堆增长火焰图

-
- 💡 查看提示: -
    -
  • 滚动: 图表可能很大,请滚动查看所有函数
  • -
  • 缩放: 使用鼠标滚轮可以缩放视图
  • -
  • 拖拽: 按住鼠标左键可以拖动图表
  • -
  • 搜索: 使用 Ctrl+F 搜索特定函数
  • -
  • 点击: 点击节点可以高亮显示相关调用
  • -
-
-
- - - - - - - -
-
加载中...
- - - - -)HTML"; - std::string WebResources::getIndexPage() { return std::string(INDEX_PAGE); } -std::string WebResources::getCpuSvgViewerPage() { - return std::string(CPU_SVG_VIEWER_PAGE); -} - -std::string WebResources::getHeapSvgViewerPage() { - return std::string(HEAP_SVG_VIEWER_PAGE); -} - -std::string WebResources::getGrowthSvgViewerPage() { - return std::string(GROWTH_SVG_VIEWER_PAGE); -} - PROFILER_NAMESPACE_END diff --git a/tests/test_full_flow.cpp b/tests/test_full_flow.cpp index 606b04e..a26452a 100644 --- a/tests/test_full_flow.cpp +++ b/tests/test_full_flow.cpp @@ -7,6 +7,27 @@ #include #include +namespace { + +/// Keep one core busy for at least @p ms milliseconds. +/// +/// gperftools samples whichever threads are *running*, so a window spent in +/// sleep() legitimately yields an empty profile. Tests that require a non-empty +/// profile must therefore generate real CPU work for the whole window instead of +/// sleeping through it -- otherwise they fail whenever the machine happens to be +/// idle, which is exactly how this test failed under parallel ctest. +void burnCPUFor(int ms) { + const auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(ms); + volatile uint64_t sink = 0; + while (std::chrono::steady_clock::now() < deadline) { + for (int i = 0; i < 20000; ++i) { + sink += static_cast(i) * 2654435761u; + } + } +} + +} // namespace + // 测试1: 验证 gperftools 生成的 profile 文件格式 TEST(FullFlowTest, GperftoolsGeneratesValidProfile) { const char* profile_path = "/tmp/test_full.prof"; @@ -25,7 +46,8 @@ TEST(FullFlowTest, GperftoolsGeneratesValidProfile) { std::sort(data.begin(), data.end()); } - std::this_thread::sleep_for(std::chrono::milliseconds(500)); + // Keep sampling real work rather than idling: see burnCPUFor(). + burnCPUFor(500); // 停止 profiler ProfilerStop(); @@ -163,9 +185,17 @@ TEST(FullFlowTest, ConcurrentProfilingRequests) { TEST(FullFlowTest, GetRawCPUProfile) { profiler::ProfilerManager profiler; + // Sample while work is actually running: the profiler only records samples + // from executing threads, so a 1s window with nothing to sample returns an + // empty profile by design. The workload used to be absent here, which made + // the non-empty assertion below depend on machine load. + std::thread burner{[]() { burnCPUFor(1200); }}; + // 获取原始 profile 数据(采样 1 秒) std::string profile_data = profiler.getRawCPUProfile(1); + burner.join(); + // 应该返回非空数据 ASSERT_FALSE(profile_data.empty()) << "getRawCPUProfile returned empty data"; diff --git a/tests/test_http_handlers.cpp b/tests/test_http_handlers.cpp index 6f29cdd..7bab364 100644 --- a/tests/test_http_handlers.cpp +++ b/tests/test_http_handlers.cpp @@ -8,6 +8,7 @@ #include "../include/profiler_manager.h" #include #include +#include #include #include #include @@ -21,6 +22,20 @@ namespace { +/// Build ChartOptions concisely for tests. +profiler::ChartOptions opts(int duration = 1, profiler::ChartRenderer renderer = profiler::ChartRenderer::FlameGraph, + bool inline_display = true) { + profiler::ChartOptions o; + o.duration = duration; + o.renderer = renderer; + o.inline_display = inline_display; + return o; +} + +} // namespace + +namespace { + /// Handlers that never touch the filesystem or spawn pprof can be tested /// directly. Anything that renders an SVG shells out to ./pprof and is covered /// by test_full_flow instead. @@ -60,21 +75,12 @@ TEST_F(HttpHandlersTest, StatusUsesMillisecondDurationKeys) { } // --------------------------------------------------------------------------- -// output_type validation +// renderer parsing // --------------------------------------------------------------------------- - -TEST_F(HttpHandlersTest, InvalidOutputTypeIsRejectedWith400) { - auto resp = handlers.handleCpuAnalyze(1, "iciclegraph"); - - EXPECT_EQ(resp.status, 400); - EXPECT_NE(resp.body.find("output_type"), std::string::npos) << "error message should name the offending parameter"; -} - -TEST_F(HttpHandlersTest, EmptyOutputTypeIsRejected) { - auto resp = handlers.handleCpuAnalyze(1, ""); - EXPECT_EQ(resp.status, 400); -} - +// (an unknown renderer is not an error: parseChartRenderer falls back to +// FlameGraph rather than failing the request, so there is no 400 case to assert. +// The mapping itself is covered by ChartRendererParsing.) +// // --------------------------------------------------------------------------- // HandlerResponse helpers // --------------------------------------------------------------------------- @@ -152,9 +158,9 @@ TEST(ConcurrentCpuProfilingTest, RequestsFailFastWhileASessionIsSampling) { // Every CPU entry point must refuse immediately, not wait for the sampler. const auto t0 = std::chrono::steady_clock::now(); - EXPECT_EQ(handlers.handleCpuAnalyze(5, "pprof").status, 409); - EXPECT_EQ(handlers.handleCpuSvgRaw(5).status, 409); - EXPECT_EQ(handlers.handleCpuFlamegraphRaw(5).status, 409); + EXPECT_EQ(handlers.handleCpuChart(opts(5)).status, 409); + EXPECT_EQ(handlers.handleCpuChart(opts(5, profiler::ChartRenderer::CallGraph)).status, 409); + EXPECT_EQ(handlers.handleCpuChart(opts(5)).status, 409); const auto elapsed = std::chrono::steady_clock::now() - t0; EXPECT_LT(elapsed, std::chrono::milliseconds(100)) << "rejection must be immediate, not queued behind the running session"; @@ -275,8 +281,8 @@ TEST(HostOwnedSessionTest, HandlersReportTheHostSessionAsBusy) { ASSERT_EQ(resp.headers.count("X-Go-Pprof"), 1u); // The /api/* endpoints report a conflict. - EXPECT_EQ(handlers.handleCpuAnalyze(1, "pprof").status, 409); - EXPECT_EQ(handlers.handleCpuSvgRaw(1).status, 409); + EXPECT_EQ(handlers.handleCpuChart(opts(1)).status, 409); + EXPECT_EQ(handlers.handleCpuChart(opts(1, profiler::ChartRenderer::CallGraph)).status, 409); profiler.stopCPUProfiler(); } @@ -288,11 +294,11 @@ TEST(HostOwnedSessionTest, HeapAnalysisRefusesToTakeOverAHostSession) { ASSERT_TRUE(profiler.startHeapProfiler("/tmp/test_host_owned_heap.prof")); ASSERT_TRUE(profiler.isProfilerRunning(profiler::ProfilerType::HEAP)); - auto result = profiler.analyzeHeapProfile("pprof"); + auto result = profiler.analyzeHeapProfile(1, "pprof"); EXPECT_NE(result.find("heap profiling already in use"), std::string::npos) << result; EXPECT_TRUE(profiler.isProfilerRunning(profiler::ProfilerType::HEAP)); - EXPECT_EQ(handlers.handleHeapAnalyze("pprof").status, 409); + EXPECT_EQ(handlers.handleHeapChart(opts(1)).status, 409); EXPECT_TRUE(profiler.stopHeapProfiler()); } @@ -311,7 +317,7 @@ class HeapAnalysisHolder { public: explicit HeapAnalysisHolder(profiler::ProfilerManager& profiler) : future_(std::async(std::launch::async, - [&profiler]() -> std::string { return profiler.analyzeHeapProfile("pprof"); })) {} + [&profiler]() -> std::string { return profiler.analyzeHeapProfile(1, "pprof"); })) {} ~HeapAnalysisHolder() { if (future_.valid()) { @@ -352,7 +358,7 @@ TEST(ConcurrentHeapAnalysisTest, SecondAnalysisIsRejectedNotRaced) { // Must be refused immediately, and must not touch the running analysis. const auto t0 = std::chrono::steady_clock::now(); - auto result = profiler.analyzeHeapProfile("pprof"); + auto result = profiler.analyzeHeapProfile(1, "pprof"); const auto elapsed = std::chrono::steady_clock::now() - t0; EXPECT_NE(result.find("heap profiling already in use"), std::string::npos) << result; @@ -367,7 +373,7 @@ TEST(ConcurrentHeapAnalysisTest, HandlerReportsConflict) { HeapAnalysisHolder holder{profiler}; ASSERT_TRUE(waitForHeapClaim(profiler)); - auto resp = handlers.handleHeapAnalyze("pprof"); + auto resp = handlers.handleHeapChart(opts(1)); EXPECT_EQ(resp.status, 409); EXPECT_NE(resp.body.find("heap profiling already in use"), std::string::npos) << resp.body; @@ -389,18 +395,77 @@ TEST(ConcurrentHeapAnalysisTest, ClaimIsReleasedAfterwards) { // Heap analysis API shape // --------------------------------------------------------------------------- -TEST_F(HttpHandlersTest, HeapAnalysisHasNoDurationParameter) { - // Regression guard: heap profiling is allocation driven, so no duration is - // accepted. This must stay compilable with a single argument. - // (If a duration parameter were reintroduced this call would not compile.) - static_assert(std::is_invocable_v, - "handleHeapAnalyze must take only output_type"); +TEST_F(HttpHandlersTest, ChartEndpointsTakeDurationAndRenderer) { + // Each analysis endpoint takes a ChartOptions carrying duration, renderer and + // delivery mode. The duration is a *collection window*, not a sampling rate: + // the rate is fixed by TCMALLOC_SAMPLE_PARAMETER at process start. A longer + // window covers more allocations, which is what a sparsely allocating process + // needs. Guard the contract so it cannot regress into "duration is ignored". + using H = profiler::ProfilerHttpHandlers; + + static_assert(std::is_invocable_v, + "handleCpuChart must take ChartOptions"); + static_assert(std::is_invocable_v, + "handleHeapChart must take ChartOptions"); + static_assert(std::is_invocable_v, + "handleGrowthChart must take ChartOptions"); + static_assert(std::is_invocable_v, + "handleHeapSnapshot must take (ChartOptions, as_profile)"); +} + +TEST_F(HttpHandlersTest, ChartDurationIsClamped) { + // The HTTP layer documents duration as clamped to 1..300, and it must behave + // that way for every endpoint: the backends disagreed (getRawCPUProfile + // rejected out-of-range values, getRawHeapProfileSample clamped them), which + // made the same query succeed or fail depending on which profiler served it. + EXPECT_EQ(profiler::clampChartDuration(0), 1); + EXPECT_EQ(profiler::clampChartDuration(-5), 1); + EXPECT_EQ(profiler::clampChartDuration(1), 1); + EXPECT_EQ(profiler::clampChartDuration(10), 10); + EXPECT_EQ(profiler::clampChartDuration(300), 300); + EXPECT_EQ(profiler::clampChartDuration(301), 300); + EXPECT_EQ(profiler::clampChartDuration(9999), 300); } -TEST_F(HttpHandlersTest, HeapAnalyzeRejectsInvalidOutputType) { - auto resp = handlers.handleHeapAnalyze("bogus"); - EXPECT_EQ(resp.status, 400); +TEST_F(HttpHandlersTest, ChartRendererParsing) { + // "callgraph" is the pprof/graphviz diagram, "flamegraph" is FlameGraph. The + // old parameter was called output_type and its values named a tool rather + // than the resulting picture, which is what these names replace. + EXPECT_EQ(profiler::parseChartRenderer("callgraph"), profiler::ChartRenderer::CallGraph); + EXPECT_EQ(profiler::parseChartRenderer("flamegraph"), profiler::ChartRenderer::FlameGraph); + // Unknown or empty values fall back rather than failing the request. + EXPECT_EQ(profiler::parseChartRenderer(""), profiler::ChartRenderer::FlameGraph); + EXPECT_EQ(profiler::parseChartRenderer("pprof"), profiler::ChartRenderer::FlameGraph); +} + +TEST_F(HttpHandlersTest, StateBasedRenderingReportsTheMissingSamplingVariable) { + // The two sources fail differently, and the message must say which one was + // asked for: the state-based path needs TCMALLOC_SAMPLE_PARAMETER before + // process start, the window-based one does not need it at all. + // + // These tests run without that variable set, so getRawHeapSample() yields + // nothing while the window path can still collect. + profiler::ProfilerManager plain; + + ASSERT_TRUE(plain.getRawHeapSample().empty()) << "precondition: sampling is off in the test process"; + + profiler::ProfilerHttpHandlers h(plain); + auto state = h.handleHeapSnapshot(opts(1), /*as_profile=*/false); + EXPECT_EQ(state.status, 500); + EXPECT_NE(state.body.find("TCMALLOC_SAMPLE_PARAMETER"), std::string::npos) << state.body; +} + +TEST_F(HttpHandlersTest, WindowBasedRenderingDoesNotMentionTheVariable) { + profiler::ProfilerManager plain; + profiler::ProfilerHttpHandlers h(plain); + + // Duration is clamped, and the window path must not blame the environment + // variable it does not depend on. + auto resp = h.handleHeapChart(opts(0)); + if (resp.status == 500) { + EXPECT_EQ(resp.body.find("TCMALLOC_SAMPLE_PARAMETER"), std::string::npos) + << "window-based failure must not blame a variable it does not need: " << resp.body; + } } // --------------------------------------------------------------------------- @@ -435,6 +500,40 @@ TEST(ProfilerLifecycleTest, StopHeapProfilerReportsState) { EXPECT_FALSE(profiler.isProfilerRunning(profiler::ProfilerType::HEAP)); } +TEST(ProfilerLifecycleTest, StopHeapProfilerWritesRealProfileContent) { + // The state assertions above would pass even if the profile file came out + // empty. stopHeapProfiler() gets its data by dumping to disk and reading that + // file back (GetHeapProfile()'s buffer has no portable deallocator), so the + // path is worth checking end to end: the derived dump name has to be found and + // its contents written to the caller's requested path. + profiler::ProfilerManager profiler; + + const std::string output = "/tmp/test_stop_heap_content.prof"; + std::remove(output.c_str()); + + ASSERT_TRUE(profiler.startHeapProfiler(output)); + + // Allocate while the profiler runs, so there is something to record. + { + std::vector> live; + for (int i = 0; i < 64; ++i) + live.emplace_back(64 * 1024, static_cast(i)); + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + } + + ASSERT_TRUE(profiler.stopHeapProfiler()); + + std::ifstream file(output, std::ios::binary); + ASSERT_TRUE(file.is_open()) << "profile not written to " << output; + const std::string content((std::istreambuf_iterator(file)), std::istreambuf_iterator()); + + EXPECT_FALSE(content.empty()) << "profile file is empty"; + // A gperftools heap profile starts with a header line naming the format. + EXPECT_NE(content.find("heap profile:"), std::string::npos) << "unexpected content: " << content.substr(0, 120); + + std::remove(output.c_str()); +} + TEST(ProfilerLifecycleTest, StartingHeapProfilerTwiceFails) { profiler::ProfilerManager profiler; @@ -494,7 +593,7 @@ TEST(ProfilerLifecycleTest, StaleHeapProfileIsNotReused) { } }); - (void)profiler.analyzeHeapProfile("pprof"); + (void)profiler.analyzeHeapProfile(1, "pprof"); const auto afterFirst = heapFiles(); auto newSince = [&](const std::set& a, const std::set& b) { diff --git a/tests/test_result_parsing.cpp b/tests/test_result_parsing.cpp new file mode 100644 index 0000000..0805d0a --- /dev/null +++ b/tests/test_result_parsing.cpp @@ -0,0 +1,125 @@ +/// @file test_result_parsing.cpp +/// @brief Tests for interpreting profiler-core return values +/// +/// These cover a class of bug that reached users: a failure the HTTP layer could +/// not recognise, so it either reported success or replaced the reason with a +/// generic message. + +#include "../src/internal/result_parsing.h" +#include +#include + +using profiler::internal::isEmptyHeapSample; +using profiler::internal::isJsonError; +using profiler::internal::jsonErrorMessage; + +// --------------------------------------------------------------------------- +// isJsonError / jsonErrorMessage +// --------------------------------------------------------------------------- + +TEST(JsonErrorTest, RecognisesCoreErrorStrings) { + EXPECT_TRUE(isJsonError(R"({"error": "Failed to start CPU profiler"})")); + EXPECT_TRUE(isJsonError(R"({"error": "cpu profiling already in use"})")); + EXPECT_TRUE(isJsonError(R"({"error": "No heap profile data was produced. Details here."})")); +} + +TEST(JsonErrorTest, DoesNotMistakeSvgOrTextForAnError) { + EXPECT_FALSE(isJsonError("")); + EXPECT_FALSE(isJsonError("")); + EXPECT_FALSE(isJsonError("")); + EXPECT_FALSE(isJsonError(R"({"err": "wrong key"})")) << "only {\"error\" is an error payload"; + EXPECT_FALSE(isJsonError("heap profile: 1: 20000 @ heap_v2/524288")); + EXPECT_FALSE(isJsonError("{}")); +} + +TEST(JsonErrorTest, ExtractsTheInnerMessage) { + // The exact shape the core emits, including an embedded apostrophe. + EXPECT_EQ(jsonErrorMessage(R"({"error": "cpu profiling already in use"})"), "cpu profiling already in use"); + EXPECT_EQ(jsonErrorMessage(R"({"error": "Failed to start CPU profiler"})"), "Failed to start CPU profiler"); +} + +TEST(JsonErrorTest, ExtractedMessageRoundTripsThroughAnotherWrap) { + // Regression guard for the double-wrap bug: the old code passed the whole + // JSON string to HandlerResponse::error(), producing + // {"error":"{"error": "..."} + // which is not valid JSON. Re-wrapping the *message* must stay parseable. + const std::string core_error = R"({"error": "cpu profiling already in use"})"; + const std::string message = jsonErrorMessage(core_error); + const std::string wrapped = "{\"error\":\"" + message + "\"}"; + + EXPECT_EQ(wrapped.find_first_of('{', 1), std::string::npos) + << "wrapped response must contain a single brace group: " << wrapped; + EXPECT_EQ(wrapped, R"({"error":"cpu profiling already in use"})"); +} + +TEST(JsonErrorTest, KeepsLongMessagesIntact) { + // The heap "no data" message is long and contains punctuation and a URL-ish + // word; it must survive extraction unchanged. + const std::string core_error = + R"({"error": "No heap profile data was produced. The heap profiler only records samples for allocations made while it is running; try /tmp/cpp_profiler."})"; + const std::string message = jsonErrorMessage(core_error); + EXPECT_EQ(message.rfind("No heap profile data was produced.", 0), 0); + EXPECT_NE(message.find("try /tmp/cpp_profiler."), std::string::npos); + EXPECT_EQ(message.back(), '.'); +} + +TEST(JsonErrorTest, FallsBackToTheInputWhenUnparseable) { + // Defensive: a malformed string is returned as-is rather than truncated to + // something meaningless. + const std::string odd = "{\"err\": \"no error key here\"}"; + EXPECT_EQ(jsonErrorMessage(odd), odd); + EXPECT_EQ(jsonErrorMessage("not json at all"), "not json at all"); +} + +// --------------------------------------------------------------------------- +// isEmptyHeapSample +// --------------------------------------------------------------------------- + +TEST(EmptyHeapSampleTest, DetectsWhatTcmallocEmitsWhenSamplingIsOff) { + // Verbatim shape produced by GetHeapSample() with TCMALLOC_SAMPLE_PARAMETER + // unset: a %warn advisory that still looks like a valid profile, which is why + // the old `empty()` check never fired. + const std::string off = "%warn\n" + "%warn This heap profile does not have any data in it, because\n" + "%warn the application was run with heap sampling turned off.\n" + "%warn To get useful data from GetHeapSample(), you must\n" + "%warn set the environment variable TCMALLOC_SAMPLE_PARAMETER to\n" + "%warn a positive sampling period, such as 524288.\n" + "%warn\n" + "heap profile: 0: 0 [ 0: 0] @ heap_v2/0\n" + "\n" + "MAPPED_LIBRARIES:\n" + "00400000-0040b000 r-xp 00000000 00:00 535369 /tmp/app\n"; + + EXPECT_FALSE(off.empty()) << "precondition: this is why empty() was not enough"; + EXPECT_TRUE(isEmptyHeapSample(off)); +} + +TEST(EmptyHeapSampleTest, DetectsZeroRateEvenWithoutTheWarning) { + // Some paths emit just the zero-rate header; the marker alone must suffice. + EXPECT_TRUE(isEmptyHeapSample("heap profile: 0: 0 [ 0: 0] @ heap_v2/0\n")); +} + +TEST(EmptyHeapSampleTest, DetectsTheEmptyString) { + EXPECT_TRUE(isEmptyHeapSample("")); +} + +TEST(EmptyHeapSampleTest, AcceptsARealSample) { + const std::string real = "heap profile: 1: 20000 [ 1: 20000] @ heap_v2/524288\n" + " 1: 20000 [ 1: 20000] @ 0x401d8a 0x4080cc\n" + "\n" + "MAPPED_LIBRARIES:\n"; + + EXPECT_FALSE(isEmptyHeapSample(real)); +} + +TEST(EmptyHeapSampleTest, AProfileCarryingTheZeroRateMarkerIsRejectedWhereverItAppears) { + // The marker is searched for anywhere in the text, not only at the start: + // a profile whose rate line says heap_v2/0 has no samples regardless of what + // else it carries, and pprof would report it as empty too. + const std::string zero_rate_with_frames = "heap profile: 0: 0 [ 0: 0] @ heap_v2/0\n" + "MAPPED_LIBRARIES:\n" + "00400000-0040b000 r-xp 00000000 00:00 535369 /tmp/app\n"; + + EXPECT_TRUE(isEmptyHeapSample(zero_rate_with_frames)); +} diff --git a/tests/test_thread_stacks.cpp b/tests/test_thread_stacks.cpp new file mode 100644 index 0000000..59f6714 --- /dev/null +++ b/tests/test_thread_stacks.cpp @@ -0,0 +1,171 @@ +/// @file test_thread_stacks.cpp +/// @brief Tests for thread stack capture and the /api/thread/stacks output +/// +/// Only depends on profiler_core (no Drogon), so it runs in the minimal +/// configuration as well. + +#include "../include/profiler/http_handlers.h" +#include "../include/profiler_manager.h" +#include +#include +#include +#include +#include + +namespace { + +/// A thread parked on a condition variable, i.e. genuinely blocked. +/// +/// The capture is signal-driven, so a thread has to exist and be waiting for the +/// test to have something to find. +class BlockedThread { +public: + BlockedThread() + : worker_([this]() { + std::unique_lock lock(mutex_); + // Announce readiness before parking, so the test cannot + // capture while this thread is still starting up and get + // an empty result that passes by accident. + ready_ = true; + ready_cv_.notify_all(); + cv_.wait(lock, [this]() { return release_; }); + }) { + std::unique_lock lock(mutex_); + ready_cv_.wait(lock, [this]() { return ready_; }); + } + + ~BlockedThread() { + { + std::lock_guard lock(mutex_); + release_ = true; + } + cv_.notify_all(); + if (worker_.joinable()) + worker_.join(); + } + + BlockedThread(const BlockedThread&) = delete; + BlockedThread& operator=(const BlockedThread&) = delete; + +private: + std::mutex mutex_; + std::condition_variable cv_; + std::condition_variable ready_cv_; + bool release_ = false; + bool ready_ = false; + std::thread worker_; +}; + +} // namespace + +TEST(ThreadStacksTest, CaptureProducesASymbolisedHeader) { + profiler::ProfilerManager profiler; + + const std::string stacks = profiler.getThreadCallStacks(); + + ASSERT_FALSE(stacks.empty()) << "capture produced nothing"; + EXPECT_NE(stacks.find("Thread Call Stacks"), std::string::npos); + EXPECT_NE(stacks.find("Total threads captured:"), std::string::npos); +} + +TEST(ThreadStacksTest, ReportsThreadNamesAlongsideTids) { + // A bare tid is unreadable after the fact; the name is what identifies the + // thread. Linux supplies it via /proc//comm and the kernel truncates it + // to 15 characters, which is what ps and top show too. + profiler::ProfilerManager profiler; + BlockedThread parked; + + const std::string stacks = profiler.getThreadCallStacks(); + ASSERT_FALSE(stacks.empty()); + // A capture that found nothing would still contain the header, so require a + // real capture before checking its shape. + ASSERT_EQ(stacks.find("Total threads captured: 0"), std::string::npos) << "capture found no threads at all:\n" + << stacks; + + // Find a "Thread ():" line and check the name is plausible. + const std::string marker = "Thread "; + std::size_t pos = stacks.find(marker); + ASSERT_NE(pos, std::string::npos); + + bool found_named_thread = false; + while (pos != std::string::npos) { + const std::size_t open = stacks.find('(', pos); + const std::size_t close = stacks.find(')', pos); + const std::size_t line_end = stacks.find('\n', pos); + // The parentheses must belong to this header line (i.e. precede any colon + // that ends it), otherwise we are looking at an unrelated bracket later. + if (open != std::string::npos && close != std::string::npos && close < line_end && open < close) { + const std::string name = stacks.substr(open + 1, close - open - 1); + if (!name.empty() && name.size() <= 15) { + found_named_thread = true; + break; + } + } + pos = stacks.find(marker, pos + marker.size()); + } + EXPECT_TRUE(found_named_thread) << "no 'Thread ():' header found in:\n" << stacks; +} + +TEST(ThreadStacksTest, BlockedThreadsReportTheirBlockingFrame) { + // The point of the endpoint is "where is each thread stuck", and the answer + // is the first frame that is not part of the capture machinery itself. + profiler::ProfilerManager profiler; + BlockedThread parked; + + const std::string stacks = profiler.getThreadCallStacks(); + ASSERT_FALSE(stacks.empty()); + ASSERT_EQ(stacks.find("Total threads captured: 0"), std::string::npos) << "capture found no threads at all:\n" + << stacks; + + const char* machinery[] = {"signalHandler", "__restore_rt", "__syscall_cancel", "__internal_syscall_cancel", + "pthread_kill", "raise"}; + bool found_real_frame = false; + std::size_t pos = 0; + while ((pos = stacks.find(" #", pos)) != std::string::npos) { + const std::size_t eol = stacks.find('\n', pos); + const std::string frame = stacks.substr(pos, eol - pos); + bool is_machinery = false; + for (const char* m : machinery) { + if (frame.find(m) != std::string::npos) { + is_machinery = true; + break; + } + } + if (!is_machinery) { + found_real_frame = true; + break; + } + pos = eol; + } + EXPECT_TRUE(found_real_frame) << "every frame looked like capture machinery:\n" << stacks; +} + +TEST(ThreadStacksTest, HandlerServesTheCapture) { + profiler::ProfilerManager profiler; + profiler::ProfilerHttpHandlers handlers(profiler); + + BlockedThread parked; + const auto resp = handlers.handleThreadStacks(); + + ASSERT_EQ(resp.status, 200); + EXPECT_NE(resp.body.find("Thread Call Stacks"), std::string::npos); + EXPECT_EQ(resp.body.find("Total threads captured: 0"), std::string::npos) << "endpoint captured no threads:\n" + << resp.body; + + // Thread names must survive the handler too, since this is the endpoint the + // panel calls. Assert the actual "Thread ():" shape rather than + // merely that some parenthesis exists somewhere. + bool named = false; + for (std::size_t pos = resp.body.find("Thread "); pos != std::string::npos; + pos = resp.body.find("Thread ", pos + 1)) { + const auto open = resp.body.find('(', pos); + const auto close = resp.body.find(')', pos); + const auto line_end = resp.body.find('\n', pos); + if (open != std::string::npos && close != std::string::npos && close < line_end && open < close && + close > open + 1) { + named = true; + break; + } + } + EXPECT_TRUE(named) << "no named thread header in:\n" << resp.body; +}