Skip to content

Commit e5a6795

Browse files
committed
ci(macrobenchmark): Decode Sauce's JSON-lines device log
The recovery worked on the device but not in the parser. Sauce returns device.log as JSON lines -- {"tag", "message", "level", ...} -- so the payload arrives with its quotes escaped and cannot be regexed straight out of the raw line. Decode each entry and match against its message, falling back to the raw line so the same parser still handles `adb logcat` output from a local run. Enable pipefail in the workflow step too. Piping into tee meant the step exited on tee's status, so this very failure reported success while producing no results. Verified against the device.log from run 31117974318: timeToInitialDisplayMs min 443.5 / median 477.3 / max 571.2 over 12 iterations on a Pixel 9 Pro XL.
1 parent 4f26d59 commit e5a6795

2 files changed

Lines changed: 25 additions & 3 deletions

File tree

.github/workflows/integration-tests-macrobenchmark.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ jobs:
7070
- name: Recover benchmark results from the device log
7171
if: always() && env.SAUCE_USERNAME != null
7272
run: |
73+
# Without pipefail the step passes on `tee`'s exit code, so a failed recovery
74+
# would report success while silently producing no results.
75+
set -o pipefail
7376
python3 scripts/parse-macrobenchmark-log.py ./artifacts \
7477
--json-out ./artifacts/benchmarkData.json | tee -a "$GITHUB_STEP_SUMMARY"
7578

scripts/parse-macrobenchmark-log.py

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,33 @@
1919
CHUNK_RE = re.compile(r"SentryBenchmarkData\s*:\s*\[(\d+)/(\d+)\](.*)$")
2020

2121

22+
def log_messages(log_file):
23+
"""Yields the message text of every log entry.
24+
25+
Sauce hands back device.log as JSON lines -- {"tag", "message", "level", ...} -- which
26+
means the payload arrives with its quotes escaped, so it has to be decoded rather than
27+
regexed out of the raw line. Plain-text lines are passed through unchanged so the same
28+
parser works on `adb logcat` output from a local run.
29+
"""
30+
# Sauce device logs occasionally carry undecodable bytes; don't die on them.
31+
for line in log_file.read_text(errors="replace").splitlines():
32+
line = line.strip()
33+
if line.startswith("{"):
34+
try:
35+
yield json.loads(line).get("message", "")
36+
continue
37+
except json.JSONDecodeError:
38+
pass
39+
yield line
40+
41+
2242
def collect_chunks(log_files):
2343
"""Returns the chunk texts keyed by index, plus the expected total."""
2444
chunks = {}
2545
total = None
2646
for log_file in log_files:
27-
# Sauce device logs occasionally carry undecodable bytes; don't die on them.
28-
for line in log_file.read_text(errors="replace").splitlines():
29-
match = CHUNK_RE.search(line)
47+
for message in log_messages(log_file):
48+
match = CHUNK_RE.search(message)
3049
if not match:
3150
continue
3251
index, chunk_total, text = int(match.group(1)), int(match.group(2)), match.group(3)

0 commit comments

Comments
 (0)