[Bug]: Boost corrupts non-TTY/piped/redirected output → silent failures in scripts

[sverdlov93]

Summary

Boost mutates a wrapped command's stdout (multi-line compression + a human-readable [Boost compressed N lines … boost retrieve <id>] marker) even when stdout is not a TTY — i.e. when it is piped, redirected to a file, or captured in a command substitution $(...). This silently corrupts output that another process consumes, with no error and no signal, breaking ordinary shell scripting and CLI automation.

Separate from #42 (the docker shim argv bug), but same root theme: boost altering a command's behavior in a context where it shouldn't.

Impact (real incident)

While investigating a CI issue I ran a standard log-fetch loop:

for jid in $(gh api repos/OWNER/REPO/actions/runs/$RUNID/jobs --jq '.jobs[].id'); do
  gh api repos/OWNER/REPO/actions/jobs/$jid/logs > logs/$jid.log
done
# -> fetched 0 logs. No error. Silent.

It produced zero usable logs. There was no error message and no indication boost was involved. I only found the cause by re-running the inner command with DISABLE_BOOST=1, which showed the data was actually there. Over the rest of the investigation I had to prefix 123 gh/git data-fetch commands with DISABLE_BOOST=1 to get reliable output (full inventory below).

Failure modes

When the wrapped command's stdout feeds another consumer, boost's compression/marker corrupts it:

• Command substitution — for x in $(gh ... --jq '.jobs[].id') → the value list is replaced/garbled by the marker → the loop iterates nothing.
• Redirection to a file — gh ... /logs > file → the file receives the [Boost compressed …] marker instead of the actual log → every downstream grep/parse finds nothing.
• jq / structured pipelines — gh ... --json ... --jq ... → parsed fields no longer match the real output.

Root cause

Boost applies its interactive output transformation unconditionally, regardless of whether stdout is a terminal. Compression and the human-readable marker are only meaningful for an interactive reader; when stdout is a pipe/file/substitution they are pure corruption.

Expected behavior

1. Auto-detect non-TTY stdout and pass through raw. If isatty(stdout) is false (piped, redirected, captured), boost must emit the original bytes unmodified — no compression, no marker.
2. Never inject the [Boost compressed …] marker into a byte stream another process consumes. The marker belongs only in interactive/agent-terminal output.
3. Fail loud, not silent. If boost ever alters output that turns out to be consumed programmatically, there should be a detectable signal rather than silently empty/garbled results.

Why the existing opt-out is insufficient

DISABLE_BOOST=1 <command> works, but you only reach for it after a confusing silent failure — by definition you don't know boost is the cause until you've already lost time debugging. Default-safe pass-through on non-TTY stdout removes the need for the per-command opt-out entirely.

---

Appendix — complete inventory of boost-disabled commands

During this single investigation I disabled boost 123 times. Every case is a command whose stdout was consumed programmatically (command substitution, pipe to jq/sort/grep, or redirect to a file). Breakdown by command shape (from the session transcript):

Command shape	Count	Consumption pattern
gh api .../actions/runs/...	28	--jq + $(...)
gh run view ... --json ... --jq	25	$(...) / JSON parse
gh run list ... --json ... --jq	23	$(...) / | sort | uniq -c
gh api .../actions/jobs/$id/logs	22	redirect > file, then grep/sed
gh api .../runs/$id/jobs --jq	3	$(...) loop over job IDs
gh api .../actions/runners --jq	1	JSON parse
gh api .../fly-desktop/.../jobs --jq	1	JSON parse
gh api .../boost/contents/...	1	JSON parse
gh issue create / gh issue comment / gh label list	4	URL/label parse
git log ... --pretty=...	4	parse commit dates/SHAs
git grep -nE ...	1	parse matches
echo "DISABLE_BOOST=1" >> $GITHUB_ENV	1	tested as a mitigation in CI — did not fix the shim

Verbatim examples

1. The silent-failure loop (WITHOUT boost) — fetched 0 logs, no error:

for jid in $(gh api repos/jfrog/fly-service/actions/runs/$RUNID/jobs --paginate --jq '.jobs[].id'); do
  gh api repos/jfrog/fly-service/actions/jobs/$jid/logs 2>/dev/null > mx/$jid.log
done
# → ls mx/*.log: no matches; downstream grep found nothing

2. Same loop WITH DISABLE_BOOST=1 — works correctly:

for jid in $(DISABLE_BOOST=1 gh api repos/jfrog/fly-service/actions/runs/$RUNID/jobs --jq '.jobs[].id'); do
  DISABLE_BOOST=1 gh api repos/jfrog/fly-service/actions/jobs/$jid/logs > mx/$jid.log 2>/dev/null
done

3. Polling run status (command substitution):

runid=$(DISABLE_BOOST=1 gh run list --repo jfrog/fly-service \
  --workflow 'Runner Docker Diagnostics' --limit 1 --json databaseId --jq '.[0].databaseId')

st=$(DISABLE_BOOST=1 gh run view "$runid" --repo jfrog/fly-service --json status --jq '.status')

4. Structured JSON extraction (jq):

DISABLE_BOOST=1 gh run view "$runid" --repo jfrog/fly-service \
  --json headSha,conclusion --jq '{sha:.headSha[0:8],conclusion}'

DISABLE_BOOST=1 gh run view "$runid" --repo jfrog/fly-service \
  --json jobs --jq '.jobs[] | "\(.conclusion) - \(.name)"'

5. Failure-rate tally (pipe to sort | uniq):

DISABLE_BOOST=1 gh run list --repo jfrog/fly-service \
  --workflow ai-evaluation-tests.yml --branch master --limit 40 \
  --json conclusion --jq '.[].conclusion' | sort | uniq -c

6. git history parsing:

DISABLE_BOOST=1 git log --oneline -15 --date=short --pretty='%h %ad %s'
DISABLE_BOOST=1 git log --diff-filter=A --date=short --pretty='%ad %s' -- <path>
DISABLE_BOOST=1 git grep -nE '<pattern>'

7. Mitigation test that FAILED (for cross-reference with #42):

# exported into a GitHub Actions job before jfrog/fly-action@v1
- run: echo "DISABLE_BOOST=1" >> "$GITHUB_ENV"
# fly-action still failed — the runner launches actions via boost_node and the
# /tmp/boost/tools PATH shims remain in effect, so the env toggle never reached
# the spawned process. (See #42.)

Key takeaways for triage

• The corruption is silent — empty/garbled results with exit code 0 and no stderr.
• It affects every non-interactive consumption path: $(...), | jq, | sort, > file.
• DISABLE_BOOST=1 is a reliable per-command workaround but is only discoverable after a failure.
• The single highest-value fix: pass through unmodified when stdout is not a TTY.

Environment

• Boost binary: /tmp/boost/tools/boost-arm64, statically-linked ELF aarch64, BuildID 397bf566827c6454219a2eba05fcdec8c27c2b2c (rebuilt 2026-06-18)
• Wrapped commands affected in practice: gh api, gh run, gh issue, gh label, git log, git grep