fix(download): fix download speed/ETA accuracy (sliding window estimator + XetPoller scan targets)

[mmogr]

Closes #525

Problem

Download speed and ETA indicators in both the GUI and CLI fluctuated wildly. After implementing the sliding window estimator (see below), the symptom persisted: system monitor showed a stable 90 MB/s, but gglib displayed values that jumped from 4 → 250 → 50 → 1000 → 30 MB/s.

Two separate bugs were found and fixed.

---

Bug 1 — Slow/wrong estimator algorithm

Both the GUI bridge task and the CLI progress bars used an exponentially-weighted average with α=0.02. At a 250 ms tick interval that gives a ~34-tick half-life (~8.5 s convergence lag), and speed initialises at 0.0 so it takes ~28 s to reach 90% of the true value. Two independent copies of this bug existed in the codebase.

A second issue: indicatif's {bytes_per_sec} and {eta} placeholders are computed internally by indicatif from set_position() calls and cannot be overridden — so even if the bridge task sent correct speed/ETA values, the CLI bars would still show indicatif's own fluctuating estimate.

Bug 2 — XetPoller scanning the wrong filesystem locations (root cause of spikes)

When hf_xet is active, huggingface_hub bypasses tqdm.update() entirely, so the Python subprocess never emits Progress events. XetPoller compensates by stat-polling a list of target paths and reporting the total bytes on disk every 250 ms.

Two problems with how those targets were chosen:

1. Atomic rename spike — targets included dest/{filename} (the final output file). When a transfer completes, huggingface_hub atomically renames .incomplete → final file. That makes the full file size appear in one 250 ms tick: e.g. a 5 GB file produces a momentary 5 GB / 0.25 s = 20 GB/s reading. The SlidingWindowRate then spends several seconds averaging this spike away while displaying wild intermediate values.

2. Global HF cache not scanned — hf_xet frequently writes blobs to the system-wide hub cache (~/.cache/huggingface/hub/models--owner--repo/blobs/) rather than the local dest/.cache staging directory. When that path wasn't in the target list, XetPoller saw 0 bytes for the first ~30 s of a download (explaining the "stuck at 4 MB/s" observation), then received the full blob size in one tick when the rename fired.

---

Fix

Shared SlidingWindowRate struct (progress/rate.rs) — retains timestamped byte-count samples over an 8-second window and reports Δbytes / Δtime across that window. This matches what system monitors show, requires no tuning constants, and is accurate from the second sample onward.

GUI / SSE path (manager/mod.rs) — the bridge task now calls rate.record() + rate.speed_bps() on every tick. DownloadEvent::progress() in gglib-core derives eta_seconds from speed_bps, so GUI ETA is also fixed with no changes to the core crate.

CLI indicatif bypass (cli_emitter.rs, cli_exec/exec/progress.rs) — {bytes_per_sec} and {eta} slots removed from all templates and replaced with {prefix}. Each bar now holds a SlidingWindowRate; speed and ETA are formatted as "X.XX MB/s eta MM:SS" and written via bar.set_prefix(), completely bypassing indicatif's internal estimator. set_position() is still called so {bytes} / {total_bytes} / {bar} continue to work correctly.

PlainProgress (non-TTY) now also emits ETA — it was previously omitted entirely.

XetPoller scan targets (cli_exec/exec/python_bridge.rs, xet_poller.rs):

• Removed per-file final-path targets (eliminates the rename spike)
• Added the global HF hub blob directory for the repo as a second target, resolving via HF_HUB_CACHE → HUGGINGFACE_HUB_CACHE → HF_HOME/hub → $HOME/.cache/huggingface/hub (same priority as Python's huggingface_hub); missing paths are silently skipped
• Made the high-water-mark check strictly monotonic: if polled bytes decrease (e.g. mid-rename), last_bytes resets to 0 so the next genuine increase fires immediately

---

Commits

1. feat(download): add SlidingWindowRate and format_eta to progress module — new rate.rs with full unit test suite
2. fix(download): replace EWA bridge task speed with SlidingWindowRate — manager/mod.rs; removes calculate_speed_eta helper
3. fix(download): bypass indicatif estimator in CliDownloadEventEmitter — cli_emitter.rs
4. fix(download): replace EWA in FancyProgress and PlainProgress — cli_exec/exec/progress.rs
5. docs(download): document SlidingWindowRate and format_eta in progress README
6. fix(download): resolve all strict clippy warnings
7. fix(download): fix XetPoller scan targets to eliminate speed spikes — root cause fix for hf-xet downloads

[mmogr]

Added follow-up fix in 04a4e1b to address residual speed spikes reported after the prior root-cause patch.

What changed:

• XetPoller directory scans now count only in-flight *.incomplete files
• Synthetic bytes are baseline-relative (current - startup) so pre-existing cache artifacts don't pollute current-request progress
• Synthetic downloaded is clamped to expected total when known

Validation:

• cargo test --package gglib-download (109 passed)
• cargo clippy --package gglib-download -- -D warnings

If speed still fluctuates after this, I can add temporary debug telemetry to log raw poller samples (raw sum, baseline, emitted bytes, and per-tick delta) for one reproducer run.

[mmogr]

Small correction: I accidentally opened then deleted a pending review while posting follow-up notes. No code or review state changed; the intended status update is the latest comment plus commit 04a4e1b.

[mmogr]

Consolidated summary (author update): residual fluctuation fix landed in commit 04a4e1b, now in branch. Key change is that synthetic progress uses baseline-relative growth of in-flight .incomplete files only, rather than absolute cache-directory size.