Raw RGBA Matplotlib frames over DataChannels; canvaskit web default; restore flet-charts tests#6673
Raw RGBA Matplotlib frames over DataChannels; canvaskit web default; restore flet-charts tests#6673FeodorFitsner wants to merge 5 commits into
Conversation
Skip per-frame diffing and PNG encode/decode when the client runs on the same machine: MatplotlibChart now streams uncompressed RGBA full frames (opcode 0x04) straight from Agg's buffer, displayed with a single decodeImageFromPixels + swap + dispose on the Dart side. Remote WebSocket clients keep the compact PNG full+diff pipeline. - Add Connection.local_data_transport capability flag, set by socket, dart_bridge and Pyodide transports; MatplotlibChart auto-selects the frame format from it - Fix O(n^2) length-prefixed packet reassembly in the Dart socket transport (flattened the accumulation buffer on every incoming chunk; multi-MB frames arrive in dozens of chunks) - Single-copy DataChannel frame framing in FletSocketServer - Default flet build web / flet publish renderer to canvaskit: with "auto" Chromium selects dart2wasm/skwasm whose JS <-> Dart typed-data boundary costs make byte-streaming Pyodide apps ~6-7x slower per frame; also let tool.flet.web.renderer take effect in flet publish (argparse default shadowed it) - three_d example: report raw frames in the stats bar and stop the refresh loop when the session is destroyed three_d figure at 1600x1000 @ DPR 2 (24 MB/frame, local socket): 7.4 fps (PNG diff) -> 18.7 fps (raw), matplotlib's own render is now the dominant per-frame cost.
…removal - New guide: Android site-packages ship zipped; path-hungry packages (matplotlib, scikit-learn) need extract_packages to be shipped extracted to disk - New guide: x86 removed from Android target architectures; --arch x86 now fails the build upfront - Register both in the breaking-changes index and sidebar; add the missing index link for the app-files-unpacked guide - Cross-link the guides from the Android publishing docs
…vaskit docs The flet-charts integration tests have been broken since the examples reorg (055cd09) deleted the importable example modules they used — they failed at collection, so golden screenshot regressions went undetected. Tests: - Add example_apps.py loader that imports an example app's main.py by path, and repoint all 8 test modules at the relocated apps under examples/extensions/charts (old example_N names mapped by content: e.g. bar_chart/example_1 -> interactive_bar_chart) - test_three_d: use a full-page screenshot like the sibling tests; the page-controls capture path measures intrinsic dimensions, which the LayoutBuilder-based matplotlib canvas cannot provide - Regenerate all macOS goldens, each visually verified; the matplotlib tests now exercise the raw RGBA frame pipeline on every run - Ignore *_actual.png diagnostic screenshots Examples — the reorg wrapped every chart in a SafeArea without expand=True, so charts rendered blank (layout exception) or shrunken: - Add the missing expand chain in bar_chart x2, line_chart x2, pie_chart x3, plotly_chart x4, radar_chart, scatter_chart and candlestick_chart examples - three_d: make the stats bar scrollable (was overflowing narrow windows), count raw frames in it, and stop the refresh loop when the session is destroyed on window close Docs — record the canvaskit web renderer default: publish guide (with rationale), --web-renderer CLI help, WebRenderer docstrings.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Deploying flet-website-v2 with
|
| Latest commit: |
508a48b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f6964b6c.flet-website-v2.pages.dev |
| Branch Preview URL: | https://fast-mpl.flet-website-v2.pages.dev |
|
|
||
| ```toml | ||
| [tool.flet.android] | ||
| extract_packages = ["matplotlib", "scikit-learn"] |
There was a problem hiding this comment.
From my testing, the entry is the package's import name — its top-level directory in site-packages — not the PyPI/distribution name. So sklearn, not scikit-learn; cv2, not opencv-python. (matplotlib matches only because its import name happens to equal its PyPI name)
| extract_packages = ["matplotlib", "scikit-learn"] | |
| extract_packages = ["matplotlib", "sklearn"] |
Other packages that need an entry:
| package | entry | why |
|---|---|---|
astropy |
"astropy" |
reads astropy/CITATION via __file__ at import |
thinc |
"thinc" |
reads thinc/backends/_custom_kernels.cu via __file__ at import |
spacy |
"spacy", "thinc" |
imports thinc at load (so it hits thinc's _custom_kernels.cu) and reads its own lang data via __file__ — list both |
opencv-python |
"cv2" |
cv2's bootstrap resolves its config files and loads its native extension through __file__-relative paths, so it must ship as a real directory |
There was a problem hiding this comment.
Fixed in 3101832 — examples now use sklearn, added an explicit note that entries are import names (top-level directory under site-packages) rather than PyPI distribution names, and folded your tested packages (cv2, astropy, thinc, spacy+thinc) into the known-packages table. Thanks for testing these!
| instead; x86 images are only published for long-obsolete API levels. | ||
|
|
||
| Note that the architectures available for a build also depend on the | ||
| [bundled Python version](../../../publish/index.md#bundled-python) — see |
There was a problem hiding this comment.
#bundled-python exists? Checked on my end, but cant find it.
Also, the comment tries to point to a section (at least, from my understanding) which could be showing "android abi against python version table", which we dont currently have. Could be worth adding such a table or mentioning in some other form which abis each python version supports.
There was a problem hiding this comment.
Good catch — the anchor never existed; the section is #choosing-a-python-version (same broken link was in publish/android.md too). Fixed both in 3101832 and inlined an ABI × Python version support matrix in the x86 guide, complementing the per-ABI list in android.md#supported-target-architectures. Also fixed a typo'd NDK anchor (#86-64 → #x86-64) while there.
There was a problem hiding this comment.
Correction in 508a48b: the ABI matrix I added (and the pre-existing "armeabi-v7a — Python 3.12 only" / PEP 738 note in android.md it was based on) was wrong — flet-dev/python-build publishes all three ABIs including armeabi-v7a for every supported Python version (verified against the 20260708 manifest). Both docs now state the same thing: all three ABIs, every bundled Python version; upstream's PEP 738 32-bit drop doesn't apply to Flet's own CPython builds.
- extract_packages entries are import names (top-level directory under site-packages), not PyPI distribution names: sklearn, not scikit-learn. Say so explicitly and fix the examples. - Extend the known-packages table with tested entries: cv2 (opencv-python), astropy, thinc, spacy. - Fix broken publish/index.md#bundled-python anchors (the section is #choosing-a-python-version) in the x86 guide and android.md, and inline an ABI x Python version support matrix in the x86 guide. - Fix typo'd NDK ABI anchor (#86-64 -> #x86-64) in android.md.
flet-dev/python-build publishes arm64-v8a, x86_64 and armeabi-v7a distributions for every supported Python version (verified against the 20260708 manifest) — the "Python 3.12 only" restriction and the derived PEP 738 note in android.md were wrong, and the x86 migration guide repeated them. State the same thing in both places: all three ABIs, every bundled Python version; upstream's PEP 738 drop does not apply to Flet's own CPython builds.
2 participants
Summary
Fast Matplotlib rendering path: when the client runs on the same machine (embedded native, Pyodide, dev socket),
MatplotlibChartnow streams uncompressed RGBA full frames straight from Agg's buffer over the DataChannel — no numpy diffing, no PNG encode/decode — displayed with a singledecodeImageFromPixels+ swap + dispose. Remote WebSocket clients keep the compact PNG full+diff pipeline (auto-selected per connection via a newConnection.local_data_transportcapability flag).Wire format
New opcode on the existing widget DataChannel:
[0x04][width u32 LE][height u32 LE][premultiplied RGBA8888]. Ack/backpressure unchanged; unknown opcodes are now acked so a newer Python never hangs an older client.Transport fixes found along the way
FletSocketServer.Benchmarks (three_d figure, 1600×1000 logical @ DPR 2 → 24 MB/frame, local socket)
matplotlib's own render is now the dominant per-frame cost.
canvaskit web renderer default
With
auto, Chromium browsers select the dart2wasm/skwasm build, where every JS ↔ Dart byte-buffer crossing pays a WasmGC boundary conversion instead of a memcpy. Measured on a Pyodide app: 67 ms vs 10 ms per-frame display (Edge/skwasm vs Safari/canvaskit fallback) for the same 24 MB raw frames. Flet web apps exchange bytes with the Python runtime on every UI update, soflet build web/flet publishnow default tocanvaskit(override via--web-rendereror[tool.flet.web].renderer, whichflet publishpreviously ignored due to an argparse default shadowing it). Documented in the publish guide, CLI help andWebRendererdocstrings.Restored flet-charts integration tests
The suite had been failing at collection since the examples reorg deleted the modules it imported — golden regressions went undetected. Tests now load the relocated example apps by path, all macOS goldens are regenerated and visually verified, and the matplotlib tests exercise the raw frame pipeline on every run. Reviving the suite immediately surfaced 12 broken chart examples (
SafeAreawrappers missingexpand=True→ blank or shrunken charts), all fixed.*_actual.pngdiagnostic screenshots are now gitignored.Test plan
pytest packages/flet-charts/integration_tests/examples/— 24/24 green (localhost socket → raw 0x04 path + goldens)flet rundesktop (raw),flet run --web(PNG fallback), Pyodide build in Safari + Edge (raw; fps parity after canvaskit default)flutter testtransport tests,flutter analyze,ruffcleanSummary by Sourcery
Introduce a fast raw RGBA Matplotlib rendering path over local DataChannels, switch Flet web builds to use CanvasKit by default for better byte-buffer performance, and restore chart integration tests to cover the reorganized examples and golden screenshots.
New Features:
Enhancements:
Build:
Documentation:
Tests:
Chores: