GTK4 apps return empty accessibility tree (1 "unknown" node) — likely atspi crate GTK4 object-path handling

[jarrodcolburn]

Summary

get_app_state (and list_apps) return an empty accessibility tree for GTK4 applications — a single root node with role: "unknown" and child_count: 0 — while GTK3 / Chromium / Electron apps work fine. The same apps expose their full trees over AT-SPI (verified with libatspi/pyatspi), so the data is present on the bus; only this tool fails to read it.

I traced it to what looks like a client-side limitation in the atspi Rust crate (atspi = "0.29.0") around GTK4's accessible object-path scheme, not anything in this repo's own logic.

Environment

• @agent-sh/computer-use-linux 0.2.8 (npm prebuilt binary)
• Ubuntu / GNOME Shell 46.0, Wayland session
• at-spi2-core 2.52, GNOME Shell window-targeting extension installed & active (can_list_windows: true)
• Diagnostics otherwise green: can_build_accessibility_tree: true, AT-SPI enabled, window targeting works

Reproduction

1. Launch any GTK4 app on a session, e.g. gapplication launch org.gnome.Nautilus
2. get_app_state targeting it by pid/window (the window resolves correctly)
3. Result:

"accessibility_tree": [
  { "name": null, "role": "unknown", "child_count": 0,
    "object_ref": ":1.21/org/a11y/atspi/accessible/root", "depth": 0 }
],
"accessibility_tree_raw_count": 1

Evidence that the protocol is fine (it's not GTK4-fundamental)

Controlled test, all apps launched fresh, all deb-packaged (ruling out snap/flatpak/appimage confinement and launch-timing):

App	Toolkit	pyatspi nodes	this tool reads
Nautilus	GTK4	751	1 (unknown)
gnome-text-editor	GTK4	158	1 (unknown)
baobab	GTK4	55	1 (unknown)
gnome-disks	GTK3	99	✅ full (application → frame "Disks")

Querying the same GTK4 root objects directly over the a11y bus (via GDBus) returns correct data — GetChildren, ChildCount, GetRoleName, Name, GetChildAtIndex all succeed:

:1.21 (Nautilus  GTK4)  role=application  ChildCount=1  GetChildren()=1  ✓
:1.27 (text-editor GTK4) role=application  ChildCount=1  GetChildren()=1  ✓
:1.28 (baobab     GTK4)  role=application  ChildCount=1  GetChildren()=1  ✓
:1.29 (gnome-disks GTK3) role=application  ChildCount=1  GetChildren()=1  ✓

So the failure is entirely client-side.

Likely root cause

The registry enumeration is identical for all apps, but the child object paths differ by toolkit:

App	Toolkit	child object path returned by GetChildren
gnome-disks / terminal	GTK3	/org/a11y/atspi/accessible/72 (conventional)
Nautilus	GTK4	/org/gnome/Nautilus/a11y/e3ed3925_…
gnome-text-editor	GTK4	/org/gnome/TextEditor/a11y/4f41edb1_…
baobab	GTK4	/org/gnome/baobab/a11y/fb0fa87f_…

GTK4's native AT-SPI backend registers accessibles under app-specific D-Bus paths (/org/gnome/<App>/a11y/<uuid>) instead of the legacy /org/a11y/atspi/accessible/<N>. Object paths are opaque per the AT-SPI spec, and libatspi follows them fine, but the atspi crate appears not to.

This matches a known upstream issue: odilia-app/atspi#69 "Compatibility with GTK4" (open since 2023-03). It's marked closed via PR #70 ("add an AccessibleId variant that can serialize a path"), yet atspi 0.29.0 as used here still produces empty GTK4 trees — so the fix may be incomplete/regressed, or additional handling is needed for the tree-walk path.

Note that even the root read here returns role: "unknown" / child_count: 0 despite being at the conventional /org/a11y/atspi/accessible/root (which answers correctly over raw D-Bus), so the workaround likely needs to cover the property reads (name, get_role_name, child_count), not just child traversal.

Suggested directions

• Bump/patch the atspi dependency and verify GTK4 path handling, or
• Bypass the crate's high-level accessors in read_node/snapshot_tree (proxy.get_children()/name()/get_role_name()/child_count()) with direct zbus calls on the existing AccessibilityConnection — these are confirmed to work against GTK4 objects over raw D-Bus.

Impact is significant for GNOME, since most modern GNOME apps (Nautilus, Files, Settings, Text Editor, Calculator, baobab) are GTK4. The window/input/screenshot features and GTK3/Chromium/Electron AT-SPI all work great — thanks for the project.

[avifenesh]

thanks for the detailed report - the pyatspi vs this-tool table and the raw gdbus dumps are what made this fast to pin down, so really appreciate the legwork.

dug into it and it turned out to be a different root cause than the atspi-path-parsing one. the crate actually deserializes the gtk4 app-specific paths (/org/gnome/<App>/a11y/<uuid>) just fine - i confirmed that by hand against the live tree.

the real problem is the atspi P2P trait. i open every node via object_as_accessible (src/atspi_tree.rs), which routes reads over a peer-to-peer socket when the app advertises one - and when it doesnt, the fallback builds an AccessibleProxy with a path but no destination. on the shared a11y bus that proxy cant address the app, so every call fails with ServiceUnknown, and read_node swallows that into role: "unknown" / child_count: 0 - exactly the empty tree you saw.

the giveaway is that the working/broken split tracks p2p support, not the toolkit. probed it live on gnome 50:

• gnome-disks (gtk3) - advertises p2p - worked
• electron / chrome - advertise p2p - worked
• nautilus (gtk4) - no p2p - empty
• ptyxis, mutter-x11-frames - no p2p - empty

modern gtk4 apps just dont implement the legacy GetApplicationBusAddress, so they always land on the broken fallback.
fix is in #32 - drop the P2P trait and open proxies with ObjectRefExt::as_accessible_proxy, which always pins the destination to the object's bus name. verified live: nautilus goes from 1 node to a full 28-node tree (descending those /org/gnome/Nautilus/a11y/<uuid> paths), gnome-disks unchanged at 111.

will ship it in the next release.

[avifenesh]

shipped in v0.2.9 - just went out on npm (@agent-sh/computer-use-linux@0.2.9) and crates.io.

installed the published package fresh and ran it against a live nautilus on gnome 50 - the released binary pulls the full 28-node tree now, descending the /org/gnome/Nautilus/a11y/<uuid> paths down to the window/group nodes, where it used to stop at the single unknown root. gtk3 apps unchanged.

thanks again for finding this and for the writeup - the table is what let me chase it down to the actual p2p fallback instead of guessing. give the latest a try when you get a chance and let me know if any gtk4 app still comes back empty for you.