Skip to content

Add recursive display locking under XInitThreads#54

Open
jserv wants to merge 2 commits into
mainfrom
display-lock
Open

Add recursive display locking under XInitThreads#54
jserv wants to merge 2 commits into
mainfrom
display-lock

Conversation

@jserv

@jserv jserv commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Upstream locking.c installs a non-recursive per-display mutex when a client calls XInitThreads. It self-deadlocks the moment one locked path re-enters another: an app-level XLockDisplay followed by any internal LockDisplay, or one public Xlib entry calling another. Repoint the lock hooks at a single process-global recursive mutex so all such nesting is safe. Single-threaded clients never install the hooks, so LockDisplay stays a no-op and the lock is never touched. Release the upstream per-display lock allocation at close (previously leaked).

Serialize the SDL-queue drain under the display lock in both drainSdlEventsToPutBack and discardQueuedEventsForWindow, so XPending, XEventsQueued, and window teardown no longer race XCheckTypedEvent's locked drain on SDL_PeepEvents(GETEVENT) and the pipe wake-byte accounting.

Make the request counter (SET_X_SERVER_REQUEST) and the process-global lastEventSerial relaxed atomics; both are bumped and read across Xlib entry points and the SDL event thread.


Summary by cubic

Make display locking recursive under XInitThreads and serialize both SDL drains and wake reconciliation to remove self-deadlocks and lost wake-ups. Adds atomic request/event counters and a ThreadSanitizer gate; single‑threaded apps are unaffected.

  • Bug Fixes

    • Redirected display->lock_fns to a process‑global recursive mutex; nested XLockDisplay/internal LockDisplay and Xlib re‑entry are safe. Freed per‑display lock allocations on XCloseDisplay.
    • Serialized SDL drains under the display lock: drainSdlEventsToPutBack and the entire discardQueuedEventsForWindow now hold it, removing races with XCheckTypedEvent and wake‑byte accounting.
    • Fixed wake‑pipe lost wakeups by moving the wake byte and qlen updates under eventQueueLengthLock, and holding the same lock across resetEventWakeups’ discard‑then‑rewrite. Verified with an XTest that the connection fd wakes even with a client SDL_SetEventFilter.
    • Made the X request counter and process‑global event serial relaxed atomics (SET_X_SERVER_REQUEST, atomicLoadRequest). Added a threaded lock/queue stress test and an XTest case.
  • Refactors

    • Gave file‑local globals internal linkage and documented the lock order at the mutex declarations.
    • CI: added a dedicated ThreadSanitizer job that builds with -fsanitize=thread and runs only the display‑lock threading test (LIBX11_COMPAT_THREAD_TEST_ONLY) under the dummy driver.

Written for commit 2d19f02. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

@jserv jserv force-pushed the display-lock branch 6 times, most recently from 30a52a0 to 2bd5137 Compare July 12, 2026 22:11
Upstream locking.c installs a non-recursive per-display mutex when a
client calls XInitThreads. It self-deadlocks the moment one locked path
re-enters another: an app-level XLockDisplay followed by any internal
LockDisplay, or one public Xlib entry calling another. Repoint the lock
hooks at a single process-global recursive mutex so all such nesting is
safe. Single-threaded clients never install the hooks, so LockDisplay
stays a no-op and the lock is never touched. Release the upstream
per-display lock allocation at close (previously leaked).

Serialize the SDL-queue drain under the display lock: in
drainSdlEventsToPutBack and across the whole of
discardQueuedEventsForWindow, so XPending, XEventsQueued, and window
teardown no longer race XCheckTypedEvent's locked drain on
SDL_PeepEvents(GETEVENT) and the pipe wake-byte accounting.

Make the request counter (SET_X_SERVER_REQUEST) and the process-global
lastEventSerial relaxed atomics; both are bumped and read across Xlib
entry points and the SDL event thread.

Document the total lock order at the mutex declarations. A threading
test in tests/check.c hammers the event queue under XLockDisplay with
recursive re-entry and an app lock held across internal-locking calls,
and a tests/test-xtest.c case checks that an external XTest push still
wakes the connection fd when a client installs its own SDL event filter.

The build, debug-build, and sanitize jobs run the unit suite (including
test_display_lock_threads) under ASan + UBSan, which catch memory and UB
bugs but not data races. The display lock and its serialized event-queue
drains are the one race-critical addition, so give them a ThreadSanitizer
gate: a dedicated job builds the library and check binary with
-fsanitize=thread and runs only the threading test offscreen. ASan and
TSan cannot share a build, hence a separate job. A data race aborts the
run and fails the job.
resetEventWakeups discarded every wake-pipe byte and rewrote the count while the
SDL filter onSdlEvent could enqueue concurrently (onSdlEvent does not hold the
display lock), so the reconcile could discard a wake byte the filter had just
written and overwrite the matching qlen bump, dropping a wakeup for a thread
blocked in poll(ConnectionNumber). Move the wake-pipe byte and the qlen counter
together under eventQueueLengthLock in ENQUEUE_EVENT_IN_PIPE / READ_EVENT_IN_PIPE,
and hold that same lock across resetEventWakeups' discard-then-rewrite, so the
reconcile is atomic against producers.

wakeEventPipeForExternalEvent (the XTest / in-process-snapshot wake path) is left
unchanged: an earlier attempt to restructure it stalled the mosaic in-process
snapshot smoke, and the drain reconcile is a separate path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant