feat: Web-based remote desktop streaming via noVNC#71
Conversation
There was a problem hiding this comment.
Pull request overview
Adds web-based remote desktop “windows” to run-kit’s existing tmux session/window model, enabling GUI workflows in the same UI as terminal streaming via a noVNC + Xvfb + x11vnc stack.
Changes:
- Backend: introduces desktop window creation + resolution change endpoint, tmux window option helpers, and relay branching (PTY vs VNC WebSocket proxy).
- Frontend: adds
DesktopClient(noVNC),DesktopBottomBarcontrols, UI creation entry points, and window type-based rendering. - Docs/spec: captures the architecture, conventions (e.g.,
desktop:prefix), and quality checklist.
Reviewed changes
Copilot reviewed 32 out of 34 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| fab/changes/260323-a805-web-based-remote-desktop/tasks.md | Task breakdown for implementing desktop streaming feature. |
| fab/changes/260323-a805-web-based-remote-desktop/spec.md | Detailed behavioral/architecture spec for desktop windows + relay. |
| fab/changes/260323-a805-web-based-remote-desktop/intake.md | Feature intake/background and design decisions. |
| fab/changes/260323-a805-web-based-remote-desktop/checklist.md | QA checklist for the feature’s behaviors and edge cases. |
| fab/changes/260323-a805-web-based-remote-desktop/.status.yaml | FAB pipeline status/metadata for the change. |
| fab/changes/260323-a805-web-based-remote-desktop/.history.jsonl | FAB stage transition history. |
| docs/memory/run-kit/ui-patterns.md | Documents desktop UI patterns (type switch, bottom bar, creation entry points). |
| docs/memory/run-kit/tmux-sessions.md | Documents desktop window convention + per-window tmux option usage. |
| docs/memory/run-kit/architecture.md | Documents unified relay branching and desktop streaming architecture. |
| app/frontend/tests/msw/handlers.ts | Updates MSW mocks to include window.type and desktop creation behavior. |
| app/frontend/src/types/novnc.d.ts | Adds local noVNC RFB type declarations. |
| app/frontend/src/types.ts | Adds WindowInfo.type union for terminal vs desktop windows. |
| app/frontend/src/components/top-bar.tsx | Adds “+ New Desktop” entry to window breadcrumb dropdown. |
| app/frontend/src/components/top-bar.test.tsx | Updates tests for new WindowInfo.type and new prop wiring. |
| app/frontend/src/components/sidebar.test.tsx | Updates test fixtures for new WindowInfo.type field. |
| app/frontend/src/components/desktop-client.tsx | New noVNC client component with reconnect/backoff behavior. |
| app/frontend/src/components/desktop-client.test.tsx | Tests DesktopClient mount/unmount + configuration behavior. |
| app/frontend/src/components/desktop-bottom-bar.tsx | New desktop-specific bottom bar (clipboard, resolution, fullscreen, palette). |
| app/frontend/src/components/dashboard.tsx | Adds Desktop badges and “+ New Desktop” creation button on session cards. |
| app/frontend/src/components/dashboard.test.tsx | Updates dashboard test fixtures for new WindowInfo.type. |
| app/frontend/src/components/breadcrumb-dropdown.tsx | Extends dropdown to support multiple action items (actions array). |
| app/frontend/src/app.tsx | Renders Desktop vs Terminal clients based on currentWindow.type; adds palette actions. |
| app/frontend/src/api/client.ts | Adds createDesktopWindow and changeDesktopResolution API calls. |
| app/frontend/pnpm-lock.yaml | Locks @novnc/novnc dependency. |
| app/frontend/package.json | Adds @novnc/novnc dependency. |
| app/backend/internal/validate/validate.go | Adds ValidateResolution helper. |
| app/backend/internal/validate/validate_test.go | Adds tests for resolution validation. |
| app/backend/internal/tmux/tmux.go | Adds WindowInfo.Type, parses type from desktop: prefix; adds window option helpers. |
| app/backend/internal/tmux/tmux_test.go | Tests window type detection and parsing updates. |
| app/backend/api/windows.go | Adds desktop window creation, port allocation + startup script, and resolution change endpoint. |
| app/backend/api/windows_test.go | Adds tests for desktop creation and resolution change paths. |
| app/backend/api/sessions_test.go | Extends mock tmux ops with window option support for new handlers. |
| app/backend/api/router.go | Adds tmux option methods to interface + registers resolution route. |
| app/backend/api/relay.go | Branches relay: terminal PTY vs desktop VNC WebSocket proxy. |
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| return fmt.Sprintf(`export DISPLAY=:%d && `+ | ||
| `Xvfb :%d -screen 0 %sx24 &>/dev/null & `+ | ||
| `sleep 1 && `+ | ||
| `WM=""; `+ | ||
| `if command -v x-session-manager &>/dev/null; then WM=x-session-manager; `+ | ||
| `elif [ -n "$XDG_CURRENT_DESKTOP" ]; then `+ | ||
| `case "$XDG_CURRENT_DESKTOP" in `+ | ||
| `GNOME) command -v mutter &>/dev/null && WM=mutter;; `+ | ||
| `KDE) command -v kwin &>/dev/null && WM=kwin;; `+ | ||
| `XFCE) command -v xfwm4 &>/dev/null && WM=xfwm4;; `+ | ||
| `esac; `+ | ||
| `fi; `+ | ||
| `if [ -z "$WM" ]; then `+ | ||
| `for wm in openbox fluxbox i3 xfwm4 mutter kwin; do `+ | ||
| `if command -v "$wm" &>/dev/null; then WM="$wm"; break; fi; `+ | ||
| `done; `+ | ||
| `fi; `+ | ||
| `[ -n "$WM" ] && $WM &>/dev/null & `+ | ||
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws`, |
There was a problem hiding this comment.
The startup script relies on shell-specific syntax (&>/dev/null) and assumes a POSIX-like interactive shell. Since this is executed via tmux send-keys in whatever shell the user has configured (fish, dash, etc.), desktop startup can fail. Prefer running the script via a known shell (e.g. sh -lc ...) and use portable redirections (>/dev/null 2>&1).
| return fmt.Sprintf(`export DISPLAY=:%d && `+ | |
| `Xvfb :%d -screen 0 %sx24 &>/dev/null & `+ | |
| `sleep 1 && `+ | |
| `WM=""; `+ | |
| `if command -v x-session-manager &>/dev/null; then WM=x-session-manager; `+ | |
| `elif [ -n "$XDG_CURRENT_DESKTOP" ]; then `+ | |
| `case "$XDG_CURRENT_DESKTOP" in `+ | |
| `GNOME) command -v mutter &>/dev/null && WM=mutter;; `+ | |
| `KDE) command -v kwin &>/dev/null && WM=kwin;; `+ | |
| `XFCE) command -v xfwm4 &>/dev/null && WM=xfwm4;; `+ | |
| `esac; `+ | |
| `fi; `+ | |
| `if [ -z "$WM" ]; then `+ | |
| `for wm in openbox fluxbox i3 xfwm4 mutter kwin; do `+ | |
| `if command -v "$wm" &>/dev/null; then WM="$wm"; break; fi; `+ | |
| `done; `+ | |
| `fi; `+ | |
| `[ -n "$WM" ] && $WM &>/dev/null & `+ | |
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws`, | |
| return fmt.Sprintf(`sh -lc 'export DISPLAY=:%d && `+ | |
| `Xvfb :%d -screen 0 %sx24 >/dev/null 2>&1 & `+ | |
| `sleep 1 && `+ | |
| `WM=""; `+ | |
| `if command -v x-session-manager >/dev/null 2>&1; then WM=x-session-manager; `+ | |
| `elif [ -n "$XDG_CURRENT_DESKTOP" ]; then `+ | |
| `case "$XDG_CURRENT_DESKTOP" in `+ | |
| `GNOME) command -v mutter >/dev/null 2>&1 && WM=mutter;; `+ | |
| `KDE) command -v kwin >/dev/null 2>&1 && WM=kwin;; `+ | |
| `XFCE) command -v xfwm4 >/dev/null 2>&1 && WM=xfwm4;; `+ | |
| `esac; `+ | |
| `fi; `+ | |
| `if [ -z "$WM" ]; then `+ | |
| `for wm in openbox fluxbox i3 xfwm4 mutter kwin; do `+ | |
| `if command -v "$wm" >/dev/null 2>&1; then WM="$wm"; break; fi; `+ | |
| `done; `+ | |
| `fi; `+ | |
| `[ -n "$WM" ] && $WM >/dev/null 2>&1 & `+ | |
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws'`, |
| <DesktopClient /> + <DesktopBottomBar /> | ||
| ) : ( | ||
| <TerminalClient /> + <BottomBar /> |
There was a problem hiding this comment.
The TSX example in the code block uses + between components (<DesktopClient /> + <DesktopBottomBar />), which isn't valid JSX and may confuse readers. Consider showing them as siblings (e.g., fragment or separate lines) to reflect actual rendering.
| <DesktopClient /> + <DesktopBottomBar /> | |
| ) : ( | |
| <TerminalClient /> + <BottomBar /> | |
| <> | |
| <DesktopClient /> | |
| <DesktopBottomBar /> | |
| </> | |
| ) : ( | |
| <> | |
| <TerminalClient /> | |
| <BottomBar /> | |
| </> |
| `app/frontend/src/components/desktop-bottom-bar.tsx` — desktop-specific controls replacing the terminal `BottomBar` when a desktop window is active. | ||
|
|
||
| **Controls** (left to right): | ||
| 1. **Clipboard paste** button — reads browser clipboard via `navigator.clipboard.readText()`, sends to VNC session via `rfb.clipboardPasteFrom`. Graceful degradation if clipboard API unavailable |
There was a problem hiding this comment.
This doc line references rfb.clipboardPasteFrom without indicating whether it's a method call vs property assignment. Since the code should invoke clipboard paste as an API call, it would be clearer to document the correct usage (e.g. calling clipboardPasteFrom(text) if that's the intended API).
| 1. **Clipboard paste** button — reads browser clipboard via `navigator.clipboard.readText()`, sends to VNC session via `rfb.clipboardPasteFrom`. Graceful degradation if clipboard API unavailable | |
| 1. **Clipboard paste** button — reads browser clipboard via `navigator.clipboard.readText()`, sends to VNC session via `rfb.clipboardPasteFrom(text)`. Graceful degradation if clipboard API unavailable |
| try { | ||
| const text = await navigator.clipboard.readText(); | ||
| if (text && rfbRef.current) { | ||
| rfbRef.current.clipboardPasteFrom = text; |
There was a problem hiding this comment.
clipboardPasteFrom is being assigned a string here. In noVNC's RFB API this is typically a method (e.g. clipboardPasteFrom(text)), so assigning will overwrite the function and break clipboard for the remainder of the session. Update this to call the method (and guard if the API differs), and align the local type definitions accordingly.
| rfbRef.current.clipboardPasteFrom = text; | |
| const rfb = rfbRef.current as { | |
| clipboardPasteFrom?: ((text: string) => void) | string; | |
| }; | |
| if (typeof rfb.clipboardPasteFrom === "function") { | |
| rfb.clipboardPasteFrom(text); | |
| } else if ("clipboardPasteFrom" in rfb) { | |
| // Fallback for non-standard implementations where this is a writable property | |
| (rfb as any).clipboardPasteFrom = text; | |
| } |
| // Store VNC port as tmux window option via internal/tmux (not in shell script) | ||
| if err := s.tmux.SetWindowOption(session, windowIndex, "@rk_vnc_port", strconv.Itoa(port), server); err != nil { | ||
| slog.Error("failed to set VNC port window option", "err", err) | ||
| // Non-fatal — relay will fail to connect but desktop still works |
There was a problem hiding this comment.
If SetWindowOption(@rk_vnc_port) fails, the desktop window won't be connectable via /relay (and resolution changes will fail) because the port is discovered exclusively through this option. Treat this as a fatal error: return a 5xx and consider killing the newly created tmux window to avoid leaving an unusable desktop process behind.
| // Store VNC port as tmux window option via internal/tmux (not in shell script) | |
| if err := s.tmux.SetWindowOption(session, windowIndex, "@rk_vnc_port", strconv.Itoa(port), server); err != nil { | |
| slog.Error("failed to set VNC port window option", "err", err) | |
| // Non-fatal — relay will fail to connect but desktop still works | |
| // Store VNC port as tmux window option via internal/tmux (not in shell script). | |
| // If this fails, the VNC port cannot be discovered by /relay or for resolution changes, | |
| // so treat it as a fatal error. | |
| if err := s.tmux.SetWindowOption(session, windowIndex, "@rk_vnc_port", strconv.Itoa(port), server); err != nil { | |
| slog.Error("failed to set VNC port window option", "err", err) | |
| writeError(w, http.StatusInternalServerError, "Failed to configure desktop VNC port") | |
| return |
| `pkill -f 'Xvfb :%d' 2>/dev/null; pkill -f 'x11vnc.*:%d' 2>/dev/null; sleep 0.5 && `+ | ||
| `export DISPLAY=:%d && `+ | ||
| `Xvfb :%d -screen 0 %sx24 &>/dev/null & `+ | ||
| `sleep 1 && `+ | ||
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws`, |
There was a problem hiding this comment.
The resolution restart script repeats the same shell portability issue (&>/dev/null) and is also executed via send-keys in the user's shell. Consider using the same sh -lc wrapper (and portable redirects) here as in desktop creation so resolution changes work regardless of the user's default shell.
| `pkill -f 'Xvfb :%d' 2>/dev/null; pkill -f 'x11vnc.*:%d' 2>/dev/null; sleep 0.5 && `+ | |
| `export DISPLAY=:%d && `+ | |
| `Xvfb :%d -screen 0 %sx24 &>/dev/null & `+ | |
| `sleep 1 && `+ | |
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws`, | |
| `sh -lc 'pkill -f "Xvfb :%d" 2>/dev/null; pkill -f "x11vnc.*:%d" 2>/dev/null; sleep 0.5 && `+ | |
| `export DISPLAY=:%d && `+ | |
| `Xvfb :%d -screen 0 %sx24 >/dev/null 2>&1 & `+ | |
| `sleep 1 && `+ | |
| `exec x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -ws'`, |
| async function connect() { | ||
| if (cancelled || !containerRef.current) return; | ||
|
|
||
| // Clear container before creating new RFB | ||
| containerRef.current.innerHTML = ""; | ||
|
|
||
| const { default: RFB } = await import("@novnc/novnc/lib/rfb"); | ||
| if (cancelled || !containerRef.current) return; | ||
|
|
||
| const wsUrl = `${wsProto}//${window.location.host}/relay/${encodeURIComponent(sessionName)}/${windowIndex}?server=${encodeURIComponent(server)}`; | ||
|
|
||
| const rfb = new RFB(containerRef.current, wsUrl); | ||
| rfb.scaleViewport = true; |
There was a problem hiding this comment.
connect() is an async function invoked without awaiting and without a try/catch. If the dynamic import fails or new RFB(...) throws, this becomes an unhandled promise rejection and the component won't schedule a reconnect. Wrap the body of connect() in try/catch and trigger the same reconnect/backoff path on failures.
| windowIndex={windowIndex} | ||
| server={server} | ||
| onSessionNotFound={() => navigate({ to: "/", replace: true })} | ||
| onRfbRef={(rfb) => { rfbInstanceRef.current = rfb; }} |
There was a problem hiding this comment.
onRfbRef is passed as an inline arrow function. Because DesktopClient includes onRfbRef in its effect dependencies (via cleanup), any App re-render will change this prop identity and force a disconnect/reconnect loop for the desktop stream. Make this callback stable (e.g. useCallback) or refactor DesktopClient to avoid depending on callback identity for connection lifecycle.
| onRfbRef={(rfb) => { rfbInstanceRef.current = rfb; }} | |
| onRfbRef={useCallback((rfb) => { rfbInstanceRef.current = rfb; }, [])} |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 32 out of 34 changed files in this pull request and generated 8 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // allocateFreePort finds a free TCP port using the net.Listen trick. | ||
| func allocateFreePort() (int, error) { | ||
| l, err := net.Listen("tcp", ":0") | ||
| if err != nil { | ||
| return 0, err | ||
| } | ||
| port := l.Addr().(*net.TCPAddr).Port | ||
| l.Close() | ||
| return port, nil |
There was a problem hiding this comment.
allocateFreePort() uses net.Listen(":0") and then closes the listener before starting x11vnc. This has a small TOCTOU race where another process could claim the port between close and x11vnc bind, causing startup/connect failures. If reliability matters, consider keeping the listener open until the VNC process is ready (or implement a retry loop on bind failure).
| // Find the newly created window index | ||
| windows, err := s.tmux.ListWindows(session, server) | ||
| if err != nil { | ||
| writeError(w, http.StatusInternalServerError, err.Error()) | ||
| return | ||
| } | ||
| windowIndex := -1 | ||
| for _, win := range windows { | ||
| if win.Name == windowName { | ||
| windowIndex = win.Index | ||
| } | ||
| } |
There was a problem hiding this comment.
Desktop window creation finds the new window index by scanning ListWindows() for win.Name == windowName. tmux allows duplicate window names, so this can select an existing window with the same name (or the last match) and then send the startup script / set options on the wrong window. Prefer having CreateWindow return the created index (e.g., tmux new-window -P -F '#{window_index}' ...) or otherwise identify the new window unambiguously (diffing window lists before/after).
| r.Post("/api/sessions/{session}/windows/{index}/select", s.handleWindowSelect) | ||
| r.Post("/api/sessions/{session}/windows/{index}/split", s.handleWindowSplit) | ||
| r.Post("/api/sessions/{session}/windows/{index}/resolution", s.handleWindowResolution) | ||
| r.Get("/api/sessions/{session}/windows/{index}/desktop-info", s.handleDesktopInfo) |
There was a problem hiding this comment.
This router registers /api/sessions/{session}/windows/{index}/desktop-info, but the handler currently depends on @rk_ws_port which isn't set anywhere, so the endpoint appears non-functional/dead code. If it's not needed for the noVNC flow, consider removing the route to avoid maintaining unused API surface.
| r.Get("/api/sessions/{session}/windows/{index}/desktop-info", s.handleDesktopInfo) |
| <textarea | ||
| ref={kbdInputRef} | ||
| aria-hidden="true" | ||
| className="absolute opacity-0 w-0 h-0 pointer-events-none" | ||
| style={{ position: "fixed", top: -9999, left: -9999 }} | ||
| autoComplete="off" | ||
| autoCorrect="off" | ||
| autoCapitalize="off" | ||
| spellCheck={false} | ||
| onKeyDown={handleKbdKeyDown} | ||
| onInput={handleKbdTextInput} | ||
| onBlur={() => setKbdActive(false)} | ||
| /> |
There was a problem hiding this comment.
The hidden textarea is focusable (used to open the mobile keyboard) but is marked aria-hidden="true". Focusable elements should not be aria-hidden; this can confuse screen readers and violates accessibility expectations. Consider removing aria-hidden and instead using an appropriate aria-label/role, and keep it out of the tab order via tabIndex={-1} if needed.
| WebSocket-to-WebSocket proxy between the browser and x11vnc. No PTY involved. | ||
|
|
||
| Per connection: | ||
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port` — returns WebSocket close code `4002` if not found | ||
| 2. Dials `ws://localhost:{port}` to connect to x11vnc's built-in WebSocket server (localhost only, no external connections) | ||
| 3. Bidirectional copy: two goroutines relay messages between browser WebSocket and VNC WebSocket | ||
| 4. On disconnect (either side): `sync.Once` cleanup closes both WebSocket connections |
There was a problem hiding this comment.
The docs describe the desktop relay as WebSocket-to-WebSocket (dialing ws://localhost:{port}) and x11vnc running with -ws, but the current implementation in api/relay.go proxies browser WebSocket ↔ raw TCP VNC and the startup script in api/windows.go starts x11vnc without -ws. Please align the documentation/spec with the actual implementation (or update the implementation to match the documented -ws WebSocket flow) to avoid misleading operators and future contributors.
| WebSocket-to-WebSocket proxy between the browser and x11vnc. No PTY involved. | |
| Per connection: | |
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port` — returns WebSocket close code `4002` if not found | |
| 2. Dials `ws://localhost:{port}` to connect to x11vnc's built-in WebSocket server (localhost only, no external connections) | |
| 3. Bidirectional copy: two goroutines relay messages between browser WebSocket and VNC WebSocket | |
| 4. On disconnect (either side): `sync.Once` cleanup closes both WebSocket connections | |
| WebSocket-to-TCP proxy between the browser and a local x11vnc server. No PTY involved — the backend terminates the browser WebSocket and speaks raw VNC over TCP to x11vnc (which is started without `-ws`). | |
| Per connection: | |
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port` — returns WebSocket close code `4002` if not found | |
| 2. Dials `localhost:{port}` over TCP to connect to x11vnc's VNC server (loopback only, no external connections) | |
| 3. Bidirectional copy: goroutines relay messages between the browser WebSocket and the TCP VNC connection (WebSocket → TCP, TCP → WebSocket) | |
| 4. On disconnect (either side): `sync.Once` cleanup closes both the WebSocket and the TCP connection |
| // Detect window type BEFORE WebSocket upgrade so desktop can use hijack | ||
| windows, err := s.tmux.ListWindows(session, server) | ||
| if err != nil || windows == nil { | ||
| slog.Warn("session not found", "session", session) | ||
| conn.WriteMessage(websocket.CloseMessage, | ||
| websocket.FormatCloseMessage(4004, "Session not found")) | ||
| http.Error(w, "Session not found", http.StatusNotFound) | ||
| return | ||
| } | ||
| var windowType string | ||
| windowFound := false | ||
| for _, win := range windows { | ||
| if win.Index == winIdx { | ||
| windowType = win.Type | ||
| windowFound = true | ||
| break | ||
| } | ||
| } | ||
| if !windowFound { | ||
| http.Error(w, "Window not found", http.StatusNotFound) | ||
| return |
There was a problem hiding this comment.
TerminalClient relies on the relay WebSocket closing with code 4004 to detect a missing session/window and navigate away. After this change, session/window validation happens before upgrading and returns 404 via http.Error, which causes the browser WebSocket to fail with code 1006 and the client to reconnect indefinitely. Consider always upgrading first (for the terminal path) and then sending a WebSocket close frame with 4004 for "session/window not found" to preserve existing client behavior.
| } | ||
|
|
||
| // desktopStartupScript generates a bash script to launch Xvfb, detect WM, and x11vnc. | ||
| // Written to a temp file and executed, avoiding send-keys one-liner parsing issues. |
There was a problem hiding this comment.
The comment says the desktop startup script is "Written to a temp file and executed", but the implementation returns an inline bash -c '...' string sent via send-keys. Either update the comment to match reality or implement the temp-file approach; as-is this is misleading for future maintenance/debugging.
| // Written to a temp file and executed, avoiding send-keys one-liner parsing issues. | |
| // Returns a bash -c command string suitable for inline execution (e.g., via send-keys). |
| // handleDesktopInfo returns the websockify port for a desktop window. | ||
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | ||
| session := chi.URLParam(r, "session") | ||
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } | ||
|
|
||
| index, ok := parseWindowIndex(r) | ||
| if !ok { | ||
| writeError(w, http.StatusBadRequest, "Invalid window index") | ||
| return | ||
| } | ||
|
|
||
| server := serverFromRequest(r) | ||
|
|
||
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return | ||
| } | ||
|
|
||
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) |
There was a problem hiding this comment.
handleDesktopInfo reads @rk_ws_port, but this option isn't set anywhere in the backend (desktop creation sets @rk_vnc_port and the relay connects directly). As implemented this endpoint will always return 404, and it also adds extra API surface not referenced by the frontend/spec. Consider removing the route/handler or implementing it consistently (including setting @rk_ws_port).
| // handleDesktopInfo returns the websockify port for a desktop window. | |
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | |
| session := chi.URLParam(r, "session") | |
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | |
| writeError(w, http.StatusBadRequest, errMsg) | |
| return | |
| } | |
| index, ok := parseWindowIndex(r) | |
| if !ok { | |
| writeError(w, http.StatusBadRequest, "Invalid window index") | |
| return | |
| } | |
| server := serverFromRequest(r) | |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | |
| if err != nil { | |
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | |
| return | |
| } | |
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) | |
| // handleDesktopInfo is currently not supported and always returns 404. | |
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | |
| // This endpoint previously attempted to read the tmux option "@rk_ws_port", | |
| // which is not set anywhere in the backend. To avoid exposing a broken | |
| // and unused API surface, we explicitly return 404 here. | |
| writeError(w, http.StatusNotFound, "Desktop info endpoint is not supported") |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 32 out of 34 changed files in this pull request and generated 8 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // Store VNC port as tmux window option | ||
| if err := s.tmux.SetWindowOption(session, windowIndex, "@rk_vnc_port", strconv.Itoa(port), server); err != nil { | ||
| slog.Error("failed to set VNC port window option", "err", err) |
There was a problem hiding this comment.
SetWindowOption failure is logged but the request still returns 201. If this fails, the relay can't discover @rk_vnc_port and the desktop will be unusable; it seems better to treat this as a hard error and fail the create request (or retry) rather than continuing.
| slog.Error("failed to set VNC port window option", "err", err) | |
| slog.Error("failed to set VNC port window option", "err", err) | |
| writeError(w, http.StatusInternalServerError, "Failed to set VNC port window option") | |
| return |
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | ||
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | ||
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | ||
| return | ||
| } |
There was a problem hiding this comment.
Resolution changes also overwrite a predictable /tmp/rk-desktop-{port}.sh path. Same security/cleanup concerns as creation (symlink clobber + accumulation). Use a securely-created temp file (CreateTemp/O_EXCL) and/or remove the file after use.
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | |
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| tmpFile, err := os.CreateTemp("", "rk-desktop-*.sh") | |
| if err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to create startup script file") | |
| return | |
| } | |
| if _, err := tmpFile.Write([]byte(script)); err != nil { | |
| tmpFile.Close() | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| if err := tmpFile.Chmod(0700); err != nil { | |
| tmpFile.Close() | |
| writeError(w, http.StatusInternalServerError, "Failed to set startup script permissions") | |
| return | |
| } | |
| if err := tmpFile.Close(); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to finalize startup script") | |
| return | |
| } | |
| scriptFile := tmpFile.Name() |
| sleep 3 | ||
| fi | ||
|
|
||
| x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage |
There was a problem hiding this comment.
x11vnc is started without restricting the bind address. Since -nopw is used, this likely exposes an unauthenticated VNC server on all interfaces by default. To keep the security posture aligned with the relay model, bind to localhost only (e.g., x11vnc -localhost / -listen 127.0.0.1) so remote clients cannot bypass run-kit's relay.
| x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage | |
| x11vnc -display :%d -rfbport %d -nopw -forever -shared -noxdamage -localhost |
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | ||
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | ||
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | ||
| return | ||
| } |
There was a problem hiding this comment.
The startup script writes to a predictable filename in /tmp based on the chosen port. On multi-user systems this pattern is vulnerable to symlink/hardlink clobbering and also leaves behind persistent files. Consider using os.CreateTemp (or similar) to create the script file securely and cleaning it up after execution (or storing the path in a tmux option if it must be reused).
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | |
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| tmpFile, err := os.CreateTemp("", "rk-desktop-*.sh") | |
| if err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| if _, err := tmpFile.Write([]byte(script)); err != nil { | |
| tmpFile.Close() | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| if err := tmpFile.Chmod(0700); err != nil { | |
| tmpFile.Close() | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| if err := tmpFile.Close(); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| scriptFile := tmpFile.Name() |
| DESKTOP_ID=desktop-%d | ||
| export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$DESKTOP_ID" | ||
| export XDG_CONFIG_HOME="$HOME/.config/$DESKTOP_ID" | ||
| export XDG_DATA_HOME="$HOME/.local/share/$DESKTOP_ID" | ||
| export XDG_CACHE_HOME="$HOME/.cache/$DESKTOP_ID" | ||
| export XDG_STATE_HOME="$HOME/.local/state/$DESKTOP_ID" | ||
| mkdir -p "$XDG_RUNTIME_DIR" "$XDG_CONFIG_HOME" "$XDG_DATA_HOME" "$XDG_CACHE_HOME" "$XDG_STATE_HOME" | ||
| chmod 0700 "$XDG_RUNTIME_DIR" |
There was a problem hiding this comment.
The script defaults XDG_RUNTIME_DIR to /run/user/$(id -u)/.... In non-systemd/container environments this directory may not exist and the user may not have permission to create under /run/user, causing desktop startup to fail. Consider falling back to a user-writable base (e.g. /tmp) when the directory doesn't exist or isn't writable.
| // handleDesktopInfo returns the websockify port for a desktop window. | ||
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | ||
| session := chi.URLParam(r, "session") | ||
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } | ||
|
|
||
| index, ok := parseWindowIndex(r) | ||
| if !ok { | ||
| writeError(w, http.StatusBadRequest, "Invalid window index") | ||
| return | ||
| } | ||
|
|
||
| server := serverFromRequest(r) | ||
|
|
||
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return |
There was a problem hiding this comment.
handleDesktopInfo reads @rk_ws_port, but nothing in this change-set ever sets that option (only @rk_vnc_port is set). This route will always 404 and adds API surface that appears unused by the frontend. Either remove this endpoint/route, or set/read the correct option and add coverage for it.
| {/* Hidden textarea to trigger mobile virtual keyboard */} | ||
| <textarea | ||
| ref={kbdInputRef} | ||
| aria-hidden="true" |
There was a problem hiding this comment.
The hidden textarea is programmatically focused to bring up mobile keyboards, but it is marked aria-hidden="true". Hiding a focusable element from assistive tech is an accessibility violation and can confuse screen readers. Consider removing aria-hidden and instead use an appropriate aria-label, or ensure it can’t receive focus via normal navigation (e.g., tabIndex={-1}) while still being focusable programmatically.
| aria-hidden="true" | |
| tabIndex={-1} | |
| aria-label="Virtual keyboard input" |
| // Detect window type BEFORE WebSocket upgrade so desktop can use hijack | ||
| windows, err := s.tmux.ListWindows(session, server) | ||
| if err != nil || windows == nil { | ||
| slog.Warn("session not found", "session", session) | ||
| conn.WriteMessage(websocket.CloseMessage, | ||
| websocket.FormatCloseMessage(4004, "Session not found")) | ||
| http.Error(w, "Session not found", http.StatusNotFound) | ||
| return | ||
| } |
There was a problem hiding this comment.
The session-not-found path now returns an HTTP 404 before the WebSocket upgrade. TerminalClient relies on receiving WS close code 4004 to stop reconnecting and redirect; with a handshake 404 it will likely loop reconnects instead. Consider upgrading first and sending a WebSocket close (e.g., 4004) for not-found cases (both terminal and desktop).
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 35 changed files in this pull request and generated 8 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Send C-c to kill the running bash -c (which kills Xvfb, x11vnc, WM), | ||
| // then send the full startup script at the new resolution. | ||
| if err := s.tmux.SendKeys(session, index, "C-c", server); err != nil { | ||
| slog.Error("failed to send C-c", "err", err) | ||
| } | ||
| time.Sleep(1 * time.Second) | ||
|
|
||
| script := desktopStartupScript(displayNum, port, body.Resolution) | ||
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | ||
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | ||
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | ||
| return | ||
| } | ||
| if err := s.tmux.SendKeys(session, index, scriptFile, server); err != nil { | ||
| writeError(w, http.StatusInternalServerError, err.Error()) | ||
| return | ||
| } | ||
|
|
There was a problem hiding this comment.
This resolution-change handler blocks the request with time.Sleep(1 * time.Second) after sending C-c. That adds fixed latency and ties up a handler goroutine. Prefer making the delay/restart sequencing happen inside the tmux-side script (or otherwise avoid sleeping in the HTTP request path).
| // Send C-c to kill the running bash -c (which kills Xvfb, x11vnc, WM), | |
| // then send the full startup script at the new resolution. | |
| if err := s.tmux.SendKeys(session, index, "C-c", server); err != nil { | |
| slog.Error("failed to send C-c", "err", err) | |
| } | |
| time.Sleep(1 * time.Second) | |
| script := desktopStartupScript(displayNum, port, body.Resolution) | |
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | |
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| if err := s.tmux.SendKeys(session, index, scriptFile, server); err != nil { | |
| writeError(w, http.StatusInternalServerError, err.Error()) | |
| return | |
| } | |
| // Prepare the startup script for the new resolution. | |
| script := desktopStartupScript(displayNum, port, body.Resolution) | |
| scriptFile := fmt.Sprintf("/tmp/rk-desktop-%d.sh", port) | |
| if err := os.WriteFile(scriptFile, []byte(script), 0700); err != nil { | |
| writeError(w, http.StatusInternalServerError, "Failed to write startup script") | |
| return | |
| } | |
| // Perform the tmux restart sequence asynchronously to avoid blocking the HTTP handler. | |
| go func(session string, index int, server string, scriptFile string) { | |
| // Send C-c to kill the running bash -c (which kills Xvfb, x11vnc, WM), | |
| // then send the full startup script at the new resolution after a short delay. | |
| if err := s.tmux.SendKeys(session, index, "C-c", server); err != nil { | |
| slog.Error("failed to send C-c", "err", err) | |
| } | |
| time.Sleep(1 * time.Second) | |
| if err := s.tmux.SendKeys(session, index, scriptFile, server); err != nil { | |
| slog.Error("failed to send startup script path", "err", err) | |
| } | |
| }(session, index, server, scriptFile) |
| - Launch WM in background (`&`) | ||
| 5. Start `x11vnc -display :N -rfbport {port} -nopw -forever -shared -noxdamage -ws` (the `-ws` flag enables WebSocket on the same VNC port) | ||
| <!-- clarified: x11vnc -ws flag — x11vnc uses -ws to enable built-in WebSocket support on the VNC listen port, allowing direct browser-to-x11vnc WebSocket connections --> |
There was a problem hiding this comment.
The spec mandates starting x11vnc with -ws and implies a WebSocket-to-WebSocket relay. The current backend implementation proxies browser WebSocket ↔ raw TCP VNC and the startup script does not use -ws. Please update the spec to match the implemented transport (or change the implementation to match this spec).
| try { | ||
| const text = await navigator.clipboard.readText(); | ||
| if (text && rfbRef.current) { | ||
| rfbRef.current.clipboardPasteFrom = text; |
There was a problem hiding this comment.
handleClipboardPaste assigns rfbRef.current.clipboardPasteFrom = text. In noVNC, clipboard sending is typically performed via a method call rather than a string field, so this is likely a runtime no-op (and conflicts with correcting the local typings). Update this to invoke the correct RFB clipboard API.
| rfbRef.current.clipboardPasteFrom = text; | |
| rfbRef.current.clipboardPasteFrom(text); |
| // Allow colon only in "desktop:" prefix (used for desktop window type detection). | ||
| // Bare colons elsewhere are forbidden because tmux uses : as session:window separator. | ||
| if strings.Contains(name, ":") && !strings.HasPrefix(name, "desktop:") { | ||
| return fmt.Sprintf("%s cannot contain colons (except desktop: prefix)", label) |
There was a problem hiding this comment.
ValidateName is used for session names as well as window names. Allowing : for any name starting with desktop: permits session names containing :, which will conflict with tmux target syntax (session:window) and can break core operations. Recommend keeping ValidateName strict (no :) and instead handling the desktop: prefix as an internal window-naming concern (validated separately in window creation/rename).
| // Allow colon only in "desktop:" prefix (used for desktop window type detection). | |
| // Bare colons elsewhere are forbidden because tmux uses : as session:window separator. | |
| if strings.Contains(name, ":") && !strings.HasPrefix(name, "desktop:") { | |
| return fmt.Sprintf("%s cannot contain colons (except desktop: prefix)", label) | |
| // Colons are forbidden because tmux uses : as session:window separator. | |
| if strings.Contains(name, ":") { | |
| return fmt.Sprintf("%s cannot contain colons", label) |
|
|
||
| // handleDesktopInfo returns the websockify port for a desktop window. | ||
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | ||
| session := chi.URLParam(r, "session") | ||
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } | ||
|
|
||
| index, ok := parseWindowIndex(r) | ||
| if !ok { | ||
| writeError(w, http.StatusBadRequest, "Invalid window index") | ||
| return | ||
| } | ||
|
|
||
| server := serverFromRequest(r) | ||
|
|
||
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return | ||
| } | ||
|
|
||
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) | ||
| } |
There was a problem hiding this comment.
handleDesktopInfo appears to be websockify-specific (it talks about returning a WebSocket port), but the rest of the desktop implementation uses only @rk_vnc_port and proxies directly to the VNC port. If this endpoint isn’t part of the intended API, consider removing it to avoid dead/legacy surface area.
| // handleDesktopInfo returns the websockify port for a desktop window. | |
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | |
| session := chi.URLParam(r, "session") | |
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | |
| writeError(w, http.StatusBadRequest, errMsg) | |
| return | |
| } | |
| index, ok := parseWindowIndex(r) | |
| if !ok { | |
| writeError(w, http.StatusBadRequest, "Invalid window index") | |
| return | |
| } | |
| server := serverFromRequest(r) | |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | |
| if err != nil { | |
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | |
| return | |
| } | |
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) | |
| } |
| Each desktop window runs three processes inside a tmux window: | ||
| 1. **Xvfb** `:N -screen 0 {width}x{height}x24` — virtual framebuffer (software rendering, default 1920x1080) | ||
| 2. **Window manager** (optional) — auto-detected from host: `x-session-manager` > `$XDG_CURRENT_DESKTOP` mapping > PATH probe (`openbox`, `fluxbox`, `i3`, `xfwm4`, `mutter`, `kwin`). Bare X11 if nothing found | ||
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage -ws` — VNC server with built-in WebSocket support (`-ws` flag) |
There was a problem hiding this comment.
The doc says x11vnc is started with -ws for built-in WebSocket support, but the generated startup script currently starts x11vnc without -ws and the relay doesn’t dial a ws:// endpoint. Please reconcile this flag/behavior in the documentation vs implementation.
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage -ws` — VNC server with built-in WebSocket support (`-ws` flag) | |
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage` — VNC server exposing the desktop over TCP; WebSocket transport is handled by the Go relay |
| // Verify the session exists and select the target window | ||
| // Detect window type BEFORE WebSocket upgrade so desktop can use hijack | ||
| windows, err := s.tmux.ListWindows(session, server) | ||
| if err != nil || windows == nil { |
There was a problem hiding this comment.
Relay now returns an HTTP 404 before upgrading to WebSocket when the session is missing. TerminalClient’s reconnect/redirect logic relies on receiving WS close code 4004; an HTTP handshake failure won’t surface that code to the client. Consider upgrading first and then sending a WebSocket close frame with 4004 (as before) for not-found cases.
| if err != nil || windows == nil { | |
| if err != nil || windows == nil { | |
| // For WebSocket clients, complete the upgrade and then send a WS close frame | |
| // with code 4004 so TerminalClient can handle reconnect/redirect logic. | |
| if websocket.IsWebSocketUpgrade(r) { | |
| conn, uerr := upgrader.Upgrade(w, r, nil) | |
| if uerr != nil { | |
| return | |
| } | |
| defer conn.Close() | |
| closeMsg := websocket.FormatCloseMessage(4004, "Session not found") | |
| // Best-effort send of the close control frame with a short deadline. | |
| _ = conn.WriteControl(websocket.CloseMessage, closeMsg, time.Now().Add(time.Second)) | |
| return | |
| } | |
| // For non-WebSocket requests, preserve the existing HTTP 404 behavior. |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return | ||
| } |
There was a problem hiding this comment.
This endpoint reads @rk_ws_port, but the desktop creation flow sets only @rk_vnc_port and nothing sets @rk_ws_port, so this will consistently fail at runtime. Either set/read the correct option, or remove the endpoint.
2539dd9 to
a4f399b
Compare
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. OpenSSF Scorecard
Scanned Files
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 34 out of 36 changed files in this pull request and generated 8 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Count existing desktop windows to generate next number | ||
| existingWindows, _ := s.tmux.ListWindows(session, server) | ||
| n := 1 | ||
| for _, w := range existingWindows { | ||
| if strings.HasPrefix(w.Name, "desktop:") { | ||
| n++ | ||
| } | ||
| } |
There was a problem hiding this comment.
The auto-numbering strategy increments n for every existing desktop window, which can collide if there are gaps (e.g., existing desktop:1 and desktop:3 makes n==3). Consider using max(existingNumeric)+1 or picking the smallest unused integer.
| // Count existing desktop windows to generate next number | |
| existingWindows, _ := s.tmux.ListWindows(session, server) | |
| n := 1 | |
| for _, w := range existingWindows { | |
| if strings.HasPrefix(w.Name, "desktop:") { | |
| n++ | |
| } | |
| } | |
| // Determine the smallest unused desktop index to avoid collisions. | |
| existingWindows, _ := s.tmux.ListWindows(session, server) | |
| used := make(map[int]bool) | |
| for _, w := range existingWindows { | |
| if strings.HasPrefix(w.Name, "desktop:") { | |
| suffix := strings.TrimPrefix(w.Name, "desktop:") | |
| if n, err := strconv.Atoi(suffix); err == nil && n > 0 { | |
| used[n] = true | |
| } | |
| } | |
| } | |
| n := 1 | |
| for { | |
| if !used[n] { | |
| break | |
| } | |
| n++ | |
| } |
| const handleClipboardPaste = useCallback(async () => { | ||
| try { | ||
| const text = await navigator.clipboard.readText(); | ||
| if (text && rfbRef.current) { | ||
| rfbRef.current.clipboardPasteFrom = text; | ||
| } |
There was a problem hiding this comment.
Clipboard paste is implemented as rfbRef.current.clipboardPasteFrom = text;, which matches the current (incorrect) d.ts but won’t work if noVNC expects clipboardPasteFrom(text) (method) as described elsewhere in the docs. Switch to the correct noVNC clipboard API call and align the local typings accordingly.
| Each desktop window runs three processes inside a tmux window: | ||
| 1. **Xvfb** `:N -screen 0 {width}x{height}x24` — virtual framebuffer (software rendering, default 1920x1080) | ||
| 2. **Window manager** (optional) — auto-detected from host: `x-session-manager` > `$XDG_CURRENT_DESKTOP` mapping > PATH probe (`openbox`, `fluxbox`, `i3`, `xfwm4`, `mutter`, `kwin`). Bare X11 if nothing found | ||
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage -ws` — VNC server with built-in WebSocket support (`-ws` flag) |
There was a problem hiding this comment.
The x11vnc command shown here includes -ws, but the generated startup script in app/backend/api/windows.go does not. Either add -ws in the script or remove it from the docs so readers don’t assume the VNC port is a WebSocket server.
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage -ws` — VNC server with built-in WebSocket support (`-ws` flag) | |
| 3. **x11vnc** `-display :N -rfbport {port} -nopw -forever -shared -noxdamage` — VNC server |
| // Create tmux window with desktop: prefix. Auto-number if no name given. | ||
| desktopName := body.Name | ||
| if desktopName == "" || desktopName == "desktop" { | ||
| // Count existing desktop windows to generate next number | ||
| existingWindows, _ := s.tmux.ListWindows(session, server) |
There was a problem hiding this comment.
The desktop branch includes logic to auto-generate a name when body.Name == "", but the handler currently validates body.Name earlier (before branching) which would reject empty names. If you intend name to be optional for desktop windows, move/conditionalize the name validation so this auto-naming path is reachable.
| // Send C-c to kill the running bash -c (which kills Xvfb, x11vnc, WM), | ||
| // then send the full startup script at the new resolution. | ||
| if err := s.tmux.SendKeys(session, index, "C-c", server); err != nil { | ||
| slog.Error("failed to send C-c", "err", err) | ||
| } | ||
| time.Sleep(1 * time.Second) | ||
|
|
There was a problem hiding this comment.
Sleeping inside the HTTP handler (time.Sleep(1s)) will tie up a server goroutine per request and makes response latency dependent on timing. Consider doing the restart entirely inside the tmux window (a script that terminates existing processes and starts new ones) without needing a server-side sleep, or make the restart async.
| r.Post("/api/sessions/{session}/windows/{index}/select", s.handleWindowSelect) | ||
| r.Post("/api/sessions/{session}/windows/{index}/split", s.handleWindowSplit) | ||
| r.Post("/api/sessions/{session}/windows/{index}/resolution", s.handleWindowResolution) | ||
| r.Get("/api/sessions/{session}/windows/{index}/desktop-info", s.handleDesktopInfo) |
There was a problem hiding this comment.
This route registers /desktop-info, but the corresponding handler reads @rk_ws_port which isn’t set anywhere and nothing calls this endpoint. Removing it would reduce API surface and avoid confusion.
| r.Get("/api/sessions/{session}/windows/{index}/desktop-info", s.handleDesktopInfo) |
| // Detect window type BEFORE WebSocket upgrade so desktop can use hijack | ||
| windows, err := s.tmux.ListWindows(session, server) | ||
| if err != nil || windows == nil { | ||
| slog.Warn("session not found", "session", session) | ||
| conn.WriteMessage(websocket.CloseMessage, | ||
| websocket.FormatCloseMessage(4004, "Session not found")) | ||
| http.Error(w, "Session not found", http.StatusNotFound) | ||
| return | ||
| } | ||
| var windowType string | ||
| windowFound := false | ||
| for _, win := range windows { | ||
| if win.Index == winIdx { | ||
| windowType = win.Type | ||
| windowFound = true | ||
| break | ||
| } | ||
| } | ||
| if !windowFound { | ||
| http.Error(w, "Window not found", http.StatusNotFound) | ||
| return | ||
| } |
There was a problem hiding this comment.
For desktop windows, session/window-not-found errors currently return plain HTTP 404s before upgrading to WebSocket, while terminal windows use WebSocket close codes (e.g., 4004). For consistency (and so clients can reliably detect not-found), consider upgrading (with the desktop upgrader) and closing with the same close codes/messages as the terminal path.
| - **THEN** it reads `@rk_vnc_port` via `tmux show-options -wv -t {session}:{window}` | ||
| - **AND** dials a WebSocket connection to `ws://localhost:{port}` | ||
| - **AND** bidirectionally copies data between browser and VNC WebSockets | ||
|
|
There was a problem hiding this comment.
This relay section describes dialing ws://localhost:{port} and doing a WebSocket-to-WebSocket proxy, but the current implementation dials tcp://127.0.0.1:{port} and bridges WebSocket↔TCP (websockify-style). Please align the spec with the implementation to avoid confusion for future changes and debugging.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 39 out of 41 changed files in this pull request and generated 6 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Wait for async import and connection | ||
| await vi.waitFor(() => { | ||
| expect(mockRFB).toHaveBeenCalled(); | ||
| }); |
There was a problem hiding this comment.
This test uses vi.waitFor(...), but elsewhere in the codebase the async retry helper comes from Testing Library (waitFor from @testing-library/react). Unless you’ve explicitly added a vi.waitFor helper, this will fail at runtime. Import and use Testing Library’s waitFor for consistency (or add a shared helper if preferred).
| session := chi.URLParam(r, "session") | ||
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } | ||
|
|
||
| index, ok := parseWindowIndex(r) | ||
| if !ok { | ||
| writeError(w, http.StatusBadRequest, "Invalid window index") | ||
| return | ||
| } | ||
|
|
||
| server := serverFromRequest(r) | ||
|
|
||
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return | ||
| } | ||
|
|
||
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) |
There was a problem hiding this comment.
handleDesktopInfo//desktop-info reads @rk_ws_port, but nothing in this PR sets that tmux window option (desktop windows only set @rk_vnc_port), and the frontend doesn’t call this endpoint. This makes the route effectively dead and potentially confusing to maintain. Either remove the endpoint/route, or implement and document setting @rk_ws_port if it’s meant to be part of the desktop lifecycle.
| session := chi.URLParam(r, "session") | |
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | |
| writeError(w, http.StatusBadRequest, errMsg) | |
| return | |
| } | |
| index, ok := parseWindowIndex(r) | |
| if !ok { | |
| writeError(w, http.StatusBadRequest, "Invalid window index") | |
| return | |
| } | |
| server := serverFromRequest(r) | |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | |
| if err != nil { | |
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | |
| return | |
| } | |
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) | |
| // This endpoint is currently not part of the desktop lifecycle and no code | |
| // sets the @rk_ws_port tmux option it previously attempted to read. | |
| // To avoid exposing a misleading or non-functional API surface, we explicitly | |
| // report it as unsupported instead of attempting to access unset state. | |
| writeError(w, http.StatusNotFound, "Desktop info endpoint is not supported") |
| ### macOS Setup | ||
|
|
||
| ```bash | ||
| brew install --cask xquartz | ||
| brew install x11vnc | ||
| ``` | ||
|
|
||
| **Log out and back in** (or reboot) after installing XQuartz. | ||
|
|
There was a problem hiding this comment.
The macOS setup instructions here recommend installing XQuartz + x11vnc, but the current implementation on runtime.GOOS == "darwin" starts rk-virtual-display (CGVirtualDisplay + ScreenCaptureKit + libvncserver) instead of Xvfb/x11vnc. Please update the README to reflect the actual macOS dependency chain (how to build/install rk-virtual-display, required Homebrew deps like libvncserver, and the needed macOS permissions) or switch the macOS startup script back to the documented XQuartz+x11vnc pipeline.
| ## macOS Support | ||
|
|
||
| macOS uses the same Xvfb + x11vnc pipeline as Linux, via [XQuartz](https://www.xquartz.org/). | ||
|
|
||
| ### Setup | ||
|
|
||
| ```bash | ||
| brew install --cask xquartz | ||
| brew install x11vnc | ||
| ``` | ||
|
|
||
| After installing XQuartz, **log out and back in** (or reboot) so the X11 environment is available. The startup script automatically adds `/opt/X11/bin` to PATH. | ||
|
|
||
| ### How It Works | ||
|
|
||
| Identical to Linux — Xvfb creates a virtual X display, x11vnc serves it over VNC, run-kit relays it to the browser. Each desktop is fully isolated with its own display number, VNC port, and XDG directories. | ||
|
|
||
| ### Window Managers | ||
|
|
||
| XQuartz ships with `quartz-wm`, which is automatically detected and used if no other WM is found. You can also install X11 window managers via Homebrew (e.g., `brew install openbox`, `brew install i3`). | ||
|
|
||
| ### Troubleshooting | ||
|
|
||
| | Symptom | Fix | | ||
| |---------|-----| | ||
| | `Xvfb not found` | Install XQuartz: `brew install --cask xquartz`, then log out/in | | ||
| | `x11vnc not found` | `brew install x11vnc` | | ||
| | Black screen after install | Reboot — XQuartz needs a fresh login session | | ||
| | No window decorations | Install a WM: `brew install openbox` | | ||
|
|
There was a problem hiding this comment.
This macOS section documents an XQuartz + Xvfb + x11vnc pipeline, but the backend now generates a different macOS startup script that executes rk-virtual-display (native CGVirtualDisplay + libvncserver). Please update this section (and troubleshooting rows like “Xvfb not found”) to match the current macOS implementation and its prerequisites/permissions, otherwise users will follow instructions that don’t apply.
| #import <Foundation/Foundation.h> | ||
| #import <CoreGraphics/CoreGraphics.h> | ||
| #import <IOSurface/IOSurface.h> | ||
| #import <ScreenCaptureKit/ScreenCaptureKit.h> | ||
| #import "CGVirtualDisplayPrivate.h" | ||
| #include <rfb/rfb.h> | ||
| #include <signal.h> |
There was a problem hiding this comment.
main.m uses pthread_mutex_t, memcpy, strcmp, atoi, fprintf/printf, usleep, and AXIsProcessTrustedWithOptions without including the headers that declare them. On modern Clang/C99 this will typically fail to compile due to missing prototypes/types. Add the needed system headers (e.g. pthread.h, string.h, stdio.h, stdlib.h, unistd.h, and an Accessibility header under ApplicationServices) and CoreMedia/CoreVideo headers for CMSampleBufferRef/CVPixelBuffer* usage.
| // Create tmux window with desktop: prefix. Auto-number if no name given. | ||
| desktopName := body.Name | ||
| if desktopName == "" || desktopName == "desktop" { | ||
| // Count existing desktop windows to generate next number | ||
| existingWindows, _ := s.tmux.ListWindows(session, server) |
There was a problem hiding this comment.
The desktopName == "" branch here is effectively unreachable because earlier in this handler ValidateName(body.Name, "Window name") rejects empty names. If you want desktop windows to allow omitted/empty name (auto-numbering), the validation needs to happen after defaulting (or be conditional for type:"desktop").
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 43 changed files in this pull request and generated 7 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)
app/backend/api/windows.go:42
- handleWindowCreate validates body.Name before branching on body.Type, which prevents creating a desktop window with an empty name even though the desktop branch tries to auto-number when name is empty. Move/adjust the name validation so desktop creation can allow an empty name (or validate after the auto-number/defaulting logic).
var body struct {
Name string `json:"name"`
CWD string `json:"cwd"`
Type string `json:"type"`
Resolution string `json:"resolution"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "Invalid JSON body")
return
}
if errMsg := validate.ValidateName(body.Name, "Window name"); errMsg != "" {
writeError(w, http.StatusBadRequest, errMsg)
return
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #import <Foundation/Foundation.h> | ||
| #import <CoreGraphics/CoreGraphics.h> | ||
| #import <IOSurface/IOSurface.h> | ||
| #import <ScreenCaptureKit/ScreenCaptureKit.h> | ||
| #import "CGVirtualDisplayPrivate.h" | ||
| #include <rfb/rfb.h> | ||
| #include <signal.h> | ||
|
|
There was a problem hiding this comment.
main.m uses pthread_mutex_t, memcpy/strcmp/atoi/fprintf, and usleep, but doesn't include the standard headers that declare them. This can fail to compile on stricter toolchains. Add the missing includes (e.g. pthread.h, string.h, stdlib.h, stdio.h, unistd.h) explicitly instead of relying on transitive includes.
| // handleDesktopInfo returns the websockify port for a desktop window. | ||
| func (s *Server) handleDesktopInfo(w http.ResponseWriter, r *http.Request) { | ||
| session := chi.URLParam(r, "session") | ||
| if errMsg := validate.ValidateName(session, "Session name"); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } | ||
|
|
||
| index, ok := parseWindowIndex(r) | ||
| if !ok { | ||
| writeError(w, http.StatusBadRequest, "Invalid window index") | ||
| return | ||
| } | ||
|
|
||
| server := serverFromRequest(r) | ||
|
|
||
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | ||
| return | ||
| } | ||
|
|
||
| writeJSON(w, http.StatusOK, map[string]string{"wsPort": wsPortStr}) | ||
| } |
There was a problem hiding this comment.
handleDesktopInfo reads @rk_ws_port, but this option is never set anywhere in the PR, so this endpoint will always 404 and the name/comment still refer to "websockify" even though the relay now dials x11vnc directly. Either remove this endpoint/route, or implement setting and using @rk_ws_port consistently (and update naming/docs accordingly).
| // Move event | ||
| CGEventRef moveEvent = CGEventCreateMouseEvent(NULL, kCGEventMouseMoved, point, kCGMouseButtonLeft); | ||
| if (moveEvent) { | ||
| CGEventPost(kCGHIDEventTap, moveEvent); | ||
| CFRelease(moveEvent); | ||
| } |
There was a problem hiding this comment.
handlePointerEvent always posts kCGEventMouseMoved for pointer motion, even when a button is held. This prevents proper drag interactions in the remote desktop (dragging typically requires kCGEventLeftMouseDragged / kCGEventRightMouseDragged / kCGEventOtherMouseDragged depending on the held button). Track whether a button is down and post the appropriate dragged event type when moving.
| CGEventRef event = CGEventCreateKeyboardEvent(NULL, 0, down ? true : false); | ||
| if (!event) return; | ||
|
|
||
| // Handle basic ASCII keysyms | ||
| if (keySym >= 0x20 && keySym <= 0x7E) { | ||
| UniChar ch = (UniChar)keySym; | ||
| CGEventKeyboardSetUnicodeString(event, 1, &ch); | ||
| } else if (keySym == 0xFF0D) { // Return |
There was a problem hiding this comment.
handleKeyEvent creates a keyboard event with keycode 0 by default and will still post it for any keysym that isn't explicitly mapped. That means unknown/special keysyms can generate unintended keycode-0 events. Consider returning early (no event) for unmapped keysyms, or expanding the mapping so only intentional events are posted.
| dispatch_semaphore_wait(sem, dispatch_time(DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC)); | ||
|
|
||
| if (setupError) { | ||
| fprintf(stderr, "ERROR: Screen capture setup failed: %s\n", | ||
| [[setupError localizedDescription] UTF8String]); | ||
| fprintf(stderr, "Grant Screen Recording permission in System Settings > Privacy & Security.\n"); | ||
| return 1; | ||
| } |
There was a problem hiding this comment.
ScreenCaptureKit setup waits up to 10s on a semaphore, but the return value of dispatch_semaphore_wait is ignored. If the wait times out, setupError may still be nil and the program will continue without a configured capture stream. Check for timeout and fail fast with a clear error if initialization didn't complete in time.
| // Create tmux window with desktop: prefix. Auto-number if no name given. | ||
| desktopName := body.Name | ||
| if desktopName == "" || desktopName == "desktop" { | ||
| // Count existing desktop windows to generate next number | ||
| existingWindows, _ := s.tmux.ListWindows(session, server) | ||
| n := 1 | ||
| for _, w := range existingWindows { | ||
| if strings.HasPrefix(w.Name, "desktop:") { | ||
| n++ | ||
| } | ||
| } | ||
| desktopName = strconv.Itoa(n) | ||
| } | ||
| windowName := "desktop:" + desktopName | ||
| var resolvedCwd string |
There was a problem hiding this comment.
Desktop creation always prefixes the requested name with "desktop:", but ValidateName now allows user input that already starts with "desktop:". If a client sends name="desktop:dev", this will create a window named "desktop:desktop:dev". Consider normalizing user input by stripping a leading "desktop:" before adding the prefix, or rejecting names that already include the prefix.
| // handleDesktopRelay proxies between a browser WebSocket and x11vnc's raw TCP VNC port. | ||
| func (s *Server) handleDesktopRelay(w http.ResponseWriter, r *http.Request, session string, windowIndex int, server string) { | ||
| portStr, err := s.tmux.GetWindowOption(session, windowIndex, "@rk_vnc_port", server) | ||
| if err != nil { | ||
| slog.Warn("VNC port not found", "session", session, "window", windowIndex, "err", err) | ||
| http.Error(w, "VNC port not found", http.StatusBadGateway) | ||
| return | ||
| } | ||
| port, err := strconv.Atoi(portStr) | ||
| if err != nil { | ||
| http.Error(w, "Invalid VNC port", http.StatusBadGateway) | ||
| return | ||
| } | ||
|
|
||
| vncAddr := fmt.Sprintf("127.0.0.1:%d", port) | ||
| vncConn, err := net.DialTimeout("tcp", vncAddr, 10*time.Second) | ||
| if err != nil { | ||
| slog.Error("failed to connect to VNC server", "addr", vncAddr, "err", err) | ||
| http.Error(w, "VNC connection failed", http.StatusBadGateway) | ||
| return | ||
| } |
There was a problem hiding this comment.
PR description/spec mention a WebSocket-to-WebSocket proxy to x11vnc, but handleDesktopRelay actually implements a WebSocket-to-TCP proxy (net.DialTimeout to 127.0.0.1:port). Please align the implementation and docs/PR description (either update docs/spec to reflect WS↔TCP, or switch to dialing x11vnc's WebSocket mode and proxy WS↔WS).
3616362 to
2e8ecb6
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 43 changed files in this pull request and generated 6 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Create tmux window with desktop: prefix. Auto-number if no name given. | ||
| desktopName := body.Name | ||
| if desktopName == "" || desktopName == "desktop" { | ||
| // Count existing desktop windows to generate next number | ||
| existingWindows, _ := s.tmux.ListWindows(session, server) | ||
| n := 1 | ||
| for _, w := range existingWindows { | ||
| if strings.HasPrefix(w.Name, "desktop:") { | ||
| n++ | ||
| } | ||
| } | ||
| desktopName = strconv.Itoa(n) | ||
| } | ||
| windowName := "desktop:" + desktopName | ||
| var resolvedCwd string |
There was a problem hiding this comment.
In the desktop creation path, the "auto-number if no name given" branch checks desktopName == "", but handleWindowCreate currently rejects empty name earlier, so that part of the condition is unreachable. Also, since ValidateName now permits names starting with desktop:, a caller could pass name: "desktop:foo" and end up with a double prefix (desktop:desktop:foo). Consider (a) allowing empty name only for desktop windows, and (b) explicitly rejecting : in the user-provided desktop label since the handler itself adds the desktop: prefix.
| if [ "$(uname)" = "Darwin" ] && [ -f "$REPO_ROOT/tools/rk-virtual-display/Makefile" ]; then | ||
| if brew --prefix libvncserver &>/dev/null; then | ||
| echo "==> Building rk-virtual-display (macOS virtual display helper)..." | ||
| make -C "$REPO_ROOT/tools/rk-virtual-display" clean all | ||
| cp "$REPO_ROOT/tools/rk-virtual-display/rk-virtual-display" "$REPO_ROOT/dist/" |
There was a problem hiding this comment.
With set -e, calling brew without checking that Homebrew exists will make this build script fail on macOS hosts that don't have brew installed. Guard with command -v brew >/dev/null (or similar) before running brew --prefix ... so the build still succeeds when Homebrew isn't present.
| sendPointerToCanvas("mousedown", 0); | ||
| sendPointerToCanvas("mouseup", 0); |
There was a problem hiding this comment.
The comment says this long-press timer simulates a right-click, but it sends a left-click (button=0) down/up. If the intent is right-click, use the right mouse button (button=2) and ensure the buttons bitmask matches so noVNC interprets it as a context-click.
| sendPointerToCanvas("mousedown", 0); | |
| sendPointerToCanvas("mouseup", 0); | |
| sendPointerToCanvas("mousedown", 2); | |
| sendPointerToCanvas("mouseup", 2); |
| const handleClipboardPaste = useCallback(async () => { | ||
| try { const text = await navigator.clipboard.readText(); if (text && rfbRef.current) rfbRef.current.clipboardPasteFrom = text; } catch { /* denied */ } | ||
| setMenuOpen(false); |
There was a problem hiding this comment.
Clipboard paste is implemented by assigning to rfb.clipboardPasteFrom, which is not how noVNC's API works (it is typically a method call). As-is this will likely be a no-op at runtime (or overwrite the function), breaking clipboard paste. Call the clipboard API method instead (and adjust the TypeScript typings accordingly).
| if err := s.tmux.SendKeys(session, index, "C-c", server); err != nil { | ||
| slog.Error("failed to send C-c", "err", err) | ||
| } | ||
| time.Sleep(1 * time.Second) |
There was a problem hiding this comment.
Using time.Sleep inside the HTTP handler makes the resolution-change endpoint block for at least 1s and is brittle (it doesn't guarantee the old processes have exited, and it ties server latency to timing assumptions). Prefer a non-blocking approach (e.g., have the restart script handle teardown/startup sequencing, or poll the tmux window/process state) rather than sleeping in the request path.
| time.Sleep(1 * time.Second) |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | ||
| if err != nil { | ||
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") |
There was a problem hiding this comment.
This handler reads @rk_ws_port, but that option is never set anywhere in this PR (desktop windows store @rk_vnc_port). As written it will always 404; consider removing this endpoint or switching it to the correct option/key that the rest of the desktop flow uses.
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_ws_port", server) | |
| if err != nil { | |
| writeError(w, http.StatusNotFound, "Not a desktop window or websockify port not set") | |
| wsPortStr, err := s.tmux.GetWindowOption(session, index, "@rk_vnc_port", server) | |
| if err != nil { | |
| writeError(w, http.StatusNotFound, "Not a desktop window or VNC/websockify port not set") |
497c866 to
fcb3759
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 43 changed files in this pull request and generated 5 comments.
Files not reviewed (1)
- app/frontend/pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #import <Foundation/Foundation.h> | ||
| #import <CoreGraphics/CoreGraphics.h> | ||
| #import <IOSurface/IOSurface.h> | ||
| #import <ScreenCaptureKit/ScreenCaptureKit.h> | ||
| #import "CGVirtualDisplayPrivate.h" | ||
| #include <rfb/rfb.h> | ||
| #include <signal.h> | ||
|
|
There was a problem hiding this comment.
main.m uses pthread_mutex_t/pthread_mutex_lock, usleep, and AXIsProcessTrustedWithOptions but does not include the corresponding headers (<pthread.h>, <unistd.h>, and an Accessibility header such as <ApplicationServices/ApplicationServices.h> / <ApplicationServices/AXUIElement.h>). With -Wall, this is likely to fail compilation (unknown types / implicit declarations). Add the missing includes near the top of the file.
| // Keepalive pings every 10s | ||
| go func() { | ||
| ticker := time.NewTicker(10 * time.Second) | ||
| defer ticker.Stop() | ||
| for range ticker.C { | ||
| if err := conn.WriteControl(websocket.PingMessage, nil, time.Now().Add(5*time.Second)); err != nil { | ||
| cleanup() | ||
| return | ||
| } | ||
| } | ||
| }() | ||
|
|
||
| // Browser WebSocket → VNC TCP | ||
| go func() { | ||
| defer cleanup() | ||
| for { | ||
| _, msg, err := conn.ReadMessage() | ||
| if err != nil { | ||
| return | ||
| } | ||
| if _, err := vncConn.Write(msg); err != nil { | ||
| return | ||
| } | ||
| } | ||
| }() | ||
|
|
||
| // VNC TCP → Browser WebSocket | ||
| buf := make([]byte, 32*1024) | ||
| for { | ||
| n, err := vncConn.Read(buf) | ||
| if err != nil { | ||
| return | ||
| } | ||
| if err := conn.WriteMessage(websocket.BinaryMessage, buf[:n]); err != nil { | ||
| return |
There was a problem hiding this comment.
handleDesktopRelay writes to the same *websocket.Conn from multiple goroutines (the keepalive ping goroutine via WriteControl and the VNC->WS loop via WriteMessage). Gorilla WebSocket connections are not safe for concurrent writers and this can panic/corrupt frames. Use a single writer goroutine (e.g., channel) or a sync.Mutex to serialize all writes (including pings).
| // Desktop window creation | ||
| if body.Type == "desktop" { | ||
| resolution := body.Resolution | ||
| if resolution == "" { | ||
| resolution = "1920x1080" | ||
| } | ||
| if errMsg := validate.ValidateResolution(resolution); errMsg != "" { | ||
| writeError(w, http.StatusBadRequest, errMsg) | ||
| return | ||
| } |
There was a problem hiding this comment.
Because name validation happens before branching on body.Type, a terminal request can now create a window named desktop:... (allowed by ValidateName) and it will later be misclassified as a desktop window by parseWindows, breaking relay behavior. Also, desktop creation intends to auto-number when name is empty, but an empty name currently fails the earlier validation so that path is unreachable. Consider moving/adjusting validation per type: reject desktop: prefix for terminal windows, and validate the final derived desktop window name (or allow empty name) for desktop windows.
| ### macOS Setup | ||
|
|
||
| ```bash | ||
| brew install --cask xquartz | ||
| brew install x11vnc | ||
| ``` | ||
|
|
||
| **Log out and back in** (or reboot) after installing XQuartz. | ||
|
|
||
| ### Linux Setup |
There was a problem hiding this comment.
The README’s “macOS Setup” section currently instructs installing XQuartz + x11vnc, but the implementation on macOS starts rk-virtual-display (built from tools/rk-virtual-display) instead. Please align the README with the actual macOS dependency/install flow (including how rk-virtual-display gets built/installed) to avoid broken setup steps.
| WebSocket-to-WebSocket proxy between the browser and x11vnc. No PTY involved. | ||
|
|
||
| Per connection: | ||
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port` — returns WebSocket close code `4002` if not found | ||
| 2. Dials `ws://localhost:{port}` to connect to x11vnc's built-in WebSocket server (localhost only, no external connections) | ||
| 3. Bidirectional copy: two goroutines relay messages between browser WebSocket and VNC WebSocket | ||
| 4. On disconnect (either side): `sync.Once` cleanup closes both WebSocket connections | ||
|
|
||
| Client-side WebSocket reconnection: exponential backoff (1s, 2s, 4s, 8s, 16s, max 30s) on unexpected close. Shows `[reconnecting...]` in terminal. Re-sends resize on successful reconnect. Skips reconnect on component unmount. On close code `4004` (session/window not found): shows `[session not found]` and navigates to `/` instead of reconnecting. Terminal page connects via `ws://${location.host}/relay/{session}/{window}?server={runkit|default}` — same host, server param from session metadata. |
There was a problem hiding this comment.
This section describes the desktop relay as WebSocket-to-WebSocket to ws://localhost:{port} and mentions WS close codes for not-found cases, but the current implementation in api/relay.go dials a raw TCP VNC connection (net.DialTimeout("tcp", ...)) and currently returns HTTP 404 before upgrading in some not-found paths. Please update the architecture doc to match the actual transport and error/close semantics so operators and future contributors aren’t misled.
| WebSocket-to-WebSocket proxy between the browser and x11vnc. No PTY involved. | |
| Per connection: | |
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port` — returns WebSocket close code `4002` if not found | |
| 2. Dials `ws://localhost:{port}` to connect to x11vnc's built-in WebSocket server (localhost only, no external connections) | |
| 3. Bidirectional copy: two goroutines relay messages between browser WebSocket and VNC WebSocket | |
| 4. On disconnect (either side): `sync.Once` cleanup closes both WebSocket connections | |
| Client-side WebSocket reconnection: exponential backoff (1s, 2s, 4s, 8s, 16s, max 30s) on unexpected close. Shows `[reconnecting...]` in terminal. Re-sends resize on successful reconnect. Skips reconnect on component unmount. On close code `4004` (session/window not found): shows `[session not found]` and navigates to `/` instead of reconnecting. Terminal page connects via `ws://${location.host}/relay/{session}/{window}?server={runkit|default}` — same host, server param from session metadata. | |
| WebSocket-to-TCP proxy between the browser and the local VNC server (for example, x11vnc). No PTY involved. | |
| Per connection: | |
| 1. Reads `@rk_vnc_port` window option via `tmux show-options -wv -t {session}:{window} @rk_vnc_port`. If the option is missing or invalid, the handler responds with an HTTP 404 (session/window/VNC port not found) and does **not** upgrade to WebSocket. | |
| 2. Dials `localhost:{port}` over plain TCP to connect to the VNC server (localhost only, no external connections). | |
| 3. Bidirectional copy: two goroutines relay bytes between the browser WebSocket and the TCP VNC connection. | |
| 4. On disconnect (either side): `sync.Once` cleanup closes both the browser WebSocket and the TCP VNC connection. | |
| Client-side WebSocket reconnection: exponential backoff (1s, 2s, 4s, 8s, 16s, max 30s) on unexpected close. Shows `[reconnecting...]` in terminal. Re-sends resize on successful reconnect. Skips reconnect on component unmount. When the relay endpoint indicates “not found” (for example, HTTP 404 before the WebSocket is established, due to a missing session/window or `@rk_vnc_port`), the UI shows `[session not found]` and navigates to `/` instead of reconnecting. Terminal page connects via `ws://${location.host}/relay/{session}/{window}?server={runkit|default}` — same host, server param from session metadata. |
Integrate desktop windows into run-kit's session model using Xvfb + x11vnc + noVNC. Desktop windows are tmux-managed VNC servers that stream through the existing WebSocket relay. Backend: unified window creation with type parameter, VNC WebSocket proxy in relay handler, dynamic port/display allocation, resolution validation and change endpoint. Frontend: DesktopClient component with noVNC scaleViewport, desktop bottom bar (clipboard, resolution, fullscreen), creation from command palette/breadcrumb/dashboard.
- Downgrade @novnc/novnc from 1.6.0 to 1.5.0 (1.6.0 has broken CJS with top-level await that Rollup can't handle) - Remove x11vnc -ws flag (not supported in v0.9.16) - Replace WebSocket-to-WebSocket relay with WebSocket-to-TCP proxy (x11vnc speaks raw VNC, not WebSocket) - Add desktop-specific WebSocket upgrader with 'binary' subprotocol - Fix useEffect dependency causing reconnect on every SSE update (~2.5s) — store callbacks in refs instead of deps - Skip activeWindow sync for desktop windows (VNC doesn't use tmux attach, so tmux active window concept doesn't apply) - Wait for window type from SSE before rendering to prevent TerminalClient connecting to desktop relay - Fix WM detection: resolve x-session-manager symlink to detect startplasma-x11, wrap full DEs with dbus-run-session - Rewrite startup script as proper bash -c block instead of one-liner chain (fixes shell parsing issues with backgrounding) - Add desktop-info API endpoint for future use Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hidden textarea triggers the virtual keyboard on mobile. Characters go through onInput (how mobile keyboards work), special keys (Enter, Backspace, arrows) go through onKeyDown. Both forward X11 keysyms to noVNC via sendKey. Kbd button positioned on the right side of the desktop bottom bar. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix zero-sized noVNC canvas (inner container needed w-full h-full) - Fix activeWindow sync redirecting away from desktop windows (use ref) - Fix resolution change: keep shell alive (no exec), send C-c + fresh startup script instead of one-liner that typed into x11vnc stdin - Add portrait resolutions (720x1280, 1080x1920, 1440x2560) with grouped dropdown (Portrait/Landscape) - Add mobile keyboard support via hidden textarea + onInput handler - Add WebSocket keepalive pings for desktop relay - Add pinch-to-zoom touch handlers for mobile desktop viewing - Fix WM detection: resolve x-session-manager symlink, wrap plasma with dbus-run-session, use proper bash -c script block - Add clipViewport and ResizeObserver for noVNC scaling - Add desktop-info API endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Write startup script to temp file instead of send-keys (tmux send-keys truncates at ~1024 bytes, cutting off WM detection) - Add per-desktop XDG isolation (XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_RUNTIME_DIR, etc.) so browsers and other single-instance apps run independently across desktops - Fix XDG_RUNTIME_DIR permissions (chmod 0700, required by D-Bus) - Remove colon from desktop ID paths (D-Bus rejects paths with :) - Fix resolution change: send C-c to kill running script, then execute fresh startup script from temp file - Remove exec from x11vnc launch (keep shell alive for resolution changes) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Chrome ignores XDG_CONFIG_HOME and PATH wrappers when launched from KDE app menu (.desktop files use absolute Exec= paths). Fix: patch .desktop files in XDG_DATA_HOME/applications/ to point to wrapper scripts that add --user-data-dir per desktop. Each desktop gets an independent Chrome profile. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Auto-number desktop windows (desktop:1, desktop:2, etc.) instead of redundant desktop:desktop - Disable KDE Wallet in virtual sessions (blocks browser network requests waiting for unlock) - Add --password-store=basic to Chrome wrapper - Allow desktop: prefix in window rename validation (colon was previously forbidden for all names) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers architecture, startup sequence, DE support (KDE/GNOME/Xfce/ Openbox/i3), per-desktop isolation, resolution management, Chrome isolation workarounds, naming conventions, and troubleshooting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lists server requirements (xvfb, x11vnc, DE packages) and usage instructions. Links to detailed docs/desktop-streaming.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On macOS, desktop windows connect to the built-in Screen Sharing VNC server (port 5900) instead of starting Xvfb + x11vnc. Resolution change returns 400 on macOS (can't resize the real display). Docs updated with macOS setup instructions and three future options (Docker-based Linux desktops, native virtual displays via CGVirtualDisplay). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace macOS Screen Sharing approach with a native virtual display helper (rk-virtual-display) that creates an isolated virtual monitor using CGVirtualDisplay + ScreenCaptureKit + libvncserver. This gives macOS the same experience as Linux: isolated virtual desktops with custom resolutions, no auth, and full mouse/keyboard control. - Add tools/rk-virtual-display: Objective-C CLI that creates a virtual display, captures it via ScreenCaptureKit, and serves over VNC - Remove macOS Screen Sharing code path and VNC handshake reorder - Remove credential prompt UI from desktop-client.tsx (no auth needed) - Remove macOS guard on resolution change - Fix daemon restart failing when kill-session exits the tmux server - Build script auto-builds rk-virtual-display on macOS - Update docs with macOS setup (brew install libvncserver) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detailed implementation plan for 6 improvements to make the mobile desktop streaming experience competitive with native VNC apps: trackpad mode, right-click/scroll gestures, modifier key bar, scroll vs pinch discrimination, double-tap zoom, and drag support. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trackpad mode: drag moves cursor indirectly (like a laptop trackpad), long press clicks at cursor position. Toggle via Tap/Pad button in bottom bar. Canvas pointer-events disabled in trackpad mode to prevent noVNC from intercepting touches. Modifier key bar (touch devices only): sticky Ctrl/Alt/Shift/Meta toggles, double-tap to lock. Esc, Tab, arrow keys. Modifiers wrap the next keypress then auto-clear (unless locked). Bottom bar now scrollable (overflow-x-auto) to fit all controls. Known limitation: after long press click + release, one tap needed before cursor moves again (mobile browser limitation with synthetic mouse events on pointer-events:none canvas). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite desktop bottom bar to mirror terminal bottom bar: Esc, Tab, Ctrl, Alt, Fn dropdown (F1-F12 + PgUp/PgDn/Home/End/Ins/Del), ArrowPad, desktop menu (monitor icon with trackpad/paste/fullscreen/ resolution), ⌘K, keyboard toggle (right end), dismiss keyboard. Uses same useModifierState hook and ArrowPad component as terminal bar. Desktop-specific actions (touch mode, clipboard, resolution) in a popup menu triggered by the monitor icon. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In trackpad mode, when zoomed in, the view automatically scrolls to follow the cursor as it approaches the edge of the visible area. 40px edge margin, 15px pan speed per touch frame. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fcb3759 to
71853bd
Compare
Summary
run-kit currently streams tmux terminal windows to the browser via xterm.js, but some workflows require a graphical desktop — browser testing, GUI applications, visual debugging. This adds desktop streaming into run-kit's existing session/window model using Xvfb + x11vnc + noVNC, so users get terminals and desktops in the same UI, managed by the same tmux lifecycle.
Changes
type: "desktop", spins up Xvfb + WM detection + x11vnc in a tmux window/relay/{session}/{window}auto-detects window type, branches to VNC WebSocket proxy for desktop windowsWindowInfo.Typederived fromdesktop:window name prefix (convention over configuration)net.Listen(":0")avoids collisions; VNC port stored in tmux window option (@rk_vnc_port)POST /api/sessions/{session}/windows/{index}/resolutionrestarts Xvfb at new sizescaleViewportfor client-side scaling with aspect ratioChange
Stats
intake → spec → tasks → apply → review → hydrate → ship