Skip to content

fix(computer-use): 修复权限检查和应用列表获取的问题#157

Open
mcjjin wants to merge 4 commits intoclaude-code-best:mainfrom
mcjjin:main
Open

fix(computer-use): 修复权限检查和应用列表获取的问题#157
mcjjin wants to merge 4 commits intoclaude-code-best:mainfrom
mcjjin:main

Conversation

@mcjjin
Copy link
Copy Markdown

@mcjjin mcjjin commented Apr 6, 2026

修复 macOS 平台下权限检查的 JXA 回退逻辑,确保在没有原生模块时仍能正确检测权限
改进应用列表获取方式,使用 mdls 获取真实的 bundleId 而非生成伪 ID

Summary by CodeRabbit

Release Notes

  • Bug Fixes
    • Improved application discovery with enhanced detection mechanisms that better identify installed applications and retrieve proper identification information
    • Enhanced robustness by implementing fallback verification methods for accessibility and screen recording permissions when native platform modules are unavailable
    • Increased resilience of escape hotkey functionality by safely handling scenarios where native module components may be missing

mcjjin and others added 4 commits April 6, 2026 13:56
修复 macOS 平台下权限检查的 JXA 回退逻辑,确保在没有原生模块时仍能正确检测权限
改进应用列表获取方式,使用 mdls 获取真实的 bundleId 而非生成伪 ID
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 6, 2026

📝 Walkthrough

Walkthrough

This pull request updates the macOS computer-use backend to improve application discovery and permission handling. The app enumeration now scans multiple standard directories and retrieves bundle identifiers via Spotlight metadata instead of AppleScript. Permission checks add JXA-based fallback probes when the native Swift module is unavailable, and the hotkey module access is now protected with optional chaining.

Changes

Cohort / File(s) Summary
Application Discovery
packages/@ant/computer-use-swift/src/backends/darwin.ts
Replaced AppleScript-based bundle identifier construction with directory scanning (/Applications, ~/Applications, /System/Applications) and mdls metadata queries; added deduplication by bundleId.
Permission Handling Fallbacks
src/utils/computerUse/hostAdapter.ts
Added JXA-based probe functions for accessibility (osascript System Events enumeration) and screen recording (screencapture to /dev/null); updated ensureOsPermissions to use fallback probes when Swift tcc module is unavailable.
Hotkey Module Safety
src/utils/computerUse/escHotkey.ts
Applied optional chaining (hotkey?.registerEscape, hotkey?.unregister, hotkey?.notifyExpectedEscape) to gracefully handle missing hotkey module without throwing errors.

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly Related PRs

  • PR #98 — Modifies the same macOS computer-use backend files (darwin.ts, hostAdapter.ts) and app listing/permission handling logic.
  • PR #108 — Changes the same darwin app enumeration using Spotlight/mdls and hostAdapter permission checks with fallback mechanisms.
  • PR #137 — Updates identical macOS-specific paths for app discovery via mdls, permission probes using JXA, and optional hotkey handling.

Suggested Reviewers

  • KonghaYao

Poem

🐰 Whiskers twitch with glee,
Swift and JXA dance free,
mdls brings light where AppleScript once did gleam,
Fallbacks blooming, a resilient macOS dream!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title uses Chinese text and doesn't provide clear information about the changes in English, making it difficult for non-Chinese speakers to understand the main changeset from PR history. Consider using English in the PR title for broader team accessibility, or provide an English translation: 'fix(computer-use): Fix permission checks and app list retrieval on macOS'
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts:
- Around line 166-172: The current scan only matches "*.app" at the top level
and misses nested app bundles; update the command built for Bun.spawn (where
proc is created using expanded from dirs) to recursively find .app bundles
(e.g., use find "${expanded}" -type d -name '*.app' -print0 and iterate over
results) so nested apps like /System/Applications/Utilities/Terminal.app are
included; adjust the parsing logic that reads each result (the code that
extracts path, name, and bundle id) to handle the find output (null-separated or
newline-separated) and continue populating allApps/InstalledApp entries
accordingly.
- Around line 171-179: The current loop sets displayName from the bundle
directory name (basename "$f"), which breaks matching against
InstalledApp.displayName; update the shell pipeline in the for loop that builds
each line so it extracts a localized display name from bundle metadata (e.g.,
use mdls -name kMDItemDisplayName or fall back to
kMDItemCFBundleName/kMDItemDisplayName extraction) and only use basename "$f"
when no metadata display name is present, then keep using the parsed displayName
variable when pushing into allApps (refer to the for (...) command that defines
path|displayName|bundleId and the allApps.push({ bundleId, displayName, path })
code).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fafb91a5-1337-4678-aebf-7425e0816b37

📥 Commits

Reviewing files that changed from the base of the PR and between 3ea64ee and d92553d.

⛔ Files ignored due to path filters (1)
  • contributors.svg is excluded by !**/*.svg
📒 Files selected for processing (3)
  • packages/@ant/computer-use-swift/src/backends/darwin.ts
  • src/utils/computerUse/escHotkey.ts
  • src/utils/computerUse/hostAdapter.ts

Comment on lines +166 to +172
const dirs = ['/Applications', '~/Applications', '/System/Applications']
const allApps: InstalledApp[] = []
for (const dir of dirs) {
const expanded = dir.startsWith('~') ? join(process.env.HOME ?? '~', dir.slice(1)) : dir
const proc = Bun.spawn(
['bash', '-c', `for f in "${expanded}"/*.app; do [ -d "$f" ] || continue; bid=$(mdls -name kMDItemCFBundleIdentifier "$f" 2>/dev/null | sed 's/.*= "//;s/"//'); name=$(basename "$f" .app); echo "$f|$name|$bid"; done`],
{ stdout: 'pipe', stderr: 'pipe' },
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Scan nested app folders too.

Line 171 only matches *.app directly under each root. That misses common locations like /System/Applications/Utilities/Terminal.app and vendor subfolders under /Applications, so those apps never reach listInstalled() and cannot be granted/launched later.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts around lines 166 -
172, The current scan only matches "*.app" at the top level and misses nested
app bundles; update the command built for Bun.spawn (where proc is created using
expanded from dirs) to recursively find .app bundles (e.g., use find
"${expanded}" -type d -name '*.app' -print0 and iterate over results) so nested
apps like /System/Applications/Utilities/Terminal.app are included; adjust the
parsing logic that reads each result (the code that extracts path, name, and
bundle id) to handle the find output (null-separated or newline-separated) and
continue populating allApps/InstalledApp entries accordingly.

Comment on lines +171 to +179
['bash', '-c', `for f in "${expanded}"/*.app; do [ -d "$f" ] || continue; bid=$(mdls -name kMDItemCFBundleIdentifier "$f" 2>/dev/null | sed 's/.*= "//;s/"//'); name=$(basename "$f" .app); echo "$f|$name|$bid"; done`],
{ stdout: 'pipe', stderr: 'pipe' },
)
const text = await new Response(proc.stdout).text()
await proc.exited
for (const line of text.split('\n').filter(Boolean)) {
const [path, displayName, bundleId] = line.split('|', 3)
if (path && displayName && bundleId && bundleId !== '(null)') {
allApps.push({ bundleId, displayName, path })
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Don't use the bundle directory name as displayName.

resolveRequestedApps() in packages/@ant/computer-use-mcp/src/toolCalls.ts, Lines 791-830 matches user requests against InstalledApp.displayName. Using basename "$f" here changes that field to the filesystem name, so localized/system apps can stop matching what the user actually sees. Populate displayName from bundle metadata and only fall back to the basename when metadata is missing.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/`@ant/computer-use-swift/src/backends/darwin.ts around lines 171 -
179, The current loop sets displayName from the bundle directory name (basename
"$f"), which breaks matching against InstalledApp.displayName; update the shell
pipeline in the for loop that builds each line so it extracts a localized
display name from bundle metadata (e.g., use mdls -name kMDItemDisplayName or
fall back to kMDItemCFBundleName/kMDItemDisplayName extraction) and only use
basename "$f" when no metadata display name is present, then keep using the
parsed displayName variable when pushing into allApps (refer to the for (...)
command that defines path|displayName|bundleId and the allApps.push({ bundleId,
displayName, path }) code).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant