Skip to content

Support Wait command & delayed screenshots; sanitize screen-element history; Groq image payload serialization and tests#100

Closed
Android-PowerUser wants to merge 1 commit into
test-termux-command-function-in-screen-operatorfrom
limit-screen-elements-to-3-most-recent-f82fst
Closed

Support Wait command & delayed screenshots; sanitize screen-element history; Groq image payload serialization and tests#100
Android-PowerUser wants to merge 1 commit into
test-termux-command-function-in-screen-operatorfrom
limit-screen-elements-to-3-most-recent-f82fst

Conversation

@Android-PowerUser
Copy link
Copy Markdown
Owner

Motivation

  • Add explicit support for delaying screenshot execution from parsed AI commands and ensure delayed screenshots can be cancelled.
  • Prevent long-running chat history growth from repeated "Screen elements:" sections by marking older sections as no longer relevant.
  • Fix Groq API request payload to serialize image URLs as an object and make polymorphic serialization explicit for Groq content.

Description

  • Added a Wait command (Command.Wait) and parsing support in CommandParser with a Wait(n) pattern, and updated the UI to display wait commands via PhotoReasoningScreen.
  • Implemented delayed screenshot handling in ScreenOperatorAccessibilityService with pendingScreenshotDelayMillis, pendingDelayedScreenshotRunnable, executeTakeScreenshotCommand() and cancelPendingDelayedScreenshot(), and ensured clearCommandQueue() cancels pending delayed screenshots.
  • Introduced PhotoReasoningScreenElementHistoryPolicy to sanitize chat messages by keeping only the latest relevant Screen elements: sections and replacing older ones with a "no longer relevant" marker, then wired it into multiple places in PhotoReasoningViewModel, PhotoReasoningHistoryBuilder, PhotoReasoningMessageMutations, and chat persistence logic.
  • Reworked Groq client serialization in ScreenCaptureApiClients.kt by adding ServiceGroqRequest, ServiceGroqMessage, and polymorphic sealed ServiceGroqContent types (ServiceGroqTextContent, ServiceGroqImageContent) where image_url is serialized as an object containing url.
  • Small Gradle change enabling unit test default value via testOptions.unitTests.isReturnDefaultValues = true in app/build.gradle.kts.
  • Added unit tests: CommandParserTest now includes a Wait parsing test, PhotoReasoningScreenElementHistoryPolicyTest to validate sanitization behavior, and ScreenCaptureApiClientsTest to verify Groq request JSON shape.

Testing

  • Ran unit tests under app/src/test: CommandParserTest, PhotoReasoningScreenElementHistoryPolicyTest, and ScreenCaptureApiClientsTest which exercised Wait parsing, history sanitization, and Groq serialization respectively, and all tests passed.
  • Verified that delayed screenshot flow posts and cancels delayed runnables via the new methods in ScreenOperatorAccessibilityService during unit/integration checks and did not regress existing command execution paths.

Codex Task

Copy link
Copy Markdown
Contributor

@amazon-q-developer amazon-q-developer Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Review Summary

This PR implements Wait command support, delayed screenshots, screen element history sanitization, and Groq serialization fixes. The changes are well-structured overall with good test coverage.

Critical Issues Found

  • Integer overflow risk in Wait command delay calculation that could cause negative delays

Changes Reviewed

  • ✅ Wait command implementation and parsing
  • ✅ Delayed screenshot execution with cancellation support
  • ✅ Groq API client serialization improvements with proper polymorphic types
  • ✅ Unit test coverage for new functionality
  • ✅ Gradle configuration for unit tests

The implementation correctly handles delayed screenshots, command queue integration, and cleanup. Please address the overflow issue before merging.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Comment on lines +243 to +245
pendingScreenshotDelayMillis = command.seconds
.coerceAtLeast(0L)
.coerceAtMost(Long.MAX_VALUE / 1000L) * 1000L
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛑 Logic Error: Integer overflow risk when multiplying seconds by 1000. If command.seconds equals Long.MAX_VALUE / 1000L (9223372036854775), multiplying by 1000 causes overflow to negative value. The coerceAtMost check happens before multiplication, allowing the overflow.

Suggested change
pendingScreenshotDelayMillis = command.seconds
.coerceAtLeast(0L)
.coerceAtMost(Long.MAX_VALUE / 1000L) * 1000L
pendingScreenshotDelayMillis = command.seconds
.coerceAtLeast(0L)
.coerceAtMost(Long.MAX_VALUE / 1000L - 1L) * 1000L

@Android-PowerUser Android-PowerUser deleted the branch test-termux-command-function-in-screen-operator May 11, 2026 18:20
@Android-PowerUser Android-PowerUser deleted the limit-screen-elements-to-3-most-recent-f82fst branch May 11, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant