pytest errors

[KeithFrohn]

Hi, I think the project is terrific. I would like to contribute.

I hit some errors when running pytest. Perhaps some folders are missing?

• Agent.ai.prompts.renderer
• Agent.platforms.locators.mobile
• Agent.platforms.collectors.xml_collector

I'm not sure why gradio_client is not found. I see it listed in pip list.

See output below.

(.venv) (base) keith@MacBookPro Agent % pytest tests/utest -q     

==================================================================================================================== ERRORS ====================================================================================================================
_______________________________________________________________________________________________ ERROR collecting tests/utest/ai/test_renderer.py _______________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/ai/test_renderer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/ai/test_renderer.py:2: in <module>
    from Agent.ai.prompts.renderer import UIRenderer
E   ModuleNotFoundError: No module named 'Agent.ai.prompts.renderer'
________________________________________________________________________________________ ERROR collecting tests/utest/platforms/test_mobile_locator.py _________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/platforms/test_mobile_locator.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/platforms/test_mobile_locator.py:2: in <module>
    from Agent.platforms.locators.mobile import MobileLocatorBuilder
E   ModuleNotFoundError: No module named 'Agent.platforms.locators.mobile'
_________________________________________________________________________________________ ERROR collecting tests/utest/platforms/test_xml_collector.py _________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/platforms/test_xml_collector.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/platforms/test_xml_collector.py:3: in <module>
    from Agent.platforms.collectors.xml_collector import XMLCollector
E   ModuleNotFoundError: No module named 'Agent.platforms.collectors.xml_collector'
_______________________________________________________________________________________________ ERROR collecting tests/utest/vlm/test_client.py ________________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/vlm/test_client.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/vlm/test_client.py:3: in <module>
    from Agent.ai.vlm._client import OmniParserClient, OmniParserError
Agent/ai/vlm/__init__.py:1: in <module>
    from ._client import OmniParserClient, OmniParserError
Agent/ai/vlm/_client.py:9: in <module>
    from gradio_client import Client, handle_file
E   ModuleNotFoundError: No module named 'gradio_client'
______________________________________________________________________________________________ ERROR collecting tests/utest/vlm/test_interface.py ______________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/vlm/test_interface.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/vlm/test_interface.py:3: in <module>
    from Agent.ai.vlm.interface import OmniParserOrchestrator
Agent/ai/vlm/__init__.py:1: in <module>
    from ._client import OmniParserClient, OmniParserError
Agent/ai/vlm/_client.py:9: in <module>
    from gradio_client import Client, handle_file
E   ModuleNotFoundError: No module named 'gradio_client'
_______________________________________________________________________________________________ ERROR collecting tests/utest/vlm/test_parser.py ________________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/vlm/test_parser.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/vlm/test_parser.py:2: in <module>
    from Agent.ai.vlm._parser import OmniParserResultProcessor
Agent/ai/vlm/__init__.py:1: in <module>
    from ._client import OmniParserClient, OmniParserError
Agent/ai/vlm/_client.py:9: in <module>
    from gradio_client import Client, handle_file
E   ModuleNotFoundError: No module named 'gradio_client'
______________________________________________________________________________________________ ERROR collecting tests/utest/vlm/test_selector.py _______________________________________________________________________________________________
ImportError while importing test module '/Users/keith/Programming/RobotFrameworkAgent/Agent/tests/utest/vlm/test_selector.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/utest/vlm/test_selector.py:3: in <module>
    from Agent.ai.vlm._selector import OmniParserElementSelector
Agent/ai/vlm/__init__.py:1: in <module>
    from ._client import OmniParserClient, OmniParserError
Agent/ai/vlm/_client.py:9: in <module>
    from gradio_client import Client, handle_file
E   ModuleNotFoundError: No module named 'gradio_client'
=========================================================================================================== short test summary info ============================================================================================================
ERROR tests/utest/ai/test_renderer.py
ERROR tests/utest/platforms/test_mobile_locator.py
ERROR tests/utest/platforms/test_xml_collector.py
ERROR tests/utest/vlm/test_client.py
ERROR tests/utest/vlm/test_interface.py
ERROR tests/utest/vlm/test_parser.py
ERROR tests/utest/vlm/test_selector.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 7 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2 skipped, 7 errors in 0.83s

[hassineabd]

Hi @KeithFrohn ,

thanks for trying the project and for taking the time to report this (and for offering to contribute).

Here is my quick reply :

• a chunk of these unit tests are currently obsolete

• they reference modules that have been moved/renamed during recent refactors (Agent.ai.prompts.renderer, Agent.platforms.locators.mobile, Agent.platforms.collectors.xml_collector, etc.).

• these unit tests are not wired into CI at the moment, so they haven’t been kept in sync. ( It should be deleted for now i think )

• At this stage the project is still in “heavy architecture churn” mode, so unit testing isn’t the priority yet (changes are major and frequent).

• the current focus is architecting the agent and validating design choices experimentally.

How you can contribute right now (most useful)

• Opening issues for features, architectural improvements, and gaps you notice
• Documentation (getting-started, architecture overview, how components fit together, dev workflow)
• Testing strategy proposals (what to test now vs later,, agent testing is different and based on signals and metrics.. i think that a set of end to end testing is way better than unit tests at this stage )

Research papers gathering :

• this project is a research-informed agent, so another high-impact contribution is helping document / evaluate solutions from papers and experiments
• best practices to reduce hallucinations (prompting + verification patterns)
• comparing UI parsers / UI grounding approaches (e.g., OmniParser vs others like cu1 and owl-vl )
• best patterns for verification prompts (visual assertions, evidence-first outputs, calibration)

Also: we have (separately) a benchmarking effort for AI-based visual assertions (calibration, limits, what can/can’t be reliably verified).

If you’re interested, I’d love to involve you there as well.

[KeithFrohn]

Yes, I'm interested. Where would you like me to start? How best to communicate?

[hassineabd]

Hello @KeithFrohn , I sent you an invitation to join the organization so you have access to all repositories (even private one)
happy to have you joining, i will setup slack communication as well
but i am more convinced by making discussions on github so we can get more people involved
feel free to make propositions at any subject and start any discussion about any topic