[AI Agents Extension] Adding some integration tests for the extension#6337
[AI Agents Extension] Adding some integration tests for the extension#6337
Conversation
Signed-off-by: trangevi <trangevi@microsoft.com>
Signed-off-by: trangevi <trangevi@microsoft.com>
Signed-off-by: trangevi <trangevi@microsoft.com>
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive integration tests for the Azure AI Agents extension, including test infrastructure, utilities, and two test suites covering agent initialization and deployment workflows. The changes also include a bug fix in the init command that properly extracts agent names from template structures.
Key Changes:
- New test infrastructure with suite setup/teardown that provisions Azure resources for integration testing
- Integration tests for agent initialization (
initcommand) with various manifest sources and target directories - Integration tests for agent deployment (
deploycommand) with API validation - Bug fix in init.go to correctly extract agent names from ContainerAgent or PromptAgent templates
- Updated dependencies to support integration testing (testify, vcr, Azure SDK packages)
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_utils.go | Utility functions for executing azd commands, parsing outputs, and logging test execution with verbose mode support |
| cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_suite.go | Test suite setup/teardown infrastructure that provisions Azure resources using azd templates and manages test environment lifecycle |
| cli/azd/extensions/azure.ai.agents/test/integrationTests/initTests/init_test.go | Integration tests verifying agent initialization with different manifest sources (GitHub URLs) and target directory configurations |
| cli/azd/extensions/azure.ai.agents/test/integrationTests/deployTests/deploy_test.go | Integration tests for agent deployment with API validation including conversation creation and agent response verification |
| cli/azd/extensions/azure.ai.agents/internal/cmd/init.go | Bug fix to extract agent name from template structures (ContainerAgent/PromptAgent) instead of using deprecated manifest.Name field, plus improved error handling |
| cli/azd/extensions/azure.ai.agents/go.mod | Added testify for test assertions and updated Azure SDK dependencies for integration test support |
| cli/azd/extensions/azure.ai.agents/go.sum | Corresponding checksum updates for new and updated dependencies |
| args := []string{ | ||
| "init", | ||
| "-e", fmt.Sprintf("azd-extension-integration-tests-%s", suffix), | ||
| "--location", "northcentralus", |
There was a problem hiding this comment.
Hardcoded location 'northcentralus' should be replaced with a configurable value. The functional tests in cli/azd/test/functional/cli_test.go use environment variables (AZD_TEST_AZURE_LOCATION) or fallback to user config (defaults.location). Consider following this pattern for flexibility across different test environments.
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_utils.go
Outdated
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_suite.go
Outdated
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/deployTests/deploy_test.go
Outdated
Show resolved
Hide resolved
| var ( | ||
| // Verbose logging flag | ||
| verboseLogging bool | ||
| // Current test context for logging | ||
| currentTestName string | ||
| ) |
There was a problem hiding this comment.
Using package-level global variables (verboseLogging and currentTestName) for state management is not thread-safe and may cause issues if tests run in parallel. Consider using context or test-specific state instead. The functional tests in cli/azd/test/functional/cli_test.go use a properly initialized config struct pattern that is thread-safe.
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_utils.go
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_suite.go
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/deployTests/deploy_test.go
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_utils.go
Outdated
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.agents/test/integrationTests/testUtilities/test_utils.go
Outdated
Show resolved
Hide resolved
|
Still in progress |
This reverts commit f91db64.
Signed-off-by: trangevi <trangevi@microsoft.com>
Signed-off-by: trangevi <trangevi@microsoft.com>
Signed-off-by: trangevi <trangevi@microsoft.com>
|
There could be a case here for adding support for live testing in extension pipelines: #6561 |
Signed-off-by: trangevi <trangevi@microsoft.com>
Signed-off-by: trangevi <trangevi@microsoft.com>
|
Hi @@trangevi. Thank you for your interest in helping to improve the Azure Developer CLI experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment. Otherwise, we'll close this out in 7 days. |

No description provided.