feat(ci): 增加覆盖率测试 #20

OhYee · 2025-12-18T03:45:33Z

Thank you for creating a pull request to contribute to Serverless Devs agentrun-sdk-python code! Before you open the request please answer the following questions to help it be more easily integrated. Please check the boxes "[ ]" with "[x]" when done too.
Please select one of the PR types below to complete

Fix bugs

Bug detail

The specific manifestation of the bug or the associated issue.

Pull request tasks

Add test cases for the changes
Passed the CI test

Update docs

Reason for update

Why do you need to update your documentation?

Pull request tasks

Update Chinese documentation
Update English documentation

Add contributor

Contributed content

Code
Document

Content detail

if content_type == 'code' || content_type == 'document':
    please tell us `PR url`，like: https://github.com/Serverless-Devs/agentrun-sdk-python/pull/1
else:
    please describe your contribution in detail

Others

Reason for update

Why do you need to update your documentation?

Copilot

Pull request overview

This PR adds comprehensive coverage testing infrastructure to the CI pipeline, enabling automated tracking of both full and incremental code coverage.

Introduces a custom Python script to run tests and calculate coverage metrics (full and incremental)
Integrates coverage checks into the CI workflow with automatic test package publishing
Adds test infrastructure improvements to support Google ADK mocking

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`scripts/check_coverage.py`	New script that runs pytest with coverage, calculates full and incremental coverage metrics, and validates against configurable thresholds
`coverage.yaml`	Configuration file defining coverage requirements (95% default) with per-directory overrides (currently set to 0% for gradual rollout)
`.github/workflows/ci.yml`	New CI workflow that runs tests, coverage checks, and automatically publishes test packages to PyPI when agentrun code changes
`.github/workflows/release-test.yml`	Updated to support being called from other workflows with configurable version bump type
`Makefile`	Added convenience targets for running tests, type checks, and coverage reports
`tests/unittests/integration/test_integration.py`	Enhanced mock patching to support Google ADK's module-level litellm imports
`pyproject.toml`	Fixed mypy python_version from incorrect "0.0.8" to "3.10" and added langchain_mcp_adapters dependency
`.gitignore`	Added coverage.json to ignored files

Comments suppressed due to low confidence (1)

scripts/check_coverage.py:26

Import of 'os' is not used.

import os

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/ci.yml

Copilot · 2025-12-18T09:41:24Z

.github/workflows/ci.yml

+        # 默认为 patch
+        BUMP_TYPE="patch"
+        case "$BUMP_TYPE" in
+          major)
+            MAJOR=$((MAJOR + 1))
+            MINOR=0
+            PATCH=0
+            ;;
+          minor)
+            MINOR=$((MINOR + 1))
+            PATCH=0
+            ;;
+          patch)
+            PATCH=$((PATCH + 1))
+            ;;
+        esac


Hardcoded constant: The BUMP_TYPE is always set to "patch" and the subsequent case statement is unnecessary. Consider removing the case statement entirely and just implementing the patch logic directly, or make BUMP_TYPE configurable if different bump types are needed in the future.

Suggested change

# 默认为 patch

BUMP_TYPE="patch"

case "$BUMP_TYPE" in

major)

MAJOR=$((MAJOR + 1))

MINOR=0

PATCH=0

;;

minor)

MINOR=$((MINOR + 1))

PATCH=0

;;

patch)

PATCH=$((PATCH + 1))

;;

esac

# 始终进行 patch 版本号递增

PATCH=$((PATCH + 1))

scripts/check_coverage.py

Copilot · 2025-12-18T09:41:25Z

coverage.yaml

+  # server 模块保持默认的 95% 覆盖率要求
+  agentrun/server:
+    full:
+      branch_coverage: 0
+      line_coverage: 0
+    incremental:
+      branch_coverage: 0
+      line_coverage: 0


Comment is misleading: The comment says "server 模块保持默认的 95% 覆盖率要求" (server module maintains default 95% coverage requirement), but the actual values below are set to 0%, not 95%. Either update the comment to match the actual configuration or update the values if the comment is correct.

- Add `workflow_call` trigger to allow this workflow to be invoked by other workflows - Default version bump type to `patch` when input is unspecified - Improve robustness of LLM mocking in integration tests by patching module-level imports - Add new CI workflow file, coverage configuration, and coverage checking script - Update dependencies and fix mypy configuration This commit enhances the release testing workflow to support automated triggering from other workflows, ensuring more flexible and reliable CI/CD processes. It also improves test stability by addressing edge cases in LLM mocking and introduces better tooling for code coverage analysis. Change-Id: Iba20089a66ad6e996dfb5930b6d039058d1b1646 Signed-off-by: OhYee <oyohyee@oyohyee.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: OhYee <oyohyee@oyohyee.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 10 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-18T11:30:13Z

scripts/check_coverage.py

+    print("🚀 运行覆盖率测试...")
+    test_path = args.test_path
+    if test_path == "tests/":  # 如果用户未指定 test-path 参数，则使用 unittests
+        test_path = "tests/unittests/"


The comment says "如果用户未指定 test-path 参数，则使用 unittests" but the condition checks for "tests/" which is actually the default value in the argument parser, not whether the user specified it. This logic overwrites the default behavior and may be confusing. Consider clarifying the comment or restructuring this logic.

Suggested change

test_path = "tests/unittests/"

if test_path == "tests/": # 当 test-path 参数为默认值 'tests/' 时，实际使用 'tests/unittests/' 作为测试路径

Copilot · 2025-12-18T11:30:14Z

.github/workflows/release-test.yml

        
-        # 根据用户选择递增版本
-        BUMP_TYPE="${{ inputs.version_bump }}"
+        # 根据用户选择递增版本（默认为 patch）


The comment states "根据用户选择递增版本（默认为 patch）" but the actual default is set by the workflow_call input definition on line 23, not by this line. The comment could be clearer about where the default comes from.

Suggested change

# 根据用户选择递增版本（默认为 patch）

# 根据用户选择递增版本（默认值由 workflow inputs 定义，此处未提供时回退为 patch）

Copilot · 2025-12-18T11:30:14Z

scripts/check_coverage.py

+    full: CoverageThreshold = None
+    incremental: CoverageThreshold = None
+    directory_overrides: dict[str, dict[str, CoverageThreshold]] = None
+    exclude_directories: list[str] = None


The type hint uses dict[str, dict[str, CoverageThreshold]] but this syntax requires Python 3.9+. Since the project targets Python 3.10, this is acceptable, but consider documenting the minimum Python version requirement in the script's docstring for clarity.

Copilot · 2025-12-18T11:30:14Z

scripts/check_coverage.py

+            parts = f.split("/")
+            if len(parts) >= 2:
+                discovered_dirs.add("/".join(parts[:2]))
+


The hard-coded path prefix "agentrun/" makes this script specific to this project structure. Consider making this configurable or deriving it from the source parameter to improve reusability.

Suggested change

parts = f.split("/")

if len(parts) >= 2:

discovered_dirs.add("/".join(parts[:2]))

parts = f.split("/")

if len(parts) >= 2:

discovered_dirs.add("/".join(parts[:2]))

Copilot · 2025-12-18T11:30:14Z

.github/workflows/ci.yml

+  # 仅在 push 时触发测试
+  push:


The comment states "仅在 push 时触发测试" (only triggered on push), but this will run on every push to any branch. Consider adding branch filters or documenting whether this is intentional, especially since the workflow includes automated package publishing which might not be desired for all branches.

Suggested change

# 仅在 push 时触发测试

push:

# 仅在 main 分支 push 时触发测试和发布

push:

branches:

- main

Copilot · 2025-12-18T11:30:15Z

.github/workflows/ci.yml

+      if: steps.changes.outputs.agentrun_changed == 'true'
+      run: |
+        python -m twine check dist/*
+        echo "Package verification completed"
+
+    - name: Publish to PyPI


The workflow automatically publishes packages to PyPI on every push where agentrun/ files change, without requiring manual approval or additional checks. This could lead to unintended package publications. Consider adding a condition to only publish on specific branches (e.g., main) or requiring manual approval for production releases.

Copilot · 2025-12-18T11:30:15Z

.github/workflows/ci.yml

+        fi
+        
+        # 解析版本号
+        IFS='.' read -r MAJOR MINOR PATCH <<< "$CURRENT_VERSION"
+        
+        # 默认为 patch
+        BUMP_TYPE="patch"
+        case "$BUMP_TYPE" in
+          major)
+            MAJOR=$((MAJOR + 1))
+            MINOR=0
+            PATCH=0
+            ;;
+          minor)
+            MINOR=$((MINOR + 1))
+            PATCH=0


The BUMP_TYPE variable is set to "patch" but never uses any other value since it's hard-coded. The subsequent case statement checking for major/minor/patch is unnecessary. Either remove the unused branches or make BUMP_TYPE configurable.

Suggested change

fi

# 解析版本号

IFS='.' read -r MAJOR MINOR PATCH <<< "$CURRENT_VERSION"

# 默认为 patch

BUMP_TYPE="patch"

case "$BUMP_TYPE" in

major)

MAJOR=$((MAJOR + 1))

MINOR=0

PATCH=0

;;

minor)

MINOR=$((MINOR + 1))

PATCH=0

# 始终执行 patch 版本号递增

PATCH=$((PATCH + 1))

Copilot · 2025-12-18T11:30:15Z

scripts/check_coverage.py

+            return cls()
+


The warning message uses emoji but doesn't specify what default configuration values are being used. Consider adding the default threshold values to help users understand what standards will be applied.

Suggested change

return cls()

default_config = cls()

print(

f"⚠️ 配置文件 {path} 不存在，使用默认配置："

f"全量覆盖率分支 {default_config.full.branch_coverage}%，"

f"行 {default_config.full.line_coverage}%；"

f"增量覆盖率分支 {default_config.incremental.branch_coverage}%，"

f"行 {default_config.incremental.line_coverage}%"

)

return default_config

Copilot · 2025-12-18T11:30:16Z

scripts/check_coverage.py

+            if summary.get("num_branches", 0) > 0:
+                branch_ratio = file_statements / max(
+                    summary.get("num_statements", 1), 1
+                )
+                total_branches += int(
+                    summary.get("num_branches", 0) * branch_ratio
+                )
+                covered_branches += int(
+                    summary.get("covered_branches", 0) * branch_ratio
+                )
+        else:


The branch coverage calculation for incremental coverage uses a simplified ratio-based approach which may not accurately reflect actual branch coverage for changed lines. The calculation multiplies the file's total branches by the ratio of changed statements to total statements, which assumes branches are evenly distributed. This can lead to misleading incremental branch coverage metrics when branches are concentrated in unchanged portions of the code.

Suggested change

if summary.get("num_branches", 0) > 0:

branch_ratio = file_statements / max(

summary.get("num_statements", 1), 1

)

total_branches += int(

summary.get("num_branches", 0) * branch_ratio

)

covered_branches += int(

summary.get("covered_branches", 0) * branch_ratio

)

else:

# 分支覆盖率：对包含变更行的文件按文件级汇总

total_branches += summary.get("num_branches", 0)

covered_branches += summary.get("covered_branches", 0)

Copilot · 2025-12-18T11:30:16Z

scripts/check_coverage.py

+
+import argparse
+import json
+import subprocess


Import of 'os' is not used.

Suggested change

import subprocess

OhYee force-pushed the test-coverage branch from 1b4bb4f to 88e38d4 Compare December 18, 2025 04:23

OhYee changed the title ~~feat(server): 实现 AG-UI 协议和生命周期钩子系统~~ feat(ci): 增加覆盖率测试 Dec 18, 2025

OhYee requested a review from Copilot December 18, 2025 09:37

Copilot started reviewing on behalf of OhYee December 18, 2025 09:37 View session

Copilot AI reviewed Dec 18, 2025

View reviewed changes

OhYee force-pushed the test-coverage branch from b09962f to 2a7b6d0 Compare December 18, 2025 11:20

OhYee requested a review from Copilot December 18, 2025 11:20

Copilot started reviewing on behalf of OhYee December 18, 2025 11:21 View session

OhYee and others added 2 commits December 18, 2025 19:27

Update .github/workflows/ci.yml

3945d0b

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: OhYee <oyohyee@oyohyee.com>

Update scripts/check_coverage.py

454f6e6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: OhYee <oyohyee@oyohyee.com>

Copilot AI reviewed Dec 18, 2025

View reviewed changes

OhYee merged commit 2582fa6 into main Dec 18, 2025
2 checks passed

OhYee deleted the test-coverage branch December 18, 2025 11:32

	test_path = "tests/unittests/"
	if test_path == "tests/": # 当 test-path 参数为默认值 'tests/' 时，实际使用 'tests/unittests/' 作为测试路径

	# 根据用户选择递增版本（默认为 patch）
	# 根据用户选择递增版本（默认值由 workflow inputs 定义，此处未提供时回退为 patch）

-  # 仅在 push 时触发测试
-  push:
+  # 仅在 main 分支 push 时触发测试和发布
+  push:
+    branches:
+      - main

-            return cls()
+            default_config = cls()
+            print(
+                f"⚠️  配置文件 {path} 不存在，使用默认配置："
+                f"全量覆盖率分支 {default_config.full.branch_coverage}%，"
+                f"行 {default_config.full.line_coverage}%；"
+                f"增量覆盖率分支 {default_config.incremental.branch_coverage}%，"
+                f"行 {default_config.incremental.line_coverage}%"
+            )
+            return default_config

feat(ci): 增加覆盖率测试 #20

feat(ci): 增加覆盖率测试 #20

Uh oh!

Conversation

OhYee commented Dec 18, 2025

Fix bugs

Update docs

Add contributor

Others

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants