Skip to content

chore: Improve token efficience#49

Draft
vinilana wants to merge 4 commits intomainfrom
chore/token-efficience
Draft

chore: Improve token efficience#49
vinilana wants to merge 4 commits intomainfrom
chore/token-efficience

Conversation

@vinilana
Copy link
Copy Markdown
Owner

This pull request introduces and documents a new "compact-first" MCP (Machine Context Protocol) contract for dotcontext, significantly improving token efficiency and workflow flexibility. It updates the documentation to explain the new compact profiles, provides benchmark results demonstrating substantial payload size reductions, and adds example payloads for both compact and legacy modes. Configuration examples are also updated to highlight the use of the new compact execution profile.

Major documentation and feature updates:

Introduction of compact MCP profiles and workflow options:

  • Added detailed documentation in README.md about the new compact MCP contract, including available profiles (standalone, planning, execution), their intended use cases, and how to select them via the DOTCONTEXT_MCP_PROFILE environment variable. Aliases for codex and claude-code now map to execution.
  • Updated configuration examples to default to the execution profile for optimal efficiency.

Benchmarking and efficiency evidence:

  • Added a comprehensive benchmark report in docs/benchmarks/mcp-token-efficiency/README.md comparing compact, legacy, and non-MCP payloads, showing 89–91% token reduction versus legacy and 17–40% versus non-MCP. The report includes scenario summaries, tool surface breakdowns, and raw payload statistics.
  • Included raw compact and legacy payload examples for key MCP actions (e.g., getCodebaseMap, scaffoldPlan, workflow-init, workflow-status, workflow-advance) to docs/benchmarks/mcp-token-efficiency/, providing concrete evidence of the efficiency improvements. [1] [2] [3] [4] [5] [6] [7]

These changes make it easier for users and developers to adopt the new compact protocol, understand its benefits, and configure their tools for maximum efficiency.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a compact-first MCP contract (with selectable server profiles) to reduce token overhead during PREVC workflow loops, while keeping verbose/legacy expansions available via explicit flags. It also adds benchmarking + documentation to quantify and demonstrate payload reductions.

Changes:

  • Add profile-aware MCP server tool registration (standalone/planning/execution) and workflow help resources (workflow://guide/{topic}).
  • Implement compact workflow/tool payloads (bundle caching, bundleId, revision, notModified polling) with opt-in legacy/verbose expansions.
  • Update planning/context tooling to default to smaller payloads and add a benchmark harness + checked-in fixtures/docs.

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/workflow/types.ts Adds compact phase execution bundle type used by MCP execution-mode payloads.
src/workflow/status/statusManager.ts Stabilizes migration timestamps used in legacy workflow migrations.
src/workflow/orchestrator.ts Adds getPhaseExecutionBundle and reuses it in orchestration responses.
src/services/workflow/workflowService.ts Exposes getPhaseExecutionBundle via the service layer.
src/services/mcp/README.md Documents profiles, compact vs verbose flags, workflow surface, and benchmarks.
src/services/mcp/mcpServer.ts Adds profile-aware tool surface + workflow help resources.
src/services/mcp/gatewayTools.ts Re-exports new compact runtime helpers/types from gateway layer.
src/services/mcp/gateway/workflowStatus.ts Implements compact workflow-status payload + revision/notModified polling.
src/services/mcp/gateway/workflowManage.ts Returns compact deltas for manage actions; legacy expansions become opt-in.
src/services/mcp/gateway/workflowInit.ts Returns compact init payloads with bundle+revision and opt-in legacy path.
src/services/mcp/gateway/workflowHandlers.test.ts Adds tests for compact workflow handler defaults and JSON serialization.
src/services/mcp/gateway/workflowCompact.test.ts Adds focused tests for compact workflow payloads and polling semantics.
src/services/mcp/gateway/workflowAdvance.ts Returns compact advance payloads with bundle+revision and opt-in legacy path.
src/services/mcp/gateway/types.ts Adds new MCP gateway request flags (verbose/guidance/content/profile).
src/services/mcp/gateway/runtime.ts Introduces compact runtime helpers + caching (manifests, bundles, revisions).
src/services/mcp/gateway/response.ts Makes JSON responses compact by default; adds optional pretty printing.
src/services/mcp/gateway/index.ts Re-exports runtime helpers/types for gateway consumers.
src/services/mcp/gateway/context.ts Threads compact/verbose preferences into scaffolding/planning tooling.
src/services/mcp/gateway/agent.ts Adds compact agent responses and manifest caching support.
src/services/ai/tools/scaffoldPlanTool.ts Makes scaffoldPlan compact by default with opt-in inline content/instructions.
src/services/ai/tools/planningToolsCompact.test.ts Adds tests for compact defaults in planning tools.
src/services/ai/tools/getCodebaseMapTool.ts Defaults codebase-map section to architecture (vs all).
src/services/ai/tools/fillScaffoldingTool.ts Adds optional inline context + context resource refs and multi-context caching.
src/services/ai/schemas.ts Updates schemas for new compact defaults + optional expansion fields.
scripts/benchmark-mcp-efficiency.js Adds benchmark harness that emits token/size comparisons + fixtures.
README.md Documents compact profiles, defaults, and benchmark snapshot.
docs/mcp-token-efficiency.md Adds narrative benchmark summary and interface notes.
docs/benchmarks/mcp-token-efficiency/README.md Adds generated benchmark report output.
docs/benchmarks/mcp-token-efficiency/summary.json Adds machine-readable benchmark summary output.
docs/benchmarks/mcp-token-efficiency/notModified-workflow-status.json Adds polling fixture for notModified response.
docs/benchmarks/mcp-token-efficiency/legacy-workflow-status.json Adds legacy workflow-status payload fixture.
docs/benchmarks/mcp-token-efficiency/legacy-workflow-init.json Adds legacy workflow-init payload fixture.
docs/benchmarks/mcp-token-efficiency/legacy-workflow-advance.json Adds legacy workflow-advance payload fixture.
docs/benchmarks/mcp-token-efficiency/legacy-scaffoldPlan.json Adds legacy scaffoldPlan payload fixture.
docs/benchmarks/mcp-token-efficiency/legacy-getCodebaseMap.json Adds legacy getCodebaseMap payload fixture.
docs/benchmarks/mcp-token-efficiency/compact-workflow-status.json Adds compact workflow-status payload fixture.
docs/benchmarks/mcp-token-efficiency/compact-workflow-init.json Adds compact workflow-init payload fixture.
docs/benchmarks/mcp-token-efficiency/compact-workflow-advance.json Adds compact workflow-advance payload fixture.
docs/benchmarks/mcp-token-efficiency/compact-scaffoldPlan.json Adds compact scaffoldPlan payload fixture.
docs/benchmarks/mcp-token-efficiency/compact-getCodebaseMap.json Adds compact getCodebaseMap payload fixture.
Comments suppressed due to low confidence (1)

src/services/mcp/gateway/workflowManage.ts:173

  • createDoc no longer returns a legacy message, even when includeLegacy is requested. If includeLegacy is intended to preserve backward-compatible payloads, consider conditionally including the previous message field in legacy mode (similar to other actions in this handler).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 226 to 231
const FillSingleFileInputSchema = z.object({
repoPath: z.string().describe('Repository path'),
filePath: z.string().describe('Absolute path to the scaffold file to fill')
filePath: z.string().describe('Absolute path to the scaffold file to fill'),
includeContext: z.boolean().default(true).optional()
.describe('Include semantic context inline for compatibility clients')
});
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FillSingleFileInputSchema defaults includeContext to true, which forces semantic context to be embedded inline even when callers omit includeContent (the compact default). This undermines the compact-first MCP contract; consider defaulting includeContext to false for fillSingle (matching fillScaffolding), and only including semanticContext when explicitly requested.

Copilot uses AI. Check for mistakes.
Comment on lines +286 to +296
private async buildPathKey(repoPath: string, relativePaths: string[]): Promise<string> {
const mtimes = await Promise.all(relativePaths.map(async (relativePath) => {
const targetPath = path.join(repoPath, relativePath);
try {
const stat = await fs.stat(targetPath);
return `${relativePath}:${Math.floor(stat.mtimeMs)}`;
} catch {
return `${relativePath}:0`;
}
}));

Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExecutionStateCache.buildPathKey only uses the directory mtime for .context/agents / .context/skills. On most filesystems, editing an existing file won’t update the directory mtime, so manifests/bundles can become stale without any cache invalidation. Consider incorporating file mtimes (or a content hash) of files discovered by the registries, or at least the latest mtime of files in these directories.

Suggested change
private async buildPathKey(repoPath: string, relativePaths: string[]): Promise<string> {
const mtimes = await Promise.all(relativePaths.map(async (relativePath) => {
const targetPath = path.join(repoPath, relativePath);
try {
const stat = await fs.stat(targetPath);
return `${relativePath}:${Math.floor(stat.mtimeMs)}`;
} catch {
return `${relativePath}:0`;
}
}));
private async getLatestMtimeMs(targetPath: string): Promise<number> {
try {
const stat = await fs.stat(targetPath);
let latest = Math.floor(stat.mtimeMs);
if (!stat.isDirectory()) {
return latest;
}
const entries = await fs.readdir(targetPath);
for (const entry of entries) {
const childPath = path.join(targetPath, entry);
try {
const childMtime = await this.getLatestMtimeMs(childPath);
if (childMtime > latest) {
latest = childMtime;
}
} catch {
// Ignore errors for individual children and continue.
}
}
return latest;
} catch {
return 0;
}
}
private async buildPathKey(repoPath: string, relativePaths: string[]): Promise<string> {
const mtimes = await Promise.all(
relativePaths.map(async (relativePath) => {
const targetPath = path.join(repoPath, relativePath);
const mtime = await this.getLatestMtimeMs(targetPath);
return `${relativePath}:${mtime}`;
})
);

Copilot uses AI. Check for mistakes.
Comment on lines +101 to +126
const compactState = {
success: true,
name: summary.name,
scale: getScaleName(summary.scale as ProjectScale),
currentPhase: {
code: summary.currentPhase,
name: PHASE_NAMES_EN[summary.currentPhase],
},
progress: summary.progress,
isComplete: summary.isComplete,
phases: compactPhaseStates(status.phases),
activeAgents: compactActiveAgents(status.agents),
gates: {
canAdvance: gates.canAdvance,
...(gates.blockingReason ? { blockedBy: gates.blockingReason } : {}),
},
approval: approval ? {
planCreated: approval.plan_created,
planApproved: approval.plan_approved,
...(approval.approved_by ? { approvedBy: String(approval.approved_by) } : {}),
...(approval.approved_at ? { approvedAt: approval.approved_at } : {}),
} : null,
bundleId: bundle.bundleId,
};
response.revision = executionStateCache.getRevision(repoPath, compactState);

Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revision is computed from compactState, but this state omits profile (and may differ in shape from the workflow-status state used for polling). A client that stores the revision returned by workflow-advance and then polls workflow-status can get persistent mismatches (breaking notModified). Consider building revisions from a single shared CompactWorkflowState shape (including profile), reused across init/status/advance/manage.

Copilot uses AI. Check for mistakes.
Comment on lines +102 to +116
activeAgent: params.to,
revision: executionStateCache.getRevision(repoPath, {
success: true,
name: summary.name,
scale: getScaleName(summary.scale as ProjectScale),
currentPhase: {
code: summary.currentPhase,
name: PHASE_NAMES_EN[summary.currentPhase],
},
progress: summary.progress,
isComplete: summary.isComplete,
phases: compactPhaseStates(status.phases),
activeAgents: compactActiveAgents(status.agents),
}),
};
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revision is derived from a compact workflow state that omits profile and other fields included by workflow-status (e.g., gates/approval/bundleId depending on handler). This can make revisions inconsistent across endpoints, breaking revision-based polling when clients use a revision returned by workflow-manage. Consider reusing the same CompactWorkflowState shape (including profile) everywhere you compute revisions.

Copilot uses AI. Check for mistakes.
Comment on lines 131 to +138
private registerGatewayTools(): void {
const wrap = <TParams>(
toolName: string,
handler: (params: TParams) => Promise<MCPToolResponse>
) => this.wrapWithActionLogging(toolName, handler);
let registeredCount = 0;

if (this.hasTool('explore')) {
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The server profile selected via DOTCONTEXT_MCP_PROFILE/options.profile is not propagated into tool handler params. As a result, resolveResponsePreferences() will default responses to profile: "standalone" unless each client call passes profile, and revisions may be computed against the wrong profile. Consider injecting profile: params.profile ?? this.profile in the wrapper before calling handlers.

Copilot uses AI. Check for mistakes.
verbose?: boolean;
includeGuidance?: boolean;
includeContent?: boolean;
includeContext?: boolean;
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ContextParams introduces includeContext, but the MCP tool schema only exposes includeContent, and handleContext only uses includeContent to control inline context. This unused/duplicated flag is likely to confuse API consumers; consider removing includeContext or wiring it consistently (e.g., aliasing it to includeContent).

Suggested change
includeContext?: boolean;

Copilot uses AI. Check for mistakes.
Comment on lines +184 to +189
class ExecutionStateCache {
private agentManifestCache = new Map<string, CachedManifest<AgentManifestEntry>>();
private skillManifestCache = new Map<string, CachedManifest<SkillManifestEntry>>();
private bundleCache = new Map<string, CachedBundle>();
private revisionCache = new Map<string, string>();

Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revisionCache is keyed by repoPath + digest(state), so every distinct workflow state creates a new entry and the map can grow without bound in a long-lived MCP server (especially with frequent polling). Consider storing only the latest revision per repo (or using an LRU/TTL) to avoid unbounded memory growth.

Copilot uses AI. Check for mistakes.
Comment on lines +79 to +120
async function createFixtureRepo(prefix) {
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
await fs.ensureDir(path.join(tempDir, '.context', 'docs'));
await fs.ensureDir(path.join(tempDir, 'src'));
await fs.writeJson(
path.join(tempDir, 'package.json'),
{
name: 'mcp-benchmark-fixture',
version: '1.0.0',
},
{ spaces: 2 }
);
await fs.writeFile(path.join(tempDir, 'src', 'index.ts'), 'export const fixture = true;\n');
await fs.writeJson(
path.join(tempDir, '.context', 'docs', 'codebase-map.json'),
{
stack: { runtime: 'node', language: 'typescript' },
structure: { root: ['src', '.context'] },
architecture: {
layers: [
{ name: 'CLI', directories: ['src'], dependsOn: ['services'] },
{ name: 'MCP', directories: ['src/services/mcp'], dependsOn: ['workflow'] },
],
},
symbols: {
classes: [],
interfaces: [],
functions: [],
types: [],
enums: [],
},
publicAPI: [],
dependencies: [],
stats: { files: 2 },
keyFiles: [],
navigation: {},
},
{ spaces: 2 }
);

return tempDir;
}
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The benchmark script writes raw handler/tool payloads directly to docs/benchmarks/..., but the payloads include machine-specific absolute temp paths (e.g., /tmp/.../.context/...). This makes benchmark artifacts non-reproducible and can cause noisy diffs. Consider normalizing/redacting repo paths (e.g., replace the temp root with a placeholder) before persisting fixtures.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants