Skip to content

AGENTS.md

Latest commit

 

History

History
630 lines (479 loc) · 20.1 KB

AGENTS.md

File metadata and controls

630 lines (479 loc) · 20.1 KB

AGENTS.md — deepctl Developer Guide

Comprehensive guide for AI coding assistants and human contributors working on the deepctl CLI.

Project Overview

deepctl is the official Deepgram CLI for speech recognition and audio intelligence. It's a plugin-based Python CLI built on Click, Rich, and Pydantic, distributed as a UV workspace monorepo.

  • Entry points: deepctl, deepgram, dg (all equivalent)
  • Python: >=3.10
  • Package manager: uv
  • Linter/formatter: ruff (not black/flake8)
  • Type checker: mypy (strict mode)
  • Tests: pytest

Architecture

deepctl/
├── src/deepctl/                    # Root package (entry point, CLI group)
│   └── main.py                     # main() → Click group → PluginManager
├── packages/
│   ├── deepctl-core/               # BaseCommand, Config, Auth, Output, Client
│   ├── deepctl-shared-utils/       # Validation helpers (audio, URLs, dates)
│   ├── deepctl-cmd-*/              # Built-in command packages
│   ├── deepctl-plugin-example/     # Reference external plugin
│   └── ...
├── scripts/
│   └── generate_readmes.py         # README auto-generation
└── .github/workflows/              # CI/CD

How Commands Are Discovered

The CLI uses Python entry points for zero-config plugin discovery:

  1. main() creates a Click group and calls PluginManager.load_plugins(cli)
  2. PluginManager scans three entry point groups in order:
    • deepctl.commands — built-in top-level commands
    • deepctl.plugins — external plugin commands
    • Plugin venv entries from ~/.deepctl/plugins/venv/
  3. For each BaseGroupCommand, it also scans:
    • deepctl.subcommands.<group-name> — built-in subcommands
    • deepctl.subplugins.<group-name> — plugin subcommands
  4. Each entry point class is instantiated and wrapped in a Click command

Data Flow

User runs: dg transcribe audio.wav --output json
  │
  ▼
main() → preprocess_hyphenated_commands() → cli()
  │
  ▼
Click routes to TranscribeCommand (discovered via entry points)
  │
  ▼
BaseCommand.execute(ctx, **kwargs)
  ├── Config from ctx.obj
  ├── AuthManager(config)
  ├── DeepgramClient(config, auth_manager)
  ├── auth_manager.guard()  [if requires_auth]
  ├── handle(config, auth_manager, client, **kwargs)  ← you implement this
  └── output_result(result, config)  ← auto JSON/YAML/table/CSV

Creating a New Command Package

1. Directory Structure

packages/deepctl-cmd-<name>/
├── pyproject.toml
├── README.md                   # Auto-generated by make readmes
├── src/deepctl_cmd_<name>/
│   ├── __init__.py
│   ├── command.py              # YourCommand(BaseCommand)
│   └── models.py               # Pydantic result models
└── tests/
    ├── __init__.py
    └── unit/
        ├── __init__.py
        └── test_<name>.py

2. pyproject.toml

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "deepctl-cmd-<name>"
version = "0.1.10" # x-release-please-version
description = "<Description> command for deepctl"
readme = "README.md"
license = "MIT"
authors = [{ name = "Deepgram", email = "devrel@deepgram.com" }]
maintainers = [{ name = "Deepgram", email = "devrel@deepgram.com" }]
classifiers = [
    "Development Status :: 3 - Alpha",
    "Intended Audience :: Developers",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]
keywords = ["deepgram", "cli", "<name>"]
requires-python = ">=3.10"
dependencies = [
    "deepctl-core>=0.1.10",
    "click>=8.0.0",
    "rich>=13.0.0",
    "pydantic>=2.0.0",
]

[project.entry-points."deepctl.commands"]
<name> = "deepctl_cmd_<name>.command:<Name>Command"

[tool.setuptools]
package-dir = { "" = "src" }

[tool.setuptools.packages.find]
where = ["src"]
include = ["deepctl_cmd_<name>*"]

[tool.uv.sources]
deepctl-core = { workspace = true }

Critical: The # x-release-please-version comment is required on the version = line only (not on dependency lines) — release-please uses these markers.

3. models.py

"""Models for <name> command."""

from deepctl_core import BaseResult
from pydantic import BaseModel, Field


class SomeInfo(BaseModel):
    """Data model for your command."""
    id: str
    name: str


class YourResult(BaseResult):
    """Result from <name> command."""
    # BaseResult provides: status="success", message=None
    items: list[SomeInfo] = Field(default_factory=list)

4. command.py

"""<Name> command for deepctl."""

from typing import Any

from deepctl_core import AuthManager, BaseCommand, Config, DeepgramClient
from rich.console import Console

from .models import YourResult

console = Console()


class YourCommand(BaseCommand):
    name = "<name>"
    help = "Full help text shown in dg <name> --help"
    short_help = "One-liner for dg --help listing"

    requires_auth = False       # Set True if command needs API key
    requires_project = False    # Set True if command needs project ID
    ci_friendly = True          # Affects confirm()/prompt() in CI

    examples = [
        "dg <name>",
        "dg <name> --option value",
    ]
    agent_help = (
        "Extended description for AI agents and MCP tool use. "
        "Explain what the command does and when to use it."
    )

    def get_arguments(self) -> list[dict[str, Any]]:
        return [
            # Positional argument (singular "name" key):
            {
                "name": "input",
                "help": "The input value",
                "type": str,
                "required": True,
            },
            # Option (plural "names" key + is_option):
            {
                "names": ["--format", "-f"],
                "help": "Output format",
                "type": str,
                "default": "json",
                "is_option": True,
            },
            # Flag:
            {
                "names": ["--verbose", "-v"],
                "help": "Enable verbose output",
                "is_flag": True,
                "is_option": True,
            },
        ]

    def handle(
        self,
        config: Config,
        auth_manager: AuthManager,
        client: DeepgramClient,
        **kwargs: Any,
    ) -> Any:
        input_val = kwargs.get("input")
        fmt = kwargs.get("format", "json")

        # Do work...
        console.print(f"[blue]Processing:[/blue] {input_val}")

        return YourResult(
            status="success",
            message="Done",
        )

5. __init__.py

"""<Name> command package for deepctl."""

from .command import YourCommand
from .models import YourResult

__all__ = ["YourCommand", "YourResult"]

6. Wire Into the Workspace

Update these files:

/pyproject.toml (root):

# In [project] dependencies:
"deepctl-cmd-<name>>=0.0.1",

# In [tool.uv.sources]:
deepctl-cmd-<name> = { workspace = true }

/.github/release-please-config.json:

// In "packages":
"packages/deepctl-cmd-<name>": {
  "component": "deepctl-cmd-<name>",
  "include-component-in-tag": true
}

/.github/.release-please-manifest.json:

"packages/deepctl-cmd-<name>": "0.0.1"

/.github/workflows/test.yml: Add packages/deepctl-cmd-<name>/tests to the pytest path list.

7. Sync and Verify

uv sync                    # Install new package
make check                 # ruff format, ruff check, mypy
uv run pytest packages/deepctl-cmd-<name>/tests/ -v
make readmes               # Regenerate READMEs

Creating a Group Command (with Subcommands)

Use BaseGroupCommand when your command has subcommands (like dg debug audio, dg debug network).

Parent Group Package

from deepctl_core import BaseGroupCommand

class MyGroupCommand(BaseGroupCommand):
    name = "mygroup"
    help = "Group of related subcommands"
    invoke_without_command = False  # Shows help when called bare

    def handle_group(self, config, auth_manager, client, **kwargs):
        """Optional: runs when group itself is invoked."""
        pass

Register as deepctl.commands:

[project.entry-points."deepctl.commands"]
mygroup = "deepctl_cmd_mygroup.command:MyGroupCommand"

Subcommand Package

Each subcommand is a separate package extending BaseCommand, registered under deepctl.subcommands.<group-name>:

[project.entry-points."deepctl.subcommands.mygroup"]
sub1 = "deepctl_cmd_mygroup_sub1.command:Sub1Command"

The CLI supports hyphenated invocation: dg mygroup-sub1 is equivalent to dg mygroup sub1.

Programmatic Subcommands

Alternatively, override setup_commands() to define subcommands directly in the group (see SkillsCommand for reference):

def setup_commands(self) -> list[click.Command]:
    return [self._create_status_command(), self._create_install_command()]

Creating an External Plugin

External plugins use deepctl.plugins instead of deepctl.commands:

[project.entry-points."deepctl.plugins"]
myplugin = "deepctl_plugin_myplugin.command:MyPluginCommand"

Users install plugins via dg plugin install <package-name>. Plugins are isolated in ~/.deepctl/plugins/venv/ for non-development installs.

To add subcommands under an existing group:

[project.entry-points."deepctl.subplugins.debug"]
myplugin = "deepctl_plugin_myplugin.command:MySubcommand"

See packages/deepctl-plugin-example/ for a complete reference.

Core APIs

deepctl_core Public Exports

from deepctl_core import (
    # Base classes
    BaseCommand, BaseGroupCommand, BaseResult, ErrorResult,
    # Infrastructure
    Config, AuthManager, DeepgramClient,
    # Output
    OutputFormatter, get_console, setup_output,
    print_error, print_info, print_output, print_success, print_warning,
    # Auth
    AuthenticationError,
    # Plugins
    PluginInfo, PluginManager,
    # Profiles
    ProfileInfo, ProfilesResult,
    # Timing
    TimingContext, enable_timing, get_timing_summary,
    is_timing_enabled, print_timing_summary,
)

Config System

  • User config: ~/.config/deepctl/config.yaml (Linux) / ~/Library/Application Support/deepctl/config.yaml (macOS)
  • Project config: ./deepgram.yaml in current directory
  • Environment variables: DEEPGRAM_API_KEY, DEEPGRAM_PROJECT_ID, DEEPGRAM_BASE_URL, DEEPGRAM_OUTPUT_FORMAT, DEEPGRAM_PROFILE
  • Profile precedence: explicit --profile flag > active_profile (from dg profiles switch) > default_profile

Auth System

  • Credential precedence: explicit --api-key flag > OS keyring (profile-scoped) > config file > env vars
  • Keyring service: com.deepgram.dx.deepctl, keys stored as api-key.<profile_name>
  • Device flow: OAuth via community.deepgram.com/api/auth/device/*
  • auth_manager.guard() verifies the API key by hitting GET /v1/projects
  • auth_manager.get_api_key() returns the key following the precedence chain
  • auth_manager.is_authenticated() returns True if any valid key is available

Output System

The --output / -o global flag controls format: json (default), yaml, table, csv.

Commands that print Rich output directly (tables, status messages) should override output_result() to suppress the automatic JSON dump in default mode:

def output_result(self, result: Any, config: Config) -> None:
    from deepctl_core.output import get_output_format
    if get_output_format() in ("default",):
        return  # Rich output already printed
    super().output_result(result, config)

DeepgramClient

Wraps the deepgram-sdk. The SDK client is lazily created on first access. Key methods:

  • client.transcribe_file(path, options) / client.transcribe_url(url, options)
  • client.get_projects() / client.get_project(id)
  • client.get_usage(project_id, start, end)
  • client.validate_api_key() / client.test_connection()

Shared Utilities (deepctl-shared-utils)

from deepctl_shared_utils import (
    FileInfo,               # Pydantic model for file metadata
    validate_audio_file,    # Check file exists, is audio, is readable
    validate_date_format,   # ISO 8601 validation
    validate_url,           # URL format + optional HEAD probe
)

Argument Specification Format

The get_arguments() return value is a list of dicts. Key differences:

Type Key for name is_option Example
Positional arg "name" (singular string) absent or False {"name": "source", "type": str, "required": True}
Option "names" (list of strings) True {"names": ["--model", "-m"], "type": str, "is_option": True}
Flag "names" (list of strings) True {"names": ["--verbose"], "is_flag": True, "is_option": True}
Multi-value "names" (list of strings) True {"names": ["-H", "--header"], "multiple": True, "is_option": True}

Click converts --my-option to my_option in kwargs. Arguments are processed in reverse order (Click decorator stacking).

Project Tools

Makefile Targets

Target Description
make dev Full dev cycle: format + lint-fix + test
make check Quick quality check: format-check + lint-check + typecheck (no tests)
make test Run pytest
make format Auto-format with ruff
make lint-fix Auto-fix lint issues
make typecheck Run mypy (strict)
make readmes Regenerate all READMEs from pyproject.toml metadata
make readmes-check Check READMEs are up to date (CI)
make clean Remove build artifacts and caches

Aliases: make t (test), make tl (lint), make f (format), make l (lint-fix), make q (check)

README Generation

READMEs are auto-generated from pyproject.toml metadata. Never manually edit sections between <!-- BEGIN:* --> and <!-- END:* --> markers.

make readmes                          # Regenerate all
python3 scripts/generate_readmes.py   # Same thing

The script updates:

  • Root README.md: architecture tree, commands table, packages table
  • Each sub-package README.md: description, commands, dependencies
  • Skips deepctl-plugin-example (has manual README)

Skills System

The skills system generates AI coding assistant integration files:

dg skills status    # Show detected AI CLIs and installed skills
dg skills install   # Install skill files for detected AI CLIs
dg skills update    # Regenerate skill files (e.g., after adding a command)
dg skills remove    # Remove installed skill files

Supported AI CLIs: Claude Code, Codex, Cursor, Gemini, Aider, Amazon Q, OpenCode, Cline.

After adding a new command, run dg skills update to include it in skill files.

Testing

Running Tests

uv run pytest                                          # All tests
uv run pytest packages/deepctl-cmd-<name>/tests/ -v    # Single package
uv run pytest -x                                       # Stop on first failure
uv run pytest -xvs                                     # Verbose with stdout

Test Patterns

from unittest.mock import Mock, patch
from deepctl_core import AuthManager, Config, DeepgramClient

@pytest.fixture
def mock_config():
    config = Mock(spec=Config)
    config.get.return_value = None
    return config

@pytest.fixture
def mock_auth_manager():
    auth = Mock(spec=AuthManager)
    auth.get_api_key.return_value = None
    auth.is_authenticated.return_value = False
    return auth

@pytest.fixture
def mock_client():
    return Mock(spec=DeepgramClient)

class TestYourCommand:
    def test_handle(self, mock_config, mock_auth_manager, mock_client):
        cmd = YourCommand()
        result = cmd.handle(
            config=mock_config,
            auth_manager=mock_auth_manager,
            client=mock_client,
            some_arg="value",
        )
        assert result.status == "success"

Use @patch("deepctl_cmd_<name>.command.some_module") to mock external dependencies. Use tmp_path fixture for filesystem tests.

CI Matrix

Tests run on Python 3.10-3.14 across Ubuntu, Windows, and macOS. The lint job runs ruff format, ruff check, and mypy on all src/ and packages/*/src paths.

Code Quality Rules

Ruff Configuration

  • Line length: 88
  • Target: Python 3.10
  • Key rules: pycodestyle (E/W), pyflakes (F), isort (I), bugbear (B), comprehensions (C4), pyupgrade (UP), unused-args (ARG), simplify (SIM), type-checking (TCH), ruff-specific (RUF)
  • Per-file ignores: __init__.py ignores F401; test_*.py ignores F401 and ARG; command.py ignores ARG002/ARG005/RUF012

Mypy Configuration

  • Strict mode enabled globally
  • Override: deepctl_cmd_mcp.* disables call-arg (FastMCP incomplete stubs)

Import Conventions

  • Use from __future__ import annotations for forward references
  • Move type-only imports into TYPE_CHECKING blocks (enforced by TCH rules)
  • Ruff auto-sorts imports (isort rules)

Release Process

Versioning

Each package is versioned independently via release-please.

  • # x-release-please-version comments on version = lines mark what gets bumped (NOT on dependency lines)
  • .github/.release-please-manifest.json tracks current versions per package
  • .github/release-please-config.json defines package components

Release Flow

  1. PRs merge to main
  2. release-please.yml creates/updates a release PR with changelogs
  3. Merging the release PR creates a v* tag
  4. release.yml builds all packages and publishes to PyPI via twine

Adding a New Package to Releases

  1. Add to .github/release-please-config.json "packages" section
  2. Add to .github/.release-please-manifest.json with initial version (e.g. "0.0.1")

Existing Commands Reference

Command Package Auth Description
dg login deepctl-cmd-login No Browser-based or API key authentication
dg logout deepctl-cmd-login No Clear credentials
dg profiles deepctl-cmd-login No Manage named profiles
dg transcribe deepctl-cmd-transcribe Yes Speech-to-text for files and URLs
dg projects deepctl-cmd-projects Yes List/manage Deepgram projects
dg usage deepctl-cmd-usage Yes View usage statistics
dg api deepctl-cmd-api Yes Raw API requests (dg api GET /v1/projects)
dg init deepctl-cmd-init No Scaffold starter apps from templates gallery
dg mcp deepctl-cmd-mcp No Run MCP server for AI assistants
dg debug deepctl-cmd-debug No Debug group (audio, browser, network, probe, stream)
dg debug probe deepctl-cmd-debug-probe No Live ffprobe analysis during streaming
dg ffprobe deepctl-cmd-ffprobe No FFprobe configuration
dg update deepctl-cmd-update No Check for and install updates
dg plugin deepctl-cmd-plugin No Manage external plugins
dg skills deepctl-cmd-skills No AI coding assistant integrations

Key Patterns to Follow

  1. One package per command — each command is an independent installable package
  2. Entry points for discovery — no hardcoded imports in the main CLI
  3. BaseCommand for leaf commands — implement handle() and get_arguments()
  4. BaseGroupCommand for groups — use entry points or setup_commands() for subcommands
  5. Pydantic models for results — subclass BaseResult for structured output
  6. Rich for human output — tables, spinners, colored text; override output_result() to control when JSON is emitted
  7. Override output_result() — if your command prints Rich output, suppress auto-JSON in default mode
  8. make readmes after changes — always regenerate READMEs after adding/modifying packages
  9. dg skills update after adding commands — regenerate AI assistant skill files
  10. Version markers on version = lines only — # x-release-please-version (not on dependency lines)