feat: Logfire observability adoption - config, base integration, tests

[lpayne-dev]

Summary

This PR adds optional Logfire observability to AKD.

It’s off by default, so nothing changes for local dev or tests.

If you enable it via env vars, you can trace:

• agent runs

• tool executions

• streaming lifecycle events

in Logfire (or stdout if you’re not sending to cloud).

---

Details

What changed

This PR introduces a new akd.observability module that centralizes Logfire initialization and tracing helpers, including init_observability() (no-op when disabled or running under pytest), span_agent_run, span_tool_execution, and log_stream_event for streaming lifecycle tracing. All logging is redacted and truncated before sending to prevent leakage of sensitive data such as API keys or large payloads.

It adds a TelemetrySettings configuration model in akd.configs.project with env-driven controls for enabling logging, setting service and environment labels, configuring sample rate, toggling redaction behavior, and choosing whether to send data to Logfire cloud. Logfire telemetry remains disabled by default to ensure local development and tests behave exactly as before.

The .env.example file now documents the new TELEMETRY_* variables, and .gitignore excludes .logfire/ to prevent local credentials or Logfire artifacts from being committed.

Script entrypoints (demo, planner, lit-agent), dependency files (pyproject.toml, uv.lock), and relevant streaming/tool/search integrations were updated to support clean Logfire initialization and hook integration.

Finally, a new test suite under tests/observability/test_logfire.py validates redaction behavior, confirms safe initialization when disabled or under pytest, and ensures there are no import-time side effects.

---

Checks

• Closed
• Tested Changes
• Stakeholder Approval

[github-actions]

✅ Tests passed

📊 Test Results

• Passed: 588
• Failed: 0
• Skipped: 39
• Warnings: 259
• Coverage: 78%

Branch: feature/logfire-adoption
PR: #355
Commit: 0692def

📋 Full coverage report and logs are available in the workflow run.

[NISH1001]

@lpayne-dev thanks for this. There was a major refactor in the base agent last week for makign the downstream agents adopt better (eg: akd-ext).
https://github.com/NASA-IMPACT/accelerated-discovery/tree/refactor/run-session

This introduces the concept of session in akd and also lots of clean-ups for simplifying the downstream subclasses.

---

So, I fear there will be lots of merge conflict with that. For now let's put this observability as a draft PR and not merge until we merge the refactored code.