Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 58 additions & 12 deletions skills/trace-claude-code/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ Claude Code Session (root trace)

Four hooks capture the complete workflow:

| Hook | What it captures |
|------|------------------|
| **SessionStart** | Creates root trace when you start Claude Code |
| **PostToolUse** | Captures every tool call (file reads, edits, terminal commands) |
| **Stop** | Captures conversation turns (your message + Claude's response) |
| **SessionEnd** | Logs session summary when you exit |
| Hook | What it captures |
| ---------------- | --------------------------------------------------------------- |
| **SessionStart** | Creates root trace when you start Claude Code |
| **PostToolUse** | Captures every tool call (file reads, edits, terminal commands) |
| **Stop** | Captures conversation turns (your message + Claude's response) |
| **SessionEnd** | Logs session summary when you exit |

## Quick setup

Expand Down Expand Up @@ -117,12 +117,52 @@ Replace `/path/to/hooks/` with the actual path to this skill's hooks directory.

### Environment variables

| Variable | Required | Description |
|----------|----------|-------------|
| `TRACE_TO_BRAINTRUST` | Yes | Set to `"true"` to enable tracing |
| `BRAINTRUST_API_KEY` | Yes | Your Braintrust API key |
| `BRAINTRUST_CC_PROJECT` | No | Project name (default: `claude-code`) |
| `BRAINTRUST_CC_DEBUG` | No | Set to `"true"` for verbose logging |
| Variable | Required | Description |
| ---------------------------- | -------- | ------------------------------------------------------------------------------- |
| `TRACE_TO_BRAINTRUST` | Yes | Set to `"true"` to enable tracing |
| `BRAINTRUST_API_KEY` | Yes | Your Braintrust API key |
| `BRAINTRUST_CC_PROJECT` | No | Project name (default: `claude-code`) |
| `BRAINTRUST_CC_DEBUG` | No | Set to `"true"` for verbose logging |
| `BRAINTRUST_REDACT_ENABLED` | No | Set to `"false"` to disable secret redaction (default: `"true"`) |
| `BRAINTRUST_REDACT_PATTERNS` | No | Comma-separated list of additional regex patterns to redact |
| `BRAINTRUST_SKIP_FILES` | No | Comma-separated list of file patterns to skip content for (e.g., `*.env,*.pem`) |

### Secret redaction

By default, the plugin redacts common secrets before sending data to Braintrust:

**Automatically redacted patterns:**

- API keys: `sk-*`, `ghp_*`, `gho_*`, `xoxb-*`, `xoxp-*`, `AKIA*`, `npm_*`, `pypi-*`
- JWT tokens
- Generic secrets: `password=`, `secret=`, `api_key=`, `token=`, `database_url=`

**Files with redacted content:**

- `.env`, `.env.*`, `.env.local`, `.env.production`
- `*credentials*`, `*secrets*`
- `*.pem`, `*.key`, `*.p12`, `id_rsa*`, `id_ed25519*`

To add custom redaction patterns:

```json
{
"env": {
"BRAINTRUST_REDACT_PATTERNS": "my_api_key_.*,custom_secret_.*",
"BRAINTRUST_SKIP_FILES": "*.secret,config/keys/*"
}
}
```

To disable redaction entirely (not recommended):

```json
{
"env": {
"BRAINTRUST_REDACT_ENABLED": "false"
}
}
```

## Viewing traces

Expand All @@ -133,6 +173,7 @@ After running Claude Code with tracing enabled:
3. Click **Logs** to see all traced sessions

Each trace shows:

- **Session root**: The overall Claude Code session
- **Turns**: Each conversation exchange (user input → assistant response)
- **Tool calls**: Individual operations (file reads, edits, terminal commands)
Expand All @@ -142,11 +183,13 @@ Each trace shows:
Traces are hierarchical:

- **Session** (root span)

- `span_attributes.type`: `"task"`
- `metadata.session_id`: Unique session identifier
- `metadata.workspace`: Project directory

- **Turn** (child of session)

- `span_attributes.type`: `"llm"`
- `input`: User message
- `output`: Assistant response
Expand All @@ -163,11 +206,13 @@ Traces are hierarchical:
### No traces appearing

1. **Check hooks are running:**

```bash
tail -f ~/.claude/state/braintrust_hook.log
```

2. **Verify environment variables** in `.claude/settings.local.json`:

- `TRACE_TO_BRAINTRUST` must be `"true"`
- `BRAINTRUST_API_KEY` must be valid

Expand All @@ -191,6 +236,7 @@ chmod +x /path/to/hooks/*.sh
### Missing jq command

Install jq:

- **macOS**: `brew install jq`
- **Ubuntu/Debian**: `sudo apt-get install jq`

Expand Down
143 changes: 142 additions & 1 deletion skills/trace-claude-code/hooks/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,147 @@ get_timestamp() {
date -u +"%Y-%m-%dT%H:%M:%S.000Z"
}

# Get system info for metadata
# Secret redaction configuration
export REDACT_ENABLED="${BRAINTRUST_REDACT_ENABLED:-true}"
export REDACT_PATTERNS="${BRAINTRUST_REDACT_PATTERNS:-}"
export SKIP_FILES="${BRAINTRUST_SKIP_FILES:-}"

# Default patterns to redact (common API keys and secrets)
DEFAULT_REDACT_PATTERNS=(
'sk-[a-zA-Z0-9]{20,}' # OpenAI/Anthropic API keys
'sk-proj-[a-zA-Z0-9]{20,}' # OpenAI project keys
'ghp_[a-zA-Z0-9]{36,}' # GitHub personal access tokens
'gho_[a-zA-Z0-9]{36,}' # GitHub OAuth tokens
'github_pat_[a-zA-Z0-9_]{22,}' # GitHub fine-grained PATs
'xoxb-[a-zA-Z0-9-]+' # Slack bot tokens
'xoxp-[a-zA-Z0-9-]+' # Slack user tokens
'AKIA[A-Z0-9]{16}' # AWS access key IDs
'eyJ[a-zA-Z0-9_-]*\.eyJ[a-zA-Z0-9_-]*' # JWT tokens
'npm_[a-zA-Z0-9]{36}' # npm tokens
'pypi-[a-zA-Z0-9]{36,}' # PyPI tokens
)

# Default file patterns to skip content for
DEFAULT_SKIP_FILE_PATTERNS=(
'*.env'
'*.env.*'
'.env.local'
'.env.production'
'.env.development'
'*credentials*'
'*secrets*'
'*.pem'
'*.key'
'*.p12'
'*.pfx'
'id_rsa*'
'id_ed25519*'
'*.asc'
)

# Check if redaction is enabled
redaction_enabled() {
[ "$(echo "$REDACT_ENABLED" | tr '[:upper:]' '[:lower:]')" = "true" ]
}

# Check if a file path matches skip patterns
should_skip_file_content() {
local file_path="$1"
[ -z "$file_path" ] && return 1

local filename
filename=$(basename "$file_path")

# Check user-defined patterns first
if [ -n "$SKIP_FILES" ]; then
IFS=',' read -ra USER_PATTERNS <<< "$SKIP_FILES"
for pattern in "${USER_PATTERNS[@]}"; do
pattern=$(echo "$pattern" | xargs) # trim whitespace
case "$filename" in
$pattern) return 0 ;;
esac
case "$file_path" in
$pattern) return 0 ;;
esac
done
fi

# Check default patterns
for pattern in "${DEFAULT_SKIP_FILE_PATTERNS[@]}"; do
case "$filename" in
$pattern) return 0 ;;
esac
done

return 1
}

# Redact secrets from a string
# Usage: echo "$content" | redact_secrets
redact_secrets() {
local input
input=$(cat)

# Skip if redaction disabled
if ! redaction_enabled; then
echo "$input"
return 0
fi

local result="$input"

# Apply default patterns
for pattern in "${DEFAULT_REDACT_PATTERNS[@]}"; do
result=$(echo "$result" | sed -E "s/$pattern/[REDACTED]/g" 2>/dev/null || echo "$result")
done

# Apply generic key=value patterns for common secret names
result=$(echo "$result" | sed -E \
-e 's/(password["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(secret["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(api_key["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(apikey["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(token["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(private_key["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(access_token["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(refresh_token["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(client_secret["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
-e 's/(database_url["\x27]?\s*[:=]\s*["\x27]?)([^"\x27\s,}]+)/\1[REDACTED]/gi' \
2>/dev/null || echo "$result")

# Apply user-defined patterns
if [ -n "$REDACT_PATTERNS" ]; then
IFS=',' read -ra USER_PATTERNS <<< "$REDACT_PATTERNS"
for pattern in "${USER_PATTERNS[@]}"; do
pattern=$(echo "$pattern" | xargs) # trim whitespace
result=$(echo "$result" | sed -E "s/$pattern/[REDACTED]/g" 2>/dev/null || echo "$result")
done
fi

echo "$result"
}

# Redact JSON content (handles nested structures)
# Usage: redact_json "$json_string"
redact_json() {
local json="$1"

if ! redaction_enabled; then
echo "$json"
return 0
fi

# Pass through redact_secrets for pattern matching
echo "$json" | redact_secrets
}

# Create a redacted placeholder for sensitive files
get_redacted_file_placeholder() {
local file_path="$1"
jq -n --arg path "$file_path" \
'{"content": "[REDACTED - SENSITIVE FILE]", "file": $path, "redacted": true}'
}

get_hostname() {
hostname 2>/dev/null || echo "unknown"
}
Expand All @@ -238,3 +378,4 @@ get_username() {
get_os() {
uname -s 2>/dev/null || echo "unknown"
}

16 changes: 14 additions & 2 deletions skills/trace-claude-code/hooks/post_tool_use.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,22 @@ debug "PostToolUse input: $(echo "$INPUT" | jq -c '.' 2>/dev/null | head -c 500)

# Extract tool info
TOOL_NAME=$(echo "$INPUT" | jq -r '.tool_name // empty' 2>/dev/null)
TOOL_INPUT=$(echo "$INPUT" | jq -c '.tool_input // {}' 2>/dev/null)
TOOL_OUTPUT=$(echo "$INPUT" | jq -c '.tool_response // .output // {}' 2>/dev/null)
SESSION_ID=$(echo "$INPUT" | jq -r '.session_id // empty' 2>/dev/null)

# Extract file path for file-based tools (for redaction check)
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // .tool_input.path // empty' 2>/dev/null)

# Check if this is a sensitive file that should have content redacted
if [ -n "$FILE_PATH" ] && should_skip_file_content "$FILE_PATH"; then
debug "Sensitive file detected, redacting content: $FILE_PATH"
TOOL_INPUT=$(echo "$INPUT" | jq -c '.tool_input | del(.content) | . + {content: "[REDACTED - SENSITIVE FILE]"}' 2>/dev/null)
TOOL_OUTPUT=$(get_redacted_file_placeholder "$FILE_PATH")
else
# Apply standard redaction to inputs and outputs
TOOL_INPUT=$(echo "$INPUT" | jq -c '.tool_input // {}' 2>/dev/null | redact_secrets)
TOOL_OUTPUT=$(echo "$INPUT" | jq -c '.tool_response // .output // {}' 2>/dev/null | redact_secrets)
fi

# Skip if no tool name
[ -z "$TOOL_NAME" ] && { debug "No tool name, skipping"; exit 0; }
[ -z "$SESSION_ID" ] && { debug "No session ID, skipping"; exit 0; }
Expand Down
17 changes: 11 additions & 6 deletions skills/trace-claude-code/hooks/stop_hook.sh
Original file line number Diff line number Diff line change
Expand Up @@ -135,19 +135,24 @@ create_llm_span() {
local start_time=$(iso_to_epoch "$start_ts")
local end_time=$(iso_to_epoch "$end_ts")

# Input is the conversation history up to this point
local input_json="$input_history"
# Apply secret redaction to conversation history and output
local input_json
input_json=$(echo "$input_history" | redact_secrets)
local redacted_output_text
redacted_output_text=$(echo "$output_text" | redact_secrets)
local redacted_tool_calls
redacted_tool_calls=$(echo "$tool_calls_json" | redact_secrets)

# Format output - include tool_calls if present
local output_json
local has_tool_calls=$(echo "$tool_calls_json" | jq 'length > 0' 2>/dev/null)
local has_tool_calls=$(echo "$redacted_tool_calls" | jq 'length > 0' 2>/dev/null)
if [ "$has_tool_calls" = "true" ]; then
output_json=$(jq -n \
--arg content "${output_text:-}" \
--argjson tool_calls "$tool_calls_json" \
--arg content "${redacted_output_text:-}" \
--argjson tool_calls "$redacted_tool_calls" \
'{role: "assistant", content: $content, tool_calls: $tool_calls}')
else
output_json=$(jq -n --arg content "$output_text" '{role: "assistant", content: $content}')
output_json=$(jq -n --arg content "$redacted_output_text" '{role: "assistant", content: $content}')
fi

local event=$(jq -n \
Expand Down