feat(revenue-analytics): Add setup-revenue-analytics command#372
feat(revenue-analytics): Add setup-revenue-analytics command#372arthurdedeus wants to merge 14 commits intomainfrom
Conversation
…l detection pipeline Wire up `setup-revenue-analytics` yargs subcommand in bin.ts with language detection, Stripe SDK/call pattern scanning, PostHog distinct_id detection, runtime Stripe docs fetching with fallback, and agent prompt builder. Supports Node, Python, Ruby, PHP, Go, Java, and .NET.
Cover Node, Python, Ruby, PHP, Go, Java, .NET for Stripe SDK detection, customer creation scanning, and charge pattern matching. Test PostHog distinct_id extraction from identify/capture calls across languages. Fix Python distinct_id pattern ordering for keyword arguments.
Test prompt assembly for all 7 languages, distinct_id inclusion/fallback, checkout session handling, and Stripe docs fallback when fetcher fails.
Add comprehensive patterns from PostHog docs for all SDK variants: - Frontend: identify(), get_distinct_id(), React Native - Backend: capture() with distinctId property, alias() - Android/Kotlin: PostHog.identify(distinctId = ...) named params - Java: posthog.capture/identify/alias with method call args - .NET: Identify/CaptureAsync, DistinctId property assignment Also skip test files, filter placeholder values from docs examples, and fix the "agent will ask" message — the agent now searches the codebase itself and falls back to a TODO placeholder.
…ceholders The Stripe docs fallback examples used made-up properties like user.posthogDistinctId that the agent would copy verbatim. Replace all with <POSTHOG_DISTINCT_ID> placeholders. Rewrite prompt to add a "CRITICAL FIRST STEP" section that teaches the agent HOW to determine the actual distinct_id value: - Search for posthog.identify() — first arg is the distinct_id - Search for posthog.capture() — look for distinctId property - Trace the value to the Stripe call site - Explicitly warn: "Do NOT invent properties like user.posthogDistinctId"
…ction Language detection now searches up to depth 3 with ignore patterns, finding package.json/requirements.txt/Gemfile etc. in subdirectories like frontend/ or backend/. Stripe package detection uses glob search (**/) instead of reading fixed root paths. Version extraction resolves lockfiles relative to the directory where the package file was found, not from installDir.
Python projects using uv or Poetry may not have requirements.txt. Add uv.lock and poetry.lock to the Stripe package detection sources, matching the `name = "stripe"` TOML pattern used by both lockfiles.
Incorporate four safety tenets into the prompt: 1. Never fabricate — don't substitute wrong identifiers 2. Thread the value — propagate as optional param, skip if impossible 3. Minimize API calls — deduplicate Customer.modify, don't add before every charge 4. Follow abstractions — modify Stripe utility layers, not business logic
…pdates The agent now emits status messages at each milestone so the user sees progress instead of a static "Setting up revenue analytics..." spinner: identifying distinct_id, updating customer creation, adding metadata for existing customers, verifying changes.
Remove explicit pushStatus call from handleSDKMessage — spinner.message already calls pushStatus internally in both InkUI and LoggingUI, so the [STATUS] line was appearing twice.
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
arthurdedeus
force-pushed
the
feat/revenue-analytics-setup
branch
from
cbcd32f to
6cc29aa
Compare
… alert Replace single-pass tag strip with a do/while loop that repeats until stable. This handles reconstructed tags from nested fragments (e.g. <scr<script>ipt>) and satisfies CodeQL's js/incomplete-multi-character-sanitization rule.
…scaping Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…scaping Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
edwinyjlim
left a comment
edwinyjlim
left a comment
There was a problem hiding this comment.
oh yeah! lots of goodies here
I'm gonna park this PR in draft. We need to rework this architecture so it's more streamlined and skills-driven way with the changes we have coming down the pipeline.
@arthurdedeus We're probably reverse uno card this and tag you in a new wizard PR that installs revenue analytics via skills
There was a problem hiding this comment.
we're moving completely to skills-based architecture, so we can always have dynamic and up-to-date context that's decoupled from the source package
lots of good stuff in this prompt, but let's figure out how to move this to https://github.com/PostHog/context-mill
There was a problem hiding this comment.
there's a ton of duplication of agent-runner going on here: logging, health check, auth, MCP URL, settings conflicts, agent init/run, error handling
we're working on making the agent-runner and agent-interface services more composable and modular, so different runs can reuse the same boot sequence as we expand the wizard
| const STRIPE_DOCS_URLS = { | ||
| customerCreate: 'https://docs.stripe.com/api/customers/create', | ||
| customerUpdate: 'https://docs.stripe.com/api/customers/update', | ||
| }; |
There was a problem hiding this comment.
this is cool! we can source this upstream in the context mill so the wizard is decoupled from context building
|
Thanks for taking a swing at this, Arthur! Appreciate you giving it the time. we definitely want the wizard to support this, but we can’t merge this direction. There’s a few signals why:
Still, this starting place is loads of information to harmonize with a proper wizard architecture:
So that’s the basic shape of the path we’d take here, but wanted to offer the option in case it was interesting for you to give a rewrite a try with our guidance? the only way we can make the wizard work long term is by betting really hard on the capabilities of models and agents, and this approach does require a different approach than conventional code investments. Happy to show you the way if it’s interesting, or we’ll take it from here, your pick! one other thing: are the docs you mentioned live yet? |
|
Thank you so much for the review @edwinyjlim and @daniloc! This is 100% discovery/experimental stuff on my end, so I really appreciate all the feedback. The docs are not yet live, I'll merge it once we get the wizard going for the setup. While I was building, I figured most of the pattern-matching stuff was going to be throw-away work, but I went on regardless just so I could get it to a working state. IMO the prompts are the most valuable assets from this PR (not that they're perfect, but they seemed effective in the tests). I'm happy to take on the work here with some guidance, really keen to learn more about how it all works and build more stuff on our AI platform. |
4 participants
In Revenue analytics, we want to standardize the way we connect revenue data to PostHog persons. To do that, we will be prescriptive in having
posthog_person_distinct_idprop in Stripe customer metadata so that we are able to automatically create a join.To guide the users on how to implement this, I'm updating the docs with a step-by-step guide on how to get the metadata populated with that field. But I thought we could also get the wizard to do that for them. So this is where the idea for this new setup command was born.
This wizard command should introspect the user codebase and:
Here's a Loom I've recorded demoing it in a demo repo.
Part of PostHog/posthog#52270