combine plugins multiple sources to simplify UI by florinnania · Pull Request #25 · HexRaysSA/plugin-repository

florinnania · 2026-03-23T08:36:13Z

Simplify Plugin Repository UI — Eliminate the API

The UI currently does a big merge at runtime in DataMerger.js — fetches from multiple sources, transforms, deduplicates, enriches. This PR moves all of that into a Python build script that runs offline and outputs a single ready-to-use JSON. The UI just loads it. No API, no runtime merging.

The inputs

JSON	What	Where
A — API export	All 169 plugins as they exist in the API today	One-time dump → `api-plugins.json`
B — Plugin repos	96 HCLI-indexed plugins (have `ida-plugin.json`)	`plugin-repository.json` on GitHub (auto-generated)
C — Tags	Contest winners, favourites, hidden gems	`tags.json` on GitHub

Plus GitHub metadata (repositories-metadata.json — stars, forks, dates) and categories (currently from the API, you'll create a categories.json).

Important: C (tags.json on GitHub) is incomplete — it only has favourite and plugin_contest_2024. The full tag set (contest years 2016-2024, placements first_place/second_place/third_place, award_winning, etc.) only exists in A (the API). So the build script gets tags from both A and C, merged.

What the script does

Basically what DataMerger.js does today, but at build time:

Merge A + B (B wins when duplicates — richer data)
Auto-tag all B plugins with plugin_manager_ready
Enrich every plugin with GitHub metadata (stars, forks, language, dates, avatar, etc.)
Merge tags from A + C (A has the full tag set, C may have additions)
Resolve README URLs (check if they exist, store the working URL — no 404 guessing in the UI)
Normalize category slugs
Output one plugins-combined.json ready for the UI

williballenthin

The direction of this PR is fine, but some changes are needed.

there are some oversights that need to be fixed, including adding a second copy of the plugin repo. And, LLM-generated text that doesn't seem quite right (exactly 12 categories is a strict output requirement?).
some of the presentation-layer data should move into the web UI, and not be stored in this repo.
we should consider removing the legacy plugins from plugins.hex-rays.com, which will simplify the code, including removing quite a bit from this PR.

I realize that for (2) and (3) you're following the assignment, but I think we (including @pal-hexrays if necessary) should realign rather than push this through.

williballenthin · 2026-03-23T09:37:48Z

scripts/merge_plugins.py

+#
+# Inputs:
+#   A  api-plugins.json       All plugins from the Hex-Rays API (one-time dump)
+#   B  hcli-plugins.json      HCLI-indexed plugins (plugin-repository.json from GitHub)


this should be plugin-repository.json

Done. At the beginning I saw that they are not identical and I was afraid to update plugin-repository.json because I was thinking that maybe other services are counting on it.

right, plugin-repository.json is auto-generated periodically, and shouldn't be modified directly.

hcli-plugins.json

williballenthin · 2026-03-23T09:40:00Z

github-metadata.json

i think this should be generated on demand, rather than committed, since it changes so frequently. thoughts?

added fetch-metadata and merge-plugins depends on fetch-metadata, so it always gets fresh data automatically.

CLAUDE.md

williballenthin · 2026-03-23T09:45:23Z

api-plugins.json

this might be outside the scope of this PR, but I think we should remove the legacy plugins from plugins.hex-rays.com. at this point we have ~100 HCLI-compatible plugins, so the index is sufficiently populated and the ecosystem is healthy. removing the legacy plugins will simplify code in many places, as well as reduce confusion about which plugins show up where.

@pal-hexrays @hx-geoffrey thoughts?

williballenthin · 2026-03-23T09:46:33Z

api-categories.json

imho, this presentation layer information belongs in the web user interface codebase, not here. which icons are used for each category and how they are translated from ID -> human reasonable name is the domain of the UI, not of the backend json data.

api-categories.json

williballenthin · 2026-03-23T09:50:50Z

scripts/merge_plugins.py

+#
+# Outputs:
+#   plugins-combined.json     All plugins (HCLI + legacy), enriched and ready for the UI
+#   categories.json           12 category definitions with pluginCount


is "12" part of the API contract? or simply a snapshot of the current count? this reads like LLM-authored content and i can't tell the intent.

Comment updated. 12 was a snapshot of the current count but if I got it right categories will stay the same in future.

scripts/merge_plugins.py

williballenthin · 2026-03-23T09:57:08Z

also, this should have been first: thank you @florinnania! I'm happy we're moving in this direction and you're helping to make it so :-)

williballenthin · 2026-03-24T09:26:12Z

justfile



+fetch-metadata:
+    curl -sSfL -o github-metadata.json https://hexrayssa.github.io/plugin-repository/plugins/github.com/repositories-metadata.json


the step collect-starts just before this emits repositories-metadata.json so lets just use that?

williballenthin · 2026-03-24T09:26:33Z

plugin-repository.json

why is this file modified?

combine plugins to simplify UI

790f010

florinnania requested a review from williballenthin March 23, 2026 08:38

williballenthin requested changes Mar 23, 2026

View reviewed changes

fnania added 5 commits March 23, 2026 14:55

delete pluginCount and add schema

6bf6ac2

fix identation

05edf78

deleted agent context

f53fab2

use plugin-repository.json instead of hcli-plugins.json

d4e7e06

revert plugin-repository.json and get fresh metadata

4ff9041

williballenthin reviewed Mar 24, 2026

View reviewed changes

plugin-repository.json

Copy link

Collaborator

williballenthin Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this file modified?



		fetch-metadata:
		curl -sSfL -o github-metadata.json https://hexrayssa.github.io/plugin-repository/plugins/github.com/repositories-metadata.json

Conversation

florinnania commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Simplify Plugin Repository UI — Eliminate the API

The inputs

What the script does

Uh oh!

williballenthin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

williballenthin commented Mar 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

florinnania commented Mar 23, 2026 •

edited

Loading