From 00c6bf4e4128c93efcb68dfe8fa9747084e58bd2 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 14 May 2026 01:59:23 +0000 Subject: [PATCH 1/2] Refresh CLAUDE.md for the layered src/ architecture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrites the AI assistant context doc to match the post-PR-#39 architecture. Updates the source-layer diagram (transport → sources → store → domain → services → http), bumps the data-source count to 5 (adds L2BEAT), lists the new endpoints (/scaling, /metrics, /refresher, /clients, /relations/:id/graph, /keywords, /stats, /validate), notes pino logging, JSON Schema validation, and ESLint, and updates env vars + tests folder layout. The "Common tasks" section is rewritten to point at src/ paths instead of dataService.js / index.js. --- CLAUDE.md | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..cd756e0 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,197 @@ +# CLAUDE.md — Chains API + +## Project Overview + +Chains API is a Node.js service that aggregates blockchain chain data from five external sources, maintains an in-memory index, and exposes it via a REST API (Fastify), MCP stdio server, and MCP HTTP server. No database — data is fetched from remote JSON/Markdown sources, indexed in memory, and optionally cached to disk for stale-first startup. + +## Quick Reference + +```bash +npm install # Install dependencies (Node >=20 required) +npm start # Start REST API on port 3000 +npm run dev # Start with --watch for auto-reload +npm run mcp # Start MCP stdio server +npm run mcp:http # Start MCP HTTP server on port 3001 +npm test # Run all tests (vitest run) +npm run test:watch # Run tests in watch mode +npm run test:coverage # Run tests with v8 coverage report +npm run lint # ESLint on src/ +``` + +## Architecture + +The codebase is organized into a layered `src/` structure. Legacy root-level files (`dataService.js`, `index.js`, `mcp-tools.js`, etc.) remain as thin facades that re-export from `src/` so older imports and tests keep working. + +``` +External HTTP sources + ↓ +src/transport/fetch.js ← proxy-aware fetch wrapper + ↓ +src/sources/{l2beat,slip44}.js ← per-source parsers / fetchers + ↓ +src/store/ ← in-memory index + disk cache + ├─ indexer.js (build indexed.byChainId / byName / all) + ├─ queries.js (search, getChain, getRelations, …) + ├─ cache.js (stale-first disk persistence) + └─ snapshot.js (export / reload coordination) + ↓ +src/domain/ ← pure business logic + ├─ relations.js (l2Of, testnetOf, parentOf, mainnetOf) + └─ keywords.js (search keyword index) + ↓ +src/services/ ← background tasks + ├─ chainRefresher.js (unified rolling RPC + L2BEAT refresher) + ├─ rpcHealth.js (RPC liveness checks) + ├─ l2beatRefresher.js (legacy shim → chainRefresher) + ├─ validation.js (16 cross-source validation rules) + └─ loader.js (initial data load) + ↓ +src/http/ ← Fastify routes + ├─ app.js (Fastify factory) + └─ routes/*.js (one file per resource) +``` + +## Tech Stack + +- **Runtime:** Node.js >=20, ES Modules (`"type": "module"`) +- **HTTP:** Fastify v5 (REST API), Express v5 (MCP HTTP server) +- **MCP SDK:** `@modelcontextprotocol/sdk` v1.26+ +- **Logging:** pino structured JSON logs (no `console.*` in src/) +- **Validation:** AJV via Fastify's JSON Schema, with `ajv-errors` for friendly messages +- **Testing:** Vitest v4 with `@vitest/coverage-v8`, `fast-check` for property-based fuzz tests +- **Linting:** ESLint v10 (`eslint.config.js`, flat config) +- **CI/CD:** GitHub Actions — test, SonarQube scan, Docker build/push to GHCR +- **Containerization:** Docker (node:20-alpine), Docker Compose + +## Data Sources + +1. **TheGraph Networks Registry** — Network/subgraph endpoint data +2. **Chainlist** — RPC endpoint lists (`rpcs.json`) +3. **Chain ID Network** — Basic chain metadata (`chains.json`) +4. **SLIP-0044** — Coin type registry (parsed from Markdown table) +5. **L2BEAT** — L2 classification (stage, category, stack, DA layer, TVS, activity); live API with checked-in fallback at `data/l2beat-fallback.json` + +Source URLs are configurable via `DATA_SOURCE_*` environment variables (see `config.js`). + +## Testing + +**Framework:** Vitest with globals enabled (no explicit imports needed for `describe`, `it`, `expect`). + +``` +tests/ +├── unit/ +│ ├── store/ (indexer, cache, queries) +│ ├── sources/ (l2beat, slip44) +│ ├── services/ (chainRefresher, l2beatRefresher, validation) +│ ├── domain/ (relations) +│ ├── http/ (admin, metrics, helpers) +│ ├── transport/ (fetch) +│ ├── dataService.test.js (legacy facade) +│ └── index.test.js (legacy facade) +├── integration/ (full API + api.fuzz.test.js property tests) +├── fixtures/ (shared test data) +└── helpers/ (test utilities) +``` + +**Conventions:** +- New tests live under `tests/unit//` matching the source path +- Test timeout: 30 seconds (configured in `vitest.config.js`) +- Coverage target: ≥80% line coverage (enforced by CI/SonarQube) +- All tests must pass before Docker image is built in CI + +**Running a single test file:** +```bash +npx vitest run tests/unit/store/indexer-l2beat.test.js +``` + +## Code Conventions + +- **ES Modules only** — `import`/`export`, not `require` +- **No build step** — source files run directly with Node +- **Config via environment** — all tunables go through `config.js` with typed parsing +- **Structured logging** — `import { logger } from '../util/logger.js'`; never `console.log` in `src/` +- **Schema-first routes** — every Fastify route declares a JSON Schema for `querystring`/`params`/`body`; typos like `?tags=` (vs `?tag=`) return 400 +- **Rate limiting** — global, search, and reload endpoints each have separate limits + +## Environment Variables + +Copy `.env.example` to `.env` for local configuration. Key variables: + +| Variable | Default | Purpose | +|----------|---------|---------| +| `PORT` | `3000` | REST API port | +| `MCP_PORT` | `3001` | MCP HTTP server port | +| `CORS_ORIGIN` | `*` | Allowed CORS origins | +| `PROXY_URL` | (empty) | HTTP/HTTPS proxy URL | +| `DATA_CACHE_ENABLED` | `true` | Enable disk caching | +| `DATA_CACHE_FILE` | `.cache/chains-api-data.json` | Cache file path | +| `LOG_LEVEL` | `info` | pino log level | +| `CHAIN_REFRESHER_TICK_MS` | `1000` | Unified refresher tick interval | +| `DATA_SOURCE_L2BEAT_API` | `https://l2beat.com/api/scaling-summary` | L2BEAT endpoint | +| `L2BEAT_FETCH_TIMEOUT_MS` | `10000` | L2BEAT live fetch timeout | +| `RPC_MONITOR_LOOP` | `false` | Enable continuous RPC monitoring (legacy; superseded by chainRefresher) | + +See `config.js` and `.env.example` for the full list. + +## CI/CD Pipeline + +GitHub Actions workflows in `.github/workflows/`: + +1. **`docker-build.yml`** — On push to main/tags/PRs: runs `npm ci`, `npm run test:coverage`, SonarQube scan, then builds and pushes Docker image to GHCR +2. **`static.yml`** — Deploys `public/` to GitHub Pages on push to main +3. **`auto-tag.yml`** — Auto-creates git tags from `package.json` version on main + +**Quality gates:** Coverage ≥80%, duplication ≤3%, no critical security vulnerabilities. + +## Docker + +```bash +docker compose up # Start both REST API and MCP HTTP server +docker compose up chains-api # Start only the REST API +``` + +Services: `chains-api` (port 3000) and `chains-api-mcp` (port 3001). Both have health checks on `/health`. + +## API Endpoints (REST) + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/` | API info | +| GET | `/health` | Per-source freshness + per-refresher status + overall `ok`/`degraded`/`down` | +| GET | `/sources` | Data source loaded state | +| GET | `/chains` | All chains (optional `?tag=`) | +| GET | `/chains/:id` | Chain by ID | +| GET | `/search?q=` | Search chains | +| GET | `/endpoints` | All endpoints | +| GET | `/endpoints/:id` | Endpoints by chain | +| GET | `/relations` | All chain relations | +| GET | `/relations/:id` | Relations by chain | +| GET | `/relations/:id/graph` | Relation subgraph for chain | +| GET | `/slip44` | All SLIP-0044 coin types | +| GET | `/slip44/:coinType` | Coin type by ID | +| GET | `/scaling` | L2BEAT projects | +| GET | `/scaling/:id` | L2BEAT project by chain ID | +| GET | `/scaling/status` | L2BEAT refresh status | +| GET | `/clients` | Execution-client registry | +| GET | `/clients/:id` | Client by id | +| GET | `/rpc-monitor` | RPC health results | +| GET | `/rpc-monitor/:id` | RPC results by chain | +| GET | `/keywords` | Indexed search keywords | +| GET | `/stats` | Aggregate counts | +| GET | `/validate` | Run 16 cross-source validation rules | +| GET | `/export` | Export cached data | +| GET | `/metrics` | Prometheus exposition (counters + gauges) | +| GET | `/refresher` | Unified refresher cursor + queue depth | +| POST | `/reload` | Reload all data sources | + +## Common Tasks + +**Add a new API endpoint:** Create or edit a file in `src/http/routes/`, declare the JSON Schema, register the route. Add tests in `tests/unit/http/` and/or `tests/integration/api.test.js`. + +**Add a new MCP tool:** Define schema and handler in `mcp-tools.js`. Both MCP servers (`mcp-server.js`, `mcp-server-http.js`) consume tools from this shared module. + +**Add a new data source:** Add a fetcher under `src/sources/`, an indexer pass in `src/store/indexer.js`, wire it into `src/services/loader.js`, expose any user-facing data via a new route in `src/http/routes/`. Update `config.js` for the source URL and add tests under `tests/unit/sources/` and `tests/unit/store/`. + +**Modify environment config:** Edit `config.js` using the existing `parseIntEnv`/`parseStringEnv`/`parseBooleanEnv` helpers. Update `.env.example` with the new variable and default. + +**Add a validation rule:** Add the rule to `src/services/validation.js`, increment the rule count in tests, expose a per-rule counter via `src/util/metrics.js` so `/metrics` tracks it. From 18756fb840ff7975c85040840a469a31bdd8a271 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 14 May 2026 01:59:34 +0000 Subject: [PATCH 2/2] Fix missing chainlist L2/testnet relations + add sources toggle UI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bug fix ------- Chainlist entries with parent.type === 'L2' or 'testnet' were not being processed for relations — only the bridge URLs were extracted, so a chain that appeared only in chainlist (with a parent declaration) never got its l2Of / testnetOf relation, the corresponding L2 tag, or the reverse parentOf / mainnetOf relation on the parent. In src/store/indexer.js, indexChainlistSource now calls processL2ParentRelation and processTestnetParentRelation in the chainlist second pass with source='chainlist'. Both helpers take an optional source parameter (default 'chains') so the same code path can attribute relations correctly. addReverseRelations already propagates the source to the reverse side, so chainlist-sourced relations also produce correctly-attributed reverse relations on the parent chain. Existing dedup logic (existingRelation check by kind + chainId) handles the case where both chains.json and chainlist declare the same parent. UI -- public/* adds a Sources toggle panel (bottom-right) that lets users enable/disable individual data sources (Chain ID Network, Chainlist, The Graph, SLIP-0044, L2BEAT). Toggling a source filters the 3D graph to show only chains whose sources array intersects the enabled set, and rebuilds the relations map accordingly. Tests ----- tests/unit/store/indexer-chainlist-relations.test.js (new file, 5 cases): - L2 relation extraction from chainlist parent - testnet relation extraction from chainlist parent - no duplicate relations when chains.json and chainlist agree - reverse parentOf created with source='chainlist' - reverse mainnetOf created with source='chainlist' --- public/app.js | 60 ++++++- public/index.html | 39 +++++ public/style.css | 100 ++++++++++++ src/store/indexer.js | 15 +- .../store/indexer-chainlist-relations.test.js | 148 ++++++++++++++++++ 5 files changed, 354 insertions(+), 8 deletions(-) create mode 100644 tests/unit/store/indexer-chainlist-relations.test.js diff --git a/public/app.js b/public/app.js index 2c723dc..628c41d 100644 --- a/public/app.js +++ b/public/app.js @@ -8,9 +8,12 @@ const COLORS = { }; // Global State +const ALL_SOURCES = ['chains', 'chainlist', 'theGraph', 'slip44', 'l2beat']; +let allChains = []; let graphData = { nodes: [], links: [] }; let filteredData = { nodes: [], links: [] }; let currentFilter = 'all'; +let enabledSources = new Set(ALL_SOURCES); let myGraph = null; // ─── Utility: Debounce ─── @@ -229,6 +232,33 @@ function initUI() { document.getElementById('detailsPanel')?.classList.add('hidden'); }); } + + // Sources Toggle + const sourcesToggle = document.getElementById('sourcesToggle'); + const sourcesDropdown = document.getElementById('sourcesDropdown'); + if (sourcesToggle && sourcesDropdown) { + sourcesToggle.addEventListener('click', () => { + sourcesDropdown.classList.toggle('hidden'); + }); + + document.addEventListener('click', (e) => { + if (!e.target.closest('#sourcesPanel')) { + sourcesDropdown.classList.add('hidden'); + } + }); + + sourcesDropdown.querySelectorAll('input[data-source]').forEach(checkbox => { + checkbox.addEventListener('change', () => { + const source = checkbox.dataset.source; + if (checkbox.checked) { + enabledSources.add(source); + } else { + enabledSources.delete(source); + } + rebuildGraphFromSources(); + }); + }); + } } async function fetchExportData() { @@ -269,10 +299,12 @@ function addRelation(relations, rel, chain) { async function fetchData() { try { const exportData = await fetchExportData(); - const chains = exportData.data.indexed.all; - const relations = buildRelationsMap(chains); + allChains = exportData.data.indexed.all; - processGraphData(chains, relations); + const visibleChains = filterChainsBySources(allChains); + const visibleRelations = buildRelationsMap(visibleChains); + + processGraphData(visibleChains, visibleRelations); updateStats(); document.getElementById('loadingOverlay').classList.add('hidden'); renderGraph(); @@ -285,6 +317,28 @@ async function fetchData() { } } +function filterChainsBySources(chains) { + if (enabledSources.size === ALL_SOURCES.length) return chains; + return chains.filter(c => + c.sources && c.sources.some(s => enabledSources.has(s)) + ); +} + +function rebuildGraphFromSources() { + if (!allChains.length) return; + + const visibleChains = filterChainsBySources(allChains); + const visibleRelations = buildRelationsMap(visibleChains); + + processGraphData(visibleChains, visibleRelations); + applyFilters(); + updateStats(); + + if (myGraph) { + myGraph.graphData(filteredData); + } +} + function updateStats() { const total = graphData.nodes.length; const mainnets = graphData.nodes.filter(n => n.type === 'Mainnet').length; diff --git a/public/index.html b/public/index.html index 2a8071a..c9de8a6 100644 --- a/public/index.html +++ b/public/index.html @@ -57,6 +57,45 @@

Blockchain Networks

+ +
+ + +
+
Mainnet
diff --git a/public/style.css b/public/style.css index 95affb3..1f4487d 100644 --- a/public/style.css +++ b/public/style.css @@ -195,6 +195,101 @@ h1 { .filter-btn[data-filter="Testnet"].active { border-color: var(--color-testnet); color: var(--color-testnet); } .filter-btn[data-filter="Beacon"].active { border-color: var(--color-beacon); color: var(--color-beacon); } +/* ─── Sources Panel ─── */ +.sources-panel { + position: absolute; + bottom: 20px; + right: 20px; + padding: 0; + z-index: 20; +} + +.sources-toggle { + display: flex; + align-items: center; + gap: 6px; + background: none; + border: none; + color: var(--text-muted); + font-size: 0.75rem; + font-weight: 500; + cursor: pointer; + padding: 8px 14px; + transition: color 0.2s; + font-family: inherit; +} + +.sources-toggle:hover { + color: var(--text-main); +} + +.sources-dropdown { + padding: 8px 12px; + display: flex; + flex-direction: column; + gap: 6px; + border-top: 1px solid var(--panel-border); +} + +.sources-dropdown.hidden { + display: none; +} + +.source-toggle { + display: flex; + align-items: center; + gap: 8px; + font-size: 0.75rem; + color: var(--text-muted); + cursor: pointer; + padding: 4px 0; + transition: color 0.15s; + user-select: none; +} + +.source-toggle:hover { + color: var(--text-main); +} + +.source-toggle input[type="checkbox"] { + appearance: none; + -webkit-appearance: none; + width: 14px; + height: 14px; + border: 1.5px solid rgba(255, 255, 255, 0.2); + border-radius: 3px; + background: transparent; + cursor: pointer; + position: relative; + flex-shrink: 0; + transition: background 0.15s, border-color 0.15s; +} + +.source-toggle input[type="checkbox"]:checked { + background: var(--accent); + border-color: var(--accent); +} + +.source-toggle input[type="checkbox"]:checked::after { + content: ''; + position: absolute; + top: 1px; + left: 4px; + width: 4px; + height: 7px; + border: solid white; + border-width: 0 1.5px 1.5px 0; + transform: rotate(45deg); +} + +.source-dot { + width: 6px; + height: 6px; + border-radius: 50%; + display: inline-block; + flex-shrink: 0; +} + /* ─── Legend ─── */ .legend { position: absolute; @@ -565,6 +660,11 @@ canvas { outline: none; } gap: 8px; max-width: calc(100% - 24px); } + + .sources-panel { + bottom: 12px; + right: 12px; + } } @media (max-width: 480px) { diff --git a/src/store/indexer.js b/src/store/indexer.js index 41a7a8a..a16b7f4 100644 --- a/src/store/indexer.js +++ b/src/store/indexer.js @@ -56,7 +56,7 @@ function mergeBridges(chain, newBridges) { }); } -function processL2ParentRelation(chain, indexed) { +function processL2ParentRelation(chain, indexed, source = 'chains') { if (chain.parent?.type !== 'L2' || !chain.parent?.chain) return; const match = chain.parent.chain.match(/^eip155-(\d+)$/); @@ -80,14 +80,14 @@ function processL2ParentRelation(chain, indexed) { kind: 'l2Of', network: chain.parent.chain, chainId: parentChainId, - source: 'chains' + source }); } mergeBridges(indexed.byChainId[chainId], chain.parent.bridges); } -function processTestnetParentRelation(chain, indexed) { +function processTestnetParentRelation(chain, indexed, source = 'chains') { if (chain.parent?.type !== 'testnet' || !chain.parent?.chain) return; const match = chain.parent.chain.match(/^eip155-(\d+)$/); @@ -107,7 +107,7 @@ function processTestnetParentRelation(chain, indexed) { kind: 'testnetOf', network: chain.parent.chain, chainId: mainnetChainId, - source: 'chains' + source }); } } @@ -353,7 +353,12 @@ function indexChainlistSource(chainlist, indexed) { chainlist.forEach(chainData => { const chainId = chainData.chainId; if (chainId === undefined || chainId === null || Number.isNaN(Number(chainId))) return; - if (indexed.byChainId[chainId] && chainData.parent?.bridges) { + if (!indexed.byChainId[chainId]) return; + + processL2ParentRelation(chainData, indexed, 'chainlist'); + processTestnetParentRelation(chainData, indexed, 'chainlist'); + + if (chainData.parent?.bridges) { mergeBridges(indexed.byChainId[chainId], chainData.parent.bridges); } }); diff --git a/tests/unit/store/indexer-chainlist-relations.test.js b/tests/unit/store/indexer-chainlist-relations.test.js new file mode 100644 index 0000000..170bef3 --- /dev/null +++ b/tests/unit/store/indexer-chainlist-relations.test.js @@ -0,0 +1,148 @@ +import { describe, it, expect } from 'vitest'; +import { indexData } from '../../../src/store/indexer.js'; + +describe('indexer — chainlist parent relations', () => { + it('extracts L2 relations from chainlist parent field', () => { + const chains = [ + { chainId: 1, name: 'Ethereum Mainnet' } + ]; + + const chainlist = [ + { + chainId: 42161, + name: 'Arbitrum One', + parent: { + type: 'L2', + chain: 'eip155-1', + bridges: [{ url: 'https://bridge.arbitrum.io' }] + } + } + ]; + + const result = indexData(null, chainlist, chains, null); + + expect(result.byChainId[42161].tags).toContain('L2'); + expect(result.byChainId[42161].relations).toContainEqual( + expect.objectContaining({ + kind: 'l2Of', + chainId: 1, + source: 'chainlist' + }) + ); + expect(result.byChainId[42161].bridges).toBeDefined(); + expect(result.byChainId[42161].bridges).toHaveLength(1); + }); + + it('extracts testnet relations from chainlist parent field', () => { + const chains = [ + { chainId: 1, name: 'Ethereum Mainnet' } + ]; + + const chainlist = [ + { + chainId: 5, + name: 'Goerli', + parent: { + type: 'testnet', + chain: 'eip155-1' + } + } + ]; + + const result = indexData(null, chainlist, chains, null); + + expect(result.byChainId[5].relations).toContainEqual( + expect.objectContaining({ + kind: 'testnetOf', + chainId: 1, + source: 'chainlist' + }) + ); + }); + + it('does not duplicate relations when both chains.json and chainlist have same parent', () => { + const chains = [ + { chainId: 1, name: 'Ethereum Mainnet' }, + { + chainId: 10, + name: 'Optimism', + parent: { + type: 'L2', + chain: 'eip155-1' + } + } + ]; + + const chainlist = [ + { + chainId: 10, + name: 'Optimism', + parent: { + type: 'L2', + chain: 'eip155-1' + } + } + ]; + + const result = indexData(null, chainlist, chains, null); + + const l2Relations = result.byChainId[10].relations.filter( + r => r.kind === 'l2Of' && r.chainId === 1 + ); + expect(l2Relations).toHaveLength(1); + }); + + it('creates reverse parentOf relations for chainlist-sourced L2 relations', () => { + const chains = [ + { chainId: 1, name: 'Ethereum Mainnet' } + ]; + + const chainlist = [ + { + chainId: 42161, + name: 'Arbitrum One', + parent: { + type: 'L2', + chain: 'eip155-1' + } + } + ]; + + const result = indexData(null, chainlist, chains, null); + + expect(result.byChainId[1].relations).toContainEqual( + expect.objectContaining({ + kind: 'parentOf', + chainId: 42161, + source: 'chainlist' + }) + ); + }); + + it('creates reverse mainnetOf relations for chainlist-sourced testnet relations', () => { + const chains = [ + { chainId: 1, name: 'Ethereum Mainnet' } + ]; + + const chainlist = [ + { + chainId: 5, + name: 'Goerli', + parent: { + type: 'testnet', + chain: 'eip155-1' + } + } + ]; + + const result = indexData(null, chainlist, chains, null); + + expect(result.byChainId[1].relations).toContainEqual( + expect.objectContaining({ + kind: 'mainnetOf', + chainId: 5, + source: 'chainlist' + }) + ); + }); +});