Skip to content

docs(arkdock): Arkdock documentation#1

Open
kmchen-arklex wants to merge 26 commits into
mainfrom
feat/arkdock-documentation
Open

docs(arkdock): Arkdock documentation#1
kmchen-arklex wants to merge 26 commits into
mainfrom
feat/arkdock-documentation

Conversation

@kmchen-arklex

@kmchen-arklex kmchen-arklex commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds complete Mintlify documentation for Arkdock covering all core features: agents, scenarios, simulations, evaluations, annotations, knowledge, metrics, and settings
  • Updates docs.json with Arkdock branding, navigation groups, navbar, footer socials, and contextual AI tool options
  • Includes 7 product screenshots and a quickstart guide covering signup through first evaluation result

Preview

https://arklex-06dfaf56-feat-arkdock-documentation.mintlify.app/overview

Test plan

  • Preview locally with mint dev and verify all navigation links resolve
  • Confirm social links and sign-in URL in docs.json are correct for production
  • Check images render on all pages that reference them (quickstart, evaluations, settings)

Adds complete Mintlify docs for Arkdock covering: overview with workflow
diagram, quickstart, agents, scenarios, simulations, evaluations,
annotations, knowledge, metrics, and settings. Updates docs.json with
Arkdock branding, navigation, navbar, footer, and contextual AI options.
Includes agent-ready components, ai-tools setup guides, api-reference
template, and essentials reference pages from the Mintlify starter kit.
@mintlify

mintlify Bot commented Jun 4, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
arklex 🟡 Building Jun 4, 2026, 3:29 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@mintlify

mintlify Bot commented Jun 4, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
arkdock 🟡 Building Jun 4, 2026, 3:29 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@kmchen-arklex kmchen-arklex changed the title docs(arkdock): add full product documentation docs(arkdock): Arkdock documentation Jun 4, 2026
@kmchen-arklex kmchen-arklex marked this pull request as draft June 4, 2026 12:37
@xinyang-articulate

Copy link
Copy Markdown
  1. evaluations.mdx:39, quickstart.mdx:102, overview.mdx:129

    Docs say users can choose the evaluation judge provider/model. The current UI only supports name, simulation, and metrics. See frontend/apps/arkdock-web/src/pages/EvaluationsPage/NewEvaluationSheet.jsx and frontend/apps/arkdock-web/src/services/evaluation/index.js.

  2. overview.mdx:93, overview.mdx:132

    Docs mention "Also run an evaluation" during simulation setup. That UI is currently commented out/TODO in frontend/apps/arkdock-web/src/pages/SimulationsPage/NewSimulationSheet.jsx, so this should not be documented as available.

  3. agents.mdx:113, agents.mdx:123

    Agent deletion behavior is wrong. Backend rejects deleting an agent if simulations/evaluations still reference it. See backend/base/arkdock/agent.go and backend/controllers/arkdock/agent.go.

  4. knowledge.mdx:53, knowledge.mdx:89

    Website ingestion docs are outdated. The UI has Add Domain for crawling multiple links and Add Link for one website/YouTube URL. See frontend/apps/arkdock-web/src/pages/KnowledgePage/index.jsx and chatbots/arkdock-python/service/routers/knowledge.py.

  5. settings.mdx:14, settings.mdx:28, quickstart.mdx:140

    Settings docs use fields/actions that are not in the UI. Profile has photo/name/company/email/role, not password management. Roles are Owner and Member, not Admin. See frontend/apps/arkdock-web/src/pages/SettingsPage/ and frontend/apps/arkdock-web/src/constants/userRole.js.

  6. evaluations.mdx:173, annotations.mdx:31

    Annotation save behavior is inconsistent. One page says auto-save, the other says click Save Annotation. The UI uses an explicit Save Annotation button. See frontend/apps/arkdock-web/src/pages/EvaluationDetailPage/AnnotationsTab.jsx.

  7. annotations.mdx:57

    Resolution rules do not match the backend. Backend uses strict majority and conflict handling, not averaging/spread thresholds. See backend/base/arkdock/annotation.go and frontend/apps/arkdock-web/src/pages/EvaluationDetailPage/ResolveDialog.jsx.

  8. agents.mdx:12, simulations.mdx:14, evaluations.mdx:61

    Several table/card fields do not match the UI. Agent cards do not show average failure rate, simulation tables do not show completion rate, and evaluation detail stats do not show average score/status counts. See frontend/apps/arkdock-web/src/pages/AgentsPage/, frontend/apps/arkdock-web/src/components/SimulationTable/, and frontend/apps/arkdock-web/src/pages/EvaluationDetailPage/.

  9. metrics.mdx:38, metrics.mdx:58

    Custom metric docs are too simplified. The form requires system prompt, user prompt template, scope, type, and type-specific config. There is no metric-level color in the create payload. See frontend/apps/arkdock-web/src/pages/MetricsPage/index.jsx and frontend/apps/arkdock-web/src/services/metric/index.js.

  10. scenarios.mdx:20

    Scenario creation flow is described incorrectly. The UI starts with a picker, and conversation-history generation is feature-flagged. See frontend/apps/arkdock-web/src/pages/ScenariosPage/AddScenariosPicker.jsx and frontend/apps/arkdock-web/src/pages/ScenariosPage/index.jsx.

  11. index.mdx, README.md, AGENTS.md, agent-ready/, essentials/

    Mintlify starter content is still present. These pages are not in the nav, but they may still be served or indexed. Remove them or replace them with Arkdock content before publishing.

- Remove LLM judge provider/model selector from evaluations, quickstart, and
  overview — the new evaluation form only has name, simulation, and metrics
- Remove "Also run an evaluation" from simulation docs — that UI is commented
  out pending backend support
- Fix agent deletion: backend rejects deletion when simulations/evaluations
  still reference the agent; update card fields to show simulation + evaluation
  counts instead of average failure rate
- Fix knowledge website section: document Add Domain (multi-page crawl) and
  Add Link (single URL/YouTube) buttons instead of the removed Add Website
- Fix settings profile fields: photo, name, company, read-only email and role;
  no password management; rename Admin role to Owner throughout
- Fix annotation save behavior: explicit Save Annotation button, not auto-save
- Fix resolution rules: strict majority vote with admin override for ties,
  not averaging or spread thresholds
- Remove Completion Rate column from simulation table (not in actual UI)
- Expand custom metric form docs to include type, scope, system prompt, user
  prompt template, and type-specific config; remove Color as an API field
- Add picker-first flow description to scenario creation; gate Conversation
  History mode behind feature flag note
- Remove all Mintlify starter content (index.mdx, agent-ready/, essentials/,
  ai-tools/, api-reference/, development.mdx); replace with Arkdock content
@kmchen-arklex

Copy link
Copy Markdown
Contributor Author

Hi @xinyang-articulate , all comments are fixed. Thanks~~

@kmchen-arklex kmchen-arklex marked this pull request as ready for review June 10, 2026 13:29
Comment thread overview.mdx Outdated
Comment thread overview.mdx Outdated
Comment thread agents.mdx Outdated
Comment thread simulations.mdx Outdated
@jackytungArticulateAI

Copy link
Copy Markdown
  1. Overview lacks a pain-point hook
    The first sentence describes what Arkdock is, but readers need to feel why the problem is hard before they care about the solution.

Consider opening with the pain point, e.g. "You've built an AI agent. How do you know it actually works for real users before you ship it?", then introducing Arkdock as the answer.

  1. The Mermaid diagram is data-flow oriented, not user-journey oriented
    The current diagram maps data inputs/outputs, which works for engineers but not for PMs or QA. Consider adding a simpler linear flow at the top of Quickstart showing the five actions in order: Connect Agent → Create Scenarios → Run Simulation → Run Evaluation → Review & Annotate. One glance should tell a reader what they need to do and in what order.

  2. Annotation Calibration deserves more prominence
    This is one of Arkdock's most differentiated features (the human-annotation + LLM-judge calibration loop), but it only appears deep in the Evaluations page.
    The "What makes it different?" section in Overview is the right place for this.
    A simple diagram of the feedback loop would make it land: LLM judge scores → human flags disagreements → calibration surfaces drift → team refines metric prompt → judge improves.

  3. No use-case entry point (good to have)
    Every page is a feature reference, but there's no page (or section) that starts from "here's a scenario you might be in." Even a short list of two or three common use cases in Overview, e.g. regression testing before deploying a new agent version, A/B comparing two system prompts, would help readers immediately self-identify and understand why they're reading.

- Add pain-point hook to Overview opening paragraph
- Add common use cases section to Overview (regression testing, A/B prompts,
  baseline establishment)
- Simplify Mermaid flowchart node labels for a cleaner display; mark Knowledge
  Base as optional with a dashed edge
- Add human-annotation calibration loop diagram to the "Human-in-the-loop
  calibration" section in Overview
- Add linear 5-step user-journey diagram to the top of Quickstart
- Add hex color swatches to status tables in agents.mdx and simulations.mdx
- evaluations.mdx: fix FAQ, annotations input range, Final Score column, and
  Goal Completion cross-reference (0-1 rather than 1-5)
- overview.mdx: fix custom metrics description
- metrics.mdx: fix Goal Completion cross-reference, type-specific config
  example, and example rubric (remove Score 0, use Score 1 as lowest)
@kmchen-arklex

Copy link
Copy Markdown
Contributor Author

Add a contact & support page

Moved the full Metric Alignment section (starting a run, phases, statuses,
reviewing results, cancelling, history, and alignment-specific FAQ) out of
metrics.mdx into a new metric-alignment.mdx page. Replaced the section in
metrics.mdx with a short two-sentence link-out. Added an "Advanced" nav
group in docs.json containing the new page.
- Updated quickstart.mdx
- Updated overview.mdx
- Created images/overview-diagram.png
- Created images/calibration-loop.png
- Created faq.mdx
- Updated docs.json

Mintlify-Source: dashboard-editor
- Updated quickstart.mdx

Mintlify-Source: dashboard-editor
- Updated quickstart.mdx

Mintlify-Source: dashboard-editor
New support.mdx covers email contact, what to include in a request,
quick links to relevant docs pages, and billing/account guidance.
Added a "Help" nav group in docs.json containing the new page.
Replaced the verbose multi-section layout with the same structure as
docs.arklex.ai/v0.3.x/contact-email: a brief Get Help paragraph with
an email link and a two-card Connect With Us section (GitHub, LinkedIn).
Removed the onboarding checklist note, nested sub-steps, roles table,
duplicate step entries, billing step, and verbose prose. Rewrote as a
single Steps block covering account creation through reviewing results,
matching the lean reference style at docs.arklex.ai/v0.3.x/quickstart.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants