Skip to content

destinations.md

Latest commit

 

History

History
202 lines (158 loc) · 13.7 KB

destinations.md

File metadata and controls

202 lines (158 loc) · 13.7 KB
Raw

Library Destinations Architecture

Audplexus pushes each downloaded audiobook to one or more library destinations simultaneously. This doc covers the data model, runtime contract, and code path so a reviewer can grok the multi-destination feature without reading every commit.

High-level flow

Audible              ┌──────────────────────────────────────────────┐
   │                 │  Audplexus                                   │
   │ aax/aaxc        │                                              │
   ▼                 │   Download   ──▶  Decrypt   ──▶  Organize   │
┌──────┐             │      │              │                │       │
│ APIs │             │      ▼              ▼                ▼       │
└──────┘             │  audible.db    /downloads/       /audiobooks/│
                     │                                    │         │
                     │                                    ▼         │
                     │                        DestinationManager.FanOut
                     │                              │  │  │         │
                     │                       ┌──────┘  │  └──────┐  │
                     │                       ▼         ▼         ▼  │
                     │                  TriggerScan TriggerScan TriggerScan
                     │                  ItemMatch   ItemMatch   ItemMatch
                     │                  SeriesGroup SeriesGroup SeriesGroup
                     │                  FranchiseTag(if Cap)              │
                     └──────┬─────────────┬──────────┬──────────┬────────┘
                            ▼             ▼          ▼          ▼
                       Plex API      Emby API   Jellyfin   Audiobookshelf
                       (token)       (api key)  (MediaBrwsr)(Bearer)

Data model

Two tables, both under 005_library_destinations migration.

library_destinations

One row per configured destination. Multiple rows of the same type are allowed.

Column Notes
id UUIDv4 generated by the application. Stable across edits.
display_name Free-form, shown on cards (e.g. "Living Room Plex").
type plex | emby | jellyfin | abs
enabled Soft toggle. Disabled rows are skipped by fan-out and reconcile.
url, api_key, plex_token, plex_section_id, library_id Per-type config columns (some unused per type). CHECK constraint enforces the required subset.
audiobook_path, destination_path Per-destination path translation source/target. Optional — falls back to global libraryDir + cached server-side path when unset.
last_health_check_at / last_health_check_ok / last_health_check_err Populated by Test Connection button + every successful or failed scan/reconcile.

api_key and plex_token are sensitive; they have json:"-" for marshal redaction AND a custom String() Stringer so fmt.Printf("%v", row) shows <redacted> not the raw token.

book_library_destinations

One row per (book, destination) pair. Carries the destination-side identity and a state machine.

Column Notes
book_id, destination_id Composite PK. ON DELETE CASCADE both ways.
server_item_id, server_item_title Destination's internal handle to the book (Plex ratingKey, Emby/Jellyfin ItemId, ABS UUID).
sync_state pending | syncing | synced | failed | orphaned | removed_from_destination
last_attempted_at, last_succeeded_at, last_error, attempt_count Independent of sync_state so an operator can see "tried recently but failed" cleanly.
disabled_reason Set when admin chose to stop retrying despite failed.
per_op_outcomes JSON {operation: {status, at, detail, duration_ms}}. Open-set per backend (the mediaserver.Outcome.Operation field).

Runtime contract

mediaserver.Backend.OnBookOrganized is the per-book post-organize entry point. MUST be synchronous, context-bound, idempotent, and return one Outcome per logical operation (scan trigger, item match, series grouping, etc.).

type Outcome struct {
    Operation     string         // "scan_trigger", "item_match", "series_grouping", "franchise_tag", ...
    Status        OutcomeStatus  // succeeded | skipped_existing | unsupported | failed | deferred | skipped_not_configured
    Detail        string         // human-readable
    Err           error          // populated only on Failed
    ServerItemID  string         // populated when item-match succeeded
    DurationMs    int64
}

Critical rule: backends never silently no-op. A backend that doesn't support an operation returns OutcomeUnsupported (with errors.Is(o.Err, ErrUnsupported)); a backend that's mid-config returns OutcomeSkippedNotConfigured. Capabilities are advertised via Backend.Capabilities() for UI affordance hiding (e.g. don't show "Franchise tagging" toggle on Plex destinations) but the runtime contract is the typed Outcome.

This was the central fix in PR-0: the old fire-and-forget pattern had TriggerScanForBook / EnsureBookInSeriesCollection spawning goroutines and discarding errors, which let the pipeline mark downloads Complete before the media-server work even finished — and silently when it failed.

Fan-out

DestinationManager.FanOut(ctx, OrganizedBook) iterates library_destinations WHERE enabled=1, builds a Backend per row (with WithDestination(row) binding so per-destination URL/api_key/library_id wins), and runs each OnBookOrganized concurrently:

  • Bounded by maxConcurrency (default 3, configurable). Prevents N parallel image uploads against slow home servers.
  • Per-destination context.WithTimeout(ctx, 2*time.Minute). One stuck destination doesn't hold up the others.
  • Each destination's outcomes are recorded into book_library_destinations.per_op_outcomes via recordOutcomes. The recording uses the OUTER ctx (not the per-destination ctx) on purpose — we want to persist the failure outcome even if the per-destination timeout fired.

Failure isolation: one destination's error never affects the others. The pipeline blocks on wg.Wait() so the download isn't marked Complete until every destination's OnBookOrganized has returned (or its 2-min timeout fired).

Sync flow (Library Scan + Reconcile)

DestinationManager.TriggerScanAll and ReconcileAll are the multi-dest analogs of the legacy single-backend TriggerLibraryScan and ReconcileLibrary. Same fan-out shape, same bounded concurrency, same per-destination timeouts (60s for scans, 10min for reconcile).

TriggerScanAll returns max of the per-destination item counts, NOT sum — in single-source-of-truth setups (3 destinations pointing at the same /audiobooks mount) summing would 3x the count and read as bogus on the dashboard. Per-destination truth lives on the dashboard cards.

Capabilities map

Each backend declares its Capabilities():

Capability Plex Emby Jellyfin ABS
trigger_scan ✔ ✔ ✔ ✔
per_item_refresh ✔ ✔ ✔ ✔
series_grouping ✔ ✔ ✔ ✔ (implicit via tags)
franchise_tag ✔ ✔
image_upload ✔ ✔
item_count ✔ ✔ ✔ ✔
author_images ✔ ✔
boxset_covers ✔ ✔

UI uses these to hide affordances that don't apply per-destination. The runtime contract still reports OutcomeUnsupported if a missing-capability operation is invoked — never a silent no-op.

Backend-specific notes

Plex

  • Auth: token via X-Plex-Token query param.
  • Section scan: POST /library/sections/{id}/refresh. With ?path= for targeted folder.
  • Section path comes from /library/sections/{id} Locations array, cached in settings.

Emby

  • Auth: api_key=... query param OR X-Emby-Token header.
  • Library refresh: POST /Library/Refresh. Per-folder via POST /Items/{id}/Refresh?Recursive=true.
  • BoxSet collections: POST /Collections + POST /Collections/{id}/Items.
  • Tag write: POST /Items/{id} with full BaseItemDto and LockedFields: ["Tags"].

Jellyfin

  • Auth: Authorization: MediaBrowser Token="...", Client="Audplexus", Device=.... Legacy X-Emby-Token works on 10.x but dies in Jellyfin 12.0 (PR jellyfin#13306). New code uses the proper header from day one.
  • Item filter: IncludeItemTypes=AudioBook. Jellyfin has first-class BaseItemKind.AudioBook; Emby returns books as both Audio and MusicAlbum (the filter in this code differs accordingly).
  • Image upload: concrete Content-Type: image/jpeg required — Jellyfin's ImageController rejects image/* and image/jpg with 400.
  • LockedFields: wire-compatible ["Tags"] but the server side is an enum, so unknown strings silently no-op.

Audiobookshelf

  • Auth: Authorization: Bearer <api_key>. Admin-scope key required for scan endpoints (post-v2.26.0 JWT rewrite, 2025-07).
  • Scan: POST /api/libraries/{id}/scan.
  • Item match: GET /api/libraries/{id}/search?q=<ASIN>. ABS has no native ASIN filter; we verify media.metadata.asin client-side.
  • Series: ABS auto-detects from embedded series and series-part ID3-style atoms (the Audiobook-rich tag profile writes these). Falls back to PATCH /api/items/{id}/media with metadata.series=[{name,sequence}] if tags are absent.
  • Folder watcher (chokidar) is on by default in ABS; the scan trigger is belt-and-suspenders — idempotent so calling both is safe.

Test Connection probe

POST /destinations/test (new row) and POST /destinations/:id/test (saved row, secrets carried over) build a Backend with the form values, call LibraryItemCount with a 10s ctx timeout, and return an HTML fragment for HTMX swap. On success or failure the row's last_health_check_* columns are updated so the dashboard's "Healthy/Failed" badge reflects the most recent test.

What this design DOESN'T do (intentionally)

  • No retry queue. Failed operations on a destination get recorded; the next reconcile or download triggers them again. Caller decides retry policy.
  • No transactional consistency across destinations. Each destination is independent; a partial-success state is captured in book_library_destinations and surfaced in Diagnostics.
  • No real-time WebSocket updates from destination servers. Health is poll-based (Test Connection on demand, scan/reconcile during sync).
  • No load shedding under heavy queue. The bounded-concurrency budget prevents pile-up at the destination side; if Audplexus itself is overloaded that's the download pipeline's problem, not the destination layer's.

File map

internal/mediaserver/
    mediaserver.go         Backend interface, Type constants, factory
    outcome.go             Outcome, OutcomeStatus, helper constructors,
                           Capability constants
    plex.go                PlexBackend (token + section scan + collections)
    emby.go                EmbyBackend (api key + BoxSet + tags + images)
    jellyfin.go            JellyfinBackend (MediaBrowser auth + AudioBook
                           filter + Content-Type strict)
    abs.go                 ABSBackend (Bearer + ASIN match + native series)
    match.go               normalizeTitle (HTML-decode, smart-quote norm,
                           "&"/"and" equiv, leading-article strip)
    franchise.go           franchiseFromSeries ("Star Wars: Foundation
                           Trilogy" -> "Star Wars")
    path.go                translateScanPath (local -> server-visible)

internal/library/
    destinations.go        DestinationManager: ListEnabled, FanOut,
                           TriggerScanAll, ReconcileAll, BuildBackend
    firstboot_destinations.go  SynthesizeLibraryDestinationsIfEmpty,
                           hasPlexLegacyConfig, hasEmbyLegacyConfig
                           (handles upgrade from v0.2.x without explicit
                           MEDIA_SERVER set)
    pipeline_stages.go     handleProcessStage calls FanOut BEFORE
                           marking download Complete

internal/database/
    migrations/005_library_destinations.up.sql  schema (sqlite)
    migrations_postgres/005_library_destinations.up.sql  schema (postgres)
    library_destinations_sqlite.go    CRUD (sqlite)
    library_destinations_postgres.go  CRUD (postgres)
    models.go              LibraryDestination, BookDestination structs
                           with secret-redacting Stringer

internal/web/
    destinations_handlers.go     Settings UI handlers + Test Connection
    server.go                    Routes + DestinationSummaries view-model
                                 + recordDestinationHealth helper
    templates/destinations_*.html  Type picker, type-specific edit form,
                                   delete confirm
    templates/dashboard_summary.html  Per-destination cards (replaces the
                                      single-active-backend stat-cards)

See also

  • WCAG-aware UI patterns documented inline in templates (cards as <ul role="list">, aria-live="polite" on the live dashboard list, aria-busy on the Test button, <details> collapsible help text per field).
  • Codex pre-PR review captured 3 real bugs — default-Plex upgrade synthesis (P1), buildDecryptArgs muxer flag (P2), per-destination scan paths (P2). All fixed; regression tests in firstboot_destinations_test.go and tag_test.go.