Add client media upload and cache pipeline by Eoic · Pull Request #16 · PapyrusReader/client

Eoic · 2026-06-27T15:45:51Z

Summary

Add authenticated media API methods for usage, upload, download, and delete
Add book file/cover media IDs to the local sync model and upload mapper
Queue media uploads after signed-in imports and retry after sync/upload completion
Show real server file-storage usage in Storage & sync
Lazy-load private uploaded covers and cache downloaded book files in platform-local storage

Depends on

Add media storage pipeline server#3

Verification

flutter analyze
flutter test

…ndling - Implemented BookDownloadService for downloading book files. - Added retry mechanism for failed media uploads in MediaUploadQueue. - Enhanced BookDetailsPage to include download option in context menu. - Created deleteBookWithMediaCleanup service to handle book deletion and media upload cleanup. - Updated ProfilePage to display failed media uploads and provide retry option. - Added tests for new download and delete functionalities, ensuring proper behavior.

Copilot

Pull request overview

This PR adds an authenticated media pipeline to the Papyrus Flutter client: scoped media upload queueing (per server profile + user), authenticated media API integration (usage/upload/download/delete), and a stable local cover rendering path that reuses Flutter’s ImageCache while persisting bytes in platform-local storage (native filesystem / web OPFS).

Changes:

Add scoped authenticated media support (models, API client/repository methods, upload queue + processing hooks, and storage usage UI).
Implement private cover rendering via a stable ImageProvider, plus consistent book-cover surfaces that support coverMediaId.
Add local book-file caching + download flows and deterministic import/delete commit/cleanup services with extensive tests.

Reviewed changes

Copilot reviewed 83 out of 84 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
docs/superpowers/specs/2026-07-11-media-storage-hardening-design.md	New design spec for scoped media storage, upload queue, and cover caching behavior.
docs/superpowers/specs/2026-07-11-cover-image-cache-design.md	New design spec for stable `ImageProvider` identity for local covers.
docs/superpowers/plans/2026-07-11-stable-local-cover-image-cache.md	Implementation plan for stable cover cache provider + widget integration.
app/web/book_worker.js	Add OPFS handlers for storing book files and cover CRUD/promotion/clearing with scoped locking.
app/pubspec.yaml	Add `http_parser` as a direct dependency for multipart content types.
app/pubspec.lock	Lockfile update for `http_parser` direct dependency.
app/lib/widgets/topics/manage_topics_sheet.dart	Use `PrivateBookCover` for consistent private/public cover rendering.
app/lib/widgets/shelves/shelf_card.dart	Use `PrivateBookCover` in shelf mosaic previews (supports private covers).
app/lib/widgets/shelves/move_to_shelf_sheet.dart	Use `PrivateBookCover` for cover rendering in the move-to-shelf sheet.
app/lib/widgets/shared/book_group_header.dart	Add `bookId`/`coverMediaId` plumbing and render via `PrivateBookCover`.
app/lib/widgets/library/book_list_item.dart	Replace network-image rendering with `PrivateBookCover`.
app/lib/widgets/library/book_card.dart	Replace network-image rendering with `PrivateBookCover`.
app/lib/widgets/dashboard/recently_added_section.dart	Render dashboard covers via `PrivateBookCover`.
app/lib/widgets/dashboard/continue_reading_card.dart	Render the continue-reading cover via `PrivateBookCover`.
app/lib/widgets/context_menu/book_context_menu.dart	Add Download action + align labels (Download/Delete) and cover rendering.
app/lib/widgets/book/private_book_cover.dart	New unified cover widget: public URL / inline data / private cached media / pending / guest.
app/lib/widgets/book_details/book_header.dart	Pass `bookId` + `coverMediaId` through to details header cover.
app/lib/widgets/book_details/book_cover_image.dart	Route cover rendering through `PrivateBookCover`.
app/lib/widgets/add_book/import_book_sheet.dart	Move import commit to a deterministic commit boundary + enqueue scoped uploads.
app/lib/utils/bulk_book_actions.dart	Bulk delete now removes pending uploads and best-effort local media cleanup.
app/lib/utils/book_actions.dart	Implement Download + Delete actions using media cache + cleanup services.
app/lib/services/book_import_service.dart	Web worker integration extended for OPFS book caching + scoped cover storage requests.
app/lib/services/book_import_service_stub.dart	Native implementation for book file storage + scoped cover store/read/delete/promote/clear.
app/lib/services/book_import_commit_service.dart	New service to commit imports with ordered persistence + compensation.
app/lib/services/book_download_service.dart	New cross-platform book-file download service + safe filenames.
app/lib/services/book_download_service_platform_web.dart	Web download implementation via Blob + anchor download.
app/lib/services/book_download_service_platform_io.dart	IO download implementation via FilePicker save + file write when needed.
app/lib/services/book_delete_cleanup_service.dart	New helper to delete a book with pending-uploads + local media cleanup.
app/lib/providers/sync_settings_provider.dart	Support quota override in storage label formatting.
app/lib/providers/notes_provider.dart	Expose `coverMediaId` for grouped book headers.
app/lib/providers/bookmarks_provider.dart	Expose `coverMediaId` for grouped book headers.
app/lib/providers/annotations_provider.dart	Expose `coverMediaId` for grouped book headers.
app/lib/providers/auth_provider.dart	Expose authenticated `downloadMedia` method.
app/lib/powersync/sync_profile_switch_queue.dart	New serializing queue for profile switches that preserves latest request.
app/lib/powersync/storage_sync_controller.dart	Display server media usage + failed media upload count/label.
app/lib/powersync/powersync_book_mapper.dart	Map new `file_media_id` / `cover_media_id` fields through PowerSync.
app/lib/powersync/papyrus_schema.dart	Add `file_media_id` / `cover_media_id` columns to schema.
app/lib/powersync/papyrus_powersync_connector.dart	Add `onUploadComplete` hook to trigger media-upload processing.
app/lib/pages/profile_page.dart	Surface media usage + failed upload retry in Storage & sync UI; clear cover cache on clear-auth-cache.
app/lib/pages/notes_page.dart	Pass `bookId` + `coverMediaId` to `BookGroupHeader`.
app/lib/pages/bookmarks_page.dart	Pass `bookId` + `coverMediaId` to `BookGroupHeader`.
app/lib/pages/annotations_page.dart	Pass `bookId` + `coverMediaId` to `BookGroupHeader`.
app/lib/pages/book_details_page.dart	Add Download/Delete actions; ensure book file is cached before reading when needed.
app/lib/models/shelf.dart	Extend `CoverPreview` to include `bookId` and `mediaId`.
app/lib/models/book.dart	Add `fileMediaId` / `coverMediaId` fields to book model + JSON/copyWith.
app/lib/media/media_upload_queue.dart	New scoped, persisted media upload queue with single-flight processing and retries.
app/lib/media/media_storage_scope.dart	New media scope identity (profile + user) with safe path parts.
app/lib/media/media_models.dart	New media models: kinds, payload, asset, usage, and upload exception.
app/lib/media/media_cache_service.dart	New lazy download + validation + caching for private book files and covers.
app/lib/media/local_cover_image_provider.dart	New filesystem-backed `ImageProvider` key for stable Flutter image cache identity.
app/lib/media/cover_upload_persistence.dart	New helper to promote pending covers after successful upload (best-effort).
app/lib/media/cover_storage_bucket.dart	New enum describing cover storage buckets (cached/pending/guest books).
app/lib/main.dart	Wire up scope activation, media queue processing, media usage refresh, and services/providers.
app/lib/data/data_store.dart	Add repository-bound awaitable mutations + preserve local cover media id through transient sync snapshots.
app/lib/auth/auth_repository.dart	Add authenticated media methods (usage/upload/download/delete).
app/lib/auth/auth_api_client.dart	Implement `/v1/media` client calls: usage, multipart upload, download, delete.
app/test/widgets/shared/book_group_header_test.dart	New test verifying `BookGroupHeader` forwards bookId to cover renderer.
app/test/widgets/library/book_list_item_test.dart	Add test verifying bookId forwarding to `PrivateBookCover`.
app/test/widgets/library/book_card_test.dart	Add test verifying bookId forwarding to `PrivateBookCover`.
app/test/widgets/context_menu/book_context_menu_test.dart	New test for Download/Delete label/order and download callback.
app/test/widgets/book_details/book_cover_image_test.dart	New test verifying `BookCoverImage` forwards bookId to `PrivateBookCover`.
app/test/services/book_import_service_test.dart	Add tests for storing downloaded book bytes with correct extension + replacement.
app/test/services/book_import_commit_service_test.dart	New tests for import commit ordering, scoping, compensation, and context guards.
app/test/services/book_delete_cleanup_service_test.dart	New tests for delete cleanup ordering and coverage of cover representations.
app/test/powersync/sync_profile_switch_queue_test.dart	New tests for profile-switch queue serialization and error handling.
app/test/powersync/storage_sync_controller_test.dart	New test for failed media upload label exposure in UI controller.
app/test/powersync/powersync_book_mapper_test.dart	Extend mapper tests for new media-id fields.
app/test/pages/profile_storage_sync_test.dart	Update profile storage/sync page harness to provide media queue and adjust pumps.
app/test/pages/book_details_delete_test.dart	New widget tests for delete cleanup + download behavior + menu ordering.
app/test/media/media_storage_scope_test.dart	New tests for scope identity, equality, and path safety.
app/test/media/media_profile_switch_contract_test.dart	New contract tests that assert key lifecycle wiring in `main.dart`/import sheet.
app/test/media/media_cache_service_test.dart	New tests for lazy cache/download behavior + cover download coalescing.
app/test/media/local_cover_image_provider_test.dart	New tests for stable provider identity reuse + eviction semantics.
app/test/media/cover_upload_persistence_test.dart	New tests for upload/promotion best-effort behavior.
app/test/data/book_repository_data_store_test.dart	Extend DataStore tests for new async mutation semantics and media-id retention.
app/test/auth/auth_api_client_test.dart	Add tests for media usage, download, and multipart upload request mapping.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+  void _onMenuAction(String action) async {
    switch (action) {
+      case 'download':
+        await _onDownloadBookFile();
      case 'delete':
-        // TODO: Confirm and delete
-        ScaffoldMessenger.of(context).showSnackBar(const SnackBar(content: Text('Delete functionality coming soon')));
+        await _confirmDeleteBook();
    }


+    final actualBytes = bytes.offsetInBytes == 0 && bytes.lengthInBytes == bytes.buffer.lengthInBytes
+        ? bytes
+        : Uint8List.fromList(bytes);
+    final jsBuffer = actualBytes.buffer.toJS;
+    final message = JSObject();
+    message['type'] = 'storeFile'.toJS;
+    message['format'] = normalizedExtension.toJS;
+    message['bookId'] = bookId.toJS;
+    message['fileData'] = jsBuffer;
+
+    worker.postMessage(message, [jsBuffer].toJS);


+    final mergedBooks = books
+        .map((book) {
+          final localBook = _books[book.id];
+          if (book.coverMediaId == null && localBook?.coverMediaId != null) {
+            // PowerSync can briefly emit the downloaded server row before its
+            // pending local media-reference update is acknowledged. Keep the
+            // established local reference through that transient null snapshot.
+            return book.copyWith(coverMediaId: localBook!.coverMediaId);
+          }
+          return book;
+        })
+        .toList(growable: false);


          onStatusChange?.call(ReadingStatus.inProgress);
        case 'finished':
          onStatusChange?.call(ReadingStatus.completed);
+        case 'download':
+          onDownload?.call();
        case 'delete':


…istent actions

… handle loading state

…te related providers for consistent state management

Add client media upload and cache pipeline

6ffe589

Eoic mentioned this pull request Jun 27, 2026

Add media storage pipeline PapyrusReader/server#3

Merged

Eoic added 28 commits July 5, 2026 01:16

docs: design media storage hardening

8517b6a

docs: plan media storage hardening

b0722db

test: align book menu labels

1df18f2

feat: define scoped media identity

6cdc91c

feat: persist scoped covers in local files

7ba5163

feat: render private covers from persistent cache

2a93ee1

fix: scope and serialize media uploads

98aae0e

fix: process media immediately after enqueue

ee0c80e

fix: clean scoped cover files

422efc7

style: format media scope test

7689ad7

fix: close media lifecycle races

f0b3586

docs: define local-first cover lifecycle

7a5d306

docs: plan local-first cover lifecycle

c0628ad

feat: add pending and guest cover storage

7935715

fix: make cover promotion race safe

1ad0281

fix: coordinate cover writes across workers

112c3e9

fix: queue cover file references instead of bytes

7ed2e3e

fix: wire pending cover queue readers

13f7618

fix: isolate queued media read failures

ec56e37

feat: persist imported covers outside metadata

dae456d

fix: make import cover commit atomic

7722028

fix: await import metadata compensation

fb1e178

fix: preserve synchronous data store mutations

ebfc660

fix: bind imports to captured library repository

b91137f

feat: render local covers by book id

fed071d

fix: reload local covers across library contexts

3f8c142

fix: subscribe after local cover source transition

cd2dde8

Eoic added 11 commits July 11, 2026 20:33

feat: promote uploaded covers into local cache

a726987

fix: keep cover promotion diagnostics non-fatal

1115150

fix: clean pending and guest covers on delete

196438c

fix: preserve uploaded cover ids across stale sync

4f8870d

docs: design stable local cover image cache

02999e7

docs: plan stable local cover image cache

3f79205

feat: add stable local cover image provider

4630bde

fix: reuse decoded covers across page transitions

02965dd

test: cover pending and guest page cache reuse

21d254a

fix: evict decoded covers after local removal

8b53613

fix: preserve pending cover cache on no-op promotion

795bfba

Eoic marked this pull request as ready for review July 12, 2026 12:53

Copilot AI review requested due to automatic review settings July 12, 2026 12:53

Copilot started reviewing on behalf of Eoic July 12, 2026 12:54 View session

feat: enhance cover image handling with book and media IDs

1fa596f

Copilot AI reviewed Jul 12, 2026

View reviewed changes

Eoic added 7 commits July 12, 2026 15:59

refactor: rename PrivateBookCover to CoverImage and update references

e7788dd

feat: implement hot restart cleanup and enhance book cover handling

634370c

feat: improve loading state handling in DataStore and related components

48b26da

feat: add loading placeholder for cover images during loading state

9f1f750

feat: restructure BookEditPage layout for improved alignment and pers…

1f5c86e

…istent actions

feat: add waitUntilLoaded method to DataStore and update providers to…

eacc903

… handle loading state

feat: enhance Book model to support clearing optional fields and upda…

f56f279

…te related providers for consistent state management

Eoic merged commit 039651c into master Jul 12, 2026
1 check passed

Eoic deleted the codex/media-storage-pipeline branch July 12, 2026 23:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add client media upload and cache pipeline#16

Add client media upload and cache pipeline#16
Eoic merged 48 commits into
masterfrom
codex/media-storage-pipeline

Eoic commented Jun 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Eoic commented Jun 27, 2026

Summary

Depends on

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants