feat: v6 file format support + remove rmapi/unipdf dependencies#441
feat: v6 file format support + remove rmapi/unipdf dependencies#441joagonca wants to merge 25 commits into
Conversation
Made CGo build more compliant
Resolve minor conflicts in go.mod (add rmc-go + juju/fslock deps), go.sum (take v6_rmc version), and blobstore.go (add imports).
Remove rmapi and unipdf dependencies, replacing with: - internal/archive: pure-Go reMarkable archive reader - internal/encoding/rm: pure-Go .rm binary format decoder - pdf_cairo.go: Cairo-based PDF renderer (replaces unipdf) - pdfcpu: PDF merging (replaces unipdf) Conflict resolutions: - Dockerfile/Makefile: Cairo is now the mandatory default build - go.mod: combined deps from both branches, Go 1.25.1 - blobstore.go: keep v6 routing logic, RenderRmapi -> RenderPDF - documents.go: keep v6 routing logic, RenderRmapi -> RenderPDF - imports: rmapi/archive -> internal/archive everywhere
…slock Two callsites in blobstore.go still used the old fslock.Lock()/Handle API. Migrate to fslock.New() + LockWithTimeout() consistent with v6_rmc.
Apply common.Sanitize(id) at function entry, consistent with ExportDocument. Removes filepath.Base wrapping since getPathFromUser already applies sanitizeFileName. Addresses CodeQL path traversal warnings.
V6 nounipdf
Bring in 20 upstream commits: - feat: passcode reset for rm1/rm2 - feat: rmdoc import/export - fix: generation-0 root missing - fix: SupportedFileTypes for 3.25 - fix: /pair connect paths - fix: RMAPI_HOST typo - fix(ci): changelog append - docs: 3.25/3.26 compat, mTLS guide, setup cleanups README conflict resolved: keep our v6+doc-rendering rows, add upstream passcode row.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| meta := sanitizedID + storage.MetadataFileExt | ||
| fullPath := fs.getPathFromUser(uid, meta) | ||
| err = os.Rename(fullPath, path.Join(trashDir, meta)) | ||
| err = os.Rename(fullPath, filepath.Join(trashDir, meta)) |
| meta := sanitizedID + storage.MetadataFileExt | ||
| fullPath := fs.getPathFromUser(uid, meta) | ||
| err = os.Rename(fullPath, path.Join(trashDir, meta)) | ||
| err = os.Rename(fullPath, filepath.Join(trashDir, meta)) |
|
I've merged both Also, I've validated the CodeQL suggestions, and I don't think the still pending ones are valid :) But open to the discussion. |
|
Why are you removing working features? |
|
My bad - they aren't removed. This was 'written' after doing the merge of both v6 and remove_unipdf branches, but before bringing in the upstream changes (since they differed in time, this has caused some git confusion). I've compared files from both upstream and this PR, and they're practically identical. So no features were removed. I just forgot to update the description after cherry-picking the changes, as rebase didn't work out of the box. EDIT: The only visible difference to upstream, is the presence of |
|
Fixes: #255 |
|
Hi everyone, first thanks for all your effort on contributing on this package and specifically on this PR. I just downloaded the docker from @crazyfacka and the build the docker from @joagonca repo. In both cases, I can download the notebooks in pdf and in rmdoc .However, if I annotated a pdf (such as paper) in the remarkable, then the downloaded .pdf has the annotations but the pdf (the actual paper with figures and text) is missing, so white page. Am I the only one experiencing this? |
|
@LopezBanos You're not. That is a known limitation for now. |
|
I'm currently testing out this commit, and it seems to be working quite well - thanks so much for your work on this. I am seeing a quirk, though: for multi-page documents, the page order in the PDF downloaded from rmfakecloud doesn't retain the correct page order. After a bit of poking around, it seems that pages appear in the PDF in descending alphabetical order of their id hashes. The correct sequence would be the ordering of the For example, for a three page document with page hash IDs:
The generated PDF would have the pages in the sequence: 2, 3, 1 |
6 participants
Summary
This PR adds native support for the reMarkable v6 file format (Paper Pro, software 3.0+) and removes the
rmapiandunipdfdependencies.What's changed
v6 file format support
.rmfile version (v3/v5/v6) by reading the 43-byte header before attempting to parsermc-golibrary for native in-process PDF rendering via Cairocontent.jsonexporter/rmc_go.go,exporter/version.go,exporter/version_test.goRemove rmapi + unipdf dependencies
github.com/juruen/rmapireplaced by:internal/archive— pure-Go reMarkable.ziparchive readerinternal/encoding/rm— pure-Go.rmbinary format decoder (v3/v5)github.com/unidoc/unipdf/v3replaced by:github.com/ungerik/go-cairo— Cairo-based PDF surface rendering via CGogithub.com/pdfcpu/pdfcpu— pure-Go PDF merging (annotations onto background)exporter/license.go(thego:linknamehack to inject a UniPDF community license key)RenderRmapi()renamed toRenderPDF()throughoutBuild changes
libcairo2-dev,pkg-config)CGO_ENABLED=1and-tags cairoare the default for all build targetspdf_stub.goprovides a clear error message if built without Cairoscratch→debian:bookworm-slim(needed forlibcairo2runtime)github.com/joagonca/rmc-go v1.1.1github.com/danjacques/gofslock→github.com/juju/fslockOther changes
Removed schema v4 hash tree support (hardcoded to v3)RemovedHASH_SCHEMA_VERSIONenvironment variableRemoved embedded MQTT broker and screen sharing infrastructureRemoveDocument()against path traversal (appliescommon.Sanitize()consistently)New dependency:
rmc-gormc-gois a Go library wrapping the reMarkable.rmv6 format renderer. It uses Cairo for native PDF/SVG generation, supporting colour and the new pen types introduced with the Paper Pro.Testing
go build -tags cairo ./cmd/rmfakecloud/✅go test -tags cairo ./...✅ (all existing + new version detection tests)go vet -tags cairo ./...✅Breaking changes
libcairo2-dev) is now a required build dependencyHASH_SCHEMA_VERSIONenv var no longer supportedMQTT-based screen sharing removed (was not functional remotely)