Skip to content

Conversation

@cristianoc
Copy link
Collaborator

@cristianoc cristianoc commented Dec 15, 2025

Vendor skip-lite library and integrate reactive analysis capabilities:

  • Vendor skip-lite marshal_cache and reactive_file_collection modules
  • Add ReactiveAnalysis module for incremental file processing
  • Add CLI flags: -cmt-cache, -reactive, -runs
  • Add README.md with usage and benchmark instructions

Benchmark results (~5000 files):

  • Standard: CMT processing 0.78s, Total 1.01s
  • Reactive (warm): CMT processing 0.01s, Total 0.20s
  • Speedup: 74x for CMT processing, 5x total

The reactive mode caches processed file_data and uses read_cmt_if_changed to skip unchanged files entirely on subsequent runs.

@cristianoc
Copy link
Collaborator Author

On real project kindly provided:

=== Run 1/3 ===
Reactive: 1888 files processed, 0 from cache

=== Timing ===
  CMT processing:     2.795s (90.3%)
    - Wall clock:        2.795s
    - Result collection: 0.000ms (aggregate)
  Analysis:           0.299s (9.7%)
    - Merging:          147.725ms
    - Solving:          151.455ms
  Reporting:          0.000s (0.0%)
  Total:              3.094s

=== Run 2/3 ===
Reactive: 0 files processed, 1888 from cache

=== Timing ===
  CMT processing:     0.009s (3.7%)
    - Wall clock:        0.009s
    - Result collection: 0.000ms (aggregate)
  Analysis:           0.231s (96.3%)
    - Merging:          126.980ms
    - Solving:          103.579ms
  Reporting:          0.000s (0.0%)
  Total:              0.239s

=== Run 3/3 ===
Reactive: 0 files processed, 1888 from cache

=== Timing ===
  CMT processing:     0.008s (3.3%)
    - Wall clock:        0.008s
    - Result collection: 0.000ms (aggregate)
  Analysis:           0.248s (96.7%)
    - Merging:          139.188ms
    - Solving:          108.941ms
  Reporting:          0.000s (0.0%)
  Total:              0.257s

Base automatically changed from reanalyze-reduce-global-state-ast to master December 15, 2025 14:52
Vendor skip-lite library and integrate reactive analysis capabilities:

- Vendor skip-lite marshal_cache and reactive_file_collection modules
- Modify C++ code to handle ReScript CMT file format (CMI+CMT headers)
- Add CmtCache module for mmap-based CMT file reading
- Add ReactiveAnalysis module for incremental file processing
- Add CLI flags: -cmt-cache, -reactive, -runs
- Add README.md with usage and benchmark instructions

Benchmark results (~5000 files):
- Standard: CMT processing 0.78s, Total 1.01s
- Reactive (warm): CMT processing 0.01s, Total 0.20s
- Speedup: 74x for CMT processing, 5x total

The reactive mode caches processed file_data and uses read_cmt_if_changed
to skip unchanged files entirely on subsequent runs.
- Change --create-sourcedirs to default to true (always create .sourcedirs.json)
- Hide the flag from help since it's now always enabled
- Add deprecation warning when flag is explicitly used
- Fix package name mismatches in test projects:
  - deadcode rescript.json: sample-typescript-app -> @tests/reanalyze-deadcode
  - rescript-react package.json: @tests/rescript-react -> @rescript/react
Simplify the reactive_file_collection implementation by making 'v t
be the concrete record type used at runtime, removing the unused
phantom fields and internal wrapper type. This eliminates warning 69
about unused record fields and relies directly on a single record with
its process function stored as an Obj.t-based callback.
@cristianoc cristianoc force-pushed the reactive-analysis-experiment branch from 1b2651a to 9265b64 Compare December 15, 2025 15:00
- CmtCache: rewritten using Unix.stat for file change detection
  (mtime, size, inode) instead of C++ mmap cache
- ReactiveFileCollection: new pure OCaml module for reactive file
  collections with delta-based updates
- ReactiveAnalysis: refactored to use ReactiveFileCollection,
  collection passed as parameter (no global mutable state)
- Deleted skip-lite vendor directory (C++ code no longer needed)

This eliminates the Linux/musl C++ compilation issue while
maintaining the same incremental analysis performance:
- Cold run: ~1.0s
- Warm run: ~0.01s (90x faster, skips unchanged files)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants