Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ jobs:
shared-key: cargo-${{ inputs.runner }}

# - name: Integration tests
# run: cargo nextest run --tests integration
# run: cargo nextest run --tests it
2 changes: 1 addition & 1 deletion .github/workflows/unit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
shared-key: cargo-${{ inputs.runner }}

- name: Unit tests (no features)
run: cargo nextest run
run: cargo nextest run --lib

- name: Doc tests
run: cargo test --doc --all-features
63 changes: 63 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# SparseIO Agent Guide

## Library Development Standards

SparseIO is a Rust library, not an application runtime. Prefer futures-oriented crates and APIs that remain usable from Tokio, smol, async-std, and other executors.

- Do not add Tokio as a normal library dependency unless the public API explicitly requires Tokio types.
- Runtime-specific crates are acceptable in tests, examples, or feature-gated integrations when they do not leak into the core API.
- Use executor-neutral primitives for library behavior. For example, prefer timer futures such as `futures-timer` over `tokio::time` in core code.
- Keep async traits object-safe for registry and builder usage. The current backend traits use `async_trait` and are stored behind trait objects.
- Prefer owned `Bytes` when transferring large payloads into storage-like APIs. Avoid forcing callers or mock implementations to clone large byte buffers.

## Commenting Standards

Write comments for future maintainers, not for the compiler.

- Add brief rustdoc comments to structs, traits, public methods, and `pub(crate)` utilities that are meant to be reused across tests or modules.
- Describe behavior, constraints, or intent that is useful when changing the code later.
- Avoid comments that restate the function name or obvious Rust syntax.
- Keep comments short. One sentence is usually enough.
- Use inline comments sparingly, only when a small implementation detail would otherwise be easy to misread.
- Keep tests readable through names and assertions first; add comments only when the scenario itself is non-obvious.
- Avoid in-line context comments regarding conversations as they provide no value to the codebase. A good rule of thumb is that if it wont provide value to someone 5 commits down the road, it doesn't deserve to be a comment.

## Project Layout

The crate is organized around a small public API with implementation details split by responsibility.

- Keep public API surfaces in the top-level `src/` modules.
- Keep backend traits and shared public types separate from concrete implementations.
- Keep test helpers and low-level support code under `src/utils/`.
- Keep architecture and design context in `docs/`; update it when behavior or architecture changes materially.
- Prefer small modules with clear ownership over broad utility buckets.

## Test Utilities

Test utilities should stay lightweight, deterministic, and easy to reason about.

- Prefer simple in-memory implementations over elaborate fixtures.
- Keep simulated latency and randomness deterministic.
- Use explicit zero-latency profiles in tests that need immediate completion or precise timing.
- Avoid making test helpers more capable than the scenarios they support.

## Dependency Guidance

Keep the core dependency graph small and runtime-neutral.

- Normal dependencies should support library internals or public API requirements.
- Dev dependencies may use Tokio for `#[tokio::test]` and other test-only needs.
- Before adding a dependency, check whether the standard library or an existing crate dependency is sufficient.
- Avoid dependencies that force a specific executor, global runtime, or background task model into core library code.

## Verification

Run the relevant checks before handing off code changes.

```bash
cargo fmt --check
cargo test
cargo clippy --all-targets -- -D warnings
```

The repository currently has rustfmt settings that may warn on stable Rust about nightly-only options. Treat formatting failures as actionable; the nightly-option warnings alone are expected.
1 change: 1 addition & 0 deletions CLAUDE.md
82 changes: 82 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,8 @@ license = "Apache-2.0"
[dependencies]
parking_lot = "0.12"
bytes = "1.11"
async-trait = "0.1"

[dev-dependencies]
tokio = { version = "1", features = ["macros", "rt", "time"] }
Comment thread
mhambre marked this conversation as resolved.
futures-timer = "3.0"
16 changes: 8 additions & 8 deletions src/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,21 +93,21 @@ mod tests {

use super::Builder;
use crate::ReaderRegistry;
use crate::utils::metadatas::StubMetadata;
use crate::utils::readers::StubReader;
use crate::utils::writers::StubWriter;
use crate::utils::metadatas::MockMetadata;
use crate::utils::readers::MockReader;
use crate::utils::writers::MockWriter;

fn writer() -> StubWriter {
StubWriter
fn writer() -> MockWriter {
MockWriter::new()
}

fn metadata() -> StubMetadata {
StubMetadata
fn metadata() -> MockMetadata {
MockMetadata::new()
}

fn registry() -> ReaderRegistry {
let mut registry = ReaderRegistry::new();
registry.register("stub", StubReader);
registry.register("stub", MockReader::new(bytes::Bytes::new()));
registry
}

Expand Down
16 changes: 9 additions & 7 deletions src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,22 @@ impl ReaderRegistry {
mod tests {
use std::io;

use async_trait::async_trait;
use bytes::Bytes;

use super::ReaderRegistry;
use crate::Reader;
use crate::utils::readers::StubReader;
use crate::utils::readers::MockReader;

struct LenReader(usize);

#[async_trait]
impl Reader for LenReader {
fn len(&self) -> io::Result<usize> {
async fn len(&self) -> io::Result<usize> {
Ok(self.0)
}

fn read_at(&self, _offset: usize, _length: usize) -> io::Result<Bytes> {
async fn read_at(&self, _offset: usize, _length: usize) -> io::Result<Bytes> {
unimplemented!("registry tests should not call reader read_at")
}
}
Expand All @@ -69,21 +71,21 @@ mod tests {
fn register_makes_reader_retrievable_by_name() {
let mut registry = ReaderRegistry::new();

registry.register("stub", StubReader);
registry.register("stub", MockReader::new(Bytes::new()));

assert!(registry.get("stub").is_some());
assert!(registry.get("other").is_none());
}

#[test]
fn register_replaces_existing_reader_for_name() {
#[tokio::test]
async fn register_replaces_existing_reader_for_name() {
let mut registry = ReaderRegistry::new();

registry.register("source", LenReader(10));
registry.register("source", LenReader(20));

let reader = registry.get("source").expect("replacement reader should be registered");

assert_eq!(reader.len().expect("len should be readable"), 20);
assert_eq!(reader.len().await.expect("len should be readable"), 20);
}
}
20 changes: 12 additions & 8 deletions src/traits.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,42 @@

use std::io;

use async_trait::async_trait;
use bytes::Bytes;

/// Reads explicit byte ranges from an upstream data source.
#[async_trait]
#[allow(clippy::len_without_is_empty)]
pub trait Reader: Send + Sync {
/// Return the total number of bytes in the source object.
fn len(&self) -> io::Result<usize>;
async fn len(&self) -> io::Result<usize>;

/// Read `length` bytes beginning at `offset`.
fn read_at(&self, offset: usize, length: usize) -> io::Result<Bytes>;
async fn read_at(&self, offset: usize, length: usize) -> io::Result<Bytes>;
Comment on lines 8 to +16
}

/// Stores and retrieves cached byte ranges.
#[async_trait]
pub trait Writer: Send + Sync {
/// Write bytes into the cache under `key`.
fn write(&self, key: &str) -> io::Result<()>;
async fn write(&self, key: &str, value: Bytes) -> io::Result<()>;

/// Read a byte range from the cache.
fn read(&self, key: &str) -> io::Result<Option<Bytes>>;
async fn read(&self, key: &str) -> io::Result<Option<Bytes>>;

/// Delete all cached data associated with `key`.
fn delete(&self, key: &str) -> io::Result<()>;
async fn delete(&self, key: &str) -> io::Result<()>;
}

/// Key-value metadata store used for coverage maps and CAS reference counts.
#[async_trait]
pub trait Metadata: Send + Sync {
/// Retrieve a value from the metadata store.
fn get(&self, key: &str) -> io::Result<Option<Bytes>>;
async fn get(&self, key: &str) -> io::Result<Option<Bytes>>;

/// Persist a value in the metadata store.
fn set(&self, key: &str, value: Bytes) -> io::Result<()>;
async fn set(&self, key: &str, value: Bytes) -> io::Result<()>;

/// Delete a value from the metadata store.
fn delete(&self, key: &str) -> io::Result<()>;
async fn delete(&self, key: &str) -> io::Result<()>;
}
Loading
Loading