Skip to content

plan.md

Latest commit

 

History

History
43 lines (29 loc) · 4.25 KB

plan.md

File metadata and controls

43 lines (29 loc) · 4.25 KB

Of course. Here is a technical implementation plan geared toward an LLM helping you build out your Rust application.

Phase 1: Core Functionality

1. Project Setup and Configuration

  • Goal: Create a new Rust project and set up the basic structure for configuration.
  • LLM Prompt: "Create a new binary Rust project named attic-lite. Add the necessary dependencies to Cargo.toml for command-line argument parsing (clap), configuration file management (serde, toml), and error handling (anyhow). Create a Config struct in a config.rs module that can be deserialized from a TOML file. The configuration should include S3 bucket details (bucket name, region, endpoint, access key, secret key) and the number of upload threads."

2. S3 Uploader

  • Goal: Implement the logic for uploading files to your S3-compatible storage.
  • LLM Prompt: "In a new s3_uploader.rs module, create an S3Uploader struct that holds the S3 client and configuration. Implement a new function that initializes the S3 client from the configuration. Then, create an upload_nar async function that takes a file path and an object key as input, and uploads the file to the configured S3 bucket using the aws-sdk-s3 crate. This function should be able to handle streaming the file content."

3. Nix Store Watcher

  • Goal: Implement the file system watcher to detect new store paths.
  • LLM Prompt: "Create a store_watcher.rs module. Implement a function watch_store that takes a tokio channel sender as an argument. This function should use the notify crate to watch the /nix/store directory recursively. When a Remove event is detected for a file with a .lock extension, extract the store path and send it through the channel. The function should run in a separate thread."

Phase 2: Integration and Logic

4. Main Application Logic

  • Goal: Tie the configuration, watcher, and uploader together in the main application loop.
  • LLM Prompt: "In main.rs, initialize the configuration and the S3Uploader. Create a tokio multi-threaded runtime. Spawn the watch_store function in a new task. In the main task, create a loop that receives store paths from the channel. For now, just log the received store paths to the console."

5. Closure Calculation and Deduplication

  • Goal: Before uploading, calculate the closure of a store path and check which paths already exist in the cache.
  • LLM Prompt: "Create a nix.rs module. Implement a function get_store_path_closure that takes a store path and returns a Vec<String> of all its dependencies by shelling out to nix-store -qR. In s3_uploader.rs, create a function check_paths_exist that takes a list of store paths and returns a new list containing only the paths that do not already exist in the S3 bucket. You can check for existence by making a HeadObject request for the corresponding .narinfo object for each path. In your main loop, when you receive a path, get its closure and filter it using check_paths_exist."

6. NAR Generation and Upload

  • Goal: For the missing paths, generate the NAR and upload it.
  • LLM Prompt: "In nix.rs, create a function create_nar that takes a store path, shells out to nix-store --dump, and returns a stream of the NAR content. In your main loop, for each missing path, spawn a new tokio task. Inside the task, call create_nar to get the NAR stream and then use s3_uploader.upload_nar to upload it. Use a Semaphore to limit the number of concurrent uploads based on the upload_threads configuration."

Phase 3: Finalizing Touches

7. .narinfo Generation

  • Goal: After a NAR is successfully uploaded, generate and upload the corresponding .narinfo file.
  • LLM Prompt: "In nix.rs, create a function get_nar_info that takes a store path and shells out to nix-store --query --get-nar-info. This will return the necessary information to construct the .narinfo file. After a NAR is uploaded successfully, call this function, create the .narinfo content, and upload it to S3."

By breaking down the project into these focused tasks, you can effectively use an LLM to generate the boilerplate and core logic for each component, allowing you to focus on the integration and refinement of your application.