DScale

A fast, deterministic simulation framework for testing and benchmarking distributed systems. It simulates network latency, bandwidth constraints, and process execution in an event-driven environment with support for both single-threaded and parallel execution modes.

Usage

1. Install

In your project

cargo add dscale

2. Define Messages

Messages must implement the Message trait, which allows defining a virtual_size for bandwidth simulation.

use dscale::Message;

struct MyMessage {
    data: u32,
}

impl Message for MyMessage {
    fn virtual_size(&self) -> usize {
        // Size in bytes used for bandwidth simulation.
        // Can be much bigger than real memory size to simulate heavy payloads.
        1000
    }
}

// Or (if there is no need in bandwidth)
impl Message for MyMessage {}

3. Implement Process Logic

Implement ProcessHandle to define how your process reacts to initialization, messages, and timers.

use dscale::{ProcessHandle, Rank, MessagePtr, TimerId, Jiffies};
use dscale::{broadcast, send_to, schedule_timer_after, rank, debug_process};
use dscale::global::configuration;

#[derive(Default)]
struct MyProcess;

impl ProcessHandle for MyProcess {
    fn on_start(&mut self) {
        debug_process!("Starting process {} of {}", rank(), configuration::process_number());
        schedule_timer_after(Jiffies(100));
    }

    fn on_message(&mut self, from: Rank, message: MessagePtr) {
        if let Some(msg) = message.try_as_type::<MyMessage>() {
            debug_process!("Received message from {}: {}", from, msg.data);
        }
    }

    fn on_timer(&mut self, _id: TimerId) {
        broadcast(MyMessage { data: 42 });
    }
}

4. Run the Simulation

Use SimulationBuilder to configure the topology, network constraints, and start the simulation.

use dscale::{SimulationBuilder, Jiffies, BandwidthConfig, Distributions};

fn main() {
    let mut runner = SimulationBuilder::default()
        .add_pool::<MyClient>("Client", 1)
        .add_pool::<MyServer>("Server", 3)
        .within_pool_latency("Client", Distributions::Uniform(Jiffies(1), Jiffies(5)))
        .within_pool_latency("Server", Distributions::Uniform(Jiffies(1), Jiffies(5)))
        .between_pool_latency("Client", "Server", Distributions::Normal {
            mean: Jiffies(10),
            std_dev: Jiffies(2),
            low: Jiffies(5),
            high: Jiffies(20),
        })
        .vnic_bandwidth(BandwidthConfig::Bounded(1000)) // 1000 bytes per Jiffy
        .time_budget(Jiffies(1_000_000))
        .simple()
        .build();

    runner.run_full_budget();
}

Parallel Execution

For large simulations, enable parallel execution to distribute process steps across multiple threads:

let mut runner = SimulationBuilder::default()
    .add_pool::<MyProcess>("Nodes", 1000)
    .within_pool_latency("Nodes", Distributions::Uniform(Jiffies(1), Jiffies(10)))
    .time_budget(Jiffies(1_000_000))
    .parallel(Threads::Specific(8)) // use 8 worker threads
    .build();

runner.run_full_budget();

When is parallel mode efficient?

1. A lot of simulated processes (at least 200-300)
2. on_message execution takes most of simulation time
3. Independent work inside on_message (not so much synchronization)

Public API

Simulation Control

• SimulationBuilder: Configures the simulation environment.
  • default: Creates simulation with no processes and default parameters.
  • seed: Sets the random seed for deterministic execution.
  • time_budget: Sets the maximum simulation duration.
  • add_pool: Creates a named pool of processes. (All processes also join GLOBAL_POOL)
  • within_pool_latency(pool, distribution): Configures latency between processes within a pool.
  • between_pool_latency(pool_a, pool_b, distribution): Configures latency between two pools (symmetric). Every pool pair must have latency configured before calling build.
  • vnic_bandwidth: Configures per-process network bandwidth limits for "virtual" NIC.
    • Bounded(usize): Limits bandwidth (bytes per jiffy).
    • Unbounded: No bandwidth limits (default).
  • simple: Selects single-threaded execution (default). Mutually exclusive with parallel — calling both panics.
  • parallel(threads): Selects parallel execution with the given number of worker threads. Mutually exclusive with simple — calling both panics.
  • build: Finalizes configuration and returns a simulation runner.
• run_full_budget: Runs the simulation until the time budget is exhausted.
• run_steps: Runs the simulation until it performs the requested number of steps or the global budget is exhausted.
• run_sub_budget: Runs the simulation until the sub-budget starting from current timepoint or global budget are exhausted.

Network Topology

• GLOBAL_POOL:
  • Implicit pool containing all processes. broadcast uses this pool.
• Distributions:
  • Uniform(low, high): Uniform distribution over [low, high].
  • Bernoulli(p, value): With probability p the latency is value, otherwise 0.
  • Normal { mean, std_dev, low, high }: Truncated normal distribution clamped to [low, high].

Process Interaction (Context-Aware)

These functions are available globally but must be called within the context of a running process step.

• broadcast: Shortcut for broadcast_within_pool(GLOBAL_POOL).
• broadcast_within_pool: Sends a message to all processes within a named pool.
• send_to: Sends a message to a specific process by rank.
• send_random: Shortcut for send_random_from_pool(GLOBAL_POOL).
• send_random_from_pool: Sends a message to a random process within a named pool.
• schedule_timer_after: Schedules a timer for the current process, returns a TimerId.
• rank: Returns the rank of the currently executing process. (Ranks start at 0)
• now: Returns the current simulation time.
• list_pool: Returns a slice of all process ranks in a pool.
• choose_from_pool: Picks a random process rank from a named pool.
• global_unique_id: Generates a globally unique monotonic ID.

Configuration (dscale::global::configuration)

• seed: Returns the deterministic seed for the current process.
• process_number: Returns total number of processes in the simulation.

Key-Value Store (dscale::global::kv)

Thread-safe store for passing shared state, metrics, or configuration between processes or back to the host.

• set(key, value): Stores a value under the given key.
• get(key) -> T: Retrieves a clone of the value (panics if missing or wrong type).
• modify(key, f): Mutates the value in place.

Macros

• debug_process!: Logs a debug message prefixed with the current simulation time and process rank. Available at the crate root (use dscale::debug_process).

Helpers (dscale::helpers)

• Combiner: Collects values until a threshold is reached, then yields them all at once. Useful for quorum-based logic.

Message Downcasting (MessagePtr)

• try_as_type::<T>(): Attempts to downcast to T, returns Option<&T>.
• as_type::<T>(): Downcasts to T, panics if the type does not match.
• is::<T>(): Returns true if the message is of type T.

Logging Configuration (RUST_LOG)

DScale output is controlled via the RUST_LOG environment variable.

• RUST_LOG=info: Shows high-level simulation status and a progress bar.
• RUST_LOG=debug: Enables all debug_process! macro output and all internal simulation events.
• RUST_LOG=full::path::to::your::file::or::crate=debug,another::path=debug: Filter events only for your specific file or crate.

Note: RUST_LOG=debug and path-level debug filters only work without the --release flag.

Thanks to

• https://gitlab.com/whirl-framework
• https://github.com/jepsen-io/maelstrom
• https://github.com/systems-group/anysystem
• https://www.open-mpi.org
• https://www.nsnam.org