Skip to content

OpenRange as physical-space simulation  #219

@larstalian

Description

@larstalian

Vision

OpenRange as a physical-space simulation substrate: an entire
building, with offices, desks, hallways, kitchens; people moving
through it, sitting at desks, talking, opening laptops. The cyber
webapp pack is the first scenario rendered on top of this — not the
goal.

The goal is a generic multi-agent environment where world state is
grounded (positions, rooms, objects, conversations) and agents
(LLM-driven employees, attackers, defenders, future household /
warehouse / logistics agents) act through that grounding rather than
through a thin HTTP-only API.

Why this matters

For cyber specifically (which is where we have the strongest pull
today):

  • Insider scenarios with a body. Phishing isn't a click; it's
    an employee at a specific desk, mid-conversation with a colleague,
    who clicks at 2:47pm. Physical context is what makes the
    attacker's chain realistic and the defender's correlation
    problem hard.
  • Hybrid attacks. Tailgating, USB drops, badge cloning. None of
    these have a cyber-only abstraction.
  • Realistic noise. Why is this laptop suddenly hitting
    /admin? Because Maria walked over and they're pair-debugging.
    Co-occurrences ground signal vs. noise.

For everything else: the same engine can host warehouse robotics,
household assistance, office-process simulation, multi-agent
logistics. Cyber is a forcing function, not the destination.

Reference: Concordia

google-deepmind/concordia
is the closest existing thing. LLM-driven agents composed from
components (memory, observation, plan, somatic state, goal),
acting against a GameMaster that resolves each tick — with
explicit support for spatial/physical state. Their examples/
folder is the right place to read; the API is small, the patterns
are in the configs.

The agent shape they land on maps cleanly onto our AgentNPC +
AgentBackend work in #74 / #217.

Where we are

Already in tree:

  • AgentNPC + AgentBackend — agent loops with tools, opt-in
    LLM, swappable provider. Concordia's "agent driven by some model"
    primitive is essentially this.
  • Dashboard with a Three.js 3D office renderer (sim-canvas) —
    desks, walls, walking characters, idle NPC wandering. Currently
    fed by topology services + green_personas; ready to scale.

Missing for the full vision:

  • Physical backing parallel to HTTPBacking — exposes
    move_to(room), observe_nearby(), pick_up(object),
    say_to(actor, text) as the interface mapping. NPCs/agents
    bind their tools to these instead of (or alongside) http_get.
  • GameMaster-shaped tick model. Concordia's GM resolves
    everyone's intended actions per tick; our current _step_npcs
    iterates with no contention model. Fine for HTTP, picky for
    physical (two NPCs picking up the same object, queuing for the
    printer, etc.).
  • Per-actor memory + observation streams. Today NPCs are
    stateless across ticks. Concordia's associative memory component
    is a good shape to copy when a real physical pack needs it.
  • A real physical pack. Probably a small office (5-10 employees,
    one floor, low-fidelity rooms) is the right first cut — domestic
    / warehouse can come later. Choice constrains what the backing's
    interface needs to look like first.

Phasing

Each step is its own PR; don't try to design the whole system up
front.

  1. Office-shaped demo on top of the cyber pack. No new
    abstractions. Spawn ~6 chatter NPCs in the existing renderer,
    wire speech-event rendering. Goal: a gif that shows the vision.
    (In flight — see linked PR.)
  2. Physical backing protocol. Define the interface shape
    (move_to, observe_nearby, say_to, ...). Implement a
    minimal in-memory backing driving the existing dashboard
    renderer.
  3. First standalone physical pack. Small office, separate from
    the cyber pack — proves the substrate generalizes.
  4. GameMaster tick. Per-tick action resolution with simple
    contention rules (one actor per object, rate-limited speech).
  5. Memory + observation primitives. Concordia-shape components
    (associative recall, recent buffer). Gated on a pack that
    actually needs them.
  6. LLM-driven employees as the default. Agent loops driving
    movement / speech from persona + memory + nearby observations,
    not from canned phrases.

Out of scope (for this issue)

  • Cost/budget story for LLM-driven NPCs (separate concern).
  • Defender-vs-attacker game theory.
  • Specific Concordia API choices (different runtime model; we may
    borrow the patterns, not the codebase).

Acceptance

  • Per phase: each step lands as its own PR with its own acceptance
    criteria. This issue is the umbrella, not a single deliverable.
  • Issue stays open until phase 3 lands (first standalone physical
    pack) — at that point the substrate has proven itself and the
    remaining phases are incremental polish.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions