[feature] Portals! #1484

KCarretto · 2026-01-01T21:26:45Z

We take your TCP, UDP, or Bytes messages and teleport them anywhere in the world! This enables SOCKS5 proxying from a local client application through any imix agent 🎉

This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>

- Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>

Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>

github-actions · 2026-01-01T21:42:27Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
2829 ±0	2829 ±0	0 ±0	0 ±0	0 ±0	0 ±0	1ms ±0

Previous Results

Build 🏗️	Result 🧪	Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
#262	✅	2829	2829	0	0	0	0	30.8s

Insights

Average Tests per Run	Total Flaky Tests	Total Failed	Slowest Test (p95)
2829	0	0	43.6s

Slowest Tests

Test 📝	Results 📊	Duration (avg) ⏱️	Duration (p95) ⏱️
eldritch: random::string_impl::tests::test_string_uniform	3	39.8s	43.6s
eldritch: random::string_impl::tests::test_string_uniform	3	39.8s	43.6s
eldritch: random::string_impl::tests::test_string_uniform	3	39.8s	43.6s
eldritch: process::info_impl::tests::test_info_default	3	3.5s	10.0s
eldritch: process::info_impl::tests::test_info_default	3	3.5s	10.0s
eldritch: process::info_impl::tests::test_info_default	3	3.5s	10.0s
eldritch: pivot::port_scan_impl::tests::test_portscan_return_type_starlark_dict_from_interpreter	3	3.1s	9.2s
eldritch: pivot::port_scan_impl::tests::test_portscan_return_type_starlark_dict_from_interpreter	3	3.1s	9.2s
eldritch: pivot::port_scan_impl::tests::test_portscan_return_type_starlark_dict_from_interpreter	3	3.1s	9.2s
eldritch: pivot::ssh_copy_impl::tests::test_pivot_ssh_copy	3	4.4s	9.1s

_{🎉 No failed tests in this run. | 🍂 No flaky tests in this run.}

Github Test Reporter by CTRF 💚

🔄 This comment has been updated

hulto · 2026-01-02T18:14:57Z

Tome should probably output the command to connect to the newly created portal.

hulto · 2026-01-02T18:31:45Z

Looks like DNS requests are being forwarded

KCarretto · 2026-01-02T18:50:23Z

Tome should probably output the command to connect to the newly created portal.

It doesn't get the ID back so it can't, but we could provide this on the UI side (via portal query)

KCarretto · 2026-01-02T18:50:44Z

Looks like DNS requests are being forwarded

Is that an issue? AFAIK that's based on client config

hulto · 2026-01-02T19:26:43Z

Looks like DNS requests are being forwarded

Is that an issue? AFAIK that's based on client config

A couple socks5h implementations don't actually do the DNS part. Just making sure ours does since it's part of the spec.

hulto · 2026-01-02T21:58:52Z

Add docs on TAVERN_API_TOKEN and the portal auth flow.

TAVERN_API_TOKEN is required when SSH'd to a Kali VM can't define the auth redir port for SSH port forwarding.
Make clear this token is different than the web oauth token

hulto · 2026-01-03T00:12:17Z

Tested with RDP can't quite game over it yet but works otherwise!
Tested nmap syn scan seemed to work great!
Tested with a prod deployment and seemed to work.

hulto

LGTM

hulto · 2026-01-03T00:13:26Z

implants/lib/pb/build.rs

+        .codec_path("crate::xchacha::ChachaCodec")
+        .build_client(false)
+        .build_server(false)
+        .compile(&["trace.proto"], &["../../../tavern/portals/proto/"])


Do we want trace compiled in for production builds? I think we probably want to gate this behind debug builds.

I think we do, it's useful to help understand network latency and could be really helpful for us to figure out why something isn't working. It's just two small protobufs, so I don't think the functionality will impact binary size much.

hulto · 2026-01-03T00:19:53Z

implants/lib/portals/portal-stream/Cargo.toml

+
+[features]
+default = ["tokio"]
+tokio = ["dep:tokio"]


This seems like it might be redundant.
Is their a situation where we don't compile with tokio?

Yes, you might want to import the synchronous version of the reader / writer (e.g. in a no_std context)

* Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

- Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

* feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest

* Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com>

* initial stubbing and renaming * feat: implement tavern/portals/stream package (#1462) This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor reader to use functional API * Implement PubSub Multiplexer (Mux) (#1466) * Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * both grpc endpoints * stub out portal-stream package * Implement portal-stream crate (#1470) * Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement SOCKS5 Proxy using tavern/portals/stream (#1467) * Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement Portal Infrastructure in ImixV2 (#1471) * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * some changes * Add integration test * Added TRACE Bytes kind * added trace protos * Implement Trace Motes for Portals (#1473) * feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * update flag to --portal * update buf size * Update portal API to send keepalive motes (#1472) - Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * add tokio-console support for imixv2 * Fix SOCKS5 proxy cold start hang by avoiding BiLock on TcpStream (#1476) Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * socks proxy sends registration message now * socks5 trace must send registration message * Add E2E Portals Workflow and Playwright Test (#1478) Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * cargo fmt * remove spammy log line * fix some tests * add create_portal fake impl * revert some changes to grpc.rs * remove grpc explicit sizes * update socks5 proxy to support auth and portals to support gcp pubsub * oops * minor cleanup * fix tests * Fix e2e workflow * use env var for auth * wait for socks to start before continuing * Add benchmark tests for cryptocodec (#1485) Added a new test file `tavern/internal/cryptocodec/cryptocodec_bench_test.go` to measure the throughput of `Encrypt` and `Decrypt` methods in `CryptoSvc`. Includes benchmarks for both encryption and decryption operations using standard payload sizes. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix race condition in TestPortalIntegration (#1487) The `TestPortalIntegration` test was flaky and often hung because the Agent would publish messages to the portal before the User had successfully subscribed to the output topic. This resulted in messages being dropped (as evidenced by "message sent to topic with no subscribers" warnings from the in-memory pubsub) and the User reader blocking indefinitely. This commit replaces the arbitrary `time.Sleep` with a deterministic synchronization mechanism. The User now sends a "ping" message to the Agent immediately after opening the portal. The Agent waits to receive this ping before proceeding. This ensures that the User's portal connection (and thus the underlying pubsub subscription) is fully established before the Agent attempts to send any data. This reduces test execution time and eliminates the race condition. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix proxy upstream address parsing * minor fixes & cleanup * fix collapsible if * cargo fmt * fix portal workflow * Add Tavern User Guide for Authentication and API Token (#1501) * Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup * Merge origin/main, resolve conflicts, and fix go generate (#1515) - Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest * Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Squidli <32434695+cmp5987@users.noreply.github.com> Co-authored-by: Hulto <7121375+hulto@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

commit 2a86d51 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Mon Jan 5 22:46:36 2026 -0500 Split stdlib into seperate impl files. (#1538) * Split stdlib into seperate impl files. * fmt commit 7ffd027 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Mon Jan 5 21:24:30 2026 -0500 Multi transport builds (#1517) * Add multi-callback * Add DSN configuration * Force lowercase * Auto grab pubkey with multi-callback using 0th element. * Update v1 * Update callback rotation * fmt * Remove agent callback_uris and idx in favor of config's * Remove callback_uri from transport init replaced by config * remove callback_uri direct ref * fmt * Cleanup warnings * Address code review feedback - Remove unused imports and variables in agent.rs - Switch to and_then syntax for cleaner Option chaining - Refactor parse_dsn to return Transport struct directly - Convert default parameters to const values - Bubble up errors instead of returning defaults - Use named enum variants instead of numeric constants in tests - Update all test cases to use TransportType enum 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Hulto <hulto@users.noreply.github.com> * fmt * Fix test_parse_callback_interval_valid test The test was failing because parse_callback_interval() now returns anyhow::Result<u64> instead of u64. Updated the test to properly unwrap the Result before comparison. Co-authored-by: Hulto <hulto@users.noreply.github.com> * Update enum name * Fix tests * Fix tests * Setup tests * fix? --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Hulto <hulto@users.noreply.github.com> commit 2d11111 Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 23:32:42 2026 -0500 update devcontainer go version to 1.24.3 (#1532) commit 40e9e6d Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 23:30:45 2026 -0500 [bug] Close Portals (#1530) * added close message * feat: add integration test for portal closure (#1531) Added `tavern/internal/portals/portal_close_test.go` to verify that when an agent's `CreatePortal` stream ends, the corresponding user `OpenPortal` streams are notified with a CLOSE mote and subsequently terminated. Implemented a retry mechanism with backoff for portal creation checks in the test setup. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup test --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 2e1506c Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Mon Jan 5 02:38:56 2026 +0000 Refactor File entity to Asset (#1529) Renames the `File` entity to `Asset` across the entire codebase, including: - Ent schema definitions and relationships - Generated Ent code - GraphQL schema, resolvers, and queries - Business logic in C2, CDN, and Tomes packages - Test data and test cases This aligns the terminology to be more generic and suitable for various types of resources managed by the system. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 42c30b9 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Sun Jan 4 19:30:21 2026 -0500 Update ZIG_VERSION to 0.16.0-dev.1859 commit f075895 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Sun Jan 4 19:11:52 2026 -0500 Update Go base image to version 1.24.0 commit cc831c2 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sun Jan 4 18:43:26 2026 -0500 fix(logo): Fix logo flashing (#1509) * fix(logo): Fix logo flashing * fix(z-index): Fix z-index of table commit 490b0cd Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Sun Jan 4 18:10:50 2026 -0500 Improve tome automation scheduling window (#1527) * Improve tome automation scheduling window Modified tome automation logic to queue tasks if their schedule falls within the upcoming beacon check-in interval window (now to now + interval), rather than just checking the current minute. This prevents beacons with long sleep intervals from missing scheduled tasks. Added `TestHandleTomeAutomation_IntervalWindow` to verify the fix and prevent regressions. Also fixed a missing sqlite3 driver import in `tome_automation_test.go`. * Refactor tome automation to use time.Duration Updated `handleTomeAutomation` signature to accept `time.Duration` for interval instead of `int`, addressing PR feedback. Updated call sites and tests to pass duration. * Refactor tome automation for lookahead and range support Updated `handleTomeAutomation` to support lookahead scheduling for standard cron expressions (queueing tasks scheduled within the upcoming beacon interval) while enforcing strict current-time matching for range-based schedules (e.g., `* 6-12 * * *`). Refactored method to accept `time.Duration` for interval. Added comprehensive tests for interval window logic and cron range behavior. Fixed missing sqlite3 driver in tests. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit ee68dea Author: Kirill Rudakov <simbiont666@gmail.com> Date: Mon Jan 5 00:21:14 2026 +0300 fix: fix #1516 (#1524) * fix: fix #1516 * fix: fixes second part of #1516 (type of ActiveTransport as enum variant name) --------- Co-authored-by: Kirill Rudakov <kirill.rudakov@gorparkovki.ru> commit 27c1744 Author: KCarretto <Kcarretto@gmail.com> Date: Sun Jan 4 16:17:50 2026 -0500 [feature] Tome automation (#1526) * added new fields for tome automation * Implement Tome automation in ClaimTasks API (#1525) * Implement Tome automation for ClaimTasks Refactored ClaimTasks to include logic for automatically queuing Quests/Tasks based on Tome triggers: - RunOnNewBeaconCallback - RunOnFirstHostCallback - RunOnSchedule (cron) Added Prometheus metrics for automation errors and implemented non-blocking error handling. * Refactor Tome automation into helper function Moved Tome automation logic from `ClaimTasks` to `handleTomeAutomation` to improve code readability and maintainability. * Add comprehensive tests for Tome automation Added `tome_automation_test.go` containing `TestHandleTomeAutomation` to verify: - Triggers: New Beacon, New Host, Schedule. - Constraints: Scheduled host restrictions (allow/deny). - Deduplication: Ensuring a tome is only queued once per callback even if multiple triggers match. This addresses PR feedback requesting unit/integration tests for the new `handleTomeAutomation` helper. * Refactor handleTomeAutomation to use early return Simplified the control flow in `handleTomeAutomation` by returning early on error, reducing nesting and improving readability. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit c20e22d Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sat Jan 3 20:08:38 2026 -0500 UI improve table (#1519) * feat(table): Improve table layout * fix(styling): Change rows shown, add tests, fix styling commit 4d773c1 Author: KCarretto <Kcarretto@gmail.com> Date: Sat Jan 3 15:42:51 2026 -0500 update portals e2e to be used during merge queue (#1518) commit 605e830 Author: KCarretto <Kcarretto@gmail.com> Date: Sat Jan 3 14:14:38 2026 -0500 [feature] Portals! (#1484) * initial stubbing and renaming * feat: implement tavern/portals/stream package (#1462) This commit introduces a new package `stream` in `tavern/portals` which provides utilities for handling ordered streams of `portalpb.Mote` messages. Key features: - `payloadSequencer`: Handles atomic sequence ID generation and mote creation. - `OrderedWriter`: Wraps a sender function (like a gRPC stream Send) to automatically sequence and write messages. - `OrderedReader`: Wraps a receiver function (like a gRPC stream Recv) to reorder incoming messages, handling out-of-order delivery with configurable buffering and stale stream detection. This package is designed to support both client and server sides of the portal stream. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor reader to use functional API * Implement PubSub Multiplexer (Mux) (#1466) * Implement PubSub Multiplexer (Mux) for Portals - Created `Mux` package in `tavern/internal/portals/mux`. - Implemented dual-mode operation: In-Memory (for dev) and GCP PubSub (for prod). - Implemented `CreatePortal` and `OpenPortal` lifecycle methods with resource provisioning. - Implemented `Publish` and `Subscribe` logic with local broadcasting (fast path) and global PubSub (slow path). - Added `HistoryBuffer` for message replay. - Added intelligent topic caching to handle `mempubsub` quirks and improve performance. - Added Prometheus metrics for observability. - Verified with comprehensive unit tests using `enttest` and `mempubsub`. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used `sync.RWMutex` for thread safety (`activeSubs`, `subscribers`, `histMu`). - **Observability:** Added Prometheus metrics for message counts. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, and Portal opening. * Refactor Mux and Address PR Feedback - **Structure:** Refactored `Mux` to use composed structs (`SubscriptionManager`, `SubscriberRegistry`, `TopicManager`, `HistoryManager`) for clearer locking semantics. - **Safety:** - Handled race conditions in `CreatePortal` and `OpenPortal` by re-checking state after lock acquisition. - Updated `teardown` logic to use `client.Portal.UpdateOneID` for reliability. - Standardized on `context.Background()` for shutdown operations to prevent context leaks. - **Features:** - Added `WithSubscriberBufferSize` to configure channel buffers. - Added `WithHistoryReplay` option to `Subscribe` for optional history. - Added `mux_messages_dropped_total` metric. - **Concurrency:** Moved global lock handling into granular manager structs to reduce contention. - **Correctness:** Fixed `CreatePortal` to use task-based lookup for dependencies and removed invalid `portalID` parameter usage in logic (though kept signature for now as per instructions). Tests passed. * Implement PubSub Multiplexer (Mux) for Portals - **Mux Package:** Created `tavern/internal/portals/mux` to route messages between local streams and global PubSub. - **Dual-Mode Operation:** Implemented support for In-Memory (dev) and GCP PubSub (prod) drivers via `gocloud.dev/pubsub`. - **Portal Lifecycle:** Added `CreatePortal` and `OpenPortal` methods managing resource provisioning, database records, and subscription lifecycles. - **Message Routing:** Implemented `Publish` (fast-path local dispatch, slow-path global send) and `Subscribe` (local channel registration). - **History Management:** Added `HistoryBuffer` in `history.go` for message replay to new subscribers. - **Concurrency & Robustness:** - Handled race conditions in shared subscription creation (`OpenPortal`). - Managed `receiveLoop` lifecycle using stored `cancelFuncs`. - Added `AlreadyExists` error handling for resource creation. - Used composed structs (`SubscriptionManager`, `SubscriberRegistry`) for granular locking. - **Observability:** Added Prometheus metrics for message counts and dropped messages. - **Testing:** Comprehensive unit tests covering In-Memory flow, Portal creation, Portal opening, and Benchmarks. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * both grpc endpoints * stub out portal-stream package * Implement portal-stream crate (#1470) * Implement portal-stream crate with sequencer, reader, and writer logic - Added `implants/lib/portals/portal-stream` crate. - Implemented `PayloadSequencer` for atomic sequence ID generation. - Implemented `OrderedReader` for reordering incoming messages with timeout and buffer handling. - Implemented `OrderedWriter` for sequencing outgoing messages. - Added comprehensive unit tests for all components. - Added crate to `implants` workspace. * Switch to anyhow for error handling in portal-stream - Replaced `thiserror` with `anyhow` in `portal-stream`. - Updated `Cargo.toml` to use `anyhow` from workspace. - Updated `reader.rs` and tests to use `anyhow::Result` and `anyhow!`. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement SOCKS5 Proxy using tavern/portals/stream (#1467) * Implement SOCKS5 proxy with gRPC tunneling - Added bin/socks5/proxy.go implementing a SOCKS5 proxy server. - Implemented tunneling over stream.OrderedWriter/Reader. - Supported TCP CONNECT and UDP ASSOCIATE commands. - Implemented robust lifecycle management and cleanup. - Added benchmarks in bin/socks5/proxy_test.go demonstrating high throughput. * Address PR comments: Refactor writes, defaults, and shutdown tracking - Refactored raw `conn.Write` calls into named helper functions. - Changed default upstream port to 8000. - Added `sync.WaitGroup` to track connection lifecycle. - Added logging for dropped motes in dispatcher. - Defined `maxStreamBufferedMessages` constant. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement Portal Infrastructure in ImixV2 (#1471) * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. * Implement portal infrastructure in imixv2 - Added `portal-stream` dependency to `imixv2`. - Updated `Transport` trait to include `create_portal` (async). - Implemented `create_portal` in `grpc` transport. - Updated `Agent` trait to include `create_portal`. - Created `imixv2/src/portal/` module with TCP, UDP, and Bytes support using `portal-stream`. - Implemented `create_portal` in `ImixAgent`. - Exposed `create_portal` via `eldritch-libpivot`. - Updated `portal-stream` to support async writers. - Updated `run_create_portal` to send initial registration message. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * some changes * Add integration test * Added TRACE Bytes kind * added trace protos * Implement Trace Motes for Portals (#1473) * feat: implement end-to-end trace motes Implements application-level tracing for Portals infrastructure using the new `tracepb` definitions. * **CLI (`bin/socks5`)**: * Added `trace` subcommand to generate trace motes, send them to the server, and print a latency report. * Refactored `proxy.go` to support subcommands. * Added `addTraceEvent` helper for modifying trace motes. * **Server (`tavern`)**: * Instrumented `api_open_portal.go` and `api_create_portal.go` to inject trace events at key checkpoints (Recv, Pub, Sub, Send). * Created `trace_helper.go` to share event injection logic. * **Agent (`imixv2`)**: * Updated `run.rs` to intercept `BYTES_PAYLOAD_KIND_TRACE` motes. * Implemented logic to add `AGENT_RECV` and `AGENT_SEND` events and immediately echo the mote back. * added retry to trace --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * update flag to --portal * update buf size * Update portal API to send keepalive motes (#1472) - Added keepalive ticker to sendPortalInput loop in api_create_portal.go - Sends a BYTES_PAYLOAD_KIND_KEEPALIVE mote at regular intervals - Prevents connection timeouts similar to the reverse shell implementation Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * add tokio-console support for imixv2 * Fix SOCKS5 proxy cold start hang by avoiding BiLock on TcpStream (#1476) Replaced `tokio::io::split(stream)` with `stream.into_split()` in `implants/imixv2/src/portal/tcp.rs`. The former uses a `BiLock` which can cause deadlocks when the read and write halves are accessed concurrently in separate tasks, specifically causing the "cold start" hang where the initial payload might be blocked. `into_split()` returns owned halves that operate independently. Added a regression test `implants/imixv2/src/tests/repro_issue.rs` to verify the fix. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * socks proxy sends registration message now * socks5 trace must send registration message * Add E2E Portals Workflow and Playwright Test (#1478) Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> * cargo fmt * remove spammy log line * fix some tests * add create_portal fake impl * revert some changes to grpc.rs * remove grpc explicit sizes * update socks5 proxy to support auth and portals to support gcp pubsub * oops * minor cleanup * fix tests * Fix e2e workflow * use env var for auth * wait for socks to start before continuing * Add benchmark tests for cryptocodec (#1485) Added a new test file `tavern/internal/cryptocodec/cryptocodec_bench_test.go` to measure the throughput of `Encrypt` and `Decrypt` methods in `CryptoSvc`. Includes benchmarks for both encryption and decryption operations using standard payload sizes. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix race condition in TestPortalIntegration (#1487) The `TestPortalIntegration` test was flaky and often hung because the Agent would publish messages to the portal before the User had successfully subscribed to the output topic. This resulted in messages being dropped (as evidenced by "message sent to topic with no subscribers" warnings from the in-memory pubsub) and the User reader blocking indefinitely. This commit replaces the arbitrary `time.Sleep` with a deterministic synchronization mechanism. The User now sends a "ping" message to the Agent immediately after opening the portal. The Agent waits to receive this ping before proceeding. This ensures that the User's portal connection (and thus the underlying pubsub subscription) is fully established before the Agent attempts to send any data. This reduces test execution time and eliminates the race condition. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix proxy upstream address parsing * minor fixes & cleanup * fix collapsible if * cargo fmt * fix portal workflow * Add Tavern User Guide for Authentication and API Token (#1501) * Add documentation for TAVERN_API_TOKEN and portal auth flow This commit adds a new user guide page for Tavern (`docs/_docs/user-guide/tavern.md`) detailing the purpose of `TAVERN_API_TOKEN`. It clarifies the distinction between this token and the web OAuth token and explains the "portal auth flow" for users SSH'd into remote environments (e.g., Kali VMs) where standard auth port forwarding is not feasible. * Update TAVERN_API_TOKEN docs: remove non-existent Portal Auth Flow Per code review feedback, the "Portal Auth Flow" feature does not exist yet. This commit removes that section from the documentation, leaving the explanation of what the token is and when to use it (SSH scenarios). --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * cleanup * Merge origin/main, resolve conflicts, and fix go generate (#1515) - Merged changes from origin/main, resolving conflicts in app.go, server.go, and mock transport. - Updated `golang.org/x/tools` to fix `ent` generation failure ("context without types"). - Re-ran `go generate ./...` to update generated protobuf and ent code. - Fixed compilation errors in tests due to renamed protobuf enum constants (ActiveTransport_TRANSPORT_HTTP1). Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest * Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Squidli <32434695+cmp5987@users.noreply.github.com> Co-authored-by: Hulto <7121375+hulto@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> commit f74a1fb Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Sat Jan 3 13:42:00 2026 -0500 All the polling (#1508) * feat(#1207): Add global polling * fix(ui): Add polling visuals * fix(test): Add testing and rebuild * fix(unused): Remove unused code commit a456e40 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 21:51:52 2026 -0500 Add Link entity and update CDN to use link-based file access (#1444) * Add Link entity and update CDN to use link-based file access This commit implements a new Link entity system for the CDN: - Add Link entity schema with: - path (unique, default UUIDv4) - active_before (timestamp, default epoch 0) - active_clicks (int, default 0) - edge to File entity (one-to-many relationship) - Update CDN upload handler to: - Create a new Link entity for each uploaded file - Return both file ID and link path in response - Add new CDN link download handler to: - Serve files using Link entity path instead of file name - Check active_clicks and active_before before serving - Decrement active_clicks when file is served - Return 404 if link is not active - Replace CDN route to use link-based downloads: - Changes /cdn/ endpoint from file name access to link path access - Removes ability to serve tome assets via direct file name - Maintains FetchAsset gRPC API for agent communication - Update dependencies: - Upgrade entgo.io/ent from v0.14.1 to v0.14.5 - Update related dependencies (atlas, sqlite3, tools) * go generate * Add mutations * Clarify docs * typeable random short string * Don't create link on upload * Resolve feedback --------- Co-authored-by: Claude <noreply@anthropic.com> commit ac48265 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Fri Jan 2 21:38:22 2026 -0500 Add offline filters (#1505) * feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest commit e9db24d Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 19:50:01 2026 -0500 Add host timing (#1502) commit 981df36 Author: KCarretto <Kcarretto@gmail.com> Date: Fri Jan 2 15:45:22 2026 -0500 fix ping message handling (#1500) commit c16a3c1 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Fri Jan 2 13:09:32 2026 -0500 Pb active transport (#1464) * Manual updates * go generated * Type * clippy isnt angry * todo * generate * Add extra field * Update docs. * Cleanup bad unwraps * Add extra to config and pass it through. * Fix proxy uri name. * Remove proxy_uri ref * Update docs. * Address PR review comments for active transport refactor (#1490) * Cleanup todos * go generate * fix e2e test * fmt commit 7115020 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:36:12 2026 -0500 fix(graphql): fix foreign key violation in dropAllData (#1495) * fix(graphql): fix foreign key violation in dropAllData Reorder deletion steps in `DropAllData` mutation to ensure child entities (HostCredential, HostFile, HostProcess) are deleted before their parents (Task, Beacon, Host). Add missing `HostCredential` deletion. Add a regression test case `foreign_key.yml` to verify the fix. * fix(graphql): fix foreign key violation in dropAllData Reorder deletion steps in `DropAllData` mutation to ensure child entities (HostCredential, HostFile, HostProcess) are deleted before their parents (Task, Beacon, Host). Add missing `HostCredential` deletion. Add a regression test case `foreign_key.yml` to verify the fix. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> commit 92961af Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 17:23:46 2026 +0000 Fix imixv2 interval sync issue (#1494) * Fix: Force immediate server sync on interval change in imixv2 Previously, when the agent's callback interval was increased (e.g., 5s -> 60s), the agent would update its local sleep timer but wouldn't notify the server until the *next* callback. This caused the server to mark the beacon as dead/offline because it expected a check-in based on the old, shorter interval. This change modifies `set_callback_interval` to trigger an immediate `claim_tasks` call after updating the configuration. This ensures the server receives the updated beacon metadata (including the new interval) right away. Any tasks retrieved during this forced check-in are spawned for execution to prevent task loss. * Refactor claim_tasks logic into ImixAgent::process_job_request Per PR review feedback, the logic for claiming and spawning tasks has been moved from the `set_callback_interval` method and the standalone `process_tasks` function (in `run.rs`) into a unified `process_job_request` method on `ImixAgent`. This reduces code duplication and ensures consistent behavior for task processing. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> commit 5182fe0 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 17:22:46 2026 +0000 Fix mixed Int/Float comparisons in eldritch-core (#1493) * Fix mixed Int/Float comparisons in eldritch-core Previously, mixed type comparisons (e.g., `Int` vs `Float`) fell back to ordering by type discriminant because they are different types. This caused `max(1, 0.8)` to return `0.8` because `Int` (2) < `Float` (3). This change adds a special case in `Value::cmp_helper` to handle `Int` vs `Float` and `Float` vs `Int` by converting the integer to a float and using `total_cmp`. This ensures numerical correctness for `max`, `min`, and general comparisons. Added regression test `tests/max_min_mixed.rs`. * Fix mixed Int/Float comparisons in eldritch-core Previously, mixed type comparisons (e.g., `Int` vs `Float`) fell back to ordering by type discriminant because they are different types. This caused `max(1, 0.8)` to return `0.8` because `Int` (2) < `Float` (3). This change adds a special case in `Value::cmp_helper` to handle `Int` vs `Float` and `Float` vs `Int` by converting the integer to a float and using `total_cmp`. This ensures numerical correctness for `max`, `min`, and general comparisons. Added regression test `tests/max_min_mixed.rs`. Ran `cargo fmt` to ensure code style compliance. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com> commit b84d92f Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:18:04 2026 -0500 Configure CI for GitHub Merge Queues (#1489) commit b437b87 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 12:09:29 2026 -0500 [AI] Test Coverage Expansion and Refinement (#1491) commit 651e377 Author: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri Jan 2 10:48:42 2026 -0500 Implement cursor context finder for LSP (#1439) commit beb1cab Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Thu Jan 1 22:50:13 2026 -0500 Cp add more filters (#1488) * fix(#979): Expand filtering on create quest for tome * fix(transport): Add transport filter and field tag * fix(enums): Add enum handling commit 9473a13 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Thu Jan 1 18:05:24 2026 -0500 Activate previously ignored tests (#1465) Remove #[ignore] attributes from three Rust tests that were previously disabled: - test_sys_exec_current_user: Tests exec functionality with current user - test_sys_exec_disown_no_defunct: Tests disowned processes don't become defunct - test_netstat_integration: Tests netstat functionality on systems with /proc All tests now pass successfully. Co-authored-by: Claude <noreply@anthropic.com> commit 3923f09 Author: Squidli <32434695+cmp5987@users.noreply.github.com> Date: Thu Jan 1 15:04:41 2026 -0500 Cp fix small bugs (#1483) * fix(package): Fix package * fix(filter): Fix create quest filter by hostname or beacon * fix(bug): Fix spelling and submit warning * fix(spelling): Fix spelling issues * fix(build): Run npm build commit f22e181 Author: Hulto <7121375+hulto@users.noreply.github.com> Date: Thu Jan 1 14:12:14 2026 -0500 Add diverse transport types to test data (#1482) * Add diverse Transport types to test data

KCarretto and others added 25 commits December 29, 2025 18:10

initial stubbing and renaming

2a04f9d

refactor reader to use functional API

6b41efb

both grpc endpoints

8745594

stub out portal-stream package

25f4eff

some changes

4e52d28

Add integration test

f0e8127

Added TRACE Bytes kind

43e7d8c

added trace protos

a900e0d

update flag to --portal

0d6ee4b

update buf size

d73cb9c

add tokio-console support for imixv2

95b9d81

socks proxy sends registration message now

23922de

socks5 trace must send registration message

d7c646e

Add E2E Portals Workflow and Playwright Test (#1478)

787ff97

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>

Merge branch 'main' into portals

2c0ad1d

cargo fmt

5c73af4

remove spammy log line

a19363f

KCarretto added 4 commits January 1, 2026 21:44

fix some tests

90e5515

add create_portal fake impl

a0d29c1

revert some changes to grpc.rs

cf7d1ca

remove grpc explicit sizes

50c579a

KCarretto added 7 commits January 2, 2026 04:01

fix proxy upstream address parsing

3eacb1a

minor fixes & cleanup

05f27ca

fix collapsible if

00aa327

cargo fmt

120fa78

fix portal workflow

2aeab33

Merge branch 'main' into portals

7dd2320

Merge branch 'main' into portals

0e46ab1

hulto reviewed Jan 3, 2026

View reviewed changes

google-labs-jules bot and others added 8 commits January 2, 2026 19:55

Merge branch 'main' into portals

e387801

cleanup

c19d939

Add offline filters (#1505)

121a36b

* feat(online-offline): Online Offline in progress * fix(online-offline): Change text * fix(query): Simplify query * fix(build): Build UI * fix(create-quest): Hide filter from create quest

merge origin main again

2d379a1

Merge branch 'main' into portals

985dc9f

hulto approved these changes Jan 3, 2026

View reviewed changes

KCarretto added this pull request to the merge queue Jan 3, 2026

Merged via the queue into main with commit 605e830 Jan 3, 2026
8 checks passed

KCarretto deleted the portals branch January 3, 2026 19:28

[feature] Portals! #1484

[feature] Portals! #1484

Uh oh!

Conversation

KCarretto commented Jan 1, 2026

Uh oh!

github-actions bot commented Jan 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Previous Results

Insights

Slowest Tests

Uh oh!

hulto commented Jan 2, 2026

Uh oh!

hulto commented Jan 2, 2026

Uh oh!

KCarretto commented Jan 2, 2026

Uh oh!

KCarretto commented Jan 2, 2026

Uh oh!

hulto commented Jan 2, 2026

Uh oh!

hulto commented Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hulto commented Jan 3, 2026

Uh oh!

hulto left a comment

Choose a reason for hiding this comment

Uh oh!

hulto Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

KCarretto Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

hulto Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

KCarretto Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Jan 1, 2026 •

edited

Loading

hulto commented Jan 2, 2026 •

edited

Loading