LogStream — Real-time Log Aggregator

Self-hosted log aggregation and live tail for small & mid-sized DevOps teams. A pragmatic alternative to Elasticsearch + Kibana when you do not want to operate a JVM cluster.

What it does

LogStream ingests logs from dozens of containers, services, and hosts, indexes them for full-text search, and lets engineers tail a live stream filtered by any combination of source, severity, and structured field. It is small enough to run on a single VPS and fast enough to keep up with 1M+ events per hour.

Capabilities

• Live tail - sub-second stream with severity highlighting, pausable, auto-scroll
• Query language - Lucene-style: level:error AND service:payment-* AND duration_ms>1000
• Source catalog - auto-discovers Docker / Swarm / Kubernetes containers; manual sources via Fluent-Bit / Vector / syslog
• Time-bucketed histograms - volume by severity overlaid on the timeline; click-and-drag to zoom
• Structured field extraction - JSON logs are indexed field-by-field, plain-text uses Grok patterns
• Saved searches & alerts - threshold + anomaly detection on any query, route to Telegram / Slack / webhook
• Trace correlation - click a trace_id in any log to pull every related event across services
• Retention tiering - hot (SSD, full-text) -> warm (compressed) -> cold (object storage) with automatic rollover
• Storage budget - per-source quotas; oldest events dropped first when full
• RBAC - viewer / operator / admin, with per-source scopes
• API - REST query, WebSocket subscribe, bulk export to NDJSON
• Multi-tenant ready - tenant isolation at the index level

Architecture

Layer	Technology
Frontend	React + Vite, Tailwind, virtual-scroll log table
Ingest	Rust pipeline, batched writes, 200k events/sec on 4 cores
Index	Tantivy (full-text) + ClickHouse (analytics & trends)
Storage	Local SSD -> S3-compatible cold storage
Delivery	Single binary OR Docker Compose

Try it

docker run -d \
  --name logstream \
  -p 7000:7000 \
  -v logstream_data:/var/lib/logstream \
  behruzops/logstream:latest

# point your Fluent-Bit / Vector / syslog at tcp://<host>:7001
# open http://localhost:7000  (default login: admin / admin)

Status

Production deployments handling 1.2M events/hour on a single 4-CPU VPS. Open-source for personal use, commercial license required for SaaS resale.

Author

Behruz Shuxratov - observability engineer based in Tashkent. Available for freelance work: log pipelines, SIEM-lite setups, Telegram/Slack alert routing.

• Email: behruzshuxratov58@gmail.com
• GitHub: @behruzops