XS✔ ◾ feat: add backup sidecar with restic for scheduled snapshots

ACKNOWLEDGMENTS.md

@@ -15,3 +15,11 @@ Typesense is an open-source, typo-tolerant search engine optimized for instant s
 - **Repository:** [imatefx/typesense-prometheus-exporter](https://github.com/imatefx/typesense-prometheus-exporter)
 
 Provides Prometheus metrics for Typesense server instances. Used as an optional sidecar container when `metrics.enabled` is set to `true`.
+
+## Restic
+
+- **Project:** [Restic](https://restic.net)
+- **Repository:** [restic/restic](https://github.com/restic/restic)
+- **License:** BSD-2-Clause
+
+A fast, secure, and efficient backup program. Used as the backup sidecar image for scheduled Typesense data snapshots when `backup.enabled` is set to `true`.

Changelog.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 1.1.0 ![AppVersion: 30.1](https://img.shields.io/static/v1?label=AppVersion&message=30.1&color=success&logo=) ![Helm: v3](https://img.shields.io/static/v1?label=Helm&message=v3&color=informational&logo=helm)
+
+**Release date:** 2026-03-19
+
+### Added
+
+- Backup sidecar with restic for scheduled S3 snapshots
+- Backup scripts configmap with entrypoint and backup logic
+- Pod-0-only backup enforcement
+- Configurable cron schedule, retention policy, and restic image
+- Extra env/envFrom support for backup credentials
+
 ## 1.0.0 ![AppVersion: 30.1](https://img.shields.io/static/v1?label=AppVersion&message=30.1&color=success&logo=) ![Helm: v3](https://img.shields.io/static/v1?label=Helm&message=v3&color=informational&logo=helm)
 
 **Release date:** 2026-03-19

Chart.yaml

@@ -5,7 +5,7 @@ description: >-
   Prometheus metrics, and Gateway API support. This chart is not officially
   maintained by or affiliated with the Typesense project.
 type: application
-version: 1.0.0
+version: 1.1.0
 appVersion: "30.1"
 icon: https://typesense.org/typesense-logo.svg
 home: https://github.com/hackthebox/typesense-helm
@@ -37,6 +37,10 @@ annotations:
       image: typesense/typesense:30.1
     - name: metrics-exporter
       image: imatefx/typesense-prometheus-exporter:v0.1.5
+    - name: backup
+      image: ghcr.io/restic/restic:0.18.1
   artifacthub.io/changes: |
     - kind: added
-      description: Initial open-source release
+      description: Backup sidecar with restic for scheduled S3 snapshots
+    - kind: added
+      description: Backup scripts configmap with entrypoint and backup logic

README.md

@@ -1,6 +1,6 @@
 # typesense
 
-![Version: 1.0.0](https://img.shields.io/badge/Version-1.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 30.1](https://img.shields.io/badge/AppVersion-30.1-informational?style=flat-square)
+![Version: 1.1.0](https://img.shields.io/badge/Version-1.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 30.1](https://img.shields.io/badge/AppVersion-30.1-informational?style=flat-square)
 
 [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/typesense)](https://artifacthub.io/packages/search?repo=typesense)
 
@@ -16,9 +16,11 @@ A production-ready Helm chart for deploying [Typesense](https://typesense.org),
 - **Security Hardened** -- Non-root container (UID 10000), read-only root filesystem, all capabilities dropped
 - **Gateway API & Ingress** -- Supports both Kubernetes Ingress and Gateway API (HTTPRoute)
 - **Prometheus Metrics** -- Optional sidecar exporter with ServiceMonitor support
+- **Automated Backups** -- Optional restic-based backup sidecar with cron scheduling and retention policies
 - **External Secrets** -- Optional integration with the External Secrets Operator
 - **PodDisruptionBudget** -- Protect Raft quorum during node maintenance
 - **Comprehensive Tuning** -- CORS, analytics, cache, thread pools, logging, health thresholds
+- **Battle-tested** -- Actively used in production on AWS EKS with S3-based backups
 
 ## TL;DR
 
@@ -213,11 +215,55 @@ storage:
 
 > **Warning:** Reducing `storage.size` after initial deployment has no effect. PVC resize depends on your StorageClass supporting volume expansion.
 
+### Backup
+
+The chart includes an optional backup sidecar that uses [restic](https://restic.net) to take scheduled snapshots of Typesense data and upload them to a remote repository (S3, GCS, Azure Blob, etc.).
+
+```yaml
+backup:
+  enabled: true
+  intervalSeconds: 86400       # every 24 hours
+  retention:
+    keepLast: 7
+  envFrom:
+    - secretRef:
+        name: restic-credentials
+```
+
+The `restic-credentials` Secret should contain `RESTIC_REPOSITORY`, `RESTIC_PASSWORD`, and any cloud provider credentials (e.g., `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`).
+
+For quick testing, you can pass credentials inline (not recommended for production):
+
+```yaml
+backup:
+  enabled: true
+  env:
+    - name: RESTIC_REPOSITORY
+      value: s3:s3.amazonaws.com/my-bucket/typesense
+    - name: RESTIC_PASSWORD
+      value: my-restic-password
+```
+
+Key details:
+- Backups only run on **pod-0** (other pods skip automatically)
+- The restic repository is initialized automatically on the first run
+- The backup sidecar runs as non-root (UID 10000) with all capabilities dropped
+- See [RESTORE_RUNBOOK.md](RESTORE_RUNBOOK.md) for disaster recovery procedures
+
 ## Values
 
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
 | affinity | object | `{}` | Affinity rules for pod scheduling |
+| backup.enabled | bool | `false` | Enable the backup sidecar. Requires backup.image and restic env vars to be configured. |
+| backup.env | list | `[]` | Extra env vars for the backup sidecar (e.g. RESTIC_REPOSITORY, AWS_ACCESS_KEY_ID) |
+| backup.envFrom | list | `[]` | References to existing Secrets/ConfigMaps to inject into the backup sidecar |
+| backup.image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
+| backup.image.repository | string | `"ghcr.io/restic/restic"` | Backup sidecar image. Defaults to the official restic image which includes wget and crond via BusyBox. |
+| backup.image.tag | string | `"0.18.1"` | Image tag |
+| backup.intervalSeconds | int | `86400` | Interval in seconds between backup runs (default: 86400 = 24h) |
+| backup.resources | object | `{"requests":{"cpu":"10m","memory":"32Mi"}}` | Resource requests and limits for the backup sidecar |
+| backup.retention.keepLast | int | `7` | Number of most recent restic snapshots to keep |
 | extraArgs | list | `[]` | Extra command-line arguments for Typesense server (e.g., ["--filter-by-max-ops=200"]) |
 | extraEnv | list | `[]` | Extra environment variables for the Typesense container |
 | fullnameOverride | string | `""` | Override the full name of the release (optional) |

README.md.gotmpl

@@ -18,9 +18,11 @@ A production-ready Helm chart for deploying [Typesense](https://typesense.org),
 - **Security Hardened** -- Non-root container (UID 10000), read-only root filesystem, all capabilities dropped
 - **Gateway API & Ingress** -- Supports both Kubernetes Ingress and Gateway API (HTTPRoute)
 - **Prometheus Metrics** -- Optional sidecar exporter with ServiceMonitor support
+- **Automated Backups** -- Optional restic-based backup sidecar with cron scheduling and retention policies
 - **External Secrets** -- Optional integration with the External Secrets Operator
 - **PodDisruptionBudget** -- Protect Raft quorum during node maintenance
 - **Comprehensive Tuning** -- CORS, analytics, cache, thread pools, logging, health thresholds
+- **Battle-tested** -- Actively used in production on AWS EKS with S3-based backups
 
 ## TL;DR
 
@@ -215,6 +217,41 @@ storage:
 
 > **Warning:** Reducing `storage.size` after initial deployment has no effect. PVC resize depends on your StorageClass supporting volume expansion.
 
+### Backup
+
+The chart includes an optional backup sidecar that uses [restic](https://restic.net) to take scheduled snapshots of Typesense data and upload them to a remote repository (S3, GCS, Azure Blob, etc.).
+
+```yaml
+backup:
+  enabled: true
+  intervalSeconds: 86400       # every 24 hours
+  retention:
+    keepLast: 7
+  envFrom:
+    - secretRef:
+        name: restic-credentials
+```
+
+The `restic-credentials` Secret should contain `RESTIC_REPOSITORY`, `RESTIC_PASSWORD`, and any cloud provider credentials (e.g., `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`).
+
+For quick testing, you can pass credentials inline (not recommended for production):
+
+```yaml
+backup:
+  enabled: true
+  env:
+    - name: RESTIC_REPOSITORY
+      value: s3:s3.amazonaws.com/my-bucket/typesense
+    - name: RESTIC_PASSWORD
+      value: my-restic-password
+```
+
+Key details:
+- Backups only run on **pod-0** (other pods skip automatically)
+- The restic repository is initialized automatically on the first run
+- The backup sidecar runs as non-root (UID 10000) with all capabilities dropped
+- See [RESTORE_RUNBOOK.md](RESTORE_RUNBOOK.md) for disaster recovery procedures
+
 {{ template "chart.valuesSection" . }}
 
 ## Upgrading

templates/backup-scripts-configmap.yaml

@@ -0,0 +1,77 @@
+{{- if .Values.backup.enabled }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "typesense.fullname" . }}-backup-scripts
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "typesense.labels" . | nindent 4 }}
+data:
+  entrypoint.sh: |
+    #!/bin/sh
+    set -e
+
+    INTERVAL="${BACKUP_INTERVAL_SECONDS}"
+
+    echo "[backup-sidecar] Starting with interval: ${INTERVAL}s"
+    echo "[backup-sidecar] First backup will run in ${INTERVAL}s"
+
+    trap 'echo "[backup-sidecar] Shutting down"; exit 0' TERM INT
+
+    while true; do
+      sleep "${INTERVAL}" &
+      SLEEP_PID=$!
+      wait $SLEEP_PID || exit 0
+      /scripts/backup.sh || echo "[backup-sidecar] Backup failed, will retry in ${INTERVAL}s"
+    done
+
+  backup.sh: |
+    #!/bin/sh
+    set -eu
+
+    SNAPSHOT_BASE="/usr/share/typesense/data/snapshots"
+    TIMESTAMP=$(date +%Y%m%dT%H%M%S)
+    SNAPSHOT_DIR="${SNAPSHOT_BASE}/${TIMESTAMP}"
+    KEEP_LAST="${BACKUP_KEEP_LAST}"
+
+    # Only pod-0 runs the backup
+    ORDINAL=$(hostname | awk -F'-' '{print $NF}')
+    if [ "${ORDINAL}" != "0" ]; then
+      echo "[backup] Skipping: pod ordinal ${ORDINAL} is not 0"
+      exit 0
+    fi
+
+    echo "[backup] Starting backup at ${TIMESTAMP}"
+
+    # Clean up local snapshot dir on exit (success or failure)
+    trap 'echo "[backup] Cleaning up ${SNAPSHOT_DIR}"; rm -rf "${SNAPSHOT_DIR}" 2>/dev/null || true' EXIT
+
+    # Initialize restic repository if it does not exist yet
+    echo "[backup] Initializing restic repo if needed..."
+    restic snapshots --quiet > /dev/null 2>&1 || restic init
+
+    # Remove stale restic locks from previous crashed runs
+    echo "[backup] Unlocking restic repo..."
+    restic unlock 2>/dev/null || true
+
+    # Trigger Typesense snapshot
+    echo "[backup] Triggering Typesense snapshot to ${SNAPSHOT_DIR}..."
+    RESULT=$(wget -q -O - \
+      --header="X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
+      --post-data="" \
+      "http://$(hostname -i):${TYPESENSE_PORT}/operations/snapshot?snapshot_path=${SNAPSHOT_DIR}")
+    echo "[backup] Snapshot result: ${RESULT}"
+
+    # Validate snapshot succeeded
+    echo "${RESULT}" | grep -q '"success":true' || { echo "[backup] Snapshot failed: ${RESULT}"; exit 1; }
+
+    # Upload snapshot to restic repository
+    echo "[backup] Running restic backup..."
+    restic backup "${SNAPSHOT_DIR}" --tag "typesense" --tag "${TIMESTAMP}"
+
+    # Enforce retention policy (trap handles local cleanup)
+    echo "[backup] Pruning old snapshots, keeping last ${KEEP_LAST}..."
+    restic forget --keep-last "${KEEP_LAST}" --tag "typesense" --prune
+
+    echo "[backup] Backup complete."
+{{- end }}

templates/statefulset.yaml

@@ -140,7 +140,59 @@ spec:
               containerPort: {{ .Values.metrics.port }}
           resources: {{- toYaml .Values.metrics.resources | nindent 12 }}
         {{- end }}
+        {{- if .Values.backup.enabled }}
+        - name: backup-sidecar
+          image: "{{ .Values.backup.image.repository }}:{{ .Values.backup.image.tag }}"
+          imagePullPolicy: {{ .Values.backup.image.pullPolicy }}
+          command: ["/bin/sh", "/scripts/entrypoint.sh"]
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            runAsNonRoot: true
+            runAsUser: 10000
+            runAsGroup: 3000
+            capabilities:
+              drop:
+                - ALL
+          env:
+            - name: BACKUP_INTERVAL_SECONDS
+              value: {{ .Values.backup.intervalSeconds | quote }}
+            - name: BACKUP_KEEP_LAST
+              value: {{ .Values.backup.retention.keepLast | quote }}
+            - name: TYPESENSE_PORT
+              value: {{ .Values.service.port | quote }}
+            - name: RESTIC_CACHE_DIR
+              value: /tmp/.cache/restic
+            {{- with .Values.backup.env }}
+            {{- toYaml . | nindent 12 }}
+            {{- end }}
+          envFrom:
+            - secretRef:
+                name: {{ .Values.secrets.secretName | default (printf "%s-secret" (include "typesense.fullname" .)) }}
+                optional: {{ .Values.secrets.optional }}
+            {{- with .Values.backup.envFrom }}
+            {{- toYaml . | nindent 12 }}
+            {{- end }}
+          volumeMounts:
+            - name: {{ include "typesense.fullname" . }}-data
+              mountPath: /usr/share/typesense/data
+            - name: backup-scripts
+              mountPath: /scripts
+            - name: backup-tmp
+              mountPath: /tmp
+          resources: {{- toYaml .Values.backup.resources | nindent 12 }}
+        {{- end }}
+
       volumes:
+        {{- if .Values.backup.enabled }}
+        - name: backup-scripts
+          configMap:
+            name: {{ include "typesense.fullname" . }}-backup-scripts
+            defaultMode: 0755
+        - name: backup-tmp
+          emptyDir: {}
+        {{- end }}
+
         - name: nodeslist
           configMap:
             name: {{ include "typesense.fullname" . }}-nodeslist