Create standalone OpenTelemetry stack and make some minor fixes

docker-compose-examples/grafana-monitoring-otel/.gitignore

@@ -2,3 +2,4 @@
 *.deb
 .env
 .DS_Store
+varnish-enterprise.lic

docker-compose-examples/grafana-monitoring-otel/README.md

@@ -3,26 +3,120 @@ Using `opentelemetry` simplifies the process quite a bit and notably introduces
 
 # Getting started
 
+## Standalone OpenTelemetry & Grafana setup
+
+If you want to deploy a standalone OpenTelemetry stack you can point any OTLP-emitting workload at, simply run the following command:
+
+```sh
+docker compose up -d
+```
+
+To tear down the environment, run the following command:
+
+```sh
+docker compose down -v
+```
+
+### Accessing Grafana
+
+Once the containers are healthy, open Grafana at [http://localhost:3000](http://localhost:3000). No credentials are needed, and anonymous access is granted with admin privileges. The Prometheus, Loki and Tempo datasources are already configured under **Connections → Data sources**, and you can use the **Explore** tab to query each backend.
+
+### Sending telemetry data
+
+The following ports are published to the Docker host:
+
+| Service        | Endpoint                                                       | Purpose                  |
+| -------------- | -------------------------------------------------------------- | ------------------------ |
+| Grafana        | [http://localhost:3000](http://localhost:3000)                 | UI                       |
+| OTel Collector | [http://localhost:4318](http://localhost:4318)                 | OTLP/HTTP receiver       |
+| OTel Collector | `localhost:4317`                                               | OTLP/gRPC receiver       |
+| OTel Collector | [http://localhost:8888/metrics](http://localhost:8888/metrics) | Collector self-metrics   |
+| Prometheus     | [http://localhost:9090](http://localhost:9090)                 | Prometheus UI / API      |
+| Loki           | [http://localhost:3100](http://localhost:3100)                 | Loki HTTP API            |
+| Tempo          | [http://localhost:3200](http://localhost:3200)                 | Tempo HTTP API           |
+
+You can send telemetry data to the specific services (Prometheus, Loki & Tempo) by defining the individual `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` & `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` environment variables.
+
+However, there's also an OTel Collector service that captures the different types of telemetry data and forwards that data to the corresponding service (Prometheus, Loki & Tempo). It only requires a single OTLP endpoint and can simply by configured through the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable.
+
+#### From a Docker container on the same compose network
+
+Within this compose network, services can address each other by container name on the internal ports:
+
+- `http://otel-collector:4318` (OTLP/HTTP)
+- `otel-collector:4317` (OTLP/gRPC)
+- `http://grafana:3000`
+- `http://prometheus:9090`
+- `http://loki:3100`
+- `http://tempo:3200`
+
+
+Here's an example:
+
+```sh
+export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
+export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+```
+
+#### From your host system
+
+If you're sending telemetry data from your host system, the port forwarding ensures the exposed services are available via `localhost`.
+
+Here's an example:
+
+```sh
+export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+```
+
+#### From another Docker container
+
+If your workload runs in a container that is not part of this compose project (for example, a separate `docker compose` stack or a one-off `docker run`), it cannot resolve `otel-collector` or use `localhost:4318` directly.
+
+Use `host.docker.internal` to reach the published ports on the host:
+
+```sh
+export OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318
+export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+```
+
 ## Varnish Cache
 
-- run "docker compose up -d"
-- go to http://localhost:3000
-- login with `admin:password`
+If you want to use this OpenTelemetry stack in conjunction with Varnish and automatically generate telemetry data, run the following command:
+
+```sh
+docker compose --profile varnish up -d
+```
 
-You should see the main dashboard, and you can also use the `Explore` tab to discover more metrics, or check the `Varnish logs` dashboard.
+The `load_generator` service will automatically send traffic to Varnish, which means logs, metrics and traces will be available in Grafana.
+
+You can access Grafana on [http://localhost:3000](http://localhost:3000) without the need for any credentials. You should see the main dashboard, and you can also use the `Explore` tab to discover more metrics, or check the `Varnish logs` dashboard.
 
 ![Main dashboard](../../.assets/vc-dashboard.png) ![Main dashboard](../../.assets/logs-dashboard.png)
 
+To tear down the environment, run the following command:
+
+```sh
+docker compose --profile varnish down -v
+```
+
 ## Varnish Enterprise
 
-- place the license file ( `varnish-enterprise.lic`, you can ask for one [here](https://www.varnish-software.com/contact-us/)) in `conf/`. The license should enable both `vmod-otel` and `mse4`.
-- run "docker compose -f compose-plus.yaml up -d"
-- go to http://localhost:3000
-- login with `admin:password`
+If you want to use the OpenTelemetry stack with Varnish Enterprise instead, follow these instructions:
+
+- Place the license file ( `varnish-enterprise.lic`, you can ask for one [here](https://www.varnish-software.com/contact-us/)) in `conf/`. The license should enable both `vmod-otel` and `mse4`.
+- Run `docker compose -f compose-enterprise.yaml --profile varnish up -d`
+- Access Grafana via [http://localhost:3000](http://localhost:3000)
 
 As for the Cache option, you will land on the main dashboard, but you should check the `Varnish Enterprise Metrics` dashboard which offer more in-depth metrics and support for backend health.
 
 ![Main dashboard](../../.assets/ve-dashboard.png) ![Main dashboard](../../.assets/ve-node-graph.png)
 ![Main dashboard](../../.assets/ve-trace.png) ![Main dashboard](../../.assets/ve-service-graph.png)
 
 The `Explore` tab will allow you to check on `Tempo` and traces, notably the service graph that gets generated automatically by the traces.
+
+To tear down the environment, run the following command:
+
+```sh
+docker compose --profile varnish down -v
+```

docker-compose-examples/grafana-monitoring-otel/compose-common.yaml

@@ -1,6 +1,7 @@
 services:
   load_generator:
     image: curlimages/curl
+    profiles: [varnish]
     volumes:
       - ./src/load_generator.sh:/usr/bin/load_generator
     command: /usr/bin/load_generator
@@ -9,12 +10,11 @@ services:
 
   origin-files: &origin
     command: /app/main origin-files
+    profiles: [varnish]
     volumes:
       - ./data:/orig_data
     environment: &otel_env
-      OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: http://loki:3100/otlp/v1/logs
-      OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: http://tempo:4318/v1/traces
-      OTEL_EXPORTER_OTLP_METRICS_ENDPOINT: http://prometheus:9090/api/v1/otlp/v1/metrics
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
     build:
       dockerfile_inline: |
         FROM golang
@@ -28,28 +28,68 @@ services:
     <<: *origin
     command: /app/main origin-esi
 
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:0.116.1
+    container_name: otel-collector
+    command: ["--config=/etc/otelcol/otel-collector-config.yaml"]
+    volumes:
+      - ./conf/otel-collector-config.yaml:/etc/otelcol/otel-collector-config.yaml:ro
+    ports:
+      - "4318:4318" # OTLP HTTP receiver (exposed to host)
+      - "4317:4317" # OTLP gRPC receiver (also exposed for convenience)
+      - "8888:8888" # Collector self-metrics
+    depends_on:
+      - loki
+      - prometheus
+      - tempo
+
   grafana:
     image: grafana/grafana-enterprise
-    ports:
-      - 3000:3000
+    container_name: grafana
     environment:
-      - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
+      - GF_AUTH_ANONYMOUS_ENABLED=true
+      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
+      - GF_AUTH_DISABLE_LOGIN_FORM=true
+      - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor,metricsSummary
+      - GF_INSTALL_PLUGINS=grafana-exploretraces-app
     volumes:
       - ./conf/grafana/grafana.ini:/etc/grafana/grafana.ini
-      - ./conf/grafana/provisioning:/etc/grafana/provisioning/
+      - ./conf/grafana/provisioning:/etc/grafana/provisioning:ro
+    ports:
+      - "3000:3000"
+    depends_on:
+      - loki
+      - prometheus
+      - tempo
 
   tempo:
-    image: grafana/tempo:latest
-    command: "-config.file=/etc/tempo.yaml"
+    image: grafana/tempo:2.7.0
+    container_name: tempo
+    command: -config.file=/etc/tempo/tempo-config.yaml
     volumes:
-      - ./conf/tempo.yaml:/etc/tempo.yaml
+      - ./conf/tempo-config.yaml:/etc/tempo/tempo-config.yaml:ro
+    ports:
+      - "3200:3200"
 
   prometheus:
-    image: prom/prometheus:v3
+    image: prom/prometheus:v3.1.0
+    container_name: prometheus
+    command:
+      - --config.file=/etc/prometheus/prometheus.yaml
+      - --web.enable-otlp-receiver
+      - --web.enable-remote-write-receiver
+      - --enable-feature=native-histograms
     volumes:
-      - ./conf/prometheus.yml:/etc/prometheus/prometheus.yml
-    command: --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --web.console.libraries=/usr/share/prometheus/console_libraries --web.console.templates=/usr/share/prometheus/consoles --web.enable-otlp-receiver --web.enable-remote-write-receiver --enable-feature=native-histograms
+      - ./conf/prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
+    ports:
+      - "9090:9090"
 
   loki:
-    image: grafana/loki:3.1.0
-    command: -config.file=/etc/loki/local-config.yaml
+    image: grafana/loki:3.3.2
+    container_name: loki
+    command: -config.file=/etc/loki/loki-config.yaml
+    volumes:
+      - ./conf/loki-config.yaml:/etc/loki/loki-config.yaml:ro
+    ports:
+      - "3100:3100"
+

docker-compose-examples/grafana-monitoring-otel/compose-plus.yaml → docker-compose-examples/grafana-monitoring-otel/compose-enterprise.yaml

@@ -16,7 +16,7 @@ services:
       - ./data/mse/disk:/var/lib/mse/disk
       - workdir:/var/lib/varnish
       - ./conf/varnish-enterprise.lic:/etc/varnish/varnish-enterprise.lic
-      - ./conf/default-plus.vcl:/etc/varnish/default.vcl
+      - ./conf/default-enterprise.vcl:/etc/varnish/default.vcl
     command: varnishd -F -a :80 -f /etc/varnish/default.vcl -n varnish -p esi_limit=1 -s mse4,/etc/varnish/mse4.conf
     ports:
       - "8080:80"
@@ -29,18 +29,18 @@ services:
         condition: service_completed_successfully
     image: varnish/varnish-enterprise
     user: root
+    profiles: [varnish]
 
   varnish-otel:
     image: varnish/varnish-enterprise
+    profiles: [varnish]
     command: varnish-otel
     volumes:
       - workdir:/var/lib/varnish
     depends_on:
       - varnish
     environment:
-      OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: http://loki:3100/otlp/v1/logs
-      OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: http://tempo:4318/v1/traces
-      OTEL_EXPORTER_OTLP_METRICS_ENDPOINT: http://prometheus:9090/api/v1/otlp/v1/metrics
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
       OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf
       OTEL_SERVICE_NAME: varnish
       OTEL_VARNISH_WORKDIR: varnish

docker-compose-examples/grafana-monitoring-otel/compose.yaml

@@ -4,10 +4,10 @@ include:
 services:
   varnish:
     image: varnish
+    profiles: [varnish]
     volumes:
       - workdir:/var/lib/varnish
       - ./conf/default.vcl:/etc/varnish/default.vcl
-    command: varnishd -F -a :80 -f /etc/varnish/default.vcl
     ports:
       - "8080:80"
     depends_on:
@@ -18,26 +18,25 @@ services:
 
   varnish-otel:
     image: varnish
+    profiles: [varnish]
     command: varnish-otel
     volumes:
       - workdir:/var/lib/varnish
     depends_on:
       - varnish
     environment:
       OTEL_RESOURCE_ATTRIBUTES: service.instance.id=varnish1
-      OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: http://loki:3100/otlp/v1/logs
-      OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: http://tempo:4318/v1/traces
-      OTEL_EXPORTER_OTLP_METRICS_ENDPOINT: http://prometheus:9090/api/v1/otlp/v1/metrics
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
       OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf
       OTEL_SERVICE_NAME: varnish
-      #OTEL_VARNISH_WORKDIR: varnish
+      OTEL_VARNISH_WORKDIR: varnish
     build:
       dockerfile_inline: |
         FROM varnish
         USER root
         RUN set -ex; \
             apt-get update; \
-            apt-get install varnish-otel; \
+            apt-get install -y varnish-otel; \
             rm -rf /var/lib/apt/lists/*
         USER varnish
 

docker-compose-examples/grafana-monitoring-otel/conf/grafana/grafana.ini

@@ -1,14 +1,3 @@
-[security]
-admin_user = admin
-admin_password = password
-
-#[auth]
-#disable_login_form = true
-#
-#[auth.anonymous]
-#enabled = true
-#org_role = Admin
-
 [dashboards]
 min_refresh_interval = 10s
 default_home_dashboard_path = /etc/grafana/provisioning/dashboards/varnish_metrics.json

docker-compose-examples/grafana-monitoring-otel/conf/grafana/provisioning/dashboards/varnish_logs.json

@@ -90,7 +90,7 @@
       "dashes": false,
       "datasource": {
         "type": "loki",
-        "uid": "lokiUID"
+        "uid": "loki"
       },
       "fieldConfig": {
         "defaults": {
@@ -136,7 +136,7 @@
         {
           "datasource": {
             "type": "loki",
-            "uid": "lokiUID"
+            "uid": "loki"
           },
           "expr": "sum(count_over_time({service_name=~\"$app\"} |= \"$search\" [$__interval]))",
           "legendFormat": "",
@@ -177,7 +177,7 @@
     {
       "datasource": {
         "type": "loki",
-        "uid": "lokiUID"
+        "uid": "loki"
       },
       "gridPos": {
         "h": 25,
@@ -201,7 +201,7 @@
         {
           "datasource": {
             "type": "loki",
-            "uid": "lokiUID"
+            "uid": "loki"
           },
           "expr": "{service_name=~\"$app\"} |= \"$search\" | logfmt",
           "hide": false,
@@ -226,7 +226,7 @@
         },
         "datasource": {
           "type": "loki",
-          "uid": "lokiUID"
+          "uid": "loki"
         },
         "definition": "label_values(service_name)",
         "hide": 0,