Skip to content

litewitness: add Prometheus metrics.#71

Open
warpfork wants to merge 1 commit into
FiloSottile:mainfrom
warpfork:main
Open

litewitness: add Prometheus metrics.#71
warpfork wants to merge 1 commit into
FiloSottile:mainfrom
warpfork:main

Conversation

@warpfork
Copy link
Copy Markdown

Metrics! Whee!

Patterns heavily riffed from the sunlight codebase.

Standard go and process stats are collected, and custom stats: these include the number of logs monitored, how large they are, how long requests are taking to service, etc.

The prefix levels on the metrics might be a little high; what I'm following here is the sunlight project's distinction between the command's whole metrics, and the witness component. Right now, the only metrics I've introduced to litewitness are on the witness component (and that sounds doubly redundant due to the project name), but that might not remain the case forever.

I disable the exposure of the metrics endpoint if the obscurity mode flag is engaged. Some metrics do contain log names, and so it matches the same enumeration concern that obscurity mode describes as its purpose.

(Possibly future work: I'm not super satisfied with how sometimes the logging labels and the prometheus labels have parallel evolution, for example in processAddCheckpointRequest. But attempting to unify those feels like a potential portal to frameworkitis, so leaving that as a future itch to scratch. Or, perhaps not.)

Patterns heavily riffed from the sunlight codebase.

Standard go and process stats are collected,
and custom stats: these include the number of logs monitored,
how large they are, how long requests are taking to service, etc.

The prefix levels on the metrics might be a little high;
what I'm following here is the sunlight project's distinction
between the command's whole metrics, and the witness component.
Right now, the only metrics I've introduced to litewitness are on
the witness component (and that sounds doubly redundant due to
the project name), but that might not remain the case forever.

I disable the exposure of the metrics endpoint if the obscurity mode flag is engaged.
Some metrics do contain log names, and so it matches the same enumeration
concern that obscurity mode describes as its purpose.

(Possibly future work: I'm not super satisfied with how sometimes
the logging labels and the prometheus labels have parallel evolution,
for example in `processAddCheckpointRequest`.
But attempting to unify those feels like a potential portal to frameworkitis,
so leaving that as a future itch to scratch.  Or, perhaps not.)
@eliasrudberg
Copy link
Copy Markdown

@warpfork thanks for adding this! I just tested it for one of my test witnesses and it works:

# curl -s http://127.0.0.1:2222/metrics | grep litewitness_witness_log_tree_size
# HELP litewitness_witness_log_tree_size Latest cosigned tree size for each log, by origin.
# TYPE litewitness_witness_log_tree_size gauge
litewitness_witness_log_tree_size{origin="arche2025h1.staging.ct.transparency.dev"} 1.432088055e+09
litewitness_witness_log_tree_size{origin="arche2025h2.staging.ct.transparency.dev"} 2.519545813e+09
litewitness_witness_log_tree_size{origin="arche2026h1.staging.ct.transparency.dev"} 2.835019557e+09
litewitness_witness_log_tree_size{origin="sigsum.org/v1/tree/1643169b32bef33a3f54f8a353b87c475d19b6223cbb106390d10a29978e1cba"} 478422
litewitness_witness_log_tree_size{origin="sigsum.org/v1/tree/4e89cc51651f0d95f3c6127c15e1a42e3ddf7046c5b17b752689c402e773bb4d"} 168273
litewitness_witness_log_tree_size{origin="sigsum.org/v1/tree/e93cb678c04cc52e4da380c4b19cfa14062032a722da254c65cbd95e7e76321d"} 60541

Corresponding issue: #5

@FiloSottile could you have a look and see if this can get merged?

@eliasrudberg
Copy link
Copy Markdown

@FiloSottile what do you think, could we get this merged soon? It would be very useful for us (Glasklar)!

From the perspective of Glasklar's witness deployments, for us it would also have been helpful to be able to serve metrics at localhost even if the witness endpoint that logs use is publicly reachable. @warpfork what do you think about adding an option to allow that? I guess it could be an option in addition to the existing -listen - e.g. -metrics-endpoint and then we could specify the public IP for -listen and localhost for the metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants