Stamp git commit id in the image

[jcchavezs]

As of now we are only relaying on main version of hypertrace-federated-service, however we don't know what image are we actually using because all of them are being called main. Having the ability to stamp the git commit id will clearly state what the image is and it will help on triage for issues like: "docker-compose up stopped working from yesterday to today but nothing changed".

After a research I could not find the way to do this with gradle. Any idea @aaron-steinfeld @adriancole @JBAhire @pavolloffay?

[aaron-steinfeld]

Agreed, this is a good idea. I can work out the specific mechanics of doing this, but how exactly do we want the SHA to present? As an env var like COMMIT so it can be echo'd on startup?

[jcchavezs]

I think a build arg is better as it will show up in docker inspect right and it is part of the image.

[aaron-steinfeld]

I can use a build arg to pass it in, but I guess I'm asking what to do with it next. I could see it as a label, env var, or both. Not sure if there's an expected standard here, a little googling seemed to lead to lots of people all doing it their own way.

ARG COMMIT="unknown"
LABEL commit=${COMMIT}
ENV COMMIT=${COMMIT}

[jcchavezs]

I see @aaron-steinfeld, I think I like the hypertrace-service.commit_sha=${COMMIT} and hypertrace-service.version=${VERSION}. It would be great to have the same for all the other services (like in https://github.com/hypertrace/hypertrace-service/blob/main/settings.gradle.kts#L15) running in this container. Any ideas @kotharironak @JBAhire ?

[kotharironak]

@jcchavezs This is a submodule include - (https://github.com/hypertrace/hypertrace-service/blob/main/settings.gradle.kts#L15) - commits show up in repo which include this submodule.

[jcchavezs]

Yeah so I think we need to include this repo version and all submodule versions.

[aaron-steinfeld]

Hmm - so to support submodules (in the generic sense), we'd ultimately have a Dockerfile generated like:

ARG COMMIT_SHA="unknown"
<for each submodule>
ARG <submodule>.COMMIT_SHA="unknown"
<end>

LABEL commit_sha=${COMMIT_SHA}
ENV COMMIT_SHA=${COMMIT_SHA}
<for each submodule>
LABEL <submodule>.commit_sha=${<submodule>.COMMIT_SHA}
ENV <submodule>.COMMIT_SHA=${<submodule>.COMMIT_SHA}
<end>

My votes would be:

• Hold off on the version for now, as they may not exist if we're building off latest main, and our versions don't really have meaning yet
• Do just the top level commit as a first cut. We don't need git access to do that since the SHA will be in the env, while the submodule info will involve working with git. If this is purely for debugging, the other SHAs (and any versions) are directly discoverable from that with a one liner: git checkout <SHA> && git submodule status. If we then find that it makes things clearer to add the submodule info in directly, we can go down that path.

[jcchavezs]

@aaron-steinfeld sorry for the confusion I did not mean to include version but just to show how I would stablish the naming convention. If we can have the sha for https://github.com/hypertrace/hypertrace-service/blob/main/hypertrace-service/Dockerfile that is already a win.

[kotharironak]

I think we can go with top-level commit sha (not for submodules) as of now as we have another open question on macro rep vs submodules.

Will be modified image tag too? If so, considering, we already have the main branch tag. The final tag will be as below?
docker pull hypertrace/hypertrace:main-<commit-sha>

[jcchavezs]

I am not 100% sure about adding the hash in the tag, at least not in the main registry. We don't want to encourage people using non release versions for now. @pavolloffay proposed to have a second image registry (e.g. hypertrace/hypertrace-service-dev) where we could push all master merges. That one makes sense.

In the other hand, when it comes to hypertrace/hypertrace-service-dev I would propose to use build number (provided by CI) instead of hash as it is explicit what is older/newer than what.

What do you think @kotharironak ?

[kotharironak]

@jcchavezs sounds good to me.

[pavolloffay]

Do just the top level commit as a first cut.

With git submodules this should be enough to find out the version/SHA of the submodule.

I would propose to use build number (provided by CI) instead of hash as it is explicit what is older/newer than what.

We should keep in mind the retention of the CI jobs/logs. Once the job history is lost we won't be able to correlate it with the git history, but I like the predictability in the numbering.

[jcchavezs]

@pavolloffay yeah, that concern can be addressed by doing hypertrace-service-dev-{ci_version}-{commit_sha}

[pavolloffay]

After expired CI history it will be hard to get a tag for a give SHA. There could be two tags :name-{ci_version} and :name-{commit_sha}.

[aaron-steinfeld]

I'm hesitant to introduce a third parallel concept of versioning (beyond the release version and the SHA) - especially because I don't believe build numbers have any guarantee of monotonicity. They'd be based on execution order, so a rerun of an older SHA would be considered newer than a run of a newer SHA.

There's a few concrete things here though we can get started on while we figure out if that's needed and what form it would take:

1. SHA in image
2. Move non-release builds to their own repository
3. Stamp SHAs into non release builds, pushing a <branch>-<SHA> tag in addition to the existing <branch> tag