Skip to content

Feature branch sync - pub/build_stream to main#4211

Closed
abhishek-sa1 wants to merge 770 commits intomainfrom
pub/build_stream
Closed

Feature branch sync - pub/build_stream to main#4211
abhishek-sa1 wants to merge 770 commits intomainfrom
pub/build_stream

Conversation

@abhishek-sa1
Copy link
Collaborator

Feature branch sync - pub/build_stream to main

mithileshreddy04 and others added 30 commits March 4, 2026 18:27
* gitlab fixes

* Update gitlab.yml

* update fix

* update validation

* gitlab fixes

* Update build_stream_config.yml

* Update build_stream_validation.py

* update prechecks

* Update main.yml

* Update main.yml

* Update cleanup_gitlab.yml

* Revert "Update cleanup_gitlab.yml"

This reverts commit 18abd4f.
…trol_plane is present in catalog (#4105)

* fix for adding service_kube_control_plane

* removing service_kube_control_plane_aarch64

* term should have x86_64

* renaming with x86_64
* fix for adding service_kube_control_plane

* removing service_kube_control_plane_aarch64

* term should have x86_64

* renaming with x86_64

* fix for prepare_oim input valdiation failure

* adding rechability check for aarch64 IP
precheck of  build_stream service running and gitlab running after reboot
abhishek-sa1 and others added 26 commits March 17, 2026 12:23
omnia.sh script update to remove HA support messages and prepare oim updates
…branch

Signed-off-by: pullan1 <sudha.pullalaravu@dell.com>
Add rescue block for build stream directory cleanup
Pytest for validate and the failing unit tests
Added the correct pulp cleanup Python Module missing in build_stream branch
* Fix the stage order in summary

* Fix ansible lint issues
Upgrade blocker removal for RC tag
Feature branch sync - staging to pub/build_stream
if self.client_secret:
print("✅ Using provided credentials!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 4 days ago

In general, to fix clear-text logging of sensitive information, avoid printing or logging the full secret value. Instead, either omit it entirely from logs or show only non-sensitive derived information (like length or a short masked prefix/suffix) that is sufficient for debugging without enabling misuse.

For this specific code, the minimal, non-breaking change is to stop printing the actual client_secret and replace it with a masked representation. The user still sees that a client secret is present, but not its full value. The best approach is:

  • In register_client, replace print(f" Client Secret: {self.client_secret}") with something like print(" Client Secret: [REDACTED]") or a masked version (e.g., last 4 characters only), so the secret is never output in clear text.
  • No behavior outside of logging changes; authentication flows and file formats remain the same.
  • No new imports or helper methods are strictly required; this can be done inline.

All required changes are in build_stream/tests/demo/buildstream_demo.py, in the register_client method, specifically around line 202.


Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -199,7 +199,8 @@
         if self.client_secret:
             print("✅ Using provided credentials!")
             print(f"   Client ID: {self.client_id}")
-            print(f"   Client Secret: {self.client_secret}")
+            # Do not log the client secret in clear text
+            print("   Client Secret: [REDACTED]")
             print("\n💡 Skipping registration - using existing credentials")
             return True
 
EOF
@@ -199,7 +199,8 @@
if self.client_secret:
print("✅ Using provided credentials!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")
# Do not log the client secret in clear text
print(" Client Secret: [REDACTED]")
print("\n💡 Skipping registration - using existing credentials")
return True

Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines +210 to +211
print(f"🔐 Using auth credentials: {self.auth_username}:"
f"{self.auth_password}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 4 days ago

In general, to fix clear‑text logging of sensitive information, remove credentials from log messages or, if some visibility is required, replace them with a redacted or masked form (e.g., show only the username and perhaps a non‑revealing indicator that a password is set). Passwords and other secrets should never be printed in full.

For this specific case, the best fix without changing existing functionality is to stop printing the raw password while still logging non‑sensitive context. We can keep the log line but only include the username and a placeholder such as "<redacted>" or a masked form derived from the password length, which avoids leaking the actual value. No other code depends on the string value of this log, so it is safe to alter.

Concretely, in build_stream/tests/demo/buildstream_demo.py, within ParseCatalogDemo.register_client, we should change the print(f"🔐 Using auth credentials: {self.auth_username}:{self.auth_password}") line to avoid interpolating self.auth_password. A simple and clear change is:

print(f"🔐 Using auth credentials: {self.auth_username}:<redacted>")

or, if you prefer slightly more information while still safe, you can mask it:

masked_pw = "*" * len(self.auth_password) if self.auth_password else "<none>"
print(f"🔐 Using auth credentials: {self.auth_username}:{masked_pw}")

The second option needs a temporary local variable but no new imports. Either way, no new dependencies are required, and no other parts of the file need modifications.

Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -207,8 +207,8 @@
         # These are the credentials used to register new OAuth clients
         # The vault shows: username="build_stream_register" with password_hash for "dell1234"
         # But the actual system might be using different credentials
-        print(f"🔐 Using auth credentials: {self.auth_username}:"
-              f"{self.auth_password}")
+        masked_pw = "*" * len(self.auth_password) if self.auth_password else "<none>"
+        print(f"🔐 Using auth credentials: {self.auth_username}:{masked_pw}")
         auth_header = base64.b64encode(f"{self.auth_username}:{self.auth_password}".encode()).decode()
 
         client_data = {
EOF
@@ -207,8 +207,8 @@
# These are the credentials used to register new OAuth clients
# The vault shows: username="build_stream_register" with password_hash for "dell1234"
# But the actual system might be using different credentials
print(f"🔐 Using auth credentials: {self.auth_username}:"
f"{self.auth_password}")
masked_pw = "*" * len(self.auth_password) if self.auth_password else "<none>"
print(f"🔐 Using auth credentials: {self.auth_username}:{masked_pw}")
auth_header = base64.b64encode(f"{self.auth_username}:{self.auth_password}".encode()).decode()

client_data = {
Copilot is powered by AI and may make mistakes. Always verify output.
print(f"📡 Endpoint: POST {self.base_url}/api/v1/auth/register")
print("📝 Headers:")
print(" Content-Type: application/json")
print(f" Authorization: Basic {auth_header}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 4 days ago

In general, the fix is to ensure that sensitive authentication data (passwords, secrets, full Authorization headers) are not written to logs or printed. You can still log that an Authorization header is being used, or log only non-sensitive metadata (e.g., header names, or a redacted version of the header), but must not include the actual password or its direct encodings.

For this specific script, the best fix without changing functionality is:

  • Keep using auth_header to actually send the HTTP request (so functionality is unchanged).
  • Stop printing the full header value to the console.
  • Instead, print a redacted or generic placeholder, such as "Authorization: Basic [REDACTED]". This preserves the “interactive demo” feel (user still sees that an Authorization header is used) while not exposing the sensitive credential.
  • Optionally, if you still want to log the username for debugging, you could log just self.auth_username and note that the password is hidden. However, to keep the change minimal, we’ll only redact the header value.

Concretely, in build_stream/tests/demo/buildstream_demo.py, around line 224 where print(f" Authorization: Basic {auth_header}") appears, replace this print statement with a version that does not interpolate auth_header, e.g. print(" Authorization: Basic [REDACTED]"). No new imports or helpers are required.

Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -221,7 +221,7 @@
         print(f"📡 Endpoint: POST {self.base_url}/api/v1/auth/register")
         print("📝 Headers:")
         print("   Content-Type: application/json")
-        print(f"   Authorization: Basic {auth_header}")
+        print("   Authorization: Basic [REDACTED]")
         print("📝 Request Body:")
         print(json.dumps(client_data, indent=2))
 
EOF
@@ -221,7 +221,7 @@
print(f"📡 Endpoint: POST {self.base_url}/api/v1/auth/register")
print("📝 Headers:")
print(" Content-Type: application/json")
print(f" Authorization: Basic {auth_header}")
print(" Authorization: Basic [REDACTED]")
print("📝 Request Body:")
print(json.dumps(client_data, indent=2))

Copilot is powered by AI and may make mistakes. Always verify output.
self.client_id = client_info.get('client_id') # Use server-assigned ID
print("\n✅ Client registered successfully!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 4 days ago

In general, to fix clear-text logging of sensitive information, the code should avoid logging secrets at all or log only masked/partial representations that cannot be used to authenticate. For authentication credentials such as client secrets, passwords, or tokens, the best practice is to not log them even at debug level; at most, log a truncated, non-usable form if needed for troubleshooting.

For this specific code, the best fix without changing functionality is to stop printing self.client_secret in clear text. The script already computes a masked version of the secret for human display (display_info['client_secret'] = display_info['client_secret'][:8] + "..." + display_info['client_secret'][-4:]), so we can reuse that approach. We should change the line print(f" Client Secret: {self.client_secret}") to log a masked version instead, derived from self.client_secret. Similarly, in the 409-status branch where an existing client is used, we should not log the hard-coded existing_client_secret in clear text; instead, log a masked representation or omit it. This addresses all variants of the alert because there will no longer be any sink that prints the clear-text secret. No new imports or external dependencies are required; the masking can be implemented inline with simple string slicing and conditional logic.

Concretely, in build_stream/tests/demo/buildstream_demo.py:

  • Around line 257, replace print(f" Client Secret: {self.client_secret}") with a small block that constructs a masked version of self.client_secret (e.g., first 4 and last 4 characters, with *** in between) and prints that instead, handling None or very short values gracefully.
  • Around line 296, replace print(f" Client Secret: {self.client_secret}") in the "Using existing client!" branch with the same masking logic so that the hard-coded secret is not exposed.
    These changes preserve the informational intent for a human operator while ensuring the underlying secret is not disclosed in clear text.
Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -254,7 +254,12 @@
                 self.client_id = client_info.get('client_id')  # Use server-assigned ID
                 print("\n✅ Client registered successfully!")
                 print(f"   Client ID: {self.client_id}")
-                print(f"   Client Secret: {self.client_secret}")
+                masked_secret = (
+                    self.client_secret
+                    if not isinstance(self.client_secret, str) or len(self.client_secret) < 8
+                    else f"{self.client_secret[:4]}***{self.client_secret[-4:]}"
+                )
+                print(f"   Client Secret (masked): {masked_secret}")
 
                 # Save credentials to file for future use
                 self.save_credentials(self.client_id, self.client_secret)
@@ -293,7 +298,12 @@
                     self.client_secret = existing_client_secret
                     print("✅ Using existing client!")
                     print(f"   Client ID: {self.client_id}")
-                    print(f"   Client Secret: {self.client_secret}")
+                    masked_secret = (
+                        self.client_secret
+                        if not isinstance(self.client_secret, str) or len(self.client_secret) < 8
+                        else f"{self.client_secret[:4]}***{self.client_secret[-4:]}"
+                    )
+                    print(f"   Client Secret (masked): {masked_secret}")
                     print("\n💡 These credentials are working for this session")
                     return True
                 else:
EOF
@@ -254,7 +254,12 @@
self.client_id = client_info.get('client_id') # Use server-assigned ID
print("\n✅ Client registered successfully!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")
masked_secret = (
self.client_secret
if not isinstance(self.client_secret, str) or len(self.client_secret) < 8
else f"{self.client_secret[:4]}***{self.client_secret[-4:]}"
)
print(f" Client Secret (masked): {masked_secret}")

# Save credentials to file for future use
self.save_credentials(self.client_id, self.client_secret)
@@ -293,7 +298,12 @@
self.client_secret = existing_client_secret
print("✅ Using existing client!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")
masked_secret = (
self.client_secret
if not isinstance(self.client_secret, str) or len(self.client_secret) < 8
else f"{self.client_secret[:4]}***{self.client_secret[-4:]}"
)
print(f" Client Secret (masked): {masked_secret}")
print("\n💡 These credentials are working for this session")
return True
else:
Copilot is powered by AI and may make mistakes. Always verify output.
self.client_secret = existing_client_secret
print("✅ Using existing client!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 4 days ago

In general, to fix clear-text logging of sensitive data, avoid printing secrets at all or print only a masked/partially redacted version. If the user truly needs to see the full value (e.g., first-time display), log it only to a secure channel and never to general-purpose logs or stdout.

For this script, the best fix without changing behavior significantly is:

  • Stop printing the full client_secret value in both branches.
  • Reuse the same masking approach already used for display_info['client_secret'] to print only a redacted form in messages intended for the user.
  • Keep storing the real secret in self.client_secret and in the credentials file so functionality (authentication) is unchanged.

Concretely in build_stream/tests/demo/buildstream_demo.py:

  • In the success path (status 200/201), change the print(f" Client Secret: {self.client_secret}") to print a masked version (e.g., first 4 and last 4 characters with *** in the middle).
  • In the “using existing client” path (status 409 with working existing client), likewise change print(f" Client Secret: {self.client_secret}") to print a masked version.
  • Implement the masking inline in those print statements to avoid extra helpers/imports in this demo context.
Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -254,7 +254,8 @@
                 self.client_id = client_info.get('client_id')  # Use server-assigned ID
                 print("\n✅ Client registered successfully!")
                 print(f"   Client ID: {self.client_id}")
-                print(f"   Client Secret: {self.client_secret}")
+                masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***"
+                print(f"   Client Secret: {masked_secret} (masked)")
 
                 # Save credentials to file for future use
                 self.save_credentials(self.client_id, self.client_secret)
@@ -293,7 +294,8 @@
                     self.client_secret = existing_client_secret
                     print("✅ Using existing client!")
                     print(f"   Client ID: {self.client_id}")
-                    print(f"   Client Secret: {self.client_secret}")
+                    masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***"
+                    print(f"   Client Secret: {masked_secret} (masked)")
                     print("\n💡 These credentials are working for this session")
                     return True
                 else:
EOF
@@ -254,7 +254,8 @@
self.client_id = client_info.get('client_id') # Use server-assigned ID
print("\n✅ Client registered successfully!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")
masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***"
print(f" Client Secret: {masked_secret} (masked)")

# Save credentials to file for future use
self.save_credentials(self.client_id, self.client_secret)
@@ -293,7 +294,8 @@
self.client_secret = existing_client_secret
print("✅ Using existing client!")
print(f" Client ID: {self.client_id}")
print(f" Client Secret: {self.client_secret}")
masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***"
print(f" Client Secret: {masked_secret} (masked)")
print("\n💡 These credentials are working for this session")
return True
else:
Copilot is powered by AI and may make mistakes. Always verify output.
print("📋 Request Body:")
print(" grant_type=client_credentials")
print(f" client_id={self.client_id}")
print(f" client_secret={self.client_secret[:8]}...{self.client_secret[-4:]}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 4 days ago

In general, the fix is to avoid logging secrets at all. Instead of printing the client_secret (even partially masked), log only non‑sensitive identifiers (e.g., client ID) and generic messages about what the script is doing. Secrets should still be stored and used programmatically, but never included in log or console messages.

For this specific script, the single best fix is to remove or replace any print that includes client_secret values, including partially masked ones, and any hard‑coded secret values. The important location for the CodeQL alert is line 330 where the token request body is printed. We can safely replace that line so the request logging omits the client_secret entirely, e.g., by printing a placeholder like client_secret=***REDACTED***. This change preserves the interactive/demo behavior while no longer leaking any part of the secret.

Concretely:

  • In build_stream/tests/demo/buildstream_demo.py, within ParseCatalogDemo.get_access_token, change the line that prints client_secret using slicing (self.client_secret[:8]...self.client_secret[-4:]) to a constant redacted placeholder.
  • No changes are required to how self.client_secret is loaded, stored, or used in HTTP requests: only its appearance in log/print output needs to be removed.
  • No new imports, helper methods, or definitions are required.

Suggested changeset 1
build_stream/tests/demo/buildstream_demo.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/build_stream/tests/demo/buildstream_demo.py b/build_stream/tests/demo/buildstream_demo.py
--- a/build_stream/tests/demo/buildstream_demo.py
+++ b/build_stream/tests/demo/buildstream_demo.py
@@ -327,7 +327,7 @@
         print("📋 Request Body:")
         print("   grant_type=client_credentials")
         print(f"   client_id={self.client_id}")
-        print(f"   client_secret={self.client_secret[:8]}...{self.client_secret[-4:]}")
+        print("   client_secret=***REDACTED***")
 
         self.wait_for_enter("Press ENTER to get access token...")
 
EOF
@@ -327,7 +327,7 @@
print("📋 Request Body:")
print(" grant_type=client_credentials")
print(f" client_id={self.client_id}")
print(f" client_secret={self.client_secret[:8]}...{self.client_secret[-4:]}")
print(" client_secret=***REDACTED***")

self.wait_for_enter("Press ENTER to get access token...")

Copilot is powered by AI and may make mistakes. Always verify output.
assert artifact_store.exists(non_existent_key) is False

# Delete the artifact
assert artifact_store.delete(ref.key) is True
assert artifact_store.exists(ref.key) is False

# Try to delete non-existent artifact
assert artifact_store.delete(non_existent_key) is False
content=sample_content,
content_type="application/json",
)
assert artifact_store.delete(ref.key) is True
from core.artifacts.value_objects import ArtifactKey

key = ArtifactKey("nonexistent/key/file.bin")
assert artifact_store.delete(key) is False

def test_valid_stage_names(self):
"""All canonical stage names should be accepted."""
for stage in StageType:
stages = job_info.get("stages", [])
print("\n📊 Stage Summary:")
for stage in stages:
status_emoji = "✅" if stage.get("stage_state") == "COMPLETED" else "⏳" if stage.get("stage_state") == "PENDING" else "❌"
print("\n📊 Stage Summary:")
for stage in stages:
status_emoji = "✅" if stage.get("stage_state") == "COMPLETED" else "⏳" if stage.get("stage_state") == "PENDING" else "❌"
status_emoji = (
@abhishek-sa1 abhishek-sa1 marked this pull request as ready for review March 18, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.