Feature branch sync - pub/build_stream to main#4211
Feature branch sync - pub/build_stream to main#4211abhishek-sa1 wants to merge 770 commits intomainfrom
Conversation
…re_oim validation
…4/omnia into pub/build_stream
* gitlab fixes * Update gitlab.yml * update fix * update validation * gitlab fixes * Update build_stream_config.yml * Update build_stream_validation.py * update prechecks * Update main.yml * Update main.yml * Update cleanup_gitlab.yml * Revert "Update cleanup_gitlab.yml" This reverts commit 18abd4f.
…4/omnia into pub/build_stream
…trol_plane is present in catalog (#4105) * fix for adding service_kube_control_plane * removing service_kube_control_plane_aarch64 * term should have x86_64 * renaming with x86_64
* fix for adding service_kube_control_plane * removing service_kube_control_plane_aarch64 * term should have x86_64 * renaming with x86_64 * fix for prepare_oim input valdiation failure * adding rechability check for aarch64 IP
…H123/omnia into pub/build_stream
precheck of build_stream service running and gitlab running after reboot
…4/omnia into pub/build_stream
omnia.sh script update to remove HA support messages and prepare oim updates
…branch Signed-off-by: pullan1 <sudha.pullalaravu@dell.com>
Add rescue block for build stream directory cleanup
Pytest for validate and the failing unit tests
Added the correct pulp cleanup Python Module missing in build_stream branch
* Fix the stage order in summary * Fix ansible lint issues
fix: checkmarx
Upgrade blocker removal for RC tag
…fect_fix1 defect fix aarch64
Feature branch sync - staging to pub/build_stream
| if self.client_secret: | ||
| print("✅ Using provided credentials!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, to fix clear-text logging of sensitive information, avoid printing or logging the full secret value. Instead, either omit it entirely from logs or show only non-sensitive derived information (like length or a short masked prefix/suffix) that is sufficient for debugging without enabling misuse.
For this specific code, the minimal, non-breaking change is to stop printing the actual client_secret and replace it with a masked representation. The user still sees that a client secret is present, but not its full value. The best approach is:
- In
register_client, replaceprint(f" Client Secret: {self.client_secret}")with something likeprint(" Client Secret: [REDACTED]")or a masked version (e.g., last 4 characters only), so the secret is never output in clear text. - No behavior outside of logging changes; authentication flows and file formats remain the same.
- No new imports or helper methods are strictly required; this can be done inline.
All required changes are in build_stream/tests/demo/buildstream_demo.py, in the register_client method, specifically around line 202.
| @@ -199,7 +199,8 @@ | ||
| if self.client_secret: | ||
| print("✅ Using provided credentials!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") | ||
| # Do not log the client secret in clear text | ||
| print(" Client Secret: [REDACTED]") | ||
| print("\n💡 Skipping registration - using existing credentials") | ||
| return True | ||
|
|
| print(f"🔐 Using auth credentials: {self.auth_username}:" | ||
| f"{self.auth_password}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, to fix clear‑text logging of sensitive information, remove credentials from log messages or, if some visibility is required, replace them with a redacted or masked form (e.g., show only the username and perhaps a non‑revealing indicator that a password is set). Passwords and other secrets should never be printed in full.
For this specific case, the best fix without changing existing functionality is to stop printing the raw password while still logging non‑sensitive context. We can keep the log line but only include the username and a placeholder such as "<redacted>" or a masked form derived from the password length, which avoids leaking the actual value. No other code depends on the string value of this log, so it is safe to alter.
Concretely, in build_stream/tests/demo/buildstream_demo.py, within ParseCatalogDemo.register_client, we should change the print(f"🔐 Using auth credentials: {self.auth_username}:{self.auth_password}") line to avoid interpolating self.auth_password. A simple and clear change is:
print(f"🔐 Using auth credentials: {self.auth_username}:<redacted>")or, if you prefer slightly more information while still safe, you can mask it:
masked_pw = "*" * len(self.auth_password) if self.auth_password else "<none>"
print(f"🔐 Using auth credentials: {self.auth_username}:{masked_pw}")The second option needs a temporary local variable but no new imports. Either way, no new dependencies are required, and no other parts of the file need modifications.
| @@ -207,8 +207,8 @@ | ||
| # These are the credentials used to register new OAuth clients | ||
| # The vault shows: username="build_stream_register" with password_hash for "dell1234" | ||
| # But the actual system might be using different credentials | ||
| print(f"🔐 Using auth credentials: {self.auth_username}:" | ||
| f"{self.auth_password}") | ||
| masked_pw = "*" * len(self.auth_password) if self.auth_password else "<none>" | ||
| print(f"🔐 Using auth credentials: {self.auth_username}:{masked_pw}") | ||
| auth_header = base64.b64encode(f"{self.auth_username}:{self.auth_password}".encode()).decode() | ||
|
|
||
| client_data = { |
| print(f"📡 Endpoint: POST {self.base_url}/api/v1/auth/register") | ||
| print("📝 Headers:") | ||
| print(" Content-Type: application/json") | ||
| print(f" Authorization: Basic {auth_header}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, the fix is to ensure that sensitive authentication data (passwords, secrets, full Authorization headers) are not written to logs or printed. You can still log that an Authorization header is being used, or log only non-sensitive metadata (e.g., header names, or a redacted version of the header), but must not include the actual password or its direct encodings.
For this specific script, the best fix without changing functionality is:
- Keep using
auth_headerto actually send the HTTP request (so functionality is unchanged). - Stop printing the full header value to the console.
- Instead, print a redacted or generic placeholder, such as
"Authorization: Basic [REDACTED]". This preserves the “interactive demo” feel (user still sees that an Authorization header is used) while not exposing the sensitive credential. - Optionally, if you still want to log the username for debugging, you could log just
self.auth_usernameand note that the password is hidden. However, to keep the change minimal, we’ll only redact the header value.
Concretely, in build_stream/tests/demo/buildstream_demo.py, around line 224 where print(f" Authorization: Basic {auth_header}") appears, replace this print statement with a version that does not interpolate auth_header, e.g. print(" Authorization: Basic [REDACTED]"). No new imports or helpers are required.
| @@ -221,7 +221,7 @@ | ||
| print(f"📡 Endpoint: POST {self.base_url}/api/v1/auth/register") | ||
| print("📝 Headers:") | ||
| print(" Content-Type: application/json") | ||
| print(f" Authorization: Basic {auth_header}") | ||
| print(" Authorization: Basic [REDACTED]") | ||
| print("📝 Request Body:") | ||
| print(json.dumps(client_data, indent=2)) | ||
|
|
| self.client_id = client_info.get('client_id') # Use server-assigned ID | ||
| print("\n✅ Client registered successfully!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, to fix clear-text logging of sensitive information, the code should avoid logging secrets at all or log only masked/partial representations that cannot be used to authenticate. For authentication credentials such as client secrets, passwords, or tokens, the best practice is to not log them even at debug level; at most, log a truncated, non-usable form if needed for troubleshooting.
For this specific code, the best fix without changing functionality is to stop printing self.client_secret in clear text. The script already computes a masked version of the secret for human display (display_info['client_secret'] = display_info['client_secret'][:8] + "..." + display_info['client_secret'][-4:]), so we can reuse that approach. We should change the line print(f" Client Secret: {self.client_secret}") to log a masked version instead, derived from self.client_secret. Similarly, in the 409-status branch where an existing client is used, we should not log the hard-coded existing_client_secret in clear text; instead, log a masked representation or omit it. This addresses all variants of the alert because there will no longer be any sink that prints the clear-text secret. No new imports or external dependencies are required; the masking can be implemented inline with simple string slicing and conditional logic.
Concretely, in build_stream/tests/demo/buildstream_demo.py:
- Around line 257, replace
print(f" Client Secret: {self.client_secret}")with a small block that constructs a masked version ofself.client_secret(e.g., first 4 and last 4 characters, with***in between) and prints that instead, handlingNoneor very short values gracefully. - Around line 296, replace
print(f" Client Secret: {self.client_secret}")in the "Using existing client!" branch with the same masking logic so that the hard-coded secret is not exposed.
These changes preserve the informational intent for a human operator while ensuring the underlying secret is not disclosed in clear text.
| @@ -254,7 +254,12 @@ | ||
| self.client_id = client_info.get('client_id') # Use server-assigned ID | ||
| print("\n✅ Client registered successfully!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") | ||
| masked_secret = ( | ||
| self.client_secret | ||
| if not isinstance(self.client_secret, str) or len(self.client_secret) < 8 | ||
| else f"{self.client_secret[:4]}***{self.client_secret[-4:]}" | ||
| ) | ||
| print(f" Client Secret (masked): {masked_secret}") | ||
|
|
||
| # Save credentials to file for future use | ||
| self.save_credentials(self.client_id, self.client_secret) | ||
| @@ -293,7 +298,12 @@ | ||
| self.client_secret = existing_client_secret | ||
| print("✅ Using existing client!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") | ||
| masked_secret = ( | ||
| self.client_secret | ||
| if not isinstance(self.client_secret, str) or len(self.client_secret) < 8 | ||
| else f"{self.client_secret[:4]}***{self.client_secret[-4:]}" | ||
| ) | ||
| print(f" Client Secret (masked): {masked_secret}") | ||
| print("\n💡 These credentials are working for this session") | ||
| return True | ||
| else: |
| self.client_secret = existing_client_secret | ||
| print("✅ Using existing client!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, to fix clear-text logging of sensitive data, avoid printing secrets at all or print only a masked/partially redacted version. If the user truly needs to see the full value (e.g., first-time display), log it only to a secure channel and never to general-purpose logs or stdout.
For this script, the best fix without changing behavior significantly is:
- Stop printing the full
client_secretvalue in both branches. - Reuse the same masking approach already used for
display_info['client_secret']to print only a redacted form in messages intended for the user. - Keep storing the real secret in
self.client_secretand in the credentials file so functionality (authentication) is unchanged.
Concretely in build_stream/tests/demo/buildstream_demo.py:
- In the success path (status 200/201), change the
print(f" Client Secret: {self.client_secret}")to print a masked version (e.g., first 4 and last 4 characters with***in the middle). - In the “using existing client” path (status 409 with working existing client), likewise change
print(f" Client Secret: {self.client_secret}")to print a masked version. - Implement the masking inline in those print statements to avoid extra helpers/imports in this demo context.
| @@ -254,7 +254,8 @@ | ||
| self.client_id = client_info.get('client_id') # Use server-assigned ID | ||
| print("\n✅ Client registered successfully!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") | ||
| masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***" | ||
| print(f" Client Secret: {masked_secret} (masked)") | ||
|
|
||
| # Save credentials to file for future use | ||
| self.save_credentials(self.client_id, self.client_secret) | ||
| @@ -293,7 +294,8 @@ | ||
| self.client_secret = existing_client_secret | ||
| print("✅ Using existing client!") | ||
| print(f" Client ID: {self.client_id}") | ||
| print(f" Client Secret: {self.client_secret}") | ||
| masked_secret = (self.client_secret[:4] + "***" + self.client_secret[-4:]) if self.client_secret and len(self.client_secret) > 8 else "***" | ||
| print(f" Client Secret: {masked_secret} (masked)") | ||
| print("\n💡 These credentials are working for this session") | ||
| return True | ||
| else: |
| print("📋 Request Body:") | ||
| print(" grant_type=client_credentials") | ||
| print(f" client_id={self.client_id}") | ||
| print(f" client_secret={self.client_secret[:8]}...{self.client_secret[-4:]}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
In general, the fix is to avoid logging secrets at all. Instead of printing the client_secret (even partially masked), log only non‑sensitive identifiers (e.g., client ID) and generic messages about what the script is doing. Secrets should still be stored and used programmatically, but never included in log or console messages.
For this specific script, the single best fix is to remove or replace any print that includes client_secret values, including partially masked ones, and any hard‑coded secret values. The important location for the CodeQL alert is line 330 where the token request body is printed. We can safely replace that line so the request logging omits the client_secret entirely, e.g., by printing a placeholder like client_secret=***REDACTED***. This change preserves the interactive/demo behavior while no longer leaking any part of the secret.
Concretely:
- In
build_stream/tests/demo/buildstream_demo.py, withinParseCatalogDemo.get_access_token, change the line that printsclient_secretusing slicing (self.client_secret[:8]...self.client_secret[-4:]) to a constant redacted placeholder. - No changes are required to how
self.client_secretis loaded, stored, or used in HTTP requests: only its appearance in log/print output needs to be removed. - No new imports, helper methods, or definitions are required.
| @@ -327,7 +327,7 @@ | ||
| print("📋 Request Body:") | ||
| print(" grant_type=client_credentials") | ||
| print(f" client_id={self.client_id}") | ||
| print(f" client_secret={self.client_secret[:8]}...{self.client_secret[-4:]}") | ||
| print(" client_secret=***REDACTED***") | ||
|
|
||
| self.wait_for_enter("Press ENTER to get access token...") | ||
|
|
| assert artifact_store.exists(non_existent_key) is False | ||
|
|
||
| # Delete the artifact | ||
| assert artifact_store.delete(ref.key) is True |
| assert artifact_store.exists(ref.key) is False | ||
|
|
||
| # Try to delete non-existent artifact | ||
| assert artifact_store.delete(non_existent_key) is False |
| content=sample_content, | ||
| content_type="application/json", | ||
| ) | ||
| assert artifact_store.delete(ref.key) is True |
| from core.artifacts.value_objects import ArtifactKey | ||
|
|
||
| key = ArtifactKey("nonexistent/key/file.bin") | ||
| assert artifact_store.delete(key) is False |
|
|
||
| def test_valid_stage_names(self): | ||
| """All canonical stage names should be accepted.""" | ||
| for stage in StageType: |
| stages = job_info.get("stages", []) | ||
| print("\n📊 Stage Summary:") | ||
| for stage in stages: | ||
| status_emoji = "✅" if stage.get("stage_state") == "COMPLETED" else "⏳" if stage.get("stage_state") == "PENDING" else "❌" |
| print("\n📊 Stage Summary:") | ||
| for stage in stages: | ||
| status_emoji = "✅" if stage.get("stage_state") == "COMPLETED" else "⏳" if stage.get("stage_state") == "PENDING" else "❌" | ||
| status_emoji = ( |
Feature branch sync - pub/build_stream to main