-
Notifications
You must be signed in to change notification settings - Fork 11
updating our BUS documentation to improve clarity #361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
clairemcdyre
wants to merge
5
commits into
main
Choose a base branch
from
bus_docs_update
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
8a12c19
updating our BUS documentation to improve clarity
clairemcdyre 7491590
Apply suggestion from @Copilot
clairemcdyre 34bcf2c
Update block-until-scan.mdx
clairemcdyre 93e89c8
Update block-until-scan.mdx
clairemcdyre d8e16bf
Update block-until-scan.mdx
clairemcdyre File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,90 +1,62 @@ | ||
| import { Note, BlockImage } from '@/components' | ||
| import { Note } from '@/components' | ||
|
|
||
| # Block Until Scan | ||
| # Enforce package signing and policy enforcement on all downloads | ||
|
|
||
| **Block Until Scan** is a security feature designed to enhance the integrity and security of software packages served by Cloudsmith, guaranteeing that all relevant security and compliance policy checks (licenses, vulnerabilities, package deny policies) are fully completed *before* a package is made available for download. | ||
| Cloudsmith can enforce package signing and security policies on **every** download from upstream registries, including the very first request for a new package. | ||
clairemcdyre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| --- | ||
| By default, Cloudsmith applies full signing and policy enforcement **after** the first download of a package that is proxied from an upstream registry. For stricter environments, you can enable an optional synchronous enforcement mode that blocks initial downloads until scanning and policy checks are complete. | ||
|
|
||
| By enabling Block Until Scan, all package requests originating from upstreams configured in "Cache and Proxy" mode are subjected to a mandatory scanning and policy evaluation process. This ensures that only packages meeting your organization's security policies can be accessed by users. | ||
|
|
||
| Without Block Until Scan, packages could be served to clients *before* all necessary policy checks were completed (unless the package was already synced in the repository, and policy checks had been completed prior to that). This allowed for the initial download of packages that may subsequently fail policy checks. | ||
|
|
||
| ## How to Use It | ||
|
|
||
| <Note variant="important"> | ||
| 📘 The Block Until Scan feature is in **Early Access (EA)**. To enable Block Until Scan for your workspace, please [contact Cloudsmith Support](https://cloudsmith.com/company/contact-us). | ||
|
|
||
| **Workspace-Wide Setting** | ||
|
|
||
| Please note that Block Until Scan is a global setting that affects **all repositories** within a given workspace. Its activation will impact all applicable package flows. | ||
| <Note variant="note"> | ||
| 📘 This feature is in **Early Access (EA)**. To enable it for your workspace, please [contact Cloudsmith](https://cloudsmith.com/company/contact-us). | ||
| </Note> | ||
|
|
||
| ### Requirements | ||
|
|
||
| - **Active Security Policies**: At least one active classic policy with a _quarantine_ action enabled is required. Without such a policy, the system would have no criteria to evaluate incoming packages. | ||
| - **Upstream Configuration ('Proxy and Caching' mode)**: the repository must have an upstream configured with ['Proxy and Caching'](/repositories/upstreams#supported-formats) mode enabled. Block Until Scan is explicitly applied to packages undergoing the caching and evaluation pipeline within Cloudsmith. | ||
| ## Default asynchronous enforcement behavior | ||
|
|
||
| ### Steps to Validate Block Until Scan | ||
| By default, Cloudsmith enforces policy evaluation and package signing **after** the first download of a package that is proxied from an upstream registry: | ||
|
|
||
| To verify the functionality of Block Until Scan, follow these steps: | ||
| - When a package is first requested from an upstream registry in **Cache and Proxy** mode, it is served immediately. | ||
| - Cloudsmith then caches, scans, signs, and evaluates the package for policy violations in the background. | ||
| - After this asynchronous evaluation is complete, all subsequent requests for that package are subject to full security enforcement and will be blocked if they violate your policies. | ||
|
|
||
| #### 1. Configure an Upstream | ||
| This ensures zero delay for the initial developer but means a vulnerable or non-compliant package could be downloaded once before being quarantined. | ||
|
|
||
| [Configure a repository](/repositories/upstreams) with an upstream in 'Proxy and Caching' mode. For example, enable a Python upstream with PyPI. | ||
| ## Synchronous enforcement mode | ||
|
|
||
| <Note> | ||
| 📘 Note that incoming packages from sources different to an upstream are also subjected to Block Until Scan. Downloads won't be allowed until all security checks have been completed. | ||
| </Note> | ||
| For environments that require maximum security at every point of access, Cloudsmith offers an optional **synchronous enforcement mode** (also known as **Block Until Scan**). In this mode, every package (including the first request of a new package) is fully scanned, signed, and evaluated against policies **before** it is made available to an end user. | ||
|
|
||
| #### 2. Create a Vulnerability policy | ||
| ### How it works | ||
|
|
||
| Browse to your Workspace Settings and [create a new vulnerability policy](/policy-management/vulnerability-policy). Define your policy with: | ||
| - A name. | ||
| - A [package Search Query](/artifact-management/search-filter-sort-packages) to scope the filter to `format:python AND requests`. We're targeting only packages named `requets` from Python, following the example in the next step. Adjust the policy to affect only to your desired scope if you use a different package for this example. | ||
| - A severity threshold set to `High`. | ||
| - A Quarantine action enabled. | ||
| When synchronous enforcement mode is enabled: | ||
|
|
||
| Then, click in **+ Create Policy**. | ||
| 1. When a new package is requested from an upstream registry, Cloudsmith holds the request until synchronization is complete. | ||
| 2. During synchronization, Cloudsmith parses the package metadata, performs security scanning, signs the package, and evaluates it against all active policies. | ||
| 3. If a package violates a policy with a **quarantine** action, Cloudsmith intercepts the request, moves the package to quarantine before the initial download completes, and returns `403 Forbidden` to the developer or CI/CD pipeline. | ||
| 4. If the policy is configured for other non-blocking actions (such as **add tag**), the package is still served, with the tag or other metadata applied. | ||
|
|
||
| #### 3. Pull a new package from your repository | ||
|
|
||
| Execute a command to pull a new package. For example, `requests==0.2.2`. This package contains a known vulnerability with a high severity CVE: `2018-18074`. If this package already exists in your repo, find a different one that hasn't been synced and cached before: | ||
|
|
||
| ```bash | ||
| pip download requests==1.2.0 --index-url https://dl.cloudsmith.io/public/demo-docs/awesome-repo/python/simple/ | ||
| ``` | ||
|
|
||
| <Note> | ||
| 📘 **Performance Impact** | ||
|
|
||
| When a package is not yet cached within Cloudsmith, every package and its dependencies must undergo a series of synchronous operations before being served: | ||
|
|
||
| * **Parsing**: The package metadata must be parsed. | ||
| * **Scanning**: The package is scanned for malware, vulnerabilities, and license compliance. | ||
| * **Policy Evaluation**: The package is evaluated against all active policies. | ||
| <Note variant="note" headline="Tip"> | ||
| If a developer receives a `403 Forbidden` error during a download request, they should check the affected package in Cloudsmith to see which specific security policy triggered the quarantine. | ||
| </Note> | ||
|
|
||
| #### 4. Observe Behavior | ||
| ## How to enable synchronous enforcement mode | ||
|
|
||
| When Block Until Scan is enabled, the initial download request for an uncached package will be temporarily blocked until the policy evaluation has completed: | ||
| - **Successful Scan**: If the package passes all policy checks, the download will proceed, and the package will be served normally. | ||
| - **Policy Matched**: If the package is matched against a policy, the package is processed exactly as the matched policy specifies: | ||
| - If the policy's action is **quarantine**, the package is quarantined and any download attempt returns **403** Forbidden. | ||
| - If the policy's action is **tag, allow**, or any other non-blocking action, the package is served normally (with the tag or other metadata applied). | ||
| Once the Cloudsmith team has enabled this feature for your workspace, you must also configure the following: | ||
|
|
||
| If the package fails a policy check (e.g., due to a vulnerability or license issue), or if the scanning process encounters an unrecoverable error, the download will be blocked with a `403 Forbidden` error code. | ||
| - **Configure upstreams in Cache and Proxy mode**: Configure repositories to connect to upstream registries in **Cache and Proxy** mode. If an upstream is configured in **Proxy Only** mode, downloads are held until synchronization is complete, but no policies are evaluated or enforced on that package. | ||
| - **Create at least one policy with a quarantine action**: Create at least one license or vulnerability policy with a **quarantine** action. Without such a policy, the system has no criteria to evaluate incoming packages for blocking. | ||
| - **Enable vulnerability scanning**: Enable **Scan for vulnerabilities** in the repository’s general settings so that Cloudsmith can run vulnerability checks as part of the enforcement process. | ||
|
|
||
| <Note variant="note" headline="First download without Block Until Scan"> | ||
| Without Block Until Scan, the same package would be delivered to the user immediately, before the full evaluation phase of the synchronization process had taken place. Cloudsmith would then asynchronously cache and synchronize the package after the first download. At this point, the system would apply the policy to the package and could then quarantine it. While future downloads of the package would then be blocked, this asynchronous processing means the initial downloads of the package could not be prevented if required. | ||
| </Note> | ||
| ## What to expect when synchronous enforcement mode is enabled | ||
|
|
||
| ## Summary | ||
| Enabling synchronous enforcement mode can introduce additional latency and developer friction, especially for cold fetches where the package is not yet cached in Cloudsmith. For packages that are already cached in Cloudsmith or directly at the nearest CDN location, the latency and impact is minimal. For each uncached package and its dependencies, Cloudsmith performs these synchronous operations before serving the download: | ||
|
|
||
| Block Until Scan helps your organization ensure that all software packages and their dependencies downloaded through Cloudsmith meet your security and compliance rules. | ||
| While this scanning happens before you can download packages, the advantages are significant: | ||
| - **Parsing**: Parse package metadata. | ||
| - **Scanning**: Scan for malware, vulnerabilities, and license compliance. | ||
| - **Evaluation and signing**: Evaluate the package against all active policies and sign it before making it available for download. | ||
|
|
||
| - **Enforce Security Policies across all your organization**: It blocks any package downloads until security checks has been completed. Only approved software components are used in your workloads, guaranteeing compliance with organizational standards and practices. | ||
| - **Reduced Risk**: checking packages before running them reduces your exposure to non-compliant software. | ||
| After evaluation: | ||
|
|
||
| In short, Block Until Scan is a crucial tool for securing and reducing risk in your software supply chain. | ||
| - If the package passes all policy checks, the download proceeds and the package is served normally. | ||
| - If the package violates a policy: | ||
| - If the policy action is **quarantine**, the package is quarantined and any download attempt returns `403 Forbidden`. | ||
| - If the policy action is **tag**, **allow**, or any other non-blocking action, the package is served normally, with the tag or other metadata applied. | ||
| - If package scanning encounters an unrecoverable error, the download is also blocked with a `403 Forbidden` error code. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.