Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 1 addition & 79 deletions .github/workflows/check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,54 +20,12 @@ jobs:
with:
python-version: "3.11" # minimum supported lang version

- name: Install dependencies
- name: Install Python dependencies
run: python -m pip install hatch 'click!=8.3.0'

- name: Run pre-commit hooks
run: hatch run lint:install-hooks && hatch run lint:precommit

linter:
name: linter
runs-on: ubuntu-latest
if: ${{ !startsWith(github.ref, 'refs/tags') }}

steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.11" # minimum supported lang version

- name: Install dependencies
run: python -m pip install hatch 'click!=8.3.0'

- name: Run
run: hatch run lint:check

mypy:
name: mypy
runs-on: ubuntu-latest
if: ${{ !startsWith(github.ref, 'refs/tags') }}

steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.11" # minimum supported lang version

- name: Install dependencies
run: python -m pip install hatch 'click!=8.3.0'

- name: Check MyPy
run: hatch run mypy:check

pkglint:
name: pkglint
runs-on: ubuntu-latest
Expand All @@ -89,22 +47,6 @@ jobs:
- name: Run
run: hatch run lint:pkglint

markdownlint:
# https://github.com/marketplace/actions/markdown-lint
name: markdownlint
runs-on: ubuntu-latest
if: ${{ !startsWith(github.ref, 'refs/tags') }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: articulate/actions-markdownlint@v1.1.0
with:
config: .markdownlint-config.yaml
# files: 'docs/**/*.md'
# ignore: node_modules
# version: 0.28.1

docs:
name: readthedocs
runs-on: ubuntu-latest
Expand All @@ -125,23 +67,3 @@ jobs:

- name: Run
run: hatch run docs:build

super-linter:
name: super-linter
runs-on: ubuntu-latest
if: ${{ !startsWith(github.ref, 'refs/tags') }}

steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Super-Linter
uses: super-linter/super-linter@v8.3.2 # x-release-please-version
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# To reuse the same Super-linter configuration that you use in the
# lint job without duplicating it, see
# https://github.com/super-linter/super-linter/blob/main/docs/run-linter-locally.md#share-environment-variables-between-environments
VALIDATE_ALL_CODEBASE: false
VALIDATE_MARKDOWN: true
VALIDATE_EDITORCONFIG: true
14 changes: 0 additions & 14 deletions .markdownlint-config.yaml

This file was deleted.

9 changes: 9 additions & 0 deletions .mdformat.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Use consecutive numbering for ordered lists (1, 2, 3 instead of 1, 1, 1)
number = true

# Preserve original line wrapping
wrap = "keep"

[plugin.tables]
# Strip extra spaces from GFM tables (don't pad columns for alignment)
compact_tables = true
2 changes: 1 addition & 1 deletion .mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pull_request_rules:
- "title~=^e2e:"
- files~=pyproject.toml
- files~=.github/
- files~=.markdownlint-config.yaml
- files~=.mdformat.toml
- files~=tests/
- files~=e2e/
actions:
Expand Down
65 changes: 51 additions & 14 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,26 +1,63 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v6.0.0
hooks:
- id: end-of-file-fixer # Ensures single trailing newline
- id: trailing-whitespace # Removes trailing spaces
args: [--markdown-linebreak-ext=md] # Preserve markdown line breaks
- id: check-yaml # Validates YAML syntax
- id: check-merge-conflict # Prevents merge conflict markers
- id: check-toml # Validates TOML syntax
- id: end-of-file-fixer
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: check-yaml
- id: check-merge-conflict
- id: check-toml

- repo: local
- repo: https://github.com/mgedmin/check-python-versions
rev: "0.24.0"
hooks:
- id: hatch-lint
name: hatch lint check
entry: hatch run lint:check
language: system
types: [python]
pass_filenames: false
- id: check-python-versions
args: ["--only", "pyproject.toml,.github/workflows/test.yaml"]

- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
rev: 3.2.1
hooks:
- id: editorconfig-checker
alias: ec
# Exclude files that are auto-generated or strictly formatted elsewhere
exclude: |
(?x)^(
LICENSE|
.*\.lock|
dist/.*
)$
- repo: https://github.com/hukkin/mdformat
rev: 1.0.0
hooks:
- id: mdformat
additional_dependencies:
- mdformat-gfm # GitHub Flavored Markdown support (tables, autolinks, etc.)
- mdformat-simple-breaks # Use --- for thematic breaks instead of 70 underscores
Comment on lines +31 to +37
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This linter and formatter seems to replace all * bullet points with -. It's a lot of noise that interferes with git history and git annotate. The asterisk is a valid character for bullet points. I actually prefer it over dash, because it sticks out.

Copy link
Author

@MichaelYochpaz MichaelYochpaz Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This linter and formatter seems to replace all * bullet points with -. It's a lot of noise that interferes with git history and git annotate

It modified only 4 doc files, and out of those only 2 doc files (docs/files.md docs/using.md) had their lists updated to use - instead of *. I wouldn't call it "a lot of noise".

I will also add that we use both * and - for different lists across our docs inconsistently, so any commit that would line these up to a specific convention would require similar changes, since we didn't enforce consistency until now and we use both symbols. See the list items here for example, that weren't modified since they already use -.

If you're concerned about git blame noise, I can move those lint changes to a separate commit (might require a separate PR since we squash merge?), and then add this commit to a .git-blame-ignore-revs and then it'll be ignored on git blame.

The asterisk is a valid character for bullet points. I actually prefer it over dash, because it sticks out.

I tried to check if there's a config option / plugin to override that, but couldn't find one.

I don't really mind either option, but if I have to choose between a Markdown linter that requires Node.js to run, and adds complexity to run locally, and a linter that just uses - for bullet points instead of * (completely valid Markdown), but runs locally and auto-fixes linting issues easily, I'll pick the other one that helps me work fast and more efficiently. I don't think this minor personal preference for docs that end up rendering the same way anyways, is worth blocking this PR.

If it's really a big concern, we could write a small plugin to override it (plugins seem pretty simple - example) but I'd rather not because we might break stuff unintentionally and it'll require maintenance if the linter updates and changes the API.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't call it "a lot of noise".

I'm consider about 100 changed lines a lot of noise.

If you're concerned about git blame noise, I can move those lint changes to a separate commit (might require a separate PR since we squash merge?), and then add this commit to a .git-blame-ignore-revs and then it'll be ignored on git blame.

That's a good idea. Let's move all auto-formatting changes to a separate commit.


- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.11.11
hooks:
# Linter runs before formatter so fixes (like removing imports) get formatted
- id: ruff-check
args: [--fix]
types_or: [python, pyi]
- id: ruff-format
types_or: [python, pyi]
Comment on lines +39 to +47
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block seens to fix Ruff formating and linting issues. How does pre-commit detect linting errors and fail when there is a problem?

Copy link
Author

@MichaelYochpaz MichaelYochpaz Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are any modifications made, it returns a non-zero error code (even if all the linting issues were fixed following these changes).
And the same happens if there are issues that couldn't be auto-fixed.


- repo: local
hooks:
- id: hatch-mypy
name: hatch mypy check
entry: hatch run mypy:check
language: system
types: [python]
pass_filenames: false

- id: mergify-lint
name: mergify lint
entry: hatch run lint:mergify
language: system
files: ^\.mergify\.yml$
pass_filenames: false
14 changes: 7 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,15 @@ hatch run lint:precommit # All linters and other pre-commit hooks

### Pre-commit Hooks

The project uses pre-commit hooks to automatically check file formatting:
Run all linters and formatters via pre-commit:

- **File endings**: Ensures all files end with a single newline
- **Whitespace**: Removes trailing whitespace
- **Syntax**: Validates YAML/TOML files
- **Conflicts**: Prevents committing merge conflict markers
- **Linters**: Runs the `mypy` and `ruff` linters
```bash
hatch run lint:precommit # Run all hooks manually
```

Pre-commit runs automatically on commit after installation with `hatch run lint:install-hooks`.

These run automatically on commit if installed with `hatch run lint:install-hooks`.
**Markdown formatting:** The mdformat hook formats Markdown files using a pure Python formatter with GitHub Flavored Markdown support.

## Safety and Permissions

Expand Down
5 changes: 3 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Fromager thrives on practical, well-tested contributions. This guide summarizes
### Prerequisites

- Python 3.11 or newer

- `hatch` for environment and task management

```bash
Expand Down Expand Up @@ -358,7 +359,7 @@ EnvKey = typing.Annotated[str, BeforeValidator(_validate_envkey)]
### Commands

| Task | Command |
| ------ | --------- |
| -- | -- |
| Run tests | `hatch run test:test` |
| Check code quality | `hatch run lint:check` |
| Fix formatting | `hatch run lint:fix` |
Expand All @@ -367,7 +368,7 @@ EnvKey = typing.Annotated[str, BeforeValidator(_validate_envkey)]
### Standards

| Standard | Requirement |
| ---------- | ------------- |
| -- | -- |
| Type annotations | Required for every function |
| Docstrings | Required on public APIs |
| Tests | Required for new behavior |
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ wheels from source.

Fromager is designed to guarantee that:

* Every binary package you install was built from source in a reproducible environment compatible with your own.
- Every binary package you install was built from source in a reproducible environment compatible with your own.

* All dependencies are also built from source, no prebuilt binaries.
- All dependencies are also built from source, no prebuilt binaries.

* The build tools themselves are built from source, ensuring a fully transparent toolchain.
- The build tools themselves are built from source, ensuring a fully transparent toolchain.

* Builds can be customized for your needs: applying patches, adjusting compiler options, or producing build variants.
- Builds can be customized for your needs: applying patches, adjusting compiler options, or producing build variants.

## Design Principles

Expand All @@ -23,9 +23,9 @@ Fromager automates the build process with sensible defaults that work for most P

Fromager can also build wheels in collections, rather than individually. Managing dependencies as a unified group ensures that:

* Packages built against one another remain ABI-compatible.
- Packages built against one another remain ABI-compatible.

* All versions are resolved consistently, so the resulting wheels can be installed together without conflicts.
- All versions are resolved consistently, so the resulting wheels can be installed together without conflicts.

This approach makes Fromager especially useful in Python-heavy domains like AI, where reproducibility and compatibility across complex dependency trees are essential.

Expand Down Expand Up @@ -61,9 +61,9 @@ GITHUB_TOKEN=<access_token>

## Additional docs

* [Using fromager](docs/using.md)
* [Package build customization instructions](docs/customization.md)
* [Developer instructions](docs/develop.md)
- [Using fromager](docs/using.md)
- [Package build customization instructions](docs/customization.md)
- [Developer instructions](docs/develop.md)

## What's with the name?

Expand Down
27 changes: 27 additions & 0 deletions docs/develop.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,32 @@
# Developing these tools

## Pre-commit hooks

The project uses [pre-commit](https://pre-commit.com/) to run linters and formatters automatically on each commit. This ensures consistent code quality across all contributions.

### Setup

Install the hooks once after cloning:

```bash
hatch run lint:install-hooks
```

### Running hooks

Hooks run automatically when you commit. To run all hooks manually:

```bash
hatch run lint:precommit
```

### What the hooks check

- **File formatting**: Trailing whitespace, final newlines, YAML/TOML syntax
- **Python**: Ruff (linting + formatting), mypy (type checking)
- **Markdown**: mdformat (formatting with GitHub Flavored Markdown support)
- **Config validation**: EditorConfig, Mergify, Python version consistency

## Unit tests and linter

The unit tests and linter now use [Hatch](https://hatch.pypa.io/) and a
Expand Down
32 changes: 16 additions & 16 deletions docs/files.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ fromager bootstrap --skip-constraints package1 package2

When `--skip-constraints` is used:

* No `constraints.txt` file will be created in the work directory
* Packages with conflicting version requirements can be built in the same collection
* The resulting wheel collection may not represent a pip-installable set of packages
* Other output files (`build-order.json`, `graph.json`) are still generated normally
- No `constraints.txt` file will be created in the work directory
- Packages with conflicting version requirements can be built in the same collection
- The resulting wheel collection may not represent a pip-installable set of packages
- Other output files (`build-order.json`, `graph.json`) are still generated normally

This option is useful for building large package indexes or collections where version conflicts are acceptable.

Expand Down Expand Up @@ -157,10 +157,10 @@ wheels-repo
└── simple
```

* The `build` sub-directoy holds temporary builds. We use it as the output directory when building the wheel because we can't predict the filename, and so using an empty directory with a name we know gives us a way to find the file and move it into the `downloads` directory after it's built
* The `downloads` sub-directory contains the wheels in `.whl` format that fromager builds combined with the pre-built wheels so we can create a local package index in `simple`
* The `prebuilt` sub-directory contains wheels that are being used as prebuilt
* The `simple` sub-directory is laid out as a simple local wheel index.
- The `build` sub-directoy holds temporary builds. We use it as the output directory when building the wheel because we can't predict the filename, and so using an empty directory with a name we know gives us a way to find the file and move it into the `downloads` directory after it's built
- The `downloads` sub-directory contains the wheels in `.whl` format that fromager builds combined with the pre-built wheels so we can create a local package index in `simple`
- The `prebuilt` sub-directory contains wheels that are being used as prebuilt
- The `simple` sub-directory is laid out as a simple local wheel index.

For example, the `wheels-repo` for `stevedore` package looks as follows:

Expand Down Expand Up @@ -207,14 +207,14 @@ work-dir

```

* The `build-order.json` file is an output file that contains the bottom-up order in which the dependencies need to be built for a specific wheel. You can find more details in the [build-order.json documentation](https://fromager.readthedocs.io/en/latest/files.html#build-order-json)
* The `constraints.txt` is the output file, produced by fromager, showing all of the versions of the packages that are install-time dependencies of the top-level items (note: this file is not generated when using the `--skip-constraints` option)
* The `graph.json` is an output file that contains all the paths fromager can take to resolve a dependency during building the wheel. You can find more details in the [graph.json documentation](https://fromager.readthedocs.io/en/latest/files.html#graph-json)
* The `logs` sub-directory contains detailed logs for fromager's `build-sequence` command including various settings and overrides for each individual package and its dependencies whose wheel was built by fromager. Each log file also contains information about build-backend dependencies if present for a given package
* The `work-dir` also includes sub-directories for the package and its dependencies. These sub-directories include various types of requirements files including `build-backend-requirements.txt`, `build-sdists-requirements.txt`, `build-system-requirements.txt` and the general `requirements.txt`.
* Files like `build.log` which store the logs generated by pip and `build-meta.json` that stores the metadata for the build are also located in `work-dir`.
* These sub-directories also include all the other relevant information for a particular package. Each sub-directory of the package will also contain the unpacked source code of each wheel that is used for the build if `--no-cleanup` option of fromager is used.
* For example, in the above directory structure, for `simple-package-foo` requirement, we will have a subdirectory titled `simple-package-foo` that holds the unpacked source code
- The `build-order.json` file is an output file that contains the bottom-up order in which the dependencies need to be built for a specific wheel. You can find more details in the [build-order.json documentation](https://fromager.readthedocs.io/en/latest/files.html#build-order-json)
- The `constraints.txt` is the output file, produced by fromager, showing all of the versions of the packages that are install-time dependencies of the top-level items (note: this file is not generated when using the `--skip-constraints` option)
- The `graph.json` is an output file that contains all the paths fromager can take to resolve a dependency during building the wheel. You can find more details in the [graph.json documentation](https://fromager.readthedocs.io/en/latest/files.html#graph-json)
- The `logs` sub-directory contains detailed logs for fromager's `build-sequence` command including various settings and overrides for each individual package and its dependencies whose wheel was built by fromager. Each log file also contains information about build-backend dependencies if present for a given package
- The `work-dir` also includes sub-directories for the package and its dependencies. These sub-directories include various types of requirements files including `build-backend-requirements.txt`, `build-sdists-requirements.txt`, `build-system-requirements.txt` and the general `requirements.txt`.
- Files like `build.log` which store the logs generated by pip and `build-meta.json` that stores the metadata for the build are also located in `work-dir`.
- These sub-directories also include all the other relevant information for a particular package. Each sub-directory of the package will also contain the unpacked source code of each wheel that is used for the build if `--no-cleanup` option of fromager is used.
- For example, in the above directory structure, for `simple-package-foo` requirement, we will have a subdirectory titled `simple-package-foo` that holds the unpacked source code

For example, the `work-dir` for `stevedore` package after `bootstrap` command looks as follows:

Expand Down
Loading
Loading