Skip to content

run_pylint: parallelism, pre-commit hook, and lint fixes#3381

Merged
Lestropie merged 2 commits into
devfrom
pylint
Jun 3, 2026
Merged

run_pylint: parallelism, pre-commit hook, and lint fixes#3381
Lestropie merged 2 commits into
devfrom
pylint

Conversation

@Lestropie
Copy link
Copy Markdown
Member

run_pylint was causing some grief, and I've definitely been frustrated in the past pushing Python code changes only to have the pylint CI test show that the code is completely non-executable. It can take a very long time to run though, hence not always running it through before pushing code. So I thought I'd set Claude at improving its efficiency and integrating it as a pre-commit hook. It can now run only over those files modified by a commit, and multi-thread running across files.

Notably, what is run in the pre-commit hook is not the same comprehensive check as that of the CI. I quite often leave TODO markers in one code comment intending to fix something in a subsequent comment, and so precluding the presence of those flags in individual commits would preclude that kind of usage. Some can also take longer to execute, so skipping such checks in the pre-commit hook should be beneficial.

Claude also told me off that run_pylint was ignoring some Python source code files in the repo, so I let it run across those and rectify issues.

Reworked the run_pylint script to analyse files concurrently, one
single-threaded pylint process per file, with a configurable worker
count that defaults to the host processor count. Added a --precommit
mode that layers a reduced check set over the base configuration and
wired it into a new pre-commit hook driven by staged filenames, and
broadened the default scope to all tracked Python files in the
repository. Corrected the Python sources newly brought into lint scope
by that broadened coverage, fixing genuine defects, naming and style
violations, and reindenting to the project's two-space standard.

Session prompts:
1. > Custom Bash script "run_pylint" is responsible for executing
   > pylint across the code base. A set of modifications are to be
   > implemented.
   > 1. The script should be capable of operating in a multi-threaded
   >    fashion. Analysis of each individual file can only run
   >    single-threaded, but independent processes can analyse
   >    multiple files in parallel. A command-line option must be
   >    made available to permit manual specification of how many
   >    such processes can run in parallel; in the absence of such an
   >    option, the maximal number of parallel threads should be
   >    determined by the number of processors available on the
   >    executing system.
   > 2. The script should have a "--diff" operating mode similar to
   >    that introduced to the check_syntax script. When executed in
   >    this mode, only those Python files with git staged changes
   >    are to be processed, and the command return code must
   >    reflect the presence of at least one error in at least one
   >    such file.
   > 3. The script is to be trialed as a pre-commit hook. It should
   >    be executed in --diff mode, using an alternative
   >    configuration file with a reduced set of tests. Initially
   >    only the detection of TODO flags will be excluded from the
   >    existing checks.
   > Devise a plan for these changes. Comment on the feasibility of
   > implementing step 1 while remaining as a Bash script rather
   > than porting to Python. Examine the set of conditions currently
   > imposed by the MRtrix3 pylint configuration and propose a set
   > of checks to be disabled when executing as a pre-commit hook:
   > this can consider both execution speed (pre-commit hooks
   > should be fast) and suitability (some violations might be OK to
   > be present in a specific commit but need to be rectified for CI
   > before merging a Pull Request).
2. > Apply the requisite corrections to the Python source code files
   > newly checked by the run_pylint script.

Generated-by: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 3, 2026

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 3, 2026

clang-tidy review says "All clean, LGTM! 👍"

@Lestropie Lestropie merged commit 8659dab into dev Jun 3, 2026
6 of 7 checks passed
@Lestropie Lestropie deleted the pylint branch June 3, 2026 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant