Lorikeet

Lorikeet is a Scalafix-based code quality feedback tool that lets you define custom rules using query patterns and rewrite templates. It can be used for automated grading and feedback on student assignments, as well as a dev tool to simplify writing custom Scalafix rules.

Writing rules requires no knowledge of Scala's AST or the Scalafix API, and allows you to express complex patterns and rewrites with a simple and intuitive syntax.

The included script Check.scala allows easily running a set of custom rules on a large number of student submissions, and provides detailed feedback and statistics on the results.

Development

Run Tests

sbt "tests / test"

Publish Locally

sbt "lorikeet / publishLocal"

Usage

This section describes how to use Lorikeet:

• To use it in your own project, see the Using the MetaRule section below.
• To grade student submissions, see the Running a Check on Student Submissions section below.

For more details on how to write custom rules and the syntax of query patterns and rewrite templates, see the Guide.

Using the MetaRule

1. Publish the rule locally using the command above, or use the published version if available.
2. Add sbt-scalafix and sbt-scalafmt to your project/plugins.sbt file:

addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.14.4")
addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.5.6")

3. Add the rule as a dependency and semanticdb support to your build.sbt file:

semanticdbEnabled := true
semanticdbVersion := scalafixSemanticdb.revision

scalafixDependencies += "ch.epfl.sidoniebouthors" % "lorikeet_3" % "0.1.0-SNAPSHOT"

4. Create a .lorikeet.conf file in the root of your project with your custom rule configuration (see the Guide for syntax and examples).

5. Run Scalafix on your project, specifying the rule name:

sbt "scalafix MetaRule"

6. Optionally, if you need to test and modify your rules, disable scalafix caching in your build.sbt file:

scalafixCaching := false

This is because sbt task caching will avoid rerunning a task that has already been run with the same arguments and scala input files, but changes to the custom rules configuration file .lorikeet.conf are not considered and would not trigger a re-run of scalafix.

Running a Check on Student Submissions

See script Check.scala.

This script expects a submission directory with the following structure:

submission/
├── SCIPER
│   └── 0
│       └── assignment.scala
|   └── 1
│       └── assignment.scala
|   └── ...
├── SCIPER
│   └── 0
│       └── assignment.scala
|   └── ...
└── ...

The script also expects a scaffold directory containing the lab sbt project that has been configured to use the custom rules, as described above.

To use the Check.scala script, you also need to provide a .scalafmt.conf file in the root of your scaffold project. Scalafmt will be run before and after applying the rewrites to ensure proper formatting. A minimal configuration could be:

version = 3.9.9
runner.dialect = scala3

The script will replace the assignment.scala file in the scaffold project with the ones from the submission, then compile and run scalafix check on it, collecting the results in a logs file.

Note that submissions that do not compile will be reported as such but will not be checked with scalafix. This means it may be a good idea to remove -Xfatal-warnings or other such flags from the scaffold project.

The script output will be individual diffs for each submissions, as well as individual feedback (which rules matches, with their descriptions and where specifically in the code), and a summary at the end with statistics on how many submissions passed each rule.

The console output looks something like this:

Diffs directory: ~evaluating/grading_diffs_2026.01.01_14.26.00
Lint reports directory: ~/evaluating/grading_reports_2026.01.01_14.26.00

Starting grading process...

-> ⚠️  ISSUES:  359355 / 0 -> Var Usage (1), If Simplification (12)
-> ⚠️  ISSUES:  361678 / 0 -> If Simplification (12)
-> ⚠️  ISSUES:  356669 / 0 -> Var Usage (5)
-> ⚠️  ISSUES:  380092 / 0 -> If Simplification (5)
-> ✅ SUCCESS: 377073 / 0
-> ⚠️  ISSUES:  378842 / 2 -> Var Usage (4), If Simplification (2)
-> ⚠️  ISSUES:  372197 / 0 -> Var Usage (3), If Simplification (4)
-> ⚠️  ISSUES:  344921 / 0 -> If Simplification (5)
-> ⚠️  ISSUES:  363557 / 0 -> Var Usage (12)
.....

--- SUMMARY ---
Total submissions: 421
Submissions with missing file: 0
Submissions with compile errors: 1
Submissions failing check: 378

--- STATISTICS ---
Submissions with Matches:
  If Simplification: 267
  Var Usage: 135
Total Rule Matches:
  If Simplification: 2609
  Var Usage: 752

Grading complete.

See the configuration options at the top of the script.