Gains tracking

.github/workflows/docs.yml

@@ -0,0 +1,26 @@
+name: Deploy Docs
+
+on:
+  push:
+    branches:
+      - master
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+
+      - name: Install dependencies
+        run: pip install -r requirements-docs.txt
+
+      - name: Deploy with mkdocs
+        run: mkdocs gh-deploy --force

README.md

@@ -6,6 +6,8 @@
 
 Fast, short for "Find AST", is a tool to search, prune, and edit Ruby ASTs.
 
+If you use Fast for LLMs and Agents, it includes an efficiency tracking feature to quantify the savings in terms of context and manual effort. See [Fast Gains (Efficiency Tracking)](https://jonatas.github.io/fast/gains) for more details.
+
 Ruby is a flexible language that allows us to write code in multiple different ways
 to achieve the same end result, and because of this it's hard to verify how
 the code was written without an AST.
@@ -519,6 +521,7 @@ The CLI tool takes the following flags
 - Use `-c` to search from code example
 - Use `-s` to search similar code
 - Use `-p` or `--parallel` to parallelize the search
+- Use `.gains` to show search efficiency statistics
 
 ### Define your `Fastfile`
 

docs/command_line.md

@@ -13,6 +13,26 @@ $ fast '(def match?)' lib/fast.rb
 - Use `-s` to search similar code
 - Use `-p` to or `--parallel` to use multi core search
 - Use `--validate-pattern PATTERN` to check if a pattern is syntactically correct.
+- Use `.gains` to show search efficiency statistics.
+
+## `.gains`
+
+The `.gains` command allows you to track and visualize the efficiency of your code searches. It measures "bytes searched" (the total size of files scanned) versus "bytes reported" (the actual results returned).
+
+```bash
+$ fast .gains
+```
+
+This is particularly useful when using Fast with LLMs and Agents, as it quantifies the reduction in context data that the agent had to process.
+
+You can also filter by category:
+
+```bash
+$ fast .gains cli  # Show only CLI search history
+$ fast .gains mcp  # Show only MCP (Agent) search history
+```
+
+See [Fast Gains (Efficiency Tracking)](gains.md) for more details.
 
 ## `--validate-pattern`
 

docs/gains.md

@@ -0,0 +1,84 @@
+# Fast Gains: Tracking Search Efficiency
+
+Fast includes a "Gains" feature that tracks the efficiency of your code searches and explorations. It measures the amount of data searched versus the amount of data actually reported to quantify the "savings" in terms of context and manual effort.
+
+This is particularly useful when using Fast as a tool for LLMs and Agents (via the [MCP Server](mcp_tutorial.md)), where minimizing unnecessary context is crucial for performance and cost.
+
+## How it Works
+
+The Gains feature monitors three main metrics during a search:
+
+1.  **Bytes Searched**: The total size of all files that Fast scanned.
+2.  **Bytes Reported**: The size of the actual results (AST nodes, source code, or summaries) that were returned to the user or agent.
+3.  **Savings**: The difference between Bytes Searched and Bytes Reported, often expressed as a percentage.
+
+### Example
+
+If you search a project with 10MB of source code and Fast returns a specific 1KB method definition, the "gain" is approximately 9.99MB, or a 99.9% reduction in the data you had to process manually.
+
+## Usage
+
+### Viewing the Report
+
+You can view your accumulated gains history using the `.gains` command:
+
+```bash
+fast .gains
+```
+
+This will display a summary of:
+- Total bytes searched and reported.
+- Total files scanned and matched.
+- Total savings (in bytes and percentage).
+- A bar chart of recent savings history.
+
+### Filtering Results
+
+Fast categorizes gains into two groups:
+- **CLI**: Searches performed directly through the command-line interface.
+- **MCP**: Searches performed by an LLM or Agent via the Model Context Protocol server.
+
+You can filter the report to see only one of these:
+
+```bash
+fast .gains cli
+# or
+fast .gains mcp
+```
+
+## Configuration
+
+Gains tracking is enabled by default in some contexts (like the MCP server) but can be controlled via environment variables or programmatically.
+
+### Environment Variable
+
+To disable gains tracking across all contexts (CLI and MCP), set the `FAST_GAINS` environment variable to `0`:
+
+```bash
+export FAST_GAINS=0
+```
+
+### Ruby API
+
+To enable or disable tracking in Ruby code:
+
+```ruby
+Fast.enable_gain_track!
+Fast.disable_gain_track!
+```
+
+## Storage
+
+Gains data is stored locally on your machine:
+- **Directory**: `~/.fast/`
+- **History File**: `~/.fast/gains.json`
+
+The system uses temporary files for each run and consolidates them into the main history file when you run the `.gains` command. This ensures that concurrent searches (e.g., from multiple agents) don't lose data or cause file locks.
+
+To keep the history file manageable, Fast only keeps the full report content for the last 5 runs, while retaining the metrics for all historical runs.
+
+## Why Track Gains?
+
+- **Quantify Value**: See how much manual "grepping" and reading Fast is saving you.
+- **Optimize Agent Prompts**: High "Bytes Reported" might indicate that your agent's search patterns are too broad and could be refined to save tokens.
+- **Monitor Project Growth**: Track how your project's size affects search performance over time.

docs/index.md

@@ -4,7 +4,7 @@
 
 Fast is a "Find AST" tool to help you search in the code abstract syntax tree.
 
-If you use Fast through `bin/fast-mcp`, treat it as a trusted local development tool. The MCP server can read files, rewrite files, and run experiment commands as the current user. See [Fast MCP Server Tutorial](mcp_tutorial.md) and [Using Fast for LLMs and Agents](agents.md) for the security notes and trust boundary.
+If you use Fast through `bin/fast-mcp`, treat it as a trusted local development tool. The MCP server can read files, rewrite files, and run experiment commands as the current user. See [Fast MCP Server Tutorial](mcp_tutorial.md), [Fast Gains (Efficiency Tracking)](gains.md), and [Using Fast for LLMs and Agents](agents.md) for the security notes and trust boundary.
 
 
 ??? "🍿Watch my talk at Ruby Kaigi: Grepping Ruby code like a boss" 

fast.gemspec

@@ -21,6 +21,7 @@ Gem::Specification.new do |spec|
     lib/fast/cli.rb
     lib/fast/experiment.rb
     lib/fast/git.rb
+    lib/fast/gains.rb
     lib/fast/mcp_server.rb
     lib/fast/node.rb
     lib/fast/prism_adapter.rb
@@ -38,6 +39,10 @@ Gem::Specification.new do |spec|
     bin/fast-experiment
     bin/setup
     bin/console
+    experiments/replace_create_with_build_stubbed.rb
+    experiments/let_it_be_experiment.rb
+    experiments/remove_useless_hook.rb
+    experiments/replace_update_attributes_with_update.rb
     .agents/fast-pattern-expert/SKILL.md
     LICENSE.txt
     README.md

lib/fast.rb

@@ -63,6 +63,22 @@ class SyntaxError < StandardError; end
   /x.freeze
 
   class << self
+    attr_accessor :gain_tracking_enabled
+
+    def enable_gain_track!
+      self.gain_tracking_enabled = true
+    end
+
+    def disable_gain_track!
+      self.gain_tracking_enabled = false
+    end
+
+    def gain_tracking_enabled?
+      return false if ENV['FAST_GAINS'] == '0'
+
+      @gain_tracking_enabled == true
+    end
+
     def ast_node?(node)
       node.respond_to?(:type) && node.respond_to?(:children)
     end
@@ -188,18 +204,18 @@ def search_file(pattern, file)
     # @param [String] pattern
     # @param [Array<String>] *locations where to search. Default is '.'
     # @return [Hash<String,Array<Fast::Node>>] with files and results
-    def search_all(pattern, locations = ['.'], parallel: true, on_result: nil)
+    def search_all(pattern, locations = ['.'], parallel: true, on_result: nil, on_search: nil)
       group_results(build_grouped_search(:search_file, pattern, on_result),
-                    locations, parallel: parallel)
+                    locations, parallel: parallel, on_search: on_search)
     end
 
     # Capture with pattern on a directory or multiple files
     # @param [String] pattern
     # @param [Array<String>] locations where to search. Default is '.'
     # @return [Hash<String,Object>] with files and captures
-    def capture_all(pattern, locations = ['.'], parallel: true, on_result: nil)
+    def capture_all(pattern, locations = ['.'], parallel: true, on_result: nil, on_search: nil)
       group_results(build_grouped_search(:capture_file, pattern, on_result),
-                    locations, parallel: parallel)
+                    locations, parallel: parallel, on_search: on_search)
     end
 
     # @return [Proc] binding `pattern` argument from a given `method_name`.
@@ -224,19 +240,21 @@ def build_grouped_search(method_name, pattern, on_result)
     # while it process several locations in parallel.
     # @param [Boolean] parallel runs the `group_files` in parallel
     # @return [Hash[String, Array]] with files and results
-    def group_results(group_files, locations, parallel: true)
+    def group_results(group_files, locations, parallel: true, on_search: nil)
       files = ruby_files_from(*locations)
       results =
         if parallel
           require 'parallel' unless defined?(Parallel)
           Parallel.map(files) do |file|
+            on_search&.call(file)
             group_files.call(file)
           rescue StandardError => e
             warn "Error processing #{file}: #{e.message}" if Fast.debugging
             nil
           end
         else
           files.map do |file|
+            on_search&.call(file)
             group_files.call(file)
           rescue StandardError => e
             warn "Error processing #{file}: #{e.message}" if Fast.debugging

lib/fast/cli.rb

@@ -4,6 +4,7 @@
 require 'fast/source'
 require 'fast/version'
 require 'fast/sql'
+require 'fast/gains'
 require 'coderay'
 require 'optparse'
 require 'ostruct'
@@ -82,29 +83,39 @@ def first_position_from_expression(node)
   # @param level [Integer] Skip exploring deep branches of AST when showing sexp
   # @example
   #   Fast.report(result, file: 'file.rb')
-  def report(result, show_link: false, show_permalink: false, show_sexp: false, file: nil, headless: false, bodyless: false, colorize: true, level: nil) # rubocop:disable Metrics/ParameterLists
+  def report(result, show_link: false, show_permalink: false, show_sexp: false, file: nil, headless: false, bodyless: false, colorize: true, level: nil, gains: nil) # rubocop:disable Metrics/ParameterLists
     if file
       if result.is_a?(Symbol) && !result.respond_to?(:loc)
         result.extend(SymbolExtension)
       end
       line = result.loc.expression.line if Fast.ast_node?(result) && result.respond_to?(:loc)
       if show_link
-        puts(result.link)
+        puts_and_record(result.link, gains)
       elsif show_permalink
-        puts(result.permalink)
+        puts_and_record(result.permalink, gains)
       elsif !headless
-        puts(highlight("# #{file}:#{line}", colorize: colorize))
+        puts_and_record(highlight("# #{file}:#{line}", colorize: colorize), gains)
       end
     end
-    puts(highlight(result, show_sexp: show_sexp, colorize: colorize, level: level)) unless bodyless
+    puts_and_record(highlight(result, show_sexp: show_sexp, colorize: colorize, level: level), gains) unless bodyless
+  end
+
+  def puts_and_record(content, gains)
+    puts(content)
+    gains&.record_report(content)
   end
 
   # Command Line Interface for Fast
   class Cli # rubocop:disable Metrics/ClassLength
-    attr_reader :pattern, :show_sexp, :pry, :from_code, :similar, :help, :level
+    attr_reader :pattern, :show_sexp, :pry, :from_code, :similar, :help, :level, :gains
     def initialize(args)
+      require 'fast/shortcut'
+      Fast.load_fast_files!
+      @gains = Gains.new(args.join(' '))
       args = args.dup
-      args = replace_args_with_shortcut(args) if shortcut_name_from(args)
+      unless args.include?('.gains')
+        args = replace_args_with_shortcut(args) if shortcut_name_from(args)
+      end
       @colorize = STDOUT.isatty
       @headless = false
       @bodyless = false
@@ -233,6 +244,12 @@ def self.run!(argv)
     def run!
       raise 'pry and parallel options are incompatible :(' if @parallel && @pry
 
+      if @pattern&.start_with?('.gains')
+        filter = @pattern.split(' ')[1]
+        Gains.report(filter)
+        return
+      end
+
       if @help || @files.empty? && @pattern.nil?
         puts option_parser.help
         return
@@ -253,10 +270,13 @@ def run!
 
       if @files.empty?
         ast ||= Fast.public_send( @sql ? :parse_sql : :ast, @pattern)
-        puts Fast.highlight(ast, show_sexp: @show_sexp, colorize: @colorize, sql: @sql, level: @level)
+        content = Fast.highlight(ast, show_sexp: @show_sexp, colorize: @colorize, sql: @sql, level: @level)
+        puts content
+        @gains.record_report(content)
       else
         search
       end
+      @gains.save!
     end
 
     # Create fast expression from node pattern using the command line
@@ -273,6 +293,7 @@ def search
       return Fast.debug(&method(:execute_search)) if debug_mode?
 
       execute_search do |file, results|
+        @gains.record_match(file) if results.any?
         results.each do |result|
           binding.pry if @pry # rubocop:disable Lint/Debugger
           report(file, result)
@@ -283,11 +304,13 @@ def search
     # Executes search for all files yielding the results
     # @yieldparam [String, Array] with file and respective search results
     def execute_search(&on_result)
+      on_search = ->(file) { @gains.record_search(file) }
       Fast.public_send(search_method_name,
                        @pattern,
                        @files,
                        parallel: parallel?,
-                       on_result: on_result)
+                       on_result: on_result,
+                       on_search: on_search)
     end
 
     # @return [Symbol] with `:capture_all` or `:search_all` depending the command line options
@@ -320,7 +343,8 @@ def report(file, result)
                   headless: @headless,
                   bodyless: @bodyless,
                   colorize: @colorize,
-                  level: @level)
+                  level: @level,
+                  gains: @gains)
     end
 
     def shortcut_name_from(args)
@@ -344,11 +368,6 @@ def extract_pattern_and_files(args)
     # Find shortcut by name. Preloads all `Fastfiles` before start.
     # @param name [String]
     def find_shortcut(name)
-      unless defined? Fast::Shortcut
-        require 'fast/shortcut'
-        Fast.load_fast_files!
-      end
-
       shortcut = Fast.shortcuts[name.to_sym]
       exit_shortcut_not_found(name) unless shortcut
       shortcut
@@ -360,7 +379,6 @@ def exit_shortcut_not_found(name)
       puts "Shortcut \033[1m#{name}\033[0m not found :("
       if Fast.shortcuts.any?
         puts "Available shortcuts are: #{Fast.shortcuts.keys.join(', ')}."
-        Fast.load_fast_files!
       end
       exit 1
     end