750: Teach ad-hoc hooks line parsing

CHANGELOG.md

@@ -1,5 +1,9 @@
 # Overcommit Changelog
 
+## main (unreleased)
+
+* Add configurable line parsing for command hooks (#750).
+
 ## 0.75.0
 
 * Remember all locally approved configuration and hook signatures across rebases

README.md

@@ -13,7 +13,8 @@
 In addition to supporting a wide variety of hooks that can be used across
 multiple repositories, you can also define hooks specific to a repository which
 are stored in source control. You can also easily
-[add your existing hook scripts](#adding-existing-git-hooks) without writing
+[add your existing hook scripts](#adding-existing-git-hooks) or
+[use some of your existing line-oriented analysis tools](#adding-existing-line-aware-commands) without writing
 any Ruby code.
 
 - [Requirements](#requirements)
@@ -47,6 +48,7 @@ any Ruby code.
   - [PreRebase](#prerebase)
 - [Repo-Specific hooks](#repo-specific-hooks)
   - [Adding Existing Git Hooks](#adding-existing-git-hooks)
+  - [Adding Existing Line-Aware Commands](#adding-existing-line-aware-commands)
 - [Security](#security)
   - [Disabling Signature Checking](#disabling-signature-checking)
 - [Contributing](#contributing)
@@ -256,6 +258,8 @@ Option                                  | Description
 `install_command`                       | Command the user can run to install the `required_executable` (or alternately the specified `required_libraries`). This is intended for documentation purposes, as Overcommit does not install software on your behalf since there are too many edge cases where such behavior would result in incorrectly configured installations (e.g. installing a Python package in the global package space instead of in a virtual environment).
 `skip_file_checkout`                    | Whether to skip this hook for file checkouts (e.g. `git checkout some-ref -- file`). Only applicable to `PostCheckout` hooks.
 `skip_if`                               | Array of arguments to be executed to determine whether or not the hook should run. For example, setting this to a value of `['bash', '-c', '! which my-executable']` would allow you to skip running this hook if `my-executable` was not in the bin path.
+`message_pattern`                       | Regex used to parse diagnostics from [line-aware command hooks](#adding-existing-line-aware-commands). Requires a named `file` capture; optional `line` and `type` captures provide line handling and warning classification.
+`warning_message_type_pattern`          | Optional regex (or literal substring) identifying warnings in the `type` capture of a line-aware command hook. All other diagnostics are errors.
 
 In addition to the built-in configuration options, each hook can expose its
 own unique configuration options. The `AuthorEmail` hook, for example, allows
@@ -705,6 +709,52 @@ of hook, see the [git-hooks documentation][GHD].
 
 [GHD]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
 
+### Adding Existing Line-Aware Commands
+
+Tools that report file diagnostics can use Overcommit's message handling without
+writing a Ruby hook. Add a `message_pattern` regex to an existing command hook to
+parse each line of its standard output. The command receives the applicable file
+paths after its configured command and flags, instead of Git hook arguments or
+standard input.
+
+`message_pattern` is required to enable this behavior and has no default. It must
+be a Ruby regex with a named `file` capture. An optional `line` capture enables
+`problem_on_unmodified_line` handling. An optional `type` capture can be classified
+using `warning_message_type_pattern`, which accepts a regex or a literal substring.
+Without a matching warning type, diagnostics are errors. Other captures are ignored.
+
+For example, to parse GNU/Emacs-style diagnostics such as
+`file.rb:12: warning: unused variable`:
+
+```yaml
+PreCommit:
+  CustomScript:
+    enabled: true
+    command: './bin/custom-script'
+    include: '**/*.rb'
+    message_pattern: !ruby/regexp /^(?<file>(?:\w:)?[^:]+):(?<line>\d+):[^ ]* (?<type>[^ ]+)/
+    warning_message_type_pattern: !ruby/regexp /^warning:/i
+```
+
+The normal `include`, `exclude`, and `requires_files` options control which files
+are passed to the command and whether it runs. `problem_on_unmodified_line`
+controls how diagnostics on unchanged lines affect the result.
+
+A successful command passes. For a failed command, the parsed diagnostics determine
+whether the hook fails or warns. A failed command with no standard output still
+fails and displays its standard error. Unexpected output that does not match
+`message_pattern` raises a message processing error instead of being silently ignored.
+
+Only standard output is parsed. For tools that write diagnostics to standard error,
+use a wrapper script that redirects it with `2>&1`.
+
+These options apply to command hooks without an existing built-in or repository
+Ruby implementation. See the [Ruby regex documentation][RubyRE] for named captures.
+To serialize a regex as YAML, use `require 'yaml'; puts YAML.dump(/MY-REGEXP/)` in
+`irb` and omit the leading `---` from the result.
+
+[RubyRE]: https://ruby-doc.org/core-2.7.3/Regexp.html
+
 ## Security
 
 While Overcommit can make managing Git hooks easier and more convenient,

lib/overcommit/configuration_loader.rb

@@ -28,7 +28,7 @@ def load_from_file(file, options = {})
         # default. Explicitly enable aliases if the option is available.
         yaml =
           begin
-            YAML.load_file(file, aliases: true)
+            YAML.load_file(file, permitted_classes: [Regexp, Symbol], aliases: true)
           rescue ArgumentError
             YAML.load_file(file)
           end

lib/overcommit/hook_loader/base.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'overcommit/utils/messages_utils'
+
 module Overcommit::HookLoader
   # Responsible for loading hooks from a file.
   class Base
@@ -22,6 +24,40 @@ def load_hooks
 
     private
 
+    def validate_line_aware_config(hook_config)
+      pattern = hook_config['message_pattern']
+      unless pattern.is_a?(Regexp) && pattern.names.include?('file')
+        raise Overcommit::Exceptions::HookLoadError,
+              '`message_pattern` must be a Regexp with a named `file` capture group'
+      end
+
+      warning_pattern = hook_config['warning_message_type_pattern']
+      unless warning_pattern.nil? || warning_pattern.is_a?(Regexp) || warning_pattern.is_a?(String)
+        raise Overcommit::Exceptions::HookLoadError,
+              '`warning_message_type_pattern` must be a Regexp or String'
+      end
+    end
+
+    def create_line_aware_command_hook_class(hook_base)
+      Class.new(hook_base) do
+        def run
+          result = execute(command, args: applicable_files)
+          return :pass if result.success?
+
+          output = result.stdout.split("\n")
+          return [:fail, result.stderr] if output.empty?
+
+          Overcommit::Utils::MessagesUtils.extract_messages(
+            output,
+            @config['message_pattern'],
+            Overcommit::Utils::MessagesUtils.create_type_categorizer(
+              @config['warning_message_type_pattern']
+            )
+          )
+        end
+      end
+    end
+
     attr_reader :log
 
     # Load and return a {Hook} from a CamelCase hook name.

lib/overcommit/hook_loader/plugin_hook_loader.rb

@@ -74,23 +74,38 @@ def check_for_unverified_plugins
       raise Overcommit::Exceptions::InvalidHookSignature
     end
 
-    def create_ad_hoc_hook(hook_name)
-      hook_module = Overcommit::Hook.const_get(@context.hook_class_name)
-      hook_base = hook_module.const_get('Base')
-
+    def create_git_hook_class(hook_base)
       # Implement a simple class that executes the command and returns pass/fail
       # based on the exit status
-      hook_class = Class.new(hook_base) do
+      Class.new(hook_base) do
         def run
           result = @context.execute_hook(command)
-
           if result.success?
             :pass
           else
             [:fail, result.stdout + result.stderr]
           end
         end
       end
+    end
+
+    def create_ad_hoc_hook(hook_name)
+      hook_module = Overcommit::Hook.const_get(@context.hook_class_name)
+      hook_base = hook_module.const_get('Base')
+
+      hook_config = @config.for_hook(hook_name, @context.hook_class_name)
+      hook_class =
+        if hook_config['message_pattern']
+          validate_line_aware_config(hook_config)
+          create_line_aware_command_hook_class(hook_base)
+        else
+          create_git_hook_class(hook_base)
+        end
+
+      # A later load may configure the same hook to use a different command format.
+      if hook_module.const_defined?(hook_name, false)
+        hook_module.send(:remove_const, hook_name)
+      end
 
       hook_module.const_set(hook_name, hook_class).new(@config, @context)
     rescue LoadError, NameError => e

lib/overcommit/utils/messages_utils.rb

@@ -37,6 +37,20 @@ def extract_messages(output_messages, regex, type_categorizer = nil)
         end
       end
 
+      def create_type_categorizer(warning_pattern)
+        return nil if warning_pattern.nil?
+
+        lambda do |type|
+          warning =
+            if warning_pattern.is_a?(Regexp)
+              type && warning_pattern.match?(type)
+            else
+              type&.include?(warning_pattern)
+            end
+          warning ? :warning : :error
+        end
+      end
+
       private
 
       def extract_file(match, message)

spec/overcommit/configuration_loader_spec.rb

@@ -44,6 +44,23 @@
         subject.plugin_directory.should == File.expand_path('some-directory')
       end
 
+      context 'and command hooks use regular expressions in YAML' do
+        let(:config_contents) { <<-'CFG' }
+          PreCommit:
+            CustomScript:
+              enabled: true
+              command: custom-script
+              message_pattern: !ruby/regexp /^(?<file>[^:]+):(?<line>\d+):[ ](?<type>\w+):/
+              warning_message_type_pattern: !ruby/regexp /^warning$/i
+        CFG
+
+        it 'loads both patterns as regular expressions' do
+          hook_config = subject.for_hook('CustomScript', 'PreCommit')
+          expect(hook_config['message_pattern']).to be_a(Regexp)
+          expect(hook_config['warning_message_type_pattern']).to eq(/^warning$/i)
+        end
+      end
+
       context 'and the configuration file contains a hook with no `enabled` option' do
         let(:config_contents) { <<-CFG }
           PreCommit: