ruby

Fixing Thor’s CLI Quirks

I’ve been wanting to build a CLI in Ruby, and Thor is the obvious library to use. But Thor has some weird behaviors that were driving me crazy. Here’s what I found, and how I extended Thor to fix it.

Matt Brictson August 2023 – 8 minute read

If you’d like to skip the explanation and jump straight to the code, check out thor_ext.rb in my gem template on GitHub. You can use the template to quickly bootstrap an executable Ruby gem.

What developers expect

CLIs for modern developer tools follow certain conventions that we’ve all learned to expect. If I’m building CLI, here are a few things that I consider non-negotiable.

The features of the CLI should be packaged into subcommands.
Each subcommand should be able to accept its own arguments and options.
I can get help for any subcommand by passing the -h or --help option.
If a subcommand fails or if I use it incorrectly (e.g. by passing in bad options), I should get a human-readable error message and a non-zero exit status.

There are many CLIs that follow these patterns; git is probably the most commonly used:

$ git show 53c987b --numstat

Passing arguments and options to a subcommand.

$ git show --help

GIT-SHOW(1)

NAME
    git-show - Show various types of objects

SYNOPSIS
    git show [<options>] [<object>...]

DESCRIPTION
    ...

Help for a subcommand.

$ git shoe
git: 'shoe' is not a git command. See 'git --help'.

$ echo $?
1

Errors are human-readable and return a non-zero exit status.

Many other CLIs also follow these patterns, like brew, gh, gem, rbenv, rails, and the list goes on.

Thor is… different

Thor is a popular library for building CLIs in Ruby. Let’s see how it behaves when we use it to implement a bare-bones CLI with a show subcommand.

#!/usr/bin/env ruby

require "thor"

class CLI < Thor
  desc "show [OBJECT]", "Show various types of objects"
  method_option :numstat, type: :boolean
  def show(object = "head")
    say "Got arg: #{object}"
    say "with --numstat" if options[:numstat]
  end
end

CLI.start(ARGV)

cli.rb
Thor makes it really easy to declare subcommands, help text, options, and optional arguments. Just subclass Thor and start the resulting CLI using .start. So far, so good!

The subcommand works and accepts an argument and option as expected:

$ ./cli.rb show
Got arg: head

$ ./cli.rb show 53c987b --numstat
Got arg: 53c987b
with --numstat

But if we dig deeper, things start becoming less familiar.

`-h` and `--help` don’t work

$ ./cli.rb show --help
Got arg: --help

Thor interprets --help as an argument, not an option.

Instead of the standard -h or --help option, Thor wants us to prefix the subcommand with the word help, like this:

$ ./cli.rb help show
Usage:
  cli.rb show [OBJECT]

Options:
  [--numstat], [--no-numstat]

Show various types of objects

Thor does offer nicely-formatted help, but in a different way.

Exit status is non-standard

$ ./cli.rb shoe
Could not find command "shoe".
Did you mean?  "show"

$ echo $?
0

Errors should return a non-zero exit status, but Thor doesn’t follow this rule when you mistype commands or options.

Exceptions aren’t handled

Let’s say something goes wrong in a subcommand, causing an exception to be raised:

def show
  raise "not a git repository (or any of the parent directories): .git"
end

Here’s what happens:

$ ./cli.rb show
./cli.rb:9:in `show': not a git repository (or any of the parent directories): .git (RuntimeError)
    from thor-1.2.2/lib/thor/command.rb:27:in `run'
    from thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
    from thor-1.2.2/lib/thor.rb:392:in `dispatch'
    from thor-1.2.2/lib/thor/base.rb:485:in `start'
    from ./cli.rb:15:in `<main>'

$ echo $?
1

Thor shows the full stack trace on an exception, which is not very user-friendly. The exit status is correct this time, though!

Making Thor CLIs more user-friendly

I want to use Thor, but I can’t reasonably release a CLI in 2023 that doesn’t even understand the --help option! Hence my journey to “fix” Thor’s behavior.

The first problem is that Thor treats unrecognized options as arguments. That causes --help to get passed along as an argument, which is not desireable. Thankfully this is an easy fix; you just have to opt into it:

class CLI < Thor
  check_unknown_options!

  # ...
end

The check_unknown_options! declaration tells Thor to be more strict about what options it will accept.

You’ll notice that --help still doesn’t work, but at least it is interpreted as an option now:

$ ./cli.rb show --help
Unknown switches "--help"

Now we’re getting somewhere!

Handling `-h` and `--help`

At this point, Thor now complains that -h and --help are “unknown switches”. My solution to this is to detect this error and retry the command with the arguments rearranged the way Thor prefers, which is help SUBCOMMAND (instead of SUBCOMMAND --help).

Here’s a snippet of code that accomplishes this (I’ll explain how I integrated it in a later section):

def handle_help_switches(given_args)
  yield(given_args.dup)
rescue Thor::UnknownArgumentError => e
  retry_with_args = []

  if given_args.first == "help"
    retry_with_args = ["help"] if given_args.length > 1
  elsif (e.unknown & %w[-h --help]).any?
    retry_with_args = ["help", (given_args - e.unknown).first]
  end
  raise unless retry_with_args.any?

  yield(retry_with_args)
end

This method will attempt to yield to a command with a given set of args. If Thor’s execution of the command fails due to an unknown argument, the code checks if that unknown argument was -h or --help. In that case it rearranges the arguments with help as the first argument (i.e. Thor’s preferred order), so that Thor will show the appropriate help page.

Fixing the exit status and error messages

As shown in earlier examples, Thor’s approach to handing errors either returns the wrong exit status or shows a full stack trace. I’ve found that correcting this requires some custom exception handling code. Here’s what I came up with:

def handle_exception_on_start(error, config)
  # Thor's source code comments say EPIPE errors are safe to ignore
  return if error.is_a?(Errno::EPIPE)

  # Re-raise (thus showing full stack trace) if user has opted into VERBOSE
  raise if ENV["VERBOSE"] || !config.fetch(:exit_on_failure, true)

  # Show Thor error messages (e.g. "Unknown switches") as-is;
  # otherwise prepend error class for additional context
  message = error.message.to_s
  message.prepend("[#{error.class}] ") if message.empty? || !error.is_a?(Thor::Error)

  # Print the error in red if the console supports it
  config[:shell]&.say_error(message, :red)

  # Ensure the CLI exits with a non-zero status
  exit(false)
end

This makes Thor report errors in a much friendlier way, while still giving users access to the full stack trace if they need it.

Putting it all together

To get Thor to use my error and help-switch handling methods, I had to override Thor’s start method. Here are all the patches packaged into a module:

# Configures Thor to behave more like a typical CLI, with better help and error handling.
#
# - Passing -h or --help to a command will show help for that command.
# - Unrecognized options will be treated as errors (instead of being silently ignored).
# - Error messages will be printed in red to stderr, without stack trace.
# - Full stack traces can be enabled by setting the VERBOSE environment variable.
# - Errors will cause Thor to exit with a non-zero status.
#
# To take advantage of this behavior, your CLI should subclass Thor and extend this module.
#
#   class CLI < Thor
#     extend FriendlyCLI
#   end
#
# Start your CLI with:
#
#   CLI.start
#
# In tests, prevent Kernel.exit from being called when an error occurs, like this:
#
#   CLI.start(args, exit_on_failure: false)
#
module FriendlyCLI
  def self.extended(base)
    super
    base.check_unknown_options!
  end

  def start(given_args=ARGV, config={})
    config[:shell] ||= Thor::Base.shell.new
    handle_help_switches(given_args) do |args|
      dispatch(nil, args, nil, config)
    end
  rescue StandardError => e
    handle_exception_on_start(e, config)
  end

  private

  def handle_help_switches(given_args)
    yield(given_args.dup)
  rescue Thor::UnknownArgumentError => e
    retry_with_args = []

    if given_args.first == "help"
      retry_with_args = ["help"] if given_args.length > 1
    elsif (e.unknown & %w[-h --help]).any?
      retry_with_args = ["help", (given_args - e.unknown).first]
    end
    raise unless retry_with_args.any?

    yield(retry_with_args)
  end

  def handle_exception_on_start(error, config)
    return if error.is_a?(Errno::EPIPE)
    raise if ENV["VERBOSE"] || !config.fetch(:exit_on_failure, true)

    message = error.message.to_s
    message.prepend("[#{error.class}] ") if message.empty? || !error.is_a?(Thor::Error)

    config[:shell]&.say_error(message, :red)
    exit(false)
  end
end

This module is also available on GitHub.

Trying it out

Here’s how the example CLI behaves after adding extend FriendlyCLI:

$ ./cli.rb show head
[RuntimeError] not a git repository (or any of the parent directories): .git

$ echo $?
1

Errors are nicely formatted and return a non-zero exit status. (The error is shown in red on color-capable consoles.)

$ ./cli.rb show --quietly
Unknown switches "--quietly"

$ echo $?
1

The nicer error handling also applies to unrecognized options.

$ ./cli.rb show --numstat 5c273cf
Got arg: 5c273cf
with --numstat

Valid options and arguments are still handled.

$ ./cli.rb show --help
Usage:
  cli.rb show [OBJECT]

Options:
  [--numstat], [--no-numstat]

Show various types of objects

The --help option now works as expected.

$ ./cli.rb show -h
Usage:
  cli.rb show [OBJECT]

Options:
  [--numstat], [--no-numstat]

Show various types of objects

-h works too.

Conclusion

After doing the research, I still consider Thor to be the best option for building CLIs in Ruby, despite its quirks.

Thor was born as a task runner and as an alternative to Rake, before evolving into what it is predominantly used for today: a battle-tested CLI library that underpins bundle, rails, and many other mission-critical Ruby executables.¹ This history explains some of its baggage.

I’m hoping that the patches I’ve shared in this article will make Thor a bit more approachable. If you’d like to build a CLI in Ruby, check out my gem template, which includes a ThorExt::Start module based on the code from this post.

And about that CLI I’m building… stay tuned! 😁

It’s worth noting that popular Ruby CLIs have their own workarounds for Thor’s oddities: Bundler adds support for -h and --help using this hack, for example. ↩