Build Python Click Like Command-line Applications With Argparse Part 1

Source

Click is a popular package to create Python command-Line applications. And it's for a good reason. Using decorators keeps the command-line configuration next to the handling code. It is easier to relate configuration with handling code.

The argparse experience can be improved to gain Click like decorator based configuration.

The Simple command case

Let's start with a single command build script with a few options:

  • a source positional argument to specify the build source

  • a --output option to specify the build result directory

  • a --clean flag to cleanup things before building

  • a repeatable --verbose flag to increase the verbosity level

Using argparse the classic way

When using argparse the classic way, arguments are defined in a separate location from where they will be used:

build-python-click-like-command-line-applications-with-argparse/build_argparse_classic.py (Source)

import argparse
from pathlib import Path


def parse_args(args=None):
    parser = argparse.ArgumentParser(description=build.__doc__)

    parser.add_argument("source", default="./src", type=Path, help="Source directory.")

    parser.add_argument(
        "--output", "-o", default="./build", type=Path, help="Build directory."
    )

    parser.add_argument(
        "--clean", "-c", action="store_true", help="Clean before build."
    )

    parser.add_argument(
        "--verbose",
        "-v",
        action="count",
        default=0,
        dest="verbosity",
        help="Add more verbosity.",
    )

    return parser.parse_args(args)


def build(source, output, clean, verbosity):
    """Build Stuff."""
    print("Build:")
    print(f"  - source: {source}")
    print(f"  - output: {output}")
    print(f"  - clean: {clean}")
    print(f"  - verbosity: {verbosity}")


def main(args=None):
    args = parse_args(args)
    build(args.source, args.output, args.clean, args.verbosity)


if __name__ == "__main__":
    main()

The generated help looks like:

build-python-click-like-command-line-applications-with-argparse/build_argparse_help.txt (Source)

usage: argparse_decorators.py [-h] [--verbose] [--clean] [--output OUTPUT] source

Build Stuff.

positional arguments:
  source                Source directory.

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Add more verbosity.
  --clean, -c           Clean before build.
  --output OUTPUT, -o OUTPUT
                        Build directory.

Using Click

When using click, arguments are defined where they will be used:

build-python-click-like-command-line-applications-with-argparse/build_click.py (Source)

from pathlib import Path

import click


@click.command()
@click.argument("source", default="./src", type=Path, help="Source directory.")
@click.option("--output", "-o", default="./build", type=Path, help="Build directory.")
@click.option("--clean", "-c", is_flag=True, help="Clean before build.")
@click.option("--verbose", "-v", "verbosity", count=True, help="Add more verbosity.")
def build(source, output, clean, verbosity):
    """Build Stuff."""
    print("Build:")
    print(f"  - source: {source}")
    print(f"  - output: {output}")
    print(f"  - clean: {clean}")
    print(f"  - verbosity: {verbosity}")


if __name__ == "__main__":
    build()

Using argparse the Click way

We can have something similar using argparse:

build-python-click-like-command-line-applications-with-argparse/build_argparse_decorators.py (Source)

from pathlib import Path

from argparse_decorators import add_argument as arg
from argparse_decorators import get_main


@arg("source", default="./src", type=Path, help="Source directory.")
@arg("--output", "-o", default="./build", type=Path, help="Build directory.")
@arg("--clean", "-c", action="store_true", help="Clean before build.")
@arg(
    "--verbose",
    "-v",
    action="count",
    default=0,
    dest="verbosity",
    help="Add more verbosity.",
)
def build(source, output, clean, verbosity):
    """Build Stuff."""
    print("Build:")
    print(f"  - source: {source}")
    print(f"  - output: {output}")
    print(f"  - clean: {clean}")
    print(f"  - verbosity: {verbosity}")


main = get_main(build)


if __name__ == "__main__":
    main()

At first build the main() instead of just calling build() might seems strange. However, it has its advantages:

  • main can be called with command-line args main("./src", "--clean")

  • build is not modified and its call matches its signature: build(Path("./src"), clean=True).

Those are valuablbe interfaces when testing or when using the script as a library.

How does it works

add_argument creates build.__argparse_annotations__ if it does not exists and appends information in it for later use.

Then get_main reads build.__argparse_annotations__ to build an argparse.ArgumentParser and build the main function

Full listing:

build-python-click-like-command-line-applications-with-argparse/argparse_decorators.py (Source)

import inspect
from argparse import ArgumentParser


def get_main(function):
    """Create a main function for a command.

    The main function will be callable using the command-line arguments:
    `main('--flag', '--option=option')`

    The command will be called according to its argspec using available args.
    """
    parser = get_parser(function)

    def main(*args: str):
        namespace = parser.parse_args(args if args else None)
        return call(function, namespace)

    return main


def get_parser(function):
    """Create a parser for a function.

    The parser description will use the function docstring.

    Example:

    .. code: python

        from argparse_decorators import add_argument, get_parser

        @add_argument("argument")
        def command(argument)
            ...

        parser = get_parser(command)

    It's same as:

    .. code: python

        import argparse

        from argparse_decorators import add_argument, get_parser

        def command(argument)
            ...

        parser = argparse.ArgumentParser(description=command.__doc__)
        parser.add_argument("argument")


    The parser will be configured according to __argparse_annotations__.
    """
    parser = ArgumentParser(description=function.__doc__)
    configure_parser(parser, function)
    return parser


def configure_parser(parser, command):
    """Configure a parser from a decorated function."""
    annotations = getattr(command, "__argparse_annotations__", [])
    for args, kwargs in annotations:
        parser.add_argument(*args, **kwargs)


def call(function, args):
    """Call a function according to its argspec using available cli args.

    Example:

    .. code: python

        import argparse

        from argparse_decorators import add_argument, call


        @add_argument("argument")
        def command(argument)
            assert argument == "value"


        call(command, argparse.Namespace(argument="value"))

    The parser will be configured according to __argparse_annotations__.
    """
    argspec = inspect.getargspec(function)
    call_args = {arg: getattr(args, arg) for arg in argspec.args}
    return function(**call_args)


def add_argument(*args, **kwargs):
    """Build a decorator that will add argparse annotations to a function."""
    return lambda decorated: _add_annotation(decorated, (args, kwargs))


def _add_annotation(function, argument):
    """Add argparse annotations to a function."""
    annotations = getattr(function, "__argparse_annotations__", [])
    annotations.append(argument)
    setattr(function, "__argparse_annotations__", annotations)
    return function

In part 2 we will see:

  • How to fallback to classic argparse if needed without loosing all the benefits from décorators

  • How to extend to simplify some parts (like a better --verbose declaration).