Command Line Arguments in Python

Overview

With Python being a very popular programming language, as well as having support for most operating systems and many libraries that make command-line argument processing easy - it's become widely used to create command line tools for many purposes. These tools can range from simple CLI apps to those that are more complex, like AWS' aws-cli tool.

Complex tools like this are typically controlled by the user via command line arguments, which allows the user to use specific commands, set options, and more. For example, these options could tell the tool to output additional information, read data from a specified source, or send output to a certain location.

In general, arguments are passed to CLI tools differently, depending on your operating system:

• Unix-like: - followed by a letter, like -h, or -- followed by a word, like --help
• Windows: / followed by either a letter, or word, like /help

These different approaches exist due to historical reasons. Many programs on Unix-like systems support both the single and double dash notation. The single dash notation is mostly used with single letter options, while double dashes present a more readable options list, which is particularly useful for complex options that need to be more explicit.

Keep in mind that both the name and the meaning of an argument are specific to a program - there is no general definition, other than a few common conventions like --help for further information on the usage of the tool. As the developer of a Python script, you will decide which arguments to provide to the caller and what they do. This requires proper evaluation.

As your list of available arguments grows, your code will become more complex in trying to accurately parse them. Luckily, in Python there are a number of libraries available to help you with this. We'll cover a few of the most common solutions, which range from "do-it-yourself" with sys.argv, to the "done-for-you" approach with argparse.

Handling Command Line Arguments with Python

Python 3+ and the ecosystem around supports a number of different ways of handling command line arguments. There are many libraries that facilitate parsing command-line arguments.

The built-in way is to use the sys module. In terms of names, and its usage, it relates directly to the C library (libc).

The second way is the getopt module, which handles both short and long options, including the evaluation of the parameter values.

The argparse module, which is derived from the optparse module (available up to Python 2.7).

The docopt module, which is available on GitHub, also allows the same functionality.

Recently, the absl library has also been gaining steam, as a means to replace optparse and getopt().

Each of these ways has their pros and cons, so it's worth evaluating each to see which suits your needs best.

The sys Module

This is a basic module that has been shipped with Python from the early days. It takes a very similar approach to the C library using argc/argv to access the arguments. The sys module implements the command line arguments in a simple list structure named sys.argv.

Each list element represents a single argument. The first item in the list, sys.argv[0], is the name of the Python script. The rest of the list elements, sys.argv[1] to sys.argv[n], are the command line arguments 2 through n.

As a delimiter between the arguments, a space is used. Argument values that contain a space in it have to be surrounded by quotes in order to be properly parsed by sys.

The equivalent of argc is just the number of elements in the list. To obtain this value, use the Python len() operator. We'll show this in a code example later on.

Printing the First CLI Argument

In this first example, our script will determine the way it was called. This information is kept in the first command line argument, indexed with 0. The code below shows how you obtain the name of your Python script:

import sys

print("The script has the name %s" % (sys.argv[0])

Save this code in a file named arguments-program-name.py, and then call it as shown below. The output is as follows and contains the file name, including its full path:

$ python arguments-program-name.py
The script has the name arguments-program-name.py
$ python /home/user/arguments-program-name.py
The script has the name /home/user/arguments-program-name.py

As you can see from the second call above, we not only get the name of the Python file, but also the full path used to call it.

Counting the Number of Arguments

In this second example we simply count the number of command line arguments using the built-in len() method. sys.argv is the list that we have to examine. In the code below, we get the number of arguments and then subtract 1 because one of those arguments (i.e., the first one) is always set as the name of the file, which isn't always useful to us. Thus, the actual number of arguments passed by the user is len(sys.argv) - 1:

import sys

# Count the arguments
arguments = len(sys.argv) - 1
print ("The script is called with %i arguments" % (arguments))

Save and name this file arguments-count.py. Some examples of calling this script are shown below. This includes three different scenarios:

• A call without any further command line arguments
• A call with two arguments
• A call with two arguments, where the second one is a quoted string containing a space

$ python arguments-count.py
The script is called with 0 arguments
$ python arguments-count.py --help me
The script is called with 2 arguments
$ python arguments-count.py --option "long string"
The script is called with 2 arguments

Iterating Through Arguments

Our third example outputs every single argument sent to the Python script, except the program name itself. Therefore, we loop through the command line arguments starting with the second list element. Recall that this is index 1 since lists are 0-based in Python:

import sys

# Count the arguments
arguments = len(sys.argv) - 1

# Output argument-wise
position = 1
while (arguments >= position):
    print ("Parameter %i: %s" % (position, sys.argv[position]))
    position = position + 1

Below we call our code, which was saved to the file arguments-output.py. As done with our previous example, the output illustrates three different calls:

• A call without any arguments
• A call with two arguments
• A call with two arguments, where the second argument is a quoted string containing a space

$ python arguments-output.py
$ python arguments-output.py --help me
Parameter 1: --help
Parameter 2: me
$ python arguments-output.py --option "long string"
Parameter 1: --option
Parameter 2: long string

Remember, the point of showing the quoted string example is that parameters are usually delimited by a space, unless they are surrounded by quotes.

Abseil Flags (absl)

Abseil's Flags library is meant to bring command line arguments to production, with distributed command line arguments. When a module uses command-line flags, and is imported into another module - the other module imports the flags as well, and can process them by forwarding them to the imported module.

This makes complex command-line arguments shared between modules easier and less verbose.

Additionally, the library lets you define the default values, descriptions of, and data type of the arguments, so additional checks and conversions aren't necessary.

from absl import flags
import sys

# Flag name, default value, help message.
flags.DEFINE_string('name', 'User', 'The name of the user.')

# Read sys.argv into FLAGS
FLAGS = flags.FLAGS
FLAGS(sys.argv)

print(f"Hello {FLAGS.name}!")

The supported data types are:

• DEFINE_integer()
• DEFINE_string()
• DEFINE_bool()
• DEFINE_enum()
• DEFINE_list()
• DEFINE_float()

As well as DEFINE_multi_integer(), DEFINE_multi_string() and DEFINE_multi_enum() for multi-argument input. Additionally, running --help, --helpfull, etc. print the existing flags and their descriptions, in different formats.

The library also allows you to define validations - both in terms of range, such as integer-based values having an upper_bound or lower_bound that's acceptable, and running arbitrary methods to check for values:

def validate_name(value):
    return len(value) > 15

flags.register_validator('name',
                         validate_name,
                         message='Name is over 15 characters long.',
                         flag_values=FLAGS)

Collecting these into a concrete example:

from absl import flags
import sys

flags.DEFINE_string('name', 'User', 'The name of the user.')
flags.DEFINE_integer('tasks', 0, 'The number of tasks a user has.', lower_bound=0)

FLAGS = flags.FLAGS
FLAGS(sys.argv)

print(f"{FLAGS.name} has {FLAGS.tasks} tasks to work on.")

$ python flags.py --name=John --tasks=5
John has 5 tasks to work on.

$ python flags.py --name=John --tasks=-1

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/absl/flags/_flag.py", line 180, in _parse
    return self.parser.parse(argument)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/absl/flags/_argument_parser.py", line 168, in parse
    raise ValueError('%s is not %s' % (val, self.syntactic_help))
ValueError: -1 is not a non-negative integer
...

The argparse Module

The argparse module has been available since Python 3.2, and an enhancement of the optparse module that exists up to Python 2.7. The Python documentation contains an API description and a tutorial that covers all the methods in detail.

The module offers a command line interface with a standardized output, whereas the former two solutions leave much of the work in your hands. argparse allows the verification of fixed and optional arguments, with name checking as either short or long style. As a default optional argument, it includes -h, along with its long version --help. This argument is accompanied by a default help message describing the accepted arguments.

The code below shows the parser initialization, and the output below shows the basic call, followed by the help message. In contrast to the Python calls we used in the previous examples, keep in mind to use Python 3 with these examples:

# Include standard modules
import argparse

# Initiate the parser
parser = argparse.ArgumentParser()
parser.parse_args()

$ python3 arguments-argparse-basic.py 
$ python3 arguments-argparse-basic.py -h
usage: arguments-argparse-basic.py [-h]

optional arguments:
  -h, --help  show this help message and exit
$ python3 arguments-argparse-basic.py --verbose
usage: arguments-argparse-basic.py [-h]
arguments-argparse-basic.py: error: unrecognized arguments: --verbose

In the next step, we will add a custom description to the help message for our users. Initializing the parser in this way allows an additional text. The code below stores the description in the text variable, which is explicitly given to the argparse class as the description parameter. Calling this code below, you can see what the output looks like:

# Include standard modules
import argparse

# Define the program description
text = 'This is a test program. It demonstrates how to use the argparse module with a program description.'

# Initiate the parser with a description
parser = argparse.ArgumentParser(description=text)
parser.parse_args()

$ python3 arguments-argparse-description.py --help
usage: arguments-argparse-description.py [-h]

This is a test program. It demonstrates how to use the argparse module with a
program description.

optional arguments:
  -h, --help  show this help message and exit

As the final step we will add an optional argument named -V, which has a corresponding long style argument named --version. To do so we use the method add_argument() that we call with three parameters (displayed for --version, only):

• The name of the parameter: --version
• The help text for the parameter: help="show program version"
• Action (without additional value): action="store_true"

The source code for that is displayed below. Reading the arguments into the variable called args is done via the parse_args() method from the parser object. Note that you submit both the short and the long version in one call. Finally, you check if the attributes args.V or args.version are set and output the version message:

# Include standard modules
import argparse

# Initiate the parser
parser = argparse.ArgumentParser()
parser.add_argument("-V", "--version", help="show program version", action="store_true")

# Read arguments from the command line
args = parser.parse_args()

# Check for --version or -V
if args.version:
    print("This is myprogram version 0.1")

$ python3 arguments-argparse-optional.py -V
This is myprogram version 0.1
$ python3 arguments-argparse-optional.py --version
This is myprogram version 0.1

The --version argument does not require a value to be given on the command line. That's why we set the action argument to "store_true". In other cases you might need an additional assigned value, for example if you specify a certain volume, height, or width. This is shown in the next example. As a default case, please note that all the arguments are interpreted as strings:

# Include standard modules
import argparse

# Initiate the parser
parser = argparse.ArgumentParser()

# Add long and short argument
parser.add_argument("--width", "-w", help="set output width")

# Read arguments from the command line
args = parser.parse_args()

# Check for --width
if args.width:
    print("Set output width to %s" % args.width)

Here we show what happens when submitting different argument values. This includes both the short and the long version, as well as the help message:

$ python3 arguments-argparse-optional2.py -w 10
Set output width to 10
$ python3 arguments-argparse-optional2.py --width 10
Set output width to 10
$ python3 arguments-argparse-optional2.py -h
usage: arguments-argparse-optional2.py [-h] [--width WIDTH]

optional arguments:
  -h, --help            show this help message and exit
  --width WIDTH, -w WIDTH
                        set output width

The getopt Module

As you may have noticed before, the sys module splits the command line string into single facets only. The Python getopt module goes a bit further and extends the separation of the input string by parameter validation. Based on the getopt C function, it allows both short and long options, including a value assignment.

In practice, it requires the sys module to process input data properly. To do so, both the sys module and the getopt module have to be loaded beforehand. Next, from the list of input parameters we remove the first list element (see the code below), and store the remaining list of command line arguments in the variable called argument_list:

# Include standard modules
import getopt, sys

# Get full command-line arguments
full_cmd_arguments = sys.argv

# Keep all but the first
argument_list = full_cmd_arguments[1:]

print argument_list

The arguments in argument_list can now be parsed using the getopts() method. But before doing that, we need to tell getopts() about which parameters are valid. They are defined like this:

short_options = "ho:v"
long_options = ["help", "output=", "verbose"]

This means that these arguments are ones we consider to be valid, along with some extra info:

------------------------------------------
long argument   short argument  with value
------------------------------------------
--help           -h              no
--output         -o              yes
--verbose        -v              no
------------------------------------------

You might have noticed that the o short option was preceded by a colon, :. This tells getopt that this option should be assigned a value.

This now allows us to process a list of arguments. The getopt() method requires three parameters to be configured - the list of actual arguments from argv, as well as both the valid short and long options (shown in the previous code snippet).

The method call itself is kept in a try-catch-statement to cover errors during the evaluation. An exception is raised if an argument is discovered that is not part of the list as defined before. The Python script will print the error message to the screen, and exit with error code 2:

try:
    arguments, values = getopt.getopt(argument_list, short_options, long_options)
except getopt.error as err:
    # Output error, and return with an error code
    print (str(err))
    sys.exit(2)

Finally, the arguments with the corresponding values are stored in the two variables named arguments and values. Now, you can easily evaluate these variables in your code. We can use a for-loop to iterate through the list of recognized arguments, one entry after the next.

# Evaluate given options
for current_argument, current_value in arguments:
    if current_argument in ("-v", "--verbose"):
        print ("Enabling verbose mode")
    elif current_argument in ("-h", "--help"):
        print ("Displaying help")
    elif current_argument in ("-o", "--output"):
        print (("Enabling special output mode (%s)") % (current_value))

Below you can see the output from executing this code. We'll show how the program reacts with both valid and invalid program arguments:

$ python arguments-getopt.py -h
Displaying help
$ python arguments-getopt.py --help
Displaying help
$ python arguments-getopt.py --output=green --help -v
Enabling special output mode (green)
Displaying help
Enabling verbose mode
$ python arguments-getopt.py -verbose
option -e not recognized

The last call to our program may seem a bit confusing at first. To understand it, you need to know that the shorthand options (sometimes also called flags) can be used together with a single dash. This allows your tool to more easily accept many options. For example, calling python arguments-getopt.py -vh is the same as calling python arguments-getopt.py -v -h. So in the last call above, the getopt module thought the user was trying to pass -e as an option, which is invalid.

Conclusion

In this article we showed many different methods for retrieving command line arguments in Python, including using sys, getopt, and argparse. These modules vary in functionality, some providing much more than others. sys is fully flexible, whereas both getoptand argparse require some structure. In contrast, they cover most of the complex work that sys leaves up to you. After working through the examples provided, you should be able to determine which module suits your project best.

In this article we did not talk about other solutions like the docopts module, we just mentioned it. This module follows a totally different approach, and will be explained in detail in one of the next articles.

References

• sys module
• getopt module
• Abseil flags module
• argparse module
• argparse tutorial