With Python being such a popular programming language, as well as having support for most operating systems, it's become widely used to create command line tools for many purposes. These tools can range from simple CLI apps to those that are more complex, like AWS' awscli tool.
Complex tools like this are typically controlled by the user via command line arguments, which allows the user to use specific commands, set options, and more. For example, these options could tell the tool to output additional information, read data from a specified source, or send output to a certain location.
In general, arguments are passed to CLI tools differently, depending on your operating system:
-followed by a letter, like
--followed by a word, like
/followed by either a letter, or word, like
These different approaches exist due to historical reasons. Many programs on Unix-like systems support both the single and double dash notation. The single dash notation is mostly used with single letter options, while double dashes present a more readable options list, which is particularly useful for complex options that need to be more explicit.
Note: In this article we'll solely be focusing on the Unix-like format of
Keep in mind that both the name and the meaning of an argument are specific to a program - there is no general definition, other than a few common conventions like
--help for further information on the usage of the tool. As the developer of a Python script, you will decide which arguments to provide to the caller and what they do. This requires proper evaluation.
As your list of available arguments grows, your code will become more complex in trying to accurately parse them. Luckily, in Python there are a number of libraries available to help you with this. We'll cover a few of the most common solutions, which range from "do-it-yourself" with
sys.argv, to the "done-for-you" approach with
Handling Command Line Arguments with Python
Python 3 supports a number of different ways of handling command line arguments. The built-in way is to use the
sys module. In terms of names, and its usage, it relates directly to the C library (
libc). The second way is the
getopt module, which handles both short and long options, including the evaluation of the parameter values.
Furthermore, two other common methods exist. This is the argparse module, which is derived from the
optparse module available up to Python 2.7. The other method is using the
docopt module, which is available on GitHub.
Each of these ways has their pros and cons, so it's worth evaluating each to see which suits your needs best.
The sys Module
This is a basic module that has been shipped with Python from the early days. It takes a very similar approach to the C library using
argv to access the arguments. The sys module implements the command line arguments in a simple list structure named
Each list element represents a single argument. The first item in the list,
sys.argv, is the name of the Python script. The rest of the list elements,
sys.argv[n], are the command line arguments 2 through n. As a delimiter between the arguments, a space is used. Argument values that contain a space in it have to be surrounded by quotes in order to be properly parsed by
The equivalent of
argc is just the number of elements in the list. To obtain this value, use the Python
len() operator. We'll show this in a code example later on.
Printing the First CLI Argument
In this first example, our script will determine the way it was called. This information is kept in the first command line argument, indexed with 0. The code below shows how you obtain the name of your Python script.
import sys print ("The script has the name %s" % (sys.argv)
Save this code in a file named arguments-program-name.py, and then call it as shown below. The output is as follows and contains the file name, including its full path:
$ python arguments-program-name.py The script has the name arguments-program-name.py $ python /home/user/arguments-program-name.py The script has the name /home/user/arguments-program-name.py
As you can see from the second call above, we not only get the name of the Python file, but also the full path used to call it.
Counting the Number of Arguments
In this second example we simply count the number of command line arguments using the built-in
sys.argv is the list that we have to examine. In the code below, we get the number of arguments and then subtract 1 because one of those arguments (i.e. the first one) is always set as the name of the file, which isn't always useful to us. Thus, the actual number of arguments passed by the user is
len(sys.argv) - 1.
import sys # Count the arguments arguments = len(sys.argv) - 1 print ("The script is called with %i arguments" % (arguments))
Save and name this file arguments-count.py. Some examples of calling this script is shown below. This includes three different scenarios:
- A call without any further command line arguments
- A call with two arguments
- A call with two arguments, where the second one is a quoted string containing a space
$ python arguments-count.py The script is called with 0 arguments $ python arguments-count.py --help me The script is called with 2 arguments $ python arguments-count.py --option "long string" The script is called with 2 arguments
Iterating Through Arguments
Our third example outputs every single argument sent to the Python script, except the program name itself. Therefore, we loop through the command line arguments starting with the second list element. Recall that this is index 1 since lists are 0-based in Python.
import sys # Count the arguments arguments = len(sys.argv) - 1 # Output argument-wise position = 1 while (arguments >= position): print ("Parameter %i: %s" % (position, sys.argv[position])) position = position + 1
Below we call our code, which was saved to the file arguments-output.py. As done with our previous example, the output illustrates three different calls:
- A call without any arguments
- A call with two arguments
- A call with two arguments, where the second argument is a quoted string containing a space
$ python arguments-output.py $ python arguments-output.py --help me Parameter 1: --help Parameter 2: me $ python arguments-output.py --option "long string" Parameter 1: --option Parameter 2: long string
Remember, the point of showing the quoted string example is that parameters are usually delimited by a space, unless they are surrounded by quotes.
The getopt Module
As you may have noticed before, the
sys module splits the command line string into single facets only. The Python getopt module goes a bit further and extends the separation of the input string by parameter validation. Based on the
getopt C function, it allows both short and long options, including a value assignment.
In practice, it requires the
sys module to process input data properly. To do so, both the
sys module and the
getopt module have to be loaded beforehand. Next, from the list of input parameters we remove the first list element (see the code below), and store the remaining list of command line arguments in the variable called
# Include standard modules import getopt, sys # Get full command-line arguments full_cmd_arguments = sys.argv # Keep all but the first argument_list = full_cmd_arguments[1:] print argument_list
The arguments in
argument_list can now be parsed using the
getopts() method. But before doing that, we need to tell
getopts() about which parameters are valid. They are defined like this:
short_options = "ho:v" long_options = ["help", "output=", "verbose"]
This means that these arguments are ones we consider to be valid, along with some extra info:
------------------------------------------ long argument short argument with value ------------------------------------------ --help -h no --output -o yes --verbose -v no ------------------------------------------
You might have noticed that the
o short option was proceeded by a colon,
:. This tells
getopt that this option should be assigned a value.
This now allows us to process a list of arguments. The
getopt() method requires three parameters to be configured - the list of actual arguments from
argv, as well as both the valid short and long options (shown in the previous code snippet).
The method call itself is kept in a try-catch-statement to cover errors during the evaluation. An exception is raised if an argument is discovered that is not part of the list as defined before. The Python script will print the error message to the screen, and exit with error code 2.
try: arguments, values = getopt.getopt(argument_list, short_options, long_options) except getopt.error as err: # Output error, and return with an error code print (str(err)) sys.exit(2)
Finally, the arguments with the corresponding values are stored in the two variables named
values. Now, you can easily evaluate these variables in your code. We can use a
for-loop to iterate through the list of recognized arguments, one entry after the next.
# Evaluate given options for current_argument, current_value in arguments: if current_argument in ("-v", "--verbose"): print ("Enabling verbose mode") elif current_argument in ("-h", "--help"): print ("Displaying help") elif current_argument in ("-o", "--output"): print (("Enabling special output mode (%s)") % (current_value))
Below you can see the output from executing this code. We'll show how the program reacts with both valid and invalid program arguments:
$ python arguments-getopt.py -h Displaying help $ python arguments-getopt.py --help Displaying help $ python arguments-getopt.py --output=green --help -v Enabling special output mode (green) Displaying help Enabling verbose mode $ python arguments-getopt.py -verbose option -e not recognized
The last call to our program may seem a bit confusing at first. To understand it, you need to know that the shorthand options (sometimes also called flags) can be used together with a single dash. This allows your tool to more easily accept many options. For example, calling
python arguments-getopt.py -vh is the same as calling
python arguments-getopt.py -v -h. So in the last call above, the
getopt module thought the user was trying to pass
-e as an option, which is invalid.
The argparse Module
The argparse module has been available since Python 3.2, and an enhancement of the
optparse module that exists up to Python 2.7. The Python documentation contains an API description and a tutorial that covers all the methods in detail.
The module offers a command line interface with a standardized output, whereas the former two solutions leave much of the work in your hands.
argparse allows the verification of fixed and optional arguments, with name checking as either short or long style. As a default optional argument, it includes
-h, along with its long version
--help. This argument is accompanied by a default help message describing the accepted arguments.
The code below shows the parser initialization, and the output below that shows the basic call, followed by the help message. In contrast to the Python calls we used in the previous examples, keep in mind to use Python 3 with these examples.
# Include standard modules import argparse # Initiate the parser parser = argparse.ArgumentParser() parser.parse_args()
$ python3 arguments-argparse-basic.py $ python3 arguments-argparse-basic.py -h usage: arguments-argparse-basic.py [-h] optional arguments: -h, --help show this help message and exit $ python3 arguments-argparse-basic.py --verbose usage: arguments-argparse-basic.py [-h] arguments-argparse-basic.py: error: unrecognized arguments: --verbose
In the next step, we will add a custom description to the help message for our users. Initializing the parser in this way allows an additional text. The code below stores the description in the
text variable, which is explicitly given to the
argparse class as the
description parameter. Calling this code below, you can see what the output looks like.
# Include standard modules import argparse # Define the program description text = 'This is a test program. It demonstrates how to use the argparse module with a program description.' # Initiate the parser with a description parser = argparse.ArgumentParser(description=text) parser.parse_args()
$ python3 arguments-argparse-description.py --help usage: arguments-argparse-description.py [-h] This is a test program. It demonstrates how to use the argparse module with a program description. optional arguments: -h, --help show this help message and exit
As the final step we will add an optional argument named
-V, which has a corresponding long style argument named
--version. To do so we use the method
add_argument() that we call with three parameters (displayed for
- The name of the parameter:
- The help text for the parameter:
help="show program version"
- Action (without additional value):
The source code for that is displayed in below. Reading the arguments into the variable called
args is done via the
parse_args() method from the
parser object. Note that you submit both the short and the long version in one call. Finally, you check if the attributes
args.version are set and output the version message.
# Include standard modules import argparse # Initiate the parser parser = argparse.ArgumentParser() parser.add_argument("-V", "--version", help="show program version", action="store_true") # Read arguments from the command line args = parser.parse_args() # Check for --version or -V if args.version: print("This is myprogram version 0.1")
$ python3 arguments-argparse-optional.py -V This is myprogram version 0.1 $ python3 arguments-argparse-optional.py --version This is myprogram version 0.1
--version argument does not require a value to be given on the command line. That's why we set the action argument to
"store_true". In other cases you might need an additional assigned value, for example if you specify a certain volume, height, or width. This is shown in the next example. As a default case, please note that all the arguments are interpreted as strings.
# Include standard modules import argparse # Initiate the parser parser = argparse.ArgumentParser() # Add long and short argument parser.add_argument("--width", "-w", help="set output width") # Read arguments from the command line args = parser.parse_args() # Check for --width if args.width: print("Set output width to %s" % args.width)
Here we show what happens when submitting different argument values. This includes both the short and the long version, as well as the help message.
$ python3 arguments-argparse-optional2.py -w 10 Set output width to 10 $ python3 arguments-argparse-optional2.py --width 10 Set output width to 10 $ python3 arguments-argparse-optional2.py -h usage: arguments-argparse-optional2.py [-h] [--width WIDTH] optional arguments: -h, --help show this help message and exit --width WIDTH, -w WIDTH set output width
In this article we showed many different methods for retrieving command line arguments in Python, including using
argparse. These modules vary in functionality, some providing much more than others.
sys is fully flexible, whereas both
argparse require some structure. In contrast, they cover most of the complex work that
sys leaves up to you. After working through the examples provided, you should be able to determine which module suits your project best.
In this article we did not talk about other solutions like the
docopts module, we just mentioned it. This module follows a totally different approach, and will be explained in detail in one of the next articles.