More Command-Line Goodness

In Command-Line Applications, we looked at a Python main-import switch which boiled down to this.


for file in args.file: 
    with open( file, "r" ) as source:
        process_file( source, args )

The point was that each distinct file on the command-line was processed in a more-or-less uniform way by a single function that does the "real work" for that input file.

It turns out that we often have flat files which are spreadsheets or spreadsheet-like.   Indeed, for some people (and some organizations) the spreadsheet is their preferred user interface.  As I've said before, 

Spreadsheets are the universal user interface. Everyone likes them, they're almost inescapable. And they work. There's no reason to attempt to replace the spreadsheet with a web page or a form or a desktop application. It's easier to cope with spreadsheet vagaries than to replace them.

They have problems, but they are surprisingly common.  

Enter Stingray Reader.  This is a small Python library to make it easy to have programs which read workbooks--collections of spreadsheets--or spreadsheet-like files with a degree of transparency.  

And.  It allows a clean command-line interface.

With a little care, we can reduce the main-import switch to something like this.

if __name__ == "__main__":

    logging.basicConfig( stream=sys.stderr )

    args= parse_args()

    logging.getLogger().setLevel( args.verbosity )

    builder= make_builder( args )

    try:

        for file in args:

            with workbook.open_workbook( input ) as source:

                process_workbook( source, builder )

        status= 0

    except Exception as e:

        logging.exception( e )

        status= 3

    logging.shutdown()

    sys.exit( status )

The bold lines are specific to workbook ("spreadsheet") processing.  A "builder" creates application-specific Python objects from spreadsheet rows.  The "workbook.open_workbook" is a function that builds a workbook reader based on the file name.  It can handle a number of file types.  

The process_workbook function is the "real work" function that handles a workbook of individual spreadsheets (or a spreadsheet-like file).

Command Line Applications

I'm old -- I admit it -- and I feel that command-line applications are still very, very important. Linux, for example, is packed full of almost innumerable command-line applications. In some cases, the Linux GUI tools are specifically just wrappers around the underlying command-line applications.

For many types of high-volume data processing, command-line applications are essential.

I've seen command-line applications done very badly.

Overusing Main

When writing OO programs, it's absolutely essential that the OS interface (public static void main in Java or the if __name__ == "__main__": block in Python) does as little as possible.

A good command-line program has the underlying tasks or actions defined in some easy-to-work with class hierarchy built on the Command design pattern. The actual main program part does just a few things: gather the relevant environment variables, parse command-line options and arguments, identify the configuration files, and initiate the appropriate commands. Nothing application-specific.

When the main method does application-specific work, that application functionality is buried in a method that's particularly hard to reuse. It's important to keep the application functionality away from the OS interface.

I'm finding that main programs should look something like this:


if __name__ == "__main__":
    logging.basicConfig( stream=sys.stderr )
    args= parse_args()
    logging.getLogger().setLevel( args.verbosity )
    try:
        for file in args.file:
            with open( file, "r" ) as source:
                process_file( source, args )
        status= 0
    except Exception as e:
        logging.exception( e )
        status= 3
    logging.shutdown()
    sys.exit( status )

That's it.  Nothing more in the top-level main program.  The process_file function becomes a reusable "command" and something that can be tested independently.