8. Add-on packages for AutoGen

This chapter includes several programs that either work closely with AutoGen (extracting definitions or providing special formatting functions), or else it is mkmerge. I want to promote the latter as an alternative to the builddir/srcdir schizophrenia. I hate it. :(

AutoOpts ought to appear in this list also, but since it is the primary reason why many people would even look into AutoGen at all, I decided to leave it in the list of chapters.

8.1 Automated Finite State Machine
8.2 Combined RPC Marshalling
8.3 Automated Event Management
8.4 Invoking columns
8.5 Invoking getdefs
8.6 Replacement for Stdio Formatting Library		The extensible format printing library.

8.1 Automated Finite State Machine

The templates to generate a finite state machine in C or C++ is included with AutoGen. The documentation is not. The documentation is in HTML format for viewing, or you can download FSM.

8.2 Combined RPC Marshalling

The templates and NFSv4 definitions are not included with AutoGen in any way. The folks that designed NFSv4 noticed that much time and bandwidth was wasted sending queries and responses when many of them could be bundled. The protocol bundles the data, but there is no support for it in rpcgen. That means you have to write your own code to do that. Until now. Download this and you will have a large, complex example of how to use AutoXDR for generating the marshalling and unmarshalling of combined RPC calls. There is a brief example on the web, but you should download AutoXDR.

8.3 Automated Event Management

Large software development projects invariably have a need to manage the distribution and display of state information and state changes. In other words, they need to manage their software events. Generally, each such project invents its own way of accomplishing this and then struggles to get all of its components to play the same way. It is a difficult process and not always completely successful. This project helps with that.

AutoEvents completely separates the tasks of supplying the data needed for a particular event from the methods used to manage the distribution and display of that event. Consequently, the programmer writing the code no longer has to worry about that part of the problem. Likewise the persons responsible for designing the event management and distribution no longer have to worry about getting programmers to write conforming code.

This is a work in progress. See my web page on the subject, if you are interested. I have some useful things put together, but it is not ready to call a product.

8.4 Invoking columns

This program was designed for the purpose of generating compact, columnized tables. It will read a list of text items from standard in or a specified input file and produce a columnized listing of all the non-blank lines. Leading white space on each line is preserved, but trailing white space is stripped. Methods of applying per-entry and per-line embellishments are provided. See the formatting and separation arguments below.

This program is used by AutoGen to help clean up and organize its output.

See `autogen/agen5/fsm.tpl' and the generated output `pseudo-fsm.h'.

This function was not implemented as an expression function because either it would have to be many expression functions, or a provision would have to be added to provide options to expression functions. Maybe not a bad idea, but it is not being implemented at the moment.

A side benefit is that you can use it outside of AutoGen to columnize input, a la the ls command.

This section was generated by AutoGen, the aginfo template and the option descriptions for the columns program. It documents the columns usage text and option meanings.

This software is released under the GNU General Public License.

8.4.1 columns usage help (-?)
8.4.13 by-columns option
8.4.4 col-width option (-w)
8.4.3 columns option (-c)
8.4.7 first-indent option
8.4.10 format option (-f)
8.4.6 indent option (-I)
8.4.14 input option (-i)
8.4.12 line-separation option
8.4.11 separation option (-S)
8.4.9 sort option (-s)
8.4.5 spread option
8.4.8 tab-width option
8.4.2 width option (-W)

8.4.1 columns usage help (-?)

This is the automatically generated usage text for columns:

@exampleindent 0

columns - Columnize Input Text - Ver. 1.1 USAGE: columns [ -<flag> [<val>] | --<name>[{=| }<val>] ]... Flg Arg Option-Name Description -W Num width Maximum Line Width -c Num columns Desired number of columns -w Num col-width Set width of each column Num spread maximum spread added to column width -I YES indent Line prefix or indentation YES first-indent First line prefix - requires these options: indent Num tab-width tab width -s opt sort Sort input text -f YES format Formatting string for each input -S YES separation Separation string - follows all but last YES line-separation string at end of all lines but last no by-columns Print entries in column order -i YES input Input file (if not stdin) -v opt version Output version information and exit -? no help Display usage information and exit -! no more-help Extended usage information passed thru pager Options may be specified by doubled hyphens and their name or by a single hyphen and the flag character (option value). This program was designed for the purpose of generating compact, columnized tables. It will read a list of text items from standard in or a specified input file and produce a columnized listing of all the non-blank lines. Leading white space on each line is preserved, but trailing white space is stripped. Methods of applying per-entry and per-line embellishments are provided. See the formatting and separation arguments below. This program is used by AutoGen to help clean up and organize its output.

@exampleindent 4

8.4.2 width option (-W)

This is the "maximum line width" option. This option specifies the full width of the output line, including any start-of-line indentation. The output will fill each line as completely as possible, unless the column width has been explicitly specified. If the maximum width is less than the length of the widest input, you will get a single column of output.

8.4.3 columns option (-c)

This is the "desired number of columns" option. Use this option to specify exactly how many columns to produce. If that many columns will not fit within line_width, then the count will be reduced to the number that fit.

8.4.4 col-width option (-w)

This is the "set width of each column" option. Use this option to specify exactly how many characters are to be allocated for each column. If it is narrower than the widest entry, it will be over-ridden with the required width.

8.4.5 spread option

This is the "maximum spread added to column width" option. Use this option to specify exactly how many characters may be added to each column. It allows you to prevent columns from becoming too far apart.

8.4.6 indent option (-I)

This is the "line prefix or indentation" option. If a number, then this many spaces will be inserted at the start of every line. Otherwise, it is a line prefix that will be inserted at the start of every line.

8.4.7 first-indent option

This is the "first line prefix" option.

This option has some usage constraints. It:

• must appear in combination with the following options: indent.

If a number, then this many spaces will be inserted at the start of the first line. Otherwise, it is a line prefix that will be inserted at the start of that line.

8.4.8 tab-width option

This is the "tab width" option. If an indentation string contains tabs, then this value is used to compute the ending column of the prefix string.

8.4.9 sort option (-s)

This is the "sort input text" option. Causes the input text to be sorted. If an argument is supplied, it is presumed to be a pattern and the sort is based upon the matched text. If the pattern starts with or consists of an asterisk (*), then the sort is case insensitive.

8.4.10 format option (-f)

This is the "formatting string for each input" option. If you need to reformat each input text, the argument to this option is interpreted as an sprintf(3) format that is used to produce each output entry.

8.4.11 separation option (-S)

This is the "separation string - follows all but last" option. Use this option if, for example, you wish a comma to appear after each entry except the last.

8.4.12 line-separation option

This is the "string at end of all lines but last" option. Use this option if, for example, you wish a backslash to appear at the end of every line, except the last.

8.4.13 by-columns option

This is the "print entries in column order" option. Normally, the entries are printed out in order by rows and then columns. This option will cause the entries to be ordered within columns. The final column, instead of the final row, may be shorter than the others.

8.4.14 input option (-i)

This is the "input file (if not stdin)" option. This program normally runs as a filter, reading from standard input, columnizing and writing to standard out. This option redirects input to a file.

8.5 Invoking getdefs

This program extracts AutoGen definitions from a list of source files. Definitions are delimited by `/*=<entry-type> <entry-name>\n' and `=*/\n'. From that, this program creates a definition of the following form:

#line nnn "source-file-name" entry_type = { name = entry_name; ... };

The ellipsis `...' is filled in by text found between the two delimiters, using the following rules:

1. Each entry is located by the pattern "\n[^*\n]*\\*[ \t]*([a-z][a-z0-9_]*):". Fundamentally, it finds a line that, after the first asterisk on the line, contains whitespace then a name and is immediately followed by a colon. The name becomes the name of the attribute and what follows, up to the next attribute, is its value.
2. If the first character of the value is either a single or double quote, then you are responsible for quoting the text as it gets inserted into the output definitions.
3. All the leading text on a line is stripped from the value. The leading text is everything before the first asterisk, the asterisk and all the whitespace characters that immediately follow it. If you want whitespace at the beginnings of the lines of text, you must do something like this:

   * mumble: * " this is some\n" * " indented text."

4. If the `<entry-name>' is followed by a comma, the word `ifdef' (or 'ifndef') and a name `if_name', then the above entry will appear as:

   #ifdef if_name #line nnn "source-file-name" entry_type = { name = entry_name; ... }; #endif

5. If you use of the subblock option, you can specify a nested value, See section 8.5.6 subblock option. That is, this text:

   * arg: int, this, what-it-is

   with the `--subblock=arg=type,name,doc' option would yield:

   arg = { type = int; name = this; doc = what-it-is; };

This section was generated by AutoGen, the aginfo template and the option descriptions for the getdefs program. It documents the getdefs usage text and option meanings.

This software is released under the GNU General Public License.

8.5.1 getdefs usage help
8.5.17 agarg option
8.5.9 assign option
8.5.15 autogen option
8.5.18 base-name option
8.5.10 common-assign option
8.5.11 copy option
8.5.2 defs-to-get option
8.5.8 filelist option
8.5.4 first-index option
8.5.5 input option
8.5.13 linenum option
8.5.7 listattr option
8.5.3 ordering option
8.5.14 output option
8.5.12 srcfile option
8.5.6 subblock option
8.5.16 template option

8.5.1 getdefs usage help

This is the automatically generated usage text for getdefs:

@exampleindent 0

getdefs - AutoGen Definition Extraction Tool - Ver. 1.3 USAGE: getdefs { <option-name>[{=| }<val>] }... Arg Option-Name Req? Description YES defs-to-get opt Regexp to look for after the "/*=" opt ordering opt Alphabetize or use named file - disabled as --no-ordering - enabled by default Num first-index opt The first index to apply to groups YES input YES Input file to search for defs - may appear multiple times - default option for unnamed options YES subblock opt subblock definition names - may appear multiple times YES listattr opt attribute with list of values - may appear multiple times opt filelist opt Insert source file names into defs Definition insertion options Arg Option-Name Req? Description YES assign opt Global assignments - may appear multiple times YES common-assign opt Assignments common to all blocks - may appear multiple times YES copy opt File(s) to copy into definitions - may appear multiple times opt srcfile opt Insert source file name into each def opt linenum opt Insert source line number into each def Definition output disposition options: Arg Option-Name Req? Description YES output opt Output file to open - an alternate for autogen opt autogen opt Invoke AutoGen with defs - disabled as --no-autogen - enabled by default YES template opt Template Name YES agarg opt AutoGen Argument - prohibits these options: output - may appear multiple times YES base-name opt Base name for output file(s) - prohibits these options: output Auto-supported Options: Arg Option-Name Req? Description opt version opt Output version information and exit no help opt Display usage information and exit no more-help opt Extended usage information passed thru pager opt save-opts opt Save the option state to an rc file YES load-opts opt Load options from an rc file - disabled as --no-load-opts - may appear multiple times All arguments are named options. The following option preset mechanisms are supported: - reading file /dev/null/.getdefsrc This program extracts AutoGen definitions from a list of source files. Definitions are delimited by `/*=<entry-type> <entry-name>\n' and `=*/\n'. From that, this program creates a definition of the following form: #line nnn "source-file-name" entry_type = { name = entry_name; ... }; The ellipsis '...' is filled in by text found between the two delimiters, with everything up through the first sequence of asterisks deleted on every line. Also, if ``<entry-type>'' is an asterisk (``*''), then the entry_type enclosure and the name entry will be omitted and the ellipsis will become top-level definitions.

@exampleindent 4

8.5.2 defs-to-get option

This is the "regexp to look for after the "/*="" option. If you want definitions only from a particular category, or even with names matching particular patterns, then specify this regular expression for the text that must follow the /*=.

8.5.3 ordering option

This is the "alphabetize or use named file" option.

This option has some usage constraints. It:

• is enabled by default.

By default, ordering is alphabetical by the entry name. Use, no-ordering if order is unimportant. Use ordering with no argument to order without case sensitivity. Use ordering=<file-name> if chronological order is important. getdefs will maintain the text content of file-name. file-name need not exist.

8.5.4 first-index option

This is the "the first index to apply to groups" option. By default, the first occurrance of a named definition will have an index of zero. Sometimes, that needs to be a reserved value. Provide this option to specify a different starting point.

8.5.5 input option

This is the "input file to search for defs" option.

This option has some usage constraints. It:

• is required to appear on the command line.
• may appear an unlimited number of times.

All files that are to be searched for definitions must be named on the command line or read from stdin. If there is only one input option and it is the string, "-", then the input file list is read from stdin. If a command line argument is not an option name and does not contain an assignment operator (=), then it defaults to being an input file name. At least one input file must be specified.

8.5.6 subblock option

This is the "subblock definition names" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for nested definitions. For example, with:
subblock=arg=argname,type,null defined, the text:
arg: this, char * will then expand to:
arg = { argname = this; type = "char *"; };
The "this, char *" string is separated at the commas, with the white space removed. You may use characters other than commas by starting the value string with a punctuation character other than a single or double quote character. You may also omit intermediate values by placing the commas next to each other with no intervening white space. For example, "+mumble++yes+" will expand to:
arg = { argname = mumble; null = "yes"; };.

8.5.7 listattr option

This is the "attribute with list of values" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for definitions that generally appear several times. That is, they tend to be a list of values. For example, with:
listattr=foo defined, the text:
foo: this, is, a, multi-list will then expand to:
foo = 'this', 'is', 'a', 'multi-list';
The texts are separated by the commas, with the white space removed. You may use characters other than commas by starting the value string with a punctuation character other than a single or double quote character.

8.5.8 filelist option

This is the "insert source file names into defs" option. Inserts the name of each input file into the output definitions. If no argument is supplied, the format will be:

infile = '%s';

If an argument is supplied, that string will be used for the entry name instead of infile.

8.5.9 assign option

This is the "global assignments" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into the output definitions, with only a semicolon attached.

8.5.10 common-assign option

This is the "assignments common to all blocks" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into each output definition, with only a semicolon attached.

8.5.11 copy option

This is the "file(s) to copy into definitions" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The content of each file named by these options will be inserted into the output definitions.

8.5.12 srcfile option

This is the "insert source file name into each def" option. Inserts the name of the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

srcfile = '%s';

If an argument is supplied, that string will be used for the entry name instead of srcfile.

8.5.13 linenum option

This is the "insert source line number into each def" option. Inserts the line number in the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

linenum = '%s';

If an argument is supplied, that string will be used for the entry name instead of linenum.

8.5.14 output option

This is the "output file to open" option.

This option has some usage constraints. It:

• is a member of the autogen class of options.

If you are not sending the output to an AutoGen process, you may name an output file instead.

8.5.15 autogen option

This is the "invoke autogen with defs" option.

This option has some usage constraints. It:

• is enabled by default.
• is a member of the autogen class of options.

This is the default output mode. Specifying no-autogen is equivalent to output=-. If you supply an argument to this option, that program will be started as if it were AutoGen and its standard in will be set to the output definitions of this program.

8.5.16 template option

This is the "template name" option. Specifies the template name to be used for generating the final output.

8.5.17 agarg option

This is the "autogen argument" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• must not appear in combination with any of the following options: output.

This is a pass-through argument. It allows you to specify any arbitrary argument to be passed to AutoGen.

8.5.18 base-name option

This is the "base name for output file(s)" option.

This option has some usage constraints. It:

• must not appear in combination with any of the following options: output.

When output is going to AutoGen, a base name must either be supplied or derived. If this option is not supplied, then it is taken from the template option. If that is not provided either, then it is set to the base name of the current directory.

8.6 Replacement for Stdio Formatting Library

Using the `printf' formatting routines in a portable fashion has always been a pain, and this package has been way more pain than anyone ever imagined. Hopefully, with this release of snprintfv, the pain is now over for all time.

The issues with portable usage are these:

1. Argument number specifiers are often either not implemented or are buggy. Even GNU libc, version 1 got it wrong.

2. ANSI/ISO "forgot" to provide a mechanism for computing argument lists for vararg procedures.

3. The argument array version of printf (`printfv()') is not generally available, does not work with the native printf, and does not have a working argument number specifier in the format specification. (Last I knew, anyway.)

4. You cannot fake varargs by calling `vprintf()' with an array of arguments, because ANSI does not require such an implementation and some vendors play funny tricks because they are allowed to.

These four issues made it impossible for AutoGen to ship without its own implementation of the `printf' formatting routines. Since we were forced to do this, we decided to make the formatting routines both better and more complete :-). We addressed these issues and added the following features to the common printf API:

1. The formatted output can be written to
   • a string allocated by the formatting function (`asprintf()').
   • a file descriptor instead of a file stream (`dprintf()').
   • a user specified stream (`stream_printf()').

2. The formatting functions can be augmented with your own functions. These functions are allowed to consume more than one character from the format, but must commence with a unique character. For example,

   "%{struct stat}\n"

   might be used with '{' registered to a procedure that would look up "struct stat" in a symbol table and do appropriate things, consuming the format string through the '}' character.

Gary V. Vaughan was generous enough to supply this implementation. Many thanks!!

For further details, the reader is referred to the snprintfv documentation. These functions are also available in the template processing as `sprintf' (see section 3.5.22 `sprintf' - format a string), `printf' (see section 3.5.17 `printf' - format to stdout), `fprintf' (see section 3.5.5 `fprintf' - format to a file), and `shellf' (see section 3.5.21 `shellf' - format a string, run shell).