Most of the customers I do work for are very technical. They have their own IT departments, often millions or tens of millions of dollars of equipment, and trained operators.

There are times I work for one person who isn’t all that technical but has an automation need. I did a project for one such person earlier this week.

As an IT project, it was terribly simple. It was similar to mail-merge but the output wasn’t for people to read but rather a script to drive something else. Thus, this was a metaprogramming deal: create a program to generate a script that does “the right thing” driven from data from Excel.

To keep the sheet from being tied up, the data was dumped as csv. I could have read it from Excel, but that was fine with the customer.

Of course, one reaches for JSon or XML in an IT environment, but to control this beast my customer wasn’t going to learn about balanced <>, or {}, or double-quoted keys with colons. The customer isn’t technical. Further, to control what subset of rows from the export were processed wouldn’t be good via command line arguments.

How does one work in the small? No Mongo, no PostgreSQL … nobody to keep the ecosystem. KISS — Keep It Simple Stupid — text files. No elegant recursive data-structures in the text files as those are only tolerable for US because we have awesome coding editors. The customer has Notepad.

I picked the old-school control, with a template file that held run-time configuration. That allowed control without keeping track of data “sideband” and without a need for serialization of state. It looks like this:

run-range 2 20
csv-source test1.csv

So, simple lines of space-delimited positional fields.

The body was just arbitrary lines of text with the only “special” meaning the merge-fields. It was interesting in that this template was to generate scripts for other environments and tools, so it wasn’t possible to pick “a” magic syntax for the merge fields. Instead, the delimiter for the merging was specified as one of the arguments.

Regular Expressions are awesome and Python’s implementation is great. To handle arbitrary delimiters of what could be magic regex fields (such as + or *) and then to extract them is just this:

delimiter = re.escape('*')
pattern = delimiter+"(.+?)"+delimiter
temp = re.split(pattern, src) # src is from the body text

The result of that is an array (or list, in Python) that with * as delim would convert “Hi, *first name*, how are you?” into:

['Hi, ', 'first name', ', how are you?']

That is interpreted as pairs [ constant split-match constant split-match … constant] so instead of trying a regular expression search/replace (which would have to be done iteratively) you can then walk the list emitting the constants and doing lookup/replace on the match result.

While not nearly as powerful as a true developer template engine, it requires only “user level” mail-merge skills to configure and run.

So, simple notation for configuration, template specification, and control, and simple code for interpreting the user’s template and getting the sections of constant and fill-in text.

Most of the code turned out to be validation of the files to make sure that all merge fields had appropriate sources, that there were no defined fill-ins unused, etc. Cryptic exceptions don’t work for end-users so this required very clear error messages that detailed what was wrong and where.

These small projects can be fun … and I was working with the decision maker who would also use it. No layers of management with different goals and wishes. That’s not just fun … it’s awesome.

Keep the Light!