The “Admin” is the administrative console that’s built into the Django application. It provides an easy way to administer content on any Django site. For developers who are tired of writing the same admin functionality again and again (“Add/Edit/Delete page”) it’s a huge win.

Neither one is publishing-specific, but Django (like Rails and Drupal) is a great way to deploy a web application quickly and easily. (Bookworm runs on Django, as do many of my smaller projects.)

Massive XML files are uncommon in the general web development world, where the primary roles of XML are either as configuration files, read only infrequently, or for interchange across the web, in which case the files are necessarily small. It’s rare to encounter XML measured in gigabytes or more; data at that level is usually stored in a relational database.

For that reason I find myself frustrated with many XML tools, even those ostensibly designed to handle large amounts of data.  Too often they don’t scale well or at least easily.  I don’t believe that scaling should be a black art that each individual developer needs to solve independently.  Unfortunately, in commercial products ease-of-use is a key bullet point and computationally-difficult problems are hard to summarize in a user’s guide.

I tend to recommend open-source software most strongly in two scenarios: for small projects with limited budgets and for large projects with unique challenges.  There simply isn’t going to be a one-size-fits-all application for most interesting publishing work.

This is one of many reasons I’m excited by Google’s willingness to open its Google Books archive to researchers:  Python is a first-class programming language in the Google ecosystem, and Google has a good track record of open-sourcing those internal tools with limited commercial value.  I expect a lot of interesting work to come out of that archive once it’s available.

The source documents are encoded in TEI XML and rendered to the browser using Python and lxml, three of my favorite technologies.

There are a few take-aways from this project that might benefit anyone working in a similar area and scale:

• Use a standard encoding format (in this case TEI, but choose an appropriate one based on the source content)
• Use a modern programming language, even in a humanities context (e.g. Python)
• Use modern XML parsing tools (e.g. lxml + XPath + XSLT)

The key advantage of libraries such as lxml in publishing and digitization projects is that it allows the developer to freely mix XML-native languages like XPath and XSLT with the expressive, procedural programming style of Python. I’m still amazed by how many people are “parsing” XML using regular expressions (or worse), or using plain CGI/Perl scripts to serve up content. There are easier ways!

“Free” doesn’t have to mean primitive. In fact I would argue that projects like Pinax can jump-start library or digital archive sites into the 21st century with less work than a grad student will spend crafting a bespoke Perl script.

Congratulations to Thomas Crombez and his team!

I just received my copy of Python for Unix and Linux System Administration by Noah Gift and Jeremy Jones, for which I was a technical reviewer. I’ve done several tech reviews for O’Reilly in the past, on both Python and CSS, and the least enjoyable part of the process has been the actual method of providing feedback.

At my previous job we routinely used Word (or OpenOffice) with Track Changes for collaborative editing, and as Word-based tools go I felt that worked well. For whatever reason, though, most of the pre-release books I’ve received have been in PDF, which is limiting in several ways:

1. Cut and paste from PDF, especially of source code, often does not work properly. To test the code a technical reviewer needs to ensure that they are accurately repeating exactly what is in the book.
2. There is no ability to in-line comment on particular words or phrases.
3. The copy of the text I’m reading may be days or even weeks out of date, back when the author did the PDF conversion.

Python for Unix and Linux System Adminstration was different: the authors elected to use the source code control system Subversion to manage the writing.  The text was composed in DocBook XML rather than Word or some other non-text format.  While I’m sure this was done entirely to facilitate collaboration between the authors, it had the downstream effect of making it supremely easy for me to review it:

1. Code samples were in plain text, and if they were formatted incorrectly, that was useful feedback to be able to give (especially in a language that is sensitive to whitespace, as Python is)
2. While I was told I would be able to “commit” my changes back to the authors inside of the source text, I still chose to use an external file to provide my comments.  I did this only because I wasn’t sure that the authors would be able to manage multiple commits coming in from technical reviewers, and because we hadn’t decided on a common tagging framework.  With more editorial guidance, being able to commit my comments directly into the source could be very useful (including the ability to potentially see other reviewers’ comments, and avoid repeating myself).
3. Each time I went to work on the book, I was able to get a fresh copy of the text.  I didn’t go back and re-check old sections, but it did mean that any section I worked on was always up-to-date.

When used with friendly front-end software like TortoiseSVN, Subversion isn’t even very difficult.  It’s certainly no more arcane than many professional content management systems.  Although it works best when managing text content (which could be Office-supported XML formats), it would still provide value with binary formats. It’s worth considering for any publisher that has to manage multiple, distributed editors or authors and wants to improve the process using entirely free software.

For more on the subject, Rachel Greenham has a nice tutorial explaining how authors can use Subversion with OS X. The definitive word is the Subversion book.

(I highly recommend Python for Unix and Linux System Adminstration as well, even for Python programmers who aren’t system administrators. It collects an impressive breadth of information in one place and showed me how to automate processes I hadn’t even realized needed automating.)