Publishing a novel series with pandoc publish

This is another example of the use of pandoc publish, my own wrapper and configuration for pandoc, to allow me to publish my fiction written in Markdown as both ebooks and paperback novels.

This time, I’m going to show how I can publish any of the instalments of my KESTREL techno-thrillers series. Here’s my ideal setup:

A lot of this is trivial with publish. Here’s the folder structure I use to keep the novels’ manuscripts with shared front- and back-matter:

01 Front Matter 02 Manuscripts 01 CHANGER 02 TOLL 03 JINX 03 Back Matter

Self-explanatory. The contents of the front- and back-matter directories are similarly organised, with anything that varies on a per-instalment basis in a suitable subdirectory, and named with the particular book it applies to. Here’s the front-matter directory, for example.

00 Accolades.md 01 Description Description CHANGER.md Description JINX.md Description TOLL.md 02 Series Promo.md 03 About the author.md 04 Title page.md 05 Copyright statement.md 06 Dedication.md 07 In Memoriam In Memoriam JINX.md 08 Epigraph Epigraph JINX.md Epigraph TOLL.md

This makes it easy to see which books have substantial differences. Each has its own description page (a copy of the back-cover teaser blurb, which is especially useful to have in ebook editions). Only JINX has an In Memoriam page, and only TOLL and JINX have epigraphs. These variants could also be all kept at the root level of the folder, but I find this approach less cluttered and more evident. It’s a personal choice. The back-matter directory, of course, has a similar setup with both shared files like an afterword and promotions for my other books, and instalment-specific files like the acknowledgements section.

Almost all of the files, of course, end up being customised for each instalment; those customisations just happen via the injection of metadata into placeholders. For example, the content of the title page file looks like this:

{.book-subtitle} %subtitle%
{.book-title} %title%
{.book-author} %author%

The placeholders are the %-delimited tokens. The braces-surrounded syntax applies CSS styles to block elements, as detailed in the documentation.

It’s easy to pass suitable placeholder values when building each instalment: just have a separate metadata JSON file for each one, and supply it as an argument to the build script. That leaves us with two things to do:

  1. Reformat the manuscripts as necessary to ensure the finished books look as good as possible.
  2. Ensure that only the appropriate Markdown files are included in each instalment.

We want to completely automate both of these things. Let’s deal with them one at a time.

Reformatting for publishing

The Markdown files for my books are exceptionally simple, which is just how I like it. Aside from the occasional emphasis, there are really only two kinds of formatting: the titles of chapters (and parts) are level-1 headings, and scene breaks are horizontal rules. The default styles supplied by publish already handle both of those things properly, so there’s really only one wrinkle to sort out: the printed-book convention that each part of a book (if it has them), and also each part-leading chapter, should start recto.

What this means is that those sections should begin on a facing page which is furthest from the start of the book; for example, in left-to-right languages like English, the recto pages are the right-hand ones. Thus, each part of a book (Part 1, Part 2, etc, including a prologue or epilogue) should begin on a right-hand page. Chapters which lead (or begin) each part should also start recto. Thankfully, publish already has syntax for this, so we only need to apply it to the appropriate Markdown headings in the manuscript at build time.

This sort of reformatting task is exactly what the transformations feature was designed for. The Markdown below shows what we want to end up with: the job is to insert the brace-delimited attributes which are highlighted.

# Part 2 {.recto}
# Chapter 7 {.recto}

The thing to notice is that part-leading chapter headings, by their nature, are separated from the prior part heading by just zero or more empty lines. This makes it easy to create the appropriate transformations file:

Annotate Markdown files for print:
Parts start recto										(?im)^(#\s+(?:Part\s+\d+|Prologue|Epilogue).*)$			\1 {.recto}
Part-leading chapters start recto		(?im)^(#\s+Part\s+\d+.*$\n+^#\s+Chapter\s+\d+.*)$		\1 {.recto}

This transformations file is applied to all instalments in the series, since their formatting is identical, and the resulting print-ready PDFs will have suitably-formatted part headings and leading chapter headings. This just leaves us with ensuring that each book only contains the content it’s meant to.

Excluding inapplicable files

From a human perspective, it’s easy enough to describe what we need when publishing each instalment:

Thankfully, this is also very easy to accomplish, using the advanced options available within the exclusions feature. Here’s the required exclusions file, which applies to all instalments:

include	filename	(Front|Back) Matter/	(?iM)%title%		Include only book-matter subfolder filenames with %title%
include	filepath	Manuscripts						(?iM)%title%		Include only Manuscripts paths with %title%

This achieves exactly what’s needed, using publish’s support for exclusions based on metadata to rewrite the matching patterns before running them.

All of my content files are in a directory called Books, and beside that directory in the parent folder is where I keep all the configuration files I’ve mentioned. I also have an alias set up in my terminal to let me use ppublish instead of having to invoke python with the path to the build script, for convenience. And since my shared transformations and exclusions files have the default filenames which the build script looks for automatically, I don’t need to explicitly specify them.

So, for example, from the parent directory I can build CHANGER with this command, if I want verbose output so I can keep track of what’s going on:

ppublish --input-folder Books --json-metadata-file metadata-changer.json --verbose

But that’s a bit too wordy, so I make use of publish’s support for args files to keep those arguments in a text file instead:

–verbose –input-folder Books –json-metadata-file metadata-changer.json

Then I can build the book with just this:

ppublish @changer.txt

If I only want to generate an epub instead of the default epub and pdf, I can do so with the --formats epub argument appended, and so on. And that’s it.

I’ve developed publish both intentionally and in response to my own workflow needs (current and projected), and it’s really come into its own as a way to maintain complete control over my body of work, and how I get it ready to share with others. There’s more functionality that I haven’t yet covered here, and if the idea of having a Markdown-based multi-format publishing setup that’s not dependent on any siloed desktop app appeals to you, I encourage you to try it out for yourself.

Thanks for reading.