Book conversion process

I realised that the most nostalgic part of the Nokia 3310 for me was the font, but I didn't want to try and write something, so I thought of public domain books. It was very amusing trying to design good UX for what was knowingly a terrible form factor. It was also quite satisfying to implement. I had fun :)

I built the game in Unity and relied heavily on the 'page' wrapping mode provided by TextMeshPro. Once I had setup the UI, TMPro could handle splitting the text up into screen-sized pages of text for me. I still had to supply it with the text to chunk though, and found that just sticking a whole book's worth of text into a single TMPro element caused both the Editor and the game to lag tremendously. There were also many characters included in the book texts which weren't renderable by the Nokia 3310 font.

I decided to split each book into separate text files for each chapter, as those were small enough chunks for TMPro to split into pages without lagging, and it provided a simple way to handle chapter-by-chapter navigation. In the inspector for the book menu script, for each book I list the title, author, name of a Resources folder, and number of chapters. This becomes an option in the book selection menu, and opening a book simply involves loading the 0.txt file in the indicated Resources subfolder.

Changing chapters simply involves loading the next or previous text file. Changing pages is provided by TextMeshPro. When reading a book, the number in the top left is the chapter and the number in the top right is the page.

My editing process changed over time as I improved various aspects, but the final process for adding a new book to the game was along the following lines:

1. Download the raw text for a public domain book from Project Gutenberg, making sure it has a license which allows me to redistribute it.
2. Remove the Project Gutenberg header and footer. This is not only simpler for parsing as it then doesn't need to be skipped over, but this also grants further freedoms of distribution as described in the Project Gutenberg license.
3. Set the last line to "<page>THE END". The "<page>" tag is a TextMeshPro rich text page break to force the "THE END" text to appear on its own on the last page for consistency.
4. Search through all instances of multiple consecutive spaces. These are usually custom formatting for tables, poems, quotes, signs or other special text within the book. TextMeshPro doesn't display blocks of whitespace so the formatting will break quite badly here, I usually just simplified this as much as possible and changed the formatting on a case-by-case basis.
5. Remove any footnotes, author's notes, transcription notes, or other meta information. These are usually wrapped in one of []{}<>.
6. Remove any non-narrative sections at the start or end of the book, such as an author biography, glossary, history, index, etc. Like the footnotes and other meta information these could be kept, but I wanted just the pure core book text because the display couldn't handle other kinds of information.
7. Replace all characters the Nokia font cannot render:
   1. — to --
   2. ‘ to '
   3. ’ to '
   4. “ to "
   5. ” to "
   6. Delete all underscores as these are used to indicate italics, bold or other custom formatting
   7. Search through all instances of * as these are usually also used for custom formatting
8. Replace all "\r\n\r\n" with "###" (note this is an advanced text replacement operating on the actual line break characters). The plain text books usually have character limits for how long a line can be, meaning there are artificial line breaks inserted into the text. We need to remove those as we want to use our own line breaks, but need to preserve the double line breaks used to add a blank line between paragraphs. This text replacement turns all double line breaks into a triple hash, placing the start of each paragraph on the same line as the end of the previous paragraph. If the text contains any genuine instances of a triple hash, use a different character string.
9. Replace all "\r\n" with " " (a single space). This removes the artificial line breaks.
10. Replace all "### " (triple hash followed by single space) with "\r\n\r\n". For some reason many of the blank lines in the source book texts contain a single space; this removes though while restoring the paragraph line breaks.
11. Replace all "###" with "\r\n\r\n". This restores the remaining paragraph line breaks.
12. Split into separate text documents, with 0.txt being the title and author, then 1+ being separate chapters or section breaks.