Using borb with Python to Create E-books From Project Gutenberg

Using borb with Python to Create E-books From Project Gutenberg

The Portable Document Format (PDF) is not a WYSIWYG (What You See is What You Get) format. It was developed to be platform-agnostic, independent of the underlying operating system and rendering engines.

To achieve this, PDF was constructed to be interacted with via something more like a programming language, and relies on a series of instructions and operations to achieve a result. In fact, PDF is based on a scripting language - PostScript, which was the first device-independent Page Description Language.

In this guide, we'll be using borb - a Python library dedicated to reading, manipulating and generating PDF documents. It offers both a low-level model (allowing you access to the exact coordinates and layout if you choose to use those) and a high-level model (where you can delegate the precise calculations of margins, positions, etc to a layout manager).

In this guide, we'll take a look at how to convert a UTF-8 book (from Project Gutenberg) to a PDF document.

Project Gutenberg eBooks may be freely used in the United States because most are not protected by U.S. copyright law. They may not be free of copyright in other countries.

Installing borb

borb can be downloaded from source on GitHub, or installed via pip:

$pip install borb  Installing unidecode For this project we will also use unidecode, it's a wonderful little library that converts text from UTF-8 to ASCII. Keep in mind that not every character in UTF-8 can be represented as an ASCII character. This is a lossy conversion, in principle so there will be some discrepancy every time you do a conversion: $ pip install unidecode


Creating a PDF Document with borb

Creating a PDF document using borb typically follows the same steps every time:

from borb.pdf.document import Document
from borb.pdf.page.page import Page
from borb.pdf.pdf import PDF

import typing
import re

from borb.pdf.canvas.layout.page_layout.multi_column_layout import SingleColumnLayout
from borb.pdf.canvas.layout.page_layout.page_layout import PageLayout

# Create empty Document
pdf = Document()

# Create empty Page
page = Page()

pdf.append_page(page)

# Create PageLayout
layout: PageLayout = SingleColumnLayout(page)


Creating E-books with borb

Note: We'll be dealing with raw text books. Each book will have a different structure and each book requires a different approach to rendering. This is a highly subjective (styling) and highly book-dependent task, though, the general process is the same.

The book we'll be downloading is UTF-8 encoded. Not every font supports every character. In fact, the PDF spec defines 14 standard fonts (which every reader/writer ought to have embedded), none of which support the full UTF-8 range.

So, to make our lives a bit easier, we're going to be using this little utility function to convert a str to ASCII:

from unidecode import unidecode

def to_ascii(s: str) -> str:
s_out: str = ""
for c in s:
if c == '“' or c == '”' or c == 'â':
s_out += '"'
else:
s_out += unidecode(c)
return s_out


In our example, we'll be using "The Mysterious affair at Styles" by Agatha Christie, which can be easily obtained in raw format from Project Gutenberg:

# Define which ebook to fetch
url = 'https://www.gutenberg.org/files/863/863-0.txt'

import requests
txt = requests.get(url).text

# Split to lines
lines_of_text: typing.List[str] = re.split('\r\n', txt)
lines_of_text = [to_ascii(x) for x in lines_of_text]

# Debug
print("This ebook contains %d lines... " % len(lines_of_text))


This prints:

Downloaded 361353 bytes of text...
This ebook contains 8892 lines...


The first lines of text are a general header added by Project Gutenberg. We don't really want that in our ebook so we're going to simply delete it, by checking whether a line starts with a certain pattern and slicing it off via the slice notation:

# Skip header
for i in range(0, len(lines_of_text)):
if lines_of_text[i].startswith("*** START OF THE PROJECT GUTENBERG EBOOK"):
break


This prints:

Free eBook: Git Essentials

Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

The first 24 lines are the gutenberg header...


Similarly, the last lines of text are just a copyright notice. We'll delete that as well:

# Skip footer
footer_offset: int = len(lines_of_text)
for i in range(0, len(lines_of_text)):
if "*** END OF THE PROJECT GUTENBERG EBOOK" in lines_of_text[i]:
footer_offset = i
break
lines_of_text = lines_of_text[0:footer_offset]
print("The last %d lines are the gutenberg footer .." % (len(lines_of_text) - footer_offset))


With that out of the way, we're going to process the main body of text.

This code took some trial and error and if you're working with a different book - it will take some trial and error too.

Figuring out when to insert a chapter title, when to start a new paragraph, what the table of contents is, etc. depends on the book as well. This is an opportunity to play around with borb a bit, and try to parse the input yourself with a different book:

from borb.pdf.canvas.layout.text.paragraph import Paragraph
from borb.pdf.canvas.color.color import HexColor, X11Color
from decimal import Decimal

# Main processing loop
i: int = 0
while i < len(lines_of_text):

# Process lines
paragraph_text: str = ""
while i < len(lines_of_text) and not len(lines_of_text[i]) == 0:
paragraph_text += lines_of_text[i]
paragraph_text += " "
i += 1

# Empty line
if len(paragraph_text) == 0:
i += 1
continue

# Space
if paragraph_text.isspace():
i += 1
continue

if sum([1 for x in paragraph_text.split(' ') if 'CHAPTER' in x]) > 2:
i += 1
continue

# Debug
print("Processing line %d / %d" % (i, len(lines_of_text)))

# Outline
if paragraph_text.startswith("CHAPTER"):
try:
page = Page()
pdf.append_page(page)
layout = SingleColumnLayout(page)
except:
pass
continue

# Default
try:
except:
pass

# Default behaviour
i += 1


All that's left is to store the final PDF document:

with open("output.pdf", "wb") as pdf_file_handle:
PDF.dumps(pdf_file_handle, pdf)


Conclusion

In this guide you've learned how to process a large piece of text and create a PDF out of it automatically using borb.

Creating books from raw text files is not a standard process, and you'll have to test things out and play around with the loops and the way you treat text to get it right.

Last Updated: November 2nd, 2021

Get tutorials, guides, and dev jobs in your inbox.

Joris SchellekensAuthor

I'm a software architect from Belgium, with a passion for machine learning, knowledge-based systems and graph algorithms. I'm also the author of borb, the pure python PDF library.

Project

Real-Time Road Sign Detection with YOLOv5

# python# machine learning# computer vision# pytorch

If you drive - there's a chance you enjoy cruising down the road. A responsible driver pays attention to the road signs, and adjusts their...

David Landup
Details
Project

Data Visualization in Python: The Collatz Conjecture

# python# matplotlib# data visualization

The Collatz Conjecture is a notorious conjecture in mathematics. A conjecture is a conclusion based on existing evidence - however, a conjecture cannot be proven....

Details