The Portable Document Format (PDF) is not a WYSIWYG (What You See is What You Get) format. It was developed to be platform-agnostic, independent of the underlying operating system and rendering engines.
To achieve this, PDF was constructed to be interacted with via something more like a programming language, and relies on a series of instructions and operations to achieve a result. In fact, PDF is based on a scripting language - PostScript, which was the first device-independent Page Description Language.
In this guide, we'll be using borb - a Python library dedicated to reading, manipulating and generating PDF documents. It offers both a low-level model (allowing you access to the exact coordinates and layout if you choose to use those) and a high-level model (where you can delegate the precise calculations of margins, positions, etc to a layout manager).
In this guide, we'll take a look at how to convert a UTF-8 book (from Project Gutenberg) to a PDF document.
Note: Project Gutenberg eBooks may be freely used in the United States because most are not protected by U.S. copyright law. They may not be free of copyright in other countries.
borb can be downloaded from source on GitHub, or installed via
pip install borb
For this project we will also use
unidecode, it's a wonderful little library that converts text from UTF-8 to ASCII. Keep in mind that not every character in UTF-8 can be represented as an ASCII character.
This is a lossy conversion, in principle so there will be some discrepancy every time you do a conversion:
pip install unidecode
Creating a PDF Document with borb
Creating a PDF document using borb typically follows the same steps every time:
from borb.pdf.document import Document from borb.pdf.page.page import Page from borb.pdf.pdf import PDF import typing import re from borb.pdf.canvas.layout.page_layout.multi_column_layout import SingleColumnLayout from borb.pdf.canvas.layout.page_layout.page_layout import PageLayout # Create empty Document pdf = Document() # Create empty Page page = Page() # Add Page to Document pdf.append_page(page) # Create PageLayout layout: PageLayout = SingleColumnLayout(page)
Creating Ebooks with borb
Note: We'll be dealing with raw text books. Each book will have a different structure and each book requires a different approach to rendering. This is a highly subjective (styling) and highly book-dependent task, though, the general process is the same.
The book we'll be downloading is UTF-8 encoded. Not every font supports every character. In fact, the PDF spec defines 14 standard fonts (which every reader/writer ought to have embedded), none of which support the full UTF-8 range.
So, to make our lives a bit easier, we're going to be using this little utility function to convert a
str to ASCII:
from unidecode import unidecode def to_ascii(s: str) -> str: s_out: str = "" for c in s: if c == '“' or c == '”' or c == 'â': s_out += '"' else: s_out += unidecode(c) return s_out
Next, in our main method, we're going to be downloading the UTF-8 book.
In our example, we'll be using "The Mysterious affair at Styles" by Agatha Christie, which can be easily obtained in raw format from Project Gutenberg:
# Define which ebook to fetch url = 'https://www.gutenberg.org/files/863/863-0.txt' # Download text import requests txt = requests.get(url).text print("Downloaded %d bytes of text..." % len(txt)) # Split to lines lines_of_text: typing.List[str] = re.split('\r\n', txt) lines_of_text = [to_ascii(x) for x in lines_of_text] # Debug print("This ebook contains %d lines... " % len(lines_of_text))
Downloaded 361353 bytes of text... This ebook contains 8892 lines...
The first lines of text are a general header added by Project Gutenberg. We don't really want that in our eBook so we're going to simply delete it, by checking whether a line starts with a certain pattern and slicing it off via the slice notation:
# Skip header header_offset: int = 0 for i in range(0, len(lines_of_text)): if lines_of_text[i].startswith("*** START OF THE PROJECT GUTENBERG EBOOK"): header_offset = i + 1 break while lines_of_text[header_offset].isspace(): header_offset += 1 lines_of_text = lines_of_text[header_offset:] print("The first %d lines are the gutenberg header..." % header_offset)
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
The first 24 lines are the gutenberg header...
Similarly, the last lines of text are just a copyright notice. We'll delete that as well:
# Skip footer footer_offset: int = len(lines_of_text) for i in range(0, len(lines_of_text)): if "*** END OF THE PROJECT GUTENBERG EBOOK" in lines_of_text[i]: footer_offset = i break lines_of_text = lines_of_text[0:footer_offset] print("The last %d lines are the gutenberg footer .." % (len(lines_of_text) - footer_offset))
With that out of the way, we're going to process the main body of text.
This code took some trial and error and if you're working with a different book - it will take some trial and error too.
Figuring out when to insert a chapter title, when to start a new paragraph, what the table of contents is, etc. depends on the book as well. This is an opportunity to play around with borb a bit, and try to parse the input yourself with a different book:
from borb.pdf.canvas.layout.text.paragraph import Paragraph from borb.pdf.canvas.layout.text.heading import Heading from borb.pdf.canvas.color.color import HexColor, X11Color from decimal import Decimal # Main processing loop i: int = 0 while i < len(lines_of_text): # Process lines paragraph_text: str = "" while i < len(lines_of_text) and not len(lines_of_text[i]) == 0: paragraph_text += lines_of_text[i] paragraph_text += " " i += 1 # Empty line if len(paragraph_text) == 0: i += 1 continue # Space if paragraph_text.isspace(): i += 1 continue # Contains the word 'CHAPTER' multiple times (likely to be table of contents) if sum([1 for x in paragraph_text.split(' ') if 'CHAPTER' in x]) > 2: i += 1 continue # Debug print("Processing line %d / %d" % (i, len(lines_of_text))) # Outline if paragraph_text.startswith("CHAPTER"): print("Adding Header of %d bytes .." % len(paragraph_text)) try: page = Page() pdf.append_page(page) layout = SingleColumnLayout(page) layout.add(Heading(paragraph_text, font_color=HexColor("13505B"), font_size=Decimal(20))) except: pass continue # Default try: layout.add(Paragraph(paragraph_text)) except: pass # Default behavior i += 1
All that's left is to store the final PDF document:
with open("output.pdf", "wb") as pdf_file_handle: PDF.dumps(pdf_file_handle, pdf)
In this guide you've learned how to process a large piece of text and create a PDF out of it automatically using borb.
Creating books from raw text files is not a standard process, and you'll have to test things out and play around with the loops and the way you treat text to get it right.