Guest Contributor

Getting Started with Python's Wikipedia API

<h2 id="introduction">Introduction</h2>
Email is one of the most used tools for communication in web applications because it helps you reach your users directly, build your brand, or send general notifications.
So, you are thinking about sending emails from your next great Node.js application. You are in the right place!
In this tutorial you'll learn how to send emails with HTML content and attachments using the <a rel="nofollow" target="_blank" href="https://nodemailer.com/">nodemailer</a> module, as well as set up <a rel="nofollow" target="_blank" href="https://mailtrap.io/">Mailtrap</a>, a fake SMTP server, for testing your code.
<h2 id="prerequisites">Prerequisites</h2>
To follow along, you will need to have <a rel="nofollow" target="_blank" href="https://nodejs.org/en/">Node.js</a> and <a rel="nofollow" target="_blank" href="https://www.npmjs.com/">npm</a> (Node Package Manager) installed locally.
To test sending emails from a local development machine, without having to configure a server, we will be using Mailtrap.
<h2 id="gettingstarted">Getting Started</h2>
There are many Node.js modules for sending emails. Amongst them, <code>nodemailer</code> is the most popular choice. It is a module that gives you the ability to easily send emails without hassle.
Let's go ahead and create a new folder and a new <code>package.json</code> file with the <code>npm init</code> command:
<pre><code class="hljs">$ mkdir nodejs-email
$ cd nodejs-email
$ npm init -y
</code></pre>
The <code>-y</code> flag provided will skip the step-by-step tool to scaffold out your project.
Now let's install the <code>nodemailer</code> module using <code>npm</code>:
<pre><code class="hljs">$ npm install nodemailer
</code></pre>
With the model ready for use, let's create an <code>index.js</code> file in our project directory.
In Node.js, the <code>require</code> syntax is used to load modules in to your code:
<pre><code class="hljs">const nodemailer = require(&#x27;nodemailer&#x27;);
</code></pre>
<h2 id="usingsmtpfornodemailertransport">Using SMTP for Nodemailer Transport</h2>
The <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol">Simple Mail Transfer Protocol</a> (SMTP) is a protocol for sending e-mail messages between servers. Most e-mail systems that send mail over the Internet support SMTP based sending.
And guess what? SMTP is the main transport used by <code>nodemailer</code> for delivering messages.
Creating a <code>nodemailer</code> transport is as easy as calling the following method with a few parameters:
<pre><code class="hljs">let transport = nodemailer.createTransport(options[, defaults])
</code></pre>
However, to actually send a message through our transport, we have to configure the connection first.
<h2 id="testingourcodewithmailtrap">Testing our Code with Mailtrap</h2>
Mailtrap is a &quot;fake SMTP server&quot; used for development purposes. Instead of having to test your code with your own email account, and potentially flooding your inbox with test emails, you can instead use Mailtrap as the endpoint.
Create a new account on Mailtrap if you don't already have one, and then create a new inbox and get your credentials:
<img src="https://stackabuse.s3.amazonaws.com/media/how-to-send-an-email-with-node-js-1.png" alt="">
Now, all we need to do is put the credentials into <code>nodemailer</code>'s transport object:
<pre><code class="hljs">let transport = nodemailer.createTransport({
 host: &#x27;smtp.mailtrap.io&#x27;,
 port: 2525,
 auth: {
 user: &#x27;put_your_username_here&#x27;,
 pass: &#x27;put_your_password_here&#x27;
 }
});
</code></pre>
With all of that set up, we can go ahead and send our first test email:
<pre><code class="hljs">const message = {
 from: &#x27;elonmusk@tesla.com&#x27;, // Sender address
 to: &#x27;to@email.com&#x27;, // List of recipients
 subject: &#x27;Design Your Model S | Tesla&#x27;, // Subject line
 text: &#x27;Have the most fun you can in a car. Get your Tesla today!&#x27; // Plain text body
};
transport.sendMail(message, function(err, info) {
 if (err) {
 console.log(err)
 } else {
 console.log(info);
 }
});
</code></pre>
Ta-da, that's it!
<img src="https://stackabuse.s3.amazonaws.com/media/how-to-send-an-email-with-node-js-2.png" alt="">
<h2 id="sendinganhtmlemail">Sending an HTML Email</h2>
While the previous example does the job just fine, it's what some may call &quot;boring&quot;.
Nowadays, emails tend to be styled, colorful, with big buttons and exciting and engaging visuals. To achieve this, we embed HTML into our emails.
You can create these HTML emails and send them with <code>nodemailer</code> just as easily as you could send a plain text email. All it takes is to use the <code>html</code> parameter in the <code>message</code> object instead of <code>text</code>:
<pre><code class="hljs">const message = {
 from: &#x27;elonmusk@tesla.com&#x27;,
 to: &#x27;to@email.com&#x27;,
 subject: &#x27;Design Your Model S | Tesla&#x27;,
 html: &#x27;&lt;h1&gt;Have the most fun you can in a car!&lt;/h1&gt;&lt;p&gt;Get your &lt;b&gt;Tesla&lt;/b&gt; today!&lt;/p&gt;&#x27;
};
</code></pre>
Although this example is quite simple, only including a <code>&lt;h1&gt;</code>, <code>&lt;p&gt;</code>, and <code>&lt;b&gt;</code> tag, you can include much, much more.
Voilà!
<img src="https://stackabuse.s3.amazonaws.com/media/how-to-send-an-email-with-node-js-3.png" alt="">
Keep in mind that you can also send both HTML and plain text in a single email by providing both parameters. With both versions in your email message, the recipient's email client can then choose which one to display.
<h2 id="sendingemailwithanattachment">Sending Email with an Attachment</h2>
There are countless reasons why you might want to include an attachment in your email, like sending pictures, spreadsheets for work, or videos.
As another example, if you run an e-commerce website, for example, you might want to send receipts to your customers:
<pre><code class="hljs">const nodemailer = require(&quot;nodemailer&quot;);

let transport = nodemailer.createTransport({
 // ...
});

const message = {
 from: &#x27;elonmusk@tesla.com&#x27;,
 to: &#x27;to@email.com&#x27;,
 subject: &#x27;Design Your Model S | Tesla&#x27;,
 html: &#x27;&lt;h1&gt;Have the most fun you can in a car!&lt;/h1&gt;&lt;p&gt;Get your &lt;b&gt;Tesla&lt;/b&gt; today!&lt;/p&gt;&#x27;,
 attachments: [
 { // Use a URL as an attachment
 filename: &#x27;your-testla.png&#x27;,
 path: &#x27;https://media.gettyimages.com/photos/view-of-tesla-model-s-in-barcelona-spain-on-september-10-2018-picture-id1032050330?s=2048x2048&#x27;
 }
 ]
};
transport.sendMail(message, function (err, info) {
 // ...
 });
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/how-to-send-an-email-with-node-js-4.png" alt="">
And you can use a file on disk as an attachment too. They don't need to be web-based.
<h2 id="conclusion">Conclusion</h2>
You can now easily send personalized emails in your Node.js code using the <code>nodemailer</code> module. And don't forget, the <code>nodemailer</code> <a rel="nofollow" target="_blank" href="https://nodemailer.com/about/">documentation</a> is your best friend for learning about the more advanced options.

How to Send Emails with Node.js

This article is the third in a series on working with PDFs in Python:
<ul>
<li><a target="_blank" href="/working-with-pdfs-in-python-adding-images-and-watermarks/">Reading and Splitting Pages</a></li>
<li><a target="_blank" href="/working-with-pdfs-in-python-adding-images-and-watermarks/">Adding Images and Watermarks</a></li>
<li>Inserting, Deleting, and Reordering Pages (you are here)</li>
</ul>
<h2 id="introduction">Introduction</h2>
This article is part three of a little series on working with PDFs in Python. In the previous articles we gave an introduction into reading PDF documents using Python. So far you have learned how to manipulate existing PDFs, and to read and extract the content - both text and images. Furthermore, we have discussed splitting documents into its single pages, as well as adding watermarks and barcodes.
Now in this article we will go one step further and demonstrate how to rearrange a PDF document in a few different ways.
<ul>
<li><a href="#deletingpageswithpdfrw">Deleting Pages with pdfrw</a></li>
<li><a href="#deletingpageswithpymupdf">Deleting Pages with PyMuPDF</a></li>
<li><a href="#insertingpageswithpymupdf">Inserting Pages with PyMuPDF</a></li>
<li><a href="#splittingevenandoddpageswithpypdf2">Splitting Even and Odd Pages with PyPDF2</a></li>
</ul>
<h2 id="deletingpageswithpdfrw">Deleting Pages with <code>pdfrw</code></h2>
Deleting individual pages from a PDF file is as simple as the following:
<ul>
<li>Read a PDF as an input file</li>
<li>Write selected pages to a new PDF as an output file</li>
</ul>
The following example removes the first two pages from a PDF document. Using the <a rel="nofollow" target="_blank" href="https://pypi.org/project/pdfrw/">pdfrw</a> library, the file is read with the help of the <code>PdfReader()</code> class first. Except for both the first and second page, each page is added to the output file using the <code>addpage()</code> method, and then written to disk eventually.
Figure 1 shows the output when executing the code on a four-page PDF file.
<pre><code class="hljs"># !/usr/bin/python
# Remove the first two pages (cover sheet) from the PDF

from pdfrw import PdfReader, PdfWriter

input_file = &quot;example.pdf&quot;
output_file = &quot;example-updated.pdf&quot;

# Define the reader and writer objects
reader_input = PdfReader(input_file)
writer_output = PdfWriter()

# Go through the pages one after the next
for current_page in range(len(reader_input.pages)):
 if current_page &gt; 1:
 writer_output.addpage(reader_input.pages[current_page])
 print(&quot;adding page %i&quot; % (current_page + 1))

# Write the modified content to disk
writer_output.write(output_file)
</code></pre>
<img class="img-responsive" alt="Delete the first two pages from a PDF" src="https://stackabuse.s3.amazonaws.com/media/working-with-pdfs-python-inserting-deleting-reordering-pages-1.png">
<h2 id="deletingpageswithpymupdf">Deleting Pages with PyMuPDF</h2>
The <a rel="nofollow" target="_blank" href="https://pypi.org/project/PyMuPDF/">PyMuPDF</a> library comes with quite a few sophisticated methods that simplify deleting pages from a PDF file. It allows you to specify either a single page (using the <code>deletePage()</code> method), or a range of page numbers (using the <code>deletePageRange()</code> method), or a list with the page numbers (using the <code>select()</code> method).
The following example will demonstrate how to use a list in order to select the pages to keep from the original document. Be aware that the pages that are not specified will not be part of the output document. In our case the output document contains the first, second, and fourth pages only.
<pre><code class="hljs"># !/usr/bin/python

# Recall that PyMuPDF is imported as fitz
import fitz

input_file = &quot;example.pdf&quot;
output_file = &quot;example-rearranged.pdf&quot;

# Define the pages to keep - 1, 2 and 4
file_handle = fitz.open(input_file)
pages_list = [0,1,3]

# Select the pages and save the output
file_handle.select(pages_list)
file_handle.save(output_file)
</code></pre>
<h2 id="insertingpageswithpymupdf">Inserting Pages with PyMuPDF</h2>
The <code>PyMuPDF</code> library allows you to insert pages as well. It provides the methods <code>newPage()</code> for adding completely blank pages, and <code>insertPage()</code> in order to add an existing page. The next example shows how to add a page from a different PDF document at the end of another one.
<pre><code class="hljs"># !/usr/bin/python

# Recall that PyMuPDF is imported as fitz
import fitz

original_pdf_path = &quot;example.pdf&quot;
extra_page_path = &quot;extra-page.pdf&quot;
output_file_path = &quot;example-extended.pdf&quot;

original_pdf = fitz.open(original_pdf_path)
extra_page = fitz.open(extra_page_path)

original_pdf.insertPDF(extra_page)
original_pdf.save(output_file_path)
</code></pre>
<h2 id="splittingevenandoddpageswithpypdf2">Splitting Even and Odd Pages with PyPDF2</h2>
The following example uses <a rel="nofollow" target="_blank" href="https://pypi.org/project/PyPDF2/">PyPDF2</a> and does this by taking a file, separating it into its even and odd pages, saving the even pages in the file <code>even.pdf</code>, and the odd pages in <code>odd.pdf</code>.
This Python script starts with the definition of two output files, <code>even.pdf</code> and <code>odd.pdf</code>, as well as their corresponding writer objects <code>pdf_writer_even</code> and <code>pdf_writer_odd</code>. Next, in a for-loop the script goes through the entire PDF file, and reads one page after the other. Pages with even page numbers are added to the stream <code>pdf_writer_even</code> using <code>addPage()</code>, and odd numbers are added to the stream <code>pdf_writer_odd</code>. At the end the two streams are saved to disk in separate files, as defined before.
<pre><code class="hljs">#!/usr/bin/python3

from PyPDF2 import PdfFileReader, PdfFileWriter

pdf_document = &quot;example.pdf&quot;
pdf = PdfFileReader(pdf_document)

# Output files for new PDFs
output_filename_even = &quot;even.pdf&quot;
output_filename_odd = &quot;odd.pdf&quot;

pdf_writer_even = PdfFileWriter()
pdf_writer_odd = PdfFileWriter()

# Get reach page and add it to corresponding
# output file based on page number
for page in range(pdf.getNumPages()):
 current_page = pdf.getPage(page)
 if page % 2 == 0:
 pdf_writer_odd.addPage(current_page)
 else:
 pdf_writer_even.addPage(current_page)

# Write the data to disk
with open(output_filename_even, &quot;wb&quot;) as out:
 pdf_writer_even.write(out)
 print(&quot;created&quot;, output_filename_even)

# Write the data to disk
with open(output_filename_odd, &quot;wb&quot;) as out:
 pdf_writer_odd.write(out)
 print(&quot;created&quot;, output_filename_odd)
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Re-writing and re-arranging the structure of a PDF is fairly easy with the libraries<code> pdfrw</code>, <code>PyMuPDF</code>, and <code>PyPDF2</code>. With just a few lines of Python code you can delete pages, separate them, and add new content.

Frank Hofmann

Working with PDFs in Python: Inserting, Deleting, and Reordering Pages

<h2 id="introduction">Introduction</h2>
In Python, a module is a self-contained file with Python statements and definitions. For example, <code>file.py</code>, can be considered a module named <code>file</code>. This differs from a package in that a package is a collection of modules in directories that give structure and hierarchy to the modules.
Modules help us break down large programs into small files that are more manageable. With modules, code reusability becomes a reality. Suppose we have a function that is frequently used in different programs. We can define this function in a module then import it into the various programs without having to copy its code each time.
In this article, we will see how to create Python modules and how to use them in Python code.
<h2 id="writingmodules">Writing Modules</h2>
A module is simply a Python file with the <code>.py</code> extension. The name of the file becomes the module name. Inside the file, we can have definitions and implementations of classes, variables, or functions. These can then be used in other Python programs.
Let us begin by creating a function that simply prints &quot;Hello World&quot;. To do this, create a new Python file and save it as <code>hello.py</code>. Add the following code to the file:
<pre><code class="hljs">def my_function():
 print(&quot;Hello World&quot;)
</code></pre>
If you run the above code, it will return nothing. This is because we have not told the program to do anything. It is true that we have created a function named <code>my_function()</code> within the code, but we have not called or invoked the function. When invoked, this function should print the text &quot;Hello World&quot;.
Now, move to the same directory where you have saved the above file and create a new file named <code>main.py</code>. Add the following code to the file:
<pre><code class="hljs">import hello

hello.my_function()
</code></pre>
Output
<pre><code class="hljs">Hello World
</code></pre>
The function was invoked successfully. We began by importing the module. The name of the file was <code>hello.py</code>, hence the name of the imported module is <code>hello</code>.
Also, note the syntax that we have used to invoke the function. This is called the &quot;dot notation&quot;, which allows us to call the function by first specifying the module name, and then the name of the function.
However, that is just one way of importing the module and invoking the function. We could have done it as follows:
<pre><code class="hljs">from hello import my_function

my_function()
</code></pre>
Output
<pre><code class="hljs">Hello World
</code></pre>
In the above example, the first line commands the Python interpreter to import a function named <code>my_function</code> from a module named <code>hello</code>. In such a case, you don't have to use the dot notation to access the function, you can just call it directly.
However, in the case where our <code>hello</code> module has multiple functions, the statement <code>from hello import my_function</code> will not import all <code>hello</code>'s functions into our program, only <code>my_function</code>. If you attempt to access any other function, an error will be generated. You have to import the whole module or import each individual function in order to use them.
We can define a variable within a module, which can then be used by other modules. To demonstrate this, open the file <code>hello.py</code> and add the following code to it:
<pre><code class="hljs">def my_function():
 print(&quot;Hello World&quot;)

# The variable that&#x27;ll be used in other modules
name = &quot;Nicholas&quot;
</code></pre>
Now, open the <code>main.py</code> file and modify it as follows:
<pre><code class="hljs">import hello

hello.my_function()

print(hello.name)
</code></pre>
Output
<pre><code class="hljs">Hello World
Nicholas
</code></pre>
We have successfully invoked both the function and the variable defined in the module since we imported the whole module instead of just the <code>my_function()</code> function.
We stated earlier that we can define a class within a module. Let's see how to do this in the next example. Open the <code>hello.py</code> file and modify it as follows:
<pre><code class="hljs">def my_function():
 print(&quot;Hello World&quot;)

# Defining our variable
name = &quot;Nicholas&quot;

# Defining a class
class Student:
 def __init__(self, name, course):
 self.course = course
 self.name = name

 def get_student_details(self):
 print(&quot;Your name is &quot; + self.name + &quot;.&quot;)
 print(&quot;You are studying &quot; + self.course)
</code></pre>
Here we have defined a class named <code>Student</code>. Two variables have been defined in this class, <code>name</code> and <code>course</code>. The method <code>get_student_details()</code> has also been defined within this, which prints the student details to the console.
Now, open the file <code>main.py</code> and modify it as follows:
<pre><code class="hljs">import hello

hello.my_function()

print(hello.name)

nicholas = hello.Student(&quot;Nicholas&quot;, &quot;Computer Science&quot;)
nicholas.get_student_details()
</code></pre>
Output
<pre><code class="hljs">Hello World
Nicholas
Your name is Nicholas.
You are studying Computer Science
</code></pre>
In the script above, we again used the dot notation to create an object of the <code>student</code> class from the <code>hello</code> module. We then used the <code>get_student_details()</code> function to get the student details.
Although modules mostly consist of class definitions (in most cases), it is possible for them to actually run their own code as well when imported. To demonstrate this, let us modify the <code>hello.py</code> file, where we have a definition of the function <code>my_function()</code>, along with the call to the function:
<pre><code class="hljs">def my_function():
 print(&quot;Hello World&quot;)
 
my_function()
</code></pre>
Now, open the file <code>main.py</code> and delete all the lines except the following:
<pre><code class="hljs">import hello
</code></pre>
Output
<pre><code class="hljs">Hello World
</code></pre>
The above output shows that we defined and called the function within the module. When the module is imported, it directly returns the result from the function without having to invoke the function. This behavior isn't always desired, but it's helpful for certain use-cases, like preloading data from cache when the module is imported.
<h2 id="importingallmoduleobjects">Importing all Module Objects</h2>
To import all objects (functions, variables, classes, etc.) from a module, we can use the <code>import *</code> statement. For example, to import all objects contained in the <code>hello</code> module, we can use the following statement:
<pre><code class="hljs">from hello import *
</code></pre>
After adding the above statement to a program, we will be able to use any function, variable, or class contained in the <code>hello</code> module without having to prefix it with <code>hello</code>.
<h2 id="accessingamodulefromanotherpath">Accessing a Module from Another Path</h2>
In Python, modules are used in more than one project. Hence, it makes no sense if you keep your modules in the directory of one of the projects, since other projects wouldn't be able to use it as easily.
You have a couple of options whenever you need to access a module that is not stored in the same directory as your program. Let us discuss these in the next few sections:
<h3 id="appendingpaths">Appending Paths</h3>
To import a module from another path, you first need to import the <code>sys</code> module as well as any other Python modules that you would like to use in your program.
The <code>sys</code> module is provided by the Python Standard Library and it provides functions and parameters that are system-specific. The <code>path.append()</code> function from the <code>sys</code> module can be used to add the path of the module to the current project.
To demonstrate this, cut the <code>hello.py</code> file from the directory where you have the file <code>main.py</code>. Paste it in another directory. In my case, I have pasted it in the directory &quot;F:\Python.&quot;
Now, open the file <code>main.py</code>, import the <code>sys</code> module and specify the path in which the Python interpreter will look for files. This is demonstrated below:
<pre><code class="hljs">import sys
sys.path.append(&#x27;F:/Python/&#x27;)

import hello
</code></pre>
Output
<pre><code class="hljs">Hello World
</code></pre>
In the above script, the line <code>sys.path.append('F:/Python/')</code> tells the Python interpreter to include this path in the list of paths that will be searched while importing the modules.
<h3 id="addingamoduletopythonpath">Adding a Module to Python Path</h3>
The above method works only if you import the <code>sys</code> module. If you don't import the <code>sys</code> module and specify the path to the module, an error will be generated. To make the module available to the entire system, you can add it to the path where Python normally checks for modules and packages. This way, you will not have to import the <code>sys</code> module and specify the path to the module as we have done in the previous section.
Before doing anything else, you should first identify the path that Python searches for modules and packages. Just open the command line of your operating system and run the <code>python</code> command. This will take you to the Python terminal.
Import the <code>sys</code> module as follows:
<pre><code class="hljs">import sys
</code></pre>
You can then run the following command to print out the path:
<pre><code class="hljs">print(sys.path)
</code></pre>
The output will contain at least one system path. If you do it from a programming environment, you will get several paths. In my case, I got the following:
<pre><code class="hljs">$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
&gt;&gt;&gt; import sys
&gt;&gt;&gt; print(sys.path)
[&#x27;&#x27;, &#x27;/Library/Python/2.7/site-packages/six-1.10.0-py2.7.egg&#x27;, &#x27;/Library/Python/2.7/site-packages/cffi-1.2.1-py2.7-macosx-10.9-intel.egg&#x27;, &#x27;/Library/Python/2.7/site-packages/pycparser-2.14-py2.7.egg&#x27;, &#x27;/Library/Python/2.7/site-packages/virtualenv-13.1.2-py2.7.egg&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old&#x27;, &#x27;/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload&#x27;, &#x27;/Library/Python/2.7/site-packages&#x27;]
&gt;&gt;&gt; 
</code></pre>
Your goal should be to find the one in the environment that you are currently using. You should look for something like the following:
<pre><code class="hljs">/Library/Python/2.7/site-packages
</code></pre>
Move your <code>hello.py</code> file to the path. After that, you will be able to import the <code>hello</code> module from any directory in the usual way, as shown below:
<pre><code class="hljs">import hello
</code></pre>
Output
<pre><code class="hljs">Hello World
</code></pre>
<h2 id="conclusion">Conclusion</h2>
This marks the end of this article. A module is simply a Python file with a set of variables and function definitions. A module facilitates code reusability since you can define a function in a module and invoke it from different programs instead of having to define the function in every program. Although a module is mostly used for function and class definitions, it may also export variables and class instances.

Nicholas Samuel

Creating and Importing Modules in Python

Predicting Customer Ad Clicks via Machine Learning

In this article, we'll examine how to strip characters from both ends of a string in Python.
The built-in <code>String</code> type is an essential Python structure, and comes with a built-in set of methods to simplify working with text data. There are many situations in which a programmer may want to remove unwanted characters, i.e. strip certain characters, from either the beginning or ending of a string.
The most common requirement is to strip whitespace (spaces, tabs, newline characters, etc.) from both ends of a string. This usually occurs after importing raw text data from a file, database, web service, or after accepting user input, which may contain typos in the form of extra spaces. This can be handled by the default usage of the <code>String.strip()</code> method, as seen here:
<pre><code class="hljs">&gt;&gt;&gt; orig_text = &#x27; The cow jumped over the moon! \n&#x27;
&gt;&gt;&gt; print(orig_text.strip())
The cow jumped over the moon!
&gt;&gt;&gt;
</code></pre>
Note that this method does not change the original value of the string, i.e. it does not change the string in-place. It simply returns a new string with the whitespace on either end stripped out. We can verify this by printing out the original string:
<pre><code class="hljs">&gt;&gt;&gt; print(orig_text)
 The cow jumped over the moon! 

&gt;&gt;&gt;
</code></pre>
The <code>strip</code> method also enables us to specify which types of characters we want to strip. This can be useful if we want to remove other characters besides whitespace. To do this we simply specify the characters to strip by passing an argument containing these characters to the <code>String.strip()</code> method:
<pre><code class="hljs">&gt;&gt;&gt; orig_text = &#x27;-----The cow jumped over the moon!$$$$$&#x27;
&gt;&gt;&gt; print(orig_text.strip(&#x27;-$&#x27;))
The cow jumped over the moon!
&gt;&gt;&gt;
</code></pre>
This is useful for removing characters at the start or end of a string that were used for formatting purposes, for example. So if you have a Markdown-formatted string, you can easily remove the header syntax like this:
<pre><code class="hljs">&gt;&gt;&gt; md_text = &#x27;### My Header Here&#x27; # Denotes an H3 header in Markdown
&gt;&gt;&gt; print(md_text.strip(&#x27;# &#x27;))
My Header Here
&gt;&gt;&gt;
</code></pre>
Finally, Python provides a way to strip characters from only one side of the string via the <code>String.rstrip()</code> and <code>String.lstrip()</code> methods. These methods work exactly the same way as the <code>String.strip()</code> method, but <code>String.rstrip()</code> only removes characters from the right side of the string and <code>String.lstrip()</code> only removes characters from the left side of the string:
<pre><code class="hljs">&gt;&gt;&gt; orig_text = &#x27;*****The cow jumped over the moon!*****&#x27;
&gt;&gt;&gt; print(orig_text.rstrip(&#x27;*&#x27;))
*****The cow jumped over the moon!
&gt;&gt;&gt; print(orig_text.lstrip(&#x27;*&#x27;))
The cow jumped over the moon!*****
</code></pre>
Once again we can print the original string to see that it was unaffected by these operations:
<pre><code class="hljs">&gt;&gt;&gt; print(orig_text)
*****The cow jumped over the moon!*****
</code></pre>
<h2 id="abouttheauthor">About the Author</h2>
This article was written by Jacob Stopak, a software consultant and developer with passion for helping others improve their lives through code. Jacob is the creator of <a rel="nofollow" target="_blank" href="https://initialcommit.com">Initial Commit</a> - a site dedicated to helping curious developers learn how their favorite programs are coded. Its featured project helps people <a rel="nofollow" target="_blank" href="https://initialcommit.com/blog/Learn-Git-Code-Definitive-Guide">learn Git</a> at the code level.

Jacob Stopak

The Python String strip() Function

In this article, we'll examine how to use the built-in Python <code>zip()</code> function.
The <code>zip()</code> function is a Python built-in function that allows us to combine corresponding elements from multiple sequences into a single list of tuples. The sequences are the arguments accepted by the <code>zip()</code> function. Any number of sequences can be supplied, but the most common use-case is to combine corresponding elements in two sequences.
For example, let's say we have the two lists below:
<pre><code class="hljs">&gt;&gt;&gt; vehicles = [&#x27;unicycle&#x27;, &#x27;motorcycle&#x27;, &#x27;plane&#x27;, &#x27;car&#x27;, &#x27;truck&#x27;]
&gt;&gt;&gt; wheels = [1, 2, 3, 4, 18]
</code></pre>
We can use the <code>zip()</code> function to associate elements from these two lists based on their order:
<pre><code class="hljs">&gt;&gt;&gt; list(zip(vehicles, wheels))
[(&#x27;unicycle&#x27;, 1), (&#x27;motorcycle&#x27;, 2), (&#x27;plane&#x27;, 3), (&#x27;car&#x27;, 4), (&#x27;truck&#x27;, 18)]
</code></pre>
Notice how the output is a sequence of tuples, where each tuple combines elements of the input sequences with corresponding indexes.
One important thing to note is that if the input sequences are of differing lengths, <code>zip()</code> will only match elements until the end of the shortest list is reached. For example:
<pre><code class="hljs">&gt;&gt;&gt; vehicles = [&#x27;unicycle&#x27;, &#x27;motorcycle&#x27;, &#x27;plane&#x27;, &#x27;car&#x27;, &#x27;truck&#x27;]
&gt;&gt;&gt; wheels = [1, 2, 3]
&gt;&gt;&gt; list(zip(vehicles, wheels))
[(&#x27;unicycle&#x27;, 1), (&#x27;motorcycle&#x27;, 2), (&#x27;plane&#x27;, 3)]
</code></pre>
Since the <code>wheels</code> list is shorter in this example (3 items as opposed to the 5 that <code>vehicles</code> has), the sequence stopped at &quot;plane&quot;.
As previously mentioned, the <code>zip()</code> function can be used with more than two sequences:
<pre><code class="hljs">&gt;&gt;&gt; vehicles = [&#x27;unicycle&#x27;, &#x27;motorcycle&#x27;, &#x27;plane&#x27;, &#x27;car&#x27;, &#x27;truck&#x27;]
&gt;&gt;&gt; wheels = [1, 2, 3, 4, 18]
&gt;&gt;&gt; energy_sources = [&#x27;pedal&#x27;, &#x27;gasoline&#x27;, &#x27;jet fuel&#x27;, &#x27;gasoline&#x27;, &#x27;diesel&#x27;]
</code></pre>
<pre><code class="hljs">&gt;&gt;&gt; list(zip(vehicles, wheels, energy_sources))
[(&#x27;unicycle&#x27;, 1, &#x27;pedal&#x27;), (&#x27;motorcycle&#x27;, 2, &#x27;gasoline&#x27;), (&#x27;plane&#x27;, 3, &#x27;jet fuel&#x27;), (&#x27;car&#x27;, 4, &#x27;gasoline&#x27;), (&#x27;truck&#x27;, 18, &#x27;diesel&#x27;)]
</code></pre>
One reason to connect multiple sequences like this is to create a cleaner way to iterate over the items in multiple sequences. Without the <code>zip()</code> function, we would have to do something like this:
<pre><code class="hljs">&gt;&gt;&gt; for i in range(len(vehicles)):
... print(&#x27;A &#x27; + vehicles[i] + &#x27; has &#x27; + str(wheels[i]) + &#x27; wheels and runs on &#x27; + energy_sources[i])
...
A unicycle has 1 wheels and runs on pedal
A motorcycle has 2 wheels and runs on gasoline
A plane has 3 wheels and runs on jet fuel
A car has 4 wheels and runs on gasoline
A truck has 18 wheels and runs on diesel
</code></pre>
But with the <code>zip()</code> function we can use the following cleaner syntax via tuple unpacking:
<pre><code class="hljs">&gt;&gt;&gt; for v, w, es in zip(vehicles, wheels, energy_sources):
... print(&#x27;A &#x27; + v + &#x27; has &#x27; + str(w) + &#x27; wheels and runs on &#x27; + es)
...
A unicycle has 1 wheels and runs on pedal
A motorcycle has 2 wheels and runs on gasoline
A plane has 3 wheels and runs on jet fuel
A car has 4 wheels and runs on gasoline
A truck has 18 wheels and runs on diesel
</code></pre>
One final thing to understand about the <code>zip()</code> function is that it actually returns an iterator, not a list of tuples. Note that in our first two examples above, we wrapped the <code>zip()</code> function inside the <code>list()</code> type to convert the result to a list. If we tried to display the return value of the <code>zip()</code> function directly we would see something like this:
<pre><code class="hljs">&gt;&gt;&gt; zip(vehicles, wheels)
&lt;zip object at 0x1032caf48&gt;
</code></pre>
This 'zip object' is an iterable instance of the <code>Zip</code> class, which means it will return its contents one by one in a for-loop, instead of all at once, the way a list does. This is more efficient for large sequences that would be very memory intensive if accessed all at once.
<h2 id="abouttheauthor">About the Author</h2>
This article was written by Jacob Stopak, a software consultant and developer with passion for helping others improve their lives through code. Jacob is the creator of <a rel="nofollow" target="_blank" href="https://initialcommit.com">Initial Commit</a> - a site dedicated to helping curious developers learn how their favorite programs are coded. Its featured project helps people <a rel="nofollow" target="_blank" href="https://initialcommit.com/blog/Learn-Git-Code-Definitive-Guide">learn Git</a> at the code level.

The Python zip() Function

<h2 id="overview">Overview</h2>
In this article, we'll introduce you to <a rel="nofollow" target="_blank" href="https://github.com/spring-cloud/spring-cloud-netflix/tree/master/spring-cloud-netflix-turbine">Spring Cloud Netflix Turbine</a>. It aggregates multiple Hystrix Metrics Streams into one, so that it could be displayed into a single dashboard view.
To give a small introduction to <a rel="nofollow" target="_blank" href="https://github.com/Netflix/Hystrix">Hystrix</a>. In a microservice architecture, we have many small applications that talk to each other to complete a request.
There is always a possibility that one of these downstream services won't respond correctly or simply fails completely. To prevent any cascading failures we set up a Hystrix fallback mechanism for our microservices.
Each microservice that implements Hystrix can choose to expose the Hystrix Metrics Streams (via the actuator endpoint <code>/hystrix.stream</code>) that can be viewed via the Hystrix Dashboard.
We've covered this in detail in <a target="_blank" href="/spring-cloud-hystrix">Spring Cloud: Hystrix</a> if you want to learn more.
<a rel="nofollow" target="_blank" href="https://github.com/Netflix/Turbine/wiki">Turbine</a> is an open-source tool from Netflix for aggregating multiple streams into a single stream. Spring provided a nice wrapper around it to be easily used in the Spring ecosystem.
<h2 id="setup">Setup</h2>
The setup is similar to the Spring Cloud: Hystrix setup. Here's what our back-end service looks like:
<ul>
<li>Eureka Server: Acts as a service registry and runs on port <code>8761</code>.</li>
<li>Recommendation Service: A simple REST service that has a single endpoint of <code>/recommendations</code> and runs on port <code>8070</code>.</li>
<li>User Service: A simple REST service that has a single endpoint of <code>/personalized/{id}</code> and runs on port <code>8060</code>.</li>
<li>Hystrix Turbine: A Hystrix dashboard service to display Hystrix streams running on port <code>9090</code>.</li>
</ul>
Here is what our services look like on the <a target="_blank" href="/spring-cloud-service-discovery-with-eureka/">Eureka server</a>:
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-1.png" alt="spring-cloud-turbine-setup">
Both the <code>user-service</code> and <code>recommendation-service</code> have implemented the Hystrix fallback mechanism and have the <code>/hystrix.stream</code> endpoint exposed via <a rel="nofollow" target="_blank" href="https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html">Actuator</a>:
<ul>
<li>Hystrix endpoint for user-service: <code>http://localhost:8060/actuator/hystrix.stream</code></li>
<li>Hystrix endpoint for recommendation-service: <code>http://localhost:8070/actuator/hystrix.stream</code></li>
</ul>
We can check both this individually in our Hystrix Dashboard by typing the URL in the box and clicking &quot;Monitor Stream&quot;:
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-2.png" alt="spring-cloud-turbine-dashboard">
We'll see a metric like this:
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-3.png" alt="spring-cloud-turbine-dashboard-single-stream">
Note: If you don't see any stream, then you probably have to hit the endpoints of the service whose stream you want to see. Ex: for<code>user-service</code> we have to hit <code>http://localhost:8060/personalized/1</code> to generate the stream.
<h2 id="installingturbine">Installing Turbine</h2>
You might have realized that looking at the individual stream is not very productive, especially when we have many microservices.
Turbine can aggregate all these individual <code>hystrix.stream</code>s to a single <code>turbine.stream</code>, which can be viewed on the Hystrix Dashboard.
It uses the <code>DiscoveryClient</code> interface to find out relevant services that produce <code>/hystrix.stream</code>s.
To add Turbine to your Hystrix Dashboard, add the following dependency:
<pre><code class="hljs">&lt;dependency&gt;
 &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
 &lt;artifactId&gt;spring-cloud-starter-netflix-turbine&lt;/artifactId&gt;
&lt;/dependency&gt;
</code></pre>
Note: This is a starter dependency of turbine, which by default uses Spring Cloud Eureka as the discovery server. If you are using Spring Cloud Consul, use the following dependencies:
<pre><code class="hljs">&lt;dependency&gt;
 &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
 &lt;artifactId&gt;spring-cloud-netflix-turbine&lt;/artifactId&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
 &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
 &lt;artifactId&gt;spring-cloud-starter-consul-discovery&lt;/artifactId&gt;
&lt;/dependency&gt;
</code></pre>
In this article we'll be using the starter dependency, i.e <code>spring-cloud-starter-netflix-turbine</code>.
To enable Turbine, we simply annotate our main class with <code>@EnableTurbine</code>:
<pre><code class="hljs">@SpringBootApplication
@EnableTurbine
@EnableDiscoveryClient
@EnableHystrixDashboard
public class HystrixTurbineApplication {

 public static void main(String[] args) {
 SpringApplication.run(HystrixTurbineApplication.class, args);
 }
}
</code></pre>
For Turbine to work as expected we have to add few details to our <code>application.properties</code>:
<pre><code class="hljs">server.port= 9090
spring.application.name= hystrix-turbine
eureka.client.serviceUrl.defaultZone= http://localhost:8761/eureka/
turbine.appConfig= user-service,recommendation-service
turbine.clusterNameExpression= new String(&quot;default&quot;)
</code></pre>
Here, we are telling Turbine the Eureka server location, applications whose <code>/hystrix.stream</code> it needs to fetch and <code>turbine.clusterNameExpression</code> as <code>new String(&quot;default&quot;)</code>, which gives the cluster name of &quot;default&quot;.
We can open up <code>http://localhost:9090/turbine.stream?cluster=default</code> to see the aggregate stream of both <code>user-service</code> and <code>recommendation-service</code>:
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-4.png" alt="spring-cloud-turbine-stream">
Again, if you are not viewing anything, just hit the <code>user-service</code> and <code>recommendation-service</code> endpoints to generate the streams.
We can also use this URL on our Hystrix dashboard to generate a nice aggregated view:
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-5.png" alt="spring-cloud-turbine-stream-in-dashboard">
Sometimes, you may want to use Eureka's <code>serviceId</code>s as cluster names for your dashboard. This can be done by using <code>turbine.aggregator.clusterConfig</code>:
<pre><code class="hljs">server.port= 9090
spring.application.name= hystrix-turbine
eureka.client.serviceUrl.defaultZone= http://localhost:8761/eureka/
turbine.aggregator.clusterConfig= USER-SERVICE,RECOMMENDATION-SERVICE
turbine.appConfig= user-service,recommendation-service
</code></pre>
You can also check what clusters are currently configured in your Turbine application by hitting the <code>/clusters</code> endpoint.
This endpoint can be disabled by setting <code>turbine.endpoints.clusters.enabled</code> to <code>false</code>.
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-6.png" alt="spring-cloud-turbine-cluster-endpoint">
So, now we can view <code>turbine.stream</code> as Eureka IDs such as: <code>http://localhost:9090/turbine.stream?cluster=USER-SERVICE</code>
<img src="https://s3.amazonaws.com/stackabuse/media/spring-cloud-turbine-7.png" alt="spring-cloud-turbine-service-id-cluster">
If there are multiple instances of a particular service running, Turbine will pick it up as per cluster and display it in the result.
<h2 id="conclusion">Conclusion</h2>
In this article, we've covered how to set up the Turbine on top of our Hystrix stream for an aggregated view. We first saw the classical Turbine approach of fetching Hystrix stream from all services.
As always, the code for the examples used in this article can be found on <a rel="nofollow" target="_blank" href="https://github.com/StackAbuse/spring-cloud/tree/master/spring-turbine">GitHub</a>.