Shadab Ansari

Web Browser Automation with Selenium and Java

<h2 id="introduction">Introduction</h2>
In this tutorial, we are going to learn how to use <a rel="nofollow" target="_blank" href="https://opencv.org">OpenCV</a> library in Python. OpenCV is an open source library which is supported by multiple platforms including Windows, Linux, and MacOS, and is available for use in multiple other languages as well; however, it is most commonly used in Python for Machine Learning applications, specifically in the Computer Vision domain.
Apart from its cross-platform support and availability in multiple other computer languages, which allows applications developed in it to be used on different systems, OpenCV is also, in comparison to other such libraries, fairly efficient in terms of computations, as it uses vector operations for most of its functions.
In this tutorial, we'll cover OpenCV installation on Mac, Windows, and Linux, image operations, image arithmetics, image smoothing, and geometric transformations using OpenCV. So without further ado, let's start.
<h2 id="installation">Installation</h2>
Note: Since we are going to use OpenCV in the Python language, it is an implicit requirement that you already have Python (version 3) installed on your workstation. Depending upon your OS, execute one of the following commands to install OpenCV library on your system:
Windows
<pre><code class="hljs">$ pip install opencv-python
</code></pre>
MacOS
<pre><code class="hljs">$ brew install opencv3 --with-contrib --with-python3
</code></pre>
Linux
<pre><code class="hljs">$ sudo apt-get install libopencv-dev python-opencv
</code></pre>
To check if your installation was successful or not, run the following command in either a Python shell, or your command prompt/ terminal:
<pre><code class="hljs">import cv2
</code></pre>
If you do not get an error on importing <code>cv2</code> then it was installed correctly.
<h2 id="basicimageoperations">Basic Image Operations</h2>
Now that we have installed OpenCV on our workstations, let's get our hands dirty with some of the functionalities that OpenCV offers.
<h3 id="displayanimage">Display an Image</h3>
Displaying an image using OpenCV is a two-step process; first, we have to load it, and then we can display it. Both operations are done in sequence using different functions.
To display an image, we need to know two things:
<ol>
<li>Image Path (both absolute and relative paths work)</li>
<li>Read Mode (read, write, etc.)</li>
</ol>
The function we'll use for reading/loading an image is <code>cv2.imread()</code>, which has two variations. First one is <code>IMREAD_GRAYSCALE</code>, which as the name suggests, converts the image to grayscale before reading it. The second one is <code>IMREAD_UNCHANGED</code>, which loads the image without cutting out the alpha channel. The default is <code>IMREAD_COLOR</code>, which simply reads the colored image using the RGB channels only.
Let's code an example:
<pre><code class="hljs">import cv2

my_bike = cv2.imread(&#x27;bike.png&#x27;)
</code></pre>
This will load the image of a bike from the file system and store it in the <code>my_bike</code> variable for further operations
Note: If you get an error from the above code, there are only three possible reasons for it. The first one is that the path you specified is incorrect, second is that the image file you specified doesn't exist, and the last one is that the image type (jpg/jpeg/png) in the image path is incorrect.
Let's now display the image we just read. It can be done by the <code>cv2.imshow()</code> function. If you have used Matlab, you may be familiar with these image operations.
<pre><code class="hljs">cv2.imshow(&#x27;my_bike&#x27;, my_bike)
</code></pre>
The first parameter to the <code>imshow()</code> function is the string name that you want to display on the image window. The second parameter is the image handler we created using the <code>cv2.imread()</code> function.
<h3 id="savinganimage">Saving an Image</h3>
Saving an image is a very commonly used feature, as we may need to update our image and save the changes to the file system for later use. OpenCV has an <code>cv2.imwrite()</code> function to save images.
Here is an example:
<pre><code class="hljs">cv2.imwrite(&#x27;bike.png&#x27;, my_bike)
</code></pre>
Here we specify the name and current location of the image. The resulting image is automatically saved in the current working directory.
<h2 id="arithmeticoperationsonimages">Arithmetic Operations on Images</h2>
Arithmetic operations on images refer to adding, subtracting, multiplying, or dividing multiple images to generate a new image which is an arithmetic combination of the input images. Image arithmetics has a lot of applications, like adding a watermark to an image, creating a blended combination of two images, applying different types of image filters, etc.
While there are many operations you can perform, we will only be showing two examples here, as this will then allow you to apply the concept to other arithmetic operations available in OpenCV. The first example will be the addition of two images, and the second example will be blending two images.
Let's code these two examples:
<h3 id="addingimages">Adding Images</h3>
<pre><code class="hljs">import cv2

# Read in the two images
image_1 = cv2.imread(&#x27;bike.jpg&#x27;)
image_2 = cv2.imread(&#x27;car.jpg&#x27;)

# Sum the two image arrays for all channels
result = cv2.add(image_1, image_2)

cv2.imshow(&#x27;result&#x27;, result)
cv2.waitKey(0)
cv2.destroyAllWindows()
</code></pre>
The <code>waitKey</code> command will wait for you to press a key before it moves on to the next command. This is useful so that the program will continue to display your image until a key is pressed, otherwise it will be displayed for a split second and then quickly disappear once the program has stopped executing.
<h3 id="blendingimages">Blending Images</h3>
Blending images is similar to image addition, except each image's contribution to the new resulting image can be controlled. Basically, if we want one image to be more focused, and the other one to be more faint when they get merged, we will go with blending, instead of simple addition.
Lets code it to clarify further:
<pre><code class="hljs">import cv2

# Read in the two images
image_1 = cv2.imread(&#x27;bike.jpg&#x27;)
image_2 = cv2.imread(&#x27;car.jpg&#x27;)

result = cv2.addWeighted(image_1, 0.9, image_2, 0.1)

cv2.imshow(&#x27;result&#x27;, result)
cv2.waitKey(0) # Wait for the user to press a key before continuing
cv2.destroyAllWindows()
</code></pre>
The sum of the weights given to the <code>addWeighted</code> function should be equal to 1.0. You can also give a scalar value at the end, which would be added to all the pixel values of the resultant image.
Note: The images can be of any type; however, the type should be the same for all images. For instance, if you are using PNG format, all images being used for computation should be in PNG format as well.
<h2 id="imagesmoothing">Image Smoothing</h2>
Image smoothing is a very helpful feature, which is mostly performed before the images are passed on to a machine learning model. It is mostly done to remove noise/high-frequency elements from images by passing the image through a low-pass filter. There are many filters, including box filter (averaging filter), median filter, mode filter, Gaussian filter, and many more; however, to understand image smoothing and how to do it using OpenCV, we will only cover the box filter.
Let's say you have an image of 10x10, and you want to pass it through a 3x3 box/averaging filter, how would you do it?
You'll start with the top left of the image, place your 3x3 filter there, and replace the central element with the average of all 9 elements. This was the first step, now you will move your filter one step to the right, and repeat the same process until you have covered the whole image. An example of 10x10 image, and 3x3 averaging filter are shown below for your reference:
Filter/Mask:
<img class="img-responsive" src="https://stackabuse.s3.amazonaws.com/media/introduction-opencv-python-1.png">
Filter being applied on 10x10 Image:
<img class="img-responsive" src="https://stackabuse.s3.amazonaws.com/media/introduction-opencv-python-2.png">
Now that we have discussed how it works, lets try and see how we can apply different filters on our image using OpenCV; please read the comments thoroughly to know which line of code is used for which filter:
<pre><code class="hljs">import cv2

# Load the original image
original_image = cv2.imread(&#x27;my_bike.png&#x27;)

# Filter by passing image through 3x3 averaging filter
average_image = cv2.blur(original_image,(3,3))

# Apply 3x3 gaussian filter on the original image
gaussian_image = cv2.GaussianBlur((original_image,(3,3),0))

# Apply 3x3 median filter on the original image
median_image = cv2.medianBlur(original_image,3)
</code></pre>
Note: You can view the resulting images by using the following additional code:
<pre><code class="hljs">import matplotlib.pyplot as plt

plt.imshow(average_image)
plt.show()
</code></pre>
<h2 id="imagetransformations">Image Transformations</h2>
Image transformation is the last, but one of the most important topics that we are going to cover with OpenCV. It has a lot of applications, but one of the most common ones nowadays is in Machine Learning for Data Augmentation, i.e. when you have a shortage of dataset, you augment/transform the currently available images to make them different from your existing dataset. This effectively increases your dataset size and might help in improving your model accuracy.
The list of possible transformations is a long one, including scaling, <a target="_blank" href="/affine-image-transformations-in-python-with-numpy-pillow-and-opencv/">affine</a>, rotation, translation, etc. We will only cover two of them using OpenCV to get a general idea; however, OpenCV provides supporting functions for a wide range of them. Let's start with scaling.
<h3 id="scaling">Scaling</h3>
To put it in simple words, scaling is basically just resizing your image, i.e. either making it bigger or smaller. <code>resize</code> is the function used for scaling the images in OpenCV. Resizing has three types: <code>INTER_CUBIC</code>, <code>INTER_LINEAR</code>, and <code>INTER_AREA</code>. Let's code an example using these functions for scaling; please read through the code, comments, and descriptions carefully as they will explain what exactly is going on in the code:
<pre><code class="hljs">import cv2
import numpy as np
import matplotlib.pyplot as plt

image = cv2.imread(&#x27;my_bike.jpg&#x27;)

# Scale up/expand both width and height by factor of 2
result_1 = cv2.resize(image, None, fx=2, fy=2, interpolation=cv2.INTER_CUBIC)

# Scale down/shrink both width and height by factor of 2
result_2 = cv2.resize(image, None, fx=2, fy=2, interpolation=cv2.INTER_AREA)

# Display the resulting images
plt.imshow(result_1)
plt.imshow(result_2)
plt.show()
</code></pre>
Here in the <code>resize</code> function, the <code>fx</code> parameter in represents the scale factor for width, <code>fy</code> represents the scale factor height, and <code>interpolation</code> specifies the function to be used for scaling (shrinking or expansion).
<h3 id="rotation">Rotation</h3>
Rotation allows us to move an image about the axis for a certain specified angle.
Before we learn how to rotate our images using code, we should know that there is a <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Rotation_matrix">rotation matrix</a> that is used for performing this transformation; we will not go in details of that, as OpenCV makes it very simple for us to calculate that matrix using a single function call. You will see that in the code below:
<pre><code class="hljs">import cv2
import matplotlib.pyplot as plt

# Load the image of a bike
image = cv2.imread(&#x27;my_bike.jpg&#x27;,0)

# Rows and columns
r, c = image.shape

matrix = cv2.getRotationMatrix2D((cols/2,rows/2), 180, 1)
result = cv2.warpAffine(image,matrix,(c,r))

# Display resulting rotation
plt.imshow(result)
plt.show()
</code></pre>
In the <code>getRotationMatrix2D</code> function, 180 specifies the degree by which the image should be rotated, 1 is the scaling factor, the function call would return the rotation matrix in the <code>matrix</code> variable.
The <code>warpAffine</code> function call uses the matrix we calculated from the previous method to rotate the image according to our specifications.
<h2 id="conclusion">Conclusion</h2>
To conclude it all, let's reiterate over some important points that we discussed in this article. OpenCV is a library available in multiple languages and is mostly used in conjunction with <a target="_blank" href="/numpy-tutorial-a-simple-example-based-guide/">NumPy</a>, SciPy and <a target="_blank" href="/python-data-visualization-with-matplotlib/">Matplotlib</a>, as we saw in some of the examples above, as well. Some of its functions are the same as in Matlab, and it also supports vectorized operations, hence increasing computational efficiency.
Furthermore, OpenCV is one of the best libraries out there for the Computer Vision domain, and after reading this article, you should be able to go ahead and search up some computer vision / machine learning applications that have been developed using OpenCV.
Let it be known that this article was just the tip of the iceberg, and OpenCV has a lot more to offer. Reading this should enable you to dive deeper and learn about other advanced features that OpenCV has to offer.

Muhammad Junaid Khalid

Introduction to OpenCV with Python

In this article we'll go over how the run-length encoding algorithm works, what it's used for, and how to implement its encode and decode functions in Python.
Run-length encoding (RLE) is a very simple form of data compression in which a stream of data is given as the input (i.e. &quot;AAABBCCCC&quot;) and the output is a sequence of counts of consecutive data values in a row (i.e. &quot;3A2B4C&quot;). This type of data compression is lossless, meaning that when decompressed, all of the original data will be recovered when decoded. Its simplicity in both the encoding (compression) and decoding (decompression) is one of the most attractive features of the algorithm.
Here you can see a simple example of a stream (&quot;run&quot;) of data in its original form and encoded form:
Input data:
<pre><code class="hljs">AAAAAAFDDCCCCCCCAEEEEEEEEEEEEEEEEE
</code></pre>
Output data:
<pre><code class="hljs">6A1F2D7C1A17E
</code></pre>
In this example we were able to compress the data from 34 characters down to 13.
As you may have noticed, the more consecutive values in a row, the more space we save in the resulting compression. On the other hand, if you have a sequence of data that frequently changes between values (i.e. &quot;BEFEFADED&quot;) then we won't save much space at all. In fact, we could even increase the size of our data since a single instance of a character results in 2 characters (i.e. &quot;A&quot; becomes &quot;1A&quot;) in the output of the encoding.
Because of this, RLE is only good for certain types of data and applications. For example, the <a rel="nofollow" target="_blank" href="https://pixycam.com/pixy-cmucam5/">Pixy camera</a>, which is a robotics camera that helps you easily track objects, uses RLE to compress labeled video data before transferring it from the embedded camera device to an external application. Each pixel is given a label of &quot;no object&quot;, &quot;object 1&quot;, &quot;object 2&quot;, etc. This is the perfect encoding for this application because of its simplicity, speed, and ability to compress the low-entropy label data.
<h2 id="encoding">Encoding</h2>
In order to encode a string of data, your code will need to loop through each character of the data and count the occurrences. Once you see a character that is different from the previous character, you will append the number of occurrences and the character to your encoding.
Below you'll find a simple implementation in Python:
<pre><code class="hljs"># rle-encode.py

def rle_encode(data):
 encoding = &#x27;&#x27;
 prev_char = &#x27;&#x27;
 count = 1

 if not data: return &#x27;&#x27;

 for char in data:
 # If the prev and current characters
 # don&#x27;t match...
 if char != prev_char:
 # ...then add the count and character
 # to our encoding
 if prev_char:
 encoding += str(count) + prev_char
 count = 1
 prev_char = char
 else:
 # Or increment our counter
 # if the characters do match
 count += 1
 else:
 # Finish off the encoding
 encoding += str(count) + prev_char
 return encoding
</code></pre>
From the comments you should be able to tell what is going on throughout the code. If not, it would be a good exercise to run through the code with a debugger and see it in action.
Continuing with the same file as above, here is an example of the code being executed:
<pre><code class="hljs">encoded_val = rle_encode(&#x27;AAAAAAFDDCCCCCCCAEEEEEEEEEEEEEEEEE&#x27;)
print(encoded_val)
</code></pre>
And the output:
<pre><code class="hljs">$ python rle-encode.py
6A1F2D7C1A17E
</code></pre>
<h2 id="decoding">Decoding</h2>
Decoding an RLE-encoded stream of data is actually even easier than encoding it. Like before, you iterate through the data stream one character at a time. If you see a numeric character then you add it to your <code>count</code>, and if you see a non-numeric character then you add <code>count</code> of those characters to your decoding, which is returned to the caller once you iterate through all of the input<code>data</code>.
Here is the algorithm implemented in Python:
<pre><code class="hljs"># rle-decode.py

def rle_decode(data):
 decode = &#x27;&#x27;
 count = &#x27;&#x27;
 for char in data:
 # If the character is numerical...
 if char.isdigit():
 # ...append it to our count
 count += char
 else:
 # Otherwise we&#x27;ve seen a non-numerical
 # character and need to expand it for
 # the decoding
 decode += char * int(count)
 count = &#x27;&#x27;
 return decode
</code></pre>
We can run this code on the same output that we got from our encoding:
<pre><code class="hljs">decoded_val = rle_decode(&#x27;6A1F2D7C1A17E&#x27;)
print(decoded_val)
</code></pre>
And the output is the same as our original input to the encoding function:
<pre><code class="hljs">$ python rle-decode.py
AAAAAAFDDCCCCCCCAEEEEEEEEEEEEEEEEE
</code></pre>
Note that this implementation doesn't do any error checking to ensure that we have a valid RLE data stream. If any of the input data isn't formatted properly then you'll likely encounter an error.

Scott Robinson

Run-Length Encoding

In Git, and most other <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Version_control">VCS</a> tools, branching is one of the main constructs that really make it useful for software development. These branches are almost like a new copy of your code at the current state, which can then be used to develop new code.
For example, whenever you need to create a new feature, fix a bug, or rewrite any of your code, it's a good idea to create a new branch so that none of your changes affect the &quot;master&quot; version of the code. This is important since it can be very difficult to revert code changes from memory, especially in complex systems.
There are a few ways you can create new branches in Git, with many of them differing in how your branch is created from the main branch, whether it be from your current branch, a different branch, a tag, etc.
The most common way to create a new branch is the following:
<pre><code class="hljs">$ git checkout -b &lt;branch-name&gt;
</code></pre>
This is most commonly used because it will create the branch for you from your current branch and it will switch you to that branch in a single command.
You can also optionally specify a different branch from which the new one will be created:
<pre><code class="hljs">$ git checkout -b new-branch dev-branch
Switched to branch &#x27;new-branch&#x27;
</code></pre>
Another common way is by using the <code>branch</code> command directly (which <code>checkout</code> does behind the scenes):
<pre><code class="hljs">$ git branch &lt;branch-name&gt;
</code></pre>
However, as you can see from the following example, this doesn't automatically switch us over to the new branch and keeps us on our current one:
<pre><code class="hljs">$ git branch
* master
$ git branch new-branch
$ git branch
* master
 new-branch
</code></pre>
If you want to work on the branch immediately then you'll need to switch to it manually using the <code>checkout</code> command:
<pre><code class="hljs">$ git checkout new-branch
Switched to branch &#x27;new-branch&#x27;
</code></pre>
<h2 id="creatingabranchfromacommit">Creating a Branch from a Commit</h2>
As mentioned above, there are a few other ways you can create new branches. One of those ways is by specifying a specific commit via its hash:
<pre><code class="hljs">$ git branch &lt;branch-name&gt; &lt;hash&gt;
</code></pre>
As always with Git, the entire hash doesn't actually need to be specified, just a few characters.
<pre><code class="hljs">$ git branch
* master
$ git branch commit-branch 735c5b4
$ git branch
 commit-branch
* master
</code></pre>
You can also use the <code>git checkout -b &lt;branch-name&gt; &lt;hash&gt;</code> syntax, which will create the branch and check it out, all in one command.
<h2 id="creatingabranchfromatag">Creating a Branch from a Tag</h2>
Much like creating a branch from a commit, you can also create a branch from a tag. This is especially useful since tags are, in my opinion, a better way to reference a certain point in a project's history.
So if you have created tags throughout your project's history, you can create a new branch just like before, but with a tag as the identifier.
<pre><code class="hljs">$ git branch tag-branch v0.4.12
$ git branch
 tag-branch
* master
</code></pre>
And again, the <code>git checkout -b &lt;branch-name&gt; &lt;tag&gt;</code> syntax can also be used.

Git: Create a New Branch

Building a Todo App with Flask in Python

Dan Nelson

Multiple Linear Regression with Python

Marcus Sanatan

Introduction to Phaser 3: Building Breakout

Vladimir Batoćanin

Mathematical Proof of Algorithm Correctness and Efficiency

<h2 id="introduction">Introduction</h2>
Computing has evolved over time and more and more ways have come up to make computers run even faster. What if instead of executing a single instruction at a time, we can also execute several instructions at the same time? This would mean a significant increase in the performance of a system.
Through concurrency, we can achieve this and our Python programs will be able to handle even more requests at a single time, and over time leading to impressive performance gains.
In this article, we will discuss concurrency in the context of Python programming, the various forms it comes in and we will speed up a simple program in order to see the performance gains in practice.
<h2 id="whatisconcurrency">What is Concurrency?</h2>
When two or more events are concurrent, it means that they are happening at the same time. In real life, concurrency is common since a lot of things happen at the same time all the time. In computing, things are a bit different when it comes to concurrency.
In computing, concurrency is the execution of pieces of work or tasks by a computer at the same time. Normally, a computer executes a piece of work as others wait their turn, once it is completed, the resources are freed and the next piece of work begins execution. This is not the case when concurrency is implemented as the pieces of work to be executed don't always have to wait for others to be completed. They are executed at the same time.
<h2 id="concurrencyvsparallelism">Concurrency vs Parallelism</h2>
We have defined concurrency as the execution of tasks at the same time, but how does it compare to <a target="_blank" href="/parallel-processing-in-python/">parallelism</a>, and what is it?
Parallelism is achieved when multiple computations or operations are carried out at the same time or in parallel with the goal of speeding up the computation process.
Both concurrency and parallelism are involved with performing multiple tasks simultaneously, but what sets them apart is the fact that while concurrency only takes place in one processor, parallelism is achieved through utilizing multiple CPUs to have tasks done in parallel.
<h2 id="threadvsprocessvstask">Thread vs Process vs Task</h2>
While generally speaking, threads, processes, and tasks may refer to pieces or units of work. However, in detail they are not so similar.
A thread is the smallest unit of execution that can be performed on a computer. Threads exist as parts of a process and are usually not independent of each other, meaning they share data and memory with other threads within the same process. Threads are also sometimes referred to as lightweight processes.
For example, in a document processing application, one thread could be responsible for formatting the text and another handles the auto-saving, while another is doing spell checks.
A process is a job or an instance of a computed program that can be executed. When we write and execute code, a process is created to execute all the tasks that we have instructed the computer to do through our code. A process can have a single primary thread or have several threads within it, each with its own stack, registers, and program counter. But they all share the code, data, and memory.
Some of the common differences between processes and threads are:
<ul>
<li>Processes work in isolation while threads can access the data of other threads</li>
<li>If a thread within a process is blocked, other threads can continue executing, while a blocked process will put on hold the execution of the other processes in the queue</li>
<li>While threads share memory with other threads, processes do not and each process has its own memory allocation.</li>
</ul>
A task is simply a set of program instructions that are loaded in memory.
<h2 id="multithreadingvsmultiprocessingvsasyncio">Multithreading vs Multiprocessing vs Asyncio</h2>
Having explored threads and processes, let us now delve deeper into the various ways a computer executes concurrently.
Multithreading refers to the ability of a CPU to execute multiple threads concurrently. The idea here is to divide a process into various threads that can be executed in a parallel manner or at the same time. This division of duty enhances the speed of execution of the entire process. For example, in a word processor like MS Word, a lot of things are going on when in use.
Multithreading will allow the program to auto-save the content being written, perform spell checks for the content, and also format the content. Through multithreading, all this can take place simultaneously and the user does not have to complete the document first for the saving to happen or the spell checks to take place.
Only one processor is involved during multithreading and the operating system decides when to switch tasks in the current processor, these tasks may be external to the current process or program being executed in our processor.
Multiprocessing, on the other hand, involves utilizing two or more processor units on a computer to achieve parallelism. Python implements multiprocessing by creating different processes for different programs, with each having its own instance of the Python interpreter to run and memory allocation to utilize during execution.
<a rel="nofollow" target="_blank" href="https://docs.python.org/3/library/asyncio.html">AsyncIO</a> or asynchronous IO is a new paradigm introduced in Python 3 for the purpose of writing concurrent code by using the <a target="_blank" href="/python-async-await-tutorial/">async/await</a> syntax. It is best for IO-bound and high-level networking purposes.
<h2 id="whentouseconcurrency">When to use Concurrency</h2>
The advantages of concurrency are best tapped into when solving CPU-bound or IO-bound problems.
CPU-bound problems involve programs that do a lot of computation without requiring networking or storage facilities and are only limited by capabilities of the CPU.
IO-bound problems involve programs that rely on input/output resources which sometimes may be slower than the CPU and are usually in use, therefore, the program has to wait for the current task to release the I/O resources.
It is best to write concurrent code when the CPU or I/O resources are limited and you want to speed up your program.
<h2 id="howtouseconcurrency">How to use Concurrency</h2>
In our demonstration example, we will solve a common I/O bound problem, which is downloading files over a network. We will write non-concurrent code and concurrent code and compare the time taken for each program to complete.
We will download images from <a rel="nofollow" target="_blank" href="http://imgur.com/">Imgur</a> through their API. First, we need to create an account and then <a rel="nofollow" target="_blank" href="https://api.imgur.com/oauth2/addclient">register</a> our demo application in order to access the API and download some images.
Once our application is set up on Imgur, we will receive a client identifier and client secret that we'll use to access the API. We'll save the credentials in a <code>.env</code> file since Pipenv <a rel="nofollow" target="_blank" href="https://pipenv.readthedocs.io/en/latest/advanced/#automatic-loading-of-env">automatically loads</a> the variables from the <code>.env</code> file.
<h3 id="synchronousscript">Synchronous Script</h3>
With those details, we can create our first script that will simply download a bunch of images to a <code>downloads</code> folder:
<pre><code class="hljs">import os
from urllib import request
from imgurpython import ImgurClient
import timeit

client_secret = os.getenv(&quot;CLIENT_SECRET&quot;)
client_id = os.getenv(&quot;CLIENT_ID&quot;)

client = ImgurClient(client_id, client_secret)

def download_image(link):
 filename = link.split(&#x27;/&#x27;)[3].split(&#x27;.&#x27;)[0]
 fileformat = link.split(&#x27;/&#x27;)[3].split(&#x27;.&#x27;)[1]
 request.urlretrieve(link, &quot;downloads/{}.{}&quot;.format(filename, fileformat))
 print(&quot;{}.{} downloaded into downloads/ folder&quot;.format(filename, fileformat))

def main():
 images = client.get_album_images(&#x27;PdA9Amq&#x27;)
 for image in images:
 download_image(image.link)

if __name__ == &quot;__main__&quot;:
 print(&quot;Time taken to download images synchronously: {}&quot;.format(timeit.Timer(main).timeit(number=1)))
</code></pre>
In this script, we pass an Imgur album identifier and then download all the images in that album using the function <code>get_album_images()</code>. This gives us a list of the images and then we use our function to download the images and save them to a folder locally.
This simple example gets the job done. We are able to download images from Imgur but it does not work concurrently. It only downloads one image at a time before moving on to the next image. On my machine, the script took 48 seconds to download the images.
<h3 id="optimizingwithmultithreading">Optimizing with Multithreading</h3>
Let us now make our code concurrent using Multithreading and see how it performs:
<pre><code class="hljs"># previous imports from synchronous version are maintained
import threading
from concurrent.futures import ThreadPoolExecutor

# Imgur client setup remains the same as in the synchronous version

# download_image() function remains the same as in the synchronous

def download_album(album_id):
 images = client.get_album_images(album_id)
 with ThreadPoolExecutor(max_workers=5) as executor:
 executor.map(download_image, images)

def main():
 download_album(&#x27;PdA9Amq&#x27;)

if __name__ == &quot;__main__&quot;:
 print(&quot;Time taken to download images using multithreading: {}&quot;.format(timeit.Timer(main).timeit(number=1)))
</code></pre>
In the above example, we create a <code>Threadpool</code> and set up 5 different threads to download images from our gallery. Remember threads execute on a single processor.
This version of our code takes 19 seconds. That is almost three times faster than the synchronous version of the script.
<h3 id="optimizingwithmultiprocessing">Optimizing with Multiprocessing</h3>
Now we will implement Multiprocessing over several CPUs for the same script to see how it performs:
<pre><code class="hljs"># previous imports from synchronous version remain
import multiprocessing

# Imgur client setup remains the same as in the synchronous version

# download_image() function remains the same as in the synchronous

def main():
 images = client.get_album_images(&#x27;PdA9Amq&#x27;)

 pool = multiprocessing.Pool(multiprocessing.cpu_count())
 result = pool.map(download_image, [image.link for image in images])

if __name__ == &quot;__main__&quot;:
 print(&quot;Time taken to download images using multiprocessing: {}&quot;.format(timeit.Timer(main).timeit(number=1)))
</code></pre>
In this version, we create a pool that contains the number of CPU cores on our machine and then map our function to download the images across the pool. This makes our code run in a parallel manner across our CPU and this multiprocessing version of our code takes an average of 14 seconds after multiple runs.
This is slightly faster than our version that utilizes threads and significantly faster than our non-concurrent version.
<h3 id="optimizingwithasyncio">Optimizing with AsyncIO</h3>
Let us implement the same script using AsyncIO to see how it performs:
<pre><code class="hljs"># previous imports from synchronous version remain
import asyncio
import aiohttp

# Imgur client setup remains the same as in the synchronous version

async def download_image(link, session):
 &quot;&quot;&quot;
 Function to download an image from a link provided.
 &quot;&quot;&quot;
 filename = link.split(&#x27;/&#x27;)[3].split(&#x27;.&#x27;)[0]
 fileformat = link.split(&#x27;/&#x27;)[3].split(&#x27;.&#x27;)[1]

 async with session.get(link) as response:
 with open(&quot;downloads/{}.{}&quot;.format(filename, fileformat), &#x27;wb&#x27;) as fd:
 async for data in response.content.iter_chunked(1024):
 fd.write(data)

 print(&quot;{}.{} downloaded into downloads/ folder&quot;.format(filename, fileformat))

async def main():
 images = client.get_album_images(&#x27;PdA9Amq&#x27;)

 async with aiohttp.ClientSession() as session:
 tasks = [download_image(image.link, session) for image in images]

 return await asyncio.gather(*tasks)

if __name__ == &quot;__main__&quot;:
 start_time = timeit.default_timer()

 loop = asyncio.get_event_loop()
 results = loop.run_until_complete(main())

 time_taken = timeit.default_timer() - start_time

 print(&quot;Time taken to download images using AsyncIO: {}&quot;.format(time_taken))
</code></pre>
There are few changes that stand out in our new script. First, we no longer use the normal <code>requests</code> module to download our images, but instead we use <code>aiohttp</code>. The reason for this is that <code>requests</code> is incompatible with AsyncIO since it uses Python's <code>http</code> and <code>sockets</code> module.
Sockets are blocking by nature, i.e. they cannot be paused and execution continues later on. <code>aiohttp</code> solves this and helps us achieve truly asynchronous code.
The keyword <code>async</code> indicates that our function is a coroutine (Co-operative Routine), which is a piece of code that can be paused and resumed. Coroutines multi-task cooperatively, meaning they choose when to pause and let others execute.
We create a pool where we make a queue of all the links to the images we wish to download. Our coroutine is started by putting it in the event loop and executing it until completion.
After several runs of this script, the AsyncIO version takes 14 seconds on average to download the images in the album. This is significantly faster than the multi-threaded and synchronous versions of the code, and pretty similar to the multiprocessing version.
<h2 id="performancecomparison">Performance Comparison</h2>
<table class="table table-striped" style="font-size:15px;overflow-x:auto;">
<thead>
 <tr>
 <th>Synchronous</th>
 <th>Multithreading</th>
 <th>Multiprocessing</th>
 <th>Asyncio</th>
 </tr>
</thead>
<tbody>
 <tr>
 <td>48s</td>
 <td>19s</td>
 <td>14s</td>
 <td>14s</td>
 </tr>
</tbody>
</table>
<h2 id="conclusion">Conclusion</h2>
In this post, we have covered concurrency and how it compares to parallelism. We have also explored the various methods that we can use to implement concurrency in our Python code, including multithreading and multiprocessing, and also discussed their differences.
From the examples above, we can see how concurrency helps our code run faster than it would in a synchronous manner. As a rule of thumb, Multiprocessing is best suited for CPU-bound tasks while Multithreading is best for I/O-bound tasks.
The source code for this post is available on <a rel="nofollow" target="_blank" href="https://github.com/ro6ley/python-concurrency-example">GitHub</a> for reference.

Robley Gori

Concurrency in Python