<h2 id="introduction">Introduction</h2>
One of the key driving factors to technology growth is data. Data has become more important and crucial in the tools being built as technology advances. It has become the driving factor to technology growth, how to collect, store, secure, and distribute data.
This data growth has led to an increase in the utilization of cloud architecture to store and manage data while minimizing the hassle required to maintain consistency and accuracy. As consumers of technology, we are generating and consuming data and this has necessitated the requirement of elaborate systems to help us manage the data.
The cloud architecture gives us the ability to upload and download files from multiple devices as long as we are connected to the Internet. And that is part of what AWS helps us achieve through S3 buckets.
<h2 id="whatiss3">What is S3?</h2>
<a rel="nofollow" target="_blank" href="https://aws.amazon.com/s3/">Amazon Simple Storage Service</a> (S3) is an offering by Amazon Web Services (AWS) that allows users to store data in the form of objects. It is designed to cater to all kinds of users, from enterprises to small organizations or personal projects.
S3 can be used to store data ranging from images, video, and audio all the way up to backups, or website static data, among others.
An S3 bucket is a named storage resource used to store data on AWS. It is akin to a folder that is used to store data on AWS. Buckets have unique names and based on the tier and pricing, users receive different levels of redundancy and accessibility at different prices.
Access privileges to S3 Buckets can also be specified through the AWS Console, the AWS CLI tool, or through provided APIs and libraries.
<h2 id="whatisboto3">What is Boto3?</h2>
<a rel="nofollow" target="_blank" href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">Boto3</a> is a software development kit (SDK) provided by AWS to facilitate the interaction with S3 APIs and other services such as <a rel="nofollow" target="_blank" href="https://aws.amazon.com/ec2/">Elastic Compute Cloud</a> (EC2). Using Boto3, we can list all the S3 buckets, create EC2 instances, or control any number of AWS resources.
<h2 id="whyuses3">Why use S3?</h2>
We can always provision our own servers to store our data and make it accessible from a range of devices over the Internet, so why should we use AWS's S3? There are several scenarios where it comes in handy.
First, AWS S3 eliminates all the work and costs involved in building and maintaining servers that store our data. We do not have to worry about acquiring the hardware to host our data or the personnel required to maintain the infrastructure. Instead, we can focus solely on our code and ensuring our services are in the best condition.
By using S3, we get to tap into the impressive performance, availability, and scalability capabilities of AWS. Our code will be able to scale effectively and perform under heavy loads and be highly available to our end users. We get to achieve this without having to build or manage the infrastructure behind it.
AWS offers tools to help us with analytics and audit, as well as management and reports on our data. We can view and analyze how the data in our buckets is accessed or even replicate the data into other regions to enhance the access of the data by the end-users. Our data is also encrypted and securely stored so that it is secure at all times.
Through <a target="_blank" href="/serverless-python-application-development-with-aws-chalice/">AWS Lambda</a> we can also respond to data being uploaded or downloaded from our S3 buckets and respond to users through configured alerts or reports for a more personalized and instant experience as expected from technology.
<h2 id="settingupaws">Setting Up AWS</h2>
To get started with S3, we need to set up an account on <a rel="nofollow" target="_blank" href="https://aws.amazon.com/">AWS</a> or log in to an existing one.
We will also need to set up the AWS CLI tool to be able to interact with our resources from the command line, which is available for Mac, Linux, and Windows.
We can install it by running:
<pre><code class="hljs">$ pip install awscli
</code></pre>
Once the CLI tool is set up, we can generate our credentials under our profile dropdown and use them to configure our CLI tool as follows:
<pre><code class="hljs">$ aws configure
</code></pre>
This command will give us prompts to provide our <code>Access Key ID</code>, <code>Secret Access Key</code>, default regions, and output formats. More details about configuring the AWS CLI tool can be found <a rel="nofollow" target="_blank" href="https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html">here</a>.
<h2 id="ourapplicationflaskdrive">Our Application - FlaskDrive</h2>
<h3 id="setup">Setup</h3>
Let's build a Flask application that allows users to upload and download files to and from our S3 buckets, as hosted on AWS.
We will use the Boto3 SDK to facilitate these operations and build out a simple front-end to allow users to upload and view the files as hosted online.
It is advisable to use a virtual environment when working on Python projects, and for this one we will use the <a rel="nofollow" target="_blank" href="https://pipenv.readthedocs.io/en/latest/#install-pipenv-today">Pipenv</a> tool to create and manage our environment. Once set up, we create and activate our environment with Python3 as follows:
<pre><code class="hljs">$ pipenv install --three
$ pipenv shell
</code></pre>
We now need to install <code>Boto3</code> and <code>Flask</code> that are required to build our FlaskDrive application as follows:
<pre><code class="hljs">$ pipenv install flask
$ pipenv install boto3
</code></pre>
<h3 id="implementation">Implementation</h3>
After setting up, we need to create the buckets to store our data and we can achieve that by heading over to the AWS console and choosing S3 in the Services menu.
After creating a bucket, we can use the CLI tool to view the buckets we have available:
<pre><code class="hljs">$ aws s3api list-buckets
{
 &quot;Owner&quot;: {
 &quot;DisplayName&quot;: &quot;robley&quot;,
 &quot;ID&quot;: &quot;##########################################&quot;
 },
 &quot;Buckets&quot;: [
 {
 &quot;CreationDate&quot;: &quot;2019-09-25T10:33:40.000Z&quot;,
 &quot;Name&quot;: &quot;flaskdrive&quot;
 }
 ]
}
</code></pre>
We will now create the functions to upload, download, and list files on our S3 buckets using the Boto3 SDK, starting off with the <code>upload_file</code> function:
<pre><code class="hljs">def upload_file(file_name, bucket):
 &quot;&quot;&quot;
 Function to upload a file to an S3 bucket
 &quot;&quot;&quot;
 object_name = file_name
 s3_client = boto3.client(&#x27;s3&#x27;)
 response = s3_client.upload_file(file_name, bucket, object_name)

 return response
</code></pre>
The <code>upload_file</code> function takes in a file and the bucket name and uploads the given file to our S3 bucket on AWS.
<pre><code class="hljs">def download_file(file_name, bucket):
 &quot;&quot;&quot;
 Function to download a given file from an S3 bucket
 &quot;&quot;&quot;
 s3 = boto3.resource(&#x27;s3&#x27;)
 output = f&quot;downloads/{file_name}&quot;
 s3.Bucket(bucket).download_file(file_name, output)

 return output
</code></pre>
The <code>download_file</code> function takes in a file name and a bucket and downloads it to a folder that we specify.
<pre><code class="hljs">def list_files(bucket):
 &quot;&quot;&quot;
 Function to list files in a given S3 bucket
 &quot;&quot;&quot;
 s3 = boto3.client(&#x27;s3&#x27;)
 contents = []
 for item in s3.list_objects(Bucket=bucket)[&#x27;Contents&#x27;]:
 contents.append(item)

 return contents
</code></pre>
The function <code>list_files</code> is used to retrieve the files in our S3 bucket and list their names. We will use these names to <a target="_blank" href="https://stackabuse.com/download-files-with-python/">download the files</a> from our S3 buckets.
With our S3 interaction file in place, we can build our Flask application to provide the web-based interface for interaction. The application will be a simple single-file Flask application for demonstration purposes with the following structure:
<pre><code class="hljs">.
├── Pipfile # stores our application requirements
├── __init__.py
├── app.py # our main Flask application
├── downloads # folder to store our downloaded files
├── s3_demo.py # S3 interaction code
├── templates
│ └── storage.html
└── uploads # folder to store the uploaded files
</code></pre>
The core functionality of our Flask application will reside in the <code>app.py</code> file:
<pre><code class="hljs">import os
from flask import Flask, render_template, request, redirect, send_file
from s3_demo import list_files, download_file, upload_file

app = Flask(__name__)
UPLOAD_FOLDER = &quot;uploads&quot;
BUCKET = &quot;flaskdrive&quot;

@app.route(&#x27;/&#x27;)
def entry_point():
 return &#x27;Hello World!&#x27;

@app.route(&quot;/storage&quot;)
def storage():
 contents = list_files(&quot;flaskdrive&quot;)
 return render_template(&#x27;storage.html&#x27;, contents=contents)

@app.route(&quot;/upload&quot;, methods=[&#x27;POST&#x27;])
def upload():
 if request.method == &quot;POST&quot;:
 f = request.files[&#x27;file&#x27;]
 f.save(os.path.join(UPLOAD_FOLDER, f.filename))
 upload_file(f&quot;uploads/{f.filename}&quot;, BUCKET)

 return redirect(&quot;/storage&quot;)

@app.route(&quot;/download/&lt;filename&gt;&quot;, methods=[&#x27;GET&#x27;])
def download(filename):
 if request.method == &#x27;GET&#x27;:
 output = download_file(filename, BUCKET)

 return send_file(output, as_attachment=True)

if __name__ == &#x27;__main__&#x27;:
 app.run(debug=True)
</code></pre>
This is a simple Flask application with 4 endpoints:
<ul>
<li>The <code>/storage</code> endpoint will be the landing page where we will display the current files in our S3 bucket for download, and also an input for users to upload a file to our S3 bucket,</li>
<li>The <code>/upload</code> endpoint will be used to receive a file and then call the <code>upload_file()</code> method that uploads a file to an S3 bucket</li>
<li>The <code>/download</code> endpoint will receive a file name and use the <code>download_file()</code> method to download the file to the user's device</li>
</ul>
And finally, our HTML template will be as simple as:
<pre><code class="hljs">&lt;!DOCTYPE html&gt;
&lt;html&gt;
 &lt;head&gt;
 &lt;title&gt;FlaskDrive&lt;/title&gt;
 &lt;/head&gt;
 &lt;body&gt;
 &lt;div class=&quot;content&quot;&gt;
 &lt;h3&gt;Flask Drive: S3 Flask Demo&lt;/h3&gt;
 &lt;p&gt;Welcome to this AWS S3 Demo&lt;/p&gt;
 &lt;div&gt;
 &lt;h3&gt;Upload your file here:&lt;/h3&gt;
 &lt;form method=&quot;POST&quot; action=&quot;/upload&quot; enctype=multipart/form-data&gt;
 &lt;input type=file name=file&gt;
 &lt;input type=submit value=Upload&gt;
 &lt;/form&gt;
 &lt;/div&gt;
 &lt;div&gt;
 &lt;h3&gt;These are your uploaded files:&lt;/h3&gt;
 &lt;p&gt;Click on the filename to download it.&lt;/p&gt;
 &lt;ul&gt;
 {% for item in contents %}
 &lt;li&gt;
 &lt;a href=&quot;/download/{{ item.Key }}&quot;&gt; {{ item.Key }} &lt;/a&gt;
 &lt;/li&gt;
 {% endfor %}
 &lt;/ul&gt;
 &lt;/div&gt;
 &lt;/div&gt;
 &lt;/body&gt;
&lt;/html&gt;
</code></pre>
With our code and folders set up, we start our application with:
<pre><code class="hljs">$ python app.py
</code></pre>
When we navigate to <code>http://localhost:5000/storage</code> we are welcomed by the following landing page:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-and-downloading-files-with-s3-and-boto-in-python-1.png" alt="flask_drive_landing_1">
Let us now upload a file using the input field and this is the output:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-and-downloading-files-with-s3-and-boto-in-python-2.png" alt="flask_drive_landing_2">
We can confirm the upload by checking our S3 dashboard, and we can find our image there:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-and-downloading-files-with-s3-and-boto-in-python-3.png" alt="flask_drive_s3_dashboard">
Our file has been successfully uploaded from our machine to AWS's S3 Storage.
On our FlaskDrive landing page, we can download the file by simply clicking on the file name then we get the prompt to save the file on our machines.
<h2 id="conclusion">Conclusion</h2>
In this post, we have created a Flask application that stores files on AWS's S3 and allows us to download the same files from our application. We used the Boto3 library alongside the AWS CLI tool to handle the interaction between our application and AWS.
We have eliminated the need for us having our own servers to handle the storage of our files and tapped into Amazon's infrastructure to handle it for us through the AWS Simple Storage Service. It has taken us a short time to develop, deploy and make our application available to end-users and we can now enhance it to add permissions among other features.
The source code for this project is available <a rel="nofollow" target="_blank" href="https://github.com/ro6ley/flask-drive"> here on GitHub</a>.

Robley Gori

AWS S3 is a service that allows us to easily manage file storage in the cloud. In this article, we'll manage file uploads with Python and Flask

File Management with AWS S3, Python, and Flask

<h2 id="introduction">Introduction</h2>
Much of the software and web apps we build today requires some kind of hosting for files - images, invoices, audio files, etc. The traditional way to store files was just to just save them on the server's HDD. However, saving files onto the HDD of the server comes with limitations such as not being able to scale up, being required to allocate space before using and much higher/non-flexible prices. Not to mention, requesting a huge amount of (potentially large) images can really put a strain on the server.
To offload the servers, developers started hosting files with cloud storage providers such as <a rel="nofollow" target="_blank" href="https://aws.amazon.com/s3/">AWS S3</a>, <a rel="nofollow" target="_blank" href="https://cloud.google.com/storage/">Google Cloud Storage</a>, etc.
In this article we will show you how to write Node.js code to upload files to S3.
<h2 id="whatiss3">What is S3?</h2>
S3, or Simple Storage Service, is a cloud storage service provided by Amazon Web Services (AWS). Using S3, you can host any number of files while paying for only what you use.
S3 also provides multi-regional hosting to customers by their region and thus are able to really quickly serve the requested files with minimum delay.
<h2 id="settinguptheenvironment">Setting up the Environment</h2>
<h3 id="awscredentials">AWS Credentials</h3>
To get started, you need to generate the AWS Security Key Access Credentials first. To do so, login to your AWS Management Console.
Click on your username:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-1.png" alt="">
Then select Access Keys -&gt; Create New Access Key:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-2.png" alt="">
After that you can either copy the Access Key ID and Secret Access Key from this window or you can download it as a <code>.CSV</code> file:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-3.png" alt="">
<h3 id="creatingans3bucket">Creating an S3 Bucket</h3>
Now let's create an AWS S3 Bucket with proper access. We can do this using the AWS management console or by using Node.js.
To create an S3 bucket using the management console, go to the S3 service by selecting it from the service menu:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-4.png" alt="">
Select &quot;Create Bucket&quot; and enter the name of your bucket and the region that you want to host your bucket. If you already know from which region the majority of your users will come from, it's wise to select a region as close to theirs as possible. This will ensure that the files from the server will be served in a more optimal time-frame.
The name you select for your bucket should be a unique name among all AWS users, so try a new one if the name is not available:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-5.png" alt="">
Follow through the wizard and configure permissions and other settings per your requirements.
To create the bucket using Node.js, we'll first have to set up our development environment.
<h3 id="developmentenvironment">Development Environment</h3>
So now let's get started with our example by configuring a new Node.js project:
<pre><code class="hljs">$ npm init
</code></pre>
To start using any AWS Cloud Services in Node.js, we have to install the AWS SDK (System Development Kit).
Install it using your preferred package manager - we'll use <code>npm</code>:
<pre><code class="hljs">$ npm i --save aws-sdk
</code></pre>
<h2 id="implementation">Implementation</h2>
<h3 id="creatingans3bucket">Creating an S3 Bucket</h3>
If you have already created a bucket manually, you may skip this part. But if not, let's create a file, say, <code>create-bucket.js</code> in your project directory.
Import the <code>aws-sdk</code> library to access your S3 bucket:
<pre><code class="hljs">const AWS = require(&#x27;aws-sdk&#x27;);
</code></pre>
Now, let's define three constants to store <code>ID</code>, <code>SECRET</code>, and <code>BUCKET_NAME</code>. These are used to identify and access our bucket:
<pre><code class="hljs">// Enter copied or downloaded access ID and secret key here
const ID = &#x27;&#x27;;
const SECRET = &#x27;&#x27;;

// The name of the bucket that you have created
const BUCKET_NAME = &#x27;test-bucket&#x27;;
</code></pre>
Now, we need to initialize the S3 interface by passing our access keys:
<pre><code class="hljs">const s3 = new AWS.S3({
 accessKeyId: ID,
 secretAccessKey: SECRET
});
</code></pre>
With the S3 interface successfully initialized, we can go ahead and create the bucket:
<pre><code class="hljs">const params = {
 Bucket: BUCKET_NAME,
 CreateBucketConfiguration: {
 // Set your region here
 LocationConstraint: &quot;eu-west-1&quot;
 }
};

s3.createBucket(params, function(err, data) {
 if (err) console.log(err, err.stack);
 else console.log(&#x27;Bucket Created Successfully&#x27;, data.Location);
});
</code></pre>
At this point we can run the code and test if the bucket is created on the cloud:
<pre><code class="hljs">$ node create-bucket.js
</code></pre>
If the code execution is successful you should see the success message, followed by the bucket address in the output:
<pre><code class="hljs">Bucket Created Successfully http://test-bucket-2415soig.s3.amazonaws.com/
</code></pre>
You can visit your S3 dashboard and make sure the bucket is created:
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-7.png" alt="">
To see a complete list of regions and other parameters, please take a look at the <a rel="nofollow" target="_blank" href="https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html">official documentation</a>.
<h3 id="uploadingfiles">Uploading Files</h3>
At this point, let's implement the file upload functionality. In a new file, e.g. <code>upload.js</code>, import the <code>aws-sdk</code> library to access your S3 bucket and the <code>fs</code> module to read files from your computer:
<pre><code class="hljs">const fs = require(&#x27;fs&#x27;);
const AWS = require(&#x27;aws-sdk&#x27;);
</code></pre>
We need to define three constants to store <code>ID</code>, <code>SECRET</code>, and <code>BUCKET_NAME</code> and initialize the S3 client as we did before.
Now, let's create a function that accepts a <code>fileName</code> parameter, representing the file we want to upload:
<pre><code class="hljs">const uploadFile = (fileName) =&gt; {
 // Read content from the file
 const fileContent = fs.readFileSync(fileName);

 // Setting up S3 upload parameters
 const params = {
 Bucket: BUCKET_NAME,
 Key: &#x27;cat.jpg&#x27;, // File name you want to save as in S3
 Body: fileContent
 };

 // Uploading files to the bucket
 s3.upload(params, function(err, data) {
 if (err) {
 throw err;
 }
 console.log(`File uploaded successfully. ${data.Location}`);
 });
};
</code></pre>
Before we upload the file, we need to read its contents as a buffer. After reading it, we can define the needed parameters for the file upload, such as <code>Bucket</code>, <code>Key</code>, and <code>Body</code>.
Besides these three parameters, there's a long list of other optional parameters. To get an idea of the things you can define for a file while uploading, here are a few useful ones:
<ul>
<li><code>StorageClass</code>: Define the class you want to store the object. S3 is intended to provide fast file serving. But in case files are not accessed frequently you can use a different storage class. For example, if you have files which are hardly touched you can store them in &quot;S3 Glacier Storage&quot; where the price is very low compared to &quot;S3 Standard Storage&quot;. But it will take more time to access those files in case you need it and is covered with a different service level agreement.</li>
<li><code>ContentType</code>: Sets the image MIME type. The default type will be &quot;binary/octet-stream&quot;. Adding a MIME type like &quot;image/jpeg&quot; will help browsers and other HTTP clients to identify the type of the file.</li>
<li><code>ContentLength</code>: Sets the size of the body in bytes, which comes in handy if body size cannot be determined automatically.</li>
<li><code>ContentLanguage</code>: Set this parameter to define which language the content is in. This will also help HTTP clients to identify or translate the content.</li>
</ul>
For the <code>Bucket</code> parameter, we'll use our bucket name, whereas for the <code>Key</code> parameter we'll add the file name we want to save as, and for the <code>Body</code> parameter, we'll use <code>fileContent</code>.
With that done, we can upload any file by passing the file name to the function:
<pre><code class="hljs">uploadFile(&#x27;cat.jpg&#x27;);
</code></pre>
You can replace <code>cat.jpg</code> with a file name that exists in the same directory as the code, a relative file path, or an absolute file path.
At this point, we can run the code and test out if it works:
<pre><code class="hljs">$ node upload.js
</code></pre>
If everything is fine, you should see output like what is shown below with a link to your file, which is stored in <code>data.Location</code>:
<pre><code class="hljs">File uploaded successfully. https://test-bucket-1242tsr.s3.ap-northeast-2.amazonaws.com/cat.jpg
</code></pre>
If there is any error it should be displayed on the console as well.
Additionally, you can go to your bucket in the AWS Management Console and make sure the file is uploaded.
<img src="https://stackabuse.s3.amazonaws.com/media/uploading-files-to-s3-with-nodejs-6.png" alt="">
<h2 id="conclusion">Conclusion</h2>
To offload our application servers, a popular choice of developers is to host files using storage providers such as AWS S3, Google Cloud Storage, etc. We've made a very simple Node.js app that handles file uploads to S3 using its interface via the <code>aws-sdk</code> module.
Depending on your requirements you can also configure public access to your bucket or the files using the console.
If you'd like to play around with the code, as always, you can find it on GitHub in <a rel="nofollow" target="_blank" href="https://gist.github.com/jkasun/b0e2b818e0271db9c1363f637dacf12e">this Gist</a>.

Janith Kasun

Uploading Files to AWS S3 with Node.js

Usman Malik

Text generation is one of the state-of-the-art applications of NLP. In this article, you will see how to generate text via deep learning techniques in Python using the Keras library.

Python for NLP: Deep Learning Text Generation with Keras

Dan Nelson

Analyzing API Data with MongoDB, Seaborn, and Matplotlib

Olivera Popović

Graphs in Java: Dijkstra's Algorithm

<h2 id="introduction">Introduction</h2>
Graphs are a convenient way to store certain types of data. The concept was ported from mathematics and appropriated for the needs of computer science.
Due to the fact that many things can be represented as graphs, graph traversal has become a common task, especially used in data science and machine learning.
<ul>
<li><a target="_blank" href="/graph-theory-and-graph-related-algorithms-theory-and-implementation">Graph Theory and Graph-Related Algorithm's Theory and Implementation</a>
<ul>
<li><a target="_blank" href="/graphs-in-java-representing-graphs-in-code/">Representing Graphs in Code</a></li>
<li><a target="_blank" href="/graphs-in-java-depth-first-search-dfs/">Depth-First Search (DFS)</a></li>
<li>Breadth-First Search (BFS)</li>
<li><a target="_blank" href="/graphs-in-java-dijkstras-algorithm/">Dijkstra's Algorithm</a></li>
<li><a target="_blank" href="/graphs-in-java-minimum-spanning-trees-prims-algorithm">Minimum Spanning Trees - Prim's Algorithm</a></li>
</ul>
</li>
</ul>
<h2 id="breadthfirstsearch">Breadth-First Search</h2>
Breadth First Search (BFS) visits &quot;layer-by-layer&quot;. This means that in a Graph, like shown below, it first visits all the children of the starting node. These children are treated as the &quot;second layer&quot;.
Unlike <a target="_blank" href="/graphs-in-java-depth-first-search-dfs/">Depth-First Search</a> (DFS), BFS doesn't aggressively go though one branch until it reaches the end, rather when we start the search from a node, it visits all the unvisited neighbors of that node before proceeding to all the unvisited neighbors of another node:
<img src="https://s3.stackabuse.com/media/articles/bfs-gif.gif" alt="">
<h2 id="implementation">Implementation</h2>
We'll use graphs implemented via an adjacency list, like we used for DFS. Also, we need to add the <code>visited</code> attribute alongside the <code>visit()</code> and <code>univisit()</code> methods to our <code>Node</code> class:
<pre><code class="hljs">public class Node {
 int n;
 String name;
 boolean visited;

 Node(int n, String name) {
 this.n = n;
 this.name = name;
 visited = false;
 }

 void visit() {
 visited = true;
 }

 void unvisit() {
 visited = false;
 }
}
</code></pre>
Now, let's define a <code>Graph</code>:
<pre><code class="hljs">public class Graph {

 // Each node maps to a list of all his neighbors
 private HashMap&lt;Node, LinkedList&lt;Node&gt;&gt; adjacencyMap;
 private boolean directed;

 public Graph(boolean directed) {
 this.directed = directed;
 adjacencyMap = new HashMap&lt;&gt;();
 }

 // ...
}
</code></pre>
Now, let's add the method <code>addEdge()</code>. We'll use two methods, a helper method and the actual method.
In the helper method, we'll also make a check for possible duplicate edges. Before adding an edge between <code>A</code> and <code>B</code>, we'll first remove it and only then add it. If it existed (we're adding a duplicate edge), it was removed and after adding it again, there's only one.
Though, if it didn't exist, removing a non-existing edge will result in a <code>NullPointerException</code> so we're introducing a temporary copy of the list:
<pre><code class="hljs">public void addEdgeHelper(Node a, Node b) {
 LinkedList&lt;Node&gt; tmp = adjacencyMap.get(a);

 if (tmp != null) {
 tmp.remove(b);
 }
 else tmp = new LinkedList&lt;&gt;();
 tmp.add(b);
 adjacencyMap.put(a,tmp);
}

public void addEdge(Node source, Node destination) {

 // We make sure that every used node shows up in our .keySet()
 if (!adjacencyMap.keySet().contains(source))
 adjacencyMap.put(source, null);

 if (!adjacencyMap.keySet().contains(destination))
 adjacencyMap.put(destination, null);

 addEdgeHelper(source, destination);

 // If a graph is undirected, we want to add an edge from destination to source as well
 if (!directed) {
 addEdgeHelper(destination, source);
 }
}
</code></pre>
Finally, we'll have the <code>printEdges()</code>, <code>hasEdge()</code> and <code>resetNodesVisited()</code> helper methods, which are pretty straightforward:
<pre><code class="hljs">public void printEdges() {
 for (Node node : adjacencyMap.keySet()) {
 System.out.print(&quot;The &quot; + node.name + &quot; has an edge towards: &quot;);
 for (Node neighbor : adjacencyMap.get(node)) {
 System.out.print(neighbor.name + &quot; &quot;);
 }
 System.out.println();
 }
}

public boolean hasEdge(Node source, Node destination) {
 return adjacencyMap.containsKey(source) &amp;&amp; adjacencyMap.get(source).contains(destination);
}

public void resetNodesVisited(){
 for(Node node : adjacencyMap.keySet()){
 node.unvisit();
 }
}
</code></pre>
Let's examine the BFS algorithm on the following undirected graph:
<pre><code class="hljs">Node 0 has neighbors: 1, 3, 2
Node 1 has neighbors: 0
Node 2 has neighbors: 3, 0
Node 3 has neighbors: 2, 0
</code></pre>
We can pick any node to start from, so let's start with 1. We repeat the process of adding and removing nodes from the queue until the queue is empty.
<blockquote>
A Queue is a FIFO (first-in-first-out) data structure. It works just like a real-life queue, and so entries are processed (removed from the queue) one by one in the order in which they were added.
</blockquote>
This is a very convenient data structure for BFS since we want to process nodes in the order in which we visit them, making sure that we process nodes &quot;closer&quot; to the starting node first.
Since they are added to the queue before any nodes &quot;further&quot; away from the starting node are added to the queue, we know the closer ones will be processed first.
<ol>
<li>We start off by having a queue that contains only node 1</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/graphs-in-java-breadth-first-search-(bfs)-2.png" alt="">
<ol start="2">
<li>Remove the first element from the queue, in this case 1, mark it as visited</li>
<li>Add all 1's unvisited neighbors to the queue (only 0)</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/graphs-in-java-breadth-first-search-(bfs)-3.png" alt="">
<ol start="4">
<li>Remove the first element from the queue, in this case 0, mark it as visited</li>
<li>Add all 0's unvisited neighbors to the queue (nodes 3 and 2, 1 has been marked as visited already)</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/graphs-in-java-breadth-first-search-(bfs)-4.png" alt="">
<ol start="6">
<li>Remove the first element from the queue, in this case 3, mark it as visited</li>
<li>Add all 3's unvisited neighbors to the queue (there are none)</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/graphs-in-java-breadth-first-search-(bfs)-5.png" alt="">
<ol start="8">
<li>Remove the first element from the queue, in this case 2, mark it as visited</li>
<li>Add all 2's unvisited neighbors to the queue (again, there are none)</li>
<li>The queue is now empty, BFS has finished</li>
</ol>
Our nodes are visited in the <code>1-0-3-2</code> order. It should be obvious that set of steps 2-3, 4-5, 6-7, and 8-9 are the same and that step 10 is our loop termination condition. Viewed in this way, it should be easy to write code for our <code>breadthFirstSearch(Node node)</code> method.
There are several types of <code>Queue</code> implementations in Java, but we'll use a <code>LinkedList</code> instead, since it provides all the necessary methods.
We're adding the following method to our <code>Graph</code> class:
<pre><code class="hljs">void breadthFirstSearch(Node node) {

 // Just so we handle receiving an uninitialized Node, otherwise an
 // exception will be thrown when we try to add it to queue
 if (node == null)
 return;

 // Creating the queue, and adding the first node (step 1)
 LinkedList&lt;Node&gt; queue = new LinkedList&lt;&gt;();
 queue.add(node);

 while (!queue.isEmpty()) {
 Node currentFirst = queue.removeFirst();

 // In some cases we might have added a particular node more than once before
 // actually visiting that node, so we make sure to check and skip that node if we have
 // encountered it before
 if (currentFirst.isVisited())
 continue;

 // Mark the node as visited
 currentFirst.visit();
 System.out.print(currentFirst.name + &quot; &quot;);

 LinkedList&lt;Node&gt; allNeighbors = adjacencyMap.get(currentFirst);

 // We have to check whether the list of neighbors is null before proceeding, otherwise
 // the for-each loop will throw an exception
 if (allNeighbors == null)
 continue;

 for (Node neighbor : allNeighbors) {
 // We only add unvisited neighbors
 if (!neighbor.isVisited()) {
 queue.add(neighbor);
 }
 }
 }
 System.out.println();
}
</code></pre>
Now we create our example graph in code and check whether our method works as expected:
<pre><code class="hljs">public class GraphShow {
 public static void main(String[] args) {

 Graph graph = new Graph(false);
 Node a = new Node(0, &quot;0&quot;);
 Node b = new Node(1, &quot;1&quot;);
 Node c = new Node(2, &quot;2&quot;);
 Node d = new Node(3, &quot;3&quot;);
 Node e = new Node(4, &quot;4&quot;);

 graph.addEdge(a,d);
 graph.addEdge(a,b);
 graph.addEdge(a,c);
 graph.addEdge(c,d);

 graph.breadthFirstSearch(b);
 }
}
</code></pre>
Output:
<pre><code class="hljs">1 0 3 2
</code></pre>
If you read the DFS article then you may recall that we encountered a situation where in an unconnected graph, not all of the nodes would be printed out since the algorithm would go through all the nodes it can, and then stop.
The same thing happens with BFS, and this can also happen when graphs are directed, sometimes we can't reach all the nodes. Sometimes this is the behavior we're looking for, but sometimes we want all of the nodes to be visited.
We'll do the same thing as we did in DFS, i.e. we'll keep calling BFS as long as there are any unvisited nodes. We'll make a new <code>breadthFirstSearchModified(Node node)</code> method that does this for us:
<pre><code class="hljs">void breadthFirstSearchModified(Node node) {
 breadthFirstSearch(node);

 for (Node n : adjacencyMap.keySet()) {
 if (!n.isVisited()) {
 breadthFirstSearch(n);
 }
 }
}
</code></pre>
<pre><code class="hljs">public class GraphShow {
 public static void main(String[] args) {

 Graph graph = new Graph(false);
 Node a = new Node(0, &quot;0&quot;);
 Node b = new Node(1, &quot;1&quot;);
 Node c = new Node(2, &quot;2&quot;);
 Node d = new Node(3, &quot;3&quot;);
 Node e = new Node(4, &quot;4&quot;);

 graph.addEdge(a,d);
 graph.addEdge(a,b);
 graph.addEdge(c,e);

 System.out.println(&quot;Using the unmodified version of BFS we get:&quot;);
 graph.breadthFirstSearch(a);

 graph.resetNodesVisited();
 System.out.println(&quot;Using the modified version of BFS we get:&quot;);
 graph.breadthFirstSearchModified(a);
 }
}
</code></pre>
Output:
<pre><code class="hljs">Using the unmodified version of BFS we get:
0 3 1
Using the modified version of BFS we get:
0 3 1
4 2
</code></pre>
<blockquote>
There is also something called a &quot;bidirectional&quot; BFS search. This is useful when we want to find the shortest path between two vertices (nodes).
This is achieved by simultaneously (in different threads) running a BFS from the starting node and the destination node. This, in theory, finds the shortest path between two nodes twice as fast as running BFS just from the starting node.
</blockquote>
Note: Same as with DFS, if we want to go through the neighbors in a particular order (instead of the order in which the edges were added), we can use a <code>PriorityQueue</code> instead of a <code>LinkedList</code> for the list of neighbors.
The <a target="_blank" href="/graphs-in-java-depth-first-search-dfs#orderingneighbors">code is the same</a>, we just have to implement <code>Comparable</code> and add a <code>compareTo()</code> method to our <code>Node</code> class.
<h2 id="conclusion">Conclusion</h2>
Graphs are a convenient way to store certain types of data. The concept was ported from mathematics and appropriated for the needs of computer science.
Due to the fact that many things can be represented as graphs, graph traversal has become a common task, especially used in data science and machine learning.
Breadth-First Search is one of the few graph traversal algorithms and visits nodes &quot;layer-by-layer&quot;. Unlike Depth-First Search, BFS doesn't aggressively go through one branch until it reaches the end, rather when we start the search from a node, it visits all the unvisited neighbors of that node before proceeding to all the unvisited neighbors of another node.

File Management with AWS S3, Python, and Flask

Uploading Files to AWS S3 with Node.js

Python for NLP: Deep Learning Text Generation with Keras

Analyzing API Data with MongoDB, Seaborn, and Matplotlib

Graphs in Java: Dijkstra's Algorithm

Graphs in Java: Breadth-First Search (BFS)

Graphs in Java: Depth-First Search (DFS)

Uploading Files with Spring Boot

Graphs in Java: Representing Graphs in Code