<h2 id="introduction">Introduction</h2>
As the use of software becomes more common and more and more systems are built to handle various tasks, data plays a more important role in the current and future technology scene. Information is increasingly becoming more valuable as technology advances and opens up more opportunities for its use.
It is for this reason, and many more, that safe storage and manipulation of data have become an important aspect of any system or application that is built.
<h2 id="whatisobjectrelationalmapping">What is Object-Relational Mapping?</h2>
In many systems, real-life objects are modeled as objects in systems to ease the representation and manipulation of their attributes. For instance, a phone can be modeled as an object with attributes such as its name, operating system, manufacturer, and much more as its attributes and this can be easily manipulated and stored in a database.
Object-Relational Mapping (ORM) is a technique of mapping such objects and their attributes in the database through Object-Relational Mappers. This technique also helps us convert data between incompatible systems using object-oriented programming applications.
An ORM is a library that helps us interact with multiple databases or systems easily using our language of choice. Our code is now mapped to the databases' specific query language.
Normally, we need to use a database-specific language to interact with a database. For example, to interact with a <a rel="nofollow" target="_blank" href="https://www.mysql.com/">MySQL</a> database, we need to use the Structured Query Language (SQL), but these languages may differ from platform to platform.
For example, while they are still similar, the syntax on a <a rel="nofollow" target="_blank" href="https://www.postgresql.org/">PostgreSQL</a> database is different from the query language used on a <a rel="nofollow" target="_blank" href="https://www.microsoft.com/en-us/sql-server/sql-server-2019">Microsoft SQL</a> database. An ORM helps bridge that difference and plug in our software into different database systems with ease.
Other benefits of using an ORM includes speeding up of the development process since the developers do not have to write the database access code and repeat it every time they want to access a database. Once a model is designed and manipulation code is written, it does not need to be done again, hence making the code easy to update, maintain, and reuse.
However, there are some drawbacks associated with ORMs and they include:
<ul>
<li>ORMs have a tendency to be slow in some situations performance-wise</li>
<li>For complex queries like joins, ORMs sometimes cannot substitute raw SQL queries</li>
<li>Due to the abstractions introduced by an ORM, the developer may lose understanding of SQL and how database management is achieved behind the scenes</li>
</ul>
<h2 id="hibernate">Hibernate</h2>
<a rel="nofollow" target="_blank" href="https://hibernate.org/">Hibernate</a> is a framework that enables developers to easily persist application data in relational databases using JDBC. It is an implementation of the Java Persistence API (JPA), which means that it can be utilized in any system that supports JPA, such as the standard edition (Java SE) and the enterprise edition (Java EE).
Hibernate is a lightweight and open-source tool that simplifies the creation, manipulation, and access of data from a database in Java-based applications. It works by mapping an object created from a Java class and its attributes, to data stored in the database.
Some advantages of using Hibernate include:
<ul>
<li>It is open-source and lightweight, which means that it's free to use and has a community of contributors constantly improving it</li>
<li>Hibernate utilizes a cache internally that enhances its performance</li>
<li>It is database-independent, which means it can be used to access and manipulate data in various different databases</li>
<li>It provides the functionality to simplify joins when fetching data from multiple tables</li>
<li>By creating tables automatically, the developer can focus on doing other logic</li>
<li>It's a stable framework that has been around for 18 years</li>
</ul>
<h2 id="alternatives">Alternatives</h2>
Hibernate is not the only ORM framework that we can use in our Java applications, others include:
<ul>
<li><a rel="nofollow" target="_blank" href="https://www.jooq.org/">JOOQ</a> (Java Object Oriented Querying) is a light database-mapping software library</li>
<li><a rel="nofollow" target="_blank" href="http://jdbi.org/">JDBI</a> provides access to relational data in Java in a convenient way</li>
<li><a rel="nofollow" target="_blank" href="https://blog.mybatis.org/">MyBatis</a> is an SQL mapper framework to integrate relational databases</li>
<li><a rel="nofollow" target="_blank" href="http://avaje.com/">Ebean</a> which can be used for both Java and Kotlin-based applications</li>
<li><a rel="nofollow" target="_blank" href="http://ormlite.com/">ORMLite</a> which is a lightweight framework to persist Java objects to SQL databases</li>
</ul>
These are but a few of the alternatives for Hibernate, there are definitely even more libraries and frameworks out there that suit many different scenarios and databases.
<h2 id="implementinghibernatewithspringboot">Implementing Hibernate with Spring Boot</h2>
<h3 id="projectsetup">Project Setup</h3>
For this demo project, we are going to use a PostgreSQL database and installation instructions can be found <a rel="nofollow" target="_blank" href="https://www.educba.com/install-postgresql/">here</a> for Mac OS, Linux, and Windows platforms.
Once set up, we can create our demo database, <code>phonesdemo</code>. <a rel="nofollow" target="_blank" href="https://www.pgadmin.org/download/">PgAdmin</a> provides a user interface to interact with a PostgreSQL database, but a terminal can also be used.
Let us now see Hibernate in action by using it in a sample Spring Boot API, which we will bootstrap using the <a rel="nofollow" target="_blank" href="https://start.spring.io/">Spring Initializr</a> tool:
<img src="https://stackabuse.s3.amazonaws.com/media/implementing-hibernate-with-spring-boot-and-postgresql-1.png" alt="spring_intializr">
We will be using Java 8 and Maven for our dependency management with a few dependencies:
<ul>
<li><code>Spring Web Starter</code> to help us build a web-based application</li>
<li><code>Spring Data JPA</code> to provide the data access API that Hibernate will utilize</li>
<li><code>H2 Database</code> to bring in Hibernate functionality into our project</li>
<li><code>PostgreSQL</code> to enable us to connect to a PostgreSQL database</li>
</ul>
Once we click on &quot;Generate&quot;, we will receive a zip file containing the project and we can start implementing the web application that requires a database. Once we unpack this zip file into our working folder, we can test that it is ready to be worked on by running the command:
<pre><code class="hljs">$ mvn spring-boot:run
</code></pre>
<h3 id="implementation">Implementation</h3>
Let us modify our <code>application.properties</code> to include our database details:
<pre><code class="hljs"># Database Properties
spring.datasource.url=jdbc:postgresql://localhost:5432/phonesdemo
spring.datasource.username=postgres
spring.datasource.password=

# Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL92Dialect

# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto=update
</code></pre>
Hibernate provides an H2 Console that we can use to check the status of the database and even perform data entry via a user interface. We enable it by adding the following line to our <code>application.properties</code>:
<pre><code class="hljs">spring.h2.console.enabled=true
</code></pre>
Then we fire up our application and navigate to <code>http://localhost:8080/h2-console</code> to test if everything is working. We get a page where we can test if the connection to our database works:
<img src="https://stackabuse.s3.amazonaws.com/media/implementing-hibernate-with-spring-boot-and-postgresql-2.png" alt="h2_database">
For the saved settings dropdown, we are going to choose Generic PostgreSQL and we are also going to update the JDBC URL to match the name of our test database. Then we fill in our database username and password and click on &quot;Test Connection&quot; just to make sure that our Spring Application can connect to our database. If everything is well set up, we get a success message.
If we click on &quot;Connect&quot;, we get this page:
<img src="https://stackabuse.s3.amazonaws.com/media/implementing-hibernate-with-spring-boot-and-postgresql-3.png" alt="h2_console_full">
Here we can navigate our database and even perform SQL queries.
The API we will be building will be used to store and manipulate phones and their attributes such as name and operating system. With our database in place and connected, let us create an entity class (<code>Phone.java</code>) that will map our object's attributes to the database and enable us to perform CRUD operations in the database:
<pre><code class="hljs">// Imports truncated for brevity, refer to GitHub link below for the full code

@Entity
@Table(name = &quot;phones&quot;)
@EntityListeners(AuditingEntityListener.class)
public class Phone {
 @Id
 @GeneratedValue(strategy = GenerationType.AUTO)
 private long id; // Each phone will be given an auto-generated unique identifier when stored

 @Column(name = &quot;phone_name&quot;, nullable = false)
 private String phoneName; // Save the name of the phone

 @Column(name = &quot;os&quot;, nullable = false)
 private String os; // Save the operating system running in the phone
 
 // Standard getters and setters
}
</code></pre>
The <code>@Entity</code> annotation tells Hibernate that this class represents an entity that should be persisted.
The <code>@Table</code> annotation is used for naming the table. If this annotation is omitted, the table will simply use the name of the class/entity.
Similarly the <code>@Column</code> annotations can also be omitted, but the database columns will use the field names as they are, and sometimes this isn't the preferred behavior since your column names might be snake case and field names are camel case, for example.
When we save this file and restart our server and check our database, there will be a new table called <code>phones</code> and the columns of <code>id</code>, <code>phone_name</code>, and <code>os</code> will be present.
There will be no data but this is Hibernate at work, if the table specified in the entity class does not exist, Hibernate will create it for us.
Let us now implement the controller to help us perform operations on our database through an API:
<pre><code class="hljs">@RestController
@RequestMapping(&quot;/api/v1&quot;)
public class PhoneController {
 @Autowired
 private PhoneRepository phoneRepository;

 // GET method to fetch all phones
 @GetMapping(&quot;/phones&quot;)
 public List&lt;Phone&gt; getAllPhones() {
 return phoneRepository.findAll();
 }

 // GET method to fetch phone by Id
 @GetMapping(&quot;/phones/{id}&quot;)
 public ResponseEntity&lt;Phone&gt; getPhoneById(@PathVariable(value = &quot;id&quot;) Long phoneId)
 throws Exception {
 Phone phone = phoneRepository.findById(phoneId)
 .orElseThrow(() -&gt; new Exception(&quot;Phone &quot; + phoneId + &quot; not found&quot;));
 return ResponseEntity.ok().body(phone);
 }
 
 // POST method to create a phone
 @PostMapping(&quot;/phones&quot;)
 public Phone createPhone(@Valid @RequestBody Phone phone) {
 return phoneRepository.save(phone);
 }
 
 // PUT method to update a phone&#x27;s details
 @PutMapping(&quot;/phones/{id}&quot;)
 public ResponseEntity&lt;Phone&gt; updatePhone(
 @PathVariable(value=&quot;id&quot;) Long phoneId, @Valid @RequestBody Phone phoneDetails
 ) throws Exception {
 Phone phone = phoneRepository.findById(phoneId)
 .orElseThrow(() -&gt; new Exception(&quot;Phone &quot; + phoneId + &quot; not found&quot;));

 phone.setPhoneName(phoneDetails.getPhoneName());
 phone.setOs(phoneDetails.getOs());

 final Phone updatedPhone = phoneRepository.save(phone);
 return ResponseEntity.ok(updatedPhone);
 }
 
 // DELETE method to delete a phone
 @DeleteMapping(&quot;/phone/{id}&quot;)
 public Map&lt;String, Boolean&gt; deletePhone(@PathVariable(value=&quot;id&quot;) Long phoneId) throws Exception {
 Phone phone = phoneRepository.findById(phoneId)
 .orElseThrow(() -&gt; new Exception(&quot;Phone &quot; + phoneId + &quot; not found&quot;));

 phoneRepository.delete(phone);
 Map&lt;String, Boolean&gt; response = new HashMap&lt;&gt;();
 response.put(&quot;deleted&quot;, Boolean.TRUE);
 return response;
 }
}
</code></pre>
In our controller class, we annotate our class by <code>@RestController</code> to indicate that this is the request handler class that will be handling the REST functionality for our API. We then define methods to handle each of the four RESTful operations: <code>GET</code>, <code>POST</code>, <code>PUT</code>, and <code>DELETE</code>. These methods will provide an interface for us to interact with our API and manage data.
Our API, in turn, will utilize Hibernate to reflect our operations on the said data to our database.
Let us start by creating a single phone through our API:
<img src="https://stackabuse.s3.amazonaws.com/media/implementing-hibernate-with-spring-boot-and-postgresql-4.png" alt="phone_create">
We get the response from our API, but let us check the database using the H2 Console to confirm:
<img src="https://stackabuse.s3.amazonaws.com/media/implementing-hibernate-with-spring-boot-and-postgresql-5.png" alt="database_confirm">
As you can see, from the screen-shot above, to fetch the data in our database, we use the SQL command <code>SELECT * FROM phones</code>. In order to achieve the same in our code through the ORM, it is as simple as using the line:
<pre><code class="hljs">phoneRepository.findAll();
</code></pre>
This is more friendly and familiar to us since it is achieved in the same language that we are using while implementing the rest of our project.
<h2 id="conclusion">Conclusion</h2>
We have successfully created a phone object and saved its attributes to our PostgreSQL database by using Hibernate in our Spring Boot API. We can add more phones, delete phones, and update phone data by interacting with the API and Hibernate will reflect the changes in our database.
Hibernate has made it easier for us to interact with a database from our Spring Boot application and manage our data. The development time has also been significantly reduced since we do not have to write the SQL commands to manage the data ourselves.
The source code for this project is available <a rel="nofollow" target="_blank" href="https://github.com/ro6ley/java-hibernate-example">here on GitHub</a>.

Robley Gori

Implementing Hibernate with Spring Boot and PostgreSQL

Olivera Popović

Common String Operations in Java

Muhammad Junaid Khalid

Python's Bokeh Library for Interactive Data Visualization

Usman Malik

Python for NLP: Creating TF-IDF Model from Scratch

In this article, we'll examine how to print a string without a newline character using Python.
In Python, the built-in <code>print</code> function is used to print content to the standard output, which is usually the console. By default, the print function adds a newline character at the end of the printed content, so the next output by the program occurs on the next line.
Try running this code to see an example:
<pre><code class="hljs">print(&#x27;Banana&#x27;)
print(&#x27;pudding.&#x27;)
</code></pre>
Output:
<pre><code class="hljs">Banana
pudding.
</code></pre>
As expected, the output of each <code>print</code> statement is shown on its own line.
However, in some cases we may want to output multiple strings on the same line using separate <code>print</code> statements. There are a few ways to prevent Python from adding the newline character when using the <code>print</code> function, depending on whether we are using Python 2.x or Python 3.x.
For example, this kind of functionality is useful for when you're developing a REPL or any command line application that takes input from the user, and you don't want the prompt and input text to be on different lines.
For Python 2.x, we can simply add a comma after the print function call, which will terminate the printed string with a space instead of a newline character:
<pre><code class="hljs">print(&#x27;Banana&#x27;),
print(&#x27;pudding.&#x27;)
</code></pre>
Output:
<pre><code class="hljs">Banana pudding.
</code></pre>
In Python 3.x, we can use the <code>end</code> keyword argument in the <code>print</code> method to specify the termination character for the printed string:
<pre><code class="hljs">print(&#x27;Banana&#x27;, end=&#x27; &#x27;)
print(&#x27;pudding.&#x27;)
</code></pre>
Output:
<pre><code class="hljs">Banana pudding.
</code></pre>
So in this case, a space is used as the &quot;termination&quot; character, which results in the printed strings to be on the same line and only separated by a single space.
<h2 id="abouttheauthor">About the Author</h2>
This article was written by Jacob Stopak, a software consultant and developer with a passion for helping others improve their lives through code. Jacob is the creator of <a rel="nofollow" target="_blank" href="https://initialcommit.com/codecard">Code Card</a> - a convenient tool for developers to look up, copy, and paste common code snippets.

Jacob Stopak

Python: Print without Newline

In this article, we'll examine how to use the <code>assert</code> statement in Python.
In Python, the <code>assert</code> statement is used to validate whether or not a condition is true, using the syntax:
<pre><code class="hljs">assert &lt;condition&gt;
</code></pre>
If the condition evaluates to <code>True</code>, the program continues executing as if nothing out of the ordinary happened. However, if the condition evaluates to <code>False</code>, the program terminates with an <code>AssertionError</code>.
<pre><code class="hljs">&gt;&gt;&gt; assert True
</code></pre>
Nothing happens when the code above is executed, since the condition evaluates to <code>True</code>. Alternatively, the condition in the example below evaluates to <code>False</code>:
<pre><code class="hljs">&gt;&gt;&gt; assert False
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
AssertionError
</code></pre>
For added clarity, we can add a custom error message to the assertion output as follows:
<pre><code class="hljs">&gt;&gt;&gt; assert False, &quot;This is a custom assertion message!&quot;
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
AssertionError: This is a custom assertion message!
</code></pre>
The assert statement is useful when we want to check that a variable in our code assumes the correct value and terminate the program if it doesn't. This helps prevent silent failure modes, which can occur if the program continues executing with erroneous values and can be difficult to debug.
Here is an example of an assert statement with a more meaningful condition. Let's assume that we want to ensure that a flag variable input by the user has been set to one of several correct values. If not, we will terminate execution of the program. We can do that as follows:
<pre><code class="hljs">&gt;&gt;&gt; flag = input(&quot;Enter a flag (y/n): &quot;)
Enter a flag (y/n): y
&gt;&gt;&gt; assert flag == &quot;y&quot; or flag == &quot;n&quot;, &quot;Invalid flag, must be &#x27;y&#x27; or &#x27;n&#x27;&quot;
</code></pre>
In the above example, the user inputted a valid value for the flag, so no errors are thrown. However, in the example below, an assertion is thrown due to invalid user input:
<pre><code class="hljs">&gt;&gt;&gt; flag = input(&quot;Enter a flag (y/n): &quot;)
Enter a flag (y/n): b
&gt;&gt;&gt; assert flag == &quot;y&quot; or flag == &quot;n&quot;, &quot;Invalid flag, must be &#x27;y&#x27; or &#x27;n&#x27;&quot;
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
AssertionError: Invalid flag, must be &#x27;y&#x27; or &#x27;n&#x27;
</code></pre>
When used in this way, your code should catch the thrown error and show the custom error message as output to the user so they can correct their response. This has a huge number of uses in Python programs, whether it be asserting inputs to an API route or checking that a downloaded resource contains the proper information.
Another important point to note is that when you're running the Python interpreter or Python script from the command line, the <code>–O</code> flag can be used to run the program without enforcing any assertions. This will ignore the assert statements by turning off the Python interpreter's debug mode. Be careful using this as it can lead to your code accepting invalid or dangerous inputs.
<h2 id="abouttheauthor">About the Author</h2>
This article was written by Jacob Stopak, a software consultant and developer with a passion for helping others improve their lives through code. Jacob is the creator of <a rel="nofollow" target="_blank" href="https://initialcommit.com/codecard">Code Card</a> - a convenient tool for developers to look up, copy, and paste common code snippets.

The Python Assert Statement

Nicholas Samuel

The Python Help System

This is the 13th article in my series of articles on Python for NLP. In the <a target="_blank" href="/python-for-nlp-creating-a-rule-based-chatbot/">previous article</a>, we saw how to create a simple rule-based chatbot that uses <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Cosine_similarity">cosine similarity</a> between the <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf">TF-IDF</a> vectors of the words in the corpus and the user input, to generate a response. The TF-IDF model was basically used to convert word to numbers.
In this article, we will study another very useful model that converts text to numbers i.e. the Bag of Words (BOW).
Since most of the statistical algorithms, e.g machine learning and deep learning techniques, work with numeric data, therefore we have to convert text into numbers. Several approaches exist in this regard. However, the most famous ones are Bag of Words, TF-IDF, and word2vec. Though several libraries exist, such as Scikit-Learn and NLTK, which can implement these techniques in one line of code, it is important to understand the working principle behind these word embedding techniques. The best way to do so is to implement these techniques from scratch in Python and this is what we are going to do today.
In this article, we will see how to implement the Bag of Words approach from scratch in Python. In the next article, we will see how to implement the TF-IDF approach from scratch in Python.
Before coding, let's first see the theory behind the bag of words approach.
<h2 id="theorybehindbagofwordsapproach">Theory Behind Bag of Words Approach</h2>
To understand the bag of words approach, let's first start with the help of an example.
Suppose we have a corpus with three sentences:
<ul>
<li>&quot;I like to play football&quot;</li>
<li>&quot;Did you go outside to play tennis&quot;</li>
<li>&quot;John and I play tennis&quot;</li>
</ul>
Now if we have to perform text classification, or any other task, on the above data using statistical techniques, we can not do so since statistical techniques work only with numbers. Therefore we need to convert these sentences into numbers.
<h3 id="step1tokenizethesentences">Step 1: Tokenize the Sentences</h3>
The first step in this regard is to convert the sentences in our corpus into tokens or individual words. Look at the table below:
<table class="table table-striped">
 <tr>
 <th>Sentence 1</th>
 <th>Sentence 2</th>
 <th>Sentence 3</th>
 </tr>
 <tr>
 <td>I</td>
 <td>Did</td>
 <td>John</td>
 </tr>
 <tr>
 <td>like</td>
 <td>you</td>
 <td>and</td>
 </tr>
 <tr>
 <td>to </td>
 <td>go</td>
 <td>I</td>
 </tr>
 <tr>
 <td>play</td>
 <td>outside</td>
 <td>play</td>
 </tr>
 <tr>
 <td>football</td>
 <td>to</td>
 <td>tennis</td>
 </tr>
 <tr>
 <td></td>
 <td>play</td>
 <td></td>
 </tr>
 <tr>
 <td></td>
 <td>tennis</td>
 <td></td>
 </tr>
</table>
<h3 id="step2createadictionaryofwordfrequency">Step 2: Create a Dictionary of Word Frequency</h3>
The next step is to create a dictionary that contains all the words in our corpus as keys and the frequency of the occurrence of the words as values. In other words, we need to create a histogram of the words in our corpus. Look at the following table:
<table class="table table-striped">
 <tr>
 <th>Word</th>
 <th>Frequency</th>
 </tr>
 <tr>
 <td>I</td>
 <td>2</td>
 </tr>
 <tr>
 <td>like</td>
 <td>1</td>
 </tr>
 <tr>
 <td>to</td>
 <td>2</td>
 </tr>
 <tr>
 <td>play</td>
 <td>3</td>
 </tr>
 <tr>
 <td>football</td>
 <td>1</td>
 </tr>
 <tr>
 <td>Did</td>
 <td>1</td>
 </tr>
 <tr>
 <td>you</td>
 <td>1</td>
 </tr>
 <tr>
 <td>go</td>
 <td>1</td>
 </tr>
 <tr>
 <td>outside</td>
 <td>1</td>
 </tr>
 <tr>
 <td>tennis</td>
 <td>2</td>
 </tr>
 <tr>
 <td>John</td>
 <td>1</td>
 </tr>
 <tr>
 <td>and</td>
 <td>1</td>
 </tr>
</table>
In the table above, you can see each word in our corpus along with its frequency of occurrence. For instance, you can see that since the word <code>play</code> occurs three times in the corpus (once in each sentence) its frequency is 3.
In our corpus, we only had three sentences, therefore it is easy for us to create a dictionary that contains all the words. In the real world scenarios, there will be millions of words in the dictionary. Some of the words will have a very small frequency. The words with very small frequency are not very useful, hence such words are removed. One way to remove the words with less frequency is to sort the word frequency dictionary in the decreasing order of the frequency and then filter the words having a frequency higher than a certain threshold.
Let's sort our word frequency dictionary:
<table class="table table-striped">
 <tr>
 <th>Word</th>
 <th>Frequency</th>
 </tr>
 <tr>
 <td>play</td>
 <td>3</td>
 </tr>
 <tr>
 <td>tennis</td>
 <td>2</td>
 </tr>
 <tr>
 <td>to</td>
 <td>2</td>
 </tr>
 <tr>
 <td>I</td>
 <td>2</td>
 </tr>
 <tr>
 <td>football</td>
 <td>1</td>
 </tr>
 <tr>
 <td>Did</td>
 <td>1</td>
 </tr>
 <tr>
 <td>you</td>
 <td>1</td>
 </tr>
 <tr>
 <td>go</td>
 <td>1</td>
 </tr>
 <tr>
 <td>outside</td>
 <td>1</td>
 </tr>
 <tr>
 <td>like</td>
 <td>1</td>
 </tr>
 <tr>
 <td>John</td>
 <td>1</td>
 </tr>
 <tr>
 <td>and</td>
 <td>1</td>
 </tr>
</table>
<h3 id="step3creatingthebagofwordsmodel">Step 3: Creating the Bag of Words Model</h3>
To create the bag of words model, we need to create a matrix where the columns correspond to the most frequent words in our dictionary where rows correspond to the document or sentences.
Suppose we filter the 8 most occurring words from our dictionary. Then the document frequency matrix will look like this:
<table class="table table-striped">
 <tr>
 <th></th>
 <th>Play</th>
 <th>Tennis</th>
 <th>To</th>
 <th>I</th>
 <th>Football</th>
 <th>Did </th>
 <th>You</th>
 <th>go</th>
 </tr>
 <tr>
 <td>Sentence 1</td>
 <td>1</td>
 <td>0</td>
 <td>1</td>
 <td>1</td>
 <td>1</td>
 <td>0</td>
 <td>0</td>
 <td>0</td>
 </tr>
 <tr>
 <td>Sentence 2</td>
 <td>1</td>
 <td>1</td>
 <td>1</td>
 <td>0</td>
 <td>0</td>
 <td>1</td>
 <td>1</td>
 <td>1</td>
 </tr>
 <tr>
 <td>Sentence 3</td>
 <td>1</td>
 <td>1</td>
 <td>0</td>
 <td>1</td>
 <td>0</td>
 <td>0</td>
 <td>0</td>
 <td>0</td>
 </tr>
</table>
It is important to understand how the above matrix is created. In the above matrix, the first row corresponds to the first sentence. In the first, the word &quot;play&quot; occurs once, therefore we added 1 in the first column. The word in the second column is &quot;Tennis&quot;, it doesn't occur in the first sentence, therefore we added a 0 in the second column for sentence 1. Similarly, in the second sentence, both the words &quot;Play&quot; and &quot;Tennis&quot; occur once, therefore we added 1 in the first two columns. However, in the fifth column, we add a 0, since the word &quot;Football&quot; doesn't occur in the second sentence. In this way, all the cells in the above matrix are filled with either 0 or 1, depending upon the occurrence of the word. Final matrix corresponds to the bag of words model.
In each row, you can see the numeric representation of the corresponding sentence. For instance, the first row shows the numeric representation of Sentence 1. This numeric representation can now be used as input to the statistical models.
Enough of the theory, let's implement our very own bag of words model from scratch.
<h2 id="bagofwordsmodelinpython">Bag of Words Model in Python</h2>
The first thing we need to create our Bag of Words model is a dataset. In the previous section, we manually created a bag of words model with three sentences. However, real-world datasets are huge with millions of words. The best way to find a random corpus is Wikipedia.
In the first step, we will scrape the Wikipedia article on <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Natural_language_processing">Natural Language Processing</a>. But first, let's import the required libraries:
<pre><code class="hljs">import nltk 
import numpy as np 
import random 
import string

import bs4 as bs 
import urllib.request 
import re 
</code></pre>
As we did in the previous article, we will be using the <a rel="nofollow" target="_blank" href="https://beautiful-soup-4.readthedocs.io/en/latest/">Beautifulsoup4</a> library to parse the data from Wikipedia. Furthermore, <a target="_blank" href="/using-regex-for-text-manipulation-in-python/">Python's regex library</a>, <code>re</code>, will be used for some preprocessing tasks on the text.
Next, we need to scrape the Wikipedia article on natural language processing.
<pre><code class="hljs">raw_html = urllib.request.urlopen(&#x27;https://en.wikipedia.org/wiki/Natural_language_processing&#x27;) 
raw_html = raw_html.read()

article_html = bs.BeautifulSoup(raw_html, &#x27;lxml&#x27;)

article_paragraphs = article_html.find_all(&#x27;p&#x27;)

article_text = &#x27;&#x27;

for para in article_paragraphs: 
 article_text += para.text
</code></pre>
In the script above, we import the raw HTML for the Wikipedia article. From the raw HTML, we filter the text within the paragraph text. Finally, we create a complete corpus by concatenating all the paragraphs.
The next step is to split the corpus into individual sentences. To do so, we will use the <code>sent_tokenize</code> function from the NLTK library.
<pre><code class="hljs">corpus = nltk.sent_tokenize(article_text)
</code></pre>
Our text contains punctuations. We don't want punctuations to be the part of our word frequency dictionary. In the following script, we first convert our text into lower case and then will remove the punctuation from our text. Removing punctuation can result in multiple empty spaces. We will remove the empty spaces from the text using regex.
Look at the following script:
<pre><code class="hljs">for i in range(len(corpus )):
 corpus [i] = corpus [i].lower()
 corpus [i] = re.sub(r&#x27;\W&#x27;,&#x27; &#x27;,corpus [i])
 corpus [i] = re.sub(r&#x27;\s+&#x27;,&#x27; &#x27;,corpus [i])
</code></pre>
In the script above, we iterate through each sentence in the corpus, convert the sentence to lower case, and then remove the punctuation and empty spaces from the text.
Let's find out the number of sentences in our corpus.
<pre><code class="hljs">print(len(corpus))
</code></pre>
The output shows 49.
Let's print one sentence from our corpus:
<pre><code class="hljs">print(corpus[30])
</code></pre>
Output:
<pre><code class="hljs">in the 2010s representation learning and deep neural network style machine learning methods became widespread in natural language processing due in part to a flurry of results showing that such techniques 4 5 can achieve state of the art results in many natural language tasks for example in language modeling 6 parsing 7 8 and many others 
</code></pre>
You can see that the text doesn't contain any special character or multiple empty spaces.
Now we have our own corpus. The next step is to tokenize the sentences in the corpus and create a dictionary that contains words and their corresponding frequencies in the corpus. Look at the following script:
<pre><code class="hljs">wordfreq = {}
for sentence in corpus:
 tokens = nltk.word_tokenize(sentence)
 for token in tokens:
 if token not in wordfreq.keys():
 wordfreq[token] = 1
 else:
 wordfreq[token] += 1
</code></pre>
In the script above we created a dictionary called <code>wordfreq</code>. Next, we iterate through each sentence in the corpus. The sentence is tokenized into words. Next, we iterate through each word in the sentence. If the word doesn't exist in the <code>wordfreq</code> dictionary, we will add the word as the key and will set the value of the word as 1. Otherwise, if the word already exists in the dictionary, we will simply increment the key count by 1.
If you are executing the above in the <a rel="nofollow" target="_blank" href="https://www.spyder-ide.org/">Spyder</a> editor like me, you can go the variable explorer on the right and click <code>wordfreq</code> variable. You should see a dictionary like this:
<img class="img-responsive" src="https://stackabuse.s3.amazonaws.com/media/python-nlp-creating-bag-words-model-from-scratch-1.png">
You can see words in the &quot;Key&quot; column and their frequency of occurrences in the &quot;Value&quot; column.
As I said in the theory section, depending upon the task at hand, not all of the words are useful. In huge corpora, you can have millions of words. We can filter the most frequently occurring words. Our corpus has 535 words in total. Let us filter down to the 200 most frequently occurring words. To do so, we can make use of Python's <code>heap</code> library.
Look at the following script:
<pre><code class="hljs">import heapq
most_freq = heapq.nlargest(200, wordfreq, key=wordfreq.get)
</code></pre>
Now our <code>most_freq</code> list contains 200 most frequently occurring words along with their frequency of occurrence.
The final step is to convert the sentences in our corpus into their corresponding vector representation. The idea is straightforward, for each word in the <code>most_freq</code> dictionary if the word exists in the sentence, a 1 will be added for the word, else 0 will be added.
<pre><code class="hljs">sentence_vectors = []
for sentence in corpus:
 sentence_tokens = nltk.word_tokenize(sentence)
 sent_vec = []
 for token in most_freq:
 if token in sentence_tokens:
 sent_vec.append(1)
 else:
 sent_vec.append(0)
 sentence_vectors.append(sent_vec)
</code></pre>
In the script above we create an empty list <code>sentence_vectors</code> which will store vectors for all the sentences in the corpus. Next, we iterate through each sentence in the corpus and create an empty list <code>sent_vec</code> for the individual sentences. Similarly, we also tokenize the sentence. Next, we iterate through each word in the <code>most_freq</code> list and check if the word exists in the tokens for the sentence. If the word is a part of the sentence, 1 is appended to the individual sentence vector <code>sent_vec</code>, else 0 is appended. Finally, the sentence vector is added to the list <code>sentence_vectors</code> which contains vectors for all the sentences. Basically, this <code>sentence_vectors</code> is our bag of words model.
However, the bag of words model that we saw in the theory section was in the form of a matrix. Our model is in the form of a list of lists. We can convert our model into matrix form using this script:
<pre><code class="hljs">sentence_vectors = np.asarray(sentence_vectors)
</code></pre>
Basically, in the following script, we converted our list into a two-dimensional numpy array using <code>asarray</code> function. Now if you open the <code>sentence_vectors</code> variable in the variable explorer of the Spyder editor, you should see the following matrix:
<img class="img-responsive" src="https://stackabuse.s3.amazonaws.com/media/python-nlp-creating-bag-words-model-from-scratch-2.png">
You can see the Bag of Words model containing 0 and 1.
<h2 id="conclusion">Conclusion</h2>
Bag of Words model is one of the three most commonly used word embedding approaches with TF-IDF and Word2Vec being the other two.
In this article, we saw how to implement the Bag of Words approach from scratch in Python. The theory of the approach has been explained along with the hands-on code to implement the approach. In the <a target="_blank" href="/python-for-nlp-creating-tf-idf-model-from-scratch/">next article</a>, we will see how to implement the TF-IDF approach from scratch in Python.

Python for NLP: Creating Bag of Words Model from Scratch

Daniele Paliotta

Introduction to GANs with Python and TensorFlow