<h2 id="introduction">Introduction</h2>
It's not an overstatement to say that information and data runs the world. Almost any application, from social media and e-commerce websites to simple time tracker and drawing apps, relies on the very basic and fundamental task of storing and retrieving data in order to run as expected.
Amazon's <a rel="nofollow" target="_blank" href="https://aws.amazon.com/rds/">Relational Database Service</a> (RDS) provides an easy way to get a database set up in the cloud using any of a wide range of relational database technologies. In this article, we're going to set up a database on RDS, and store data on it with a Node application.
<h2 id="prerequisites">Prerequisites</h2>
<h3 id="amazonwebservices">Amazon Web Services</h3>
<a rel="nofollow" target="_blank" href="https://aws.amazon.com/">Amazon Web Services</a> (AWS) provides a collection of tools for building applications in the cloud. As RDS is an AWS service, we'll need to get set up with an AWS account.
AWS has a free tier for a lot of awesome stuff, and RDS is no exception - you get to use 750 hours (31 days) a month of <code>db.t2.micro</code> database usage for free for 12 months.
<h3 id="postman">Postman</h3>
<a target="_blank" href="/getting-started-with-postman/">Postman</a> is a useful tool for creating and sending requests. We'll be using this in the Node half of the article in order to test the code we've written.
Postman is optional, and really, you can use any tool to test out the endpoint, even your Internet browser.
<h2 id="settingupanrdsinstance">Setting Up an RDS Instance</h2>
First, we're going to create our RDS instance cluster. Head to AWS and <a rel="nofollow" target="_blank" href="https://signin.aws.amazon.com/">log in</a>.
Once you're logged in, click on 'Services' in the top left, and then search for 'RDS'. You'll be presented with a page that looks something like this:
<img src="https://stackabuse.s3.amazonaws.com/media/working-with-aws-rds-with-node-and-express-1.png" alt="setting_up_rds_instance">
On the menu on the left, select 'Databases'. This would normally display a list of RDS instance clusters that we've created, but we don't have any yet.
To create one, click the orange 'Create database' button. You should be presented with a page that looks like:
<img src="https://stackabuse.s3.amazonaws.com/media/working-with-aws-rds-with-node-and-express-2.png" alt="create_rds_database">
AWS has recently introduced an 'Easy create' method for creating new RDS instances, so let's use that.
Under 'Engine type' we'll use 'Amazon Aurora', which is Amazon's own database engine optimized for RDS. For the edition, we'll leave this set to 'Amazon Aurora with MySQL 5.6 compatibility'.
Under 'DB instance size' select the 'Dev/Test' option - this is a less powerful (and cheaper) instance type, but is still more than enough for what we need it for.
The 'DB cluster identifier' is the name of the database cluster that we're creating. Let's call ours <code>my-node-database</code> for now.
For the master username, leave it as <code>admin</code>. Finally, we have the option to have a master password generated automatically. For the ease of this tutorial, let's set our own.
Make sure it's secure as this is the master username and password!
Finally, scroll down and click 'Create database'. It takes a few minutes to fully provision an RDS instance:
<img src="https://stackabuse.s3.amazonaws.com/media/working-with-aws-rds-with-node-and-express-3.png" alt="create_rds_database">
Before getting started on our Node application, we need to make sure we can connect to the instance. Select the instance you just created (it'll be the option that ends in <code>instance-1</code>) and take a note of the value under 'Endpoint'.
On the right-hand side, under 'VPC security groups', click the link - this will take you to the security group that's been set up for the database. Security groups are essentially firewall rules as to who is and isn't allowed to make connections to a resource.
Currently, this one is set to only allow connections from resources that have the same security group.
Selecting the 'Actions' drop down at the top, navigate to 'Edit inbound rules'. In this dialog, click 'Add rule'. For the new rule, under 'Type', select 'All traffic'. Under 'Source', select 'Anywhere'.
You should end up with something that looks like this:
<img src="https://stackabuse.s3.amazonaws.com/media/working-with-aws-rds-with-node-and-express-4.png" alt="edit_inbound_rules">
Head back to RDS, and 'Modify' the instance. Here, at the 'Network &amp; Security' section, under 'Public accessibility', we want to select 'Yes' - this allows us to connect to the database from our machines - otherwise we would need to <a target="_blank" href="/deploying-node-js-apps-to-aws-ec2-with-docker/">put our application on an EC2 instance</a>, which isn't the focus of this article.
Your RDS instance should now be ready to go! Let's write some code to interact with it.
<h2 id="nodeapplication">Node Application</h2>
In order to interact with our application, we're going to create a very simple API that allows us to store user profiles via Express.js. Before we do that, we need to create a table inside our RDS instance to store data in.
Let's create a folder, move into it and initialize a blank Node.js application with the default configuration:
<pre><code class="hljs">$ mkdir node-rds
$ cd node-rds
$ npm init -y
</code></pre>
Then, let's install the required dependencies:
<pre><code class="hljs">$ npm install express --save
$ npm install mysql --save
</code></pre>
And finally, we want to create two JavaScript files - one of them will be our Express app, the other will be a single-use script to create a table in our database:
<pre><code class="hljs">$ touch index.js
$ touch dbseed.js
</code></pre>
<h3 id="tablecreationscript">Table Creation Script</h3>
Let's start off with the <code>dbseed.js</code> file:
<pre><code class="hljs">const mysql = require(&#x27;mysql&#x27;);

const con = mysql.createConnection({
 host: &quot;&lt;DB_ENDPOINT&gt;&quot;,
 user: &quot;admin&quot;,
 password: &quot;&lt;DB_PASSWORD&gt;&quot;
});

con.connect(function(err) {
 if (err) throw err;
 console.log(&quot;Connected!&quot;);
 con.end();
});
</code></pre>
Make sure to swap out <code>&lt;DB_ENDPOINT&gt;</code> for the endpoint that we noted down earlier, and fill in the password. What this piece of code will do is attempt to connect to the database - if it succeeds, it'll run an anonymous function that logs 'Connected!', and then immediately close the connection.
We can quickly check to see if it's properly set up by running:
<pre><code class="hljs">$ node dbseed.js
</code></pre>
The app should return 'Connected!'. If it doesn't, there's likely an issue with the security settings - go back to the RDS set-up and make sure you've done everything correctly.
Now that we know that we can definitely connect to our database, we need to create a table. Let's modify our anonymous function:
<pre><code class="hljs">con.connect(function(err) {
 if (err) throw err;

 con.query(&#x27;CREATE DATABASE IF NOT EXISTS main;&#x27;);
 con.query(&#x27;USE main;&#x27;);
 con.query(&#x27;CREATE TABLE IF NOT EXISTS users(id int NOT NULL AUTO_INCREMENT, username varchar(30), email varchar(255), age int, PRIMARY KEY(id));&#x27;, function(error, result, fields) {
 console.log(result);
 });
 con.end();
});
</code></pre>
The expected output should look something like:
<pre><code class="hljs">OkPacket {
 fieldCount: 0,
 affectedRows: 0,
 insertId: 0,
 serverStatus: 2,
 warningCount: 0,
 message: &#x27;&#x27;,
 protocol41: true,
 changedRows: 0
}
</code></pre>
Now that we've got a table to work with, let's set up the Express app to insert and retrieve data from our database.
<h3 id="insertuserendpoint">Insert User Endpoint</h3>
Let's write a <code>POST</code> request for user creation in our <code>index.js</code> file:
<pre><code class="hljs">app.post(&#x27;/users&#x27;, (req, res) =&gt; {
 if (req.query.username &amp;&amp; req.query.email &amp;&amp; req.query.age) {
 console.log(&#x27;Request received&#x27;);
 con.connect(function(err) {
 con.query(`INSERT INTO main.users (username, email, age) VALUES (&#x27;${req.query.username}&#x27;, &#x27;${req.query.email}&#x27;, &#x27;${req.query.age}&#x27;)`, function(err, result, fields) {
 if (err) res.send(err);
 if (result) res.send({username: req.query.username, email: req.query.email, age: req.query.age});
 if (fields) console.log(fields);
 });
 });
 } else {
 console.log(&#x27;Missing a parameter&#x27;);
 }
});
</code></pre>
What this code does is pretty straightforward - it packs the information from the <code>POST</code> request and inserts it into a query. This query will create a user in the RDS database and log the results.
Let's check if it works:
<pre><code class="hljs">$ node index.js
</code></pre>
In Postman we want to create a <code>POST</code> request to our server. Let's input the required parameters and send the request:
<img src="https://stackabuse.s3.amazonaws.com/media/working-with-aws-rds-with-node-and-express-5.png" alt="postman_request_to_rds">
We should be getting the same data back, which means that it's working correctly and our endpoint has inserted a user.
<h3 id="getusersendpoint">Get Users Endpoint</h3>
Let's devise a simple <code>GET</code> endpoint for a more user-friendly way to check for results. Below the <code>POST</code> request handler, let's make another handler:
<pre><code class="hljs">app.get(&#x27;/users&#x27;, (req, res) =&gt; {
 con.connect(function(err) {
 con.query(`SELECT * FROM main.users`, function(err, result, fields) {
 if (err) res.send(err);
 if (result) res.send(result);
 });
 });
});
</code></pre>
In your browser, navigate to <code>localhost:3000/users</code> and you should be presented with all the inserted users.
<h2 id="whatnext">What Next?</h2>
<h3 id="secureyourdatabase">Secure your Database</h3>
Earlier, we made our RDS instance publicly accessible, and with a completely open security group. You could <a target="_blank" href="/deploying-node-js-apps-to-aws-ec2-with-docker/">get set up on EC2</a> and begin to tighten the security on your instance by removing public access and locking down your security group.
You should also take a look at the codebase as well - it's functional, but might be vulnerable to attacks such as SQL injection - it's worth spending time to make sure that users can't perform actions on your database that you don't intend.
<h3 id="experimentwithreadwrite">Experiment with Read/Write</h3>
If you dive a bit deeper into RDS, you can set up instances specifically to read or write from - this can help with securing your instance, but also if you're dealing with lots of database operations, this can help optimize your database's performance.
<h3 id="trydifferentconfigurations">Try Different Configurations</h3>
We used AWS' 'Easy create' to make our database - it's worth going back and trying out the various configurations that you could set RDS up with to meet your particular needs.
<h3 id="terminateyourrdsinstance">Terminate your RDS Instance!</h3>
If you're not planning on continuing to use your RDS instance, make sure to terminate it! Otherwise you'll rack up a hefty bill within a month.
<h2 id="conclusion">Conclusion</h2>
You've created an RDS instance and populated it with data through a Node/Express endpoint. This is a great springboard to leap into more complex web applications that store and serve data for its users, and with data rising meteorically as a valuable commodity this is a good time to think on what sort of data you want to operate with.
To end, remember that any data you store is your duty to protect - always make sure you understand how to build a secure application before you start putting people's personal details in it!!

Josh Simpson

Amazon's RDS provides an easy way to get a database set up with a wide range of relational database technologies. In this article, we'll be using AWS RDS with Node.js.

Using AWS RDS with Node.js and Express.js

Janith Kasun

JWT are very popular for handling HTTP authentication and authorization, which we'll be doing through Node.js and Express in this article.

Authentication and Authorization with JWTs in Express.js

<h2 id="introduction">Introduction</h2>
<a rel="nofollow" target="_blank" href="https://aws.amazon.com">Amazon Web Services</a> (AWS) offers a wide range of on-demand and reliable computing services. It does this by hiding the infrastructure management and its complexities, thus simplifying the process of provisioning and running cloud infrastructure.
AWS allows IT companies and developers to focus on creating better solutions for their products with scalable and on demand web services which make it easier to increase or decrease any resource as the application evolves over time.
One of these products is the Simple Storage Service, or S3, which allows you to cheaply store files at scale.
<h3 id="s3bucket">S3 Bucket</h3>
Amazon's <a rel="nofollow" target="_blank" href="https://aws.amazon.com/s3">Simple Storage Service</a> allows users to manage their static data reliably and efficiently by storing them on Amazon's servers. The stored data can be accessed at any time from anywhere over the Internet.
Data stored in an S3 bucket is accessible through Amazon Management Console, which is a UI interface, as well as the <a rel="nofollow" target="_blank" href="https://aws.amazon.com/cli/">AWS Command Line Interface</a> and <a rel="nofollow" target="_blank" href="https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html">S3 REST API</a> for developers.
<h3 id="springcloudaws">Spring Cloud AWS</h3>
AWS services can be integrated into Java applications using <a rel="nofollow" target="_blank" href="https://spring.io">Spring</a>, which is a well known and the most commonly used Java web framework. <a rel="nofollow" target="_blank" href="https://spring.io/projects/spring-cloud-aws">Spring Cloud for Amazon Web Services</a> allows developers to access AWS services with a smaller code footprint and simple integration.
<h2 id="springamazons3client">Spring Amazon S3 Client</h2>
The S3 Bucket in which our data is stored is accessible through Spring's Amazon S3 client, offering general operations for managing data on the server. In this section we'll show how to include this client library in your project, and then later we'll take a look at some of the common S3 operations available via the client.
<h3 id="mavendependencies">Maven Dependencies</h3>
The first step to integrating AWS into a Spring project is, of course, to import the required dependencies. In this case, we'll be using <code>speing-cloud-starter-aws</code>, which is contains the <code>spring-cloud-aws-context</code> and <code>spring-cloud-aws-autoconfigure</code> dependencies:
<pre><code class="hljs">&lt;dependency&gt;
 &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
 &lt;artifactId&gt;spring-cloud-starter-aws&lt;/artifactId&gt;
&lt;/dependency&gt;
</code></pre>
<h3 id="springcloudawsconfiguration">Spring Cloud AWS Configuration</h3>
Since we're using Spring Boot, naturally, most of the configuration is done by the framework itself. Though, AWS-related configurations should be specified in the <code>application.yaml</code> file:
<pre><code class="hljs">cloud:
 aws:
 region.static: &quot;[S3 Bucket region]&quot;
 credentials:
 accessKey: &quot;xxxxxxx&quot;
 secretKey: &quot;xxxxxxx&quot;
</code></pre>
Please keep in mind that the names and structures of the properties are strictly formatted and will be used by Spring Boot to set up a valid connection with the AWS Services.
The &quot;overseeing&quot; object for the management of requests to the S3 bucket, using Amazon's S3 client, is an instance of the <code>AmazonS3</code> class:
<pre><code class="hljs">@Autowired
private AmazonS3 amazonS3Client;
</code></pre>
We'll be autowiring it as a standard Spring bean and the properties from our <code>.yaml</code> file will be used to strap it up and prep it for work.
<h2 id="uploadingfilestos3">Uploading Files to S3</h2>
Any type of file can be uploaded to an S3 Bucket, although one of the most common use-cases is for image storage. Keeping the files/images stored in the cloud makes it easier to access them and keep them safe in a stable and fast service that's able to scale up the resources required to serve your files.
<h3 id="directfilemultipartfileupload">Direct File/Multipart File Upload</h3>
Once Amazon's S3 Client is functional, you can upload a new file simply by calling the <code>putObject()</code> method:
<pre><code class="hljs">amazonS3Client.putObject(new PutObjectRequest(&quot;bucketName&quot;, &quot;fileKey&quot;, file));
</code></pre>
Where <code>bucketName</code> is the S3 bucket name to which you want to upload the file. <code>fileKey</code> is a String value which will uniquely identify the file being uploaded and <code>file</code> is a valid <code>File</code> object.
If you have a multi-part file coming into your application or microservice through an exposed REST endpoint, it can also be uploaded. This requires little additional code and we simply convert the <code>MultipartFile</code> to a <code>File</code>:
<pre><code class="hljs">File file = new File(&quot;FileName&quot;);
try {
 FileOutputStream fileOutputStream = new FileOutputStream(file)
 fileOutputStream.write(multipartFile.getBytes());
} catch (IOException e) {
 /* Handle Exception */
}
</code></pre>
What we've done is simply convert a multipart to a regular Java <code>File</code> object using a <code>FileOutputStream</code>. Once converted, it can be uploaded to S3 bucket using the same <code>putObject()</code> method from before.
<h3 id="uploadingfilesasmetadata">Uploading Files as Metadata</h3>
When handling requests and data received through API end-points, keeping a copy of the data on our own server before uploading it to our S3 bucket is cost-ineffective and just increases our application's size unnecessarily since the main resource for file storage will be the S3 Bucket.
To avoid needing to keep a copy, we can use the <code>PutObjectRequest</code> from Amazon's API to upload the file to the bucket by sending it via an <code>InputStream</code> and providing file details in the form of metadata:
<pre><code class="hljs">ObjectMetadata objectMetaData = new ObjectMetadata();
objectMetaData.setContentType(multipartFile.getContentType());
objectMetaData.setContentLength(multipartFile.getSize());

try {
 PutObjectRequest putObjectRequest = new PutObjectRequest(&quot;bucketName&quot;, &quot;fileName&quot;, multipartFile.getInputStream(), objectMetaData);
 amazonS3Client.putObject(putObjectRequest);
} catch (IOException e) {
 /* Handle Exception */
}
</code></pre>
The size and the content type of the file are specified via the <code>ObjectMetaData</code> object. The file's input stream is added to the <code>PutObjectRequest</code> along with S3 bucket name to which we're uploading as well as the file name to associate with.
Once the <code>PutObjectRequest</code> object is created, it can be sent to the S3 Bucket using the <code>putObject()</code> method, just like before.
<h3 id="uploadingfileswithpublicview">Uploading Files with Public View</h3>
Sometimes we may wish for the uploaded files to be publicly available. A reader shouldn't need authorization to view the images on a blog post, for example. So far, we've uploaded files that require our authorization to view.
AWS' S3 provides us with options to set the access level on each uploaded file, during their upload.
To change the access level and give public access, let's slightly modify the data upload request:
<pre><code class="hljs">new PutObjectRequest(&quot;bucketName&quot;, &quot;fileName&quot;, multipartFile.getInputStream(), objectMetaData)
 .withCannedAcl(CannedAccessControlList.PublicRead)
</code></pre>
Adding the <code>CannedAccessControlList.PublicRead</code> property to the <code>PutObjectRequest</code> gives read-only public access for the file being uploaded - allowing anyone with the correct URL to access/view the file.
Once the <code>PutObjectRequest</code> object is created, it can then be uploaded to an S3 bucket using the same <code>putObject()</code> method as before.
<h2 id="downloadingfilesfroms3">Downloading Files from S3</h2>
Once uploaded, you can easily download files from your bucket using the <code>getObject()</code> method via the <code>AmazonS3</code> class's instance.
The returned object is packed in an <code>S3Object</code> instance, which can then be streamed into a regular <code>Object</code>:
<pre><code class="hljs">S3Object s3Object = amazonS3Client.getObject(&quot;bucketName&quot;, &quot;fileName&quot;);
S3ObjectInputStream inputStream = s3Object.getObjectContent();
byte[] bytes = StreamUtils.copyToByteArray(inputStream);
File file = new File(&quot;File_Name&quot;);
try {
 FileOutputStream fileOutputStream = new FileOutputStream(file)
 fileOutputStream.write(bytes);
} catch (IOException e) {
 /* Handle Exception */
}
</code></pre>
If the request to download the file is made through a REST endpoint, we can return the file data to the calling entity without creating a <code>File</code> by using Spring's <code>ResponseEntity</code>:
<pre><code class="hljs">S3Object s3Object = amazonS3Client.getObject(&quot;bucketName&quot;, &quot;fileName&quot;);
S3ObjectInputStream inputStream = s3Object.getObjectContent();
byte[] bytes = StreamUtils.copyToByteArray(inputStream);
String contentType = s3Object.getObjectMetadata().getContentType();
return ResponseEntity.ok().contentType(contentType).body(bytes);
</code></pre>
This way we don't have to create a file on our server when downloading from the S3 bucket, the file is simply returned to the caller in the API response.
<h2 id="deletefilefroms3bucket">Delete file from S3 Bucket</h2>
Deleting files from an S3 bucket is the simplest task and all you need to know is the absolute path to the file.
Calling the <code>deleteObject()</code> method with the bucket name and the complete file name deletes the file from the server, if it exists:
<pre><code class="hljs">amazonS3Client.deleteObject(&quot;bucketName&quot;, &quot;fileName&quot;);
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Amazon's S3 provides a convenient way to store file data in the cloud and gives a reliable medium to access data whenever needed.
With Spring Cloud AWS, developers can easily access Amazon's S3 services from their application to perform necessary operations.
For Spring Boot applications all the connection management to the Amazon server is handled by Spring itself, making things simpler than using the regular <code>aws-java-sdk</code> with plain Java applications.
As evident from the code snippets above accessing S3 bucket using Spring Cloud AWS is quite simple and the code footprint is very small as well.
The source code for the project is available on <a rel="nofollow" target="_blank" href="https://github.com/TaimoorChoudhary/spring-boot-s3-bucket">GitHub</a>.

Taimoor Choudhary

AWS S3 allows users to manage their static data reliably and efficiently. Through Spring Cloud's module, we can easily access and use S3 from our Spring projects.

Spring Cloud: AWS S3

<h2 id="introduction">Introduction</h2>
Python allows you to convert strings, integers, and floats interchangeably in a few different ways. The simplest way to do this is using the basic <code>str()</code>, <code>int()</code>, and <code>float()</code> functions. On top of this, there are a couple of other ways as well.
Before we get in to converting strings to numbers, and converting numbers to strings, let's first see a bit about how strings and numbers are represented in Python.
Note: For simplicity of running and showing these examples we'll be using the Python interpreter.
<h2 id="strings">Strings</h2>
String literals in Python are declared by surrounding a character with double (&quot;) or single quotation (') marks. Strings in Python are really just arrays with a Unicode for each character as an element in the array, allowing you to use indices to access a single character from the string.
For example, we can access individual characters of these strings by specifying an index:
<pre><code class="hljs">&gt;&gt;&gt; stringFirst = &quot;Hello World!&quot;
&gt;&gt;&gt; stringSecond = &#x27;Again!&#x27;
&gt;&gt;&gt; stringFirst[3]
&#x27;l&#x27;
&gt;&gt;&gt; stringSecond[3]
&#x27;i&#x27;
</code></pre>
<h2 id="numerics">Numerics</h2>
A numeric in Python can be an <code>integer</code>, a <code>float</code>, or a <code>complex</code>.
Integers can be a positive or negative whole number. Since Python 3, integers are unbounded and can practically hold any number. Before Python 3, the top boundary was 231-1 for 32-bit runtimes and 263-1 for 64-bit runtimes.
Floats have unlimited length as well, but a floating-point number must contain a decimal point.
Complex numerics must have an imaginary part, which is denoted using <code>j</code>:
<pre><code class="hljs">&gt;&gt;&gt; integerFirst = 23
&gt;&gt;&gt; floatFirst = 23.23
&gt;&gt;&gt; complextFirst = 1 + 23j
</code></pre>
<h2 id="convertingstringstonumerics">Converting Strings to Numerics</h2>
<h3 id="usingtheintfunction">Using the int() Function</h3>
If you want to convert a string to an integer, the simplest way would be to use <code>int()</code> function. Just pass the string as an argument:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23&quot;
&gt;&gt;&gt; y = &quot;20&quot;
&gt;&gt;&gt; z = int(x) - int(y)
&gt;&gt;&gt; z
3
</code></pre>
This works as expected when you're passing a string-representation of an integer to <code>int()</code>, but you'll run in to trouble if the string you pass doesn't contain an integer value. If there are any non-numeric characters in your string then <code>int()</code> will raise an exception:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23a&quot;
&gt;&gt;&gt; z = int(x)
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
ValueError: invalid literal for int() with base 10: &#x27;23a&#x27;
</code></pre>
This same exception will even be raised if a valid float string is passed:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23.4&quot;
&gt;&gt;&gt; z = int(x)
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
ValueError: invalid literal for int() with base 10: &#x27;23.4&#x27;
</code></pre>
The <code>int()</code> function does have another useful feature than just converting strings to integers, it also allows you to convert numbers from any base to a base 10 integer. For example, we can convert the following binary string to a base 10 integer using the <code>base</code> parameter:
<pre><code class="hljs">&gt;&gt;&gt; int(&#x27;1101100&#x27;, base=2)
108
</code></pre>
The same can be done for any other base, like hexadecimal (base 16):
<pre><code class="hljs">&gt;&gt;&gt; int(&#x27;6C&#x27;, base=16)
108
</code></pre>
<h3 id="usingthefloatfunction">Using the float() Function</h3>
Converting string literals to floats is done via the <code>float()</code> function:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23.23&quot;
&gt;&gt;&gt; y = &quot;23.00&quot;
&gt;&gt;&gt; z = float(x) - float(y)
&gt;&gt;&gt; z
0.23000000000000043
</code></pre>
Notice that the resulting value is not entirely accurate, as it should just be <code>0.23</code>. This has to do with <a rel="nofollow" target="_blank" href="https://0.30000000000000004.com/">floating point math</a> issues rather than the conversion from string to number.
The <code>float()</code> function offers a bit more flexibility than the <code>int()</code> function since it can parse and convert both floats and integers:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23&quot;
&gt;&gt;&gt; y = &quot;20&quot;
&gt;&gt;&gt; z = float(x) - float(y)
&gt;&gt;&gt; z
3.0
</code></pre>
Unlike <code>int()</code>, <code>float()</code> doesn't raise an exception when it receives a non-float numeric value.
However, it will raise an exception if a non-numeric value is passed to it:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;23a&quot;
&gt;&gt;&gt; z = float(x)
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
ValueError: could not convert string to float: &#x27;23a&#x27;
</code></pre>
While <code>float()</code> does not have the ability to convert non-base 10 numbers like <code>int()</code> does, it does have the ability to convert numbers represented in scientific notation (aka e-notation):
<pre><code class="hljs">&gt;&gt;&gt; float(&#x27;23e-5&#x27;)
0.00023
&gt;&gt;&gt; float(&#x27;23e2&#x27;)
2300.0
</code></pre>
<h3 id="usingthecomplexfunction">Using the complex() Function</h3>
Converting string literals to complex numerics is done via the <code>complex()</code> function. To do so, the string must follow specific formatting. Particularly, it must be formatted without white spaces around the <code>+</code> or <code>-</code> operators:
<pre><code class="hljs">&gt;&gt;&gt; x = &quot;5+3j&quot;
&gt;&gt;&gt; y = &quot;3+1j&quot;
&gt;&gt;&gt; z = complex(x) + complex(y)
&gt;&gt;&gt; z
(8+4j)
</code></pre>
Having any extra spaces between the <code>+</code> or <code>-</code> operators will result in a raised exception:
<pre><code class="hljs">&gt;&gt;&gt; z = complex(&quot;5+ 3j&quot;)
Traceback (most recent call last):
 File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
ValueError: complex() arg is a malformed string
</code></pre>
Like the <code>float()</code> function, <code>complex()</code> is also more relaxed in the types of numbers it allows. For example, the imaginary part of the number can be completely omitted, and both integers and floats can be parsed as well:
<pre><code class="hljs">&gt;&gt;&gt; complex(&quot;1&quot;)
(1+0j)
&gt;&gt;&gt; complex(&quot;1.2&quot;)
(1.2+0j)
</code></pre>
As you can see, however, this should not be used as a more flexible replacement for <code>int</code>/<code>float</code> since it automatically adds the imaginary part of the number to the stringified version.
<h2 id="convertingnumericstostrings">Converting Numerics to Strings</h2>
<h3 id="usingthestrfunction">Using the str() Function</h3>
The <code>str()</code> function can be used to change any numeric type to a string.
The function <code>str()</code> is available from Python 3.0+ since the strings in Python 3.0+ are Unicode by default. However, this is not true for Python versions below 3.0 - in which to achieve the same goal, the <code>unicode()</code> function is used:
<pre><code class="hljs">&gt;&gt;&gt; str(23) # Integer to String
&#x27;23&#x27;
&gt;&gt;&gt; str(23.3) # Float to String
&#x27;23.3&#x27;
&gt;&gt;&gt; str(5+4j) # Complex to String
&#x27;(5+4j)&#x27;
</code></pre>
The nice thing about <code>str()</code> is that it can handle converting any type of number to a string, so you don't need to worry about choosing the correct method based on what type of number you're converting.
<h3 id="usingtheformatfunction">Using the format() Function</h3>
Another way of converting numerics to strings is using the <code>format()</code> function, which allows you to set placeholders within a string and then convert another data type to a string and fill the placeholders.
To use the function, simply write a string followed by <code>.format()</code> and pass the arguments for the placeholders.
Here's an example:
<pre><code class="hljs">&gt;&gt;&gt; &quot;My age is {}&quot;.format(21)
&#x27;My age is 21&#x27;
</code></pre>
The arguments in the <code>.format()</code> function can also be referred to individually, using their positions or variable names:
<pre><code class="hljs">&gt;&gt;&gt; &quot;You get {product} when you multiply {1} with {0}&quot;.format(5.5, 3, product=16.5)
&#x27;You get 16.5 when you multiply 3 with 5.5&#x27;
</code></pre>
Note that underneath the hood, the <code>.format()</code> function simply uses <code>str()</code> to convert the arguments to strings. So essentially this is a similar way of converting numbers to strings as the previous section, but <code>.format()</code> acts as a convenience function for formatting a string.
<h2 id="conclusion">Conclusion</h2>
Python allows you to convert strings, integers, and floats interchangeably in a few different ways. The simplest way to do this is using the basic <code>str()</code>, <code>int()</code>, and <code>float()</code> functions. On top of this, there are a couple of other ways as well such as the <code>format()</code> function. Just keep in mind that the <code>int()</code>, <code>float()</code>, and <code>complex()</code> functions have their limitations and may raise exceptions if the input string isn't formatted exactly as they expect.

Guest Contributor

Converting data types is a common and trivial task in any developer's workflow. Python provides several ways to go about this task.

Convert Strings to Numbers and Numbers to Strings in Python

Robley Gori

AWS EC2 offers provisioning and hosting applications on the cloud with automatic scaling. We'll be deploying a Django app to EC2 with Docker.

Deploying Django Applications to AWS EC2 with Docker

<h2 id="introduction">Introduction</h2>
The sweetest <a rel="nofollow" target="_blank" href="https://en.wikipedia.org/wiki/Syntactic_sugar">syntactic sugar</a> added to Java up until now are definitely <a target="_blank" href="/lambda-expressions-in-java">Lambda Expressions</a>.
Java is a verbose language and that can get in the way of productivity and readability. Reducing boilerplate and repetitive code has always been a popular task with Java developers, and clean, readable, concise code is generally sought after.
Lambda Expressions removed the need to type cumbersome boilerplate code when it comes to some common tasks by allowing developers to call them without them belonging to a class and passing them as if they were objects.
These expressions have seen major usage with the <a target="_blank" href="/introduction-to-java-8-streams/">Java Streams API</a>, and Spring's <a rel="nofollow" target="_blank" href="https://docs.spring.io/spring/docs/current/spring-framework-reference/web-reactive.html">WebFlux</a> framework for creating reactive, dynamic applications.
Another, really useful feature added to Java 8, are method references, which make Lambda Expressions just that much more concise and simple, by invoking (referencing) the methods using a method name when the Lambda Expression would've been used simply to call a method.
<h2 id="methodreferences">Method References</h2>
Method references are essentially shortened Lambda Expressions, used for invoking methods.
They consist of two parts:
<pre><code class="hljs">Class::method;
</code></pre>
And a common example would be printing out the results of say, subscribing to a publisher service or a Java Stream:
<pre><code class="hljs">someCodeChain.subscribe(System.out::println);
</code></pre>
Let's go over an example of imperative code, which we'll then turn to functional code via Lambda Expressions and then finally shorten via Method References.
We'll be making a simple class:
<pre><code class="hljs">public class Employee {
 private int id;
 private String name;
 private int wage;
 private String position;

 // Constructor, getters and setters

 @Override
 public String toString() {
 return &quot;Name: &quot; + name + &quot;, Wage: &quot; + wage + &quot;, Position: &quot; + position;
 }

 public int compareTo(Employee employee) {
 if (this.wage &lt;= employee.wage) {
 return 1;
 } else {
 return -1;
 }
 }
}
</code></pre>
If we formed this class into a collection, such as an <code>ArrayList</code>, we couldn't sort it using the utility method <code>.sort()</code> since it doesn't implement the <code>Comparable</code> interface.
What we can do is define a <code>new Comparator</code> for these objects while calling the <code>.sort()</code> method:
<pre><code class="hljs">Employee emp1 = new Employee(1, &quot;David&quot;, 1200, &quot;Developer&quot;);
Employee emp2 = new Employee(2, &quot;Tim&quot;, 1500, &quot;Developer&quot;);
Employee emp3 = new Employee(3, &quot;Martha&quot;, 1300, &quot;Developer&quot;);

ArrayList&lt;Employee&gt; employeeList = new ArrayList&lt;&gt;();
employeeList.add(emp1);
employeeList.add(emp2);
employeeList.add(emp3);

Collections.sort(employeeList, new Comparator&lt;Employee&gt;() {
 public int compare(Employee emp1, Employee emp2) {
 return emp1.compareTo(emp2);
 }
});

System.out.println(employeeList);
</code></pre>
Running this code will yield:
<pre><code class="hljs">[Name: Tim, Wage: 1500, Position: Developer, Name: Martha, Wage: 1300, Position: Developer, Name: David, Wage: 1200, Position: Developer]
</code></pre>
Here, the anonymous class (<code>Comparator</code>) is defining the comparison criteria. We can make it a lot simpler and shorter by using a Lambda Expression:
<pre><code class="hljs">Collections.sort(employeeList, (e1, e2) -&gt; e1.compareTo(e2));
</code></pre>
Running this piece of code will yield:
<pre><code class="hljs">[Name: Tim, Wage: 1500, Position: Developer, Name: Martha, Wage: 1300, Position: Developer, Name: David, Wage: 1200, Position: Developer]
</code></pre>
Then again, since all we're doing with this Lambda Expression is call a single method, we can reference just that method:
<pre><code class="hljs">Collections.sort(employeeList, Employee::compareTo);
</code></pre>
And this will also yield:
<pre><code class="hljs">[Name: Tim, Wage: 1500, Position: Developer, Name: Martha, Wage: 1300, Position: Developer, Name: David, Wage: 1200, Position: Developer]
</code></pre>
<h2 id="methodreferencetypes">Method Reference Types</h2>
Method references can be used in a couple of different scenarios:
<ul>
<li>Static Methods: <code>Class::staticMethodName</code></li>
<li>Instance Methods of Particular Objects: <code>object::instanceMethodName</code></li>
<li>Instance Methods of Arbitraty Objects: <code>Class::methodName</code></li>
<li>Constructor Reference: <code>Class::new</code></li>
</ul>
Let's go over all of these types through some simple examples.
<h3 id="staticmethodreferences">Static Method References</h3>
You can reference any <code>static</code> method of a class by simply calling its containing class with the method name.
Let's define a class with a <code>static</code> method and then reference it from another class:
<pre><code class="hljs">public class ClassA {
 public static void raiseToThePowerOfTwo(double num) {
 double result = Math.pow(num, 2);
 System.out.println(result);
 }
}
</code></pre>
And now, from another class, let's use the <code>static</code> utility method:
<pre><code class="hljs">public class ClassB {
 public static void main(String[] args) {
 List&lt;Double&gt; integerList = new ArrayList&lt;&gt;();
 integerList.add(new Double(5));
 integerList.add(new Double(2));
 integerList.add(new Double(6));
 integerList.add(new Double(1));
 integerList.add(new Double(8));
 integerList.add(new Double(9));

 integerList.forEach(ClassA::raiseToThePowerOfTwo);
 }
}
</code></pre>
Running this piece of code will yield:
<pre><code class="hljs">25.0
4.0
36.0
1.0
64.0
81.0
</code></pre>
There are many Java classes that offer <code>static</code> utility methods which can be used here. In our example we've used a custom method, albeit not a very useful one in this instance.
<h3 id="instancemethodsofparticularobjects">Instance Methods of Particular Objects</h3>
You can call a method from a particular instantiated object by referencing the method using the reference variable of the object.
This is most often illustrated through a custom comparator. We'll use the same <code>Employee</code> class from before and the same list to highlight the difference between these two:
<pre><code class="hljs">public class Employee {
 private int id;
 private String name;
 private int wage;
 private String position;

 // Constructor, getters and setters

 @Override
 public String toString() {
 return &quot;Name: &quot; + name + &quot;, Wage: &quot; + wage + &quot;, Position: &quot; + position;
 }

 public int compareTo(Employee employee) {
 if (this.wage &lt;= employee.wage) {
 return 1;
 } else {
 return -1;
 }
 }
}
</code></pre>
Now, let's define a <code>CustomComparator</code>:
<pre><code class="hljs">public class CustomComparator {
 public int compareEntities(Employee emp1, Employee emp2) {
 return emp1.compareTo(emp2);
 }
}
</code></pre>
And finally, let's populate a list and sort it:
<pre><code class="hljs">Employee emp1 = new Employee(1, &quot;David&quot;, 1200, &quot;Developer&quot;);
Employee emp2 = new Employee(2, &quot;Tim&quot;, 1500, &quot;Developer&quot;);
Employee emp3 = new Employee(3, &quot;Martha&quot;, 1300, &quot;Developer&quot;);

ArrayList&lt;Employee&gt; employeeList = new ArrayList&lt;&gt;();
employeeList.add(emp1);
employeeList.add(emp2);
employeeList.add(emp3);

// Initializing our CustomComparator
CustomComparator customComparator = new CustomComparator();

// Instead of making a call to an arbitrary Employee
// we&#x27;re now providing an instance and its method
Collections.sort(employeeList, customComparator::compareEntities);

System.out.println(employeeList);
</code></pre>
Running this code will also yield:
<pre><code class="hljs">[Name: Tim, Wage: 1500, Position: Developer, Name: Martha, Wage: 1300, Position: Developer, Name: David, Wage: 1200, Position: Developer]
</code></pre>
The main difference is that by adding another layer, through the <code>CustomComparator</code>, we can add more functionality for comparison and take it away from the class itself. A class like <code>Employee</code> shouldn't be burdened with complex comparison logic and this results in cleaner and more readable code.
On the other hand, sometimes we don't wish to define custom comparators and introducing one is simply too much of a hassle. In such cases, we'd call a method from an arbitrary object of a particular type, shown in the next section.
<h3 id="instancemethodsofarbitraryobjects">Instance Methods of Arbitrary Objects</h3>
This example has already been shown in the beginning of the article when we've boiled down the imperative approach into a functional approach via Lambda Expressions.
Though, for good measure, since this approach is used really often, let's take a look at another example:
<pre><code class="hljs">List&lt;Integer&gt; integerList = new ArrayList&lt;&gt;();
integerList.add(new Integer(5));
integerList.add(new Integer(2));
integerList.add(new Integer(6));
integerList.add(new Integer(1));
integerList.add(new Integer(8));
integerList.add(new Integer(9));

// Referencing the non-static compareTo method from the Integer class
Collections.sort(integerList, Integer::compareTo);

// Referencing static method
integerList.forEach(System.out::print);
</code></pre>
Running this piece of code would yield:
<pre><code class="hljs">125689
</code></pre>
While this may seem like it's the same as a call to a static method, it's not. This is equivalent to calling the Lambda Expression:
<pre><code class="hljs">Collections.sort(integerList, (Integer a, Integer b) -&gt; a.compareTo(b));
</code></pre>
Here, the distinction is more obvious. If we were to call a <code>static</code> method, it would look like:
<pre><code class="hljs">Collections.sort(integerList, (Integer a, Integer b) -&gt; SomeClass.compare(a, b));
</code></pre>
<h3 id="referencingconstructors">Referencing Constructors</h3>
You can reference a class' constructor in the same way you'd reference a <code>static</code> method.
You could use a reference to a constructor instead of classic class instantiation:
<pre><code class="hljs">// Classic instantiation
Employee employee = new Employee();

// Constructor reference
Employee employee2 = Employe::new;
</code></pre>
Based on the context, if multiple constructors are present, the adequate one will be used if referenced:
<pre><code class="hljs">Stream&lt;Employee&gt; stream = names.stream().map(Employee::new);
</code></pre>
Due to a stream of names, if a <code>Employee(String name)</code> constructor is present, it will be used.
Another way you could use constructor references is when you'd wanna map out a stream into an array, while keeping the particular type. If you were to simply map it and then call <code>toArray()</code>, you'd get an array of <code>Object</code>s instead of your particular type.
If we tried, say:
<pre><code class="hljs">Employee[] employeeArray = employeeList.toArray();
</code></pre>
Of course, we'd be greeted with a compiler error since we <code>.toArray()</code> returns an array of <code>Object</code>s. Casting it won't help either:
<pre><code class="hljs">Employee[] employeeArray = (Employee[]) employeeList.toArray();
</code></pre>
But this time, it'll be a runtime exception - <code>ClassCastException</code>.
We can avoid that with:
<pre><code class="hljs">// Making a list of employees
List&lt;String&gt; employeeList = Arrays.asList(&quot;David&quot;, &quot;Scott&quot;);

// Mapping a list to Employee objects and returning them as an array
Employee[] employeeArray = employeeList.stream().map(Employee::new).toArray(Employee[]::new);

// Iterating through the array and printing information
for (int i = 0; i &lt; employeeArray.length; i++) {
 System.out.println(employeeArray[i].toString());
}
</code></pre>
And with that, we get the output:
<pre><code class="hljs">Name: David, Wage: 0, Position: null
Name: Scott, Wage: 0, Position: null
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Method References are a type of Lambda Expressions that are used to simply reference a method in their call. With them, writing code can be a lot more concise and readable.
Lambda Expressions have introduced Java Developers to a more functional approach in programming which allows them to avoid writing verbose code for simple operations.

David Landup

Reducing boilerplate code has always been a popular task for Java developers. Method References allow us to shorten Lambda Expressions further down into concise, readable statements.

Method References in Java 8

A common operation in many programming languages is to check if a string contains another string. While it's a simple and common task, the method names often differ between programming languages. For example, here is a small sample of the methods used to achieve this in various languages:
<ul>
<li>Java: <code>String.contains()</code>, <code>String.indexOf()</code>, etc.</li>
<li>Python: <code>in</code> operator, <code>String.index()</code>, <code>String.find()</code></li>
<li>Go: <code>strings.Contains()</code></li>
<li>Ruby: <code>string.include?</code></li>
</ul>
You get the point. There are a million ways to do this, and it seems like each language implements it differently.
Anyway, let's see a few of the ways in which you can check if a string contains a substring in JavaScript.
Note: The first two methods shown below also work on arrays, which tells you if an array contains a given value (or the index of it for <code>indexOf()</code>). Keep this in mind when reading the article as it'll likely help you for similar use-cases.
<h2 id="thestringincludesmethod">The <code>String.includes()</code> Method</h2>
This method was introduced in ES6 and is typically the preferred method for simple use-cases. If all you need to do is get a boolean value indicating if the substring is in another string, then this is what you'll want to use.
It works like this:
<pre><code class="hljs">&gt; let str = &#x27;stackabuse&#x27;;
&gt; let substr = &#x27;stack&#x27;;
&gt; str.includes(substr);
true
</code></pre>
As you can see, a boolean value is returned since the string &quot;stack&quot; is a substring of <code>stackabuse</code>.
This is a case sensitive search, so the following will not match the substring:
<pre><code class="hljs">&gt; let str = &#x27;StackAbuse&#x27;;
&gt; let substr = &#x27;stack&#x27;;
&gt; str.includes(substr);
false
</code></pre>
While this is enough for most use-cases, the <code>includes()</code> method also provides another option that may be useful to you. A second argument can be provided that tells the method at which index to start the search. So if you know that the substring isn't contained in the first 50 characters (or you just don't want it to match those characters), then you can use the method like this:
<pre><code class="hljs">str.includes(substr, 50);
</code></pre>
An offset of below 0 just starts the search from index 0, and an offset greater than <code>string.length</code> returns <code>false</code> since the search starts from <code>string.length</code>.
<h2 id="thestringindexofmethod">The <code>String.indexOf()</code> Method</h2>
The <code>String.indexOf()</code> method is much like the previous <code>includes()</code> method (and it's suitable to be used in a polyfill for <code>includes()</code> as well), but the only difference is the return value. Instead of returning a boolean value indicating the presence of the substring, it actually returns the index location of the substring, or -1 if it isn't present.
Here's an example:
<pre><code class="hljs">&gt; let str = &#x27;stackabuse&#x27;;
&gt; let substr = &#x27;abuse&#x27;;
&gt; str.indexOf(substr);
5
&gt; str.indexOf(&#x27;apple&#x27;);
-1
</code></pre>
As you can see, this method returns the 0-based index position of the substring, and a -1 when the substring wasn't found.
Just like the <code>includes()</code> method, the <code>indexOf()</code> method is case sensitive and also supports the offset parameter:
<pre><code class="hljs">&gt; let str = &#x27;StackAbuse&#x27;;
&gt; let substr = &#x27;abuse&#x27;;
&gt; str.indexOf(substr);
-1
&gt; str.indexOf(&#x27;Abu&#x27;, 3);
5
&gt; str.indexOf(&#x27;Abu&#x27;, 6);
-1
</code></pre>
This method is useful for when you need to know the exact location of the substring, however, it's not as clean when simply using it as a boolean:
<pre><code class="hljs">let str = &#x27;stackabuse&#x27;;
let substr = &#x27;stack&#x27;;

if (str.indexOf(substr) &gt; -1) {
 console.log(&#x27;Found the substring!&#x27;);
}
</code></pre>
In cases like this you should use the <code>includes()</code> method instead as it's more error-prone.
<h2 id="regex">Regex</h2>
One of the more useful and powerful ways to check for a substring is to use regular expressions, or regex. Using regex for a task like this gives you a lot more flexibility than with the previous methods where you can only check for a constant string. While regex is too big of a subject to completely cover here, we can at least take a look at some of the useful features for our use-case.
Checking for substrings in a string with regex can be achieved using the <code>RegExp.test()</code> method:
<pre><code class="hljs">&gt; let str = &#x27;stackabuse&#x27;;
&gt; /stack/.test(str);
true
</code></pre>
Unlike the previous two methods, we can now do case insensitive searches with the <code>i</code> flag:
<pre><code class="hljs">&gt; let str = &#x27;StackAbuse&#x27;;
&gt; /stack/i.test(str);
true
</code></pre>
As a more complex example, let's say you want to see if a string contains a zip code (5 digit postal codes), but you don't really care what zip code is in the string. This kind of problem can't be solved using <code>includes()</code> or <code>indexOf()</code>. But with regex, we can easily test this:
<pre><code class="hljs">&gt; let str = &#x27;My zip code is 90210&#x27;;
&gt; /\d{5}/.test(str);
true
&gt; str = &#x27;My address is 123 Fake St.&#x27;;
&gt; /\d{5}/.test(str);
false
</code></pre>
While JavaScript isn't necessarily known for its speed anyway, keep in mind that this will be slower than a simpler method like <code>includes()</code>, so you should only use it for the more complex cases that can't be solved with a simpler solution.
<h2 id="conclusion">Conclusion</h2>
Manipulating and inspecting strings is one of the most common tasks performed in many programming languages, especially with user-facing code. For many different reasons, like validating strings, you'll need to check if a string contains a substring. In this article we saw a few different methods provided by JavaScript that allows you to do exactly that.

Scott Robinson

Checking if a String contains a substring is a common operation and slightly differs between languages. JavaScript offers several ways to do this.

JavaScript: Check if String Contains a Substring

Speech recognition, as the name suggests, refers to automatic recognition of human speech. Speech recognition is one of the most important tasks in the domain of human computer interaction. If you have ever interacted with Alexa or have ever ordered Siri to complete a task, you have already experienced the power of speech recognition.
Speech recognition has various applications ranging from automatic transcription of speech data (like voice-mails) to interacting with robots via speech.
In this tutorial, you will see how we can develop a very simple speech recognition application that is capable of recognizing speech from audio files, as well as live from a microphone. So, let's begin without further ado.
Several speech recognition libraries have been developed in Python. However we will be using the <a rel="nofollow" target="_blank" href="https://pypi.org/project/SpeechRecognition/">SpeechRecognition</a> library, which is the simplest of all the libraries.
<h2 id="installingspeechrecognitionlibrary">Installing SpeechRecognition Library</h2>
Execute the following command to install the library:
<pre><code class="hljs">$ pip install SpeechRecognition
</code></pre>
<h2 id="speechrecognitionfromaudiofiles">Speech Recognition from Audio Files</h2>
In this section, you will see how we can translate speech from an audio file to text. The audio file that we will be using as input can be downloaded from <a rel="nofollow" target="_blank" href="https://www.voiptroubleshooter.com/open_speech/american/OSR_us_000_0018_8k.wav">this link</a>. Download the file to your local file system.
The first step, as always, is to import the required libraries. In this case, we only need to import the <code>speech_recognition</code> library that we just downloaded.
<pre><code class="hljs">import speech_recognition as speech_recog
</code></pre>
To convert speech to text the one and only class we need is the <code>Recognizer</code> class from the <code>speech_recognition</code> module. Depending upon the underlying API used to convert speech to text, the <code>Recognizer</code> class has following methods:
<ul>
<li><code>recognize_bing()</code>: Uses Microsoft Bing Speech API</li>
<li><code>recognize_google()</code>: Uses Google Speech API</li>
<li><code>recognize_google_cloud()</code>: Uses Google Cloud Speech API</li>
<li><code>recognize_houndify()</code>: Uses Houndify API by SoundHound</li>
<li><code>recognize_ibm()</code>: Uses IBM Speech to Text API</li>
<li><code>recognize_sphinx()</code>: Uses PocketSphinx API</li>
</ul>
Among all of the above methods, the <code>recognize_sphinx()</code> method can be used offline to translate speech to text.
To recognize speech from an audio file, we have to create an object of the <code>AudioFile</code> class of the <code>speech_recognition</code> module. The path of the audio file that you want to translate to text is passed to the constructor of the <code>AudioFile</code> class. Execute the following script:
<pre><code class="hljs">sample_audio = speech_recog.AudioFile(&#x27;E:/Datasets/my_audio.wav&#x27;)
</code></pre>
In the above code, update the path to the audio file that you want to transcribe.
We will be using the <code>recognize_google()</code> method to transcribe our audio files. However, the <code>recognize_google()</code> method requires the <code>AudioData</code> object of the <code>speech_recognition</code> module as a parameter. To convert our audio file to an <code>AudioData</code> object, we can use the <code>record()</code> method of the <code>Recognizer</code> class. We need to pass the <code>AudioFile</code> object to the <code>record()</code> method, as shown below:
<pre><code class="hljs">with sample_audio as audio_file:
 audio_content = recog.record(audio_file)
</code></pre>
Now if you check the type of the <code>audio_content</code> variable, you will see that it has the type <code>speech_recognition.AudioData</code>.
<pre><code class="hljs">type(audio_content)
</code></pre>
Output:
<pre><code class="hljs">speech_recognition.AudioData
</code></pre>
Now we can simply pass the <code>audio_content</code> object to the <code>recognize_google()</code> method of the <code>Recognizer()</code> class object and the audio file will be converted to text. Execute the following script:
<pre><code class="hljs">recog.recognize_google(audio_content)
</code></pre>
Output:
<pre><code class="hljs">'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas tank degrees office 30 face before you go out the race was badly strained and hung them the stray cat gave birth to kittens the young girl gave no clear response the meal was called before the bells ring what weather is in living'
</code></pre>
The above output shows the text of the audio file. You can see that the file has not been 100% correctly transcribed, yet the accuracy is pretty reasonable.
<h3 id="settingdurationandoffsetvalues">Setting Duration and Offset Values</h3>
Instead of transcribing the complete speech, you can also transcribe a particular segment of the audio file. For instance, if you want to transcribe only the first 10 seconds of the audio file, you need to pass 10 as the value for the <code>duration</code> parameter of the <code>record()</code> method. Look at the following script:
<pre><code class="hljs">sample_audio = speech_recog.AudioFile(&#x27;E:/Datasets/my_audio.wav&#x27;)
with sample_audio as audio_file:
 audio_content = recog.record(audio_file, duration=10)

recog.recognize_google(audio_content)
</code></pre>
Output:
<pre><code class="hljs">'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas'
</code></pre>
In the same way, you can skip some part of the audio file from the beginning using the <code>offset</code> parameter. For instance, if you do not want to transcribe the first 4 seconds of the audio, pass 4 as the value for the <code>offset</code> attribute. As an example, the following script skips the first 4 seconds of the audio file and then transcribes the audio file for 10 seconds.
<pre><code class="hljs">sample_audio = speech_recog.AudioFile(&#x27;E:/Datasets/my_audio.wav&#x27;)
with sample_audio as audio_file:
 audio_content = recog.record(audio_file, offset=4, duration=10)

recog.recognize_google(audio_content)
</code></pre>
Output:
<pre><code class="hljs">'take the winding path to reach the lake no closely the size of the gas tank web degrees office dirty face'
</code></pre>
<h3 id="handlingnoise">Handling Noise</h3>
An audio file can contain noise due to several reasons. Noise can actually affect the quality of speech to text translation. To reduce noise, the <code>Recognizer</code> class contains <code>adjust_for_ambient_noise()</code> method, which takes the <code>AudioData</code> object as a parameter. The following script shows how you can improve transcription quality by removing noise from the audio file:
<pre><code class="hljs">sample_audio = speech_recog.AudioFile(&#x27;E:/Datasets/my_audio.wav&#x27;)
with sample_audio as audio_file:
 recog.adjust_for_ambient_noise(audio_file)
 audio_content = recog.record(audio_file)

recog.recognize_google(audio_content)
</code></pre>
Output:
<pre><code class="hljs">'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas tank web degrees office 30 face before you go out the race was badly strained and hung them the stray cat gave birth to kittens the younger again no clear response the mail was called before the bells ring what weather is in living'
</code></pre>
The output is quite similar to what we got earlier; this is due to the fact that the audio file had very little noise already.
<h2 id="speechrecognitionfromlivemicrophone">Speech Recognition from Live Microphone</h2>
In this section you will see how you can transcribe live audio received via a microphone on your system.
There are several ways to process audio input received via microphone, and various libraries have been developed to do so. One such library is <a rel="nofollow" target="_blank" href="https://pypi.org/project/PyAudio/">PyAudio</a>. Execute the following script to install the <code>PyAudio</code> library:
<pre><code class="hljs">$ pip install PyAudio
</code></pre>
Now the source for the audio to be transcribed is a microphone. To capture the audio from a microphone, we need to first create an object of the <code>Microphone</code> class of the <code>Speach_Recogniton</code> module, as shown here:
<pre><code class="hljs">mic = speech_recog.Microphone()
</code></pre>
To see the list of all the microphones in your system, you can use the <code>list_microphone_names()</code> method:
<pre><code class="hljs">speech_recog.Microphone.list_microphone_names()
</code></pre>
Output:
<pre><code class="hljs">['Microsoft Sound Mapper - Input',
 'Microphone (Realtek High Defini',
 'Microsoft Sound Mapper - Output',
 'Speakers (Realtek High Definiti',
 'Microphone Array (Realtek HD Audio Mic input)',
 'Speakers (Realtek HD Audio output)',
 'Stereo Mix (Realtek HD Audio Stereo input)']
</code></pre>
This is a list of microphones available in my system. Keep in mind that your list will likely look different.
The next step is to capture the audio from the microphone. To do so, you need to call the <code>listen()</code> method of the <code>Recognizer()</code> class. Like the <code>record()</code> method, the <code>listen()</code> method also returns the <code>speech_recognition.AudioData</code> object, which can then be passed to the <code>recognize_google()</code> method.
The following script prompts the user to say something in the microphone and then prints whatever the user has said:
<pre><code class="hljs">with mic as audio_file:
 print(&quot;Speak Please&quot;)

 recog.adjust_for_ambient_noise(audio_file)
 audio = recog.listen(audio_file)

 print(&quot;Converting Speech to Text...&quot;)
 print(&quot;You said: &quot; + recog.recognize_google(audio))
</code></pre>
Once you execute the above script, you will see the following message:
<pre><code class="hljs">Please say something
</code></pre>
At this point of time, say whatever you want and then pause. Once you have paused, you will see the transcription of whatever you said. Here is the output I got:
<pre><code class="hljs">Converting Speech to Text...
You said: hello this is normally from stack abuse abuse this is an article on speech recognition I hope you will like it and this is just a test speech and when I will stop speaking are you in today thank you for Reading
</code></pre>
It is important to mention that if <code>recognize_google()</code> method is not able to match the words you speak with any of the words in its repository, an exception is thrown. You can test this by saying some unintelligible words. You should see the following exception:
<pre><code class="hljs">Speak Please
Converting Speech to Text...
---------------------------------------------------------------------------
UnknownValueError Traceback (most recent call last)
&lt;ipython-input-27-41218bc8a239&gt; in &lt;module&gt;
 8 print(&quot;Converting Speech to Text...&quot;)
 9
---&gt; 10 print(&quot;You said: &quot; + recog.recognize_google(audio))
 11
 12

~\Anaconda3\lib\site-packages\speech_recognition\__init__.py in recognize_google(self, audio_data, key, language, show_all)
 856 # return results
 857 if show_all: return actual_result
--&gt; 858 if not isinstance(actual_result, dict) or len(actual_result.get(&quot;alternative&quot;, [])) == 0: raise UnknownValueError()
 859
 860 if &quot;confidence&quot; in actual_result[&quot;alternative&quot;]:

UnknownValueError:
</code></pre>
A better approach is to use the <code>try</code> block when the <code>recognize_google()</code> method is called as shown below:
<pre><code class="hljs">with mic as audio_file:
 print(&quot;Speak Please&quot;)

 recog.adjust_for_ambient_noise(audio_file)
 audio = recog.listen(audio_file)

 print(&quot;Converting Speech to Text...&quot;)

 try:
 print(&quot;You said: &quot; + recog.recognize_google(audio))
 except Exception as e:
 print(&quot;Error: &quot; + str(e))
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Speech recognition has various useful applications in the domain of human computer interaction and automatic speech transcription. This article briefly explains the process of speech transcription in Python via the <code>speech_recognition</code> library and explains how to translate speech to text when the audio source is an audio file or live microphone.

Usman Malik

Speech recognition is one of the most important tasks in human-computer interaction. We'll be using the SpeechRecognition library to perform this task in Python.

Introduction to Speech Recognition with Python

<h2 id="introduction">Introduction</h2>
<a rel="nofollow" target="_blank" href="https://yaml.org/">YAML</a> files are nowadays being used extensively for defining properties of tools and applications due to the very human-readable syntax.
Besides containing configuration properties, they're also often used for data transmission/serialization, similar to how JSON is used.
Reading and writing YAML files is quickly becoming a basic developer skill, similar to how reading and writing JSON and XML files is.
<h2 id="whatisyaml">What is YAML?</h2>
YAML Ain't Markup Language (YAML) is a simple, human-readable data-serialization language, very much alike JSON. It's easier to read and comprehend and has the ability to reference other items within a YAML file, as well as embed other formats such as JSON and XML within itself.
It's not meant to be a replacement for the current solutions and is mainly used for configuration files due to the fact that it's both easy to read and edit, and supports comments, making it a lot more team-friendly.
Most YAML files can be converted to JSON, and vice-versa as YAML is a superset of JSON. Here's an example of a YAML file:
<pre><code class="hljs">--- #Employee Info
name: David
wage: 1500
position: Developer
techstack:
 - Java
 - Spring
 - Hibernate
</code></pre>
The syntax is really simple, a dictionary (our employee entity) is represented with <code>&lt;key&gt;: &lt;value&gt;</code>.
After defining a few properties such as the <code>name</code>, <code>wage</code>, and <code>position</code>, a list is included, again, indented with each list item beginning with a <code>-</code>.
Each of these items can also be a dictionary as well:
<pre><code class="hljs">---
name: David
wage: 1500
position: Developer
colleagues:
 - name: Martha
 wage: 1500
 position: Developer
 - name: Jim
 wage: 1800
 position: DevOps Engineer
</code></pre>
You can shorten dictionaries and form more complex collections, though that's out of the scope of this tutorial.
With a refresher on YAML, we're ready to write some code that reads/writes YAML files. To achieve this, we can use either of the two popular libraries: <a rel="nofollow" target="_blank" href="https://github.com/FasterXML/jackson">Jackson</a> or <a rel="nofollow" target="_blank" href="https://bitbucket.org/asomov/snakeyaml/wiki/Home">SnakeYAML</a>. In this article, we'll be focusing on Jackson.
<h2 id="readingyamlwithjackson">Reading YAML with Jackson</h2>
Jackson is an extremely popular Java-based library used for parsing and manipulating <a target="_blank" href="/reading-and-writing-json-in-java/">JSON</a> and <a target="_blank" href="/reading-and-writing-xml-in-java/">XML</a> files.
Needless to say, it also allows us to parse and manipulate YAML files in a similar fashion to how we're already used to doing with the two previously mentioned formats.
By using Jackson as our library, we're in a familiar environment and many Java applications already have Jackson for other purposes anyway.
First, let's add Jackson to our project via Maven:
<pre><code class="hljs">&lt;dependency&gt;
 &lt;groupId&gt;com.fasterxml.jackson.core&lt;/groupId&gt;
 &lt;artifactId&gt;jackson-dataformat-yaml&lt;/artifactId&gt;
 &lt;version&gt;{$version}&lt;/version&gt;
&lt;/dependency&gt;

&lt;dependency&gt;
 &lt;groupId&gt;com.fasterxml.jackson.core&lt;/groupId&gt;
 &lt;artifactId&gt;jackson-databind&lt;/artifactId&gt;
 &lt;version&gt;{$version}&lt;/version&gt;
&lt;/dependency&gt;
</code></pre>
We'll need both <code>jackson-databind</code> from which we'll extract the <code>ObjectMapper</code> class and the <code>jackson-dataformat-yaml</code> dependency from which we'll extract the <code>YAMLFactory</code> class.
Just like we did with JSON and XML, the main class we'll be using for this task will be the <code>ObjectMapper</code> class. Its <code>readValue()</code> method is used to map the source (YAML file) into the result (an object of a class).
Let's first define a simple class that we'd like to map to. With respect to the previous YAML example, let's make an <code>Employee</code> class:
<pre><code class="hljs">public class Employee {

 public Employee(String name, int wage, String position, List&lt;Employee&gt; colleagues) {
 this.name = name;
 this.wage = wage;
 this.position = position;
 this.colleagues = colleagues;
 }

 // Without a default constructor, Jackson will throw an exception
 public Employee() {}

 private String name;
 private int wage;
 private String position;
 private List&lt;Employee&gt; colleagues;

 // Getters and setters

 @Override
 public String toString() {
 return &quot;\nName: &quot; + name + &quot;\nWage: &quot; + wage + &quot;\nPosition: &quot; + position + &quot;\nColleagues: &quot; + colleagues + &quot;\n&quot;;
 }
</code></pre>
Now, we'll read a YAML file, say <code>person.yaml</code>, with the contents:
<pre><code class="hljs">---
name: David
wage: 1500
position: Developer
colleagues:
 - name: Martha
 wage: 1500
 position: Developer
 - name: Jim
 wage: 1800
 position: DevOps Engineer
</code></pre>
We'll map this file to an instance of our <code>Employee</code> class using Jackson's <code>ObjectMapper</code> class:
<pre><code class="hljs">// Loading the YAML file from the /resources folder
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
File file = new File(classLoader.getResource(&quot;person.yaml&quot;).getFile());

// Instantiating a new ObjectMapper as a YAMLFactory
ObjectMapper om = new ObjectMapper(new YAMLFactory());

// Mapping the employee from the YAML file to the Employee class
Employee employee = om.readValue(file, Employee.class);

// Printing out the information
System.out.println(&quot;Employee info &quot; + employee.toString());

// Access the first element of the list and print it as well
System.out.println(&quot;Accessing first element: &quot; + employee.getColleagues().get(0).toString());
</code></pre>
Running this piece of code will yield us:
<pre><code class="hljs">Employee info
Name: David
Wage: 1500
Position: Developer
Colleagues: [
Name: Martha
Wage: 1500
Position: Developer
Colleagues: null
,
Name: Jim
Wage: 1800
Position: DevOps Engineer
Colleagues: null
]

Accessing first element:
Name: Martha
Wage: 1500
Position: Developer
Colleagues: null
</code></pre>
Our <code>person.yaml</code> file was mapped to an instance of an object which we can then use for whatever is the intended purpose.
<h2 id="writingyamlwithjackson">Writing YAML with Jackson</h2>
With reading and mapping out of the way, let's also go ahead and write to a YAML file.
As opposed to using the <code>readValue()</code> method, we'll be using the <code>writeValue()</code> method, specifying where the resulting file will end up and the object from which we're mapping into the YAML file.
Let's instantiate an employee and assign some values:
<pre><code class="hljs">List&lt;Employee&gt; colleagues = new ArrayList&lt;Employee&gt;();

colleagues.add(new Employee(&quot;Mary&quot;, 1800, &quot;Developer&quot;, null));
colleagues.add(new Employee(&quot;Jane&quot;, 1200, &quot;Developer&quot;, null));
colleagues.add(new Employee(&quot;Tim&quot;, 1600, &quot;Developer&quot;, null));
colleagues.add(new Employee(&quot;Vladimir&quot;, 1000, &quot;Developer&quot;, null));

// We want to save this Employee in a YAML file
Employee employee = new Employee(&quot;David&quot;, 1500, &quot;Developer&quot;, colleagues);

// ObjectMapper is instantiated just like before
ObjectMapper om = new ObjectMapper(new YAMLFactory());

// We write the `employee` into `person2.yaml`
om.writeValue(new File(&quot;/src/main/resources/person2.yaml&quot;), employee);
</code></pre>
Running this piece of code will yield us a YAML file with the contents:
<pre><code class="hljs">---
name: &quot;David&quot;
wage: 1500
position: &quot;Developer&quot;
colleagues:
- name: &quot;Mary&quot;
 wage: 1800
 position: &quot;Developer&quot;
 colleagues: null
- name: &quot;Jane&quot;
 wage: 1200
 position: &quot;Developer&quot;
 colleagues: null
- name: &quot;Tim&quot;
 wage: 1600
 position: &quot;Developer&quot;
 colleagues: null
- name: &quot;Vladimir&quot;
 wage: 1000
 position: &quot;Developer&quot;
 colleagues: null
</code></pre>
As you can see, the Java <code>employee</code> object was serialized in to a YAML-formatted file by using Jackson's support of YAML.
<h2 id="conclusion">Conclusion</h2>
YAML-formatted files are becoming increasingly popular for defining properties of tools and applications due to the very human-readable syntax. Besides containing configuration properties, they're also being used more and more for data transmission, similar to how JSON is used, although their use in this area is still not as prevalent in the wild.
Jackson is an extremely popular Java-based library used for parsing and manipulating <a target="_blank" href="/reading-and-writing-json-in-java/">JSON</a> and <a target="_blank" href="/reading-and-writing-xml-in-java/">XML</a> files. In addition, it was extended to allow developers to also work with the YAML file format.

YAML is a simple, human-readable data-serialization language, much like JSON. Using Jackson, a popular Java library present in many projects, we can easily read and write YAML.