<h2 id="introduction">Introduction</h2>
Saving a String into files can be done in a few ways using Java. In this article, we'll show some common methods for writing a String into a file.
Here's a list of all the classes and methods we'll go over:
<ul>
<li><a href="#files.writestring">Files.writeString()</a></li>
<li><a href="#files.write">Files.write()</a></li>
<li><a href="#filewriter">FileWriter</a></li>
<li><a href="#bufferedwriter">BufferedWriter</a></li>
<li><a href="#printwriter">PrintWriter</a></li>
</ul>
<h2 id="fileswritestring">Files.writeString()</h2>
Since Java 11, the <code>Files</code> class contains a useful utility method <code>Files.writeString()</code>. This method comes in two variants. The most basic form requires a <code>Path</code> of the file to write to and the textual contents. The other variant also accepts an optional <code>CharSet</code>:
<pre><code class="hljs">Path path = Paths.get(&quot;output.txt&quot;);
String contents = &quot;Hello&quot;;

try {
 Files.writeString(path, contents, StandardCharsets.UTF_8);
} catch (IOException ex) {
 // Handle exception
}
</code></pre>
There is little room for flexibility here, but it works great if you need to write something down into a file quickly.
<h2 id="fileswrite">Files.write()</h2>
A String, like other objects, can be converted into a <code>byte[]</code>. The <code>Files.write()</code> method deals with bytes:
<pre><code class="hljs">Path path = Paths.get(&quot;output.txt&quot;);
String someString = &quot;Hello World&quot;;
byte[] bytes = someString.getBytes();

try {
 Files.write(path, bytes);
} catch (IOException ex) {
 // Handle exception
}
</code></pre>
There is no need to close any resources since we haven't opened any resources ourselves.
<h2 id="filewriter">FileWriter</h2>
<code>FileWriter</code> is one of the simplest ways to write some textual contents into a file. We'll create a <code>File</code> instance and pass it into the <code>FileWriter</code> constructor to &quot;bridge&quot; them.
Then, we simply use the <code>FileWriter</code> instance to write to it:
<pre><code class="hljs">File output = new File(&quot;output.txt&quot;);
FileWriter writer = new FileWriter(output);

writer.write(&quot;This text was written with a FileWriter&quot;);
writer.flush();
writer.close();
</code></pre>
After using the writer, it's important to flush and close the resources. Alternatively, you can do this with the <a target="_blank" href="/the-try-with-resources-statement-in-java/"><code>try-with-resources</code></a> syntax:
<pre><code class="hljs">try(FileWriter writer = new FileWriter(&quot;output.txt&quot;)) {
 writer.write(&quot;This text was written with a FileWriter&quot;); 
}
catch(IOException e){
 // Handle the exception
}
</code></pre>
<h2 id="bufferedwriter">BufferedWriter</h2>
<code>BufferedWriter</code> is a wrapper object that is used around objects of type <code>Writer</code>. If we have an existing <code>Writer</code> such as <code>FileWriter</code>, we can wrap it inside a <code>BuffereWriter</code>.
<code>BufferedWriter</code> is best used when there are multiple <code>write()</code> operations for a file. In this case, those multiple writes are temporarily stored into an internal buffer and written into a file only when there is enough content. This avoids having to store each new chunk of text into a file and instead provides an appropriate buffer for temporary storage.
Using a <code>BufferedWriter</code> is much more efficient than <code>FileWriter</code> for multiple writes, but it doesn't help with a single write.
In fact, using <code>BufferedWriter</code> for a single write would cause unnecessary overhead. For such simple cases, <code>FileWriter</code> is a much better option.
Let's create a <code>BufferedWriter</code>:
<pre><code class="hljs">BufferedWriter writer = new BufferedWriter(new FileWriter(&quot;output.txt&quot;));
</code></pre>
<code>BufferedWriter</code> and <code>FileWriter</code> both extend <code>Writer</code> so they have the same methods:
<pre><code class="hljs">try(BufferedWriter writer = new BufferedWriter(new FileWriter(&quot;output.txt&quot;))) {
 writer.write(&quot;Written with BufferedWriter);
}
catch(IOException e){
 // Handle the exception
}
</code></pre>
<h2 id="printwriter">PrintWriter</h2>
<code>PrintWriter</code> lets us format the text before writing it down. It contains methods we're used to, such as <code>printf()</code>, <code>println()</code>, etc. Let's create a <code>PrintWriter</code>:
<pre><code class="hljs">File output = new File(&quot;output.txt&quot;);
PrintWriter writer = new PrintWriter(output);
</code></pre>
A better way to work with a <code>PrintWriter</code> is with the <code>try-with-resources</code> syntax:
<pre><code class="hljs">try(PrintWriter writer = new PrintWriter(new FileWriter(&quot;output.txt&quot;))) {
 // Write using the PrintWriter instance
} catch {
 // Handle Exception
}
</code></pre>
Once we have a <code>PrintWriter</code> instance, let's explore some of the methods it provides.
<h3 id="printwriterwithappend">PrintWriter with append()</h3>
<code>PrintWriter</code>, like the <code>StringBuilder</code> provides the <code>append()</code> method which allows us to append contents to the end of an existing file.
Let's <code>appent()</code> some text to an empty <code>writer</code>:
<pre><code class="hljs">writer.append(&quot;Welcome to my fruit store! From me, you can buy:\n&quot;);
writer.append(&quot;Apples&quot;);
writer.append(&quot;\n&quot;);
writer.append(&quot;Oranges&quot;);
writer.append(&quot;\n&quot;);
writer.append(&quot;Bananas&quot;);
</code></pre>
The <code>append()</code> method returns the <code>PrintWriter</code> object it was called upon. This makes it possible to chain <code>append()</code> methods and organize them more neatly:
<pre><code class="hljs">writer.append(&quot;Welcome to my fruit store! From me, you can buy:\n&quot;);
writer.append(&quot;Apples\n&quot;).append(&quot;Oranges\n&quot;).append(&quot;Bananas\n&quot;);
</code></pre>
<h3 id="printwriterwithprint">PrintWriter with print()</h3>
<code>PrintWriter</code> contains methods for formatted printing. These include <code>print()</code>, <code>printf()</code>, and <code>println()</code>:
<pre><code class="hljs">writer.print(&quot;Welcome to my fruit store %f&quot;, 2.0);
writer.printf(&quot;From me, you can buy %s and %s.&quot;, &quot;apples&quot;, &quot;oranges&quot;);
</code></pre>
<h3 id="printwriterwithwrite">PrintWriter with write()</h3>
With <code>write()</code>, we can write many different types of textual contents into the stream. Examples include char arrays, Strings, and integers:
<pre><code class="hljs">char[] hello = {&#x27;H&#x27;, &#x27;e&#x27;, &#x27;l&#x27;, &#x27;l&#x27;, &#x27;o&#x27;, &#x27;!&#x27;, &#x27;\n&#x27;};

writer.write(hello);
writer.write(&quot;Welcome to my fruit store\n&quot;);
writer.write(&quot;From me, you can buy apples and oranges&quot;);
</code></pre>
The <code>write()</code> method only accepts content without formatting options so it's similar to <code>print()</code>, but can't format Strings.
To finalize each <code>PrintWriter</code> &quot;session&quot; of either appending, printing, or writing, it's important to flush and close the stream:
<pre><code class="hljs">writer.flush();
writer.close();
</code></pre>
The <code>flush()</code> method &quot;flushes&quot; the contents into the file and <code>close()</code> permanently closes the stream.
Note: If you use the <code>try-with-resources</code> syntax, it'll flush and close the stream automatically.
<h2 id="conclusion">Conclusion</h2>
In this article, we've shown some common methods for writing strings into files. There are many options because there are many possible use-cases.
We've covered the <code>Files.writeString()</code>, <code>Files.write()</code> methods, as well as the <code>FileWriter</code>, <code>BufferedWriter</code> and <code>PrintWriter</code> classes.

Luka Čupić

In this tutorial, we'll save or write a String into a File in Java using Files.writeString(), Files.write(), FileWriter, BufferedWriter and PrintWriter with formatting.

Java: Save/Write String in to a File

Java: Save/Write String Into a File

<h2 id="introduction">Introduction</h2>
<a rel="nofollow noopener" target="_blank" href="https://nodejs.org/api/async_hooks.html#async_hooks_async_hooks">Async Hooks</a> are a core module in Node.js that provides an API to track the lifetime of asynchronous resources in a Node application. An asynchronous resource can be thought of as an object that has an associated callback.
Examples include, but are not limited to: Promises, Timeouts, TCPWrap, UDP etc. The whole list of asynchronous resources that we can track using this API can be found <a rel="nofollow noopener" target="_blank" href="https://nodejs.org/api/async_hooks.html#async_hooks_type">here.</a>
The Async Hooks feature was introduced in 2017, in Node.js version 8 and is still experimental. This means that backwards-incompatible changes may still be made to future releases of the API. That being said, it's currently not deemed fit for production.
In this article, we will take a deeper look at Async Hooks - what they are, why they are important, where we can use them and how we can leverage them for a particular use case, that is, request context handling in a Node.js and Express application.
<h2 id="whatareasynchooks">What are Async Hooks?</h2>
As stated earlier, the Async Hooks class is a core Node.js module that provides an API for tracking asynchronous resources in your Node.js application. This also includes tracking of resources created by native Node modules such as <code>fs</code> and <code>net</code>.
During the lifetime of an async resource, there are 4 events which fire and we can track, with Async Hooks. These include:
<ol>
<li><code>init</code> - Called during the construction of the async resource</li>
<li><code>before</code> - Called before the callback of the resource is called</li>
<li><code>after</code> - Called after the callback of the resource has been invoked</li>
<li><code>destroy</code> - Called after the async resource is destroyed</li>
<li><code>promiseResolve</code> - Called when the <code>resolve()</code> function of a Promise is invoked.</li>
</ol>
Below is a summarized snippet of the Async Hooks API from the <a rel="nofollow noopener" target="_blank" href="https://nodejs.org/api/async_hooks.html#async_hooks_overview">overview in the Node.js documentation</a>:
<pre><code class="hljs">const async_hooks = require(&#x27;async_hooks&#x27;);

const exec_id = async_hooks.executionAsyncId();
const trigger_id = async_hooks.triggerAsyncId();
const asyncHook = async_hooks.createHook({
 init: function (asyncId, type, triggerAsyncId, resource) { },
 before: function (asyncId) { },
 after: function (asyncId) { },
 destroy: function (asyncId) { },
 promiseResolve: function (asyncId) { }
});
asyncHook.enable();
asyncHook.disable();
</code></pre>
The <code>executionAsyncId()</code> method returns an identifier of the current execution context.
The <code>triggerAsyncId()</code> method returns the identifier of the parent resource that triggered the execution of the async resource.
The <code>createHook()</code> method creates an async hook instance, taking the aforementioned events as optional callbacks.
To enable tracking of our resources, we call the <code>enable()</code> method of our async hook instance which we create with the <code>createHook()</code> method.
We can also disable tracking by calling the <code>disable()</code> function.
Having seen what the Async Hooks API entails, let's look into why we should use it.
<h2 id="whentouseasynchooks">When to use Async Hooks</h2>
The addition of Async Hooks to the core API has availed many advantages and use cases. Some of them include:
<ol>
<li>Better debugging - By using Async Hooks, we can improve and enrich stack traces of async functions.</li>
<li>Powerful tracing capabilities, especially when combined with Node's Performance API. Also, since the Async Hooks API is native, there is minimal performance overhead.</li>
<li>Web request context handling - to capture a request's information during the lifetime of that request, without passing the request object everywhere. Using Async Hooks this can be done anywhere in code and could be especially useful when tracking the behavior of users in a server.</li>
</ol>
In this article, we will be looking at how to handle request ID tracing using Async Hooks in an Express application.
<h2 id="usingasynchooksforrequestcontexthandling">Using Async Hooks for Request Context Handling</h2>
In this section, we will be illustrating how we can leverage Async Hooks to perform simple request ID tracing in a Node.js application.
<h3 id="settinguprequestcontexthandlers">Setting up Request Context Handlers</h3>
We'll begin by creating a directory where our application files will reside, then move into it:
<pre><code class="hljs">mkdir async_hooks &amp;&amp; cd async_hooks 
</code></pre>
Next, we'll need to initialize our Node.js application in this directory with <code>npm</code> and default settings:
<pre><code class="hljs">npm init -y
</code></pre>
This creates a <code>package.json</code> file at the root of the directory.
Next, we'll need to install <code>Express</code> and <code>uuid</code> packages as dependencies. We will use the <code>uuid</code> package to generate a unique ID for each incoming request.
Finally, we install the <code>esm</code> module so Node.js versions lower than v14 can run this example:
<pre><code class="hljs">npm install express uuid esm --save
</code></pre>
Next, create a <code>hooks.js</code> file at the root of the directory:
<pre><code class="hljs">touch hooks.js
</code></pre>
This file will contain the code that interacts with the <code>async_hooks</code> module. It exports two functions:
<ul>
<li>One that enables an Async Hook for an HTTP request, keeping track of its given request ID and any request data we'd like to keep.</li>
<li>The other returns the request data managed by the hook given its Async Hook ID.</li>
</ul>
Let's put that into code:
<pre><code class="hljs">require = require(&#x27;esm&#x27;)(module);
const asyncHooks = require(&#x27;async_hooks&#x27;);
const { v4 } = require(&#x27;uuid&#x27;);
const store = new Map();

const asyncHook = asyncHooks.createHook({
 init: (asyncId, _, triggerAsyncId) =&gt; {
 if (store.has(triggerAsyncId)) {
 store.set(asyncId, store.get(triggerAsyncId))
 }
 },
 destroy: (asyncId) =&gt; {
 if (store.has(asyncId)) {
 store.delete(asyncId);
 }
 }
});

asyncHook.enable();

const createRequestContext = (data, requestId = v4()) =&gt; {
 const requestInfo = { requestId, data };
 store.set(asyncHooks.executionAsyncId(), requestInfo);
 return requestInfo;
};

const getRequestContext = () =&gt; {
 return store.get(asyncHooks.executionAsyncId());
};

module.exports = { createRequestContext, getRequestContext };
</code></pre>
In this piece of code, we first require the <code>esm</code> module to provide backwards compatibility for Node versions that do not have native support for experimental module exports. This feature is used internally by the <code>uuid</code> module.
Next, we also require both the <code>async_hooks</code> and <code>uuid</code> modules. From the <code>uuid</code> module, we de-structure the <code>v4</code> function, which we shall use later to generate version 4 UUIDs.
Next, we create a store that will map each async resource to its request context. For this, we utilize a simple JavaScript map.
Next, we call the <code>createHook()</code> method of the <code>async_hooks</code> module and implement the <code>init()</code> and <code>destroy()</code> callbacks. In the implementation of our <code>init()</code> callback, we check if the <code>triggerAsyncId</code> is present in the store.
If it exists, we create a mapping of the <code>asyncId</code> to the request data stored under the <code>triggerAsyncId</code>. This in effect ensures that we store the same request object for child asynchronous resources.
The <code>destroy()</code> callback checks if the store has the <code>asyncId</code> of the resource, and deletes it if true.
To use our hook, we enable it by calling the <code>enable()</code> method of the <code>asyncHook</code> instance we've created.
Next, we create 2 functions - <code>createRequestContext()</code> and <code>getRequestContext</code> that we use to create and get our request context respectively.
The <code>createRequestContext()</code> function receives the request data and a unique ID as arguments. It then creates a <code>requestInfo</code> object from both arguments and attempts to update the store with the async ID of the current execution context as key, and the <code>requestInfo</code> as the value.
The <code>getRequestContext()</code> function, on the other hand, checks if the store contains an ID corresponding to the ID of the current execution context.
We finally export both functions using the <code>module.exports()</code> syntax.
We've successfully set up our request context handling functionality. Let's proceed to set up our <code>Express</code> server that will receive the requests.
<h3 id="settinguptheexpressserver">Setting up the Express Server</h3>
Having set up our context, we'll now proceed to create our <code>Express</code> server so we can capture HTTP requests. To do so, create a <code>server.js</code> file at the root of the directory as follows:
<pre><code class="hljs">touch server.js
</code></pre>
Our server will accept an HTTP request on port 3000. It creates an Async Hook to track every request by calling the <code>createRequestContext()</code> in a middleware function - a function that has access to an HTTP's request and response objects. The server then sends a JSON response with the data captured by the Async Hook.
Inside the <code>server.js</code> file, enter the following code:
<pre><code class="hljs">const express = require(&#x27;express&#x27;);
const ah = require(&#x27;./hooks&#x27;);
const app = express();
const port = 3000;

app.use((request, response, next) =&gt; {
 const data = { headers: request.headers };
 ah.createRequestContext(data);
 next();
});

const requestHandler = (request, response, next) =&gt; {
 const reqContext = ah.getRequestContext();
 response.json(reqContext);
 next()
};

app.get(&#x27;/&#x27;, requestHandler)

app.listen(port, (err) =&gt; {
 if (err) {
 return console.error(err);
 }
 console.log(`server is listening on ${port}`);
});
</code></pre>
In this piece of code, we require <code>express</code> and our <code>hooks</code> modules as dependencies. We then create an <code>Express</code> app by calling the <code>express()</code> function.
Next, we set up a middleware that de-structures the request headers saving them to a variable called <code>data</code>. It then calls the <code>createRequestContext()</code> function passing <code>data</code> as an argument. This ensures that the headers of the request will be preserved throughout the request's lifecycle with the Async Hook.
Finally, we call the <code>next()</code> function to go to the next middleware in our middleware pipeline or invoke the next route handler.
After our middleware, we write the <code>requestHandler()</code> function that handles a <code>GET</code> request on the server's root domain. You'll notice that in this function, we can have access to our request context through the <code>getRequestContext()</code> function. This function returns an object representing the request headers and request ID generated and stored in the request context.
We then create a simple endpoint and attach our request handler as a callback.
Finally, we make our server listen to connections on port 3000 by calling the <code>listen()</code> method of our app instance.
Before running the code, open the <code>package.json</code> file at the root of the directory and replace the <code>test</code> section of the script with this:
<pre><code class="hljs">&quot;start&quot;: &quot;node server.js&quot;
</code></pre>
This done, we can run our app with the following command:
<pre><code class="hljs">npm start
</code></pre>
You should receive a response on your terminal indicating that the app is running on port 3000, as shown:
<pre><code class="hljs">&gt; async-hooks-demo@1.0.0 start /Users/allanmogusu/StackAbuse/async-hooks-demo
&gt; node server.js

(node:88410) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time
server is listening on 3000
</code></pre>
With our app running, open a separate terminal instance and run the following <code>curl</code> command to test out our default route:
<pre><code class="hljs">curl http://localhost:3000
</code></pre>
This <code>curl</code> command makes a <code>GET</code> request to our default route. You should get a response similar to this:
<pre><code class="hljs">$ curl http://localhost:3000
{&quot;requestId&quot;:&quot;3aad88a6-07bb-41e0-ab5a-fa9d5c0269a7&quot;,&quot;data&quot;:{&quot;headers&quot;:{&quot;host&quot;:&quot;localhost:3000&quot;,&quot;user-agent&quot;:&quot;curl/7.64.1&quot;,&quot;accept&quot;:&quot;*/*&quot;}}}%
</code></pre>
Notice that the generated <code>requestId</code> and our request headers are returned. Repeating the command should generate a new request ID since we shall be making a new request:
<pre><code class="hljs">$ curl http://localhost:3000
{&quot;requestId&quot;:&quot;38da84792-e782-47dc-92b4-691f4285b172&quot;,&quot;data&quot;:{&quot;headers&quot;:{&quot;host&quot;:&quot;localhost:3000&quot;,&quot;user-agent&quot;:&quot;curl/7.64.1&quot;,&quot;accept&quot;:&quot;*/*&quot;}}}%
</code></pre>
The response contains the ID we generated for the request and the headers we captured in the middleware function. With Async Hooks, we could easily pass data from one middleware to another for the same request.
<h2 id="conclusion">Conclusion</h2>
Async Hooks provides an API for tracking the lifetime of asynchronous resources in a Node.js application.
In this article, we have looked briefly at the Async Hooks API, the functionality it provides, and how we can leverage it. We have specifically covered a basic example of how we can use Async Hooks to do web request context handling and tracing efficiently and cleanly.
Since Node.js version 14 however, the Async Hooks API ships with async local storage, an API that makes request context handling easier in Node.js. You can read more about it <a rel="nofollow noopener" target="_blank" href="https://nodejs.org/api/async_hooks.html#async_hooks_class_asynclocalstorage">here.</a> Also, the code for this tutorial can be accessed <a rel="nofollow noopener" target="_blank" href="https://github.com/Allan690/async-hooks-demo">here.</a>

Marcus Sanatan

Allan Mogusu

Node.js Async Hooks provide an API to track the lifetime of asynchronous resources in a Node application. Let's use Async Hooks to track HTTP requests.

Using Async Hooks for Request Context Handling in Node.js

<h2 id="introduction">Introduction</h2>
The <code>forEach()</code> method is part of the <code>Stream</code> interface and is used to execute a specified operation, defined by a <code>Consumer</code>.
The <code>Consumer</code> interface represents any operation that takes an argument as input, and has no output. This sort of behavior is acceptable because the <code>forEach()</code> method is used to change the program's state via side-effects, not explicit return types.
Therefore, the best target candidates for <code>Consumers</code> are lambda functions and method references. It's worth noting that <code>forEach()</code> can be used on any <code>Collection</code>.
<h2 id="foreachonlist">forEach() on List</h2>
The <code>forEach()</code> method is a terminal operation, which means that after we call this method, the stream along with all of its integrated transformations will be materialized. That is to say, they'll &quot;gain substance&quot;, rather than being streamed.
Let's generate a small list:
<pre><code class="hljs">List&lt;Integer&gt; list = new ArrayList&lt;Integer&gt;();
list.add(1);
list.add(2);
list.add(3);
</code></pre>
Traditionally, you could write a for-each loop to go through it:
<pre><code class="hljs">for (Integer element : list) {
 System.out.print(element + &quot; &quot;);
}
</code></pre>
This would print:
<pre><code class="hljs">1 2 3
</code></pre>
Alternatively, we can use the <code>forEach()</code> method on a <code>Stream</code>:
<pre><code class="hljs">list.stream().forEach((k) -&gt; {
 System.out.print(k + &quot; &quot;);
});
</code></pre>
This also outputs:
<pre><code class="hljs">1 2 3
</code></pre>
We can make this even simpler via a <a target="_blank" href="/method-references-in-java-8">method reference</a>:
<pre><code class="hljs">list.stream().forEach(System.out::println);
</code></pre>
<h2 id="foreachonmap">forEach() on Map</h2>
The <code>forEach()</code> method is really useful if we want to avoid chaining many stream methods. Let's generate a map with a few movies and their respective IMDB scores:
<pre><code class="hljs">Map&lt;String, Double&gt; map = new HashMap&lt;String, Double&gt;();
map.put(&quot;Forrest Gump&quot;, 8.8);
map.put(&quot;The Matrix&quot;, 8.7);
map.put(&quot;The Hunt&quot;, 8.3);
map.put(&quot;Monty Python&#x27;s Life of Brian&quot;, 8.1);
map.put(&quot;Who&#x27;s Singin&#x27; Over There?&quot;, 8.9);
</code></pre>
Now, let's print out the values of each film that has a score higher than <code>8.4</code>:
<pre><code class="hljs">map.entrySet()
 .stream()
 .filter(entry -&gt; entry.getValue() &gt; 8.4)
 .forEach(entry -&gt; System.out.println(entry.getKey() + &quot;: &quot; + entry.getValue()));
</code></pre>
This results in:
<pre><code class="hljs">Forrest Gump: 8.8
The Matrix: 8.7
Who&#x27;s Singin&#x27; Over There?: 8.9
</code></pre>
Here, we've converted a <code>Map</code> to a <code>Set</code> via <code>entrySet()</code>, streamed it, filtered based on the score and finally printed them out via a <code>forEach()</code>. Instead of basing this on the return of <code>filter()</code>, we could've based our logic on side-effects and skipped the <code>filter()</code> method:
<pre><code class="hljs">map.entrySet()
 .stream()
 .forEach(entry -&gt; {
 if (entry.getValue() &gt; 8.4) {
 System.out.println(entry.getKey() + &quot;: &quot; + entry.getValue());
 }
 }
 );
</code></pre>
This results in:
<pre><code class="hljs">Forrest Gump: 8.8
The Matrix: 8.7
Who&#x27;s Singin&#x27; Over There?: 8.9
</code></pre>
Finally, we can omit both the <code>stream()</code> and <code>filter()</code> methods by starting out with <code>forEach()</code> in the beginning:
<pre><code class="hljs">map.forEach((k, v) -&gt; {
 if (v &gt; 8.4) {
 System.out.println(k + &quot;: &quot; + v);
 }
 });
</code></pre>
This results in:
<pre><code class="hljs">Forrest Gump: 8.8
The Matrix: 8.7
Who&#x27;s Singin&#x27; Over There?: 8.9
</code></pre>
<h2 id="foreachonset">forEach() on Set</h2>
Let's take a look at how we can use the <code>forEach</code> method on a <code>Set</code> in a bit more tangible context. First, let's define a class that represents an <code>Employee</code> of a company:
<pre><code class="hljs">public class Employee {
 private String name;
 private double workedHours;
 private double dedicationScore;
 
 // Constructor, Getters and Setters
 
 public void calculateDedication() {
 dedicationScore = workedHours*1.5;
 }
 
 public void receiveReward() {
 System.out.println(String
 .format(&quot;%s just got a reward for being a dedicated worker!&quot;, name));
 }
}
</code></pre>
Imagining we're the manager, we'll want to pick out certain employees that have worked overtime and award them for the hard work. First, let's make a <code>Set</code>:
<pre><code class="hljs"> Set&lt;Employee&gt; employees = new HashSet&lt;Employee&gt;();

 employees.add(new Employee(&quot;Vladimir&quot;, 60));
 employees.add(new Employee(&quot;John&quot;, 25));
 employees.add(new Employee(&quot;David&quot;, 40));
 employees.add(new Employee(&quot;Darinka&quot;, 60));
</code></pre>
Then, let's calculate each employee's dedication score:
<pre><code class="hljs">employees.stream().forEach(Employee::calculateDedication);
</code></pre>
Now that each employee has a dedication score, let's remove the ones with a score that's too low:
<pre><code class="hljs">Set&lt;Employee&gt; regular = employees.stream()
 .filter(employee -&gt; employee.getDedicationScore() &lt;= 60)
 .collect(Collectors.toSet());
employees.removeAll(regular);
</code></pre>
Finally, let's reward the employees for their hard work:
<pre><code class="hljs">employees.stream().forEach(employee -&gt; employee.receiveReward());
</code></pre>
And for clarity's sake, let's print out the names of the lucky workers:
<pre><code class="hljs">System.out.println(&quot;Awarded employees:&quot;);
employees.stream().map(employee -&gt; employee.getName()).forEach(employee -&gt; System.out.println(employee));
</code></pre>
After running the code above, we get the following output:
<pre><code class="hljs">Vladimir just got a reward for being a dedicated worker!
Darinka just got a reward for being a dedicated worker!
Awarded employees:
Vladimir
Darinka
</code></pre>
<h2 id="sideeffectsvsreturnvalues">Side-effects Vs Return Values</h2>
The point of every command is to evaluate the expression from start to finish. Everything in-between is a side-effect. In this context, it means altering the state, flow or variables without returning any values.
Let's take a look at the difference on another list:
<pre><code class="hljs">List&lt;Integer&gt; targetList = Arrays.asList(1, -2, 3, -4, 5, 6, -7);
</code></pre>
This approach is based on the returned values from an <code>ArrayList</code>:
<pre><code class="hljs">long result1 = targetList
 .stream()
 .filter(integer -&gt; integer &gt; 0)
 .count();
System.out.println(&quot;Result: &quot; + result1);
</code></pre>
This results in:
<pre><code class="hljs">Result: 4
</code></pre>
And now, instead of basing the logic of the program on the return type, we'll perform a <code>forEach()</code> on the stream and add the results to an <code>AtomicInteger</code> (streams operate concurrently):
<pre><code class="hljs">AtomicInteger result2 = new AtomicInteger();

targetList.stream().forEach(integer -&gt; {
 if (integer &gt; 0)
 result2.addAndGet(1);
 });
System.out.println(&quot;Result: &quot; + result2);
</code></pre>
<h2 id="conclusion">Conclusion</h2>
The <code>forEach()</code> method is a really useful method to use to iterate over collections in Java in a functional approach.
In certain cases, they can massively simplify the code and enhance clarity and brevity. In this article, we've gone over the basics of using a <code>forEach()</code> and then covered examples of the method on a <code>List</code>, <code>Map</code> and <code>Set</code>.
We've covered the difference between the <code>for-each</code> loop and the <code>forEach()</code>, as well as the difference between basing logic on return values versus side-effects.

Vladimir Batoćanin

In this tutorial, we'll be going over the Java Streams forEach() method. We'll cover basic usage and examples of forEach() on a List, Map and Set.

Guide to Java Streams: forEach() with Examples

<h2 id="introduction">Introduction</h2>
Reading user input is the first step towards writing useful Java software. User input can come in many forms - mouse and keyboard interactions, a network request, command-line arguments, files that are updated with data relevant for a program's execution, etc.
We're going to focus on keyboard input via something called the standard input stream. You may recognize it as Java's <code>System.in</code>.
We're going to use the <code>Scanner</code> class to make our interaction with the underlying stream easier. Since <code>Scanner</code> has some downfalls, we'll also be using the <code>BufferedReader</code> and <code>InputStreamReader</code> classes to process the <code>System.in</code> stream.
In the end, we'll Decorate the <code>InputStream</code> class and implement our own custom <code>UncloseableInputStream</code> to handle issues with the <code>Scanner</code> class.
<h2 id="javascannerclass">Java Scanner Class</h2>
The <code>java.util.Scanner</code> class is a simple scanner that can parse and handle primitive inputs, Strings and streams. Since <code>System.in</code> is just an <code>InputStream</code>, we can construct a <code>Scanner</code> as such:
<pre><code class="hljs">Scanner sc = new Scanner(System.in);
</code></pre>
This <code>Scanner</code> instance can now scan and parse booleans, integers, floats, bytes and Strings.
Let's see how we can extract information from a <code>Scanner</code> into variables we can work with:
<pre><code class="hljs">Scanner sc = new Scanner(System.in);
 
// Read an integer into a variable
int myInteger = sc.nextInt();
 
// Read a byte into a variable
byte myByte = sc.nextByte();
 
// Read a line until newline or EOF into a string
String myLine = sc.nextLine();
 
// Closing the scanner
sc.close();
</code></pre>
Again, the constructor doesn't have to take <code>System.in</code>. It can take any <code>File</code>, <code>InputStream</code>, <code>Readable</code>, <code>ReadableByteChannel</code>, <code>Path</code> (of a file to read), or even <code>String</code>. Additionally, as the second argument, it can specify a character encoding to interpret said characters by:
<pre><code class="hljs">Scanner sc = new Scanner(new FileInputStream(&quot;myFile.txt&quot;), &quot;UTF-8&quot;);
</code></pre>
Note that Scanner has to be closed when you're done working with it. The easiest way to do this is via the <a target="_blank" href="/the-try-with-resources-statement-in-java/">try-with-resources</a> statement.
<h3 id="scannermethodsforreading">Scanner Methods for Reading</h3>
Available methods for reading the next token using the scanner method are:
<table class="table-striped">
 <tr>
 <td>Method</td>
 <td>Return type</td>
 <td>Description</td>
 </tr>
 <tr>
 <td>next()</td>
 <td>String</td>
 <td>Finds and returns the next complete token from the scanner.</td>
 </tr>
 <tr>
 <td>nextByte()</td>
 <td>byte</td>
 <td>Scans the next token of the input as a byte.</td>
 </tr>
 <tr>
 <td>nextDouble()</td>
 <td>double</td>
 <td>Scans the next token of the input as a double.</td>
 </tr>
 <tr>
 <td>nextFloat()</td>
 <td>float</td>
 <td>Scans the next token of the input as a float.</td>
 </tr>
 <tr>
 <td>nextInt()</td>
 <td>int</td>
 <td>Scans the next token of the input as an int.</td>
 </tr>
 <tr>
 <td>nextLong()</td>
 <td>long</td>
 <td>Scans the next token of the input as a long.</td>
 </tr>
 <tr>
 <td>nextShort()</td>
 <td>short</td>
 <td>Scans the next token of the input as a short.</td>
 </tr>
 <tr>
 <td>nextBoolean()</td>
 <td>boolean</td>
 <td>Scans the next token of the input into a boolean value and returns that value.</td>
 </tr>
 <tr>
 <td>nextLine()</td>
 <td>String</td>
 <td>Advances this scanner past the current line and returns the input that was skipped.</td>
 </tr>
</table>
A method worth noting is the <code>hasNext()</code> method - a generic method that will return <code>true</code> if there's any type of token to be read. There are type-specific methods such as <code>hasNextInt()</code>, <code>hasNextFloat()</code>, <code>hasNextLine()</code> etc. which you can use in the same fashion.
<h3 id="problemswhenusingsysteminwithscanner">Problems When Using System.in With Scanner</h3>
A big problem with <code>System.in</code> is that it's an <code>InputStream</code>. When working with it, the <code>Scanner</code> will always expect more input until the <code>InputStream</code> is closed. Once the stream is closed, we can no longer access the input from the <code>Scanner</code>.
Besides closing itself, the <code>Scanner</code> class will also close the <code>InputStream</code> if it implements <code>Closeable</code>.
Since <code>InputStream</code> does, that means that the <code>Scanner</code> will close the <code>System.in</code> stream to the entirety of your program.
That being said, if you close a <code>Scanner</code> and thus the <code>System.in</code> as well, you can't use <code>System.in</code> again:
<pre><code class="hljs">Scanner sc = new Scanner(System.in);
System.out.println(sc.nextInt());
sc.close();
System.out.println(&quot;Closing the scanner...&quot;);

sc = new Scanner(System.in);
System.out.println(sc.nextInt());
sc.close();
System.out.println(&quot;Closing the scanner...&quot;);
</code></pre>
This results in:
<pre><code class="hljs">1
1
Closing the scanner...
Exception in thread &quot;main&quot; java.util.NoSuchElementException
 at java.util.Scanner.throwFor(Scanner.java:862)
 at java.util.Scanner.next(Scanner.java:1485)
 at java.util.Scanner.nextInt(Scanner.java:2117)
 at java.util.Scanner.nextInt(Scanner.java:2076)
 at com.company.Main.main(Main.java:18)

</code></pre>
This makes work with <code>Scanner</code> and <code>System.in</code> much more complicated. We'll be fixing this in the final section.
<h2 id="bufferedreaderandinputstreamreader">BufferedReader and InputStreamReader</h2>
Instead of a <code>Scanner</code>, you can also use a <code>BufferedReader</code> alongside an <code>InputStreamReader</code> to get the user input:
<pre><code class="hljs">BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

String line;

while((line = br.readLine()) != null){
 System.out.println(String.format(&quot;The input is: %s&quot;, line));
}
</code></pre>
Here, we just repeat back the input String with a prefix:
<pre><code class="hljs">Hello!
The input is: Hello!
I&#x27;d like to order some extra large fries.
The input is: I&#x27;d like to order some extra large fries.
^D

Process finished with exit code 0
</code></pre>
The <code>BufferedReader</code> is well-suited for reading Strings, but doesn't have built-in methods for handling numbers. To read an integer, you'd have to parse it from a String:
<pre><code class="hljs">BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

int a = Integer.parseInt(br.readLine());
System.out.println(a);
</code></pre>
This works just fine now:
<pre><code class="hljs">5
5
</code></pre>
<h2 id="customuncloseableinputstream">Custom Uncloseable InputStream</h2>
Fortunately, there's a solution to <code>Scanner</code> closing the <code>System.in</code> stream thanks to the <a href="https://stackabuse.com/structural-design-patterns-in-java/#decorator">Decorator Design Pattern</a>. We can implement our own <code>InputStream</code> and just make the <code>close()</code> method do nothing so that when <code>Scanner</code> calls it, it won't affect the underlying standard input:
<pre><code class="hljs">public class UnclosableInputStreamDecorator extends InputStream {

 private final InputStream inputStream;

 public UnclosableInputStreamDecorator(InputStream inputStream) {
 this.inputStream = inputStream;
 }

 @Override
 public int read() throws IOException {
 return inputStream.read();
 }

 @Override
 public int read(byte[] b) throws IOException {
 return inputStream.read(b);
 }

 @Override
 public int read(byte[] b, int off, int len) throws IOException {
 return inputStream.read(b, off, len);
 }

 @Override
 public long skip(long n) throws IOException {
 return inputStream.skip(n);
 }

 @Override
 public int available() throws IOException {
 return inputStream.available();
 }

 @Override
 public synchronized void mark(int readlimit) {
 inputStream.mark(readlimit);
 }

 @Override
 public synchronized void reset() throws IOException {
 inputStream.reset();
 }

 @Override
 public boolean markSupported() {
 return inputStream.markSupported();
 }

 @Override
 public void close() throws IOException {
 // Do nothing
 }
}
</code></pre>
When we modify our troublesome code to use the custom <code>InputStream</code>, it will execute without issues:
<pre><code class="hljs">public class ScannerDemo {
 public static void main(String[] args) {
 
 Scanner sc = new Scanner(new UnclosableInputStreamDecorator(System.in));
 System.out.println(sc.nextInt());
 sc.close();
 System.out.println(&quot;Closing the scanner...&quot;);

 sc = new Scanner(new UnclosableInputStreamDecorator(System.in));
 System.out.println(sc.nextInt());
 sc.close();
 System.out.println(&quot;Closing the scanner...&quot;);
 }
}
</code></pre>
Running this will result in:
<pre><code class="hljs">1
1
Closing the scanner...
1
1
Closing the scanner...
</code></pre>
<h2 id="conclusion">Conclusion</h2>
In this article, we've covered how to use the <code>Scanner</code> class to read user input. We've then used the <code>BufferedReader</code> class alongside the <code>InputStreamReader</code> as an alternative approach.
Finally, we've implemented our own <code>InputStream</code> to avoid the issue of <code>Scanner</code> closing the <code>System.in</code> stream for the whole program.
Hopefully, you've learned how to handle basic console input in Java and some common errors that you might encounter along the way.

Darinka Zobenica

In this article, we'll use the Scanner, BufferedReader and InputStreamReader classes to get user input in Java. We'll also implement a custom InputStream class for processing.

How to Get User Input in Java

Olivera Popović

In this tutorial, we'll be diving into the basic data types in Python such as int, float, str, bool and duct. We'll also be covering examples of their usage and some caveats.

Guide to Basic Data Types in Python with Examples

Leodanis Pozo Ramos

Variance and Standard Deviation measure the spread of a dataset. In this tutorial, we'll learn how these functions work and how to code them in Python.

Calculating Variance and Standard Deviation in Python

<h2 id="introduction">Introduction</h2>
One of the most popular services available on Amazon Web Services is EC2, which stands for Elastic Compute Cloud. EC2 makes it easy for developers and users to create and manage cloud-hosted virtual machines, also known as EC2 instances.
EC2 provides a web interface which makes it easy to spin up and work with VM instances, including configuration, authentication, connection, maintenance, storage, scaling, and more.
In addition to creating and working with the EC2 service through the web interface, AWS provides SDK's that allow VM operations to be performed directly from code.
In this article, we'll discuss using the Java SDK and API to create an EC2 instance, start and stop the instance, reboot it, back it up into an image, and restoring it from the backup.
<h2 id="setupawscredentialsforthejavasdk">Setup AWS Credentials for the Java SDK</h2>
Let's start by learning how to create a set of AWS credentials, which are required to access AWS and make API calls through the SDK. The easiest way to do this is to log into the AWS console and create a new IAM (Identity and Access Management) role.
Follow these steps to do this:
<ol>
<li>
Log into the <a rel="noopener nofollow" href="https://aws.amazon.com" target="_blank">AWS Console</a>.
</li>
<li>
Click on the Services menu in the top left of the screen, search for IAM, and click on the dropdown option that appears.
</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/automating-aws-ec2-management-with-java-01.png" alt="AWS IAM Services Menu">
<ol start="3">
<li>Under the Security Status heading, expand the Create individual IAM users option and click the Manage Users button.</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/automating-aws-ec2-management-with-java-02.png" alt="AWS IAM Manage Users">
<ol start="4">
<li>
Click the Add user button.
</li>
<li>
Enter the user name for your new IAM user and check the box for Programmatic access.
</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/automating-aws-ec2-management-with-java-03.png" alt="AWS IAM User Details">
<ol start="6">
<li>
Click the Next: Permissions button and then select Attach existing policies directly.
</li>
<li>
Type EC2 into the search box and in the results, check the box for AmazonEC2FullAccess.
</li>
</ol>
<img src="https://stackabuse.s3.amazonaws.com/media/automating-aws-ec2-management-with-java-04.png" alt="AWS IAM EC2 Permissions">
<ol start="8">
<li>
Click the Next: Tags button, then click the Next: Review button.
</li>
<li>
Review the IAM user configuration and click the Create user button.
</li>
<li>
You'll be taken to a confirmation page, where you can copy out the Access key ID and Secret access key which are the credentials you'll use to access the AWS API through the Java SDK.
</li>
</ol>
By default, the SDK will look up for the credentials in the Default credential profile file, which is a file typically located at <code>~/.aws/credentials</code> on your local machine. You'll need to create this file yourself and add the IAM credentials into it.
To configure this yourself, create the new file <code>~/.aws/credentials</code> and add the following contents, replacing the access key and secret key with the values from your newly created IAM user in the AWS console:
<pre><code class="hljs">aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY
</code></pre>
Create a default region file for the AWS SDK to use by adding a new file called <code>~/.aws/config</code> with the following contents (you can replace the region with one closer to where your users live for optimal performance):
<pre><code class="hljs">region = US_WEST_2
</code></pre>
The local environment should now be configured for the AWS Java SDK to authenticate successfully.
<h2 id="installingthesdkwithmaven">Installing the SDK with Maven</h2>
If you're using Maven, add the following dependency to include the AWS Java SDK:
<pre><code class="hljs">&lt;dependency&gt;
 &lt;groupId&gt;com.amazonaws&lt;/groupId&gt;
 &lt;artifactId&gt;aws-java-sdk-ec2&lt;/artifactId&gt;
 &lt;version&gt;${version}&lt;/version&gt;
&lt;/dependency&gt;
</code></pre>
Or, if you're using Gradle:
<pre><code class="hljs">compile group: &#x27;com.amazonaws&#x27;, name: &#x27;aws-java-sdk&#x27;, version: &#x27;${version}&#x27;
</code></pre>
Now, instead of just the EC2 dependency, you could use <code>aws-java-sdk</code>, which is the entire SDK. There's a lot of dependencies in the entire SDK - 219, to be exact, so if you're only using EC2, there's no need to download them all.
<h2 id="creatinganec2instance">Creating an EC2 Instance</h2>
The AWS Java SDK for EC2 provides several classes that can be used to create a new instance. These are located in the <code>software.amazon.awssdk</code> library. These classes are:
<ul>
<li>Region: Represents an AWS hosting region to perform the SDK operations in.</li>
<li>Ec2Client: Used to set up and configure a client to connect to AWS EC2 over web services.</li>
<li>InstanceType: Represents a VM instance type/size such as <code>T2.MICRO</code>.</li>
<li>RunInstancesRequest: Represents a web request for creating EC2 instances.</li>
<li>RunInstancesResponse: Represents a web response from the AWS EC2 service after instance creation.</li>
</ul>
Let's see how we can use these to create an EC2 instance:
<pre><code class="hljs">Region region = Region.US_WEST_2;
String amiId = &quot;ami-0e34e7b9ca0ace12d&quot;;

Ec2Client ec2 = Ec2Client.builder()
 .region(region)
 .build();

RunInstancesRequest runRequest = RunInstancesRequest.builder()
 .imageId(amiId)
 .instanceType(InstanceType.T1_MICRO)
 .maxCount(1)
 .minCount(1)
 .build();

RunInstancesResponse response = ec2.runInstances(runRequest);
</code></pre>
Here, we set up a <code>Region</code> to a region close to the end-users for optimal performance. If a <code>Region</code> is not programmatically specified, the default region from the <code>~/.aws/config</code> is used.
Then, we've specified our AMI in a String. This is a unique identifier for the machine image we're using and can be obtained in the AWS Console.
With those two in place, we've instantiated an <code>Ec2Client</code> object and set the region. After that, using the <code>RunInstancesRequest</code> class, we've populated the information for a request.
The request is sent via the <code>runInstances()</code> method and the response is packed in a <code>RunInstancesResponse</code> instance.
After this, the instance can be seen in the AWS Console:
<img src="https://stackabuse.s3.amazonaws.com/media/automating-aws-ec2-management-with-java-05.png" alt="AWS EC2 instance running in console">
Let's now manipulate the instance we created, beginning with starting and stopping it.
<h2 id="startingandstoppinganinstance">Starting and Stopping an Instance</h2>
Building upon the classes used in the previous example, and in the same fashion, to send requests for starting and stopping of an instance, we have:
<ul>
<li>StartInstancesRequest: Represents a web request for starting EC2 instances.</li>
<li>StopInstancesRequest: Represents a web request for stopping EC2 instances.</li>
</ul>
Let's see how we can use these two classes to start and stop an EC2 instance:
<pre><code class="hljs">Region region = Region.US_WEST_2;
String instanceId = &quot;i-025dea8c69e83bb61&quot;;

Ec2Client ec2 = Ec2Client.builder()
 .region(region)
 .build();

StartInstancesRequest request = StartInstancesRequest.builder()
 .instanceIds(instanceId)
 .build();

ec2.startInstances(request);

StopInstancesRequest request = StopInstancesRequest.builder()
 .instanceIds(instanceId)
 .build();

ec2.stopInstances(request);
</code></pre>
We've set the <code>Region</code> and made note of our instance ID. Then, we've built an <code>Ec2Client</code>, like last time.
Finally, we've formulated a <code>StartInstancesRequest</code> with the instance ID and sent it via the <code>startInstances()</code> method.
The <code>StopInstancesRequest</code> is built the exact same way and is sent via the <code>stopInstances()</code> method.
After starting or stopping an EC2 instance in this way, the new state of the instance should be visible in the AWS EC2 console.
For convenience, both the stopping and starting actions can be performed in a single reboot request.
<h2 id="rebootinganinstance">Rebooting an Instance</h2>
In the same fashion, we're introduced to the <code>RebootInstancesRequest</code> class:
<ul>
<li>RebootInstancesRequest: Represents a web request for starting EC2 instances.</li>
</ul>
Let's see how we can construct a request to reboot an instance:
<pre><code class="hljs">Region region = Region.US_WEST_2;
String instanceId = &quot;i-025dea8c69e83bb61&quot;;

Ec2Client ec2 = Ec2Client.builder()
 .region(region)
 .build();

RebootInstancesRequest request = RebootInstancesRequest.builder()
 .instanceIds(instanceId)
 .build();

ec2.rebootInstances(request);
</code></pre>
We've set the <code>Region</code> and <code>instanceId</code> and constructed an <code>Ec2Client</code>.
Using the <code>builder()</code> method of the <code>RebootInstancesRequest</code>, we build a request with just the <code>instanceId</code>.
The request is sent via the <code>rebootInstances()</code> method, after which, the instance is rebooted.
Next, we'll look into automating the common task of backing up an existing EC2 instance to an AMI. We'll also discuss how to restore the image to a new EC2 instance.
<h2 id="backupaninstancetoanamiamazonmachineimage">Back up an Instance to an AMI (Amazon Machine Image)</h2>
As with the previous tasks, this one is taken care of with a request as well. To construct requests for backing up instances to images, we've got:
<ul>
<li>CreateImageRequest: Represents a web request for creating a new AMI.</li>
</ul>
Let's see how we can use the <code>CreateImageRequest</code> class to back up an image based on an existing instance:
<pre><code class="hljs">Region region = Region.US_WEST_2;
String instanceId = &quot;i-025dea8c69e83bb61&quot;;

Ec2Client ec2 = Ec2Client.builder()
 .region(region)
 .build();

CreateImageRequest request = CreateImageRequest.builder()
 .instanceId(instanceId)
 .name(&quot;newimage&quot;)
 .build();

ec2.createImage(request);
</code></pre>
After creating a new AMI from an existing EC2 instance in this way, the new AMI will be visible in the IMAGES &gt; AMIs link on the right sidebar of the EC2 console.
<h2 id="restoreaninstancefromanami">Restore an Instance from an AMI</h2>
Luckily, restoring this AMI to create a new instance is very easy since we already learned how to create a new EC2 instance in the first example of this article.
All we need to do is replace the string <code>amiId</code> (which we originally found from the desired AMI in the AWS console) with the AMI ID from the new custom AMI that we created:
<pre><code class="hljs">Region region = Region.US_WEST_2;
String amiId = &quot;created_ami&quot;;

Ec2Client ec2 = Ec2Client.builder()
 .region(region)
 .build();

RunInstancesRequest runRequest = RunInstancesRequest.builder()
 .imageId(amiId)
 .instanceType(InstanceType.T1_MICRO)
 .maxCount(1)
 .minCount(1)
 .build();

RunInstancesResponse response = ec2.runInstances(runRequest);
</code></pre>
This will allow us to use our custom AMI as a template for creating the new EC2 instance instead of the out-of-the-box AWS AMI from their marketplace.
<h2 id="conclusion">Conclusion</h2>
In this article, we discussed how to set up and configure the AWS SDK for Java, specifically for the EC2 service.
We covered the setup of credentials for AWS SDK authentication and adding required dependencies using Maven. We also detailed the Java code for some common EC2 operations such as creating a new instance, stopping/starting the instance, rebooting the instance, and backing up/restoring the instance from an AMI.
<h2 id="abouttheauthor">About the Author</h2>
This article was written by Jacob Stopak, a software developer and consultant with a passion for helping others improve their lives through code. Jacob is the author of the <a href="https://initialcommit.io/store/baby-bitcoin-guidebook-for-developers" rel="noopener nofollow" target="_blank">Baby Bitcoin Guidebook for Developers</a>, a book that dives into Bitcoin's code to help curious developers understand how Bitcoin works under the hood.

Jacob Stopak

In this article, we'll learn how to use the AWS Java SDK to create an EC2 instance, start, stop and reboot the instance, and back it up to an image.

Automating AWS EC2 Management with Java

<h2 id="introduction">Introduction</h2>
If you are a Machine Learning Engineer, Data Scientist, or a hobbyist developing Machine Learning Models from time to time just for fun, then it is very likely that you are familiar with Tensorflow.
Tensorflow is an open-source and free framework developed by Google Brain Team written in Python, C++, and CUDA. It is used to develop, test, and deploy Machine Learning models.
Initially, TensorFlow did not have full support for multiple platforms and programming languages, and it was not very fast and efficient for training Machine Learning models, but with time and after a few updates, Tensorflow is now considered as a go-to framework for developing, training and deploying machine learning models.
<h2 id="tensorflow1x">TensorFlow 1.x</h2>
Tensorflow 1.x was also a huge leap for this framework. It introduced many new features, improved performance, and open source contributions. It introduced a high-level API for TensorFlow, which made it very easy to build prototypes in no time.
It was made compatible with Keras. But the major thing that irritated the developers was that it did not feel like taking advantage of the simplicity of Python when using TensorFlow.
In TensorFlow, every model is represented as a graph, and the nodes represent the computations in the graph. It is an example of <a rel="nofollow noopener" target="_blank" href="https://en.wikipedia.org/wiki/Symbolic_programming">&quot;Symbolic Programming&quot;</a> and whereas Python is an <a rel="nofollow noopener" target="_blank" href="https://en.wikipedia.org/wiki/Imperative_programming#:~:text=In%20computer%20science,%20imperative%20programming,for%20the%20computer%20to%20perform">&quot;Imperative Programming&quot;</a> Language.
I will not go into much detail as this is beyond the scope of this article. But the point here is that with the release of PyTorch (which is much oriented towards Imperative Programming and takes advantage of Python's dynamic behavior), newbies and research scientists found PyTorch easier to understand and learn than Tensorflow and in no time PyTorch started to gain popularity.
Every Tensorflow developer was demanding the same from Tensorflow and the Google Brain Team. Moreover, TensorFlow 1.x went through a lot of development which resulted in many APIs, i.e., <code>tf.layers, tf.contrib.layers, tf.keras</code> and the developers had many options to choose from, which resulted in conflicts.
<h2 id="announcementoftensorflow20">Announcement of Tensorflow 2.0</h2>
It was pretty obvious that the Tensorflow team had to address these issues so they announced Tensorflow 2.0.
This was a huge step because to address all the issues they had to make huge changes. Many people were faced with another learning experience, but the improvements made it worth relearning.
In the training phase, we're introduced to <code>tf.data</code> and Datasets, which allow us to import and process data with ease. Then, we're introduced to distributed training over multiple CPUs, GPUs and TPUs. For serialization, we can use the <code>SavedModel</code> to either deploy to TensorFlow Hub or services like TensorFlow Serving, TensorFlow Lite or TensorFlow.js:
<img src="https://stackabuse.s3.amazonaws.com/media/whats-new-in-tensorflow-2.png" alt="tensorflow 2.0 architecture">
Credit: <a rel="nofollow noopener" target="_blank" href="https://blog.tensorflow.org/2019/09/tensorflow-20-is-now-available.html">blog.tensorflow.org</a>
<h2 id="whatsnewintensorflow20">What's new in Tensorflow 2.0</h2>
Here is a brief overview of the most important updates that came with Tensorflow 2.
<h3 id="1deployingmodelsonmultipleplatforms">1. Deploying Models on Multiple Platforms</h3>
Tensorflow was always very well suited for production, but Tensorflow 2 improved compatibility and parity across multiple platforms.
It introduced the new platform support for the <code>SavedModel</code> format that allows us to save Tensorflow models. The novelty here is that you can deploy your saved model on any platform, i.e., on Mobile or IoT devices using <a rel="nofollow noopener" target="_blank" href="https://www.tensorflow.org/lite">Tensorflow Lite</a> or Node.js with <a target="_blank" rel="nofollow noopener" href="https://www.tensorflow.org/js">Tensorflow.js</a>. Alternatively, you can use it in production environments with <a rel="nofollow noopener" target="_blank" href="https://www.tensorflow.org/tfx/guide/serving">Tensorflow Serving</a>.
Let's take a look at how you can save a compiled model:
<pre><code class="hljs">import os
import tensorflow as tf
# Building the Model
model = tf.keras.Sequential([
 tf.keras.layers.Dense(5,actiavtion=&#x27;relu&#x27;,input_shape=(16,)),
 tf.keras.layers.Dense(1,activation=&#x27;sigmoid&#x27;)])
# Compiling the Model
model.compile(loss=&#x27;binary_crossentropy&#x27;,optimizer=&#x27;adam&#x27;)

# Saving the Model
save_path = path + &quot;/version_number/&quot;
save_path = os.path.join
tf.saved_model.save(model, save_path)
</code></pre>
And there you go. You can now deploy it using any of the aforementioned services.
<h3 id="2eagerexecution">2. Eager Execution</h3>
Before Tensorflow 2, you had to create a session to run your model. In fact, if you wanted to print the value of a variable just for debugging, you first had to create a session and then write a print statement inside that session.
You had to create slow and useless placeholders to feed the input data to the model. Basically, in Tensorflow 1.x, you would first build the entire graph and then run it, as opposed to building it while running.
This felt static and clunky, especially as opposed to PyTorch, which allowed the users to create dynamic graphs during execution.
Thankfully, this was revamped in Tensorflow 2.0, which introduced us to eager execution. Let's take a look at how we would've constructed a graph in Tensorflow 1.x vs 2.0:
<pre><code class="hljs">import tensorflow as tf

&quot;&quot;&quot;Creating the Graph&quot;&quot;&quot;
# Tensorflow 1.x
# Defining two Tensorflow variables
a = tf.Variable(4)
b = tf.Variable(5)
result = tf.multiply(a,b)
</code></pre>
Now, to access the <code>result</code> variable, we'll have to execute the graph in a session:
<pre><code class="hljs"># Creating a session
with tf.Session() as sess:
 # Initializing all the Variables
 sess.run(tf.global_variables_initializer())
 print(sess.run(result))
</code></pre>
Now, instead of that, we can just directly access them:
<pre><code class="hljs">import tensorflow as tf

# Tensorflow 2.0
a = tf.Variable(4)
b = tf.Variable(5)
# No need to create a session
print(float(a*b))
</code></pre>
<h3 id="3integrationofkeraswithtensorflow">3. Integration of Keras with Tensorflow</h3>
Keras is a Neural Net and Deep Learning API built on top of Tensorflow.
Most people start out with Keras before moving on to Tensorflow or PyTorch. It was designed for fast experimentation with deep neural nets and is thus simpler.
Before Tensorflow 2.0, it was supported by the library, but wasn't integrated. Now, it's officially a high-level API. No need to install it explicitly, it comes shipped with Tensorflow and is now accessible through <code>tf.keras</code>.
This consequently results in an API cleanup and removal of <code>tf.contrib.layers</code> <code>tf.layers</code>, etc. <a rel="nofollow noopener" target="_blank" href="https://www.tensorflow.org/api_docs/python/tf/keras"><code>tf.keras</code></a> is the go-to API now. Both <code>tf.contrib.layers</code> and <code>tf.layers</code> were doing the same thing. And with <code>tf.keras</code>, there would be triple redundancy since it contains the <code>tf.keras.layers</code> module.
The team also provided a <a target="_blank" rel="nofollow noopener" href="https://www.tensorflow.org/guide/upgrade">guide</a> to upgrade your code from Tensorflow 1.x to Tensorflow 2.0 since a lot of the older packages are now deprecated.
<h3 id="4tffunctiondecorator">4. tf.function Decorator</h3>
This is also one of the most exciting features of Tensorflow 2. The <a target="_blank" rel="nofollow noopener" href="https://www.tensorflow.org/guide/function"><code>@tf.function</code></a> decorator allows your Python functions to be automatically converted to Tensorflow Graphs.
You can still have all the advantages of graph-based execution and get rid of hefty session-based programming. By applying the <code>@tf.function</code> decorator to a function such as:
<pre><code class="hljs">@tf.function
def multiply(a, b):
 return a * b

multiply(tf.ones([2, 2]), tf.ones([2, 2]))
</code></pre>
In case you are wondering, this is automatically complemented by Autograph. It generates a graph that has the exact same effects as the function we've decorated.
<h3 id="5trainingusingdistributedcomputing">5. Training using Distributed Computing</h3>
Tensorflow 2.0 comes with improved performance for training using GPUs. According to the team, this version is 3 times faster than Tensorflow 1.x.
And as of now, Tensorflow can also work with TPUs. In fact, you can work with multiple TPUs and GPUs in a distributed computing approach.
You can read more about this in the <a rel="nofollow noopener" target="_blank" href="https://www.tensorflow.org/guide/distributed_training">official guide</a>.
<h3 id="6tfdataanddatasets">6. tf.data and Datasets</h3>
With <a rel="nofollow noopener" target="_blank" href="https://www.tensorflow.org/guide/data"><code>tf.data</code></a>, it is now very easy to build custom data pipelines. No need to use <code>feed_dict</code>. <code>tf.data</code> has support for many types of input formats, i.e., text, images, video, time-series, and much more.
It provides very clean and efficient input pipelines. For example, say we want to import a text file with some words that'll be preprocessed and used in a model. Let's do some classic preprocessing done for most NLP problems.
Let's first read the file, turn all words into lower case and split them into a list:
<pre><code class="hljs">import numpy as np

text_file = &quot;file.txt&quot;

text = open(text_file,&#x27;r&#x27;).read()

text = text.lower()
text = text.split()
</code></pre>
Then, we'll want to drop all duplicate words. This is easily done by packing them in a <code>Set</code>, converting that into a <code>List</code> and sorting it:
<pre><code class="hljs">words = sorted(list(set(text)))
</code></pre>
Now that we have sorted unique words, we'll make a vocabulary out of them. Each word will have a unique digit identifier assigned to it:
<pre><code class="hljs">vocab_to_int = {word:index for index, word in enumerate(words)}
int_to_vocab = np.array(words)
</code></pre>
Now, to convert our array of integers representing words into a Tensorflow Dataset, we'll use the <code>from_tensor_slices()</code> function provided by <code>tf.data.Dataset</code>:
<pre><code class="hljs">words_dataset = tf.data.Dataset.from_tensor_slices(words_as_int)
</code></pre>
Now, we can perform operations on this dataset, such as batching it into smaller sequences:
<pre><code class="hljs">seq_len = 50
sequences = words_dataset.batch(seq_len+1,drop_remainder=True)
</code></pre>
Now, when training, we can easily get batches from the Dataset object:
<pre><code class="hljs">for (batch_n,inp) in enumerate(dataset):
</code></pre>
Alternatively, you can directly load already existing datasets into <code>Dataset</code> objects:
<pre><code class="hljs">import tensorflow_datasets as tfds

mnist_data = tfds.load(&quot;mnist&quot;)
mnist_train, mnist_test = mnist_data[&quot;train&quot;], mnist_data[&quot;test&quot;]
</code></pre>
<h3 id="7tfkerasmodel">7. tf.keras.Model</h3>
A loved novelty is defining your own custom models by sub-classing the <code>keras.Model</code> class.
Taking a hint from PyTorch, which allows developers to create models using custom classes (customizing the classes that form a <code>Layer</code>, and thus altering the structure of the model) - Tensorflow 2.0, through Keras, allows us to define custom models as well.
Let's create a <code>Sequential</code> model, like you might using Tensorflow 1:
<pre><code class="hljs"># Creating a Model
model = tf.keras.Sequential([
tf.keras.layers.Dense(512,activation=&#x27;relu&#x27;,input_shape=(784,)),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(512,activation=&#x27;relu&#x27;),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10,activation=&#x27;softmax&#x27;)
])
</code></pre>
Now, instead of using the <code>Sequential</code> model, let's create our own model by sub-classing the <code>keras.Model</code> class:
<pre><code class="hljs"># Creating a Model
class mnist_model(tf.keras.Model):
 def __init__(self):
 super(mnist_model,self).__init__()
 self.dense1 = tf.keras.layers.Dense(512)
 self.drop1 = tf.keras.layers.Dropout(0.2)
 self.dense2 = tf.keras.layers.Dense(512)
 self.drop2 = tf.keras.layers.Dropout(0.2)
 self.dense3 = tf.keras.layers.Dense(10)

 def call(self,x):
 x = tf.nn.relu(self.dense1(x))
 x = self.drop1(x)
 x = tf.nn.relu(self.dense2(x))
 x = self.drop2(x)
 x = tf.nn.softmax(self.dense3(x))
 return x
</code></pre>
We've effectively created the same model here, though this approach allows us to fully customize and create models per our needs.
<h3 id="8tfgradienttape">8. tf.GradientTape</h3>
<code>tf.GradientTape</code> allows you to automatically calculate gradients. This is useful when using custom training loops.
You can train your model using custom training loops rather than calling <code>model.fit</code>. It gives you more control over the training process if you'd like to tweak it.
Pairing custom training loops made available by <code>tf.GradientTape</code> with custom models made available by <code>keras.Model</code> gives you control over models and training you never had before.
These quickly became very loved features in the community. Here's how you can create a custom model with decorated functions and a custom training loop:
<pre><code class="hljs">&quot;&quot;&quot;Note: We&#x27;ll be using the model created in the previous section.&quot;&quot;&quot;
# Creating the model
model = mnist_model()
# Defining the optimizer and the loss
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=False)

@tf.function
def step(model,x,y):
 &quot;&quot;&quot;
 model: in this case the mnist_model
 x: input data in batches
 y: True labels &quot;&quot;&quot;
 # Use GradientTape to monitor trainable variables
 with tf.GradientTape() as tape:
 # Computing predictions
 predictions = model(x)
 # Calculating Loss
 loss = loss_object(y,predictions)
 # Extracting all the trainable variables
 trainable_variables = model.trainable_variables()

 # Computing derivative of loss w.r.t variables/weights
 gradients = tape.gradient(loss,trainable_variables)
 # Updating the weights
 optimizer.apply_gradients(zip(gradients,trainable_variables))

 return loss
</code></pre>
Now you can just call the <code>step()</code> function by passing the model and training data in batches using a loop.
<h2 id="conclusion">Conclusion</h2>
With the arrival of Tensorflow 2.0, many setbacks have been reworked. From widening the variety of system support and new services to custom models and training loops - Tensorflow 2.0 has also introduced a new learning experience for veteran practitioners.

Guest Contributor

In this article, we'll be diving into the major changes made in a leap from Tensorflow 1.x to 2.0 - from widening the variety of system support and new services to custom models and training loops.

What's New in Tensorflow 2.0?

<h2 id="introduction">Introduction</h2>
As Python developers, most of us are familiar with Virtual Environments. One of the first things we do when working on a new project is to create an environment. We commonly use <a rel="nofollow noopener" target="_blank" href="https://virtualenv.pypa.io/en/latest/">virtualenv</a> or <a rel="nofollow noopener" target="_blank" href="https://docs.python.org/3/library/venv.html">venv</a> exactly for that purpose.
<blockquote>
Each project we work on uses different packages and may even be compatible with only one Python version.
</blockquote>
Doing something repeatedly warrants automation. In this article, we'll see how <code>direnv</code> and <code>pyenv</code> can help us do that.
As a side note, some modern IDEs already automated these steps. For example, PyCharm will create the Virtual environment when initializing a project:
<img src="https://stackabuse.s3.amazonaws.com/media/managing-python-environments-with-pyenv-and-direnv-1.png" alt="PyCharm's automated python environment management">
Although automating all these steps is a great win if we use IDEs that support such functionalities, a more generic solution should be IDE-agnostic.
<h2 id="theproblemsofvirtualenv">The Problems of <code>virtualenv</code></h2>
Imagine we found a project on GitHub and we would like to play around with it. <a rel="nofollow noopener" target="_blank" href="https://github.com/lcofre/pyweather">Pyweather</a> is a simple script that requests the extended weather forecast for our location and prints it on the terminal.
These are the steps we take in order to try the script on our machine:
<pre><code class="hljs">$ git clone https://github.com/lcofre/pyweather.git
$ cd pyweather
</code></pre>
Then we create the virtual environment and install the packages the script uses:
<pre><code class="hljs">$ virtualenv --python=python3 env
$ source env/bin/activate
(env) $ pip install requirements.txt
</code></pre>
And only then, we can execute the script:
<pre><code class="hljs">(env) $ ./pyweather.py	
</code></pre>
We created a virtual environment and saved it in the root folder of our project. While being in that folder we had to activate the environment with the <code>source</code> command.
When we finish working, we need to leave the Virtual Environment by executing <code>deactivate</code>:
<pre><code class="hljs">(env) $ deactivate
</code></pre>
All those steps are our responsibility. How many times we may have forgotten to activate an environment and installed a package globally!
Let's see how <code>direnv</code> helps us automate this.
<h2 id="direnv">direnv</h2>
<a rel="nofollow noopener" target="_blank" href="https://github.com/direnv/direnv#how-it-works"><code>direnv</code></a> was made mainly to load environment variables, depending on the current directory and has an extension for many shells.
In this example, we'll be using <code>bash</code>, but <code>direnv</code> supports many other shells as well. And what's more important for us, it allows us to manage Python Virtual Environments.
To install it we'll run the <code>bash</code> installer they provide. We could use the package manager of our distribution, but the <code>bash</code> installer will ensure we install the latest version available:
<pre><code class="hljs">$ curl -sfL https://direnv.net/install.sh | bash
</code></pre>
Now we need to hook <code>direnv</code> to <code>bash</code>. We'll edit <code>~/.bashrc</code> and then reload it:
<pre><code class="hljs">$ echo &#x27;eval &quot;$(direnv hook bash)&quot;&#x27; &gt;&gt; ~/.bashrc
$ source ~/.bashrc
</code></pre>
This way <code>direnv</code> will link itself to the shell and will be executed always before each prompt. We will never notice it's working in the background.
<code>direnv</code> will check if something needs to be loaded on the current folder. It checks the existence of a file named <code>.envrc</code>, with instructions on what should be loaded.
To load Python Virtual Environments we run the <code>layout</code> command, followed by the Python version:
<pre><code class="hljs">$ echo &#x27;layout python&#x27; &gt; .envrc
</code></pre>
Or if we want to use Python 3:
<pre><code class="hljs">$ echo &#x27;layout python3&#x27; &gt; .envrc
</code></pre>
Running these will tell <code>direnv</code> to look for a <code>python</code> or <code>python3</code> executable on the path.
As soon as we create the <code>.envrc</code> file we'll be warned that we need to allow <code>direnv</code> to access that folder. Let's do that right now:
<pre><code class="hljs">$ direnv allow
</code></pre>
<pre><code class="hljs">direnv: loading .envrc
...
New python executable in /home/myuser/untitled/.direnv/python-3.6.9/bin/python3
...
Installing setuptools, pkg_resources, pip, wheel...direnv:
done.
direnv: export +VIRTUAL_ENV ~PATH
</code></pre>
As we can see in the output, the Virtual Environment was immediately created. The prompt is not modified though, so we won't see the name of the environment written at the beginning.
Now we can install the packages we need as we did on the environment we created in the previous section:
<pre><code class="hljs">$ pip install -r requirements.txt
</code></pre>
<code>direnv</code> will silently activate the environment in the background. Whenever we move out of the directory, the environment will be deactivated:
<pre><code class="hljs">$ cd ..
</code></pre>
<pre><code class="hljs">direnv: unloading
</code></pre>
If we can use any Python version that is installed on the system, <code>direnv</code> is all we need.
Let's suppose now that our <code>pyweather</code> script requires a very specific version though.
<h2 id="pyenv">pyenv</h2>
<a rel="nofollow noopener" target="_blank" href="https://github.com/pyenv/pyenv#how-it-works"><code>pyenv</code></a> is a version management utility for Python. It allows, among other things, to change Python versions on a per-project basis. <code>direnv</code> provides support for it since version <code>2.21.0</code>, so together they can give us a higher level of control on the version we use in our environment.
Let's start by installing <code>pyenv</code>:
<pre><code class="hljs">$ curl -L https://pyenv.run | bash
</code></pre>
And then ensuring it will always be accessible to our terminal:
<pre><code class="hljs">$ echo &#x27;export PATH=&quot;~/.pyenv/bin:$PATH&quot;&#x27; &gt;&gt; ~/.bashrc
$ echo &#x27;eval &quot;$(pyenv init -)&quot;&#x27; &gt;&gt; ~/.bashrc
$ echo &#x27;eval &quot;$(pyenv virtualenv-init -)&quot;&#x27; &gt;&gt; ~/.bashrc
$ source ~/.bashrc
</code></pre>
Now let's suppose our <code>pyweather</code> script requires a very specific Python version, <code>3.6.2</code>.
First, we need to install that version of Python:
<pre><code class="hljs">$ pyenv install 3.6.2
</code></pre>
And now we can configure our project to use the specific version:
<pre><code class="hljs">$ echo &#x27;layout pyenv 3.6.2&#x27; &gt; .envrc
$ direnv allow
</code></pre>
We can confirm all works as expected by checking the Python version in the environment:
<pre><code class="hljs">$ python --version
</code></pre>
<pre><code class="hljs">Python 3.6.2
</code></pre>
If we ever need to change the Python version, it will be enough for us to change the layout in the <code>.envrc</code> file.
Thanks to both utilities we can change the layout to any Python version, and our virtual environment will be updated right away.
Another advantage of using both <code>direnv</code> and <code>pyenv</code> is that we can version our <code>.envrc</code> file in our project repository.
That way all contributors will be able to configure their environment as intended by the project, as long as they install the utilities and Python version needed.
<h2 id="conclusion">Conclusion</h2>
Virtual Environments are in a way detached from the Python development workflow. We need to remember to configure and activate it before working with our project. Thanks to <code>direnv</code> and <code>pyenv</code> we can automate all this, and entering the project folder will do all the work for us in the background.
Installation of both utilities is not straightforward, but after being done once we will save ourselves a lot of time. We will also always have the certainty that we are working with the right virtual environment and Python version.

Leandro Cofre

In this tutorial, we'll be going over direnv and pyenv tools to automate the management of Python Environments.