Usman Malik

Python for NLP: Multi-label Text Classification with Keras

Mina Krivokuća

Minimax with Alpha-Beta Pruning in Python

Olivera Popović

Unit Testing in Java with JUnit 5

Python for NLP: Creating Multi-Data-Type Classification Models with Keras

There are a number of different ways to format strings in Python, one of which is done using the <code>%</code> operator, which is known as the string formatting (or interpolation) operator. In this article we'll show you how to use this operator to construct strings with a template string and variables containing your data.
<h2 id="theoperator">The % Operator</h2>
This way of working with text has been shipped with Python since the beginning, and it's also known as C-style formatting, as it originates from the C programming language. Another description for it is simple positional formatting.
The <code>%</code> operator tells the Python interpreter to format a string using a given set of variables, enclosed in a tuple, following the operator. A very simple example of this is as follows:
<pre><code class="hljs">&#x27;%s is smaller than %s&#x27; % (&#x27;one&#x27;, &#x27;two&#x27;)
</code></pre>
The Python interpreter substitutes the first occurrence of <code>%s</code> in the string by the given string &quot;one&quot;, and the second <code>%s</code> by the string &quot;two&quot;. These <code>%s</code> strings are actually placeholders in our &quot;template&quot; string, and they indicate that strings will be placed there.
As a first example, below we demonstrate using the Python REPL how to print a string value and a float value:
<pre><code class="hljs">&gt;&gt;&gt; print(&quot;Mr. %s, the total is %.2f.&quot; % (&quot;Jekyll&quot;, 15.53))
&#x27;Mr. Jekyll, the total is 15.33.&#x27;
</code></pre>
Just like the <code>%s</code> is a placeholder for strings, <code>%f</code> is a placeholder for floating point numbers. The &quot;.2&quot; before the <code>f</code> is what indicates how many digits we want displayed after the decimal point.
These are just two simple examples of what is possible, and a lot more placeholder types are supported. Here is the full list of placeholder types in more detail:
<h3 id="c">%c</h3>
This placeholder represents a single character.
<pre><code class="hljs">&gt;&gt;&gt; print(&quot;The character after %c is %c.&quot; % (&quot;B&quot;, &quot;C&quot;))
The character after B is C.
</code></pre>
Providing more than a single character as the variable here will raise an exception.
<h3 id="s">%s</h3>
This placeholder uses string conversion via <code>str()</code> prior to formatting. So any value that can be converted to a string via <code>str()</code> can be used here.
<pre><code class="hljs">&gt;&gt;&gt; place = &quot;New York&quot;
&gt;&gt;&gt; print(&quot;Welcome to %s!&quot; % place)
Welcome to New York!
</code></pre>
Here we only have a single element to be used in our string formatting, and thus we're not required to enclose the element in a tuple like the previous examples.
<h3 id="iandd">%i and %d</h3>
These placeholders represent a signed decimal integer.
<pre><code class="hljs">&gt;&gt;&gt; year = 2019
&gt;&gt;&gt; print(&quot;%i will be a perfect year.&quot; % year)
2019 will be a perfect year.
</code></pre>
Since this placeholder expects a decimal, it will be converted to one if a floating point value is provided instead.
<h3 id="u">%u</h3>
This placeholder represents an unsigned decimal integer.
<h3 id="o">%o</h3>
This placeholder represents an octal integer.
<pre><code class="hljs">&gt;&gt;&gt; number = 15
&gt;&gt;&gt; print(&quot;%i in octal is %o&quot; % (number, number))
15 in octal is 17
</code></pre>
<h3 id="x">%x</h3>
Represents a hexadecimal integer using lowercase letters (a-f).
<pre><code class="hljs">&gt;&gt;&gt; number = 15
&gt;&gt;&gt; print(&quot;%i in hex is %02x&quot; % (number, number))
15 in hex is 0f
</code></pre>
By using the &quot;02&quot; prefix in our placeholder, we're telling Python to print a two-character hex string.
<h3 id="x">%X</h3>
Represents a hexadecimal integer using uppercase letters (A-F).
<pre><code class="hljs">&gt;&gt;&gt; number = 15
&gt;&gt;&gt; print(&quot;%i in hex is %04X&quot; % (number, number))
15 in hex is 000F
</code></pre>
And like the previous example, by using the &quot;04&quot; prefix in our placeholder, we're telling Python to print a four-character hex string.
<h3 id="e">%e</h3>
Represents an exponential notation with a lowercase &quot;e&quot;.
<h3 id="e">%E</h3>
Represents an exponential notation with an uppercase &quot;e&quot;.
<h3 id="f">%f</h3>
Represents a floating point real number.
<pre><code class="hljs">&gt;&gt;&gt; price = 15.95
&gt;&gt;&gt; print(&quot;the price is %.2f&quot; % price)
the price is 15.95
</code></pre>
<h3 id="g">%g</h3>
The shorter version of <code>%f</code> and <code>%e</code>.
<h3 id="g">%G</h3>
The shorter version of <code>%f</code> and <code>%E</code>.
The placeholders shown above allow you to format strings by specifying data types in your templates. However, these aren't the only features of the interpolation operator. In the next subsection we'll see how we can pad our strings with spaces using the <code>%</code> operator.
<h3 id="aligningtheoutput">Aligning the Output</h3>
Up until now we've only shown how to format text strings by specifying simple placeholders. With the help of an additional numerical value, you can define the total space that shall be reserved on either side of a variable in the output string.
As an example the value of <code>%10s</code> reserves 10 characters, with the extra spacing on the left side of the placeholder, and a value of <code>%-10s</code> puts any extra space to the right of the placeholder. The single padding character is a space, and cannot be changed.
<pre><code class="hljs">&gt;&gt;&gt; place = &quot;London&quot;
&gt;&gt;&gt; print (&quot;%10s is not a place in France&quot; % place) # Pad to the left
 London is not a place in France
&gt;&gt;&gt; print (&quot;%-10s is not a place in France&quot; % place) # Pad to the right
London is not a place in France
</code></pre>
Dealing with numbers works in the same way:
<pre><code class="hljs">&gt;&gt;&gt; print (&quot;The postcode is %10d.&quot; % 25000) # Padding on the left side
The postcode is 25000.
&gt;&gt;&gt; print (&quot;The postcode is %-10d.&quot; % 25000) # Padding on the right side
The postcode is 25000 .
</code></pre>
Truncating strings and rounding numbers is the counterpart to padding. Have a look at <a target="_blank" href="/rounding-numbers-in-python/">Rounding Numbers in Python</a> in order to learn more about the traps that are hiding here.
<h2 id="conclusion">Conclusion</h2>
In this article we saw how the interpolation (aka formatting) operator is a powerful way to format strings, which allows you to specify data type, floating point precision, and even spacing/padding.

Frank Hofmann

Python String Interpolation with the Percent (%) Operator

<h2 id="introduction">Introduction</h2>
When getting dressed, as one does, you most likely haven't had this line of thought:
<blockquote>
Oh, it might have been a good idea to put on my underpants before getting into my trousers.
</blockquote>
That's because we're used to sorting our actions topologically. Or in simpler terms, we're used to logically deducing which actions have to come before or after other actions, or rather which actions are prerequisites for other actions.
For example, let's say that you want to build a house, the steps would look like this:
<ol>
<li>Lay down the foundation</li>
<li>Build walls with installations</li>
<li>Put in insulation</li>
<li>Put in decorations/facade</li>
</ol>
In that exact order - it's undisputed. You can't build walls if you don't have a foundation, and you can't put in insulation if there aren't any walls.
In this guide we will be covering Topological Sorting in Java.
<h2 id="introductiontographs">Introduction to Graphs</h2>
Since Topological Sorting is applied to Directed Acyclic Graphs (DAG), we first have to talk a bit about Graphs.
A Graph is simply a data structure that represents a set of objects that have certain relations between each other - the objects being represented by nodes (circles) and the individual relations by edges (the lines).
There are many different kinds of graphs, but for the problem at hand we need to learn what is a Directed Acyclic Graph. Let's dissect the big bad math term into smaller, more comprehensible segments.
<h3 id="directed">Directed</h3>
A graph is directed if every relation between 2 objects doesn't have to be bidirectional (it has to have a direction), unlike a unidirectional graph where every relation has to go both ways.
In the graph below, the relation <code>C-A</code> is unidirectional, which means <code>C</code> has a relation with <code>A</code>, and <code>A</code> has a relation with <code>C</code>.
<img src="https://stackabuse.s3.amazonaws.com/media/topological-sorting-in-java-1.png" alt="">
On the other hand, in the following graph, the relation <code>C-A</code> is directed, which means <code>A</code> has a relation with <code>C</code>, but <code>C</code> doesn't have a relation with <code>A</code>.
<img src="https://stackabuse.s3.amazonaws.com/media/topological-sorting-in-java-2.png" alt="">
Because of this difference, we have to strictly define what are the node's neighbors:
Undirected graph:
<blockquote>
​Two nodes (A and B) are neighboring nodes if an undirected path exists between them.
</blockquote>
Directed graph:
<blockquote>
​<code>A</code> is <code>B</code>'s neighbor if a direct, directed edge exists which leads from <code>B</code> to <code>A</code>. The first direct in this definition refers to the fact that the length of the path leading from <code>B</code> to <code>A</code> has to be strictly 1.
</blockquote>
<h3 id="acyclic">Acyclic</h3>
A given graph is acyclic only if a cycle does not exist. A cycle is a path for any node <code>X</code>, which starts at <code>X</code> and leads back to <code>X</code>. The following graph is not acyclic because it contains a cycle (<code>X-B-C-X</code>).
<img src="https://stackabuse.s3.amazonaws.com/media/topological-sorting-in-java-5.png" alt="">
<h2 id="basictopologicalsortingconcept">Basic Topological Sorting Concept</h2>
So how does topological sorting look when used on a graph, and why does the graph have to be acyclic for it to work?
To answer these questions, let's strictly define what it means to topologically sort a graph:
<blockquote>
A graph is topologically sortable if a sequence <code>a1</code>, <code>a2</code>, <code>a3</code>... exists (<code>ai</code> being graph nodes), where for every edge <code>ai</code>-&gt;<code>aj</code>, <code>ai</code> comes before <code>aj</code> in the sequence.
</blockquote>
If we say that actions are represented by nodes. The above definition would basically mean that an undisputable order execution must exist.
To better understand the logic behind topological sorting and why it can't work on a graph that contains a cycle, let's pretend we're a computer that's trying to topologically sort the following graph:
<img src="https://stackabuse.s3.amazonaws.com/media/topological-sorting-in-java-4.png" alt="">
<pre><code class="hljs"># Let's say that we start our search at node X
# Current node: X
step 1: Ok, i'm starting from node X so it must be at the beginning of the sequence.
 sequence: [X]

# The only available edge from X is X-&gt;B, so we 'travel' to B
# Current node: B
step 2: Right, B comes after X in the sequence for sure.
 sequence: [X,B]

# We travel to C using the edge B-&gt;C
# Current node: C
step 3: Same thing as the last step, we add C.
 sequence: [X,B,C]

# Current node: X
step 4: WHAT IN THE TARNATION, X AGAIN?
 sequence: [X,B,C,X]
</code></pre>
This is the reason why we can't topologically sort a graph that contains a cycle, because the following two statements are both true:
<ul>
<li>X comes before B</li>
<li>B comes before X</li>
</ul>
And because of that, we can't determine the absolute order of the given actions.
Now because we're familiar with the algorithm's concepts, let's take a look at the implementation in Java.
<h2 id="implementation">Implementation</h2>
First off, let's construct classes for defining nodes and graphs, and then using said classes, define the following graph:
<img src="https://stackabuse.s3.amazonaws.com/media/topological-sorting-in-java-3.png" alt="">
<pre><code class="hljs">public class Graph {
 private List&lt;Node&gt; nodes;

 public Graph() {
 this.nodes = new ArrayList&lt;&gt;();
 }

 public Graph(List&lt;Node&gt; nodes) {
 this.nodes = nodes;
 }

 public void addNode(Node e) {
 this.nodes.add(e);
 }

 public List&lt;Node&gt; getNodes() {
 return nodes;
 }

 public Node getNode(int searchId) {
 for (Node node:this.getNodes()) {
 if (node.getId() == searchId) {
 return node;
 }
 }
 return null;
 }

 public int getSize() {
 return this.nodes.size();
 }

 @Override
 public String toString() {
 return &quot;Graph{&quot; +
 &quot;nodes=&quot; + nodes +
 &quot;}&quot;;
 }
}
</code></pre>
The graph is pretty simple, we can instantiate it empty or with a set of nodes, add nodes, retrieve them and print them out.
Now, let's move on to the <code>Node</code> class:
<pre><code class="hljs">public class Node {
 private int id;
 private List&lt;Integer&gt; neighbors;

 public Node(int id) {
 this.id = id;
 this.neighbors = new ArrayList&lt;&gt;();
 }

 public void addNeighbor(int e) {
 this.neighbors.add(e);
 }

 public int getId() {
 return id;
 }

 public List&lt;Integer&gt; getNeighbors() {
 return neighbors;
 }

 @Override
 public String toString() {
 return &quot;Node{&quot; +
 &quot;id=&quot; + id +
 &quot;, neighbors=&quot; + neighbors +
 &quot;}&quot;+ &quot;\n&quot;;
 }
}
</code></pre>
This class is quite simple as well - just a constructor and a list of neighboring nodes.
With both of our classes, let's instantiate a graph and populate it with a few nodes:
<pre><code class="hljs">public class GraphInit {
 public static void main(String[] args) {
 Graph g = new Graph();
 Node node1 = new Node(1);
 Node node2 = new Node(2);
 Node node3 = new Node(3);
 Node node4 = new Node(4);
 node1.addNeighbor(2);
 node2.addNeighbor(3);
 node4.addNeighbor(3);
 g.addNode(node1);
 g.addNode(node2);
 g.addNode(node3);
 g.addNode(node4);
 System.out.println(g);
 }
}
</code></pre>
Output:
<pre><code class="hljs">Graph{nodes=[Node{id=1, neighbors=[2]}
, Node{id=2, neighbors=[3]}
, Node{id=3, neighbors=[]}
, Node{id=4, neighbors=[3]}
]}
</code></pre>
Now let's implement the algorithm itself:
<pre><code class="hljs">private static void topoSort(Graph g) {

 // Fetching the number of nodes in the graph
 int V = g.getSize();

 // List where we&#x27;ll be storing the topological order
 List&lt;Integer&gt; order = new ArrayList&lt;&gt; ();

 // Map which indicates if a node is visited (has been processed by the algorithm)
 Map&lt;Integer, Boolean&gt; visited = new HashMap&lt;&gt;();
 for (Node tmp: g.getNodes())
 visited.put(tmp.getId(), false);

 // We go through the nodes using black magic
 for (Node tmp: g.getNodes()) {
 if (!visited.get(tmp.getId()))
 blackMagic(g, tmp.getId(), visited, order);
 }

 // We reverse the order we constructed to get the
 // proper toposorting
 Collections.reverse(order);
 System.out.println(order);
}

private static void blackMagic(Graph g, int v, Map&lt;Integer, Boolean&gt; visited, List&lt;Integer&gt; order) {
 // Mark the current node as visited
 visited.replace(v, true);
 Integer i;

 // We reuse the algorithm on all adjacent nodes to the current node
 for (Integer neighborId: g.getNode(v).getNeighbors()) {
 if (!visited.get(neighborId))
 blackMagic(g, neighborId, visited, order);
 }

 // Put the current node in the array
 order.add(v);
}
</code></pre>
If we call <code>topoSort(g)</code> for the graph initialized above, we get the following output:
<pre><code class="hljs">[4, 1, 2, 3]
</code></pre>
Which is exactly right.
<h2 id="problemmodelingusingtopologicalsorting">Problem Modeling Using Topological Sorting</h2>
In a real-world scenario, topological sorting can be utilized to write proper assembly instructions for Lego toys, cars, and buildings.
There's actually a type of topological sorting which is used daily (or hourly) by most developers, albeit implicitly. If you're thinking Makefile or just Program dependencies, you'd be absolutely correct.
A typical Makefile looks like this:
<pre><code class="hljs">area_51_invasion.out: me.c, the_boys.c, Chads.c, Karen.c, the_manager.c
 #instructions for assembly when one of the files in the dependency list is modified
</code></pre>
With this line, we define which files depend on other files, or rather, we are defining in which topological order should the files be inspected to see if a rebuild is needed.
That is, if <code>area_51_invasion.out</code> depends on <code>the_boys.c</code> and <code>the_boys.c</code> is for some reason modified, we need to rebuild <code>area_51_invasion.out</code> and everything that depends on that same file, that is everything that comes before it in the Makefile's topological order.
<h2 id="conclusion">Conclusion</h2>
Considering <code>toposort()</code> is basically something we do on a regular basis. You may have even implemented it in your software and didn't even know it. And if you haven't, I strongly suggest you give it a whirl!

Vladimir Batoćanin

Topological Sorting in Java

<h2 id="introduction">Introduction</h2>
In this article I cover the basics of creating, interacting with, inspecting, and extracting zip archive files using Java (OpenJDK 11 to be specific). The code sample used in this article is in the form of a Gradle project and hosted in <a rel="nofollow" target="_blank" href="https://github.com/amcquistan/zip-files-in-java">this GitHub repo</a> for you to run and experiment with. Please exercise caution when changing code that deletes files.
As mentioned already, the code examples here are written in using Java 11 and utilizes the <code>var</code> keyword which was introduced in Java 10 and functional programming paradigms in Java 8, so a minimum version of Java 10 is required to run them as is.
<h3 id="contents">Contents</h3>
<ul>
<li><a href="#keyjavaclassesforworkingwithziparchives">Key Java Classes for Working with Zip Archives</a></li>
<li><a href="#commonfilepathsforthecodeexamples">Common File Paths for the Code Examples</a></li>
<li><a href="#inspectingthecontentsofaziparchive">Inspecting the Contents of a Zip Archive</a></li>
<li><a href="#extractingaziparchive">Extracting a Zip Archive</a></li>
<li><a href="#writingfilesdirectlyintoanewziparchive">Writing Files Directly into a New Zip Archive</a></li>
<li><a href="#zippinganexistingFileintoanewziparchive">Zipping an Existing File into a New Zip Archive</a></li>
<li><a href="#zippingafolderintoanewziparchive">Zipping a Folder into a New Zip Archive</a></li>
</ul>
<h2 id="keyjavaclassesforworkingwithziparchives">Key Java Classes for Working with Zip Archives</h2>
I feel it's a good idea to start things off by identifying some of the prominent classes that are commonly used when dealing with zip archives in Java. These classes live in either the <code>java.util.zip</code> or <code>java.nio.file</code> packages.
<ul>
<li><a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/index.html?java/util/zip/ZipFile.html">java.util.zip.ZipFile</a> is used to read in and interact with items (<code>ZipEntry</code> instances) in a zip archive</li>
<li><a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/ZipEntry.html">java.util.zip.ZipEntry</a> is an abstraction representing a item such as a file or directory in a zip archive (ie, <code>ZipFile</code> instance)</li>
<li><a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/index.html?java/util/zip/ZipOutputStream.html">java.util.zip.ZipOutputStream</a> is an implementation of the abstract <a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/java/io/OutputStream.html">OutputStream</a> class and used to write items to a Zip file</li>
<li><a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html">java.nio.file.Files</a> is very handy utilities class for streaming and copying file data into ZipOutputStream instances or out of ZipFile instances</li>
<li><a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/java/nio/file/Paths.html">java.nio.file.Path</a> another handy utilities class for effectively working with file paths</li>
</ul>
<h2 id="commonfilepathsforthecodeexamples">Common File Paths for the Code Examples</h2>
For the example code I use two common directories to write and read data to/from which are both relative to the root of the Gradle project. Take a look at the linked Repo in the introduction, or better yet, run the samples. Just keep these two Path variables in mind as they are used often as the starting directory for inputs and outputs.
<pre><code class="hljs">public class App {

 static final Path zippedDir = Path.of(&quot;ZippedData&quot;);
 static final Path inputDataDir = Path.of(&quot;InputData&quot;);
 
 // ... other stuff 
}
</code></pre>
<h2 id="inspectingthecontentsofaziparchive">Inspecting the Contents of a Zip Archive</h2>
You can instantiate a <code>ZipFile</code> class and pass it the path to an existing zip archive, which essentially opens it like any other file, then inspect the contents by querying the <code>ZipEntry</code> enumeration contained inside it. Note that <code>ZipFile</code> implements the <a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/8/docs/api/java/lang/AutoCloseable.html">AutoCloseable</a> interface, making it a great candidate for the <a target="_blank" href="/the-try-with-resources-statement-in-java/">try-with-resources</a> Java programming construct shown below and throughout the examples here.
<pre><code class="hljs">static void showZipContents() {
 try (var zf = new ZipFile(&quot;ZipToInspect.zip&quot;)) {
 
 System.out.println(String.format(&quot;Inspecting contents of: %s\n&quot;, zf.getName()));
 
 Enumeration&lt;? extends ZipEntry&gt; zipEntries = zf.entries();
 zipEntries.asIterator().forEachRemaining(entry -&gt; {
 System.out.println(String.format(
 &quot;Item: %s \nType: %s \nSize: %d\n&quot;,
 entry.getName(),
 entry.isDirectory() ? &quot;directory&quot; : &quot;file&quot;,
 entry.getSize()
 ));
 });
 } catch (IOException e) {
 e.printStackTrace();
 }
}
</code></pre>
Running the Gradle project using the following:
<pre><code class="hljs">$ ./gradlew run
</code></pre>
This yields output for the <code>App.showZipContents</code> method of:
<pre><code class="hljs">&gt; Task :run
Inspecting contents of: ZipToInspect.zip

Item: ZipToInspect/ 
Type: directory 
Size: 0

Item: ZipToInspect/greetings.txt 
Type: file 
Size: 160

Item: ZipToInspect/InnerFolder/ 
Type: directory 
Size: 0

Item: ZipToInspect/InnerFolder/About.txt 
Type: file 
Size: 39
</code></pre>
Here you can see that this prints out all files and directories in the zip archive, even the files within directories.
<h2 id="extractingaziparchive">Extracting a Zip Archive</h2>
Extracting the contents of a zip archive onto disk requires nothing more than replicating the same directory structure as what is inside the <code>ZipFile</code>, which can be determined via <code>ZipEntry.isDirectory</code> and then copying the files represented in the <code>ZipEntry</code> instances onto disk.
<pre><code class="hljs">static void unzipAZip() {
 var outputPath = Path.of(&quot;UnzippedContents&quot;);

 try (var zf = new ZipFile(&quot;ZipToInspect.zip&quot;)) {
 
 // Delete if exists, then create a fresh empty directory to put the zip archive contents
 initialize(outputPath);

 Enumeration&lt;? extends ZipEntry&gt; zipEntries = zf.entries();
 zipEntries.asIterator().forEachRemaining(entry -&gt; {
 try {
 if (entry.isDirectory()) {
 var dirToCreate = outputPath.resolve(entry.getName());
 Files.createDirectories(dirToCreate);
 } else {
 var fileToCreate = outputPath.resolve(entry.getName());
 Files.copy(zf.getInputStream(entry), fileToCreate);
 }
 } catch(IOException ei) {
 ei.printStackTrace();
 }
 });
 } catch(IOException e) {
 e.printStackTrace();
 }
}
</code></pre>
<h2 id="writingfilesdirectlyintoanewziparchive">Writing Files Directly into a New Zip Archive</h2>
Since writing a zip archive is really nothing more than writing a stream of data to some destination (a Zip file in this case) then writing data, like String data, to a zip archive is only different in that you need to match the data being written to <code>ZipEntry</code> instances added to the <code>ZipOutputStream</code>.
Again, <code>ZipOutputStream</code> implements the <code>AutoCloseable</code> interface, so it is best to use with a try-with-resources statement. The only real catch is to remember to close your <code>ZipEntry</code> when you are done with each one to make it clear when it should no longer receive data.
<pre><code class="hljs">static void zipSomeStrings() {
 Map&lt;String, String&gt; stringsToZip = Map.ofEntries(
 entry(&quot;file1&quot;, &quot;This is the first file&quot;),
 entry(&quot;file2&quot;, &quot;This is the second file&quot;),
 entry(&quot;file3&quot;, &quot;This is the third file&quot;)
 );
 var zipPath = zippedDir.resolve(&quot;ZipOfStringData.zip&quot;);
 try (var zos = new ZipOutputStream(
 new BufferedOutputStream(Files.newOutputStream(zipPath)))) {
 for (var entry : stringsToZip.entrySet()) {
 zos.putNextEntry(new ZipEntry(entry.getKey()));
 zos.write(entry.getValue().getBytes());
 zos.closeEntry();
 }
 } catch (IOException e) {
 e.printStackTrace();
 }
}
</code></pre>
<h2 id="zippinganexistingfileintoanewziparchive">Zipping an Existing File into a New Zip Archive</h2>
If you've <a target="_blank" href="/how-to-copy-a-file-in-java/">copied a File in Java</a> before then you are essentially already a PRO at creating a zip archive from an existing file (or directory for that matter). Again, the only real difference is that you need to take a little extra caution to be sure you are matching files up to the appropriate <code>ZipEntry</code> instances.
In this example I create an input file &quot;FileToZip.txt&quot; and write some data to it &quot;Howdy There Java Friends!&quot; and then use the <a rel="nofollow" target="_blank" href="https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#copy(java.nio.file.Path,%20java.io.OutputStream)">Files.copy(Path, OutputStream)</a> to associate the <code>ZipEntry</code> with the FileToZip.txt file inside the ZippedFile.zip zip archive I'm creating with a <code>ZipOutoutStream</code> instance.
<pre><code class="hljs">static void zipAFile() {
 var inputPath = inputDataDir.resolve(&quot;FileToZip.txt&quot;);
 var zipPath = zippedDir.resolve(&quot;ZippedFile.zip&quot;);
 
 try (var zos = new ZipOutputStream(
 new BufferedOutputStream(Files.newOutputStream(zipPath)))) {
 
 Files.writeString(inputPath, &quot;Howdy There Java Friends!\n&quot;);

 zos.putNextEntry(new ZipEntry(inputPath.toString()));
 Files.copy(inputPath, zos);
 zos.closeEntry();
 } catch (IOException e) {
 e.printStackTrace();
 }
}
</code></pre>
<h2 id="zippingafolderintoanewziparchive">Zipping a Folder into a New Zip Archive</h2>
Zipping a non-empty directory becomes a little more involved, especially if you want to maintain empty directories within the parent directory. To maintain the presence of an empty directory within a zip archive you need to be sure to create an entry that is suffixed with the file system directory separator when creating it's <code>ZipEntry</code>, and then immediately close it.
In this example I create a directory named <code>foldertozip</code> containing the structure shown below, then zip it into a zip archive.
<pre><code class="hljs">tree .
.
└── foldertozip
 ├── emptydir
 ├── file1.txt
 └── file2.txt
</code></pre>
In the following code notice that I use the <code>Files.walk(Path)</code> method to traverse the directory tree of <code>foldertozip</code> and look for empty directories (<code>emptydir</code> in this example) and if / when found I concatenate the directory separator to the name within the <code>ZipEntry</code>. After this I close it as soon as I add it to the <code>ZipOutputStream</code> instance.
I also use a slightly different approach to injecting the non-directory files into the <code>ZipOutputStream</code> compared to the last example, but I am just using this different approach for the sake of variety in the examples.
<pre><code class="hljs">static void zipADirectoryWithFiles() {
 var foldertozip = inputDataDir.resolve(&quot;foldertozip&quot;); 
 var dirFile1 = foldertozip.resolve(&quot;file1.txt&quot;);
 var dirFile2 = foldertozip.resolve(&quot;file2.txt&quot;); 

 var zipPath = zippedDir.resolve(&quot;ZippedDirectory.zip&quot;);
 try (var zos = new ZipOutputStream(
 new BufferedOutputStream(Files.newOutputStream(zipPath)))) {
 
 Files.createDirectory(foldertozip);
 Files.createDirectory(foldertozip.resolve(&quot;emptydir&quot;));
 Files.writeString(dirFile1, &quot;Does this Java get you rev&#x27;d up or what?&quot;);
 Files.writeString(dirFile2, &quot;Java Java Java ... Buz Buz Buz!&quot;);

 Files.walk(foldertozip).forEach(path -&gt; {
 try {
 var reliativePath = inputDataDir.relativize(path);
 var file = path.toFile();
 if (file.isDirectory()) {
 var files = file.listFiles();
 if (files == null || files.length == 0) {
 zos.putNextEntry(new ZipEntry(
 reliativePath.toString() + File.separator));
 zos.closeEntry();
 }
 } else {
 zos.putNextEntry(new ZipEntry(reliativePath.toString()));
 zos.write(Files.readAllBytes(path));
 zos.closeEntry();
 }
 } catch(IOException e) {
 e.printStackTrace();
 }
 });
 } catch(IOException e) {
 e.printStackTrace();
 }
}
</code></pre>
<h2 id="conclusion">Conclusion</h2>
In this article I have discussed and demonstrated a modern approach to working with zip archives in Java using pure Java and no third party libraries. You may also notice I use a few more modern Java language features, such as functional programming paradigms and the <code>var</code> keyword for type inferred variables, so please make sure you are using at least Java 10 when running these examples.
As always, thanks for reading and don't be shy about commenting or critiquing below.

Adam McQuistan

Working with Zip Files in Java

Muhammad Junaid Khalid

Debugging Python Applications with the PDB Module

<h2 id="introduction">Introduction</h2>
Memory management is the process of efficiently allocating, de-allocating, and coordinating memory so that all the different processes run smoothly and can optimally access different system resources. Memory management also involves cleaning memory of objects that are no longer being accessed.
In Python, the memory manager is responsible for these kinds of tasks by periodically running to clean up, allocate, and manage the memory. Unlike C, Java, and other programming languages, Python manages objects by using reference counting. This means that the memory manager keeps track of the number of references to each object in the program. When an object's reference count drops to zero, which means the object is no longer being used, the garbage collector (part of the memory manager) automatically frees the memory from that particular object.
The user need not to worry about memory management as the process of allocation and deallocation of memory is fully automatic. The reclaimed memory can be used by other objects.
<h2 id="pythongarbagecollection">Python Garbage Collection</h2>
As explained earlier, Python deletes objects that are no longer referenced in the program to free up memory space. This process in which Python frees blocks of memory that are no longer used is called Garbage Collection. The Python <a rel="nofollow" target="_blank" href="https://docs.python.org/3/library/gc.html">Garbage Collector</a> (GC) runs during the program execution and is triggered if the reference count reduces to zero. The reference count increases if an object is assigned a new name or is placed in a container, like tuple or dictionary. Similarly, the reference count decreases when the reference to an object is reassigned, when the object's reference goes <a target="_blank" href="/local-and-global-variables-in-python/">out of scope</a>, or when an object is deleted.
The memory is a heap that contains objects and other data structures used in the program. The allocation and deallocation of this heap space is controlled by the Python Memory manager through the use of API functions.
<h2 id="pythonobjectsinmemory">Python Objects in Memory</h2>
Each variable in Python acts as an object. Objects can either be simple (containing numbers, strings, etc.) or containers (dictionaries, lists, or user defined classes). Furthermore, Python is a dynamically typed language which means that we do not need to declare the variables or their types before using them in a program.
For example:
<pre><code class="hljs">&gt;&gt;&gt; x = 5
&gt;&gt;&gt; print(x)
5
&gt;&gt;&gt; del x
&gt;&gt;&gt; print(x)
Traceback (most recent call last):
 File &quot;&lt;mem_manage&gt;&quot;, line 1, in &lt;module&gt;
 print(x)
NameError : name &#x27;x&#x27; is not defined
</code></pre>
If you look at the first 2 lines of the above program, object <code>x</code> is known. When we delete the object <code>x</code> and try to use it, we get an error stating that the variable <code>x</code> is not defined.
You can see that the garbage collection in Python is fully automated and the programmer does not need to worry about it, unlike languages like C.
<h2 id="modifyingthegarbagecollector">Modifying the Garbage Collector</h2>
The Python garbage collector has three generations in which objects are classified. A new object at the starting point of its life cycle is the first generation of the garbage collector. As the object survives garbage collection, it will be moved up to the next generations. Each of the 3 generations of the garbage collector has a threshold. Specifically, when the threshold of number of allocations minus the number of deallocations is exceeded, that generation will run garbage collection.
Earlier generations are also garbage collected more often than the higher generations. This is because newer objects are more likely to be discarded than old objects.
The <code>gc</code> module includes functions to change the threshold value, trigger a garbage collection process manually, disable the garbage collection process, etc. We can check the threshold values of different generations of the garbage collector using the <code>get_threshold()</code> method:
<pre><code class="hljs">import gc
print(gc.get_threshold())
</code></pre>
Sample Output:
<pre><code class="hljs">(700, 10, 10)
</code></pre>
As you see, here we have a threshold of 700 for the first generation, and 10 for each of the other two generations.
We can alter the threshold value for triggering the garbage collection process using the <code>set_threshold()</code> method of the <code>gc</code> module:
<pre><code class="hljs">gc.set_threshold(900, 15, 15)
</code></pre>
In the above example, we have increased the threshold value for all the 3 generations. Increasing the threshold value will decrease the frequency of running the garbage collector. Normally, we need not think too much about Python's garbage collection as a developer, but this may be useful when optimizing the Python runtime for your target system. One of the key benefits is that Python's garbage collection mechanism handles a lot of low-level details for the developer automatically.
<h2 id="whyperformmanualgarbagecollection">Why Perform Manual Garbage Collection?</h2>
We know that the Python interpreter keeps a track of references to objects used in a program. In earlier versions of Python (until version 1.6), the Python interpreter used only the reference counting mechanism to handle memory. When the reference count drops to zero, the Python interpreter automatically frees the memory. This classical reference counting mechanism is very effective, except that it fails to work when the program has reference cycles. A reference cycle happens if one or more objects are referenced to each other, and hence the reference count never reaches zero.
Let's consider an example.
<pre><code class="hljs">&gt;&gt;&gt; def create_cycle():
... list = [8, 9, 10]
... list.append(list)
... return list
... 
&gt;&gt;&gt; create_cycle()
[8, 9, 10, [...]]
</code></pre>
The above code creates a reference cycle, where the object <code>list</code> refers to itself. Hence, the memory for the object <code>list</code> will not be freed automatically when the function returns. The reference cycle problem can't be solved by reference counting. However, this reference cycle problem can be solved by changing the behavior of the garbage collector in your Python application.
To do so, we can use the <code>gc.collect()</code> function of the <code>gc</code> module.
<pre><code class="hljs">import gc
n = gc.collect()
print(&quot;Number of unreachable objects collected by GC:&quot;, n)
</code></pre>
The <code>gc.collect()</code> returns the number of objects it has collected and de-allocated.
There are two ways to perform manual garbage collection: time-based or event-based garbage collection.
Time-based garbage collection is pretty simple: the <code>gc.collect()</code> function is called after a fixed time interval.
Event-based garbage collection calls the <code>gc.collect()</code> function after an event occurs (i.e. when the application is exited or the application remains idle for a specific time period).
Let's understand the manual garbage collection work by creating a few reference cycles.
<pre><code class="hljs">import sys, gc

def create_cycle():
 list = [8, 9, 10]
 list.append(list)

def main():
 print(&quot;Creating garbage...&quot;)
 for i in range(8):
 create_cycle()

 print(&quot;Collecting...&quot;)
 n = gc.collect()
 print(&quot;Number of unreachable objects collected by GC:&quot;, n)
 print(&quot;Uncollectable garbage:&quot;, gc.garbage)

if __name__ == &quot;__main__&quot;:
 main()
 sys.exit()
</code></pre>
The output is as below:
<pre><code class="hljs">Creating garbage...
Collecting...
Number of unreachable objects collected by GC: 8
Uncollectable garbage: []
</code></pre>
The script above creates a list object that is referred to by a variable, creatively named <code>list</code>. The first element of the list object refers to itself. The reference count of the list object is always greater than zero even if it is deleted or out of scope in the program. Hence, the <code>list</code> object is not garbage collected due to the circular reference. The garbage collector mechanism in Python will automatically check for, and collect, circular references periodically.
In the above code, as the reference count is at least 1 and can never reach 0, we have forcefully garbage collected the objects by calling <code>gc.collect()</code>. However, remember not to force garbage collection frequently. The reason is that even after freeing the memory, the GC takes time to evaluate the object's eligibility to be garbage collected, taking up processor time and resources. Also, remember to manually manage the garbage collector only after your app has started completely.
<h2 id="conclusion">Conclusion</h2>
In this article, we discussed how memory management in Python is handled automatically by using reference counting and garbage collection strategies. Without garbage collection, implementing a successful memory management mechanism in Python is impossible. Also, programmers need not worry about deleting allocated memory, as it is taken care of by the Python memory manager. This leads to fewer memory leaks and better performance.

Guest Contributor

Basics of Memory Management in Python