Marcus Sanatan

Leodanis Pozo Ramos

In this tutorial, we'll learn how to compute the mean, median, and mode in Python from scratch and with the Python statistics module.

Calculating Mean, Median and Mode in Python

Calculating Mean, Median, and Mode in Python

Dhrubajyoti Bhattacharjee

In this tutorial, we'll be diving into everything you need to know about Thymeleaf in order to get started with this templating engine for Java and Spring.

Getting Started with Thymeleaf in Java and Spring

<h2 id="introductiontoanyandall">Introduction to any() and all()</h2>
In this tutorial, we'll be covering the <code>any()</code> and <code>all()</code> functions in Python.
The <code>any(iterable)</code> and <code>all(iterable)</code> are built-in functions in Python and have been around since Python 2.5 was released. Both functions are equivalent to writing a series of <code>or</code> and <code>and</code> operators respectively between each of the elements of the passed <code>iterable</code>. They are both convenience functions that shorten the code by replacing boilerplate loops.
Both methods short-circuit and return a value as soon as possible, so even with huge iterables, they're as efficient as they can be.
<h2 id="theandoroperators">The and/or Operators</h2>
Let's remind ourselves how the <code>and</code>/<code>or</code> operators work, as these functions are based on them.
<h3 id="theoroperator">The or Operator</h3>
The <code>or</code> operator evaluates to <code>True</code> if any of the conditions (operands) are <code>True</code>.
<pre><code class="hljs">print(&quot;(2 == 2) or (3 == 3) evaluates to: &quot; + str((2 == 2) or (3 == 3)))
print(&quot;(2 == 2) or (3 == 2) evaluates to: &quot; + str((2 == 2) or (3 == 2)))
print(&quot;(2 == 0) or (3 == 2) evaluates to: &quot; + str((2 == 0) or (3 == 2)))
</code></pre>
Output:
<pre><code class="hljs">(2 == 2) or (3 == 3) evaluates to: True
(2 == 2) or (3 == 2) evaluates to: True
(2 == 0) or (3 == 2) evaluates to: False
</code></pre>
We can chain multiple <code>or</code>s in a single statement, and it will again evaluate to <code>True</code> if any of the conditions are <code>True</code>:
<pre><code class="hljs">print(str(False or False or False or True or False))
</code></pre>
This results in:
<pre><code class="hljs">True
</code></pre>
<h3 id="theandoperator">The and Operator</h3>
The <code>and</code> operator evaluates to <code>True</code> only if all conditions are <code>True</code>:
<pre><code class="hljs">print(&quot;(2 == 2) and (3 == 3) evaluates to: &quot; + str((2 == 2) and (3 == 3)))
print(&quot;(2 == 2) and (3 == 2) evaluates to: &quot; + str((2 == 2) and (3 == 2)))
print(&quot;(2 == 0) and (3 == 2) evaluates to: &quot; + str((2 == 0) and (3 == 2)))
</code></pre>
Output:
<pre><code class="hljs">(2 == 2) and (3 == 3) evaluates to: True
(2 == 2) and (3 == 2) evaluates to: False
(2 == 0) and (3 == 2) evaluates to: False
</code></pre>
Similarly to <code>or</code>, we can chain multiple <code>and</code> operators, and they will evaluate to <code>True</code> only if all the operands evaluate to <code>True</code>:
<pre><code class="hljs">print(str(True and True and True and False and True))
</code></pre>
This results in:
<pre><code class="hljs">False
</code></pre>
<h2 id="any">any()</h2>
The method <code>any(iterable)</code> behaves like a series of <code>or</code> operators between each element of the <code>iterable</code> we passed. It's used to replace loops similar to this one:
<pre><code class="hljs">for element in some_iterable:
 if element:
 return True
return False
</code></pre>
We get the same result by simply calling <code>any(some_iterable)</code>:
<pre><code class="hljs">print(any([2 == 2, 3 == 2]))
print(any([True, False, False]))
print(any([False, False]))
</code></pre>
Running this piece of code would result in:
<pre><code class="hljs">True
True
False
</code></pre>
Note: Unexpected behavior may happen when using <code>any()</code> with dictionaries and <a href="#booleanconversionandanyall">data types other than boolean</a>. If <code>any()</code> is used with a dictionary, it checks whether any of the keys evaluate to <code>True</code>, not the values:
<pre><code class="hljs">dict = {True : False, False: False}

print(any(dict))
</code></pre>
This outputs:
<pre><code class="hljs">True
</code></pre>
Whereas, if <code>any()</code> checked the values, the output would have been <code>False</code>.
The method <code>any()</code> is often used in combination with the <code>map()</code> method and list comprehensions:
<pre><code class="hljs">old_list = [2, 1, 3, 8, 10, 11, 13]
list_if_even = list(map(lambda x: x % 2 == 0, old_list))
list_if_odd = [x % 2 != 0 for x in old_list]

print(list_if_even)
print(list_if_odd)

print(&quot;Are any of the elements even? &quot; + str(any(list_if_even)))
print(&quot;Are any of the elements odd? &quot; + str(any(list_if_odd)))
</code></pre>
This outputs:
<pre><code class="hljs">[True, False, False, True, True, False, False]
[False, True, True, False, False, True, True]
Are any of the elements even? True
Are any of the elements odd? True
</code></pre>
Note: If an empty <code>iterable</code> is passed to <code>any()</code>, the method returns <code>False</code>.
If you'd like to read more about <a target="_blank" href="/map-filter-and-reduce-in-python-with-examples/">the map(), filter() and reduce() functions</a>, we've got you covered!
<h2 id="all">all()</h2>
The <code>all(iterable)</code> method evaluates like a series of <code>and</code> operators between each of the elements in the <code>iterable</code> we passed. It is used to replace loops similar to this one:
<pre><code class="hljs">for element in iterable:
 if not element:
 return False
return True
</code></pre>
The method returns <code>True</code> only if every element in <code>iterable</code> evaluates to <code>True</code>, and <code>False</code> otherwise:
<pre><code class="hljs">print(all([2 == 2, 3 == 2]))
print(all([2 &gt; 1, 3 != 4]))
print(all([True, False, False]))
print(all([False, False]))
</code></pre>
This outputs:
<pre><code class="hljs">False
True
False
False
</code></pre>
Note: Just like with <code>any()</code>, unexpected behavior may happen when passing dictionaries and data types other than <code>boolean</code>. Again, if <code>all()</code> is used with a dictionary, it checks whether all of the keys evaluate to <code>True</code>, not the values.
Another similarity with <code>any()</code> is that <code>all()</code> is also commonly used in combination with the <code>map()</code> function and list comprehensions:
<pre><code class="hljs">old_list = [&quot;just&quot;, &quot;Some&quot;, &quot;text&quot;, &quot;As&quot;, &quot;An&quot;, &quot;example&quot;]
list_begins_upper = list(map(lambda x: x[0].isupper(), old_list))
list_shorter_than_8 = [len(x) &lt; 8 for x in old_list]

print(list_begins_upper)
print(list_shorter_than_8)

print(&quot;Do all the strings begin with an uppercase letter? &quot; + str(all(list_begins_upper)))
print(&quot;Are all the strings shorter than 8? &quot; + str(all(list_shorter_than_8)))
</code></pre>
This outputs:
<pre><code class="hljs">[False, True, False, True, True, False]
[True, True, True, True, True, True]
Do all the strings begin with an uppercase letter? False
Are all strings shorter than 8? True
</code></pre>
Note: If an empty <code>iterable</code> is passed to <code>all()</code>, the method returns <code>True</code>! This is because the code for <code>all()</code> checks if there are any <code>False</code> elements in the <code>iterable</code>, and in the case of an empty list there are no elements and therefore there are no <code>False</code> elements either.
<h2 id="booleanconversionandanyallfunctions">Boolean Conversion and any(), all() Functions</h2>
A common cause of confusion and errors when using any logical operators, and therefore when using <code>any()</code> and <code>all()</code> as well, is what happens when the elements aren't of the <code>boolean</code> data type. In other words, when they aren't exactly <code>True</code> of <code>False</code> but instead have to be evaluated to <code>True</code> or <code>False</code>.
Some programming languages don't evaluate non-<code>boolean</code> data types to <code>boolean</code>s. For example Java would complain if you tried something along the lines of <code>if(&quot;some string&quot;) </code> or <code>if(15)</code> and tell you that the type you used can't be converted to <code>boolean</code>.
Python on the other hand does no such thing, and will instead convert what you passed to <code>boolean</code> without warning you about it.
Python converts most things to <code>True</code> with a few exceptions:
<ul>
<li>Any numerical value equal to 0 (including 0.0) is treated as <code>False</code>. A common misconception here is that negative values (-2, -3.3,...) are treated as <code>False</code> as well, they are not treated as <code>False</code>!</li>
<li>Any empty sequence (or collection) is treated as <code>False</code>, including empty <code>string</code>s, empty <code>list</code>s, etc. Keep in mind that unexpected behavior might happen when using <code>all()</code> with an empty <code>iterable</code> (it will return <code>True</code>).</li>
<li>The actual <code>boolean</code> value <code>False</code> is of course treated as <code>False</code> as well as the special value <code>None</code>.</li>
</ul>
A few examples of how we can use the way Python &quot;boolean-izes&quot; other data types with <code>any()</code> and <code>all()</code>.
<pre><code class="hljs">list_of_strings = [&quot;yet&quot;, &quot;another&quot;, &quot;example&quot;,&quot;&quot;]
print(&quot;Do all strings have some content?&quot;, all(list_of_strings))

list_of_ints = [0, 0.0, -2, -5]
print(&quot;Are any of the ints different from 0?&quot;, any(list_of_ints))
</code></pre>
This outputs:
<pre><code class="hljs">Do all strings have some content? False
Are any of the ints different from 0? True
</code></pre>
Keep in mind that you might still want to write more readable code by not using implicit <code>boolean</code> conversion like this.
<h2 id="conclusion">Conclusion</h2>
Both the <code>any()</code> and <code>all()</code> functions are there for convenience sake and should be used only when they make the code shorter but maintain the readability.
In this article, we've jumped into the <code>any()</code> and <code>all()</code> functions and showcased their usage through several examples.

Olivera Popović

In this tutorial, we'll be going over examples and practical usage of the any() and all() convenience functions in Python.

any() and all() in Python with Examples

Darinka Zobenica

Dimitrije Stamenic

Structural Design Patterns in Python

<h2 id="introduction">Introduction</h2>
In this article, we'll discuss a concept related to serialization and deserialization in Java. Although sometimes considered as &quot;<a rel="nofollow noopener" target="_blank" href="https://stackoverflow.com/a/21562563/5943138">part of the black magic of the Java serialization API</a>&quot;, in this article we'll see that <code>serialVersionUID</code> is in fact quite straightforward and simple.
First, we'll gloss over serialization and deserialization to recollect some important ideas that we're going to need later on. Afterwards, we'll dive deeper into <code>serialVersionUID</code> and show what it is and how it works.
Finally, we'll conclude by showing an example which should tie everything together.
<h2 id="serializationanddeserialization">Serialization and Deserialization</h2>
Serialization is the process of storing the state of an object so that it can be persisted into a database, transferred over the network, written into a file, etc. How exactly does serialization work is beyond the scope of this article, but in general - it works by converting the object into a byte stream which can then be used as any other stream of information, e.g. transferred through a network socket.
Deserialization is the process opposite to serialization. It takes the byte-stream representation of an object (e.g. from a file or a socket) and converts it back into a Java object that lives inside the JVM.
Before either serialization or deserialization can be performed on an object, it is necessary that this object (i.e. its class) implements the <code>Serializable</code> interface. The <code>Serializable</code> interface is used to &quot;mark&quot; classes which can be (de)serialized.
Without a class implementing this interface, it's not possible to either serialize or deserialize objects from that class. In the words of the <a rel="nofollow noopener" target="_blank" href="https://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html">Serializable Javadoc</a>:
<blockquote>
&quot;Serializability of a class is enabled by the class implementing the java.io.Serializable interface*.
</blockquote>
<h2 id="whatistheserialversionuid">What is the serialVersionUID?</h2>
For serialization and deserialization to work properly, each serializable class must have a version number associated with it - the serialVersionUID. The purpose of this value is to make sure that the classes used both by the sender (the one that serializes) and the receiver (the one that deserializes) of the serialized object are compatible with each other.
If we think about this, it makes much more sense. There should be some mechanism of determining whether the object that was sent matches the object that was received. Otherwise, it could happen, for example, that a change was made to the class of an object before its serialization which the receiver isn't aware of.
Upon reading the object (i.e. deserialization), the reader could load the &quot;new&quot; object into the &quot;old&quot; representation. This could have annoying consequences at best, and complete disarray of the business logic at worst.
That's precisely the reason why the <code>serialVersionUID</code> exists and is typically used with all serializable objects. It is used to verify that both &quot;versions&quot; of an object (at the sender's and the receiver's side) are compatible, i.e. identical.
In case an update indeed needs to be made to the class, this can be denoted by incrementing the value of the <code>serialVersionUID</code>. The serialized version will thus have an updated UID which will be stored together with the object and delivered to the reader.
If the reader doesn't have the newest version of the class, an <code>InvalidClassException</code> will be thrown.
<h2 id="howtogenerateserialversionuid">How to Generate serialVersionUID?</h2>
As per the documentation, each serialVersionUID field must be <code>static</code>, <code>final</code>, and of type <code>long</code>. The <a target="_blank" href="https://stackabuse.com/access-modifiers-in-java/">access modifier</a> can be arbitrary, but it is strongly recommended that all declarations use the <code>private</code> modifier.
In that case, the modifier will only apply to the current class and not its subclasses which is the expected behavior; we don't want a class to be influenced by anything else other than itself. With all that said, this is how a properly constructed <code>serialVersionUID</code> might look like:
<pre><code class="hljs">private static final long serialVersionUID = 42L;
</code></pre>
Earlier we mentioned that all serializable classes must implement the <code>Serializable</code> interface.
This interface suggests that all serializable classes can declare a <code>serialVersionUID</code>, but are not obliged to. In case a class doesn't have an explicitly declared serialVersionUID value, one will be generated by the serialization runtime.
<blockquote>
However, it is strongly recommended that all serializable classes explicitly declare a <code>serialVersionUID</code> value.
</blockquote>
This is because the default, the <code>serialVersionUID</code> computation is complex, and thus sensitive to very slight differences in environments. If two different compilers are used in the serialization-deserialization process, an <code>InvalidClassException</code> may be thrown during deserialization because the classes will seemingly not match even though they contain the same contents, verbatim.
Finally, if there are any <code>transient</code> or <code>static</code> fields present in the class, they will be ignored during the serialization process and will be <code>null</code> after deserialization.
<h2 id="serialversionuidexample">serialVersionUID Example</h2>
Let's define a class that we'll be using for serialization and deserialization. Of course, it'll implement the <code>Serializable</code> interface and we'll start off with the <code>serialVersionUID</code> being <code>1</code>:
<pre><code class="hljs">public class Spaceship implements Serializable {

 private static final long serialVersionUID = 1L;

 private Pilot pilot;
 private Engine engine;
 private Hyperdrive hyperdrive;

 public void fly() {
 System.out.println(&quot;We&#x27;re about to fly high among the stars!&quot;);
 }

 // Constructor, Getters, Setters
}
</code></pre>
Next, we'll implement a <code>serializeObject()</code> method which will be responsible for serializing the object and writing it out to a <code>.ser</code> file:
<pre><code class="hljs">public void serializeObject(Spaceship spaceship) {
 ObjectOutputStream out = new ObjectOutputStream(
 new FileOutputStream(&quot;./spaceship.ser&quot;)
 );

 out.writeObject(spaceship);
 out.close();
}
</code></pre>
Our method serializes the <code>spaceship</code> object into a <code>.ser</code> file via a <code>FileOutputStream</code>. This file now contains the serialized contents of our object.
Now, let's implement a <code>deserializeObject()</code> method, which takes in that <code>.ser</code> file and constructs an object back from it:
<pre><code class="hljs">public void deserializeObject(String filepath) {
 Spaceship ship;

 ObjectInputStream in = new ObjectInputStream(
 new FileInputStream(filepath)
 );
 
 ship = (Spaceship) in.readObject();
 in.close();

 ship.fly();
}
</code></pre>
Let's call these two and observe the output:
<pre><code class="hljs">public class Main {
 public static void main(String[] args) {
 Spaceship spaceship = new Spaceship();
 serializeObject(spaceship);
 deserializeObject(&quot;./spaceship.ser&quot;);
 }
}
</code></pre>
This will result in:
<pre><code class="hljs">We&#x27;re about to fly high among the stars!
</code></pre>
Our <code>deserializeObject()</code> method loaded the serialized file into the JVM and successfully converted it into a <code>Spaceship</code> object.
To demonstrate the problem mentioned earlier regarding versioning - let's change the value of the <code>serialVersionUID</code> from <code>1L</code> to <code>2L</code> in our <code>Spaceship</code> class.
Afterwards, let's modify our <code>main()</code> method to read the file again, without writing it with the modified <code>serialVersionUID</code>:
<pre><code class="hljs">public class Main {
 public static void main(String[] args) {
 deserializeObject(&quot;./spaceship.ser&quot;);
 }
}
</code></pre>
Of course, this will result in:
<pre><code class="hljs">Exception in thread &quot;main&quot; java.io.InvalidClassException ...
</code></pre>
As expected, the reason for the exception lies in the <code>serialVersionUID</code>.
Because we haven't written the new data after updating the <code>serialVersionUID</code> value to <code>2L</code>, the serialized object still contains <code>1L</code> as its <code>serialVersionUID</code>.
However, the <code>deserializeObject()</code> method was expecting this value to be <code>2L</code> because that is the actual new value from within the <code>Spaceship</code> instance. Due to this inconsistency between the stored and the restored state of the <code>Spaceship</code> object, the exception was appropriately thrown.
<h2 id="conclusion">Conclusion</h2>
Serialization and deserialization are powerful and common techniques used for storing or transmitting data structures and objects. Sometimes it's easy to overlook certain important details such as the <code>serialVersionUID</code>, especially given the fact that IDEs typically generate it automatically.
Hopefully, it should be a bit more clear now what its purpose is and how to use it properly in the projects to come.

Luka Čupić

In this tutorial, we'll explain what is the serialVersionUID in Java and run through an example of serialization and deserialization using it.

What is the serialVersionUID in Java?

<h2 id="introduction">Introduction</h2>
Just like with all other types of files, you can use the Pandas library to read and write Excel files using Python as well. In this short tutorial, we are going to discuss how to read and write Excel files via <code>DataFrame</code>s.
In addition to simple reading and writing, we will also learn how to write multiple <code>DataFrame</code>s into an Excel file, how to read specific rows and columns from a spreadsheet, and how to name single and multiple sheets within a file before doing anything.
If you'd like to learn more about other file types, we've got you covered:
<ul>
<li><a target="_blank" href="/reading-and-writing-json-files-in-python-with-pandas/">Reading and Writing JSON Files in Python with Pandas</a></li>
<li><a target="_blank" href="/reading-and-writing-csv-files-in-python-with-pandas/">Reading and Writing CSV Files in Python with Pandas</a></li>
</ul>
<h2 id="readingandwritingexcelfilesinpythonwithpandas">Reading and Writing Excel Files in Python with Pandas</h2>
Naturally, to use Pandas, we first have to install it. The easiest method to install it is via <code>pip</code>.
If you're running Windows:
<pre><code class="hljs">$ python pip install pandas
</code></pre>
If you're using Linux or MacOS:
<pre><code class="hljs">$ pip install pandas
</code></pre>
Note that you may get a <code>ModuleNotFoundError</code> or <code>ImportError</code> error when running the code in this article. For example:
<pre><code class="hljs">ModuleNotFoundError: No module named 'openpyxl'
</code></pre>
If this is the case, then you'll need to install the missing module(s):
<pre><code class="hljs">$ pip install openpyxl xlsxwriter xlrd
</code></pre>
<h2 id="writingexcelfilesusingpandas">Writing Excel Files Using Pandas</h2>
We'll be storing the information we'd like to write to an Excel file in a <code>DataFrame</code>. Using the built-in <code>to_excel()</code> function, we can extract this information into an Excel file.
First, let's import the Pandas module:
<pre><code class="hljs">import pandas as pd
</code></pre>
Now, let's use a <a target="_blank" href="/python-dictionary-tutorial/">dictionary</a> to populate a <code>DataFrame</code>:
<pre><code class="hljs">df = pd.DataFrame({&#x27;States&#x27;:[&#x27;California&#x27;, &#x27;Florida&#x27;, &#x27;Montana&#x27;, &#x27;Colorodo&#x27;, &#x27;Washington&#x27;, &#x27;Virginia&#x27;],
 &#x27;Capitals&#x27;:[&#x27;Sacramento&#x27;, &#x27;Tallahassee&#x27;, &#x27;Helena&#x27;, &#x27;Denver&#x27;, &#x27;Olympia&#x27;, &#x27;Richmond&#x27;],
 &#x27;Population&#x27;:[&#x27;508529&#x27;, &#x27;193551&#x27;, &#x27;32315&#x27;, &#x27;619968&#x27;, &#x27;52555&#x27;, &#x27;227032&#x27;]})
</code></pre>
The keys in our dictionary will serve as column names. Similarly, the values become the rows containing the information.
Now, we can use the <code>to_excel()</code> function to write the contents to a file. The only argument is the file path:
<pre><code class="hljs">df.to_excel(&#x27;./states.xlsx&#x27;)
</code></pre>
Here's the Excel file that was created:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-1.png" alt="states spreadsheet">
Please note that we are not using any parameters in our example. Therefore, the sheet within the file retains its default name - &quot;Sheet 1&quot;. As you can see, our Excel file has an additional column containing numbers. These numbers are the indices for each row, coming straight from the Pandas <code>DataFrame</code>.
We can change the name of our sheet by adding the <code>sheet_name</code> parameter to our <code>to_excel()</code> call:
<pre><code class="hljs">df.to_excel(&#x27;./states.xlsx&#x27;, sheet_name=&#x27;States&#x27;)
</code></pre>
Similarly, adding the <code>index</code> parameter and setting it to <code>False</code> will remove the index column from the output:
<pre><code class="hljs">df.to_excel(&#x27;./states.xlsx&#x27;, sheet_name=&#x27;States&#x27;, index=False)
</code></pre>
Now, the Excel file looks like this:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-2.png" alt="states spreadsheet without index">
<h3 id="writingmultipledataframestoanexcelfile">Writing Multiple DataFrames to an Excel File</h3>
It is also possible to write multiple dataframes to an Excel file. If you'd like to, you can set a different sheet for each dataframe as well:
<pre><code class="hljs">income1 = pd.DataFrame({&#x27;Names&#x27;: [&#x27;Stephen&#x27;, &#x27;Camilla&#x27;, &#x27;Tom&#x27;],
 &#x27;Salary&#x27;:[100000, 70000, 60000]})

income2 = pd.DataFrame({&#x27;Names&#x27;: [&#x27;Pete&#x27;, &#x27;April&#x27;, &#x27;Marty&#x27;],
 &#x27;Salary&#x27;:[120000, 110000, 50000]})

income3 = pd.DataFrame({&#x27;Names&#x27;: [&#x27;Victor&#x27;, &#x27;Victoria&#x27;, &#x27;Jennifer&#x27;],
 &#x27;Salary&#x27;:[75000, 90000, 40000]})

income_sheets = {&#x27;Group1&#x27;: income1, &#x27;Group2&#x27;: income2, &#x27;Group3&#x27;: income3}
writer = pd.ExcelWriter(&#x27;./income.xlsx&#x27;, engine=&#x27;xlsxwriter&#x27;)

for sheet_name in income_sheets.keys():
 income_sheets[sheet_name].to_excel(writer, sheet_name=sheet_name, index=False)

writer.save()
</code></pre>
Here, we've created 3 different dataframes containing various names of employees and their salaries as data. Each of these dataframes is populated by its respective dictionary.
We've combined these three within the <code>income_sheets</code> variable, where each key is the sheet name, and each value is the <code>DataFrame</code> object.
Finally, we've used the <code>xlsxwriter</code> engine to create a <code>writer</code> object. This object is passed to the <code>to_excel()</code> function call.
Before we even write anything, we loop through the keys of <code>income</code> and for each key, write the content to the respective sheet name.
Here is the generated file:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-3.png" alt="multi-sheet excel file">
You can see that the Excel file has three different sheets named <code>Group1</code>, <code>Group2</code>, and <code>Group3</code>. Each of these sheets contains names of employees and their salaries with respect to the date in the three different dataframes in our code.
The engine parameter in the <code>to_excel()</code> function is used to specify which underlying module is used by the Pandas library to create the Excel file. In our case, the <a rel="nofollow noopener" target="_blank" href="https://xlsxwriter.readthedocs.io/index.html"><code>xlsxwriter</code></a> module is used as the engine for the <code>ExcelWriter</code> class. Different engines can be specified depending on their respective features.
Depending upon the Python modules installed on your system, the other options for the engine attribute are: <code>openpyxl</code> (for <code>xlsx</code> and <code>xlsm</code>), and <code>xlwt</code> (for <code>xls</code>).
Further details of using the <code>xlsxwriter</code> module with Pandas library are available at <a rel="nofollow noopener" target="_blank" href="https://xlsxwriter.readthedocs.io/working_with_pandas.html">the official documentation</a>.
Last but not least, in the code above we have to explicitly save the file using <code>writer.save()</code>, otherwise it won't be persisted on the disk.
<h2 id="readingexcelfileswithpandas">Reading Excel Files with Pandas</h2>
In contrast to writing <code>DataFrame</code> objects to an Excel file, we can do the opposite by reading Excel files into <code>DataFrame</code>s. Packing the contents of an Excel file into a <code>DataFrame</code> is as easy as calling the <code>read_excel()</code> function:
<pre><code class="hljs">students_grades = pd.read_excel(&#x27;./grades.xlsx&#x27;)
students_grades.head()
</code></pre>
For this example, we're reading <a rel="nofollow noopener" target="_blank" href="https://stackabuse.s3.amazonaws.com/files/reading-and-writing-excel-files-in-python-with-pandas-grades.xlsx">this Excel file</a>.
Here, the only required argument is the path to the Excel file. The contents are read and packed into a <code>DataFrame</code>, which we can then preview via the <code>head()</code> function.
Note: Using this method, although the simplest one, will only read the first sheet.
Let's take a look at the output of the <code>head()</code> function:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-4.png" alt="grades dataframe">
Pandas assigns a row label or numeric index to the <code>DataFrame</code> by default when we use the <code>read_excel()</code> function.
We can override the default index by passing one of the columns in the Excel file as the <code>index_col</code> parameter:
<pre><code class="hljs">students_grades = pd.read_excel(&#x27;./grades.xlsx&#x27;, sheet_name=&#x27;Grades&#x27;, index_col=&#x27;Grade&#x27;)
students_grades.head()
</code></pre>
Running this code will result in:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-5.png" alt="grades index">
In the example above, we have replaced the default index with the &quot;Grade&quot; column from the Excel file. However, you should only override the default index if you have a column with values that could serve as a better index.
<h3 id="readingspecificcolumnsfromanexcelfile">Reading Specific Columns from an Excel File</h3>
Reading a file in its entirety is useful, though in many cases, you'd really want to access a certain element. For example, you might want to read the element's value and assign it to a field of an object.
Again, this is done using the <code>read_excel()</code> function, though, we'll be passing the <code>usecols</code> parameter. For example, we can limit the function to only read certain columns. Let's add the parameter so that we read the columns that correspond to the &quot;Student Name&quot;, &quot;Grade&quot; and &quot;Marks Obtained&quot; values.
We do this by specifying the numeric index of each column:
<pre><code class="hljs">cols = [0, 1, 3]

students_grades = pd.read_excel(&#x27;./grades.xlsx&#x27;, usecols=cols)
students_grades.head()
</code></pre>
Running this code will yield:
<img src="https://stackabuse.s3.amazonaws.com/media/reading-and-writing-excel-files-python-pandas-library-6.png" alt="dataframe usecols">
As you can see, we are only retrieving the columns specified in the <code>cols</code> list.
<h2 id="conclusion">Conclusion</h2>
We've covered some general usage of the <code>read_excel()</code> and <code>to_excel()</code> functions of the Pandas library. With them, we've read existing Excel files and written our own data to them.
Using various parameters, we can alter the behavior of these functions, allowing us to build customized files, rather than just dumping everything from a <code>DataFrame</code>.

Guest Contributor

In this tutorial, we'll be reading and writing Excel files in Python with Pandas, including examples of using DataFrames and file handling.

Reading and Writing Excel Files in Python with the Pandas Library

Reading and Writing Excel (XLSX) Files in Python with the Pandas Library

<h3 id="introduction">Introduction</h3>
Binary Search is an efficient search algorithm that works on sorted arrays. It's often used as one of the first examples of algorithms that run in logarithmic time (O(logn)) because of its intuitive behavior and is a fundamental algorithm in computer science.
<blockquote>
In this article, we'll be diving into the idea behind and Python implementation of Binary Search.
</blockquote>
<h3 id="binarysearchexample">Binary Search - Example</h3>
Binary Search works on a divide-and-conquer approach and relies on the fact that the array is sorted to eliminate half of possible candidates in each iteration. More specifically, it compares the middle element of the sorted array to the element it's searching for in order to decide where to continue the search.
If the target element is larger than the middle element - it can't be located in the first half of the collection so it's discarded. The same goes the other way around.

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: If the array has an even number of elements, it doesn't matter which of the two &quot;middle&quot; elements we use to begin with.

 </div>
 </div>
 </div>
 Let's look at an example quickly before we continue to explain how binary search works:
<img src="https://stackabuse.s3.amazonaws.com/media/binary-search-in-python-1.png" alt="binary search in python visualization">
As we can see, we know for sure that, since the array is sorted, the element we're looking for (<code>18</code>) is not in the first half of the original array.

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: From now on, we'll mark the element we're looking for as <code>x</code>.

 </div>
 </div>
 </div>
 When we know in which half of the original array <code>x</code> is, we can repeat this exact process with that half and split it into halves again, discarding the half that surely doesn't contain <code>x</code>:
<img src="https://stackabuse.s3.amazonaws.com/media/binary-search-in-python-2.png" alt="binary search in python visualization">
We repeat this process until we end up with a subarray that contains only one element. We check whether that element is <code>x</code>. If it is - we found <code>x</code>, if it isn't - <code>x</code> doesn't exist in the array at all.
If you take a closer look at this, you can notice that in the worst-case scenario (<code>x</code> not existing in the array), we need to check a much smaller number of elements than we'd need to in an unsorted array - which would require something more along the lines of Linear Search, which is insanely inefficient.
To be more precise, the number of elements we need to check in the worst case is O(log2N) where N is the number of elements in the array.
This makes a bigger impact the bigger the array is:
<blockquote>
If our array had 10 elements, we would need to check only 3 elements to either find <code>x</code> or conclude it's not there. That's 33.3%.
However, if our array had 10,000,000 elements we would only need to check 24 elements. That's 0.0002%.
</blockquote>
<h3 id="binarysearchimplementation">Binary Search Implementation</h3>
Binary Search is a naturally recursive algorithm since the same process is repeated on smaller and smaller arrays until an array of size 1 has been found. However, there is of course an iterative implementation as well, and we will be showing both approaches.
<h4 id="recursive">Recursive</h4>
Let's start off with the recursive implementation as it's more natural:
<pre><code class="hljs">def binary_search_recursive(array, element, start, end):
 if start &gt; end:
 return -1

 mid = (start + end) // 2
 if element == array[mid]:
 return mid

 if element &lt; array[mid]:
 return binary_search_recursive(array, element, start, mid-1)
 else:
 return binary_search_recursive(array, element, mid+1, end)
</code></pre>

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: This code is simplified to best illustrate the use of the binary serach. It's not fully production-ready, since it lacks proper error handling. Nevertheless, you can use it to implement binary search, just be aware that you wouldn't be able to get appropriate error messages prompted to you.

 </div>
 </div>
 </div>
 Let's take a closer look at this code. We exit the recursion if the <code>start</code> element is higher than the <code>end</code> element:
<pre><code class="hljs">if start &gt; end:
 return -1
</code></pre>
This is because this situation occurs only when the element doesn't exist in the array. What happens is that we end up with only one element in the current sub-array, and that element doesn't match the one we're looking for.
At this point, the <code>start</code> is equal to the <code>end</code>. However, since the <code>element</code> isn't equal to <code>array[mid]</code>, we &quot;split&quot; the array again in such a way that we either decrease <code>end</code> by 1, or increase <code>start</code> by one, and the recursion exists on that condition.
We could have done this using a different approach:
<pre><code class="hljs">if len(array) == 1:
 if element == array[mid]:
 return mid
 else:
 return -1
</code></pre>
The rest of the code does the &quot;check middle element, continue search in the appropriate half of the array&quot; logic. We find the index of the middle element and check whether the element we're searching for matches it:
<pre><code class="hljs">mid = (start + end) // 2
if elem == array[mid]:
 return mid
</code></pre>
If it doesn't, we check whether the element is smaller or larger than the middle element:
<pre><code class="hljs">if element &lt; array[mid]:
 # Continue the search in the left half
 return binary_search_recursive(array, element, start, mid-1)
else:
 # Continue the search in the right half
 return binary_search_recursive(array, element, mid+1, end)
</code></pre>
Let's go ahead and run this algorithm, with a slight modification so that it prints out which subarray it's working on currently:
<pre><code class="hljs">element = 18
array = [1, 2, 5, 7, 13, 15, 16, 18, 24, 28, 29]

print(&quot;Searching for {}&quot;.format(element))
print(&quot;Index of {}: {}&quot;.format(element, binary_search_recursive(array, element, 0, len(array))))
</code></pre>
Running this code will result in:
<pre><code class="hljs">Searching for 18
Subarray in step 0:[1, 2, 5, 7, 13, 15, 16, 18, 24, 28, 29]
Subarray in step 1:[16, 18, 24, 28, 29]
Subarray in step 2:[16, 18]
Subarray in step 3:[18]
Index of 18: 7
</code></pre>
It's clear to see how it halves the search space in each iteration, getting closer and closer to the element we're looking for. If we tried searching for an element that doesn't exist in the array, the output would be:
<pre><code class="hljs">Searching for 20
Subarray in step 0: [4, 14, 16, 17, 19, 21, 24, 28, 30, 35, 36, 38, 39, 40, 41, 43]
Subarray in step 1: [4, 14, 16, 17, 19, 21, 24, 28]
Subarray in step 2: [19, 21, 24, 28]
Subarray in step 3: [19]
Index of 20: -1
</code></pre>
And just for the fun of it, we can try searching some large arrays and seeing how many steps it takes Binary Search to figure out whether a number exists:
<pre><code class="hljs">Searching for 421, in an array with 200 elements
Search finished in 6 steps. Index of 421: 169

Searching for 1800, in an array with 1500 elements
Search finished in 11 steps. Index of 1800: -1

Searching for 3101, in an array with 3000 elements
Search finished in 8 steps. Index of 3101: 1551
</code></pre>
<h4 id="iterative">Iterative</h4>
The iterative approach is very simple and similar to the recursive approach. Here, we just perform the checks in a <code>while</code> loop:
<pre><code class="hljs">def binary_search_iterative(array, element):
 mid = 0
 start = 0
 end = len(array)
 step = 0

 while (start &lt;= end):
 print(&quot;Subarray in step {}: {}&quot;.format(step, str(array[start:end+1])))
 step = step+1
 mid = (start + end) // 2

 if element == array[mid]:
 return mid

 if element &lt; array[mid]:
 end = mid - 1
 else:
 start = mid + 1
 return -1
</code></pre>
Let's populate an array and search for an element within it:
<pre><code class="hljs">element = 18
array = [1, 2, 5, 7, 13, 15, 16, 18, 24, 28, 29] 

print(&quot;Searching for {} in {}&quot;.format(element, array))
print(&quot;Index of {}: {}&quot;.format(element, binary_search_iterative(array, element)))
</code></pre>
Running this code gives us the output of:
<pre><code class="hljs">Searching for 18 in [1, 2, 5, 7, 13, 15, 16, 18, 24, 28, 29]
Subarray in step 0: [1, 2, 5, 7, 13, 15, 16, 18, 24, 28, 29]
Subarray in step 1: [16, 18, 24, 28, 29]
Subarray in step 2: [16, 18]
Subarray in step 3: [18]
Index of 18: 7
</code></pre>
<h3 id="conclusion">Conclusion</h3>
Binary Search is an incredible algorithm to use on large, sorted arrays, or whenever we plan to search for elements repeatedly in a single array.
The cost of sorting the array once and then using Binary Search to find elements in it multiple times is far better than using Linear Search on an unsorted array just so we could avoid the cost of sorting it.
If we're sorting the array and searching for an element just once, it's more efficient to just do a Linear Search on the unsorted array.
If you'd like to read about <a target="_blank" href="/sorting-algorithms-in-python">Sorting Algorithms in Python</a>, we've got you covered!

In this tutorial, we'll be covering Binary Search in Python, its idea, and both the iterative and recursive implementation.

Binary Search in Python

Mina Krivokuća

In this tutorial, we'll be covering how to use grep, as a command-line tool. We'll cover its options as well as practical examples and use cases.

How to Use the grep Command on GNU/Linux

François Dupire

In this tutorial, we'll dive into Relationship Mapping with JPA and Hibernate in Java, with examples of Many-to-Many, One-to-Many, Many-to-One and One-to-One annotations.