<h2 id="introduction">Introduction</h2>
<a rel="nofollow noopener" target="_blank" href="https://pandas.pydata.org/">Pandas</a> is a Python library for data analysis and manipulation. Almost all operations in <code>pandas</code> revolve around <code>DataFrame</code>s, an abstract data structure tailor-made for handling a metric ton of data.
In the aforementioned metric ton of data, some of it is bound to be missing for various reasons. Resulting in a missing (<code>null</code>/<code>None</code>/<code>Nan</code>) value in our <code>DataFrame</code>.
Which is why, in this article, we'll be discussing how to handle missing data in a Pandas <code>DataFrame</code>.
<h2 id="datainspection">Data Inspection</h2>
Real-world datasets are rarely perfect. They may contain missing values, wrong data types, unreadable characters, erroneous lines, etc.
The first step to to any proper data analysis is cleaning and organizing the data we'll later be using. We will discuss a few common problems related to data that might occur in a dataset.
We will be working with small employees dataset for this. The <code>.csv</code> file looks like this:
<pre><code class="hljs">First Name,Gender,Salary,Bonus %,Senior Management,Team
Douglas,Male,97308,6.945,TRUE,Marketing
Thomas,Male,61933,NaN,TRUE
Jerry,Male,NA,9.34,TRUE,Finance
Dennis,n.a.,115163,10.125,FALSE,Legal
,Female,0,11.598,,Finance
Angela,,,18.523,TRUE,Engineering
Shawn,Male,111737,6.414,FALSE,na
Rachel,Female,142032,12.599,FALSE,Business Development
Linda,Female,57427,9.557,TRUE,Client Services
Stephanie,Female,36844,5.574,TRUE,Business Development
,,,,,
</code></pre>
Let's import it into a <code>DataFrame</code>:
<pre><code class="hljs">df = pd.read_csv(&#x27;out.csv&#x27;)
df
</code></pre>
This results in:
<img src="https://stackabuse.s3.amazonaws.com/media/python-how-to-handle-missing-dataframe-values-in-pandas-1.png" alt="original dataframe pandas">
Taking a closer look at the dataset, we note that Pandas automatically assigns <code>NaN</code> if the value for a particular column is an empty string <code>''</code> <code>NA </code> or <code>NaN</code>. However, there are cases where missing values are represented by a custom value, for example, the string <code>'na'</code> or <code>0</code> for a numeric column.
For example, the 6th row has a value of <code>na</code> for the <code>Team</code> column, while the 5th row has a value of <code>0</code> for the <code>Salary</code> column.
<h2 id="customizingmissingdatavalues">Customizing Missing Data Values</h2>
In our dataset, we want to consider these as missing values:
<ol>
<li>A <code>0</code> value in the <code>Salary</code> column</li>
<li>An <code>na</code> value in the <code>Team</code> column</li>
</ol>
This can be achieved by using the <code>na_values</code> argument to set custom missing values. This argument represents a dictionary where the keys represent a column name and the value represents the data values that are to be considered as missing:
<pre><code class="hljs"># This means that in Salary column, 0 is also considered a missing value.
# And &#x27;na&#x27; is considered a missing value in the Team column.
df = pd.read_csv(&#x27;out.csv&#x27;, na_values={&quot;Salary&quot; : [0], &quot;Team&quot; : [&#x27;na&#x27;]})
df
</code></pre>
This results in:
<img src="https://stackabuse.s3.amazonaws.com/media/python-how-to-handle-missing-dataframe-values-in-pandas-2.png" alt="customized df">
On the other hand if we want to map a list of values which must be treated as missing values in all columns, we can pass a list of candidates which we want to globally consider as missing values to the <code>na_values</code> parameter:
<pre><code class="hljs">missing_values = [&quot;n.a.&quot;,&quot;NA&quot;,&quot;n/a&quot;, &quot;na&quot;, 0]
df = pd.read_csv(&#x27;out.csv&#x27;, na_values = missing_values)
df
</code></pre>
This results in:
<img src="https://stackabuse.s3.amazonaws.com/media/python-how-to-handle-missing-dataframe-values-in-pandas-3.png" alt="global missing values pandas">
Note that here, the value of <code>Gender</code> in the 4th row is also <code>NaN</code> now since we defined <code>n.a.</code> as a missing value above.
Which implementation to chose depends upon the nature of the dataset.
For example, some of the numeric columns in the dataset might need to treat <code>0</code> as a missing value while other columns may not. Therefore, you can use the first approach where you customize missing values based on columns.
Likewise, if we want to treat <code>0</code> for example as a missing value globally, we can utilize the second method and just pass an array of such values to the <code>na_values</code> argument.
Once we have identified all the missing values in the <code>DataFrame</code> and annotated them correctly, there are several ways we can handle missing data.
<h2 id="removingrowswithmissingvalues">Removing Rows With Missing Values</h2>
One approach would be removing all the rows which contain missing values. This can easily be done with the <code>dropna()</code> function, specifically dedicated to this:
<pre><code class="hljs"># Drops all rows with NaN values
df.dropna(axis=0,inplace=True)
</code></pre>
This results in:
<img src="https://stackabuse.s3.amazonaws.com/media/python-how-to-handle-missing-dataframe-values-in-pandas-6.png" alt="drop nan values pandas">
<code>inplace = True</code> makes all the changes in the existing <code>DataFrame</code> without returning a new one. Without it, you'd have to re-assign the <code>DataFrame</code> to itself.
The <code>axis</code> argument specifies if you're working with rows or columns - <code>0</code> being rows, and <code>1</code> being columns.
You can control whether you want to remove the rows containing at least 1 <code>NaN</code> or all <code>NaN</code> values by setting the <code>how</code> parameter in the <code>dropna</code> method.
how : {‘any’, ‘all’}
<ul>
<li><code>any</code>: if any NA values are present, drop that label</li>
<li><code>all</code>: if all values are NA, drop that label</li>
</ul>
<pre><code class="hljs">df.dropna(axis=0,inplace=True, how=&#x27;all&#x27;)
</code></pre>
This would only remove the last row from the dataset since <code>how=all</code> would only drop a row if all of the values are missing from the row.
Similarly, to drop columns containing missing values, just set <code>axis=1</code> in the <code>dropna</code> method.
<h2 id="fillingoutmissingvalues">Filling out Missing Values</h2>
It might not be the best approach to remove the rows containing missing values if such rows are abundant. They might contain valuable data in other columns and we don't want to skew the data towards an inaccurate state.
In this case, we have several options to assign appropriate values. The most common ones are listed below:
<ul>
<li>Fill NA with Mean, Median or Mode of the data</li>
<li>Fill NA with a constant value</li>
<li>Forward Fill or Backward Fill NA</li>
<li>Interpolate Data and Fill NA</li>
</ul>
Let's go through these one by one.
<h3 id="fillmissingdataframevalueswithcolumnmeanmedianandmode">Fill Missing DataFrame Values with Column Mean, Median and Mode</h3>
Let's start out with the <code>fillna()</code> method. It fills the NA-marked values with values you supply the method with.
For example, you can use the <code>.median()</code>, <code>.mode()</code> and <code>.mean()</code> functions on a column, and supply those as the fill value:
<pre><code class="hljs"># Using median
df[&#x27;Salary&#x27;].fillna(df[&#x27;Salary&#x27;].median(), inplace=True)
 
# Using mean
df[&#x27;Salary&#x27;].fillna(int(df[&#x27;Salary&#x27;].mean()), inplace=True)
 
# Using mode
df[&#x27;Salary&#x27;].fillna(int(df[&#x27;Salary&#x27;].mode()), inplace=True)
</code></pre>
Now, if a salary is missing from a person's rows, a mean, mode or median are used to fill that value. This way, you're not dropping these people from the dataset, and you're also not skewing the salary values.
While not perfect, this method allows you to introduce values that don't impact the overall dataset, since no matter how many averages you add, the average stays the same.
<h3 id="fillmissingdataframevalueswithaconstant">Fill Missing DataFrame Values with a Constant</h3>
You could also decide to fill the NA-marked values with a constant value. For example, you can put in a special string or numerical value:
<pre><code class="hljs">df[&#x27;Salary&#x27;].fillna(0, inplace=True)
</code></pre>
At the very least, these values are now actual values, instead of <code>na</code> or <code>NaN</code>.
<h3 id="forwardfillmissingdataframevalues">Forward Fill Missing DataFrame Values</h3>
This method would fill the missing values with first non-missing value that occurs before it:
<pre><code class="hljs">df[&#x27;Salary&#x27;].fillna(method=&#x27;ffill&#x27;, inplace=True)
</code></pre>
<h3 id="backwardfillmissingdataframevalues">Backward Fill Missing DataFrame Values</h3>
This method would fill the missing values with first non-missing value that occurs after it:
<pre><code class="hljs">df[&#x27;Salary&#x27;].fillna(method=&#x27;bfill&#x27;, inplace=True)
</code></pre>
<h3 id="fillmissingdataframevalueswithinterpolation">Fill Missing DataFrame Values with Interpolation</h3>
Finally, this method uses mathematical interpolation to determine what value would have been in the place of a missing value:
<pre><code class="hljs">df[&#x27;Salary&#x27;].interpolate(method=&#x27;polynomial&#x27;)
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Data cleaning and preprocessing is a very important part of every data analysis and each data science project.
In this article we went over several techniques to handle missing data which included customizing the missing data values and imputing the missing data values using different methods including mean, median, mode, a constant value, forward fill, backward fill and interpolation.

Hassan Saeed

In this tutorial, we'll go over how to handle missing data in a Pandas DataFrame. We'll cover data cleaning as well as dropping and filling values using mean, mode, median and interpolation.

Python: How to Handle Missing Data in Pandas DataFrame

<h2 id="introduction">Introduction</h2>
Having to manually format a number as a currency string can be a tedious process. You may have just a few lines of modifications to make, however, when we need to do a fair bit of conversions, it becomes very tedious.
The first step to automating these kinds of tasks will require a function. In this article, we'll be going over a few methods you can use to format numbers as currency strings in Python.
<h2 id="methodsforformattingnumbers">Methods for Formatting Numbers</h2>
We'll be going over three alternate libraries and functions which allow us to convert numbers into currency strings:
<ul>
<li>The <a rel="nofollow noopener" target="_blank" href="http://babel.pocoo.org/en/latest/">locale</a> module.</li>
<li>The <a rel="nofollow noopener" target="_blank" href="https://docs.python.org/3/library/locale.html#locale.setlocale">Babel</a> module.</li>
<li>The <code>str.format()</code> function.</li>
</ul>
The <code>locale</code> module is already included in Python, though, we'll have to install Babel to use it.
<h2 id="formatnumberascurrencystringwithlocale">Format Number as Currency String with Locale</h2>
The <code>locale</code> module comes already preinstalled with your version of Python.
This package allows developers to localize their applications. Meaning they don't have to know in which region their software will be run, they can just write a universal codebase which will dynamically change depending on the region of use.
<h3 id="initializingthelocale">Initializing the Locale</h3>
To begin using the <code>locale</code> module you first need to set the locale:
<pre><code class="hljs">import locale 

# To use default settings, set locale to None or leave the second argument blank.
print(locale.setlocale(locale.LC_ALL, &#x27;&#x27;))

# To use a specific locale (Great Britain&#x27;s locale in this case)
print(locale.setlocale(locale.LC_ALL, &#x27;en_GB&#x27;))
</code></pre>
The code above will produce the following output:
<pre><code class="hljs">English_United States.1252
en_GB
</code></pre>
To get the list of available locales, you can look it up on <a rel="nofollow noopener" target="_blank" href="https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f">MS-LCID</a>. Alternatively, you can print it out:
<pre><code class="hljs"># For the Windows operating system 
for lang in locale.windows_locale.values():
 print(lang)

# For other operating systems
for lang in locale.locale_alias.values():
 print(lang)
</code></pre>
Running any of the above variants will yield something similar to:
<pre><code class="hljs">en_GB
af_ZA
sq_AL
gsw_FR
am_ET
ar_SA
ar_IQ
ar_EG
ar_LY
ar_DZ
...
</code></pre>
<h3 id="formattingnumberswithlocale">Formatting Numbers with Locale</h3>
With your preferred locale set, you can easily format number strings:
<pre><code class="hljs">locale.setlocale(locale.LC_ALL, &#x27;&#x27;)

# If you&#x27;d like groupings - set grouping to True, else set it to false or leave it out completely
print(locale.currency(12345.67, grouping=True))
print(locale.currency(12345.67))
</code></pre>
Running the code above we get the following output:
<pre><code class="hljs">$12,345.67
$12345.67
</code></pre>
<h2 id="usingthestrformatmethod">Using the str.format() method</h2>
The next method we'll be covering is the <code>str.format()</code> method, which has the advantage of being the most straightforward one:
<pre><code class="hljs">number_string = 340020.8
# This portion is responsible for grouping the number
number_commas_only = &quot;{:,}&quot;.format(number_string)
print(number_commas_only)

# To ensure we have two decimal places
number_two_decimal = &quot;{:.2f}&quot;.format(number_string)
print(number_two_decimal)

# Both combined along with the currency symbol(in this case $)
currency_string = &quot;${:,.2f}&quot;.format(number_string)
print(currency_string)
</code></pre>
Running the code above we get the following output:
<pre><code class="hljs">340,020.8
340020.80
$340,020.80
</code></pre>
Though, this approach is hard-coded, unlike the previous one, which you can use to localize the formatting dynamically.
<h2 id="formatnumberascurrencystringwithbabel">Format Number as Currency String with Babel</h2>
Using Babel is perhaps one of the lesser known methods, however it's very user-friendly and intuitive. It comes with number and currency formatting as well as other internationalizing tasks.
Unlike Python's <code>locale</code> module, you don't have to worry about making adjustments on a global scale.
To install <code>Babel</code> via <code>pip</code>, run the following command:
<pre><code class="hljs">$ pip install Babel
...
Successfully installed Babel-2.9.0
</code></pre>
Once installed, to achieve the same results as the two other methods listed above, you can simply call <code>format_currency()</code> on a string:
<pre><code class="hljs">import babel.numbers
number_string = 340020.8

# The three needed arguments are the number, currency and locale
babel.numbers.format_currency(number_string, &quot;USD&quot;, locale=&#x27;en_US&#x27;)
</code></pre>
Running the code above we get the following output:
<pre><code class="hljs">$340,020.80
</code></pre>
To get the full list of locales available:
<pre><code class="hljs">avail_loc = babel.localedata.locale_identifiers()
print(avail_loc)
</code></pre>
Which looks something like this:
<pre><code class="hljs">[&#x27;af&#x27;, &#x27;af_NA&#x27;, &#x27;af_ZA&#x27;, &#x27;agq&#x27;, &#x27;agq_CM&#x27;, &#x27;ak&#x27;, &#x27;ak_GH&#x27;, &#x27;am&#x27;, &#x27;am_ET&#x27;,...]
</code></pre>
<h2 id="searchingfornumbersinstringsandformattingascurrency">Searching For Numbers in Strings and Formatting as Currency</h2>
Sometimes, you don't work with direct numerical input, such as the input from a user. You might be working with a sentence, or a larger, unclean corpus. We can use the <code>re</code> module to filter through different types of input, find numerical values and format them.
Let's use all three of the approaches above to format the currency in a sentence:
<pre><code class="hljs">import re
import locale
import babel.numbers
locale.setlocale(locale.LC_ALL, &#x27;en_US&#x27;)
</code></pre>
Next we come up with the regex pattern needed to match the number strings:
<pre><code class="hljs"> # This pattern is used to match any number string
 pattern = r&#x27;\d+(\.\d{1,2})?&#x27;
</code></pre>
Next we apply the three methods we've learned to the string variable <code>message</code>:
<pre><code class="hljs">message = &quot;Our current budget is 180000, we&#x27;ll need 25000.67 to cover rent, then 23400.4 for food.&quot;

# re.sub() is used to substitute substrings that match a certain pattern
# with another string, in our case the return value of a lambda function
# which will return a matching currency string.
new_message_locale = re.sub(
 pattern, lambda x: locale.currency(float(x.group()), grouping=True), message
)
new_message_str = re.sub(
 pattern, lambda x: &quot;${:,.2f}&quot;.format(float(x.group())), message
)
new_message_babel = re.sub(
 pattern,
 lambda x: babel.numbers.format_currency(float(x.group()), &quot;USD&quot;, locale=&quot;en_US&quot;),
 message,
)
</code></pre>
Let's compare the original output with the output gotten from all three methods:
<pre><code class="hljs">print(message)
print(new_message_locale)
print(new_message_str)
print(new_message_babel)
</code></pre>
<pre><code class="hljs">Our current budget is 180000, we&#x27;ll need 25000.67 to cover rent, then 23400.4 for food.
Our current budget is $180,000.00, we&#x27;ll need $25,000.67 to cover rent, then $23,400.40 for food.
Our current budget is $180,000.00, we&#x27;ll need $25,000.67 to cover rent, then $23,400.40 for food.
Our current budget is $180,000.00, we&#x27;ll need $25,000.67 to cover rent, then $23,400.40 for food.
</code></pre>
Depending on the method you prefer, the length of this script can be reduced. There are certain limitations as you may have noticed.
The script as it is, is unable to differentiate between number strings you'd like to format. However, it can be changed easily depending on your needs and use cases.
<h2 id="conclusion">Conclusion</h2>
In this article we took a look at a couple of ways of converting numbers into proper currency strings. We've covered the <code>str.format()</code> method, as well as the <code>locale</code> and <code>babel</code> modules.
Finally we combined these methods with Python's regular expression module to achieve a wider range of uses. In the end I hope you were able to learn something new from all this that can help save you time.

Daniel Pimeh

In this tutorial, we'll cover how to format a number as a currency string in Python, using the built-in str.format() method, the locale module and the Babel module.

How to Format Number as Currency String in Python

<h2 id="introduction">Introduction</h2>
Server logs are an important part of development that record the activities or events that the system is performing at any given point in time.
Spring Boot makes use of <a rel="nofollow noopener" target="_blank" href="https://commons.apache.org/proper/commons-logging/">Apache Commons' Logging</a> for its system logs by default. Additionally, by default you can use any of the logging frameworks under the <a rel="nofollow noopener" target="_blank" href="http://www.slf4j.org/docs.html">SLF4J API</a> such as Logback (which is the default), Log4J2, and Java Util Logging in Spring Boot.
Application logging is a powerful development tool that is important for production-level support and debugging. Each log entry contains information such as the timestamp, the actual method being called, a custom log message, and other contextual information. Each log entry also includes an identifier called a logging level.
In this tutorial, we'll take a look at how to perform Logging in Spring Boot, using SLF4J, as well as log levels and the configuration required to customize them, and log groups.
<h2 id="logmessagesinspringboot">Log Messages in Spring Boot</h2>
To enable logging in Spring, import <code>Logger</code> and <code>LoggerFactory</code> from the <code>org.slf4j</code> API library:
<pre><code class="hljs">import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
</code></pre>
Afterwards, declare a constant to store the logger service by instantiating a logger for the class using <code>LoggerFactory</code>:
<pre><code class="hljs">private static final Logger log = LoggerFactory.getLogger(DemoService.class);
</code></pre>
This <code>log</code> is usually defined at the very top of the global variable list, since it's a constant. By providing the class you want to log, it's fully set up and ready to be called with the provided messages.
Alternatively, you can just annotate the class with <code>@Slf4j</code> which will automatically generate a logger for the class without having to declare a constant. The annotation supplies a static variable called <code>log</code> which provides the logger utilities by default:
<pre><code class="hljs">@Slf4j
@Service
public class DemoService {
 //...
}
</code></pre>

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: The most common identifier for the logger is <code>log</code>, though you'll also see <code>logger</code> in a lot of cases. Really, you're free to call it whatever you'd like, but <code>log</code> is succinct and saves you from writing a longer name due to frequent calls.

 </div>
 </div>
 </div>
 Logging in Spring Boot comprises a few different levels. Every log has a degree of urgency or importance that is associated with a log level. Log levels are configured at runtime and each log request checks the log level to see if the request has equal or higher priority than the log level to be displayed.
If you try logging a message which is considered less important than some threshold, it just won't be logged.
Take this block for example, with messages of log importance in descending order:
<pre><code class="hljs">public void logLevelDisplay () {
 logger.error(&quot;Error message&quot;);
 logger.warn(&quot;Warning message&quot;);
 logger.info(&quot;Info message&quot;);
 logger.debug(&quot;Debug message&quot;);
 logger.trace(&quot;Trace message&quot;);
}
</code></pre>
If no configurations are set in <code>application.properties</code> or elsewhere, the log level will default to <code>INFO</code>. So if we were to call the method <code>logLevelDisplay()</code>, it would output something like this:
<pre><code class="hljs">2020-12-20 12:46:35.880 ERROR 47958 --- [nio-8080-exec-1] c.service.DemoService : Error message
2020-12-20 12:46:52.976 WARN 47958 --- [nio-8080-exec-1 c.service.DemoService : Warn message
2020-12-20 12:46:59.104 INFO 47958 --- [nio-8080-exec-1] c.service.DemoService : Info message
</code></pre>
The <code>DEBUG</code> and <code>TRACE</code> messages weren't displayed because the application's log level is set to <code>INFO</code>, which has a higher priority level than the two.
If the log level was set to <code>ERROR</code>, then only the error message would be displayed since it's the highest log level of the 5 built-in levels.
This brings us to an important topic - log levels, which deserve an explanation of their own.
<h2 id="loglevelsinspringboot">Log Levels in Spring Boot</h2>
As mentioned before, logging in Spring Boot comprises different levels. For example, Hibernate SQL queries are categorized as <code>DEBUG</code>, which is a low-urgency log level that is mainly invoked for debugging purposes.
Logging levels allows distinction between the following sample logs:
<ul>
<li>Successfully updated <code>User</code> information: Updated name <code>John</code> to <code>Jon</code>.</li>
<li>NullPointerException: <code>id</code> of <code>User</code> is <code>null</code></li>
</ul>
The first log is a general informative type of message while the second log is a <code>NullPointerException</code> message.
These logs have different levels of urgency, and distinction between them using log levels is important for filtering out logs that don't have to be output depending on the circumstance. If there's a huge error going on, you don't want it to be cluttered by a bunch of working, low-level <code>INFO</code> logs!
SLF4J provides 5 default logging levels in Spring boot:
<code>ERROR</code> - Error logs are serious issues that affect a significant part of the system or some part of your application has failed to operate. Exceptions are considered <code>ERROR</code> level logs. Other examples of error logs are database connection failures and configuration errors. <code>ERROR</code> logs are the most urgent default log level in SLF4J.
<code>WARN</code> - Warning logs are used to indicate potential problems that might cause errors and should be monitored in case they fail. Of course, the context of a warning is subjective to the developer and the situation so warning logs might vary from system to system.
<code>INFO</code> -<code>INFO</code> is the default logging level that is set by Spring Boot. If no configurations are made, the log level is automatically set to <code>INFO</code>. These types of logs are information that isn't normally needed but is useful in situations like production code debugging or determining when certain data is manipulated.
<code>DEBUG</code> - <code>DEBUG</code> logs include more detailed, specific information that isn't needed in normal situations. This is often set as a log level when a developer is trying to deep trace a problem or a bug that is hard to trace.
<code>TRACE</code> - <code>TRACE</code> is a more granular version of <code>DEBUG</code>. <code>TRACE</code> logs are exhaustive, imagine logging every single operation the system is doing, from starting a service, initializing new variables, and calling methods.
In order of urgency, <code>ERROR</code> is the most urgent while <code>TRACE</code> is the least urgent log. The default log level in Spring Boot is <code>INFO</code> when no manual configuration is set.
<h2 id="configuringloglevelsinspringboot">Configuring Log Levels in Spring Boot</h2>
Log levels can be set in the Spring environment by setting its configurations in <code>application.properties</code>.
The format to set the log level configuration is <code>logging.level.[classpath] = [level]</code>. The classpath is specified since different components of the application can be configured with different log levels, which is especially useful for code isolation and debugging.
To specify a log level for all classes that don't have their own log level settings, the <code>root</code> logger can be set using <code>logging.level.root</code>.
In <code>application.properties</code>:
<pre><code class="hljs">logging.level.root=INFO
logging.level.com.test=DEBUG
logging.level.com.service.DemoService=WARN
</code></pre>
From this configuration, every class except <code>DemoService</code> and the classes under the <code>com.test</code> classpath will have their log levels set to <code>INFO</code>, while the <code>test</code> classes and <code>DemoService</code> have their own specified log levels.
<h2 id="loggroups">Log Groups</h2>
Log groups is a useful way to set logger configurations to a group of classes with different classpaths. An example is if you want to set all the test class log levels to <code>DEBUG</code> in one go. This is possible using the configuration <code>logging.group.[groupName]</code>:
<pre><code class="hljs"># Initialize log group
logging.group.test=com.test, com.test-prod, com.apptest

# Set log level to log group
logging.level.test=DEBUG
</code></pre>
With this approach, you won't have to individually set the log level of all related components all the time.
<h2 id="conclusion">Conclusion</h2>
Knowing about the different log levels is important especially in situations like debugging in production.
Let's say a major bug has been exposed in production, and the current logs do not have enough information to diagnose the root cause of the problem. By changing the log level to <code>DEBUG</code> or <code>TRACE</code>, the logs will show much-needed information to pinpoint crucial details that may lead towards the fix.
In Spring, the log level configurations can be set in the <code>application.properties</code> file which is processed during runtime. Spring supports 5 default log levels, <code>ERROR</code>, <code>WARN</code>, <code>INFO</code>, <code>DEBUG</code>, and <code>TRACE</code>, with <code>INFO</code> being the default log level configuration.

David Landup

Rayven Yor Esplanada

In this tutorial, we'll cover how to log in Spring Boot with SLF4J. We'll cover log levels, log groups and the default Logback logger with examples.

Logging in Spring Boot with SLF4J

<h2 id="introduction">Introduction</h2>
Most web applications nowadays will require you to fill out a form at some point, be it an online banking application or a music streaming service.
And because end users are never to be trusted, we need to fool-proof our application so that it detects when the input is incorrect and returns it back to the user with an appropriate (error) message.
Form validation is a technique used to prevent users from providing data that does not meet the requirements of the application. An example would be making sure that the provided password has at least one uppercase character and a number. You can validate data on the server-side, or the client-side.
Server-side validation is the idea of validating user data after it is sent to the servers. If the data is not in the expected format, it is sent back to the user.
On the other hand, client-side validation entails validating the data input by the user in the browser before it is sent to the server. This is typically more efficient than server-side validation since it prevents the round trip from the client to server and back. It's also an early filter, which makes sure that the right data gets through, instead of dealing with the wrong data once it has been passed through.
This doesn't mean that the server shouldn't have data validation - we're just sifting through initially now.
In this tutorial, we will discuss how we can validate data in the browser using vanilla JavaScript.
<h2 id="creatingaform">Creating a Form</h2>
Let's make a simple sign-up form, which will will consist of the following fields:
<ol>
<li><code>username</code></li>
<li><code>first-password</code> - used to check the initial password for certain criteria</li>
<li><code>second-password</code> - used as confirmation, and making sure the user didn't make a typo in the <code>first-password</code> field.</li>
</ol>
... and put it into our <code>index.html</code> file:
<pre><code class="hljs">&lt;!DOCTYPE html&gt;
&lt;html lang=&quot;en&quot;&gt;
 &lt;head&gt;
 &lt;meta charset=&quot;UTF-8&quot; /&gt;
 &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot; /&gt;

 &lt;script defer src=&quot;validate.js&quot;&gt;&lt;/script&gt;

 &lt;title&gt;Form Validation&lt;/title&gt;
 &lt;/head&gt;
 &lt;body&gt;
 &lt;h1&gt;Form Validation Using JavaScript&lt;/h1&gt;

 &lt;form id=&quot;form&quot; action=&quot;/&quot; method=&quot;GET&quot;&gt;
 &lt;!-- We are only interested in client-side validation now --&gt;

 &lt;!-- All the fields are required --&gt;
 &lt;div&gt;
 &lt;label for=&quot;username&quot;&gt;Username: &lt;/label&gt;
 &lt;input type=&quot;text&quot; name=&quot;username&quot; id=&quot;username&quot; autofocus required /&gt;
 &lt;/div&gt;

 &lt;div&gt;
 &lt;label for=&quot;first-password&quot;&gt;Password: &lt;/label&gt;
 &lt;input
 type=&quot;password&quot;
 name=&quot;firstPassword&quot;
 id=&quot;first-password&quot;
 required
 /&gt;
 &lt;/div&gt;

 &lt;div&gt;
 &lt;label for=&quot;second-password&quot;&gt;Confirm Password: &lt;/label&gt;
 &lt;input
 type=&quot;password&quot;
 name=&quot;secondPassword&quot;
 id=&quot;second-password&quot;
 required
 /&gt;
 &lt;/div&gt;

 &lt;button type=&quot;submit&quot;&gt;Submit&lt;/button&gt;
 &lt;/form&gt;

 &lt;div id=&quot;show-error&quot;&gt;No validation error yet!&lt;/div&gt;
 &lt;!--We will use this div to display validation error --&gt;
 &lt;/body&gt;
&lt;/html&gt;
</code></pre>
This HTML file will render a simple form which looks like this:
<img src="https://stackabuse.s3.amazonaws.com/media/client-side-form-validation-using-vanilla-javascript-1.jpg" alt="">
<h2 id="formrequirements">Form-Requirements</h2>
Before writing any code, let's first go over the list of validations we need:
<ul>
<li>Make sure the username starts with an uppercase letter</li>
<li>The username must have at least one digit included</li>
<li>Ensure the password length is between 8 to 20 characters</li>
<li>Make sure the password contains at least one uppercase letter</li>
<li>Confirm the two passwords match</li>
</ul>
In case the user input isn't compliant with the requirements above, we want:
<ul>
<li>The text of the last <code>div</code> to change</li>
<li>Prevent the form from submitting</li>
</ul>
<h2 id="setup">Setup</h2>
First, let's make a script file <code>validate.js</code> and link it with our <code>index.html</code> file inside our <code>head</code> tag:
<pre><code class="hljs">&lt;script defer src=&quot;validate.js&quot;&gt;&lt;/script&gt;
</code></pre>
Then, let's access the relevant fields from the document:
<pre><code class="hljs">// To disable the form from submitting
const form = document.querySelector(&#x27;#form&#x27;);
// To display the error message
const errorDiv = document.querySelector(&#x27;#show-error&#x27;);
// To validate the username
const username = document.querySelector(&#x27;#username&#x27;);
// To validate the password
const firstPassword = document.querySelector(&#x27;#first-password&#x27;);
// To confirm the password
const secondPassword = document.querySelector(&#x27;#second-password&#x27;); 
</code></pre>
For the sake of simplicity, let's make it so that the form validations are only run upon clicking the <code>submit</code> button, and not in real-time:
<pre><code class="hljs">form.addEventListener(&#x27;submit&#x27;, (error) =&gt; {
 // All validation checks are run in this method. 
}
</code></pre>
<h2 id="implementingthevalidators">Implementing the Validators</h2>
<h3 id="firstletteroftheusernameisuppercase">First Letter of the Username is Uppercase</h3>
This is pretty self explanatory, if the string's first letter is the same as its uppercase variant, it means that <code>username</code> in fact starts with an uppercase letter:
<pre><code class="hljs">// We will include any and all errors in this string
let incorrectInput = &#x27;&#x27;;

const firstLetter = username.value[0];

// Return true if first letter is uppercase
const firstLetterIsUpperCase = (firstLetter === firstLetter.toUpperCase()); 

if (!firstLetterIsUpperCase) {
 incorrectInput += &#x27; The first letter of username must be uppercase.\n&#x27;;
}
</code></pre>
<h3 id="usernamecontainsatleastonedigit">Username Contains at Least One Digit</h3>
<code>/\d/</code> is a regular expression pattern that matches to a single digit, meaning if it matches even once inside the provided username, the username contains a digit:
<pre><code class="hljs">// Regex to see if a digit is in the username, returns true if there is
const usernameIncludesDigit = /\d/.test(username.value); 

if (!usernameIncludesDigit) {
 incorrectInput += &#x27;Username must include at least one digit.\n&#x27;;
}
</code></pre>
<h3 id="passwordisbetween8and20characterslong">Password is Between 8 and 20 Characters Long</h3>
The <code>length</code> string attribute should give us the number of characters in the password. A simple conditional statement should do the trick:
<pre><code class="hljs">const badPasswordLength = (firstPassword.value.length &lt; 8 || firstPassword.value.length &gt; 20);

if (badPasswordLength) {
 incorrectInput += &#x27; The password should be within 8 to 20 characters.\n&#x27;;
}
</code></pre>
<h3 id="passwordcontainsatleastoneuppercasecharacter">Password Contains at Least One Uppercase Character</h3>
This is similar to what we did for username. We just need to modify the regular expression to check for uppercase letters instead of digits:
<pre><code class="hljs">// Regex to see if a digit is in the username, returns true if there is
const passwordIncludesUppercase = /[a-z]/.test(firstPassword.value); 

if (!passwordIncludesUppercase) {
 incorrectInput += &#x27; The password should contain at least one uppercase character.\n&#x27;;
} 
</code></pre>
<h3 id="verifythatthetwopasswordsarethesame">Verify that the Two Passwords are the Same</h3>
Finally, we need to compare <code>firstPassword</code> to <code>secondPassword</code> to see if they match:
<pre><code class="hljs">if (firstPassword.value !== secondPassword.value) {
 incorrectInput += &#x27;The passwords do not match.\n&#x27;;
}
</code></pre>
<h3 id="displayingtheerrormessages">Displaying the Error Messages</h3>
After all these checks, if any condition is unsatisfied, <code>incorrectInput</code> will not be an empty string and will contain the issue raised in that conditional block.
In this case, we will change the text in <code>errorDiv</code> to show our errors in red:
<pre><code class="hljs">if (incorrectInput !== &quot;&quot;) {
 // Change the error div tag to display the error message(s)
 errorDiv.innerText = incorrectInput; 
 // Change the color of the text to red
 errorDiv.style.color = &#x27;red&#x27;; 
 // Prevent the form button from submitting again, before fixing the issues
 error.preventDefault(); 
}
</code></pre>
<h3 id="testingthecode">Testing the Code</h3>
And now, let's test out our new form with the following input:
<ul>
<li>Username: <code>johndoe</code></li>
<li>Password: <code>42</code></li>
<li>Confirm Password: <code>421</code></li>
</ul>
Which should yield the following results:
<img src="https://stackabuse.s3.amazonaws.com/media/client-side-form-validation-using-vanilla-javascript-3+(2).jpg" alt="">
<h2 id="conclusion">Conclusion</h2>
In this article we used vanilla JavaScript to validate a simple HTML form.
JavaScript enabled us to define custom validation checks that fit our use case. For example, you can add custom patterns that a password must meet in order for a user to sign up on your website.

Osama Akhtar

In this tutorial, we'll go over how to validate form input in JavaScript, for a registration form. We'll use client-side form validation in vanilla JavaScript.

Client-Side Form Validation Using Vanilla JavaScript

<h2 id="introduction">Introduction</h2>
There are many data visualization libraries in Python, yet Matplotlib is the most popular library out of all of them. Matplotlib’s popularity is due to its reliability and utility - it's able to create both simple and complex plots with little code. You can also customize the plots in a variety of ways.
In this tutorial, we'll cover how to plot Box Plots in Matplotlib.
<blockquote>
Box plots are used to visualize summary statistics of a dataset, displaying attributes of the distribution like the data’s range and distribution.
</blockquote>
<h2 id="importingdata">Importing Data</h2>
To create a Box Plot, we'll need some data to plot. We'll need to choose a dataset that contains continuous variables as features, since Box Plots visualize continuous variable distribution. We'll be working with the <a rel="nofollow noopener" target="_blank" href="https://www.kaggle.com/uciml/red-wine-quality-cortez-et-al-2009">Wine Quality</a> dataset.
We’ll begin by importing all the libraries that we need. We’ll import Pandas to read and parse the dataset, and we’ll of course need to import Matplotlib as well, or more accurately, the PyPlot module:
<pre><code class="hljs">import pandas as pd
from matplotlib import pyplot as plt
</code></pre>
Let’s check to make sure that our dataset is ready to use. We’ll print out the head of the dataset to make sure the data has been loaded properly, and we’ll also check to ensure that there are no missing data entries:
<pre><code class="hljs">dataframe = pd.read_csv(&quot;winequality-red.csv&quot;)
print(dataframe.head())
print(dataframe.isnull().values.any())
</code></pre>
<pre><code class="hljs"> fixed acidity volatile acidity citric acid ... sulphates alcohol quality
0 7.4 0.70 0.00 ... 0.56 9.4 5
1 7.8 0.88 0.00 ... 0.68 9.8 5
2 7.8 0.76 0.04 ... 0.65 9.8 5
3 11.2 0.28 0.56 ... 0.58 9.8 6
4 7.4 0.70 0.00 ... 0.56 9.4 5

[5 rows x 12 columns]
False
</code></pre>
The second print statement returns <code>False</code>, which means that there isn't any missing data. If there were, we'd have to <a target="_blank" href="/python-how-to-handle-missing-dataframe-values-in-pandas">handle missing DataFrame values</a>.
<h2 id="plotaboxplotinmatplotlib">Plot a Box Plot in Matplotlib</h2>
Let’s select some features of the dataset and visualize those features with the <code>boxplot()</code> function. We’ll make use of Pandas to extract the feature columns we want, and save them as variables for convenience:
<pre><code class="hljs">fixed_acidity = dataframe[&quot;fixed acidity&quot;]
free_sulfur_dioxide = dataframe[&#x27;free sulfur dioxide&#x27;]
total_sulfur_dioxide = dataframe[&#x27;total sulfur dioxide&#x27;]
alcohol = dataframe[&#x27;alcohol&#x27;]
</code></pre>
As usual, we can call plotting functions on the PyPlot instance (<code>plt</code>), the <code>Figure</code> instance or <code>Axes</code> instance:
<pre><code class="hljs">import pandas as pd
import matplotlib.pyplot as plt

dataframe = pd.read_csv(&quot;winequality-red.csv&quot;)

fixed_acidity = dataframe[&quot;fixed acidity&quot;]
free_sulfur_dioxide = dataframe[&#x27;free sulfur dioxide&#x27;]
total_sulfur_dioxide = dataframe[&#x27;total sulfur dioxide&#x27;]
alcohol = dataframe[&#x27;alcohol&#x27;]

fig, ax = plt.subplots()
ax.boxplot(fixed_acidity)
plt.show()
</code></pre>
Here, we've extracted the <code>fig</code> and <code>ax</code> objects from the return of the <code>subplots()</code> function, so we can use either of them to call the <code>boxplot()</code> function. Alternatively, we could've just called <code>plt.boxplot()</code>.
In any case, this results in:
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-1.png" alt="simple box plot matplotlib">
After creating the plot, we can see some of the summary statistics for our data. The Box Plot shows the median of the dataset (the vertical line in the middle), as well as the interquartile ranges (the ends of the boxes) and the minimum and maximum values of the chosen dataset feature (the far end of the “whiskers”).
We can also plot multiple columns on one figure, simply by providing more columns. This again, can be done on either the <code>plt</code> instance, the <code>fig</code> object or the <code>ax</code> object:
<pre><code class="hljs">import pandas as pd
import matplotlib.pyplot as plt

dataframe = pd.read_csv(&quot;winequality-red.csv&quot;)

fixed_acidity = dataframe[&quot;fixed acidity&quot;]
free_sulfur_dioxide = dataframe[&#x27;free sulfur dioxide&#x27;]
total_sulfur_dioxide = dataframe[&#x27;total sulfur dioxide&#x27;]
alcohol = dataframe[&#x27;alcohol&#x27;]

columns = [fixed_acidity, free_sulfur_dioxide, total_sulfur_dioxide, alcohol]

fig, ax = plt.subplots()
ax.boxplot(columns)
plt.show()
</code></pre>
This results in:
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-2.png" alt="box plot matplotlib multiple columns">
Now, we've got a lot more going on, since we've decided to plot multiple columns.
<h2 id="customizingtheplot">Customizing the Plot</h2>
As you can see, while the plots have successfully been generated, without tick labels on the X and Y-axis, it is difficult to interpret the graph.
We can customize the plot and add labels to the X-axis by using the <code>xticks</code> function. Let's pass in the number of labels we want to add and then the labels for each of those columns:
<pre><code class="hljs">fig, ax = plt.subplots()
ax.boxplot(columns)
plt.xticks([1, 2, 3, 4], [&quot;Fixed acidity&quot;, &quot;Free sulfur dioxide&quot;, &quot;Total sulfur dioxide&quot;, &quot;Alcohol&quot;], rotation=10)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-3.png" alt="label customization box plot matplotlib">
If we wanted to we could also change the orientation of the plot by altering the <code>vert</code> parameter. <code>vert</code> controls whether or not the plot is rendered vertically and it is set to <code>1</code> by default:
<pre><code class="hljs">fig, ax = plt.subplots()
ax.boxplot(fixed_acidity, vert=0)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-4.png" alt="vertical box plot matplotlib">
The <code>notch=True</code> attribute creates the notch format to the box plot, <code>patch_artist=True</code> fills the boxplot with colors:
<pre><code class="hljs">fig, ax = plt.subplots()
columns = [free_sulfur_dioxide, total_sulfur_dioxide]
ax.boxplot(columns, notch=True, patch_artist=True)
plt.xticks([1, 2], [&quot;Free sulfur dioxide&quot;, &quot;Total sulfur dioxide&quot;])
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-5.png" alt="notches and patches box plot matplotlib">
We can make use of the <code>meanline</code> argument to render the mean on the box, although this should be avoided if we are also showing notches, since they can conflict.
This must be combined with the <code>showmean</code> parameter. If possible, the mean will be visualized as a line that runs all the way across the box. If not possible, the mean will be shown as points:
<pre><code class="hljs">fig, ax = plt.subplots()
columns = [free_sulfur_dioxide, total_sulfur_dioxide]
ax.boxplot(columns, patch_artist=True, meanline=True, showmeans=True)
plt.xticks([1, 2], [&quot;Free sulfur dioxide&quot;, &quot;Total sulfur dioxide&quot;])
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-6.png" alt="showmean box plot matplotlib">
We can color the different feature columns by creating a list of hex color values and using the <code>set_facecolor</code> argument. In the below example, we <code>zip</code> the <code>boxes</code> element of the <code>box</code> variable together with the colors we want to use and then set the face color for each of those boxes.
<pre><code class="hljs">columns = [fixed_acidity, free_sulfur_dioxide, total_sulfur_dioxide, alcohol]
fig, ax = plt.subplots()
box = ax.boxplot(columns, notch=True, patch_artist=True)
plt.xticks([1, 2, 3, 4], [&quot;Fixed acidity&quot;, &quot;Free sulfur dioxide&quot;, &quot;Total sulfur dioxide&quot;, &quot;Alcohol&quot;])

colors = [&#x27;#0000FF&#x27;, &#x27;#00FF00&#x27;,
 &#x27;#FFFF00&#x27;, &#x27;#FF00FF&#x27;]

for patch, color in zip(box[&#x27;boxes&#x27;], colors):
 patch.set_facecolor(color)

plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/matplotlib-box-plot-tutorial-and-examples-7.png" alt="change box plot colors matplotlib">
<h2 id="conclusion">Conclusion</h2>
In this tutorial, we learned how to create a Box Plot in Matplotlib and Python. Then, we took a look at how you can customize it using arguments like <code>vert</code>, <code>meanline</code>, and <code>set_facecolor</code>.
If you're interested in Data Visualization and don't know where to start, make sure to check out our bundle of books on <a class="ebook-link" href="https://gum.co/data-visualization-in-python-book-bundle">Data Visualization in Python</a>:
<div style="border: 1px solid #ebebeb;padding: 15px;">
 <a class="ebook-link" rel="nofollow" target="_blank" href="https://gum.co/data-visualization-in-python-book-bundle" style="text-decoration: none;color: inherit;">
 <h4 style="margin-top: 10px;">Data Visualization in Python</h4>
 <div class="row">
 <div class="col-md-4 col-xs-12">
 <img class="img-responsive center-block" src="https://s3.stackabuse.com/media/ebooks/data-visualization-in-python/dvp-mpl-bundle-1-small.jpg">
 </div>
 <div class="col-md-8 col-xs-12 my-auto">
 Become dangerous with Data Visualization
 ✅ 30-day no-question money-back guarantee 
 ✅ Beginner to Advanced 
 ✅ Updated regularly for free (latest update in April 2021) 
 ✅ Updated with bonus resources and guides 
 </div>
 </div>
 </a>
</div>
Data Visualization in Python with Matplotlib and Pandas is a book designed to take absolute beginners to Pandas and Matplotlib, with basic Python knowledge, and allow them to build a strong foundation for advanced work with these libraries - from simple plots to animated 3D plots with interactive buttons.
<blockquote>
It serves as an in-depth guide that'll teach you everything you need to know about Pandas and Matplotlib, including how to construct plot types that aren't built into the library itself.
</blockquote>
Data Visualization in Python, a book for beginner to intermediate Python developers, guides you through simple data manipulation with Pandas, covers core plotting libraries like Matplotlib and Seaborn, and shows you how to take advantage of declarative and experimental libraries like Altair. More specifically, over the span of 11 chapters this book covers 9 Python libraries: Pandas, Matplotlib, Seaborn, Bokeh, Altair, Plotly, GGPlot, GeoPandas, and VisPy.
<blockquote>
It serves as a unique, practical guide to Data Visualization, in a plethora of tools you might use in your career.
</blockquote>

Dan Nelson

In this tutorial, we'll cover how to plot Box Plots in Matplotlib with Python. We'll cover basic Box Plots and Customization with examples in detail.

Matplotlib Box Plot - Tutorial and Examples

Geoffery, Joseph

In this tutorial, we'll go over everything you need to know to Integrate MongoDB with Flask and Python by building a functional CRUD application.

Integrating MongoDB with Flask Using Flask-PyMongo

<h2 id="introduction">Introduction</h2>
Seaborn is one of the most widely used data visualization libraries in Python, as an extension to Matplotlib. It offers a simple, intuitive, yet highly customizable API for data visualization.
In this tutorial, we'll take a look at how to plot a boxplot in Seaborn.
<blockquote>
Boxplots are used to visualize summary statistics of a dataset, displaying attributes of the distribution like the data’s range and distribution.
</blockquote>
<h2 id="importdata">Import Data</h2>
We’ll need to select a dataset with continuous features in order to create a boxplot, because boxplots display summary statistics for continuous variables - the median and range of a dataset. We’ll be working with the <a rel="nofollow noopener" target="_blank" href="http://archive.ics.uci.edu/ml/datasets/Forest+Fires">Forest Fires</a> dataset.
We’ll begin with importing Pandas to load and parse the dataset. We’ll obviously want to import Seaborn as well. Finally, we’ll import the Pyplot module from Matplotlib, so that we can show the visualizations:
<pre><code class="hljs">import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
</code></pre>
Let's use Pandas to read the CSV file and check how our <code>DataFrame</code> looks by printing its head. Additionally, we'll want to check if the dataset contains any missing values:
<pre><code class="hljs">dataframe = pd.read_csv(&quot;forestfires.csv&quot;)
print(dataframe.head())
print(dataframe.isnull().values.any())
</code></pre>
<pre><code class="hljs"> X Y month day FFMC DMC DC ISI temp RH wind rain area
0 7 5 mar fri 86.2 26.2 94.3 5.1 8.2 51 6.7 0.0 0.0
1 7 4 oct tue 90.6 35.4 669.1 6.7 18.0 33 0.9 0.0 0.0
2 7 4 oct sat 90.6 43.7 686.9 6.7 14.6 33 1.3 0.0 0.0
3 8 6 mar fri 91.7 33.3 77.5 9.0 8.3 97 4.0 0.2 0.0
4 8 6 mar sun 89.3 51.3 102.2 9.6 11.4 99 1.8 0.0 0.0
False
</code></pre>
The second print statement returns <code>False</code>, which means that there isn't any missing data. If there were, we'd have to <a target="_blank" href="/python-how-to-handle-missing-dataframe-values-in-pandas">handle missing DataFrame values</a>.
After we check for the consistency of our dataset, we want to select the continuous features that we want to visualize. We’ll save these as their own variables for convenience:
<pre><code class="hljs">FFMC = dataframe[&quot;FFMC&quot;]
DMC = dataframe[&quot;DMC&quot;]
DC = dataframe[&quot;DC&quot;]
RH = dataframe[&quot;RH&quot;]
ISI = dataframe[&quot;ISI&quot;]
temp = dataframe[&quot;temp&quot;]
</code></pre>
<h2 id="plottingaboxplotinseaborn">Plotting a Boxplot in Seaborn</h2>
Now that we have loaded in the data and selected the features that we want to visualize, we can create the Boxplots!
We can create the boxplot just by using Seaborn’s <code>boxplot</code> function. We pass in the dataframe as well as the variables we want to visualize:
<pre><code class="hljs">sns.boxplot(x=DMC)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-1.png" alt="seaborn simple boxplot">
If we want to visualize just the distribution of a categorical variable, we can provide our chosen variable as the <code>x</code> argument. If we do this, Seaborn will calculate the values on the Y-axis automatically, as we can see on the previous image.
However, if there’s a specific distribution that we want to see segmented by type, we can also provide a categorical X-variable and a continuous Y-variable.
<pre><code class="hljs">day = dataframe[&quot;day&quot;]
sns.boxplot(x=DMC, y=day)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-2.png" alt="seaborn x and y variable boxplot">
This time around, we can see a boxplot generated for each day in the week, as specified in the dataset.
If we want to visualize multiple columns at the same time, what do we provide to the <code>x</code> and <code>y</code> arguments? Well, we provide the labels for the data we want, and provide the actual data using the <code>data</code> argument.
We can create a new <code>DataFrame</code> containing just the data we want to visualize, and <code>melt()</code> it into the <code>data</code> argument, providing labels such as <code>x='variable'</code> and <code>y='value'</code>:
<pre><code class="hljs">df = pd.DataFrame(data=dataframe, columns=[&quot;FFMC&quot;, &quot;DMC&quot;, &quot;DC&quot;, &quot;ISI&quot;])
sns.boxplot(x=&quot;variable&quot;, y=&quot;value&quot;, data=pd.melt(df))
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-3.png" alt="seaborn data argument boxplot">
<h2 id="customizeaseabornboxplot">Customize a Seaborn Boxplot</h2>
<h3 id="changeboxplotcolors">Change Boxplot Colors</h3>
Seaborn will automatically assign the different colors to different variables so we can easily visually differentiate them. Though, we can also supply a list of colors to be used if we'd like to specify them.
After choosing a list of colors with hex values (or any valid Matplotlib color), we can pass them into the <code>palette</code> argument:
<pre><code class="hljs">day = dataframe[&quot;day&quot;]
colors = [&#x27;#78C850&#x27;, &#x27;#F08030&#x27;, &#x27;#6890F0&#x27;,&#x27;#F8D030&#x27;, &#x27;#F85888&#x27;, &#x27;#705898&#x27;, &#x27;#98D8D8&#x27;]
sns.boxplot(x=DMC, y=day, palette=colors)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-4.png" alt="seaborn change colors boxplot">
<h3 id="customizeaxislabels">Customize Axis Labels</h3>
We can adjust the X-axis and Y-axis labels easily using Seaborn, such as changing the font size, changing the labels, or rotating them to make the ticks easier to read:
<pre><code class="hljs">df = pd.DataFrame(data=dataframe, columns=[&quot;FFMC&quot;, &quot;DMC&quot;, &quot;DC&quot;, &quot;ISI&quot;])
boxplot = sns.boxplot(x=&quot;variable&quot;, y=&quot;value&quot;, data=pd.melt(df))
boxplot.axes.set_title(&quot;Distribution of Forest Fire Conditions&quot;, fontsize=16)
boxplot.set_xlabel(&quot;Conditions&quot;, fontsize=14)
boxplot.set_ylabel(&quot;Values&quot;, fontsize=14)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-5.png" alt="seaborn customize boxplot axis labels">
<h3 id="orderingboxplots">Ordering Boxplots</h3>
If we want to view the boxes in a specific order, we can do that by making use of the <code>order</code> argument, and supplying the column names in the order you want to see them in:
<pre><code class="hljs">df = pd.DataFrame(data=dataframe, columns=[&quot;FFMC&quot;, &quot;DMC&quot;, &quot;DC&quot;, &quot;ISI&quot;])
boxplot = sns.boxplot(x=&quot;variable&quot;, y=&quot;value&quot;, data=pd.melt(df), order=[&quot;DC&quot;, &quot;DMC&quot;, &quot;FFMC&quot;, &quot;ISI&quot;])
boxplot.axes.set_title(&quot;Distribution of Forest Fire Conditions&quot;, fontsize=16)
boxplot.set_xlabel(&quot;Conditions&quot;, fontsize=14)
boxplot.set_ylabel(&quot;Values&quot;, fontsize=14)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-6.png" alt="seaborn order boxplots">
<h2 id="creatingsubplots">Creating Subplots</h2>
If we wanted to separate out the plots for the individual features into their own subplots, we could do that by creating a figure and axes with the <code>subplots</code> function from Matplotlib. Then, we use the <code>axes</code> object and access them via their index. The <code>boxplot()</code> function accepts an <code>ax</code> argument, specifying on which <code>axes</code> it should be plotted on:
<pre><code class="hljs">fig, axes = plt.subplots(1, 2)
sns.boxplot(x=day, y=DMC, orient=&#x27;v&#x27;, ax=axes[0])
sns.boxplot(x=day, y=DC, orient=&#x27;v&#x27;, ax=axes[1])
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-7.png" alt="seaborn create boxplot subplots">
<h2 id="boxplotwithdatapoints">Boxplot with Data Points</h2>
We could even overlay a swarmplot onto the boxplot in order to see the distribution and samples of the points comprising that distribution, with a bit more detail.
In order to do this, we just create a single figure object and then create two different plots. The <code>stripplot()</code> will be overlaid over the <code>boxplot()</code>, since they're on the same <code>axes</code>/<code>figure</code>:
<pre><code class="hljs">df = pd.DataFrame(data=dataframe, columns=[&quot;FFMC&quot;, &quot;DMC&quot;, &quot;DC&quot;, &quot;ISI&quot;])
boxplot = sns.boxplot(x=&quot;variable&quot;, y=&quot;value&quot;, data=pd.melt(df), order=[&quot;DC&quot;, &quot;DMC&quot;, &quot;FFMC&quot;, &quot;ISI&quot;])
boxplot = sns.stripplot(x=&quot;variable&quot;, y=&quot;value&quot;, data=pd.melt(df), marker=&quot;o&quot;, alpha=0.3, color=&quot;black&quot;, order=[&quot;DC&quot;, &quot;DMC&quot;, &quot;FFMC&quot;, &quot;ISI&quot;])
boxplot.axes.set_title(&quot;Distribution of Forest Fire Conditions&quot;, fontsize=16)
boxplot.set_xlabel(&quot;Conditions&quot;, fontsize=14)
boxplot.set_ylabel(&quot;Values&quot;, fontsize=14)
plt.show()
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/seaborn-box-plot-tutorial-and-examples-8.png" alt="seaborn overlay stripplot over boxplot">
<h2 id="conclusion">Conclusion</h2>
In this tutorial, we've gone over several ways to plot a boxplot using Seaborn and Python. We've also covered how to customize the colors, labels, ordering, as well as overlay swarmplots and subplot multiple boxplots.
If you're interested in Data Visualization and don't know where to start, make sure to check out our bundle of books on <a class="ebook-link" href="https://gum.co/data-visualization-in-python-book-bundle">Data Visualization in Python</a>:
<div style="border: 1px solid #ebebeb;padding: 15px;">
 <a class="ebook-link" rel="nofollow" target="_blank" href="https://gum.co/data-visualization-in-python-book-bundle" style="text-decoration: none;color: inherit;">
 <h4 style="margin-top: 10px;">Data Visualization in Python</h4>
 <div class="row">
 <div class="col-md-4 col-xs-12">
 <img class="img-responsive center-block" src="https://s3.stackabuse.com/media/ebooks/data-visualization-in-python/dvp-mpl-bundle-1-small.jpg">
 </div>
 <div class="col-md-8 col-xs-12 my-auto">
 Become dangerous with Data Visualization
 ✅ 30-day no-question money-back guarantee 
 ✅ Beginner to Advanced 
 ✅ Updated regularly for free (latest update in April 2021) 
 ✅ Updated with bonus resources and guides 
 </div>
 </div>
 </a>
</div>
Data Visualization in Python with Matplotlib and Pandas is a book designed to take absolute beginners to Pandas and Matplotlib, with basic Python knowledge, and allow them to build a strong foundation for advanced work with these libraries - from simple plots to animated 3D plots with interactive buttons.
<blockquote>
It serves as an in-depth guide that'll teach you everything you need to know about Pandas and Matplotlib, including how to construct plot types that aren't built into the library itself.
</blockquote>
Data Visualization in Python, a book for beginner to intermediate Python developers, guides you through simple data manipulation with Pandas, covers core plotting libraries like Matplotlib and Seaborn, and shows you how to take advantage of declarative and experimental libraries like Altair. More specifically, over the span of 11 chapters this book covers 9 Python libraries: Pandas, Matplotlib, Seaborn, Bokeh, Altair, Plotly, GGPlot, GeoPandas, and VisPy.
<blockquote>
It serves as a unique, practical guide to Data Visualization, in a plethora of tools you might use in your career.
</blockquote>

In this tutorial, we'll cover how to plot a Box Plot in Seaborn and Python with detailed examples of plotting and customization.

Seaborn Box Plot - Tutorial and Examples

Seaborn Boxplot - Tutorial and Examples

<h2 id="introduction">Introduction</h2>
Scheduling a job in Jenkins is very useful when it comes to developing a great product. Imagine what a great advantage it is to transform a time-consuming task into an automated job. It doesn't need any supervision and as long as there are no errors in the workflow, you shouldn't have any worries.
In today's article we'll make a job that will run every five minutes and has the following output:
<pre><code class="hljs">Hello, World!
Today&#x27;s:
&lt;&lt;date&gt;&gt;
Here&#x27;s the current time:
&lt;&lt;time&gt;&gt;
</code></pre>
Additionally, we'll build a job that watches another project, and depending on its status, prints out an automated statement.
<h2 id="usecasesforautomationwithjenkins">Use-Cases for Automation with Jenkins</h2>
But before we dive into the technical part, let's take a moment to consider what can be automated in Jenkins. Of course, with a little imagination, you can tackle most problems and automate them. Let's take a look at an example that could be automated with Jenkins if you have a little (more) patience and you're also eager to deepen your Jenkins skills.
Products like Amazon Echo and Alexa could be configured to call Jenkins jobs that perform various actions. You could relatively easily integrate it into your smart home:
<ol>
<li>Say, &quot;Alexa, lock the doors&quot; (that would be the trigger, meaning Jenkins will listen to that event).</li>
<li>Alexa skill activates a Jenkins job.</li>
<li>Jenkins' job will use a script to call the smart doors locking system API.</li>
<li>Alexa waits for the Jenkins job to finish and tells you how it went, i.e. &quot;Door successfully locked&quot;, &quot;The doors are already locked&quot; or &quot;The doors could not be locked&quot;.</li>
</ol>
Of course, that's not a CI/CD job, but is well within the domain of possibilities for existing tech with Jenkins.
<h2 id="requirements">Requirements</h2>
<ul>
<li>A Jenkins server – for this tutorial, we'll use the official Docker image to set up a Jenkins server</li>
<li>A job that you'd like automated</li>
</ul>
<h2 id="creatingajenkinsjob">Creating a Jenkins Job</h2>
Login in to your Jenkins application and click on Create a Job. The type of job doesn't matter. It can be Freestyle, Maven, etc. All that matters is how we configure the scheduler.
For this article, we'll go with Pipeline. Choose a name for your project and save it:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-1.png" alt="create jenkins project">
Now, before we deal with the scheduling part, let's set up the Pipeline Groovy script which will print the message from the introduction.
Scroll down to Pipeline and add the following script:
<pre><code class="hljs">pipeline {
 agent any

 stages {
 stage(&#x27;Write Greetings&#x27;) {
 steps {
 echo &#x27;Hello, World!&#x27;
 }
 }
 
 stage(&#x27;Write Date&#x27;) {
 steps {
 print &#x27;Today\&#x27;s &#x27;
 print new Date().format(&quot;dd-MM-yyyy&quot;, TimeZone.getTimeZone(&quot;UTC&quot;))
 }
 }
 
 stage(&#x27;Write Hour&#x27;) {
 steps {
 print &#x27;Here\&#x27;s the current time &#x27;
 print new Date().format(&quot;HH:mm:ss&quot;, TimeZone.getTimeZone(&quot;UTC&quot;))
 }
 }
 }
}
</code></pre>
The script has three stages: Write Greetings, Write Date, and Write Hour. We use stages in order to better define and structure our script. Also, the stages can be viewed in the Jenkins UI.
In case one of them fails, you'll know which one:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-2.png" alt="Pipeline Groovy Script">
<h2 id="runningthejob">Running the Job</h2>
We have our job ready to be triggered! Let's click on Build Now and check the output:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-3.png" alt="Build the jenkins job">
<pre><code class="hljs">Started by user Alex
Running in Durability level: MAX_SURVIVABILITY
Running on Jenkins in /var/jenkins_home/workspace/Scheduled Pipeline
Hello, World!
Today&#x27;s:
24-12-2020
Here&#x27;s the current time:
16:00:38
Finished: SUCCESS
</code></pre>
You can also see on the Job's page the average time for every stage:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-4.png" alt="Jenkins Job Stage Details">
There are many ways to trigger a Jenkins job: manually (of course), using a Cron Expression, hooks, watching other projects, etc.
<h2 id="schedulingjenkinsjobswithcron">Scheduling Jenkins Jobs with Cron</h2>
Let's start with triggering a job via the Cron utility. We'll choose Build periodically and for the Schedule field use the following expression:
<pre><code class="hljs">H/5 * * * *
</code></pre>
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-5.png" alt="Schedule the Jenkins Job">
This expression means the job will be executed every 5 minutes. A few useful examples of other patterns could be:
<ul>
<li><code>0 0 13 1/1 * ? *</code> – every hour, starting with 13:00;</li>
<li><code>0 0 17 ? * MON-FRI *</code> – every working day, at 5 PM;</li>
</ul>
If you don't want to learn this type of formatting in order to schedule your job, you can use a Cron Expression generator like <a rel="nofollow noopener" target="_blank" href="http://www.cronmaker.com/?1">CronMaker</a>.
Let's see the result. For this, we'll have to wait. After 10 minutes you should see that 2 jobs were triggered automatically:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-6.png" alt="Jenkins Job Results">
Notice the first job was executed at 9:12 PM, then at 9:17 PM, and the last at 9:22 PM. So the rule worked! We have a job which runs every 5 minutes.
<h2 id="schedulingajenkinsjobbywatchingotherprojects">Scheduling a Jenkins Job by Watching Other Projects</h2>
Now that we've seen how we can schedule a job periodically, let's create another one that triggers itself every time the first one was successful. Create another Pipeline project, and add the following code:
<pre><code class="hljs">pipeline {
 agent any

 stages {
 stage(&#x27;It worked&#x27;) {
 steps {
 echo &#x27;The other job worked!&#x27;
 }
 }
 }
}
</code></pre>
We have our pipeline script, let's schedule it. In the Build Triggers tab, select Build after other projects are built.
Enter the project which needs to be watched and select the cases when this one should be triggered:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-7.png" alt="Watch Projects">
Save it and let's wait for a little while. You should see the new job trigger itself every time the first one succeeds. You can see the details of the first run of this second job that was triggered by the job created before:
<img src="https://stackabuse.s3.amazonaws.com/media/scheduling-a-job-with-jenkins-8.png" alt="Jenkins Job Details">
Here's the output:
<pre><code class="hljs">Started by upstream project &quot;Scheduled Pipeline&quot; build number 7
originally caused by:
 Started by timer
Running in Durability level: MAX_SURVIVABILITY
Running on Jenkins in /var/jenkins_home/workspace/The other job worked
The other job worked!
Finished: SUCCESS
</code></pre>
<h2 id="conclusion">Conclusion</h2>
Automation is very useful and one of the key things technology brings us, regardless of the process you want to automate.
In this article, we've gone over how to schedule a job in Jenkins to run periodically using a Cron pattern, as well as to run when other projects are built.

Alex Chirea

In this tutorial, we'll go over examples on how to schedule a job in Jenkins to run periodically using Cron and how to run by watching other projects.

How to Schedule a Job with Jenkins

<h2 id="introduction">Introduction</h2>
Dictionaries are one of the most used data structures in all of software development, and for a good reason. They allow us to store our data in neat key, value pairs, which in turn gives us the ability to, on average, access our data in <code>O(1)</code> time.
While using a dictionary it's important to know how to iterate over it. Not being able to recover the data you stored makes it practically useless.
In this article, we'll see how to iterate through a Python dictionary with all kinds of iterators and the <code>for</code> loop.
<h2 id="usingthekeysmethod">Using the keys() Method</h2>
Python dictionaries have a handy method which allows us to easily iterate through all initialized keys in a dictionary, <code>keys()</code>.
Keep in mind that since Python 3, this method does not return a list, it instead returns a view object. A view object is exactly like the name says, a view of some data.
This means that we can iterate through said data no problem, but if we want to actually store the list of keys, we need to materialize it. Which can easily be done by forwarding the provided view object to a <code>list</code> constructor.
Let's take a look at how it all works:
<pre><code class="hljs">my_dict = {&#x27;alpha&#x27;: 5, &#x27;beta&#x27;: 4, &#x27;gamma&#x27;: 3}

# Here we&#x27;re just &#x27;looking&#x27; at the keys,
# we&#x27;re not actually constructing a list 
# out of them
key_view = my_dict.keys()
print(&quot;Key view:&quot;, key_view)
print(&quot;Type:&quot;, type(key_view),end=&quot;\n\n&quot;)

# Here we&#x27;re materializing the keys
# into a list ofo keys
key_list = list(my_dict.keys())
print(&quot;Key list:&quot;, key_list)
print(&quot;Type: &quot;, type(key_list),end=&quot;\n\n&quot;)

# And as we can see, view can be easily be
# used for iterating over a dictionary
for key in my_dict.keys():
 print(key, &#x27;:&#x27;, my_dict[key]) 
</code></pre>
Running the code gives us the following output:
<pre><code class="hljs">Key view: dict_keys([&#x27;alpha&#x27;, &#x27;beta&#x27;, &#x27;gamma&#x27;])
Type: &lt;class &#x27;dict_keys&#x27;&gt;

Key list: [&#x27;alpha&#x27;, &#x27;beta&#x27;, &#x27;gamma&#x27;]
Type: &lt;class &#x27;list&#x27;&gt;

alpha : 5
beta : 4
gamma : 3 
</code></pre>
An alternative way of doing this would be:
<pre><code class="hljs">my_dict = {&#x27;alpha&#x27;: 5, &#x27;beta&#x27;: 4, &#x27;gamma&#x27;: 3}

for key in my_dict:
 print(key, &#x27;:&#x27;, my_dict[key])
</code></pre>
When using the <code>in</code> keyword in conjunction with a dictionary, the dictionary invokes its <code>__iter__()</code> method. This method then returns an iterator which is used to implicitly go through the keys of the provided dictionary.
<h2 id="usingthevaluesmethod">Using the values() Method</h2>
Just like the <code>keys()</code> method, the <code>values()</code> method also return a view object, but instead of iterating through keys, it iterates through values:
<pre><code class="hljs">my_dict = {&#x27;alpha&#x27;: 5, &#x27;beta&#x27;: 4, &#x27;gamma&#x27;: 3}

# Inspecting the view of the values
# in the dictionary
key_list = list(my_dict.values())
print(&quot;Value list:&quot;, key_list)
print(&quot;Type: &quot;, type(key_list), end=&quot;\n\n&quot;)

for value in my_dict.values():
 print(value, end=&quot; &quot;)
</code></pre>
Running the code gives us the following output:
<pre><code class="hljs">Value list: [5, 4, 3]
Type: &lt;class &#x27;list&#x27;&gt;
5 4 3 
</code></pre>
Unlike the previous method, this one only provides values. It is useful when you are not concerned about keys.
<h2 id="usingtheitemsmethod">Using the items() method</h2>
Just like the <code>keys()</code> and <code>values()</code> methods, the <code>items()</code> method also returns a view object, but instead of just iterating through either <code>keys</code> or <code>values</code>, it iterates through <code>(key,value)</code> pairs.
Let's take a look at how it all works:
<pre><code class="hljs">my_dict = {&#x27;alpha&#x27;: 5, &#x27;beta&#x27;: 4, &#x27;gamma&#x27;: 3}

# Inspecting the view of the (key,value) pairs
key_list = list(my_dict.items())
print(&quot;(key,value) pair list:&quot;, key_list)
print(&quot;Type: &quot;, type(key_list), end=&quot;\n\n&quot;)

for item in my_dict.items():
 print(item, end=&quot; &quot;)
</code></pre>
Running the code gives us the following output:
<pre><code class="hljs">(key,value) pair list: [(&#x27;alpha&#x27;, 5), (&#x27;beta&#x27;, 4), (&#x27;gamma&#x27;, 3)]
Type: &lt;class &#x27;list&#x27;&gt;
(&#x27;alpha&#x27;, 5) (&#x27;beta&#x27;, 4) (&#x27;gamma&#x27;, 3) 
</code></pre>
To immediately assign both keys and values simultaneously, we can use tuple unpacking and extract them by using variables for each value in a tuple:
<pre><code class="hljs">for key, value in my_dict.items():
 print(key,&#x27;:&#x27;,value)
</code></pre>
It is important to note that in older Python 2 versions where <code>items()</code>, <code>keys()</code>, and <code>values()</code> returned a copy of data from a dictionary. While in Python 3 they return a view object.
These are more effective as they provide a dynamic view and, additionally if any changes are made in the original dictionary, they will immediately be reflected in the view object (and vice versa).
<h2 id="conclusion">Conclusion</h2>
In this article, we've covered different ways to iterate over a Python dictionary. This includes the <code>keys()</code>, <code>values()</code> and <code>items()</code> methods, with a <code>for</code> loop.

Guest Contributor

In this tutorial, we'll go over how to iterate through a dictionary in Python, using keys(), values() and items() and a for loop, with examples.