David Landup

Arpendu Kumar Garai

Convert YAML Array into Java List with SnakeYAML

Turning a collection of numbers (or objects whose fields you'd like to inspect) into a distribution of those numbers is a common statistical technique, and is employed in various contexts in reporting and data-driven applications.
Given a collection:
<pre><code class="hljs">1, 1, 2, 1, 2, 3, 1, 4, 5, 1, 3
</code></pre>
You can inspect their distribution as a count (frequency of each element), and store the results in a map:
<pre><code class="hljs">{
&quot;1&quot;: 5,
&quot;2&quot;: 2,
&quot;3&quot;: 2,
&quot;4&quot;: 1,
&quot;5&quot;: 1
}
</code></pre>
Or, you can normalize the values based on the total number of values - thus expressing them in percentages:
<pre><code class="hljs">{
&quot;1&quot;: 0.45,
&quot;2&quot;: 0.18,
&quot;3&quot;: 0.18,
&quot;4&quot;: 0.09,
&quot;5&quot;: 0.09
}
</code></pre>
Or even express these percentages in a <code>0..100</code> format instead of a <code>0..1</code> format.
<blockquote>
In this guide, we'll take a look at how you can calculate a distribution from a collection - both using primitive types and objects whose fields you might want to report in your application.
</blockquote>
With the addition of functional programming support in Java - calculating distributions is easier than ever. We'll be working with a collection of numbers and a collection of <code>Book</code>s:
<pre><code class="hljs">public class Book {

 private String id;
 private String name;
 private String author;
 private long pageNumber;
 private long publishedYear;

 // Constructor, getters, setters, toString()
}
</code></pre>
<h2 id="calculatedistributionofcollectioninjava">Calculate Distribution of Collection in Java</h2>
Let's first take a look at how you can calculate a distribution for primitive types. Working with objects simply allows you to call custom methods from your domain classes to provide more flexibility in the calculations.
By default, we'll represent the percentages as a double from <code>0.00</code> to <code>100.00</code>.
<h3 id="primitivetypes">Primitive Types</h3>
Let's create a list of integers and print their distribution:
<pre><code class="hljs">List&lt;Integer&gt; integerList = List.of(1, 1, 2, 1, 2, 3, 1, 4, 5, 1, 3);
System.out.println(calculateIntegerDistribution(integerList));
</code></pre>
The distribution is calculated with:
<pre><code class="hljs">public static Map&lt;Integer, Double&gt; calculateIntegerDistribution(List&lt;Integer&gt; list) {
 return list.stream()
 .collect(Collectors.groupingBy(Integer::intValue,
 Collectors.collectingAndThen(Collectors.counting(),
 count -&gt; (Double.parseDouble(String.format(&quot;%.2f&quot;, count * 100.00 / list.size()))))));
}
</code></pre>
This method accepts a list and streams it. While streamed, the values are grouped by their integer value - and their values are counted using <code>Collectors.counting()</code>, before being collected into a <code>Map&lt;Integer, Double&gt;</code> where the keys represent the input values and the doubles represent their percentages in the distribution.
The key methods here is <code>collect()</code> which accepts two collectors. The key-collector collects by simply grouping by the key values (input elements). The value-collector collects via the <code>collectingAndThen()</code> method, which allows us to count the values and then format them in another format, such as <code>count * 100.00 / list.size()</code> which lets us express the counted elements in percentages:
<pre><code class="hljs">{1=45.45, 2=18.18, 3=18.18, 4=9.09, 5=9.09}
</code></pre>

 <div class="alert alert-reference">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-link-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Advice: If you'd like to read more about <code>groupingBy()</code>, <code>counting()</code> and <code>collectingAndThen()</code> - read our in-depth, example driven <a target="_blank" href="https://stackabuse.com/guide-to-java-8-collectors-groupingby/">&quot;Guide to Java 8 Collectors: groupingBy()&quot;</a>, <a target="_blank" href="https://stackabuse.com/guide-to-java-8-collectors-counting/">&quot;Guide to Java 8 Collectors: counting()&quot;</a> and <a target="_blank" href="https://stackabuse.com/guide-to-java-8-collectors-collectingandthen/">&quot;Guide to Java 8 Collectors: collectingAndThen()&quot;</a>.

 </div>
 </div>
 </div>
 <h3 id="sortdistributionbyvalueorkey">Sort Distribution by Value or Key</h3>
When creating distributions - you'll typically want to sort the values. More often than not, this'll be by key. Java <code>HashMap</code>s don't guarantee to preserve order of insertion, so we'll have to use a <code>LinkedHashMap</code> which does. Additionally, it's easiest to re-stream the map and re-collect it now that it's a much smaller size and much more manageable.
The previous operation can quickly collapse multiple thousand records into small maps, depending on the number of keys you're dealing with, so re-streaming isn't expensive:
<pre><code class="hljs">public static Map&lt;Integer, Double&gt; calculateIntegerDistribution(List&lt;Integer&gt; list) {
 return list.stream()
 .collect(Collectors.groupingBy(Integer::intValue,
 Collectors.collectingAndThen(Collectors.counting(),
 count -&gt; (Double.parseDouble(String.format(&quot;%.2f&quot;, count.doubleValue() / list.size()))))))
 // Stream the entries of the distribution map
 // to sort it and store in a LinkedHashMap
 .entrySet()
 .stream()
 .sorted(Map.Entry.comparingByKey())
 .collect(Collectors.toMap(e -&gt; Integer.parseInt(e.getKey().toString()),
 Map.Entry::getValue,
 (a, b) -&gt; {
 throw new AssertionError();
 },
 LinkedHashMap::new));
}
</code></pre>
<h3 id="objects">Objects</h3>
How can this be done for objects? The same logic applies! Instead of an identify function (<code>Integer::intValue</code>), we'll be using the desired field instead - such as the published year for our books. Let's create a few books, store them in a list and then calculate the distributions of the publishing years:
<pre><code class="hljs">Book book1 = new Book(&quot;001&quot;, &quot;Our Mathematical Universe&quot;, &quot;Max Tegmark&quot;, 432, 2014);
Book book2 = new Book(&quot;002&quot;, &quot;Life 3.0&quot;, &quot;Max Tegmark&quot;, 280, 2017);
Book book3 = new Book(&quot;003&quot;, &quot;Sapiens&quot;, &quot;Yuval Noah Harari&quot;, 443, 2011);
Book book4 = new Book(&quot;004&quot;, &quot;Steve Jobs&quot;, &quot;Water Isaacson&quot;, 656, 2011);

List&lt;Book&gt; books = Arrays.asList(book1, book2, book3, book4);
</code></pre>
Let's calculate the distribution of the <code>publishedYear</code> field:
<pre><code class="hljs">public static Map&lt;Integer, Double&gt; calculateDistribution(List&lt;Book&gt; books) {
 return books.stream()
 .collect(Collectors.groupingBy(Book::getPublishedYear,
 Collectors.collectingAndThen(Collectors.counting(),
 count -&gt; (Double.parseDouble(String.format(&quot;%.2f&quot;, count * 100.00 / books.size()))))))
 // Sort results by year
 .entrySet()
 .stream()
 .sorted(Map.Entry.comparingByKey())
 .collect(Collectors.toMap(e -&gt; Integer.parseInt(e.getKey().toString()),
 Map.Entry::getValue,
 (a, b) -&gt; {
 throw new AssertionError();
 },
 LinkedHashMap::new));
}
</code></pre>
Adjust the <code>&quot;%.2f&quot;</code> to set the floating point precision. This results in:
<pre><code class="hljs">{2011=50.0, 2014=25.0, 2017=25.0}
</code></pre>
50% of the given books (2/4) were published in 2011, 25% (1/4) were published in 2014 and 25% (1/4) in 2017. What if you want to format this result differently, and normalize the range in <code>0..1</code>?
<h2 id="calculatenormalizedpercentagedistributionofcollectioninjava">Calculate Normalized (Percentage) Distribution of Collection in Java</h2>
To normalize the percentages from a <code>0.0...100.0</code> range to a <code>0..1</code> range - we'll simply adapt the <code>collectingAndThen()</code> call to not multiply the count by <code>100.0</code> before dividing by the size of the collection.
Previously, the <code>Long</code> count returned by <code>Collectors.counting()</code> was implicitly converted into a double (multiplication with a double value) - so this time, we'll want to explicitly get the <code>doubleValue()</code> of the <code>count</code>:
<pre><code class="hljs"> public static Map&lt;Integer, Double&gt; calculateDistributionNormalized(List&lt;Book&gt; books) {
 return books.stream()
 .collect(Collectors.groupingBy(Book::getPublishedYear,
 Collectors.collectingAndThen(Collectors.counting(),
 count -&gt; (Double.parseDouble(String.format(&quot;%.4f&quot;, count.doubleValue() / books.size()))))))
 // Sort results by key
 .entrySet()
 .stream()
 .sorted(comparing(e -&gt; e.getKey()))
 .collect(Collectors.toMap(e -&gt; Integer.parseInt(e.getKey().toString()),
 Map.Entry::getValue,
 (a, b) -&gt; {
 throw new AssertionError();
 },
 LinkedHashMap::new));
}
</code></pre>
Adjust the <code>&quot;%.4f&quot;</code> to set the floating point precision. This results in:
<pre><code class="hljs">{2011=0.5, 2014=0.25, 2017=0.25}
</code></pre>
<h2 id="calculateelementcountfrequencyofcollection">Calculate Element Count (Frequency) of Collection</h2>
Finally - we can get the element count (frequency of all elements) in the collection by simply not dividing the count by the size of the collection! This is a fully non-normalized count:
<pre><code class="hljs"> public static Map&lt;Integer, Integer&gt; calculateDistributionCount(List&lt;Book&gt; books) {
 return books
 .stream()
 .collect(Collectors.groupingBy(Book::getPublishedYear,
 Collectors.collectingAndThen(Collectors.counting(),
 count -&gt; (Integer.parseInt(String.format(&quot;%s&quot;, count.intValue()))))))
 // Sort values by key
 .entrySet()
 .stream()
 .sorted(Map.Entry.comparingByKey())
 .collect(Collectors.toMap(e -&gt; Integer.parseInt(e.getKey().toString()),
 Map.Entry::getValue,
 (a, b) -&gt; {
 throw new AssertionError();
 },
 LinkedHashMap::new));
}
</code></pre>
This results in:
<pre><code class="hljs">{2011=2, 2014=1, 2017=1}
</code></pre>
Indeed, there are two books from 2011, and one from 2014 and 2017 each.
<h2 id="conclusion">Conclusion</h2>
Calculating distributions of data is a common task in data-rich applications, and doesn't require the use of external libraries or complex code. With functional programming support, Java made working with collections a breeze!
In this short draft, we've taken a look at how you can calculate frequency counts of all elements in a collection, as well as how to calculate distribution maps normalized to percentages between <code>0</code> and <code>1</code> as well as <code>0</code> and <code>100</code> in Java.

In this short guide, learn how to count the frequency of elements in a Java list or other collections, as well as calculate the percentage distributions normalized to `0..1` and `0..100` using Java Collectors.

Calculate Distribution from Collection in Java

Null values arise a lot in software development, and proper handling of <code>null</code> values can be turned into a science in and of itself. Jackson is the de facto library for serialization and deserialization of Java objects - and a common scenario pertains to serializing objects that have <code>null</code> fields.
<blockquote>
In this short tutorial, we'll take a look at how you can ignore null fields when serializing a POJO with Jackson.
</blockquote>
<h2 id="changingtheobjectmappersettings">Changing the ObjectMapper Settings</h2>
Jackson's ObjectMapper is the central API that maps objects to serialized values. Naturally, you can adjust and customize the way it works through various flags, one of which is:
<pre><code class="hljs">objectMapper.setSerializationInclusion(Include.NON_NULL);
</code></pre>
When serializing with this mapper, properties will only be included if they're non-null.
This sets the flag globally for any object being serialized with the <code>objectMapper</code> instance. If you're working with Spring Boot, and the ObjectMapper it's relying on isn't exposed, you can alter the flag in the <code>application.properties</code> file:
<pre><code class="hljs">spring.jackson.default-property-inclusion = non_null
</code></pre>
Or, if you're using <code>application.yml</code>:
<pre><code class="hljs">spring:
 jackson:
 default-property-inclusion: non_null
</code></pre>

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: Older versions of Spring used <code>spring.jackson.serialization-inclusion=non_null</code> instead of <code>spring.jackson.default-property-inclusion=non_null</code>.

 </div>
 </div>
 </div>
 Alternatively, you can peel back replace the default <code>ObjectMapper</code> instance by registering a new <code>@Primary</code> <code>@Bean</code>, named <code>objectMapper()</code>, which overrides the default implementation and allows full customization over the instance that'll be used by default when serializing and deserializing:
<pre><code class="hljs">@Bean
@Primary
public ObjectMapper objectMapper() {
 return new ObjectMapper()
 .setSerializationInclusion(JsonInclude.Include.NON_NULL);
}
</code></pre>
This <code>@Bean</code> should go into any file that's annotated with the <code>@Configuration</code> annotation, and is oftentimes situated either in a specialized configuration class or the main class of the Spring Boot application.
<h2 id="changingthejackson2objectmapperbuilderinstance">Changing the Jackson2ObjectMapperBuilder Instance</h2>
Spring uses the <code>HttpMessageConverters</code> to convert between HTTP messages and objects. The default converters are the <code>ObjectMapper</code> and <code>XmlMapper</code> instances instantiated via the <code>Jackson2ObjectMapperBuilder</code>.
Instead of manually providing the primary <code>ObjectMapper</code> (replacing the one provided by the builder), you can instruct Spring on how to build the object mapper instead. This allows Spring to do its high-level configurations without you tampering with the lower-level objects, while still customizing the way it works. Additionally, you can also customize the <code>XmlMapper</code> this way in one go.
To customize the builder, you register a new <code>@Bean</code> of <code>Jackson2ObjectMapperBuilderCustomizer</code> type, which customizes and returns the builder:
<pre><code class="hljs">@Bean
public Jackson2ObjectMapperBuilderCustomizer customBuilder() {
 return builder -&gt; {
 builder.serializationInclusion(JsonInclude.Include.NON_NULL);
 }
}
</code></pre>
<h2 id="jsonincludeannotation">@JsonInclude Annotation</h2>
On the other end of the spectrum, we can use annotations, as a low-code alternative! Annotations can be applied at the class-level or method-level:
<pre><code class="hljs">@JsonInclude(JsonInclude.Include.NON_NULL)
</code></pre>
This allows you to customize only certain classes, instead of the global behavior of the object mapper.

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: As of Jackson 2.0, the annotation's value is <code>JsonInclude.Include.NON_NULL</code>. For earlier versions, use <code>include=JsonSerialize.Inclusion.NON_NULL</code>.

 </div>
 </div>
 </div>
 <h3 id="jsonincludeonclasslevel">@JsonInclude on Class-Level</h3>
When you apply <code>@JsonInclude</code> at class-level, the annotation extends to all of the getters and setters:
<pre><code class="hljs">@Entity(name=&quot;property&quot;)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class Property implements Serializable {
 private static final long serialVersionUID = 1L;
 
 @Id
 @GeneratedValue(strategy=GenerationType.AUTO)
 private String id;

 @Column(name=&quot;transaction_type&quot;)
 private TransactionType transactionType;

 @Column(name=&quot;property_type&quot;)
 private PropertyType propertyType;
 
 // Constructor, getters, setters, toString(), etc.
}
</code></pre>
Any given field that has a <code>null</code> value won't be serialized.
<h3 id="jsonincludeonmethodlevel">@JsonInclude on Method-Level</h3>
Alternatively, you can apply the annotation at method-level as the most granular option:
<pre><code class="hljs">@Entity(name=&quot;property&quot;)
public class Property implements Serializable {
 private static final long serialVersionUID = 1L;
 
 @Id
 @GeneratedValue(strategy=GenerationType.AUTO)
 private String id;

 @Column(name=&quot;transaction_type&quot;)
 @JsonInclude(JsonInclude.Include.NON_NULL)
 private TransactionType transactionType;

 @Column(name=&quot;property_type&quot;)
 private PropertyType propertyType;
 
 // Constructor, getters, setters, toString(), etc.
}
</code></pre>
In this snippet - the annotation is only applied to the <code>TransactionType</code> field.
<h2 id="conclusion">Conclusion</h2>
In this guide, you've learned how to ignore null fields in Jackson, with Java and Spring Boot. This can be achieved globally by replacing the underlying implicit <code>ObjectMapper</code> instance and customizing the behavior of the new primary bean you register.
Alternatively, you can customize the <code>Jackson2ObjectMapperBuilder</code> that builds the <code>ObjectMapper</code> for your application instead of replacing the already built one. For a low-code solution - you can change the <code>spring.jackson.default-property-inclusion</code> property in your properties files.
Finally - you can use the <code>@JsonInclude</code> annotation on class-level or method-level.

In this tutorial, learn how to ignore null fields when serializing POJOs with Jackson, in Java and Spring Boot. You can do this by customizing the ObjectMapper instance, changing the default-property-inclusion property in your properties file, customizing the Jackson2ObjectMapperBuilder instance, or applying the @JsonInclude annotation.

Ignore Null Fields with Jackson in Java and Spring Boot

Java Streams have been introduced all the way back in Java 8 in 2014, in an effort to introduce verbose Java to a Functional Programming paradigm. Java Streams expose many flexible and powerful functional operations to perform collection processing in one-liners.
Filtering collections based on some predicate remains one of the most commonly used functional operations, and can be performed with a <code>Predicate</code> or more concisely - with a Lambda Expression.
<blockquote>
In this short guide, we'll take a look at how you can filter a Java 8 Stream with Lambda Expressions.
</blockquote>

 <div class="alert alert-reference">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-link-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Advice: If you aren't familiar with functional interfaces, predicates and lambda expressions - for an example-driven intuition-building guide, read our in-depth <a target="_blank" href="https://stackabuse.com/guide-to-functional-interfaces-and-lambda-expressions-in-java/">&quot;Guide to Functional Interfaces and Lambda Expressions in Java&quot;</a>!

 </div>
 </div>
 </div>
 <h2 id="filteringstreamsinjava">Filtering Streams in Java</h2>
In general, any <code>Stream</code> can be filtered via the <code>filter()</code> method, and a given predicate:
<pre><code class="hljs">Stream&lt;T&gt; filter(Predicate&lt;? super T&gt; predicate)
</code></pre>
Each element in the stream is run against the predicate, and is added to the output stream if the predicate returns <code>true</code>. You can supply a <code>Predicate</code> instance:
<pre><code class="hljs">Predicate&lt;String&gt; contains = s -&gt; s.contains(&quot;_deprecated&quot;);
List&lt;String&gt; results = stream.filter(contains).collect(Collectors.toList());
</code></pre>
Or, simplify it by providing a Lambda Expression:
<pre><code class="hljs">List&lt;String&gt; results = stream.filter(s -&gt; s.contains(&quot;_deprecated&quot;))
 .collect(Collectors.toList());
</code></pre>
Or even collapse the Lambda Expression into a method reference:
<pre><code class="hljs">// Filters empty strings by invoking s -&gt; s.isEmpty() on each element
List&lt;String&gt; results = stream.filter(String::isEmpty)
 .collect(Collectors.toList());
</code></pre>
With method references, you can't pass arguments, though, you can define methods in the object you're filtering and tailor them to be easily filterable (as long as the method doesn't accept arguments and returns a <code>boolean</code>).
Remember that streams are not collections - they're streams of collections, and you'll have to collect them back into any collection such as a <code>List</code>, <code>Map</code>, etc. to give them permanence. Additionally, all operations done on stream elements either intermediate or terminal:
<ul>
<li>Intermediate operations return a new stream with changes from the previous operation</li>
<li>Terminal operations return a data type and are meant to end a pipeline of processing on a stream</li>
</ul>
<code>filter()</code> is an intermediate operation, and is meant to be chained with other intermediate operations, before the stream is terminated. To persist any changes (such as changes to elements themselves, or filtered results), you'll have to assign the resulting output stream to a new reference variable, through a terminal operation.

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: Even when chaining many lambda expressions, you might not run into readability issues, with proper line breaks.

 </div>
 </div>
 </div>
 In the following examples, we'll be working with this list of books:
<pre><code class="hljs">Book book1 = new Book(&quot;001&quot;, &quot;Our Mathematical Universe&quot;, &quot;Max Tegmark&quot;, 432, 2014);
Book book2 = new Book(&quot;002&quot;, &quot;Life 3.0&quot;, &quot;Max Tegmark&quot;, 280, 2017);
Book book3 = new Book(&quot;003&quot;, &quot;Sapiens&quot;, &quot;Yuval Noah Harari&quot;, 443, 2011);
 
List&lt;Book&gt; books = Arrays.asList(book1, book2, book3);
</code></pre>
<h3 id="filtercollectionwithstreamfilter">Filter Collection with Stream.filter()</h3>
Let's filter this collection of books. Any predicate goes - so let's for example filter by which books have over 400 pages:
<pre><code class="hljs">List&lt;Book&gt; results = books.stream()
 .filter(b -&gt; b.getPageNumber() &gt; 400)
 .collect(Collectors.toList());
</code></pre>
This results in a list which contains:
<pre><code class="hljs">[
Book{id=&#x27;001&#x27;, name=&#x27;Our Mathematical Universe&#x27;, author=&#x27;Max Tegmark&#x27;, pageNumber=432, publishedYear=2014}, 
Book{id=&#x27;003&#x27;, name=&#x27;Sapiens&#x27;, author=&#x27;Yuval Noah Harari&#x27;, pageNumber=443, publishedYear=2011}
]
</code></pre>
When filtering, a really useful method to chain is <code>map()</code>, which lets you map objects to another value. For example, we can map each book to its name, and thus return only the names of the books that fit the predicate from the <code>filter()</code> call:
<pre><code class="hljs">List&lt;String&gt; results = books.stream()
 .filter(b -&gt; b.getPageNumber() &gt; 400)
 .map(Book::getName)
 .collect(Collectors.toList());
</code></pre>
This results in a list of strings:
<pre><code class="hljs">[Our Mathematical Universe, Sapiens]
</code></pre>
<h3 id="filtercollectiononmultiplepredicateswithstreamfilter">Filter Collection on Multiple Predicates with Stream.filter()</h3>
Commonly, we'd like to filter collections by more than one criteria. This can be done by chaining multiple <code>filter()</code> calls or using a short-circuit predicate, which checks for two conditions in a single <code>filter()</code> call.
<pre><code class="hljs"> List&lt;Book&gt; results = books.stream()
 .filter(b -&gt; b.getPageNumber() &gt; 400 &amp;&amp; b.getName().length() &gt; 10)
 .collect(Collectors.toList());
 
// Or

 List&lt;Book&gt; results2 = books.stream()
 .filter(b -&gt; b.getPageNumber() &gt; 400)
 .filter(b -&gt; b.getName().length() &gt; 10)
 .collect(Collectors.toList());
</code></pre>
When utilizing multiple criteria - the lambda calls can get somewhat lengthy. At this point, extracting them as standalone predicates might offer more clarity. Though, which approach is faster?
<h2 id="singlefilterwithcomplexconditionormultiplefilters">Single Filter with Complex Condition or Multiple Filters?</h2>
It depends on your hardware, how large your collection is, and whether you use parallel streams or not. In general - one filter with a complex condition will outperform multiple filters with simpler conditions (small-to-medium collections), or perform at the same level (very large collections). If your conditions are too long - you may benefit from distributing them over multiple <code>filter()</code> calls, for the improved readability, since performance is very similar.
The best choice is to try both, note the performance on the target device, and adjust your strategy accordingly.
GitHub user volkodavs did a filtering benchmark in throughput operations/s, and hosted the results on the <a target="_blank" href="https://github.com/volkodavs/javafilters-benchmarks">javafilters-benchmarks</a> repository. The results are summarized in an informative table:
<img src="https://s3.stackabuse.com/media/articles/java-stream-filter-with-lambda-expression-1.png" alt="">
It shows a clear diminishing of returns at larger collection sizes, with both approaches performing around the same level. Parallel streams benefit significantly at larger collection sizes, but curb the performance at smaller sizes (below ~10k elements). It's worth noting that parallel streams maintained their throughput much better than non-parallel streams, making them significantly more robust to input.

In this short guide, learn how to filter a stream with lambda expressions, method references and predicates in Java, with a single filter with multiple conditions (criteria) vs multiple filters benchmark and best practices.

Java - Filter a Stream with Lambda Expressions

<h2 id="introduction">Introduction</h2>
The <code>echo</code> command outputs a given string to the standard output pipe, typically pointing to the terminal. Although the standard output pipe can point to other interfaces - the <code>echo</code> command is typically used to print and display messages in the terminal. By default, the text of the displayed message inherits the color of other text in the terminal (which is customizable in and of itself). However - there are several ways you can alter the output color of <code>echo</code> - both for individual strings and for entire messages.
<blockquote>
In this short guide, we'll take a look at how you can change the output color of <code>echo</code> in Linux-based systems, using ANSI escape codes, <code>tput</code> and how you can make this process less verbose in Bash scripts.
</blockquote>
<h2 id="changeoutputcolorwithansiescapecodes">Change Output Color with ANSI Escape Codes</h2>
The easiest way to change the color is through ANSI escape sequences/codes. All ANSI escape codes start with the <code>Escape</code> character, which can be represented in various formats - <code>27</code> in decimal, <code>\x1B</code> in hexadecimal, as the control key <code>^[</code>, or <code>\033</code> in octal format. The sequences are then followed by the command:
<pre><code class="hljs">\033[command
</code></pre>
Where the opening bracket (Control Sequence Introducer) is optional, but helps separate the command from the escape character. When you put a color code as the command, it changes the color of the oncoming text:
<pre><code class="hljs">\033[0;34
</code></pre>
<code>0;34</code> is the code for the color blue, for example. With this alone, you can change the color of text in <code>echo</code> with:
<pre><code class="hljs">\033[0;34Text
</code></pre>
Where <code>Text</code> would be colored blue. Alternatively, consider a simple bash script to print &quot;Welcome to France&quot; in the colors of the French flag:
<pre><code class="hljs">#!/bin/bash 
BLUE=&#x27;\033[0;34m&#x27;
WHITE= &#x27;\033[0;37m&#x27; 
RED= &#x27;\033[0;31m&#x27; 
echo -e &quot;${Blue}Welcome ${WHITE}to ${RED}France&quot;
</code></pre>
The optional <code>-e</code> flag of the <code>echo</code> command allows you to use special characters like <code>\n</code> (newline) and <code>\t</code> (tab) inside of the input string.
Once you run the script:
<pre><code class="hljs">$ ./colors.sh
</code></pre>
It results in:
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-1.jpg" alt="">
The ANSI codes aren't limited to color - but can also be applied for style. The codes <code>0..9</code> represent text styles, while the codes <code>30...37</code> represent colors:
<table class="table table-striped">
 <tr>
 <td>Color</td>
 <td>Codes</td>
 <td>Text Styles</td>
 <td>Codes</td>
 </tr>
 <tr>
 <td>Black</td>
 <td>30</td>
 <td>Simple text</td>
 <td>0</td>
 </tr>
 <tr>
 <td>Red</td>
 <td>31</td>
 <td>Bold text</td>
 <td>1</td>
 </tr>
 <tr>
 <td>Green</td>
 <td>32</td>
 <td>Low intensity text</td>
 <td>2</td>
 </tr>
 <tr>
 <td>Brown/Orange</td>
 <td>33</td>
 <td>Underline text</td>
 <td>4</td>
 </tr>
 <tr>
 <td>Blue</td>
 <td>34</td>
 <td>Blinking text</td>
 <td>5</td>
 </tr>
 <tr>
 <td>Purple</td>
 <td>35</td>
 <td>Invisible text</td>
 <td>8</td>
 </tr>
 <tr>
 <td>Cyan</td>
 <td>36</td>
 <td>Strikethrough text</td>
 <td>9</td>
 </tr>
 <tr>
 <td>Light Gray</td>
 <td>37</td>
 <td></td>
 <td></td>
 </tr>
</table>
Let's create a bash script to explore some of these options:
<pre><code class="hljs">#!/bin/bash
echo -e &quot;\033[0;33mSample text&quot;
echo -e &quot;\033[1;33mBold text&quot;
echo -e &quot;\033[2;33mLow intensity text&quot;
echo -e &quot;\033[4;33mUnderline text&quot;
echo -e &quot;\033[5;33mBlinking text&quot;
echo -e &quot;\033[8;33mInvisible text&quot;
echo -e &quot;\033[9;33mStrikethrough text&quot;
</code></pre>
Running this script results in:
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-2.gif" alt="">
Similarly, you can change the background color of these texts using codes <code>40..47</code>:
<table class="table table-striped">
 <tr>
 <td>color</td>
 <td>Codes</td>
 </tr>
 <tr>
 <td>Black</td>
 <td>40</td>
 </tr>
 <tr>
 <td>Red</td>
 <td>41</td>
 </tr>
 <tr>
 <td>Green</td>
 <td>42</td>
 </tr>
 <tr>
 <td>Brown/Orange</td>
 <td>43</td>
 </tr>
 <tr>
 <td>Blue</td>
 <td>44</td>
 </tr>
 <tr>
 <td>Purple</td>
 <td>45</td>
 </tr>
 <tr>
 <td>Cyan</td>
 <td>46</td>
 </tr>
 <tr>
 <td>Light Gray</td>
 <td>47</td>
 </tr>
</table>
Changing the background of a given string boils down to the same rule as when changing the font color - the code itself changes the behavior:
<pre><code class="hljs">#!/bin/bash
BLUE=&#x27;\033[0;44m&#x27;
BLACK=&#x27;\033[0;30m&#x27;
WHITE=&#x27;\033[0;30;47m&#x27;
RED=&#x27;\033[0;41m&#x27;
echo -e &quot;${BLUE}Welcome ${WHITE}to ${RED}France&quot;
</code></pre>
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-3.jpg" alt="">

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: As a rule of thumb - you can translate font colors to background colors by adding 10. <code>30</code> is black font color, <code>40</code> is black background color.

 </div>
 </div>
 </div>
 <h2 id="changetheoutputcolorwiththetputcommand">Change the Output Color with the <code>tput</code> Command</h2>
An alternative to ANSI codes is using the <code>tput</code> command:
<pre><code class="hljs">tput setaf color_code
</code></pre>
<code>setf</code> allows for 8 colors, while <code>setaf</code> allows for 256 colors so depending on the command you're using, you can go between <code>0..7</code> and <code>0..255</code> as the color codes. Both commands dedicate <code>0..7</code> to the same color codes, while with <code>setaf</code>, <code>8..15</code> are high-intensity colors, and <code>16..231</code> are different hues of the first 8, and <code>232..255</code> are grayscale colors:
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-4.png" alt=""> 
Credit: <a target="_blank" rel="nofollow noopener noreferrer" href="https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit">Wikipedia</a>
Finally, <code>tput</code> also allows you to change the background color, add bold, lower the intensity, etc. with other commands:
<table class="table table-striped">
 <tr>
 <td>Text Styles</td>
 <td>Commands</td>
 </tr>
 <tr>
 <td>Foreground color</td>
 <td>setaf</td>
 </tr>
 <tr>
 <td>Background color</td>
 <td>setab</td>
 </tr>
 <tr>
 <td>No style</td>
 <td>sgv0</td>
 </tr>
 <tr>
 <td>Bold text</td>
 <td>bold</td>
 </tr>
 <tr>
 <td>Low-intensity text</td>
 <td>dim</td>
 </tr>
 <tr>
 <td>Underline text</td>
 <td>smul</td>
 </tr>
 <tr>
 <td>Blinking text</td>
 <td>blink</td>
 </tr>
 <tr>
 <td>Reverse text</td>
 <td>rev</td>
 </tr>
</table>
Let's create another script that uses <code>tput</code> to change the output color <code>echo</code>:
<pre><code class="hljs">#!/bin/bash
YELLOW=`tput setaf 3`
echo &quot;${YELLOW}Changing&quot;
WHITE=`tput setaf 7 bold`
echo &quot;${WHITE}Colors&quot;
BLUE=`tput setaf 4 smul`
echo &quot;${BLUE}With tput&quot;
CYAN=`tput setaf 6 blink`
echo &quot;${CYAN}is less&quot;
RED=`tput setab 1 setaf 7`
echo &quot;${RED}verbose!&quot;
</code></pre>
This script will print the following output in the terminal:
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-5.gif" alt="">
The <code>tput</code> command delivers an easy way to print any color through a simple color code. Let's now create a script that can print every color code available for the <code>tput</code> command:
<pre><code class="hljs">#!/bin/bash
tput init 
 end = $ (($ (tput colors) - 1)) 
 w = 1 
 for c
 in $ (seq 0 $end) 
 do
eval &quot;$(printf &quot; tput setaf % 3 s &quot; &quot; $c &quot;)&quot; 
 echo - n &quot;$_&quot; 
 [[$c - ge $ ((w * 2))]] 
 offset = 2 || offset = 0 
 [[$ (((c + offset) % (w - offset))) - eq $ (((w - offset) - 1))]] 
 echo 
 done 
 tput init
</code></pre>
It will print 0 to 255 color and their codes:
<img src="https://s3.stackabuse.com/media/articles/how-to-change-the-output-color-of-echo-in-linux-6.png" alt="">
<h2 id="conclusion">Conclusion</h2>
In this short guide - we've taken a look at how you can change the color of <code>echo</code>'s output in Linux - using ANSI Escape Sequences and the <code>tput</code> command.
We've also explored how you can make the process less verbose using Bash variables, tweak background colors and style text.

Guest Contributor

In this short guide, learn how to change the output color of the echo command in Linux/Unix systems, as well as how to change the text background color and apply basic formatting, using ANSI escape codes and tput.

How to Change the Output Color of Echo in Linux

<h2 id="introduction">Introduction</h2>
Object detection has been gaining steam, and improvements are being made to several approaches to solving it. In the past couple of years, YOLO-based methods have been outperforming others in terms of accuracy and speed, with recent advancements such as YOLOv7 and YOLOv6 (which was released independently, after YOLOv7).
However - all of these are concerning 2D object detection, which is a difficult task in and of itself. Recently, we've been able to successfully perform 3D object detection, and while these detectors are still at a more unstable stage than 2D object detectors, their accuracy is rising.
<blockquote>
In this guide, we'll be performing 3D object detection in Python with MediaPipe's Objectron.
</blockquote>

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: MediaPipe is Google's open source framework for building machine learning pipelines to process images, videos and audio streams, primarily for mobile devices. It's being used both internally and externally, and provides pre-trained models for various tasks, such as face detection, face meshing, hand and pose estimation, hair segmentation, object detection, box tracking, etc.

 </div>
 </div>
 </div>
 All of these can and are used for downstream tasks - such as applying filters to faces, automated camera focusing, biometric verification, hand-controlled robotics, etc. Most projects are available with APIs for Android, iOS, C++, Python and JavaScript, while some are only available for certain languages.
In this guide, we'll be working with MediaPipe's Objectron, available for Android, C++, Python and JavaScript.
<h2 id="mediapipeand3dobjectdetection">MediaPipe and 3D Object Detection</h2>
The Objectron solution was trained on the <a rel="nofollow noopener noreferrer" target="_blank" href="https://github.com/google-research-datasets/Objectron">Objectron Dataset</a>, which contains short object-centric videos. The dataset only contains 9 objects: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops and shoes, so it's not a very general dataset, but the processing and procurement of these videos is fairly expensive (camera poses, sparse point-clouds, characterization of the planar surfaces, etc. for each frame of each video), making the dataset nearly 2 terabytes in size.
The trained Objectron model (known as a solution for MediaPipe projects) is trained on four categories - shoes, chairs, mugs and cameras.
2D object detection uses the term &quot;bounding boxes&quot;, while they're actually rectangles. 3D object detection actually predicts boxes around objects, from which you can infer their orientation, size, rough volume, etc. This is a fairly difficult task to take on, especially given the lack of appropriate datasets and the cost of creating them. While difficult, the problem holds promise for various Augmented Reality (AR) applications!
The Objectron solution can run in a one-stage or two-stage mode - where the one-stage mode is better at detecting multiple objects, while the two-stage mode is better at detecting a single main object in the scene, and runs significantly faster. The single-stage pipeline uses a MobileNetV2 backbone, while the two-stage pipeline uses the TensorFlow Object Detection API.
When an object is detected in a video, further predictions aren't made for it on each frame for two reasons:
<ul>
<li>Continuous predictions introduce high jitteriness (due to the inherent stochasticity in the predictions)</li>
<li>It's expensive to run large models on every frame</li>
</ul>
The team offloads the heavy predictions to first encounters only and then tracks that box as long as the object in question is still in the scene. Once the line of sight is broken and the object is re-introduced, a prediction is made again. This makes it possible to use larger models with higher accuracy, while keeping the computational requirements low, and lowers the hardware requirements for real-time inference!
Let's go ahead and install MediaPipe, import the Objectron solution and apply it to static images and a video feed coming straight from a camera.
<h2 id="installingmediapipe">Installing MediaPipe</h2>
Let's first install MediaPipe and prepare a helper method to fetch images from a given URL:
<pre><code class="hljs">$ ! pip install mediapipe
</code></pre>
With the framework installed, let's import it alongside common libraries:
<pre><code class="hljs">import mediapipe as mp

import cv2
import numpy as np
import matplotlib.pyplot as plt
</code></pre>
Let's define a helper method to fetch images given a URL and which returns an RGB array representing that image:
<pre><code class="hljs">import PIL
import urllib

def url_to_array(url):
 req = urllib.request.urlopen(url)
 arr = np.array(bytearray(req.read()), dtype=np.int8)
 arr = cv2.imdecode(arr, -1)
 arr = cv2.cvtColor(arr, cv2.COLOR_BGR2RGB)
 return arr

mug = &#x27;https://goodstock.photos/wp-content/uploads/2018/01/Laptop-Coffee-Mug-on-Table.jpg&#x27;
mug = url_to_array(mug)
</code></pre>
Finally, we'll want to import both the Objectron solution and the drawing utilities to visualize predictions:
<pre><code class="hljs">mp_objectron = mp.solutions.objectron
mp_drawing = mp.solutions.drawing_utils
</code></pre>
<h2 id="3dobjectdetectiononstaticimageswithmediapipe">3D Object Detection on Static Images with MediaPipe</h2>
The <code>Objectron</code> class allows for several arguments, including:
<ul>
<li><code>static_image_mode</code>: Whether you're feeding in an image or a stream of images (video)</li>
<li><code>max_num_objects</code>: The maximum identifiable number of objects</li>
<li><code>min_detection_confidence</code>: The detection confidence threshold (how sure the network has to be to classify an object for the given class)</li>
<li><code>model_name</code>: Which model you'd like to load in between <code>'Cup'</code>, <code>'Shoe'</code>, <code>'Camera'</code> and <code>'Chair'</code>.</li>
</ul>
With those in mind - let's instantiate an Objectron instance and <code>process()</code> the input image:
<pre><code class="hljs"># Instantiation
objectron = mp_objectron.Objectron(
 static_image_mode=True,
 max_num_objects=5,
 min_detection_confidence=0.2,
 model_name=&#x27;Cup&#x27;)

# Inference
results = objectron.process(mug)
</code></pre>
The <code>results</code> contain the 2D and 3D landmarks of the detected object(s) as well as the rotation, translation and scale for each. We can process the results and draw the bounding boxes fairly easily using the provided drawing <code>utils</code>:
<pre><code class="hljs">if not results.detected_objects:
 print(f&#x27;No box landmarks detected.&#x27;)

# Copy image so as not to draw on the original one
annotated_image = mug.copy()
for detected_object in results.detected_objects:
 # Draw landmarks
 mp_drawing.draw_landmarks(annotated_image, 
 detected_object.landmarks_2d, 
 mp_objectron.BOX_CONNECTIONS)

 # Draw axis based on rotation and translation
 mp_drawing.draw_axis(annotated_image, 
 detected_object.rotation, 
 detected_object.translation)
 
# Plot result
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(annotated_image)
ax.axis(&#x27;off&#x27;)
plt.show()
</code></pre>
This results in:
<img src="https://s3.stackabuse.com/media/articles/3d-object-detection-3d-bounding-boxes-in-python-with-mediapipe-objectron-0.png" alt="">
<h2 id="3dobjectdetectionfromvideoorwebcamwithmediapipe">3D Object Detection from Video or Web-cam with MediaPipe</h2>
A more exciting application is on videos! You don't have to change the code much to accommodate videos, whether you're providing one from the web-cam or an existing video file. OpenCV is a natural fit for reading, manipulating and feeding video frames into the objectron model:
<pre><code class="hljs"># VideoCapture(0) tries to capture from the webcam
cap = cv2.VideoCapture(0)
# Or
# cap = cv2.VideoCapture(&#x27;filename.mp4&#x27;)

objectron = mp_objectron.Objectron(static_image_mode=False,
 max_num_objects=5,
 min_detection_confidence=0.4,
 min_tracking_confidence=0.70,
 model_name=&#x27;Cup&#x27;)
 
# Read video stream and feed into the model
while cap.isOpened():
 success, image = cap.read()

 image.flags.writeable = False
 image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
 results = objectron.process(image)

 image.flags.writeable = True
 image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
 if results.detected_objects:
 for detected_object in results.detected_objects:
 
 mp_drawing.draw_landmarks(image, 
 detected_object.landmarks_2d, 
 mp_objectron.BOX_CONNECTIONS)
 
 mp_drawing.draw_axis(image, 
 detected_object.rotation,
 detected_object.translation)

 cv2.imshow(&#x27;MediaPipe Objectron&#x27;, cv2.flip(image, 1))
 if cv2.waitKey(10) &amp; 0xFF == ord(&#x27;q&#x27;):
 break
cap.release()
cv2.destroyAllWindows()
</code></pre>
Making the image non-writable with <code>image.flags.writeable = False</code> makes the process run somewhat faster, and is an optional change. The final <code>cv2.flip()</code> on the resulting image is also optional - and simply makes the output mirrored to make it a bit more intuitive.
When run on a camera and a globally common IKEA mug, these are the results:
<img src="https://s3.stackabuse.com/media/articles/3d-object-detection-3d-bounding-boxes-in-python-with-mediapipe-objectron-1.gif" alt="">
The output is slightly jittery, but handles rotational translation well, even with a shaky hand holding the low-resolution camera. What happens when an object is taken out of the frame?
<img src="https://s3.stackabuse.com/media/articles/3d-object-detection-3d-bounding-boxes-in-python-with-mediapipe-objectron-2.gif" alt="">
The predictions stop for the object at the first detection, and box tracking clearly picks up that the object has left the frame, and performs the prediction and tracking once again as soon as the object re-enters the frame. It appears that the tracking works somewhat better when the model can see the mug handle, as the outputs are more jittery when the handle is not visible (presumably because it's harder to accurately ascertain the true orientation of the mug).
Additionally, some angles seem to produce significantly more stable outputs than others, in challenging light conditions. For mugs specifically, it helps to be able to see the lip of the mug as it helps with perspective, rather than seeing an orthogonal projection of the object.
Additionally, when tested on a transparent mug, the model had difficulties ascertaining it as a mug. This is likely an example of an out of distribution object, as most mugs are opaque and have various colors.
<h2 id="conclusion">Conclusion</h2>
3D object detection is still somewhat young, and MediaPipe's Objectron is a capable demonstration! While sensitive to lighting conditions, object types (transparent vs opaque mugs, etc.) and slightly jittery - Objectron is a good glimpse into what will soon be possible to do with higher accuracy and accessibility than ever before.

In this practical guide, learn how to perform 3D object detection (regress 3D bounding boxes) around objects in real-time with Python, OpenCV and MediaPipe, built on top of TensorFlow Object Detection.

3D Object Detection (3D Bounding Boxes) in Python with MediaPipe Objectron

<h2 id="introduction">Introduction</h2>
Object detection is a large field in computer vision, and one of the more important applications of computer vision &quot;in the wild&quot;. From it, keypoint detection (oftentimes used for pose estimation) was extracted.
Keypoints can be various points - parts of a face, limbs of a body, etc. Pose estimation is a special case of keypoint detection - in which the points are parts of a human body.
Pose estimation is an amazing, extremely fun and practical usage of computer vision. With it, we can do away with hardware used to estimate poses (motion capture suits), which are costly and unwieldy. Additionally, we can map the movement of humans to the movement of robots in Euclidean space, enabling fine precision motor movement without using controllers, which usually don't allow for higher levels of precision. Keypoint estimation can be used to translate our movements to 3D models in AR and VR, and increasingly is being used to do so with just a web cam. Finally - pose estimation can help us in sports and security.
<blockquote>
In this guide, we'll be performing real-time pose estimation from a video in Python, using the state-of-the-art YOLOv7 model.
</blockquote>
Specifically, we'll be working with a video from the 2018 winter Olympics, held in South Korea's PyeongChang:
<img src="https://s3.stackabuse.com/media/articles/real-time-pose-estimation-from-video-in-python-with-yolov7.gif" alt="">
Aljona Savchenko and Bruno Massot did an amazing performance, including overlapping bodies against the camera, fast fluid movement and spinning in the air. It'll be an amazing opportunity to see how the model handles difficult-to-infer situations!
<h2 id="yoloandposeestimation">YOLO and Pose Estimation</h2>
YOLO (You Only Look Once) is a methodology, as well as a family of models built for object detection. Since the inception in 2015, YOLOv1, YOLOv2 (YOLO9000) and YOLOv3 have been proposed by the same author(s) - and the deep learning community continued with open-sourced advancements in the continuing years.
<a target="_blank" rel="nofollow noopener noreferrer" href="https://github.com/ultralytics/yolov5">Ultralytics' YOLOv5</a> is an industry-grade object detection repository, built on top of the YOLO method. It's implemented in PyTorch, as opposed to C++ for previous YOLO models, is fully open source, and has a beautifully simple and powerful API that lets you infer, train and customize the project flexibly. It's such a staple that most new attempts at improving the YOLO method build on top of it.
This is how <a rel="nofollow noopener noreferrer" target="_blank" href="https://arxiv.org/abs/2105.04206">YOLOR (You Only Learn One Representation)</a> and YOLOv7 which built on top of YOLOR (same author) were created as well!
YOLOv7 isn't just an object detection architecture - it provides new model heads that can output keypoints (skeletons) and perform instance segmentation besides only bounding box regression, which wasn't standard with previous YOLO models. This isn't surprising, since many object detection architectures were re-purposed for instance segmentation and keypoint detection tasks earlier as well, due to the shared general architecture, with different outputs depending on the task.

 <div class="alert alert-reference">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-link-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Advice: If you're interested in reading more about instance segmentation, read our <a target="_blank" href="https://stackabuse.com/instance-segmentation-with-yolov7-in-python/">&quot;Instance Segmentation with YOLOv7 in Python&quot;</a>!

 </div>
 </div>
 </div>
 Even though it isn't surprising - supporting instance segmentation and keypoint detection will likely become the new standard for YOLO-based models, which have begun outperforming practically all other two-stage detectors a couple of years ago in terms of both accuracy and speed.
<blockquote>
This makes instance segmentation and keypoint detection faster to perform than ever before, with a simpler architecture than two-stage detectors.
</blockquote>

 <div class="alert alert-reference">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-link-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 YOLOv7 was released alongside a paper named <a rel="nofollow noopener noreferrer" target="_blank" href="https://arxiv.org/abs/2207.02696">&quot;YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors&quot;</a>, and the source code is available <a rel="nofollow noopener noreferrer" target="_blank" href="https://github.com/WongKinYiu/yolov7">on GitHub</a>.

 </div>
 </div>
 </div>
 The model itself was created through architectural changes, as well as optimizing aspects of training, dubbed &quot;bag-of-freebies&quot;, which increased accuracy without increasing inference cost.
<h2 id="installingyolov7">Installing YOLOv7</h2>
Let's start by cloning the repository to get hold of the source code:
<pre><code class="hljs">! git clone https://github.com/WongKinYiu/yolov7.git
</code></pre>
Now, let's move into the <code>yolov7</code> directory, which contains the project, and take a look at the contents:
<pre><code class="hljs">%cd yolov7
!ls
</code></pre>
<pre><code class="hljs">/content/yolov7
cfg	 figure output.mp4	 test.py 
data	 hubconf.py paper		 tools
deploy	 inference README.md	 train_aux.py
detect.py LICENSE.md requirements.txt train.py
export.py models scripts		 utils
</code></pre>

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: Calling <code>!cd dirname</code> moves you into a directory in that cell. Calling <code>%cd dirname</code> moves you into a directory across the upcoming cells as well and keeps you there.

 </div>
 </div>
 </div>
 Now, YOLO is meant to be an object detector, and doesn't ship with pose estimation weights by default. We'll want to download the weights and load a concrete model instance from them. The weights are available on the same GitHub repository, and can easily be downloaded through the CLI as well:
<pre><code class="hljs">! curl -L https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-w6-pose.pt -o yolov7-w6-pose.pt

 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 153M 100 153M 0 0 23.4M 0 0:00:06 0:00:06 --:--:-- 32.3M
</code></pre>
Once downloaded, we can import the libraries and helper methods we'll be using:
<pre><code class="hljs">import torch
from torchvision import transforms

from utils.datasets import letterbox
from utils.general import non_max_suppression_kpt
from utils.plots import output_to_keypoint, plot_skeleton_kpts

import matplotlib.pyplot as plt
import cv2
import numpy as np
</code></pre>
Great! Let's get on with loading the model and creating a script that lets you infer poses from videos with YOLOv7 and OpenCV.
<h2 id="realtimeposeestimationwithyolov7">Real-Time Pose Estimation with YOLOv7</h2>
Let's first create a method to load the model from the downloaded weights. We'll check what device we have available (CPU or GPU):
<pre><code class="hljs">device = torch.device(&quot;cuda:0&quot; if torch.cuda.is_available() else &quot;cpu&quot;)

def load_model():
 model = torch.load(&#x27;yolov7-w6-pose.pt&#x27;, map_location=device)[&#x27;model&#x27;]
 # Put in inference mode
 model.float().eval()

 if torch.cuda.is_available():
 # half() turns predictions into float16 tensors
 # which significantly lowers inference time
 model.half().to(device)
 return model

model = load_model()
</code></pre>
Depending on whether we have a GPU or not, we'll turn half-precision on (using <code>float16</code> instead of <code>float32</code> in operations), which makes inference significantly faster. Note that it's highly encouraged to perform this on a GPU for real-time speeds, as CPUs will likely lack the power to do so unless running on small videos.
Let's write a convenient method for running inference. We'll accept images as NumPy arrays (as that's what we'll be passing them later while reading the video). First, using the <code>letterbox()</code> function - we'll resize and pad the video to a shape that the model can work with. This doesn't need to be and won't be the shape (resolution) of the resulting video!
Then, we'll apply the transforms, convert the image to half precision (if a GPU is available), batch it and run it through the model:
<pre><code class="hljs">def run_inference(image):
 # Resize and pad image
 image = letterbox(image, 960, stride=64, auto=True)[0] # shape: (567, 960, 3)
 # Apply transforms
 image = transforms.ToTensor()(image) # torch.Size([3, 567, 960])
 if torch.cuda.is_available():
 image = image.half().to(device)
 # Turn image into batch
 image = image.unsqueeze(0) # torch.Size([1, 3, 567, 960])
 with torch.no_grad():
 output, _ = model(image)
 return output, image
</code></pre>
We'll return the predictions of the model, as well as the image as a tensor. These are &quot;rough&quot; predictions - they contain many activations that overlap, and we'll want to &quot;clean them up&quot; using Non-Max Suppression, and plot the predicted skeletons over the image itself:
<pre><code class="hljs">def draw_keypoints(output, image):
 output = non_max_suppression_kpt(output, 
 0.25, # Confidence Threshold
 0.65, # IoU Threshold
 nc=model.yaml[&#x27;nc&#x27;], # Number of Classes
 nkpt=model.yaml[&#x27;nkpt&#x27;], # Number of Keypoints
 kpt_label=True)
 with torch.no_grad():
 output = output_to_keypoint(output)
 nimg = image[0].permute(1, 2, 0) * 255
 nimg = nimg.cpu().numpy().astype(np.uint8)
 nimg = cv2.cvtColor(nimg, cv2.COLOR_RGB2BGR)
 for idx in range(output.shape[0]):
 plot_skeleton_kpts(nimg, output[idx, 7:].T, 3)

 return nimg
</code></pre>
With these in place, our general flow will look like:
<pre><code class="hljs">img = read_img()
outputs, img = run_inference(img)
keypoint_img = draw_keypoints(output, img)
</code></pre>
To translate that to a real-time video setting - we'll use OpenCV to read a video, and run this process for every frame. On each frame, we'll also write the frame into a new file, encoded as a video. This will necessarily slow down the process as we're running the inference, displaying it and writing - so you can speed up the inference and display by avoiding the creation of a new file and writing to it in the loop:
<pre><code class="hljs">def pose_estimation_video(filename):
 cap = cv2.VideoCapture(filename)
 # VideoWriter for saving the video
 fourcc = cv2.VideoWriter_fourcc(*&#x27;MP4V&#x27;)
 out = cv2.VideoWriter(&#x27;ice_skating_output.mp4&#x27;, fourcc, 30.0, (int(cap.get(3)), int(cap.get(4))))
 while cap.isOpened():
 (ret, frame) = cap.read()
 if ret == True:
 frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
 output, frame = run_inference(frame)
 frame = draw_keypoints(output, frame)
 frame = cv2.resize(frame, (int(cap.get(3)), int(cap.get(4))))
 out.write(frame)
 cv2.imshow(&#x27;Pose estimation&#x27;, frame)
 else:
 break

 if cv2.waitKey(10) &amp; 0xFF == ord(&#x27;q&#x27;):
 break

 cap.release()
 out.release()
 cv2.destroyAllWindows()
</code></pre>
The <code>VideoWriter</code> accepts several parameters - the output filename, the FourCC (four codec codes, denoting the codec used to encode the video), the frame rate and the resolution as a tuple. To not guess or resize the video - we've used the width and height of the original video, obtained through the <code>VideoCapture</code> instance that contains data about the video itself, such as the width, height, total number of frames, etc.
Now, we can call the method on any input video:
<pre><code class="hljs">pose_estimation_video(&#x27;../ice_skating.mp4&#x27;)
</code></pre>
This will open up an OpenCV window, displaying the inference in real-time. And also, it'll write a video file in the <code>yolov7</code> directory (since we've <code>cd</code>'d into it):
<img src="https://s3.stackabuse.com/media/articles/real-time-pose-estimation-from-video-in-python-with-yolov7.gif" alt="">

 <div class="alert alert-note">
 <div class="flex">
 
 <div class="flex-shrink-0 mr-3">
 <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
 </div>
 
 <div class="w-full">
 Note: If your GPU is struggling, or if you want to embed the results of a model like this into an application that has latency as a crucial aspect of the workflow - make the video smaller and work on smaller frames. This is a full HD 1920x1080 video, and should be able to run fast on most home systems, but if it doesn't work as well on your system, make the image(s) smaller.

 </div>
 </div>
 </div>
 <h2 id="conclusion">Conclusion</h2>
In this guide, we've taken a look at the YOLO method, YOLOv7 and the relationship between YOLO and object detection, pose estimation and instance segmentation. We've then taken a look at how you can easily install and work with YOLOv7 using the programmatic API, and created several convenience methods to make inference and displaying results easier.
Finally, we've opened a video using OpenCV, ran inference with YOLOv7, and made a function for performing pose estimation in real-time, saving the resulting video in full resolution and 30FPS on your local disk.

In this guide, learn how to perform real-time pose estimation (keypoint detection) with state-of-the-art YOLOv7 and OpenCV, in Python, with practical code following good practices.

Real-Time Pose Estimation from Video in Python with YOLOv7

During textual processing, whether you're searching for certain words and making pattern matching rules, counting the frequency of elements, etc. - punctuation can throw a wrench in your plans.
Oftentimes, you'll want to remove stopwords, punctuation, digits or otherwise some category of characters, depending on what your end goal is.
<blockquote>
In this short tutorial, we'll take a look at how to remove punctuation from a string in Java.
</blockquote>
<h2 id="removepunctuationfromstringwithregexregularexpressions">Remove Punctuation from String with RegEx (Regular Expressions)</h2>
Regular Expressions are a very natural fit here, both because they're likely going to be part of other processing parts, and because they're efficient pattern matchers! In Java, the regular expression for matching punctuation is <code>\p{Punct}</code> or a shorthand <code>\p{P}</code>.
You'll have to escape the first backslash in a string, so removing all punctuation is equivalent to matching them and replacing with an empty character:
<pre><code class="hljs">String.replaceAll(&quot;\\p{P}&quot;, &quot;&quot;)
</code></pre>
Let's apply it to a simple sentence:
<pre><code class="hljs">String text = &quot;Hi! This is, in effect, a synthetic sentence. It&#x27;s meant to have several punctuation characters!&quot;;
String clean = text.replaceAll(&quot;\\p{P}&quot;, &quot;&quot;);
System.out.println(clean);
</code></pre>
This results in:
<pre><code class="hljs">Hi This is in effect a synthetic sentence Its meant to have several punctuation characters
</code></pre>
Let's take a look at what characters are treated as punctuation here:
<pre><code class="hljs">String text = &quot;!#$%&amp;&#x27;()*+,-./:;&lt;=&gt;?@[]^_`{|}~&quot;;
String clean = text.replaceAll(&quot;\\p{P}&quot;, &quot;&quot;);
System.out.println(clean);
</code></pre>
With these special characters - which are left after removing punctuation?
<pre><code class="hljs">$+&lt;=&gt;^`|~
</code></pre>
<h2 id="removepunctuationfromstringwithoutregex">Remove Punctuation from String without RegEx</h2>
If you don't want to employ regular expressions, you can do a manual check while iterating through each character of a string. Remember to use a <code>StringBuffer</code> instead of a <code>String</code> while doing this, since strings are immutable and copies need to be made every time you want to add a character - so you'd be creating <code>string.length</code> number of strings in memory.
<code>StringBuffer</code> is mutable, and can be easily converted into an immutable string at the end of the process:
<pre><code class="hljs">public static String removePunctuations(String s) {
 StringBuffer buffer = new StringBuffer();
 for (Character c : s.toCharArray()) {
 if(Character.isLetterOrDigit(c))
 buffer.append(c);
 }
 return buffer.toString();
}
</code></pre>
Let's create a string and clean it:
<pre><code class="hljs">String text = &quot;Hello! \nHere are some special characters: !#$%&amp;&#x27;()*+,-./:;&lt;=&gt;?@[]^_`{|}~ \nWhere are they? :(\n&quot;;
System.out.println(text);
String clean = removePunctuations(text);
System.out.println(clean);
</code></pre>
<pre><code class="hljs">Hello! 
Here are some special characters: !#$%&amp;&#x27;()*+,-./:;&lt;=&gt;?@[]^_`{|}~ 
Where are they? :(

HelloHerearesomespecialcharactersWherearethey
</code></pre>
While this process is more customizable, it only checks for letters and digits. You can check manually for character codes alternatively, and only exclude some punctuation characters instead - and leave in whitespaces, line breaks, etc.
<h2 id="conclusion">Conclusion</h2>
In this short tutorial, we took a look at how you can remove punctuation or certain special characters from a string in Java, using regular expressions or a manual check in an enhanced <code>for</code> loop.

In this short tutorial, learn how to remove punctuation characters and all non-digit/non-characters from a String in Java, using Regular Expressions or an enhanced for loop.

Remove Punctuation From a String in Java

In this practical guide, learn how to perform easy but powerful and fast instance segmentation and object detection in Python with YOLOv7 and Detectron2.