Project Jupyter Notebook

<div class="alert alert-note">
                <div class="flex">
                    
                        <div class="flex-shrink-0 mr-3">
                            <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
                        </div>
                        
            <p><strong>Note:</strong> This Guided Project is part of our <a target="_blank" href="https://stackabuse.com/courses/practical-deep-learning-for-computer-vision-with-python/"><em>Practical Deep Learning for Computer Vision with Python</em> course</a>, and is meant to solidify the knowledge from the previous lesson - <em>Guide to Convolutional Neural Networks</em> in which we cover the theory and history of CNNs.</p>

                </div>
            </div>
            <p>Time to put the theory into practice! If it didn't all fit into place already, there's a good chance it will now that you can build and see the results. If not - don't worry! Once the practical application is finished, try revisiting the initial explanations in the lesson. Many people have an &quot;a-ha&quot; moment after practicing with CNNs an then re-reading the introductory parts.</p>

            <div class="alert alert-note">
                <div class="flex">
                    
                        <div class="flex-shrink-0 mr-3">
                            <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
                        </div>
                        
            <p><strong>Note:</strong> With high-level APIs such as Keras that do the heavy lifting, it's easy to forget how things work under the hood, and it's worth revisiting them in the initial phases of learning (as well as some time down the line). If you haven't had any exposition to some of the terminology used here, it might take you a bit of time to get things to click. It's easy to conect the dots looking backwards, but not so much looking forwards. This is how discoveries are made!</p>

                </div>
            </div>

David Landup

Introduction - Building a Convolutional Neural Network with Keras

<h4 id="modeldefinition">Model Definition</h4>
<p>So - we've been acquainted with the building blocks of CNNs in the previous lesson. How do we put them into good use and how do you choose which layers to put where? There are some conventions to follow, though, all of them can be tweaked or changed depending on your specific dataset, and the best way to find a good architecture for your own model is to experiment.</p>
<p>You'll also find conflicting information, such as, whether the filter size should be small or large in the beginning. Some argue that filter sizes should start out large to prune away the unimportant data and reduce computational costs, while the images are still on the larger end, making the models focus on the more salient features from the get-go. Some argue that smaller filter sizes should be used to capture as much local information as possible and then make them larger to capture more global features (following the idea of the visual cortex hierarchy). Some keep the filter sizes the same throughout the entire architecture!</p>
<blockquote>
<p>So, what's the right approach when it comes to filter sizes? Generally, I've personally found that starting out with smaller filter sizes and keeping them the same size after pooling works well, since the same filter size is then proportionally bigger in the next convolutional block, capturing more global patterns. Though, the best way to find filter sizes is to experiment.</p>
</blockquote>

Defining a Convolutional Neural Network and Training

<p>Evaluating models can be streamlined through a couple of simple methods that yield stats that you can reference later. If you've ever read a research paper - you've heard of a model's accuracy, weighted accuracy, recall (sensitivity), specificity, or precision. Keep in mind how misleading these numbers can be. In a later project, when we learn to classify Breast Cancer as malignant or benign, we'll deal with all of these metrics extensively, realizing how illusory they can be even <em>together</em>, not just as individual metrics.</p>
<p>While this is a handy way of assessing the promise of a model - just getting a classification report isn't enough. It's a great start though!</p>
<h4 id="learningcurves">Learning Curves</h4>
<p>We'll want to visualize the progress of learning through time first. It helps to see the road we took to the goal, not just the goal:</p>
<pre><code class="hljs"><span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd

model_history = pd.DataFrame(history_basek.history)
model_history[<span class="hljs-string">&#x27;epoch&#x27;</span>] = history_basek.epoch

fig, ax = plt.subplots(<span class="hljs-number">2</span>, figsize=(<span class="hljs-number">14</span>,<span class="hljs-number">8</span>))
num_epochs = model_history.shape[<span class="hljs-number">0</span>]

ax[<span class="hljs-number">0</span>].plot(np.arange(<span class="hljs-number">0</span>, num_epochs), model_history[<span class="hljs-string">&quot;accuracy&quot;</span>], label=<span class="hljs-string">&quot;Training Accuracy&quot;</span>, lw=<span class="hljs-number">3</span>)
ax[<span class="hljs-number">0</span>].plot(np.arange(<span class="hljs-number">0</span>, num_epochs), model_history[<span class="hljs-string">&quot;val_accuracy&quot;</span>], label=<span class="hljs-string">&quot;Validation Accuracy&quot;</span>, lw=<span class="hljs-number">3</span>)

ax[<span class="hljs-number">1</span>].plot(np.arange(<span class="hljs-number">0</span>, num_epochs), model_history[<span class="hljs-string">&quot;loss&quot;</span>], label=<span class="hljs-string">&quot;Training Loss&quot;</span>, lw=<span class="hljs-number">3</span>)
ax[<span class="hljs-number">1</span>].plot(np.arange(<span class="hljs-number">0</span>, num_epochs), model_history[<span class="hljs-string">&quot;val_loss&quot;</span>], label=<span class="hljs-string">&quot;Validation Loss&quot;</span>, lw=<span class="hljs-number">3</span>)

ax[<span class="hljs-number">0</span>].legend()
ax[<span class="hljs-number">1</span>].legend()

plt.tight_layout()
plt.show()
</code></pre>

Evaluating a CNN Model - The Basics

<p>There's much more to evaluating a model over metric evaluation and predicting a batch and checking manually. These are the numbers you'd put out to signify how great your model is, if you were to write a publication - however, it's still a black box system in which we have no clue as to why the street before was classified as a building and vice versa. We do know that there's an overlap, but what has the model learned to make it misclassify in a relatively obvious image like the one above?</p>

            <div class="alert alert-note">
                <div class="flex">
                    
                        <div class="flex-shrink-0 mr-3">
                            <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
                        </div>
                        
            <p><strong>Note:</strong> Before going further, let's unshuffle the test set again. I know - it's tedious, and I wish there were a shorter way to do this, but there isn't. Some of the visualizations down the line we'll make are affected by the order of data.</p>

                </div>
            </div>
            <pre><code class="hljs">test_generator = test_datagen.flow_from_directory(config[<span class="hljs-string">&#x27;TEST_PATH&#x27;</span>],
                                                 target_size=(<span class="hljs-number">150</span>,<span class="hljs-number">150</span>),
                                                 batch_size=<span class="hljs-number">32</span>,
                                                 shuffle=<span class="hljs-literal">False</span>,
                                                 class_mode=<span class="hljs-string">&#x27;categorical&#x27;</span>,
                                                 seed=<span class="hljs-number">2</span>)

y_preds = model.predict(test_generator)
</code></pre>
<h4 id="identifyingwrongpredictions">Identifying Wrong Predictions</h4>
<p>Let's start out by identifying the wrong predictions. The <code>test_generator.classes</code> property is a NumPy array of the classes - one class for each instance. It's the length of our test set (3000) and can be used to directly compare against the most confident predictions in our <code>y_preds</code>:</p>

Evaluating a CNN Model Like a Pro

<p>This concludes the first project in the course! It was a ride. To recap, here are some of the prominent concepts we've covered:</p>
<ul>
<li>Co-occurrence and the source of co-occurrence bias in datasets</li>
<li>Finding, downloading datasets, and extracting data</li>
<li>Visualizing subsets of images</li>
<li>Data loading and preprocessing</li>
<li>Promises and perils of Data Augmentation and Keras' <code>ImageDataGenerator</code> class</li>
<li>Defining a custom CNN architecture</li>
<li>Implementing LRFinder with Keras and finding learning rates automatically</li>
<li>The concept of Cyclical Learning Rates</li>
<li>Evaluating a model's classification abilities</li>
<li>Interpreting a model's predictions evaluating errors</li>
<li>What makes the network predict wrong</li>
<li>Interpreting a model's attention maps to identify what models actually learn with <code>tf-keras-vis</code> and <em>GradCam++</em></li>
<li>Interpreting what the model's convolutional layers have learned through Principal Component Analysis and t-SNE</li>
</ul>
<p>That concludes this Guided Project - <em>&quot;Building Your First Convolutional Neural Network With Keras&quot;</em>. Thank you for taking a ride with us!</p>
<blockquote>
<p>Online education is spreading through the world, and is becoming an increasingly important part of many lives. We believe that accessible, high-quality resources can help empower people that build tomorrow, and remain guided by that goal.</p>
</blockquote>
<p>At StackAbuse, we believe that learning is not a one-stop time investment. It's <em><strong>life-long</strong></em>. Especially in the volatile and rapidly changing world of Computer Science and Software Engineering. So, we've pledged to update our courses, guides, and other upcoming material to keep the pace of progress in the field. Software is updating - it's only fitting that learning resources are updating as well.</p>

Thank You for Supporting Online Education

<h3 id="theintelimageclassificationdatasetimportingandexploration">The Intel Image Classification Dataset - Importing and Exploration</h3>
<p>Let's try working with the <a rel="nofollow noopener noreferrer" target="_blank" href="https://www.kaggle.com/datasets/puneet6060/intel-image-classification"><em>Intel Image Classification</em></a> dataset. It's a great dataset to go further from, since it's not super easy to get a high accuracy from the get-go. Additionally, there are some features that are easy to mix up for a network, which will serve as a great introduction into <em>model evaluation</em> and how you can learn about what makes it trip up and misclassify an image.</p>
<p>The dataset consists of 14k images for training and 3k for testing, sized at 150x150, with 6 classes: &quot;buildings&quot;, &quot;forest&quot;, &quot;glacier&quot;, &quot;mountain&quot;, &quot;sea&quot; and &quot;street&quot;. As you could assume from the classes, it mainly consists of images of natural scenes (as much as buildings can be natural).</p>
<blockquote>
<p>It may sound like this should be a walk in the park, like the Dogs vs. Cats classification is. However, consider the classes again. A mountain oftentimes contains a forest on it. A glacier is in the sea and there are buildings all around on streets! These classes, although, seemingly totally different, are fairly intertwined.</p>
</blockquote>

            <div class="alert alert-note">
                <div class="flex">
                    
                        <div class="flex-shrink-0 mr-3">
                            <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
                        </div>
                        
            <p><strong>Note:</strong> This phenomena is known as <em>co-occurrence</em>. Some things are generally present with other things in images. A network might learn a <em>co-occurring feature</em> as <em>part of a class</em>, whereas, it might not be. Learning a co-occurring feature and assuming a class is similar to mixing up <strong>correlation</strong> with <strong>causation</strong>, and it's a major fallacy that results in a bias that can easily go unnoticed because of its very nature. Correctly identifying and evaluating wrong predictions as well as visualizing what networks learn is a great way to spot and remove this bias.</p></div></div>

Importing and Data Exploration

<h4 id="youmayknowhowtobuildacnnbutdoyouunderstandit">You may know how to build a CNN - but do you <em>understand</em> it?</h4>
<blockquote>
<p>Most resources start with pristine datasets, start at importing and finish at validation. There's much more to know.</p>
</blockquote>
<p><em>Why was a class predicted? Where was the model wrong? What made it predict wrong? What was the model's attention on? What do the learned features look like? Can we separate them into distinct classes? How close are the features that make Class N to the feature that make Class M?</em></p>
<p>Here's the latent feature space of your model visualized, and otherwise hidden away from you:</p>
<img width="50%" src="https://s3.stackabuse.com/media/guided+projects/building-your-first-convolutional-neural-network-with-keras-18.png">
<p>You're new to Deep Learning for Computer Vision and have gone through the theory and basics or you feel unsure about how to build Convolutional Neural Networks?</p>
<p>Literature is vast, and either it's too long and theoretical or too brief to be practical. <strong>In this Guided Project</strong> - we'll go through the process of building your own CNN using Keras, assuming you're familiar with the fundamentals.</p>
<p>In this project, through a practical, hand-held approach, you'll learn about:</p>
<ul>
<li>Co-occurrence and the source of co-occurrence bias in datasets</li>
<li>Finding, downloading datasets, and extracting data</li>
<li>Visualizing subsets of images</li>
<li>Data loading and preprocessing</li>
<li>Promises and perils of Data Augmentation and Keras' ImageDataGenerator class</li>
<li>Defining a custom CNN architecture</li>
<li>Implementing LRFinder with Keras and finding learning rates automatically</li>
<li>Evaluating a model's classification abilities</li>
<li>Interpreting a model's predictions and evaluating errors</li>
<li>What makes the network predict wrong</li>
<li>Interpreting a model's attention maps to identify what models actually learn with tf-keras-vis and GradCam++</li>
<li>Interpreting what the model's convolutional layers have learned through Principal Component Analysis and t-SNE</li>
<li>How similarity search engines find similar images</li>
</ul>

            <div class="alert alert-note">
                <div class="flex">
                    
                        <div class="flex-shrink-0 mr-3">
                            <img src="/assets/images/icon-information-circle-solid.svg" class="icon" aria-hidden="true" />
                        </div>
                        
            <p><strong>Note:</strong> This Guided Project is part of our in-depth course on <a target="_blank" href="https://stackabuse.com/courses/practical-deep-learning-for-computer-vision-with-python/"><em>Practical Deep Learning for Computer Vision</em></a> and assumes that you've read the previous lessons or have that prerequisite knowledge from before.</p>

                </div>
            </div>
            