{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## The Structure of Convolutional Neural Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Dense Layers\n", "\n", "A *densely-connected* layer is the easiest type of neural network layer to understand. In a densely-connected layer, each unit receives inputs from all of the units in the previous layer, and the weighted connections into one unit are independent of the connections into the other units in the layer. Here is a picture of a Dense layer with 5 units (the top row), each of which receives inputs from all 7 units in the previous layer (the bottom row):\n", "\n", "
\n", "\n", "
\n", "\n", "For each non-input unit in a network, you can think of the connections into that unit as representing the unit's \"receptive field\". For example, the receptive fields of three different units are highlighted below in red. As you can see, each receptive field covers the entire 7-unit \"input space\" of the previous layer.\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "The weighted connections into different units are completely independent of each other. During training, the backpropagation algorithm updates the weights of one unit's receptive field without regard to the other units' receptive fields. This means that each unit in a layer is free to learn to respond to input patterns in its own unique way. In other words, each connection in a densely-connected layer is an independent parameter of the network.\n", "\n", "If we remember that each unit also maintains its own independent *bias* value, then we can easily calculate the total number of parameters in the network that are updated by backpropagation. For example, for the densely-connected layer of 5 units shown above, with each unit receiving inputs from all 7 units in the previous layer, we have a total of\n", "\n", "5 $\\times$ 7 weights + 5 biases = 40 total trainable parameters\n", "\n", "which we can verify by building a network containing a Dense output layer of 5 units that receives connections from an input layer of 7 units." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import keras\n", "from keras.models import Sequential\n", "from keras.layers import Input, Dense" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 1a')\n", "network.add(Input(shape=(7,)))\n", "network.add(Dense(5, name='output'))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that although we explicitly created an `Input` layer of 7 units in the above network definition, this layer does not appear in the network summary, since input layers do not contain any trainable parameters themselves. The input layer's `shape` is only needed to determine the number of parameters in the next layer (here, the `Dense` output layer)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Self-Check 1\n", "\n", "Suppose we have a densely-connected feedforward network with an input layer of 12 units, two hidden layers of 8 units and 5 units, and an output layer of 3 units. How many total trainable parameters does this network have? (Don't forget that each hidden and output unit has its own bias value, which counts as a trainable parameter.)\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Self-Check 1')\n", "network.add(Input(shape=(12,)))\n", "network.add(Dense(8, name='hidden1'))\n", "network.add(Dense(5, name='hidden2'))\n", "network.add(Dense(3, name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "hidden1_layer_params = (12 * 8) + 8\n", "hidden2_layer_params = (8 * 5) + 5\n", "output_layer_params = (5 * 3) + 3\n", "total_params = hidden1_layer_params + hidden2_layer_params + output_layer_params\n", "print(total_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Topology Doesn't Matter\n", "\n", "In a standard feedforward network with densely-connected layers, the precise order of the connections into a unit doesn't matter. This is easy to see. Consider the unlabeled output unit below, which has an input connection from every unit in the previous layer. In this example, the input activation values are labeled **a**, **b**, **c**, etc., and the connection weights are the numbers shown in red (the connection weights are usually floating-point values rather than simple integers, but I'll use integers here for purposes of illustration). \n", "\n", "
\n", "\n", "The output unit multiplies each connection weight by the corresponding input activation, adds everything up, including the unit's bias value (not shown), and possibly applies an activation function to the result. The key thing to notice is that adding up the terms **a** $\\!\\cdot\\!$ 1 + **b** $\\!\\cdot\\!$ 2 + **c** $\\!\\cdot\\!$ 3 + $\\ldots$ + **g** $\\!\\cdot\\!$ 7 can be done in any order. Mathematically, you'll get the same result. So we could re-order the input units and their connections however we like, and the output unit will still compute the same value. For example:\n", "\n", "
\n", "\n", "Notice that in the new ordering of the input layer, which we might think of as its *topology*, the input units **a** and **f** are right next to each other, whereas originally they were far apart. But this makes no difference from the output unit's point of view, for the simple reason that addition is commutative." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Real-World Topology and Local Receptive Fields\n", "\n", "What if the input layer represents real-world perceptual data, such as pixel values in an image, or sound intensities in an audio waveform, or sensor readings taken from a robot? In this type of data, topology matters. For example, two adjacent pixel values in an input image usually correspond to two points physically close to each other in space. For audio data, adjacent input values usually correspond to sound intensities measured at two consecutive moments in time. A robot with a ring of sonar sensors might return a vector of values measuring the distances to objects at different angles from the robot, and adjacent values would correspond to readings taken in nearby spatial directions. In each of these cases, arbitrarily re-ordering the input values would cause important real-world topological information to be lost. Previously-adjacent values (pixels, sonar readings, etc.) would now be far apart in the input pattern. Of course, none of this matters from the output unit's point of view — assuming that the corresponding connection weights are also re-ordered in the same way — for the reasons outlined above. The output unit would still compute exactly the same value.\n", "\n", "
\n", "
\n", "Densely-connected layers are not sensitive to topological information that may be implicitly encoded in the ordering of values in the input data.\n", "
\n", "
\n", "\n", "In contrast, **convolutional layers** are sensitive to the topological structure of the input data. Instead of being densely-connected, each unit in a convolutional layer has a *limited* (or *local*) receptive field, that receives input from only *some* of the units in the previous layer, instead of from all of them. For example, the units in the output layer shown below have receptive fields of size 3: \n", "\n", "
\n", " \n", "Each unit receives input from a different region (of size 3) of the input layer:\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Translation Invariance and Weight Sharing\n", "\n", "You can think of a local receptive field as just a set of weighted connections into a unit, along with the unit's bias value. A particular combination of weight and bias values makes the unit sensitive to certain types of patterns or **features** that occur in the region of the input associated with the receptive field. (An example of such a feature might be an \"edge\" formed by adjacent regions of light and dark pixels in an image.) If the weights and bias values of the three receptive fields shown in the picture above are independent of each other, then each of the output units will be sensitive to different features that occur in different regions of the input pattern. For example, based on its own set of receptive-field weights (and bias), the leftmost output unit above might respond strongly to one type of feature that it detects in the leftmost three input units, while the middle output unit, using a different set of weights, might respond to a very different type of feature occurring in the middle three input units.\n", "\n", "However, it usually doesn't matter *where* exactly in the input pattern a particular feature occurs. We would like our network to learn to recognize features regardless of their precise location in the input. For example, suppose our network is able to recognize the presence of a visual feature, say a short line segment oriented at some angle, at some particular location in an input image. Now imagine that we shift the line segment over in the image by just a few pixels, keeping everything else the same. We would still want the network to recognize the line segment, despite its shifted position (sometimes called a \"translation\").\n", "\n", "To achieve this type of **translation invariance**, we need to have multiple units that will look for the *same* feature *at all possible locations* in the input. Instead of each unit having its own independent set of weights, all of the units will **share** a single set of weight and bias values, but each unit will be connected to a different local region of the input. For example, in the picture above, each of the five output units \"looks at\" a different region of the input, but they all share a common set of three weights and one bias value.\n", "\n", "The convolutional layer architecture has several major advantages. First, if the weight and bias parameters become sensitive to some type of feature during training, that feature can be detected no matter where it is located in the input, because there will be *some* output unit that is connected to the input region where the feature occurs. Second, instead of requiring 20 separate trainable parameters, that is, (3 weights + 1 bias) $\\times$ 5 output units, we only require 4 trainable parameters in all (3 weights + 1 bias), which are shared among all five output units. The set of 3 shared weights is called a **kernel**, and the kernel weights together with the bias value is called a **filter**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A One-Dimensional Convolutional Layer\n", "\n", "To make this more concrete, let's write some code that builds our example network below.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from keras.layers import Conv1D" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 2')\n", "network.add(Input(shape=(7,1)))\n", "network.add(Conv1D(filters=1, \n", " kernel_size=3,\n", " name='output'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will use a 1-dimensional convolutional layer (`Conv1D`) for the output units, where each unit is connected to a receptive field of size 3 (`kernel_size`) in the input layer. All of the units in the output layer share a single set of weight/bias parameters, so `filters` is 1. Later on, we will see that a convolutional layer can keep track of more than one set of weight/bias parameters at a time, meaning that the units can learn to recognize several different features at once. But for now we will keep things simple with just one filter.\n", "\n", "The `Input` layer's `shape` keyword specifies an input layer of seven units, but why is the shape `(7,1)` instead of `(7,)` like in our first example? Because in general, layers in a convolutional network can consist of multiple **channels**. You can think of each channel as a duplicate set of units. In our example here, we only have one set of seven input units, so the number of channels is just 1. But we must still specify the number of channels as part of the input layer's `shape`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As the summary shows, the `Conv1D` layer has an output shape of `(None, 5, 1)`, meaning five output units organized into a single output channel. The `None` is there as a reminder that Keras layers always output a **batch** of patterns at a time, and we don't know the batch size in advance. The 4 trainable parameters refer to the set of three weights plus the bias shared among all units in the layer.\n", "\n", "Why does the output layer have 5 units? Although this is what we want, we didn't explicitly specify the number of output units anywhere in the network definition, so how did Keras know to include 5 units? That is a consequence of specifying `kernel_size=3` given an input layer of size 7. Keras automatically deduced that in order to cover the entire input layer with receptive fields spaced one unit apart, a total of 5 output units are required." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multiple Input Channels\n", "\n", "What would a multiple-channel input layer look like? Suppose our input layer has two channels instead of just one. We would then have two sets (or \"banks\", or \"sublayers\", or, well, \"channels\") of seven input units each:\n", "\n", "
\n", "\n", "How are the input units in the new channel connected to the output units? Just like before: all of the input units within an output unit's receptive field, regardless of the channel, are connected to that output unit, and together the weights form the \"kernel\" that is shared among all output units. For example, the picture below shows the receptive field of the leftmost and middle output unit. The kernel now has 6 weights instead of 3, for a total of 7 trainable parameters (6 weights + 1 bias) shared among the output units:\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 3')\n", "network.add(Input(shape=(7,2))) # changed (7,1) to (7,2)\n", "network.add(Conv1D(filters=1,\n", " kernel_size=3,\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Likewise, if the input layer has 3 channels, there are 10 shared parameters (9 weights + 1 bias):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 4')\n", "network.add(Input(shape=(7,3))) # changed (7,2) to (7,3)\n", "network.add(Conv1D(filters=1,\n", " kernel_size=3,\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Size of the Input Layer\n", "\n", "Suppose we now increase the size of the input layer, keeping the receptive field size (as specified by the `kernel_size` parameter in our network definition) and the number of input channels the same. How would this change the total number of trainable parameters in the network? For example, suppose we increase the input layer size from 7 to 1000 units, while still maintaining 3 input channels and receptive fields of size 3. To completely cover the input space with receptive fields, many more output units will clearly be required, but the exact number is determined by the receptive field size. In our case, with receptive fields of size 3, the output layer size will shrink by 2 units, for a total of 998 units instead of 1000.\n", "\n", "
\n", "\n", "But the number of trainable parameters doesn't change at all. The layer still keeps track of just 10 parameters (9 weights + 1 bias), shared among all 998 output units." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 5')\n", "network.add(Input(shape=(1000,3))) # changed 7 to 1000\n", "network.add(Conv1D(filters=1,\n", " kernel_size=3,\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multiple Filters\n", "\n", "The number of channels in a layer is sometimes called its **depth**, so in the above network, the input layer has depth 3. During training, the shared parameters will become sensitive to a particular type of feature occurring within the input data (at any or all input depths). But in our example above, there is only a single set of shared parameters (specified by `filters=1`), so the network can only learn to recognize, at most, one type of feature in the input. In general, we want a convolutional layer to be able to potentially learn to recognize many different types of features in the input. Therefore, most convolutional layers use multiple filters. Remember that a \"filter\" is just a set of trainable parameters (weights + bias) that is shared among all units in a convolutional layer.\n", "\n", "The above output layer has one filter, consisting of 10 parameters. If we increase the number of filters to 2, there will be two independent sets of parameters:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 6')\n", "network.add(Input(shape=(1000,3)))\n", "network.add(Conv1D(filters=2, # increased filters from 1 to 2\n", " kernel_size=3,\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the output layer shape now shows **2 output channels**. Each output channel will hold the result of processing the input data using a different filter. You can think of each channel as being a separate \"parallel copy\" of the output units, with an output activation pattern determined by that channel's own filter (weights + bias), independently of the other channels. In a way, a convolutional layer with multiple filters has multiple distinct \"personalities\", each of which learns to respond in its own unique way to the input data. Furthermore, the output channels of a layer can serve as the input channels to the next layer in sequence.\n", "\n", "If we increase the number of filters to 4, the total number of trainable parameters doubles again, to 40, and we now have 4 output channels:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 7')\n", "network.add(Input(shape=(1000,3)))\n", "network.add(Conv1D(filters=4, # increased filters from 2 to 4\n", " kernel_size=3,\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As an illustration, the picture below shows the four channels of the output layer and the three channels of the input layer. The filter parameters for output channel 0 are highlighted in red, and are shared by all of the output units in channel 0 (here, only the leftmost unit's receptive field connections are shown). The blue connections represent the filter parameters for output channel 3, and are shared by all of the channel 3 output units (but only the connections into the rightmost unit are shown). Likewise, the output units in channel 1 share their own set of 10 parameters (not shown), as do the units in channel 2, for a total of 40 trainable parameters.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A Two-Dimensional Convolutional Layer\n", "\n", "The examples above illustrate most of the key architectural concepts of convolutional layers. However, in practice **2-dimensional** convolutional layers are much more common than 1-dimensional layers, because most convolutional neural networks are applied to the task of image processing. To understand how 2-dimensional layers work, just increase the dimensionality of the input layer by one. Instead of the input units being arranged in a linear sequence as in the examples above, they will be arranged in a \"sheet\" of units, where each unit might correspond to an individual pixel in an input image:\n", "\n", "
\n", "\n", "Different output units are connected to different local 2D receptive fields in the input image, which means that the output units themselves have a 2D topological arrangement that mirrors the arrangement of the input units. Furthermore, all of the output units share a common set of weight and bias parameters. For example, three different output units, each with 3 $\\times$ 3 receptive fields, are shown below. Each unit shares the same set of 9 connection weights (shown in red) and 1 bias value (not shown).\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "If the input layer has multiple channels (*i.e.* its depth is greater than 1), then each output unit connects to the input units *at all depths* of the receptive field. Thus you can think of a unit in a 2D convolutional layer as receiving input from a 3-dimensional **volume** of units in the previous layer. For example, color RGB images have three channels specifying the pixel intensities for red, green, and blue. With three input channels, a receptive field of size 3 $\\times$ 3 would constitute a volume of 27 pixel values, and the total number of trainable parameters would be: (3 $\\times$ 3) weights $\\times$ 3 channels + 1 bias = 28 parameters. (For clarity, the receptive field connections from channels 0 and 1 are not shown in the picture below.)\n", "\n", "
\n", "\n", "The full set of output units and connections would look something like the following picture (again, for clarity the connections from the bottom two input channels are not shown.) The size of the output layer shrinks by two units in each direction, due to the 3 $\\times$ 3 receptive field size. The 2-dimensional pattern of activation across the output units is called a **feature map**.\n", "\n", "
\n", "\n", "Here is the code to create the above layer:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from keras.layers import Conv2D" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 8')\n", "network.add(Input(shape=(5,5,3)))\n", "network.add(Conv2D(filters=1,\n", " kernel_size=(3,3),\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Adding another filter to the layer is like adding another set of output units with their own independent set of 28 trainable parameters, which are shared among all of the new output units, bringing the total number of layer parameters to 56. The new output units constitute a new **output channel**, as in the picture below. The 2-dimensional pattern of activation on each output channel represents a **feature map** of size 3 $\\times$ 3 computed from the 5 $\\times$ 5 input image. In the picture, only some of the connections from the input layer to the output layer are shown for clarity. Keep in mind that the set of connections to output channel 0 also extends to the bottom two input channels as well. Furthermore, the purple output units shown in the top output channel have their own separate set of connections from all three input channels as well.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 9')\n", "network.add(Input(shape=(5,5,3)))\n", "network.add(Conv2D(filters=2, # changed filters from 1 to 2\n", " kernel_size=(3,3),\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a final example, adding three more filters would add three more sets of 28 parameters, giving a total of [(3 $\\times$ 3 $\\times$ 3) + 1] $\\times$ 5 = 140 trainable parameters. Given an input image, the layer would produce 5 separate feature maps as output.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 10')\n", "network.add(Input(shape=(5,5,3)))\n", "network.add(Conv2D(filters=5, # changed filters from 2 to 5\n", " kernel_size=(3,3),\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since the number of trainable parameters is independent of the size of the input image, we could change the input size to be anything else, say 32 $\\times$ 32 pixels, without affecting the number of parameters. The output feature maps would then be of size 30 $\\times$ 30. In other words, the width and height of the input image doesn't matter. All units in the layer share the same weights and bias for learning a feature, no matter where the feature occurs in the input image." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 11')\n", "network.add(Input(shape=(32,32,3))) # changed input size from 5x5 pixels to 32x32 pixels\n", "network.add(Conv2D(filters=5,\n", " kernel_size=(3,3),\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of thinking of a convolutional layer as a collection of individual *units*, which all share a common set of weights and get their input from different regions of the input layer, it may be more helpful to think of a layer as a collection of **columns of units**. Each unit in a column learns a different feature of the input data based on its own set of filter parameters, which are shared across all columns in the layer. This idea is shown below, for the example of 32 $\\times$ 32 pixel RGB input images and 5 output layer filters. [[*image source*](https://cs231n.github.io/convolutional-networks)]\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Padding\n", "\n", "In the previous example, using a 3 × 3 kernel (receptive field) with an input image of size 32 × 32 pixels produced an output of size 30 × 30. If we wish to keep the size of the output the same as the input, we can automatically pad the input image with extra 0s around the edges, so that the output will have the same height and width dimensions as the input. To do this, just specify `padding='same'`, as shown in the next example." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 12')\n", "network.add(Input(shape=(32,32,3)))\n", "network.add(Conv2D(filters=5,\n", " kernel_size=(3,3),\n", " padding='same', # keeps the output size the same as the input size\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### Self-Check 2\n", "\n", "Suppose we create a 2D convolutional layer with 64 filters and 3 $\\times$ 3 receptive fields, and feed it color RGB images of size 224 $\\times$ 224 pixels. How many trainable parameters will this layer have, and what will be the size of the feature maps created by the layer?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Self-Check 2')\n", "network.add(Input(shape=(224,224,3)))\n", "network.add(Conv2D(filters=64,\n", " kernel_size=(3,3),\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "64 * (3 * 3 * 3 + 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What will the output shape be if we specify `padding='same'`?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Self-Check 2')\n", "network.add(Input(shape=(224,224,3)))\n", "network.add(Conv2D(filters=64,\n", " kernel_size=(3,3),\n", " padding='same',\n", " name='output'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pooling Layers\n", "\n", "Another important type of convolutional neural network layer is the *pooling* layer, of which there are several different varieties. Both 1D and 2D versions are available, but the basic principle is the same for each. The job of a pooling layer is to perform \"downsampling\", in which an image or other type of input pattern is reduced in size by extracting a subset of the available information from the pattern and throwing away the rest. The result is, in some sense, a more abstract representation of the input that preserves only the most essential information.\n", "\n", "One very common type of downsampling operation is called \"max pooling\", and is illustrated below. The diagram shows a 2D image of pixels on the left (with individual pixel values labeled *a*, *b*, *c*, etc.), and a smaller image on the right created by finding the maximum of each non-overlapping 2 × 2 cluster of pixels in the original image. A more concrete example is also shown. The 2 × 2 pooling size results in the 6 × 6 input image being transformed into a smaller 3 × 3 output image, half the size of the original.\n", "\n", "Other types of pooling operations are also possible, such as *average* pooling, in which each cluster of pixels is replaced by the average value of the pixels in the cluster instead of the maximum value." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2 × 2 Max Pooling\n", "\n", "\n", "\n", "#### Example\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The code below adds a 2 × 2 `MaxPooling2D` layer to our earlier (un-padded) example network." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from keras.layers import MaxPooling2D" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 13')\n", "network.add(Input(shape=(32,32,3)))\n", "network.add(Conv2D(filters=5,\n", " kernel_size=(3,3),\n", " name='conv1'))\n", "network.add(MaxPooling2D(pool_size=(2,2), # added a 2x2 max pooling layer\n", " name='pool1'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the five output channels of the `Conv2D` layer, each of size 30 × 30, are transformed by the pooling layer into five new channels, each of size 15 × 15. The number of trainable parameters of the network, however, does not change, because a pooling layer simply applies a single mathematical operation (*e.g.*, maximum, average, etc.) uniformly across its input." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3 × 3 Max Pooling\n", "\n", "Using a larger pooling size results in a smaller output image. For example, with a 3 × 3 pooling size, an input image would be transformed into an output image one third of the original size, since each 3 × 3 cluster of pixel values gets transformed into a single value, as illustrated below.\n", "\n", "\n", "\n", "#### Example\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "network = Sequential(name='Example 14')\n", "network.add(Input(shape=(32,32,3)))\n", "network.add(Conv2D(filters=5,\n", " kernel_size=(3,3),\n", " name='conv1'))\n", "network.add(MaxPooling2D(pool_size=(3,3), # changed (2,2) to (3,3)\n", " name='pool1'))\n", "network.summary(print_fn=print)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The five 30 × 30 channels are now transformed by the `MaxPooling2D` layer into five new channels of size 10 × 10." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.12" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }