Well, max-pooling and monotonely increasing non-linearities commute. This means that MaxPool(Relu(x)) = Relu(MaxPool(x)) for any input. So the result is the same in that case. So it is technically better to first subsample through max-pooling and then apply the non-linearity (if it is costly, such as the sigmoid). In practice it is often done the … Read more
You’re right to say that kernel_size defines the size of the sliding window. The filters parameters is just how many different windows you will have. (All of them with the same length, which is kernel_size). How many different results or channels you want to produce. When you use filters=100 and kernel_size=4, you are creating 100 … Read more
You could generate the subarrays using as_strided: import numpy as np a = np.array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]]) sub_shape = (3,3) view_shape = tuple(np.subtract(a.shape, sub_shape) + 1) + sub_shape strides = a.strides + … Read more
UpSampling2D is just a simple scaling up of the image by using nearest neighbour or bilinear upsampling, so nothing smart. Advantage is it’s cheap. Conv2DTranspose is a convolution operation whose kernel is learnt (just like normal conv2d operation) while training your model. Using Conv2DTranspose will also upsample its input but the key difference is the … Read more
The number of filters is the number of neurons, since each neuron performs a different convolution on the input to the layer (more precisely, the neurons’ input weights form convolution kernels). A feature map is the result of applying a filter (thus, you have as many feature maps as filters), and its size is a … Read more
The Y array in your screenshot is not a 1D array, it’s a 2D array with 300 rows and 1 column, as indicated by its shape being (300, 1). To remove the extra dimension, you can slice the array as Y[:, 0]. To generally convert an n-dimensional array to 1D, you can use np.reshape(a, a.size). … Read more
Yes, it is possible and you should also use a doubly block circulant matrix (which is a special case of Toeplitz matrix). I will give you an example with a small size of kernel and the input, but it is possible to construct Toeplitz matrix for any kernel. So you have a 2d input x … Read more
Convolution is a mathematical operator primarily used in signal processing. Numpy simply uses this signal processing nomenclature to define it, hence the “signal” references. An array in numpy is a signal. The convolution of two signals is defined as the integral of the first signal, reversed, sweeping over (“convolved onto”) the second signal and multiplied … Read more
The pooling and convolutional ops slide a “window” across the input tensor. Using tf.nn.conv2d as an example: If the input tensor has 4 dimensions: [batch, height, width, channels], then the convolution operates on a 2D window on the height, width dimensions. strides determines how much the window shifts by in each of the dimensions. The … Read more
I want to explain with picture from C3D. In a nutshell, convolutional direction & output shape is important! ↑↑↑↑↑ 1D Convolutions – Basic ↑↑↑↑↑ just 1-direction (time-axis) to calculate conv input = [W], filter = [k], output = [W] ex) input = [1,1,1,1,1], filter = [0.25,0.5,0.25], output = [1,1,1,1,1] output-shape is 1D array example) graph … Read more