Studying stability of different convolutional neural networks against additive noise

Understanding internal process of ConvNets is commonly done using visualization techniques. However, these techniques do not usually provide a tool for estimating stability of a ConvNet against noise. In this paper, we show how to analyze a ConvNet in the frequency domain. Using the frequency domain analysis, we show the reason that a ConvNet might be sensitive to a very low magnitude additive noise. Our experiments on a few ConvNets trained on different datasets reveals that convolution kernels of a trained ConvNet usually pass most of the frequencies and they are not able to effectively eliminate the effect of high frequencies. They also show that a convolution kernel with more concentrated frequency response is more stable against noise. Finally, we illustrate that augmenting a dataset with noisy images can compress the frequency response of convolution kernels.