[Paper] Very Deep Convolutional Networks for Large-Scale Image Recognition

“Very Deep Convolutional Networks for Large-Scale Image Recognition”이란 논문에 대한 리뷰입니다.

원문은 링크에서 확인할 수 있습니다.

Key

3*3 Convolution Filter with deep network

Pre-process

• Subtracting mean

Architecture:

• Convolution Layer :3*3 Filter / stride 1 / channel a factor of 2
• Maxpool Layer: 2*2 Filter with stride 2 (not all layer)
• ReLU for all hidden layer
• NO Normalization Layer

Config

• Mini-batch: 256 batch size / 0.9 momentum / 5e-4 weight decay / 1e-2 learning rate
• Dropout on FC layer with 0.5
• Learning rate scheduler: decrease by 10 when validation not improving

Insight

• Two 33 layer ~ Single 55 layer / Three 33 layer ~ Single 77 layer
• Make decision function more discriminative
• Less parameters  imposing a regularization on large filter
• Less epoch
• Due to greater depth and small convolution layers
• Pre-initialization(NO NEED, Glorot & Bengio(2010))