Back to Yann's
Home
Publications
LeNet-5
Demos
LeNet-5, convolutional
neural networks
Convolutional Neural Networks are are a special kind of multi-layer
neural networks. Like almost every other neural networks they are
trained with a version of the back-propagation algorithm. Where they
Unusual
differ is in the architecture.
Patterns
Convolutional Neural Networks are designed to recognize visual
unusual styles
patterns directly from pixel images with minimal preprocessing.
weirdos
They can recognize patterns with extreme variability (such as
handwritten characters), and with robustness to distortions and
Invariance
translation (anim) simple geometric transformations.
scale (anim)
LeNet-5 is our latest convolutional network designed for handwritten
rotation (anim)
squeezing (anim) and machine-printed character recognition.
Here is an example of LeNet-5 in action.
stroke width
(anim)
Noise
Resistance
noisy 3 and 6
noisy 2 (anim)
noisy 4 (anim)
Multiple
Character
various stills
dancing 00 (anim)
dancing 384
(anim)
Complex cases
(anim)
35 -> 53
12 -> 4-> 21
23 -> 32
30 + noise
31-51-57-61
Many more examples are available in the column on the left:
Several papers on LeNet and convolutional networks are available
on my publication page:
[LeCun et al., 1998]
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings
of the IEEE, november 1998.
[Bottou et al., 1997]
L. Bottou, Y. LeCun, and Y. Bengio. Global training of
document processing systems using graph transformer
networks. In Proc. of Computer Vision and Pattern
Recognition, Puerto-Rico, 1997. IEEE.
[LeCun et al., 1997]
Y. LeCun, L. Bottou, and Y. Bengio. Reading checks with
graph transformer networks. In International Conference on
Acoustics, Speech, and Signal Processing, volume 1, pages
151-154, Munich, 1997. IEEE.
[LeCun and Bengio, 1995a]
Y. LeCun and Y. Bengio. Convolutional networks for
images, speech, and time-series. In M. A. Arbib, editor, The
Handbook of Brain Theory and Neural Networks. MIT Press,
1995.
[LeCun et al., 1995a]
Y. LeCun, L. D. Jackel, L. Bottou, A. Brunot, C. Cortes, J. S.
Denker, H. Drucker, I. Guyon, U. A. Muller, E. Sackinger,
P. Simard, and V. Vapnik. Comparison of learning
algorithms for handwritten digit recognition. In F. Fogelman
and P. Gallinari, editors, International Conference on
Artificial Neural Networks, pages 53-60, Paris, 1995. EC2 &
Cie.
[Vaillant et al., 1994]
R. Vaillant, C. Monrocq, and Y. LeCun. Original approach
for the localisation of objects in images. IEE Proc on Vision,
Image, and Signal Processing, 141(4):245-250, August
1994.
[Matan et al., 1992b]
Ofer Matan, Christopher J. C. Burges, Yann LeCun, and
John S. Denker. Multi-digit recognition using a space
displacement neural network. In J. M. Moody, S. J. Hanson,
and R. P. Lippman, editors, Neural Information Processing
Systems, volume 4. Morgan Kaufmann Publishers, San
Mateo, CA, 1992.
[Boser et al., 1991]
B. Boser, E. Sackinger, J. Bromley, Y. LeCun, and L. Jackel.
An analog neural network processor with programmable
topology. IEEE Journal of Solid-State Circuits, 26(12):20172025, December 1991.
[LeCun et al., 1990b]
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E.
Howard, W. Hubbard, and L. D. Jackel. Handwritten digit
recognition with a back-propagation network. In David
Touretzky, editor, Advances in Neural Information
Processing Systems 2 (NIPS*89), Denver, CO, 1990. Morgan
Kaufman.
[LeCun, 1989b]
Y. LeCun. Generalization and network design strategies.
Technical Report CRG-TR-89-4, Department of Computer
Science, University of Toronto, 1989.
[LeCun et al., 1989a]
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E.
Howard, W. Hubbard, and L. D. Jackel. Backpropagation
applied to handwritten zip code recognition. Neural
Computation, 1(4):541-551, Winter 1989.
[LeCun et al., 1989d]
Y. LeCun, L. D. Jackel, B. Boser, J. S. Denker, H. P. Graf,
I. Guyon, D. Henderson, R. E. Howard, and W. Hubbard.
Handwritten digit recognition: Applications of neural net
chips and automatic learning. IEEE Communication, pages
41-46, November 1989. invited paper.
Yann LeCun
[email protected]