Constraining the Size of the Dusty Torus in Active Galactic Nuclei, An Optical/Infrared Reverberation Lag Study