We present an efficient learning-based algorithm for deformable, pairwise 3Dmedical image registration. Current registration methods optimize an energyfunction independently for each pair of images, which can be time-consuming forlarge data. We define registration as a parametric function, and optimize itsparameters given a set of images from a collection of interest. Given a newpair of scans, we can quickly compute a registration field by directlyevaluating the function using the learned parameters. We model this functionusing a CNN, and use a spatial transform layer to reconstruct one image fromanother while imposing smoothness constraints on the registration field. Theproposed method does not require supervised information such as ground truthregistration fields or anatomical landmarks. We demonstrate registrationaccuracy comparable to state-of-the-art 3D image registration, while operatingorders of magnitude faster in practice. Our method promises to significantlyspeed up medical image analysis and processing pipelines, while facilitatingnovel directions in learning-based registration and its applications. Our codeis available at https://github.com/balakg/voxelmorph.