Likelihood-free inference methods typically make use of a distance betweensimulated and real data. A common example is the maximum mean discrepancy(MMD), which has previously been used for approximate Bayesian computation,minimum distance estimation, generalised Bayesian inference, and within thenonparametric learning framework. The MMD is commonly estimated at a root-$m$rate, where $m$ is the number of simulated samples. This can lead tosignificant computational challenges since a large $m$ is required to obtain anaccurate estimate, which is crucial for parameter estimation. In this paper, wepropose a novel estimator for the MMD with significantly improved samplecomplexity. The estimator is particularly well suited for computationallyexpensive smooth simulators with low- to mid-dimensional inputs. This claim issupported through both theoretical results and an extensive simulation study onbenchmark simulators.