The paper introduces a new efficient nonlinear one-class classifierformulated as the Rayleigh quotient criterion. The method, operating in areproducing kernel Hilbert subspace, minimises the scatter of targetdistribution along an optimal projection direction while at the same timekeeping projections of target observations as distant as possible from theorigin which serves as an artificial outlier with respect to the data. Weprovide a graph embedding view of the problem which can then be solvedefficiently using the spectral regression approach. In this sense, unlikeprevious similar methods which often require costly eigen-computations of densematrices, the proposed approach casts the problem under consideration into aregression framework which avoids eigen-decomposition computations. Inparticular, it is shown that the dominant complexity of the proposed method isthe complexity of computing the kernel matrix. Additional appealingcharacteristics of the proposed one-class classifier are: 1-the ability to betrained in an incremental fashion (allowing for application in streaming datascenarios while also reducing computational complexity in the non-streamingoperation mode); 2-being unsupervised while also providing the ability for theuser to specify the expected fraction of outliers in the training set inadvance; And last but not least 3-the deployment of the kernel trick allowingfor a large class of functions by nonlinearly mapping the data into ahigh-dimensional feature space. Extensive experiments conducted on severaldatasets verifies the merits of the proposed approach in comparison with someother alternatives.