In recent years, neural implicit representations gained popularity in 3Dreconstruction due to their expressiveness and flexibility. However, theimplicit nature of neural implicit representations results in slow inferencetime and requires careful initialization. In this paper, we revisit the classicyet ubiquitous point cloud representation and introduce a differentiablepoint-to-mesh layer using a differentiable formulation of Poisson SurfaceReconstruction (PSR) that allows for a GPU-accelerated fast solution of theindicator function given an oriented point cloud. The differentiable PSR layerallows us to efficiently and differentiably bridge the explicit 3D pointrepresentation with the 3D mesh via the implicit indicator field, enablingend-to-end optimization of surface reconstruction metrics such as Chamferdistance. This duality between points and meshes hence allows us to representshapes as oriented point clouds, which are explicit, lightweight andexpressive. Compared to neural implicit representations, our Shape-As-Points(SAP) model is more interpretable, lightweight, and accelerates inference timeby one order of magnitude. Compared to other explicit representations such aspoints, patches, and meshes, SAP produces topology-agnostic, watertightmanifold surfaces. We demonstrate the effectiveness of SAP on the task ofsurface reconstruction from unoriented point clouds and learning-basedreconstruction.