Abstract
The increasing amount of graph data places requirements on the efficienttraining of graph neural networks (GNNs). The emerging graph distillation (GD)tackles this challenge by distilling a small synthetic graph to replace thereal large graph, ensuring GNNs trained on real and synthetic graphs exhibitcomparable performance. However, existing methods rely on GNN-relatedinformation as supervision, including gradients, representations, andtrajectories, which have two limitations. First, GNNs can affect the spectrum(i.e., eigenvalues) of the real graph, causing spectrum bias in the syntheticgraph. Second, the variety of GNN architectures leads to the creation ofdifferent synthetic graphs, requiring traversal to obtain optimal performance.To tackle these issues, we propose Graph Distillation with Eigenbasis Matching(GDEM), which aligns the eigenbasis and node features of real and syntheticgraphs. Meanwhile, it directly replicates the spectrum of the real graph andthus prevents the influence of GNNs. Moreover, we design a discriminationconstraint to balance the effectiveness and generalization of GDEM.Theoretically, the synthetic graphs distilled by GDEM are restricted spectralapproximations of the real graphs. Extensive experiments demonstrate that GDEMoutperforms state-of-the-art GD methods with powerful cross-architecturegeneralization ability and significant distillation efficiency. Our code isavailable at https://github.com/liuyang-tian/GDEM.