Abstract
Graph Neural Networks (GNNs) have attracted considerable attention and haveemerged as a new promising paradigm to process graph-structured data. GNNs areusually stacked to multiple layers and the node representations in each layerare computed through propagating and aggregating the neighboring node featureswith respect to the graph. By stacking to multiple layers, GNNs are able tocapture the long-range dependencies among the data on the graph and thus bringperformance improvements. To train a GNN with multiple layers effectively, somenormalization techniques (e.g., node-wise normalization, batch-wisenormalization) are necessary. However, the normalization techniques for GNNsare highly task-relevant and different application tasks prefer to differentnormalization techniques, which is hard to know in advance. To tackle thisdeficiency, in this paper, we propose to learn graph normalization byoptimizing a weighted combination of normalization techniques at four differentlevels, including node-wise normalization, adjacency-wise normalization,graph-wise normalization, and batch-wise normalization, in which theadjacency-wise normalization and the graph-wise normalization are newlyproposed in this paper to take into account the local structure and the globalstructure on the graph, respectively. By learning the optimal weights, we areable to automatically select a single best or a best combination of multiplenormalizations for a specific task. We conduct extensive experiments onbenchmark datasets for different tasks, including node classification, linkprediction, graph classification and graph regression, and confirm that thelearned graph normalization leads to competitive results and that the learnedweights suggest the appropriate normalization techniques for the specific task.Source code is released here https://github.com/cyh1112/GraphNormalization.