Cycle consistency is widely used for face editing. However, we observe thatthe generator tends to find a tricky way to hide information from the originalimage to satisfy the constraint of cycle consistency, making it impossible tomaintain the rich details (e.g., wrinkles and moles) of non-editing areas. Inthis work, we propose a simple yet effective method named HifaFace to addressthe above-mentioned problem from two perspectives. First, we relieve thepressure of the generator to synthesize rich details by directly feeding thehigh-frequency information of the input image into the end of the generator.Second, we adopt an additional discriminator to encourage the generator tosynthesize rich details. Specifically, we apply wavelet transformation totransform the image into multi-frequency domains, among which thehigh-frequency parts can be used to recover the rich details. We also noticethat a fine-grained and wider-range control for the attribute is of greatimportance for face editing. To achieve this goal, we propose a novel attributeregression loss. Powered by the proposed framework, we achieve high-fidelityand arbitrary face editing, outperforming other state-of-the-art approaches.