Abstract

We present a conceptually simple but effective funnel activation for imagerecognition tasks, called Funnel activation (FReLU), that extends ReLU andPReLU to a 2D activation by adding a negligible overhead of spatial condition.The forms of ReLU and PReLU are y = max(x, 0) and y = max(x, px), respectively,while FReLU is in the form of y = max(x,T(x)), where T(x) is the 2D spatialcondition. Moreover, the spatial condition achieves a pixel-wise modelingcapacity in a simple way, capturing complicated visual layouts with regularconvolutions. We conduct experiments on ImageNet, COCO detection, and semanticsegmentation tasks, showing great improvements and robustness of FReLU in thevisual recognition tasks.

 

Quick Read (beta)