Paper ID: 2111.14349

First Power Linear Unit with Sign

Boxi Duan

This paper proposes a novel and insightful activation method termed FPLUS, which exploits mathematical power function with polar signs in form. It is enlightened by common inverse operation while endowed with an intuitive meaning of bionics. The formulation is derived theoretically under conditions of some prior knowledge and anticipative properties, and then its feasibility is verified through a series of experiments using typical benchmark datasets, whose results indicate our approach owns superior competitiveness among numerous activation functions, as well as compatible stability across many CNN architectures. Furthermore, we extend the function presented to a more generalized type called PFPLUS with two parameters that can be fixed or learnable, so as to augment its expressive capacity, and outcomes of identical tests validate this improvement.

Submitted: Nov 29, 2021