INVESTIGATIONS ON THE SMOOTH APPROXIMATION TO ROCLU (ROBUST CAUCHY LINEAR UNIT) ACTIVATION FUNCTION USING GAUSSIAN ERROR FUNCTION

Abstract

In this article we present smooth approximation to the activationfunction f(x) named RoCLU (Robust Cauchy Linear Unit) using function F(x) dependingon the Gaussian error function. We prove the inequality for the error between these two functions. Quantitative evaluation of the maximum gradient discrepancy sup  |f'(x) − F'(x)| and corresponding critical activation points across varying scaling x<0 hyperparameter a values is also given. Some numerical examples are presented using CAS MATHEMATICA.