by **Guest** » Sat Jun 13, 2020 7:41 am

Was it your intention to make it as hard as possible for people to respond?

If it was- good work!

I first had to open a pdf, something many people refuse to do for fear of viruses. Then I had to search through it for any mention of a "derivative". There was no mention of the "chain rule" but I think you are referring to the derivative of the "Sigmoid' which is given as [tex]\frac{1}{1+ e^{-x}}[/tex]. (The text has partial derivatives. I don't know why since there is only the variable x.)

To differentiate that, I would write it as [tex](1+ e^{-x})^{-1}[/tex] and think of it as [tex]z(y)= y^{-1}[/tex] with [tex]y(x)= 1+ e^{-x}[/tex]. The "chain rule" then is [tex]\frac{dz}{dx}= \frac{dz}{dy}\frac{dy}{dx}[/tex]. [tex]\frac{dz}{dy}= (-1)y^{-1-1}= (-1)y^{-2}= -\frac{1}{y^2}[/tex] and [tex]\frac{dy}{dx}= -e^{-x}[/tex].

So, by the "chain rule", [tex]\frac{dz}{dx}= \frac{1}{y^2}e^{-x}= \frac{e^{-x}}{(1+ e^{-x})^2}[/tex].

For some reason, in this paper they separate [tex]\frac{e^{-x}}{(1+ e^{-x})^2}[/tex] into [tex]\frac{e^{-x}}{1+ e^{-x}}\frac{1}{1+ e^{-x}}[/tex] and write [tex]\frac{e^{-x}}{1+ e^{-x}}[/tex] as [tex]1- \frac{1}{1+ e^{-x}}[/tex]. That is, of course, true because [tex]1- \frac{1}{1+ e^{-x}}= \frac{1+ e^{-x}}{1+ e^{-x}}- \frac{1}{1+ e^{-x}}= \frac{1+ e^{-x}- 1}{1+ e^{-x}}[/tex].

That allows them to write [tex]\frac{d Sigmoid}{dx}= Sigmoid\times (1- Sigmoid)[/tex].

Is that what you were asking about?