Softmax function
-
Activation FunctionsMLAI/DeepLearning 2020. 1. 30. 14:32
1. Overview In artificial neural networks, the activation function of a node defines the output of that node given an input or set of inputs. A standard computer chip circuit can be seen as a digital network of activation functions that can be "ON" (1) or "OFF" (0), depending on the input. This is similar to the behavior of the linear perceptron in neural networks. However, only nonlinear activa..
-
Softmax and Cross-Entropy with CNNMLAI/DeepLearning 2019. 10. 16. 15:56
1. Overview 2. Description How come two output values add up to one? 2.1 Softmax function(Normalized exponential function) $$f_{j}(z)=\frac{e^{zj}}{\sum_{k}e^{zk}}$$ Normally, the dog and the cat neurons would have any kind of real values. Applying the softmax function which is written up over there at the top, and that would bring these values to be between zero and one and it would make them a..