
Loss functions and their final-layer activations
When making the first steps with deep learning, we grasp the idea of using a neural network to learn a function that maps data to other data. We are often told that neural networks are a powerful tool in machine learning because of their non-linearity and their ability to learn complex functions from data, which results in minizing some loss function. In this post, we will explore how the final-layer activations are dependent on the loss function of our problem. ...