How does ReLU compare to Sigmoid?

What are the main differences between ReLU and Sigmoid activation functions?

ReLU (Rectified Linear Unit) is linear for positive inputs and zero for negative inputs, while Sigmoid outputs values between 0 and 1, making it suitable for binary classification.

Why is ReLU often preferred over Sigmoid in deep learning?

ReLU helps mitigate the vanishing gradient problem, allowing models to learn faster and perform better in deeper networks.

Are there any drawbacks to using ReLU?

Yes, ReLU can suffer from the 'dying ReLU' problem, where neurons can become inactive and stop learning if they output zero consistently.

In what scenarios would you still use Sigmoid?

Sigmoid is useful in the output layer of binary classification problems, as it provides a probability-like output.