What are some common errors in gradient descent?

What are some common errors in gradient descent?

Some common errors include choosing a poor learning rate, not normalizing data, and getting stuck in local minima.

How can I choose the right learning rate?

You can use techniques like learning rate schedules, grid search, or adaptive learning rate methods like Adam.

What happens if the learning rate is too high?

If the learning rate is too high, the algorithm may overshoot the minimum and diverge instead of converging.

Can you explain local minima?

Local minima are points where the function value is lower than neighboring points, but not necessarily the lowest overall.

What are some strategies to avoid local minima?

You can use techniques like momentum, simulated annealing, or initializing with different starting points.