What are the key parameters in gradient descent?

What are the key parameters in gradient descent?

The key parameters in gradient descent include the learning rate, number of iterations, and the initial weights.

How does the learning rate affect the optimization process?

The learning rate determines the size of the steps taken towards the minimum. A small rate may take longer to converge, while a large rate can overshoot the minimum.

What happens if the learning rate is too high?

If the learning rate is too high, the algorithm may diverge, leading to oscillations or failure to converge to a minimum.

Can you explain the concept of momentum in gradient descent?

Momentum helps accelerate gradient descent by adding a fraction of the previous update to the current update, smoothing out the optimization path.