How do I implement gradient descent in Python?

How do I implement gradient descent in Python?

To implement gradient descent in Python, you need to define your cost function and its gradient. Do you have a specific function in mind?

Yes, I want to minimize the function f(x) = (x-3)^2.

Great choice! You can start by defining the function and its derivative. Would you like a code example?

Yes, please provide a simple code example.

Here's a basic implementation:

import numpy as np

def f(x):
    return (x - 3) ** 2

def df(x):
    return 2 * (x - 3)

x = 0  # Starting point
learning_rate = 0.1
for _ in range(100):
    x -= learning_rate * df(x)
print(x)  # Should be close to 3

How do I choose the learning rate?

Choosing the learning rate can be tricky. A small learning rate may take longer to converge, while a large one can overshoot. It's often a good idea to experiment with different values.