What is web scraping and how can I do it with Python?

What is web scraping and how can I do it with Python?

Web scraping is the process of extracting data from websites. You can use libraries like Beautiful Soup and Scrapy in Python.

Can you show me a simple example using Beautiful Soup?

Sure! First, install Beautiful Soup with pip install beautifulsoup4. Then, you can use it like this:

import requests
from bs4 import BeautifulSoup

url = 'http://example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.text)

What if the website has anti-scraping measures?

In that case, you might need to use techniques like rotating user agents, using proxies, or even headless browsers with Selenium.

Are there any legal issues I should be aware of?

Yes, always check the website's robots.txt file and terms of service to ensure you're allowed to scrape their data.