Can you show me some examples of web scraping using Python 3.10 and asyncio?

Can you show me some examples of web scraping using Python 3.10 and asyncio?

Sure! Here's a simple example using aiohttp and asyncio to scrape a webpage:

import aiohttp
import asyncio

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

async def main():
    url = 'https://example.com'
    html = await fetch(url)
    print(html)

asyncio.run(main())

What if I want to scrape multiple pages at once?

You can modify the main function to gather multiple tasks. Here's an example:

async def fetch_all(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(url) for url in urls]
        return await asyncio.gather(*tasks)

async def main():
    urls = ['https://example.com', 'https://example.org']
    htmls = await fetch_all(urls)
    for html in htmls:
        print(html)

How do I handle exceptions while scraping?

You can use try-except blocks within your fetch function to handle exceptions. Here's an example:

async def fetch(url):
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                response.raise_for_status()  # Raise an error for bad responses
                return await response.text()
    except Exception as e:
        print(f'Error fetching {url}: {e}')