How can I perform batch inference using PyTorch?

How can I perform batch inference using PyTorch?

To perform batch inference in PyTorch, you can use a DataLoader to load your data in batches and then pass these batches through your model.

Can you show me a code example for that?

Sure! Here's a simple example:

import torch
from torch.utils.data import DataLoader, TensorDataset

# Sample data
inputs = torch.randn(100, 3)  # 100 samples, 3 features
labels = torch.randn(100, 1)   # 100 labels

dataset = TensorDataset(inputs, labels)
dataloader = DataLoader(dataset, batch_size=10)

model = MyModel()  # Assume MyModel is defined
model.eval()  # Set the model to evaluation mode

with torch.no_grad():
    for batch_inputs, _ in dataloader:
        outputs = model(batch_inputs)
        # Process outputs

What if my model is on a GPU?

If your model is on a GPU, make sure to move your inputs to the GPU as well using .to(device) where device is your GPU device.