def benchmark(device='cuda'): a = torch.randn(4096, 4096, device=device) b = torch.randn(4096, 4096, device=device) torch.cuda.synchronize() start = time.time() for _ in range(100): torch.mm(a, b) torch.cuda.synchronize() return (time.time() - start) / 100
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu126 pytorch for cuda 12.6
PyTorch uses the Triton language for GPU programming. New CUDA versions sometimes break compatibility with the version of Triton pinned in PyTorch releases. If you encounter triton errors after upgrading to a 12.6 environment, try upgrading triton separately: def benchmark(device='cuda'): a = torch