Using CUDA with pytorch?

Is there a way to reliably enable CUDA on the whole model?

I want to run the training on my GPU. I found on some forums that I need to apply .cuda() on anything I want to use CUDA with (I've applied it to everything I could without making the program crash). Surprisingly, this makes the training even slower.

Then, I found that you could use this torch.set_default_tensor_type('torch.cuda.FloatTensor') to use CUDA. With both enabled, nothing changes. What is happening?

2 Answers

You can use the tensor.to(device) command to move a tensor to a device.

The .to() command is also used to move a whole model to a device, like in the post you linked to.

Another possibility is to set the device of a tensor during creation using the device= keyword argument, like in t = torch.tensor(some_list, device=device)

To set the device dynamically in your code, you can use

device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 

to set cuda as your device if possible.

There are various code examples on PyTorch Tutorials and in the documentation linked above that could help you.

With both enabled, nothing changes.

That is because you have already set every tensor to GPU.

Is there a way to reliably enable CUDA on the whole model?

model.to('cuda') 

I've applied it to everything I could

You only need to apply it to tensors the model will be interacting with, generally:

• the model's pramaters model.to('cuda')
• the features data features = features.to('cuda')
• the target data targets = targets.to('cuda')