I am attempting to training model using VIT to classify some images. During training the script stuck and I don't know where is my error. The classification features by some images with only two target 0 and 1 (false and true). The batch size is 32 and epochs are only 3.
Below I put the script for training model:
import torch.utils.data as data from torch.autograd import Variable import numpy as np train_loader = data.DataLoader(train_ds, batch_size=BATCH_SIZE, shuffle=True, num_workers=2) test_loader = data.DataLoader(test_ds, batch_size=BATCH_SIZE, shuffle=True, num_workers=2) # Train the model for epoch in range(EPOCHS): for step, (x, y) in enumerate(train_loader): # Change input array into list with each batch being one element x = np.split(np.squeeze(np.array(x)), BATCH_SIZE) # Remove unecessary dimension for index, array in enumerate(x): x[index] = np.squeeze(array) # Apply feature extractor, stack back into 1 tensor and then convert to tensor x = torch.tensor(np.stack(feature_extractor(x)['pixel_values'], axis=0)) # Send to GPU if available x = x.to(device) y = y.to(device) b_x = Variable(x) # batch x (image) b_y = Variable(y) # batch y (target) # Feed through model output = model(b_x, None) loss = output[0] # Calculate loss if loss is None: loss = loss_func(output, b_y) optimizer.zero_grad() loss.backward() optimizer.step() if step % 50 == 0: # Get the next batch for testing purposes test = next(iter(test_loader)) test_x = test[0] # Reshape and get feature matrices as needed test_x = np.split(np.squeeze(np.array(test_x)), BATCH_SIZE) for index, array in enumerate(test_x): test_x[index] = np.squeeze(array) test_x = torch.tensor(np.stack(feature_extractor(test_x)['pixel_values'], axis=0)) # Send to appropirate computing device test_x = test_x.to(device) test_y = test[1].to(device) # Get output (+ respective class) and compare to target test_output, loss = model(test_x, test_y) test_output = test_output.argmax(1) # Calculate Accuracy accuracy = (test_output == test_y).sum().item() / BATCH_SIZE print('Epoch: ', epoch, '| train loss: %.4f' % loss, '| test accuracy: %.2f' % accuracy) The error message is this: ValueError: array split does not result in an equal division. And it highlights the comand x = np.split(np.squeeze(np.array(x)), BATCH_SIZE).
1 Answer
I recently came across the same error. Check with the length of the train_ds. and change the BATCH_SIZE so that when you divide the len(train_ds) by BATCH_SIZE the reminder is zero.
