Validation data without targets

Question

I have a validation dataset of images to be classified by my CNN model. I want to load these images using pytorch. torchvision.datasets.ImageFolder() function doesn't work, since there are no targets, because the dataset is unclassified. I'm assuming that I need to write a custom dataset class, that I would later put in torch.utils.data.DataLoader(). I've searched online, but I'm still not really understanding how the class should look like.

I've tried this

import torch
from torch.utils.data import Dataset
from torchvision.io import read_image
import os


class Dset(Dataset):
    def __init__(self, dir: str, transform=None) -> None:
        self.transform = transform
        self.images = os.listdir(dir)
        self.dir = dir
    
    def __getitem__(self, index: int) -> torch.Tensor:
        image = read_image(f'{self.dir}/{self.images[index]}')
        if self.transform is not None:
            image = self.transform(image)
        return image

    def __len__(self) -> int:
        return len(self.images)

But after this cell (all images are in .data/)

from torchvision import transforms

batch_size = 64
transform = transforms.Compose([transforms.Grayscale(), transforms.ToTensor()])
data = Dset('data', transform=transform)
trainloader = torch.utils.data.DataLoader(data, batch_size=batch_size, shuffle=True)

images, labels = iter(trainloader)

I am encountering this error: TypeError: Input image tensor permitted channel values are [1, 3], but found 4

Update

import torch
from torch.utils.data import Dataset
from torchvision.io import read_image, ImageReadMode
import os


class Dset(Dataset):
    def __init__(self, dir: str, transform=None) -> None:
        self.transform = transform
        self.images = os.listdir(dir)
        self.dir = dir
    
    def __getitem__(self, index: int) -> torch.Tensor:
        image = read_image(f'{self.dir}/{self.images[index]}', mode=ImageReadMode.RGB)
        if self.transform is not None:
            image = self.transform(image)
        return image

    def __len__(self) -> int:
        return len(self.images)

The error was caused by the alpha channel in the images.

After fixing that, I'm encountering this: TypeError: pic should be PIL Image or ndarray. Got <class 'torch.Tensor'>

Update 2

from torchvision import transforms

batch_size = 64
transform = transforms.Compose(
[transforms.ToPILImage(), transforms.Resize((512, 512)),
transforms.Grayscale(), transforms.ToTensor()]
)
data = Dset('data', transform=transform)
trainloader = torch.utils.data.DataLoader(data, batch_size=batch_size, shuffle=True)

images = iter(trainloader)[0]

torchvision.transforms.ToTensor only converts PIL Image or numpy.ndarray to tensor.

Last line results in: TypeError: '_SingleProcessDataLoaderIter' object is not subscriptable

Answer 1

As discussed in the comments - the issue is your images have an alpha channel. You can modify the read_image function to remove the alpha channel from the input images as follows:

image = read_image(f'{self.dir}/{self.images[index]}', mode=ImageReadMode.RGB)

For other modes, you can check the ImageReadMode class.

Update1:

For the new error - according to the documentation:

ToTensor class converts a PIL Image or ndarray to tensor and scale the values accordingly.

But here you are providing tensor as an input instead of the required PIL image or ndarray.

To resolve this you may use the ToPILImage method.

Update2:

For the error: TypeError: '_SingleProcessDataLoaderIter' object is not subscriptable

Check how to Iterate through a DataLoader from the PyTorch tutorials.

Also, you may try using a for loop as well to iterate as follows:

for images in trainloader:
    # Process images here
    break # update this break statement as per your requirement