In my implementation, I used DistributedSampler(..., drop_last=False) for validation. This is okay when samples % world_size == 0, so my experiments were fine. However, if this value is non-zero by changing the dataset or number of GPUs, please use drop_last=True then replace num_val_samples with math.floor(num_val_samples / world_size) * world_size to avoid evaluating duplicated samples.
In my implementation, I used
DistributedSampler(..., drop_last=False)for validation. This is okay whensamples % world_size == 0, so my experiments were fine. However, if this value is non-zero by changing the dataset or number of GPUs, please usedrop_last=Truethen replacenum_val_sampleswithmath.floor(num_val_samples / world_size) * world_sizeto avoid evaluating duplicated samples.