Skip to content

Conversation

@rashi-ops
Copy link

This PR introduces a modular, config-driven data pipeline for the Vision Transformer experiments by separating dataset and dataloader logic into reusable components.

Issues:

  • Hard-coded dataset paths and notebook-specific data handling.
  • Dataset and dataloader logic tightly coupled with training code.

Changes:

  • Added 'dataset/my_dataset_vit' to handle .npy gravitational lensing data.
  • Added 'dataloaders/dataloaders.py' with 'create_dataloaders(cfg)' function.
  • Added Config-driven paths, batch size, and transforms in 'lensid.yaml'.

Previously, dataset and dataloader logic were hard-coded inside the notebook.
This is making reuse and experimentation difficult.
This PR decouples data handling from training, improves reusability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant