How to split dataset into train and valid?

[jexterliangsufe]

My train dataloader length is equal to valid dataloader length. Why?
In nf codes, L329 create self.val_size, but not used in creating datamodule L336

neuralforecast/neuralforecast/common/_base_model.py

Lines 329 to 343 in 90b6fd0

	self.val_size = val_size
	self.test_size = test_size
	is_local = isinstance(dataset, TimeSeriesDataset)
	if is_local:
	datamodule_constructor = TimeSeriesDataModule
	else:
	datamodule_constructor = _DistributedTimeSeriesDataModule
	datamodule = datamodule_constructor(
	dataset=dataset,
	batch_size=batch_size,
	valid_batch_size=valid_batch_size,
	num_workers=self.num_workers_loader,
	drop_last=self.drop_last_loader,
	shuffle_train=shuffle_train,
	)

Then pytorch_lightning.Trainer is called in L357 and does not use val_size if is_local=True

neuralforecast/neuralforecast/common/_base_model.py

Line 357 in 90b6fd0

trainer.fit(model, datamodule=datamodule)

Also, In TimeSeriesDataModule, the function train_dataloader and val_dataloader use the same dataset(both self.dataset).

neuralforecast/neuralforecast/tsdataset.py

Lines 394 to 412 in 90b6fd0

	def train_dataloader(self):
	loader = TimeSeriesLoader(
	self.dataset,
	batch_size=self.batch_size,
	num_workers=self.num_workers,
	shuffle=self.shuffle_train,
	drop_last=self.drop_last,
	)
	return loader
	def val_dataloader(self):
	loader = TimeSeriesLoader(
	self.dataset,
	batch_size=self.valid_batch_size,
	num_workers=self.num_workers,
	shuffle=False,
	drop_last=self.drop_last,
	)
	return loader

Is this a mistake or I use nf in a wrong way?

---

neuralforecast=1.7.2
pytorch_lightning=2.2.5

[jexterliangsufe]

I made a mistake. self.val_size used in _base_windows.py and split train/valid/predict windows correctly.

[samosun]

Hi, I have the same confusion. Correct me if I'm wrong, is the train/validation set divided on each batch?

[jexterliangsufe]

Yes.