Merge pull request #1330 from NNPDF/hotfix_hyperopt_with_overfit

Recover `overfit` option for partitions

doc/sphinx/source/n3fit/hyperopt.rst

@@ -260,7 +260,8 @@ The partitions can be chosen by adding a ``kfold::partitions`` key to the runcar
                 - data_5
 
 The ``overfit`` flag, when applied to one of the partitions, introduces this partition in the
-training data. This is useful for very broad scans where we want to find an architecture which is able to
+fitted data, i.e., the training and validation always include that partition and will work normally.
+This is useful for very broad scans where we want to find an architecture which is able to
 fit, without worrying about things like overlearning which might be a second-order problem.
 
 The ``weight`` (default 1.0) is multiplied with the loss function of the partition for which it is set.

n3fit/src/n3fit/model_trainer.py

@@ -356,12 +356,13 @@ def _model_generation(self, pdf_models, partition, partition_idx):
         splitted_pdf = splitting_layer(full_pdf_per_replica)
 
         # If we are in a kfolding partition, select which datasets are out
+        training_mask = validation_mask = experimental_mask = [None]
         if partition:
-            training_mask = [i[partition_idx] for i in self.training["folds"]]
-            validation_mask = [i[partition_idx] for i in self.validation["folds"]]
+            if partition.get("overfit", False):
+                # If overfitting, don't apply folding masks to the training/validation
+                training_mask = [i[partition_idx] for i in self.training["folds"]]
+                validation_mask = [i[partition_idx] for i in self.validation["folds"]]
             experimental_mask = [i[partition_idx] for i in self.experimental["folds"]]
-        else:
-            training_mask = validation_mask = experimental_mask = [None]
 
         # Training and validation leave out the kofld dataset
         # experiment leaves out the negation