Math is wrong for multiplier=1

[jonashaag]

Here

pytorch-gradual-warmup-lr/warmup_scheduler/scheduler.py

Line 44 in 6b5e895

warmup_lr = [base_lr * ((self.multiplier - 1.) * self.last_epoch / self.total_epoch + 1.) for base_lr in self.base_lrs]

it should have the same special case from a few lines above:

if self.multiplier == 1.0:
    warmup_lr = [base_lr * (float(self.last_epoch) / self.total_epoch) for base_lr in self.base_lrs]
else:
    warmup_lr = [base_lr * ((self.multiplier - 1.) * self.last_epoch / self.total_epoch + 1.) for base_lr in self.base_lrs]

Otherwise the calculation will always be:

warmup_lr = [base_lr * ((self.multiplier - 1.) * self.last_epoch / self.total_epoch + 1.) for base_lr in self.base_lrs]
# <=>
warmup_lr = [base_lr * (0 * self.last_epoch / self.total_epoch + 1.) for base_lr in self.base_lrs]
# <=>
warmup_lr = [base_lr * (1.) for base_lr in self.base_lrs]
# <=>
warmup_lr = [base_lr for base_lr in self.base_lrs]
# <=>
warmup_lr = self.base_lrs

[hyunseoki]

good, it works