learning rate not match?

[ifsheldon]

In the link, I see that the learning rate (of student?) starts from 0.5, but if I understand your code correctly, the learning rate should be 0 during the first 3000 steps as when training CIFAR10 model, you specified the num_wait_step to be 3000. Also, you specified the num_warmup_steps to be 5000. Can you please explain a bit? Thanks!

[kekmodel]

This is the result of the experiment before code modification. I'm going to post a new test with improved performance.