flambe.optim.linear
¶
Module Contents¶
-
class
flambe.optim.linear.
WarmupLinearScheduler
(optimizer, warmup: int, n_steps: int)[source]¶ Bases:
flambe.optim.scheduler.LambdaLR
Linear warmup and then linear decay.
Linearly increases learning rate from 0 to 1 over warmup training steps. Linearly decreases learning rate from 1. to 0. over remaining n_steps - warmup steps.
This scheduler is generally used after every training batch.