flambe.optim.linear

Module Contents

class flambe.optim.linear.WarmupLinearScheduler(optimizer, warmup: int, n_steps: int)

Bases: flambe.optim.scheduler.LambdaLR

Linear warmup and then linear decay.

Linearly increases learning rate from 0 to 1 over warmup training steps. Linearly decreases learning rate from 1. to 0. over remaining n_steps - warmup steps.

This scheduler is generally used after every training batch.

lr_lambda(self, step: int)

Compue the learning rate factor.

Parameters:step (int) – The current step. Could be training over validation steps. Returns:The output factor Return type:float