WarmupScheduler

class WarmupScheduler(optimizer, step_size, n_warmup_steps, last_epoch=- 1, verbose=False)

Parameters

• optimizer (Optimizer) – Wrapped optimizer.

• step_size (int) – Period of learning rate decay.

• n_warmup_steps (int) – Number of steps for the warmup phase

• last_epoch (int) – The index of last epoch. Default: -1.

• verbose (bool) – If True, prints a message to stdout for each update. Default: False.

Example

>>> scheduler = WarmupScheduler(optimizer, step_size=30, n_warmup_steps=100)
>>> for epoch in range(100):
>>>     train(...)
>>>     validate(...)
>>>     scheduler.step()