xlnstorch.optim.lr_scheduler.LNSReduceLROnPlateau

xlnstorch.optim.lr_scheduler.LNSReduceLROnPlateau#

class xlnstorch.optim.lr_scheduler.LNSReduceLROnPlateau(optimizer, mode='min', factor=0.1, patience=10, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0.0, eps=1e-08)#

An LNS learning rate scheduler that reduces the learning rate of each parameter group when a metric has stopped improving.

See also: torch.optim.lr_scheduler.ReduceLROnPlateau

Parameters:

  • optimizer (LNSOptimizer) – Wrapped optimizer.

  • mode (str, optional) – One of min, max. In min mode, the learning rate will be reduced when the quantity monitored has stopped decreasing; in max mode, it will be reduced when the quantity monitored has stopped increasing. Default: min.

  • factor (float | LNSTensor, optional) – Factor by which the learning rate will be reduced. new_lr = lr * factor. Default: 0.1.

  • patience (int, optional) – Number of epochs with no improvement after which learning rate will be reduced. Default: 10.

  • threshold (float | LNSTensor, optional) – Threshold for measuring the new optimum, to only focus on significant changes. Default: 1e-4.

  • threshold_mode (str, optional) – One of rel, abs. In rel mode, the threshold is a relative change; in abs mode, it is an absolute change. Default: rel.

  • cooldown (int, optional) – Number of epochs to wait before resuming normal operation after lr has been reduced. Default: 0.

  • min_lr (float | LNSTensor | List[float] | List[LNSTensor], optional) – A scalar or a list of scalars defining the lower bound on the learning rate of each parameter group. Default: 0.0.

  • eps (float | LNSTensor, optional) – Minimal decay applied to lr. If the difference between new and old lr is smaller than eps, the update is ignored. Default: 1e-8.

__init__(optimizer, mode='min', factor=0.1, patience=10, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0.0, eps=1e-08)#
Parameters:

  • optimizer (LNSOptimizer)

  • mode (str)

  • factor (float | LNSTensor)

  • patience (int)

  • threshold (float | LNSTensor)

  • threshold_mode (str)

  • cooldown (int)

  • min_lr (float | LNSTensor | List[float] | List[LNSTensor])

  • eps (float | LNSTensor)

Methods

__init__(optimizer[, mode, factor, ...])

get_last_lr()

Return last computed learning rate by current scheduler.

get_lr()

Compute learning rate using chainable form of the scheduler.

is_better(a, best)

load_state_dict(state_dict)

Load the scheduler's state.

state_dict()

Return the state of the scheduler as a dict.

step(metrics[, epoch])

Perform a step.

Attributes

in_cooldown

Contents