Layers

The xlnstorch package provides a set of layers that support LNSTensor parameters. Whilst you can use the built-in PyTorch layers with LNSTensor inputs, these layers will implement standard floating-point parameters.

Note that some of the layers in this module are only added for completeness since they don’t have any parameters. For example, the nn.LNSDropout layer is equivalent to the standard PyTorch torch.nn.Dropout layer, both will work with LNSTensor inputs, but the nn.LNSDropout layer is implemented for completeness. Any layers that do not have parameters will be denoted in the documentation.

Linear Layers	Note
LNSIdentity	No parameters
LNSLinear
LNSBilinear
LNSLazyLinear

Dropout Layers	Note
LNSDropout	No parameters
LNSDropout1d	No parameters
LNSDropout2d	No parameters
LNSDropout3d	No parameters

Convolutional Layers	Note
LNSConv1d
LNSConv2d
LNSConv3d

Normalization Layers	Note
LNSBatchNorm1d
LNSBatchNorm2d
LNSBatchNorm3d
LNSLayerNorm

Pooling Layers	Note
LNSAvgPool1d	No parameters
LNSAvgPool2d	No parameters
LNSAvgPool3d	No parameters
LNSAdaptiveAvgPool1d	No parameters
LNSAdaptiveAvgPool2d	No parameters
LNSAdaptiveAvgPool3d	No parameters
LNSMaxPool1d	No parameters
LNSMaxPool2d	No parameters
LNSMaxPool3d	No parameters

Recurrent Layers	Note
LNSRNN
LNSRNNCell
LNSLSTM
LNSLSTMCell
LNSGRU
LNSGRUCell

Transformer Layers	Note
LNSMultiheadAttention

Custom Layers

To implement your own custom layers that support LNSTensor parameters, you can subclass the base layer class provided in this module LNSModule.

This base class is a subclass of the standard PyTorch torch.nn.Module and provides the method LNSModule.register_parameter() which is equivalent to PyTorch’s method of registering parameters.

LNSModule	An LNS module that serves as a base class for all LNS layers.
LNSIdentity	An LNS identity layer that does not change the input.
LNSLinear	An LNS linear layer that performs a linear transformation on the input, \(x\).
LNSBilinear	An LNS bilinear layer that performs a bilinear transformation on two inputs, \(x_1\) and \(x_2\).
LNSLazyLinear	An LNS lazy linear layer that performs a linear transformation on the input, \(x\), without initializing the weight and bias parameters until the first forward pass.
LNSDropout	An LNS dropout layer that randomly zeroes some of the elements of the input tensor with a probability \(p\) during training.
LNSDropout1d	An LNS dropout layer that randomly zeroes out entire channels of the input tensor with a probability \(p\) during training.
LNSDropout2d	An LNS dropout layer that randomly zeroes out entire channels of the input tensor with a probability \(p\) during training.
LNSDropout3d	An LNS dropout layer that randomly zeroes out entire channels of the input tensor with a probability \(p\) during training.
LNSConv1d	An LNS 1D convolutional layer that applies a 1D convolution over the input tensor.
LNSConv2d	An LNS 2D convolutional layer that applies a 2D convolution over the input tensor.
LNSConv3d	An LNS 3D convolutional layer that applies a 3D convolution over the input tensor.
LNSBatchNorm1d	An LNS 1D normalization layer that applies a 1D batch normalization over the input tensor.
LNSBatchNorm2d	An LNS 2D normalization layer that applies a 2D batch normalization over the input tensor.
LNSBatchNorm3d	An LNS 1D normalization layer that applies a 1D batch normalization over the input tensor.
LNSLayerNorm	Applies Layer Normalization over a mini-batch of inputs.
LNSAvgPool1d	An LNS 1D average pooling layer that applies a 1D average pooling operation over the input tensor.
LNSAvgPool2d	An LNS 2D average pooling layer that applies a 2D average pooling operation over the input tensor.
LNSAvgPool3d	An LNS 3D average pooling layer that applies a 3D average pooling operation over the input tensor.
LNSAdaptiveAvgPool1d	An LNS 1D adaptive average pooling layer that applies a 1D adaptive average pooling operation over the input tensor.
LNSAdaptiveAvgPool2d	An LNS 2D adaptive average pooling layer that applies a 2D adaptive average pooling operation over the input tensor.
LNSAdaptiveAvgPool3d	An LNS 3D adaptive average pooling layer that applies a 3D adaptive average pooling operation over the input tensor.
LNSMaxPool1d	An LNS 1D maximum pooling layer that applies a 1D maximum pooling operation over the input tensor.
LNSMaxPool2d	An LNS 2D maximum pooling layer that applies a 2D maximum pooling operation over the input tensor.
LNSMaxPool3d	An LNS 3D maximum pooling layer that applies a 3D maximum pooling operation over the input tensor.
LNSRNN	An LNS multi-layer Elman RNN.
LNSRNNCell	An LNS Elman RNN cell.
LNSLSTM	An LNS multi-layer long-short term memory (LSTM) RNN.
LNSLSTMCell	An LNS long-short term memory (LSTM) cell.
LNSGRU	An LNS multi-layer gated recurrent unit (GRU) RNN.
LNSGRUCell	An LNS gated recurrent unit (GRU) cell.
LNSMultiheadAttention	An LNS multi-head attention layer.