xlnstorch.LNSTensor#

An xlnstorch.LNSTensor is a wrapper for a torch.Tensor that stores an ‘internal representation’ Tensor object along with an LNS base.

Mathematical Context and Internal Representation#

We represent a non-zero real number \(x\) in the LNS (Logarithmic Number System) using the following scheme:

Let \(B\) denote the chosen base and let

\[x = (-1) ^ {s_x} \cdot B^{X}\]

where:

\[\begin{split}X = \log_B |x| \text{ and } s_x = \begin{cases} 0 & \text{if } x \ge 0 \\ 1 & \text{if } x \le 0 \end{cases}\end{split}\]

We can pack the logarithm and sign into a single int64 value as follows:

\[x' = (\mathrm{round}(X) \ll 1) + s_x\]

Since we are quantising the logarithm, we can represent it as an integer value. It is useful to choose a base close to 1, to ensure that the quantisation does not lose too much precision. This is typically done by choosing a base of the form \(B = 2^{2^{-f}}\) for some integer \(f\), where \(f\) is the number of fractional bits in the LNS representation.

The internal representation of an LNSTensor encodes the LNS value as an integer, but for compatibility with PyTorch’s autograd system, it is stored in a tensor of type torch.float64. This allows gradients to be computed and propagated correctly during backpropagation, as PyTorch’s autograd does not support integer tensors for gradient computation.

Although the storage type is a floating point, the actual values represent integers. This means that before performing any bitwise operations or integer-specific manipulations, the values must be explicitly converted back to integers. Failing to do so can result in incorrect behavior, since bitwise operations on floating point types are not valid and may produce unexpected results. Note that if you are not implementing custom functionality, you typically do not need to worry about this, as the library handles these conversions.

This design ensures that LNSTensor objects can fully participate in PyTorch’s computation graph and benefit from automatic differentiation, while still maintaining the integrity of their integer-based internal encoding for LNS arithmetic and bit-level operations.

LNSTensor class reference#

class xlnstorch.LNSTensor#

There are two ways to create a tensor currently.

To create a tensor with pre-existing data, use xlnstorch.lnstensor().
If necessary, use the xlnstorch.LNSTensor() constructor directly, although the use of this is discouraged.

`LNSTensor.get_internal_tensor`	Converts an array-like floating point value to an LNSTensor and returns the internal packed representation for a given base.
`LNSTensor.base`
`LNSTensor.lns`	The packed representation that does carry gradients.
`LNSTensor.value`	Decode the packed integers back to ordinary floating-point numbers.
`LNSTensor.grad`	The gradient of the LNSTensor, if it has been computed.
`LNSTensor.shape`	Returns the shape of the LNSTensor.
`LNSTensor.ndim`	Alias for `dim()`.
`LNSTensor.requires_grad`	Returns whether the LNSTensor requires gradients.
`LNSTensor.grad_fn`	Returns the function that created this LNSTensor, if it was created by an operation that has a gradient function.
`LNSTensor.view`	Returns a new tensor with the same data as this LNSTensor but with a different shape.
`LNSTensor.contiguous`	Returns a contiguous copy of the LNSTensor in memory.
`LNSTensor.repeat`	Repeats the tensor along the specified dimensions.
`LNSTensor.item`	Returns the value of the LNSTensor as a Python number.
`LNSTensor.size`	Returns the size of the LNSTensor along a specified dimension or all dimensions.
`LNSTensor.numel`	Returns the total number of elements in the LNSTensor.
`LNSTensor.backward`	Computes the gradients of the current LNSTensor with respect to the graph leaves.
`LNSTensor.broadcast_to`	Broadcasts `self` to the shape `shape`.
`LNSTensor.expand`	Expands `self` to the shape `sizes`.
`LNSTensor.to`	Converts the LNSTensor to a specified device.
`LNSTensor.dim`	Returns the number of dimensions of the LNSTensor.
`LNSTensor.clone`	Returns a copy of the LNSTensor with the same data and base.
`LNSTensor.squeeze`	Returns a new LNSTensor with all specified dimensions of size 1 removed.
`LNSTensor.unsqueeze`	Returns a new LNSTensor with a dimension of size one inserted at the specified position.
`LNSTensor.detach`	Returns a new LNSTensor that is detached from the current computation graph.
`LNSTensor.requires_grad_`	Sets the requires_grad flag for the LNSTensor in-place.
`LNSTensor.numpy`	Converts the LNSTensor to a NumPy array.
`LNSTensor.xlns`	Converts the LNSTensor to an xlns type.
`LNSTensor.add`
`LNSTensor.add_`
`LNSTensor.sub`
`LNSTensor.sub_`
`LNSTensor.mul`
`LNSTensor.mul_`
`LNSTensor.div`
`LNSTensor.div_`
`LNSTensor.pow`
`LNSTensor.pow_`
`LNSTensor.neg`
`LNSTensor.neg_`
`LNSTensor.abs`
`LNSTensor.abs_`
`LNSTensor.sqrt`
`LNSTensor.sqrt_`
`LNSTensor.square`
`LNSTensor.square_`
`LNSTensor.reciprocal`
`LNSTensor.reciprocal_`
`LNSTensor.sign`
`LNSTensor.sign_`
`LNSTensor.positive`
`LNSTensor.sum`
`LNSTensor.prod`
`LNSTensor.transpose`
`LNSTensor.equal`
`LNSTensor.eq`
`LNSTensor.ne`
`LNSTensor.ge`
`LNSTensor.gt`
`LNSTensor.le`
`LNSTensor.lt`
`LNSTensor.isclose`
`LNSTensor.allclose`
`LNSTensor.any`
`LNSTensor.all`
`LNSTensor.sort`
`LNSTensor.argsort`
`LNSTensor.kthvalue`
`LNSTensor.maximum`
`LNSTensor.minimum`
`LNSTensor.tanh`
`LNSTensor.sigmoid`

xlnstorch.LNSTensor

Contents

xlnstorch.LNSTensor#

Mathematical Context and Internal Representation#

LNSTensor class reference#