xlnstorch.viz.precision_sweep_analysis

xlnstorch.viz.precision_sweep_analysis#

xlnstorch.viz.precision_sweep_analysis(op, ideal_op=None, *, precisions=None, x_range=(-1.0, 1.0), y_range=None, steps=101, device='cpu')#

Analyze how different precision levels affect operation accuracy.

Parameters:

  • op (Callable) – The operation to analyze (e.g., torch.mul, torch.add)

  • ideal_op (Callable, optional) – A reference function that computes the exact result using Decimal. If not provided, a default mapping from op to an ideal function is used if available.

  • precisions (List[int], optional) – List of precision values (f parameter) to test. Defaults to [4, 6, 8, 10, 12, 16, 20, 24]

  • x_range (Tuple[float, float]) – Range of x values to test

  • y_range (Tuple[float, float], optional) – Range of y values for binary operations

  • steps (int) – Number of sample points per dimension

  • device (torch.device | str) – Device for computations

Returns:

Dictionary containing error metrics for each precision level

Return type:

Dict[str, Any]

Contents