Skip to content

Summation of float values is not precise enough in the calculation of variance/std #25

@ajcr

Description

@ajcr

The rolling summation of float values is not precise enough in the calculation of variance/std.

>>> import rolling
>>> list(rolling.Std([0,1,1,1], 3))
[0.5773502691896258, 7.450580596923828e-09]

The first value is correct. The second value 7.450e-09 is small, but should be 0. It is not close to 0 using the default tolerances in math.isclose() for example.

This is related to #20 which caused the variance to drop below 0 (when it should have been 0).

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions