Relative Error Bound for ZFP

[treigerm]

Setting ZFP parameters according to llnl/zfp#272 (comment) .

Right now this still violates the error bounds so I am not quite sure why. Observed behavior:

• Tested on "pr", "no2" (Adjust error bound usage #51), "agb" variables
• For "pr" and "no2" around 50% of the pixels exceed the error bound
• For "agb" around 0.2 % of the pixels exceed the error bound, except for the low bound where the error bound is satisfied

I don't think our compute_keepbits function is wrong because it works for bit rounding. Additionally, even increasing the maxprec value did not seem to fix the issue. I thought that maybe ZFP implicitly assumed for some reason that the data is in double precision so I did a run with assuming 12 non-mantissa bits (instead of 9 for single precision) but that did not fix the issue either, although it reduced the percentage of pixels exceeding the error bound.

[juntyr]

The docs specify that when n = p + q, where n is the number of bits in the float and p is the number of bitplanes, the relative error is bounded by 2^(q-23) or 2^(q-52).

If that matches our implementation, then this would be a bug. Perhaps you can describe the problem to Peter in the original issue (I’m unfortunately a bit swamped at the moment)?

[treigerm]

Yep, that's what we're doing right now. Will go and raise it in the original issue.