Skip to content

Commit

Permalink
ExecutionTest::UnaryHalfOpTest#AcosHalf: Update tolerance (microsoft#…
Browse files Browse the repository at this point in the history
…6690)

Update tolerance for ExecutionTest::UnaryHalfOpTest#AcosHalf test.

Enables implementations to calculate `acos` for fp16 type by converting
to fp32, doing fp32 math, and then converting back to fp16 using
round-to-nearest-even conversing (RTNE) per D3D11 spec. For more details
please see issue microsoft#6179.

As mentioned in the linked issue, for these floating point tests a fixed
point tolerance does not really make sense. It should vary based on the
magnitude of the expected value. But we are already using this approach
in many similar test cases and the simplest fix now is to update the
tolerance to accommodate the fp32-to-ft16 conversion.

Fixes microsoft#6179
  • Loading branch information
hekota authored Jun 12, 2024
1 parent 4353db3 commit 84d39b6
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tools/clang/unittests/HLSLExec/ShaderOpArithTable.xml
Original file line number Diff line number Diff line change
Expand Up @@ -1562,7 +1562,7 @@
</Row>
<Row Name="AcosHalf">
<Parameter Name="Validation.Type">Epsilon</Parameter>
<Parameter Name="Validation.Tolerance">0.0008</Parameter>
<Parameter Name="Validation.Tolerance">0.0010</Parameter>
<Parameter Name="ShaderOp.Text"> struct SUnaryFPOp {
float16_t input;
float16_t output;
Expand Down

0 comments on commit 84d39b6

Please sign in to comment.