Opened 2 weeks ago

Closed 3 days ago

#5881 closed defect (fixed)

Rounding error for real values in external constant eval

Reported by: Jesper Mattsson Owned by: Jesper Mattsson
Priority: major Milestone:
Component: Modelica/FrontEnd Version: trunk
Keywords: Cc:

Description

When constant-evaluating external functions that return real values, how the values are serialized to be passed back to the compiler is controlled by the following define, on line 65 of jmi_evaluator_util.h:

#define JMCEVAL_realFormat "%.16f"

16 significant digits is sometimes not enough to completely describe a double value - sometimes you need 17.

Change History (13)

comment:1 Changed 2 weeks ago by Jesper Mattsson

r14129:

Changed serialization of real return values in constant evaluation of external C functions to use 17 significant digits.

comment:2 Changed 2 weeks ago by Jesper Mattsson

r14130:

Changed serialization of real return values in constant evaluation of external C functions to use scientific notation.

comment:3 Changed 2 weeks ago by Jesper Mattsson

r14134:

Added test of constant evaluation returning a value that would loose precision with the wrong printf directive.

comment:4 Changed 2 weeks ago by Jesper Mattsson

r14145:

Update test "testDGELSX" to use assert_equals, since the bug causing the difference is now fixed.

comment:5 Changed 2 weeks ago by Jesper Mattsson

r14146:

Added change log entry.

comment:6 Changed 10 days ago by Jesper Mattsson

r14181:

Fixed broken Python test.

comment:7 Changed 7 days ago by Jesper Mattsson

r14229:

Merged dev branch to trunk.

comment:8 Changed 7 days ago by Jesper Mattsson

r14230:

Merged to release branch.

comment:9 Changed 4 days ago by Jesper Mattsson

r14242:

Reverted trunk merge.

comment:10 Changed 4 days ago by Jesper Mattsson

r14243:

Reverted release branch merge.

comment:11 Changed 4 days ago by Jesper Mattsson

r14244:

Reverted changes to testDGELSX, since it was not fixed by this change.

comment:12 Changed 4 days ago by Jesper Mattsson

r14245:

Merged dev branch to trunk.

comment:13 Changed 3 days ago by Jesper Mattsson

Resolution: fixed
Status: newclosed

r14261:

Merged to release branch.

Note: See TracTickets for help on using tickets.