Scale overflow correction truncates non-overflowing double value

Posts   
 
    
Posts: 30
Joined: 19-Dec-2022
# Posted on: 16-Nov-2023 18:23:30   

Hi,

I'm sorry, but unfortunately I just stumbled over another odd behavior, but in a totally different area: We have a table "Test" with a column with the datatype Number(12,3) with the name "Duration". The resulting Entity "TestEntity" has a column "Duration" with double a C# type. Now I do the following:

var test = new TestEntity
{
    Duration = 613.444d
};

Console.WriteLine(test.Duration);

In LLBLGen 5.8.3 the output is, as I would expect, "613.444". But the odd thing is, in LLBLGen 5.10.1 (our current used version), the output is "613,443". That is not expected since 613.444 should not trigger an overflow at all. (It also let some integration tests fail wink )

Can you help us with this?

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 17-Nov-2023 07:11:04   

You need to switch ValidatePrecisionScaleSingleDoubleValues off.

Please read:

Defining the Scale overflow correction action to use

Posts: 30
Joined: 19-Dec-2022
# Posted on: 17-Nov-2023 09:45:15   

This would deactivate this behavior, yes. But why did it work in 5.8.3? We also using the scale correction currently, because any AVG could easily result in a scale overflow and so far the framework is handling this for us. Deactivating it would mean handling it manually via a round or truncate in SQL. But in 5.8 it worked nicely for us, so that changed?

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 17-Nov-2023 09:48:52   

Yes, this was a change starting from v.5.8.4

From the changelog:

We've changed the ScaleOverflowCorrectionAction.Truncate behavior for Double/Single typed entity fields when the value causes a scale overflow: If the value for some reason causes a scale overflow, we'll either truncate or round based on the set ScaleOverflowCorrectionAction action but won't throw an exception if this happens when ScaleOverflowCorrectionAction is set to None. Additionally, if the scale is bigger than 15, we will ignore the correction action. By their nature, double and single fields aren't precise and don't contain scale information, making truncating often fail. The methodology we used, string-based truncation, caused problems with double values which resulted in scientific notations and therefore the truncation functionality didn't work.

Posts: 30
Joined: 19-Dec-2022
# Posted on: 17-Nov-2023 13:17:43   

Ok, thanks very much. I also found (a bit too late) https://www.llblgen.com/tinyforum/Thread/27448 which explains the problem.

Posts: 30
Joined: 19-Dec-2022
# Posted on: 17-Nov-2023 13:23:31   

By the way: RuntimeConfiguration.Entity.SetScaleOverflowCorrectionActionToUse(ScaleOverflowCorrectionAction.Round) sets this setting only for Adapter. I don't know if that's on purpose or if a "EntityCore<IEntityFields>.ScaleOverflowCorrectionActionToUse = action;" is missing in this method.

The same applies for all RuntimeConfiguration.Entity.xyz() methods.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39617
Joined: 17-Aug-2003
# Posted on: 17-Nov-2023 15:03:28   

Markus@Init wrote:

By the way: RuntimeConfiguration.Entity.SetScaleOverflowCorrectionActionToUse(ScaleOverflowCorrectionAction.Round) sets this setting only for Adapter. I don't know if that's on purpose or if a "EntityCore<IEntityFields>.ScaleOverflowCorrectionActionToUse = action;" is missing in this method.

The same applies for all RuntimeConfiguration.Entity.xyz() methods.

That is indeed a small issue. We use EntityBase2.ScaleOverflowCorrectionActionToUse = action;, however it was understood that as the setting is defined on the EntityCore<T> class it would be set for both, but looking at the code now, I realize that is of course not the case flushed Not the finest moment of my former self smile

Hmm. Not an easy fix tbh.. (as we have to determine what to call and not sure if we can determine that) These methods are merely a set of convenience methods, as they all call into EntityBase2.<methodname>

We'll see what we can do about this in the next version

Frans Bouma | Lead developer LLBLGen Pro