HnD | decimal rounding problem, Page 1

decimal rounding problem

Posts

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 02:13:55

Hi,

My environment is LLBLGen v2.6 and I am hitting an Oracle database. The columns in the database that are mapped to decimal? types are number(38 ) nullable columns.

I am experiencing a strange problem. A user enters a decimal value into a text box, I parse it and assign it to an entity data member of type decimal?, the value is always rounded to 3 decimal places.

Here is a sample of the code:


                formulation.UpperWeightLimitPct = rowControls.UpperWeightLimitPct.Text.Length > 0 ?
                    decimal.Parse(rowControls.UpperWeightLimitPct.Text) : new Nullable<decimal>();
                formulation.LowerWeightLimitPct = rowControls.LowerWeightLimitPct.Text.Length > 0 ?
                    decimal.Parse(rowControls.LowerWeightLimitPct.Text) : new Nullable<decimal>();
                formulation.NominalWeightPct = rowControls.NominalWeightPct.Text.Length > 0 ?
                    decimal.Parse(rowControls.NominalWeightPct.Text) : new Nullable<decimal>();

Why would the rounding error take place? I've checked the parsed values and the decimal places are preserved. It is when the value is assigned to the entity field (formulation.UpperWeightLimitPct, etc.) that the rounding occurs.

Thanks,

Mike

daelmo
Support Team

Posts: 8245
Joined: 28-Nov-2005

# Posted on: 12-Nov-2008 04:16:04

What LLBLGen runtimeLibraries version are you using? Also, please check this: http://llblgen.com/TinyForum/Messages.aspx?ThreadID=12455

David Elizondo | LLBLGen Support Team

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 15:20:00

daelmo wrote:

What LLBLGen runtimeLibraries version are you using? Also, please check this: http://llblgen.com/TinyForum/Messages.aspx?ThreadID=12455

I'm using

SD.LLBLGen.Pro.DQE.OracleMS.NET20.dll SD.LLBLGen.Pro.LinqSupportClasses.NET35.dll SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 15:31:14

daelmo wrote:

What LLBLGen runtimeLibraries version are you using? Also, please check this: http://llblgen.com/TinyForum/Messages.aspx?ThreadID=12455

I also added the line

to my appSettings in the config file as described in the thread (12455) with no change in behavior.

Walaa
Support Team

Posts: 14993
Joined: 21-Aug-2005

# Posted on: 12-Nov-2008 16:31:25

daelmo wrote:

What LLBLGen runtimeLibraries version are you using?

Please check this thread to know how to get the RTL version: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7725

Which value did you set for the <add key="buildInValidationBypassMode" value="0"/>?

Also when you set <add key="scaleOverflowCorrectionActionToUse " value="0"/>, you didn't get an exception, did you?

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 17:10:54

Walaa wrote:

daelmo wrote:

What LLBLGen runtimeLibraries version are you using?

Please check this thread to know how to get the RTL version: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7725

Which value did you set for the <add key="buildInValidationBypassMode" value="0"/>?

Also when you set <add key="scaleOverflowCorrectionActionToUse " value="0"/>, you didn't get an exception, did you?

I've tried using <add key="buildInValidationBypassMode" value="0"/> and <add key="buildInValidationBypassMode" value="1"/>, both with no success.

The versions of my libraries are:

2.6.8.819 - SD.LLBLGen.Pro.DQE.OracleMS.NET20.dll 2.6.8.911 - SD.LLBLGen.Pro.LinqSupportClasses.NET35.dll 2.6.8.911 - SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll

Walaa
Support Team

Posts: 14993
Joined: 21-Aug-2005

# Posted on: 12-Nov-2008 17:16:16

Also when you set <add key="scaleOverflowCorrectionActionToUse " value="0"/>, you didn't get an exception, did you?

2.6.8.911 - SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll

To minimize the possibilities, would you please try the latest build?

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 17:53:35

Walaa wrote:

Also when you set <add key="scaleOverflowCorrectionActionToUse " value="0"/>, you didn't get an exception, did you?

??

2.6.8.911 - SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll

To minimize the possibilities, would you please try the latest build?

No, I didn't get an exception. The columns these entity fields map to are

number(38)

columns, so there is no problem with any loss of precision.

FYI

Oracle 10g MS Oracle drivers LLBLGen Pro v2.6 Final

flyfisher1952
User

Posts: 23
Joined: 19-Aug-2008

# Posted on: 12-Nov-2008 18:32:28

Walaa wrote:

Also when you set <add key="scaleOverflowCorrectionActionToUse " value="0"/>, you didn't get an exception, did you?

??

2.6.8.911 - SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll

To minimize the possibilities, would you please try the latest build?

I just tried adding the line

FormulationEntity.ScaleOverflowCorrectionActionToUse = ScaleOverflowCorrectionAction.None;

and got the following exception

The scale of value '4.0001' is larger than the scale of the field: '3' and will cause an overflow in the database. \r\nParameter name: UpperWeightLimitPct

This lead me to examine the table definitions again and found that they are number(10,3), hence the problem.

Never mind...