dynamic list - division expression problem (datatype int to float or double)

Posts   
 
    
rracer99
User
Posts: 58
Joined: 11-Mar-2007
# Posted on: 22-Apr-2007 12:31:07   

I am reading a bytes column from the database. If I divide bytes by 1024 * 1024 I get approx. MegaBytes.

If I divide by 1024 * 1024 * 1024, I get approx. GigaBytes. However this will return 0. I am assuming this is happening because the math is applied per - row, and since this is an int field, it is rounding to 0, so the AggregateFunction.Sum is still 0.

How can I apply this expression to get the proper result?



//this works, probably because my samples have bytes greater than 1024 * 1024
fields.DefineField(new EntityField2(ReportTotalsField.TotalBytesMB.ToString(), RawLogsFields.ScBytes / (1024 * 1024), AggregateFunction.Sum), 0);

//this returns 0, is this because each value is rounding down to 0?  How would this be done?
fields.DefineField(new EntityField2(ReportTotalsField.TotalBytesGB.ToString(), RawLogsFields.ScBytes / (1024 * 1024 * 1024), AggregateFunction.Sum), 4);


Thanks!

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39922
Joined: 17-Aug-2003
# Posted on: 22-Apr-2007 14:44:59   

Could you examine the query that's being executed (please enable tracing) and see if it is odd or wrong ? Also, you could try a bitshift operator instead if you're using sqlserver. dividing by 1024*1024 is simply the same as shifting the value to the right by 20.

Frans Bouma | Lead developer LLBLGen Pro
rracer99
User
Posts: 58
Joined: 11-Mar-2007
# Posted on: 23-Apr-2007 04:42:08   

This is the SQL output, I will test the bit operator suggestion next chance. Let me know if you have any other ideas in the meantime:

The statement appears correct. I still suggest my first theory, are the values being round to 0 because they are a decimal?


Generated Sql query: 
    Query: SELECT DISTINCT COUNT(DISTINCT [Logs].[dbo].[raw_logs].[c_ip]) AS [TotalUniqueIPs], COUNT(DISTINCT [Logs].[dbo].[raw_logs].[c_playerid]) AS [TotalSessions], COUNT([Logs].[dbo].[raw_logs].[c_ip]) AS [TotalHits], SUM([Logs].[dbo].[raw_logs].[sc_bytes] / @LO8c74c61) AS [TotalBytesMB], SUM([Logs].[dbo].[raw_logs].[sc_bytes] / @LO2e259f82) AS [TotalBytesGB] FROM [Logs].[dbo].[raw_logs]  WHERE ( ( [Logs].[dbo].[raw_logs].[cs_uri_stem] LIKE @CsUriStem3 AND [Logs].[dbo].[raw_logs].[date] >= @Date4 AND [Logs].[dbo].[raw_logs].[date] <= @Date5))
    Parameter: @LO8c74c61 : Int32. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 1048576.
    Parameter: @LO2e259f82 : Int32. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 1073741824.
    Parameter: @CsUriStem3 : String. Length: 8. Precision: 0. Scale: 0. Direction: Input. Value: "/lagar1%".
    Parameter: @Date4 : DateTime. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 1/1/2007 12:00:00 AM.
    Parameter: @Date5 : DateTime. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 1/1/2008 12:00:00 AM.


daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 23-Apr-2007 07:15:09   

rracer99, I've tested, and something like this should evaluate the expression as decimal (adding .0). However the bitwise way should be more efficient than the division.


//this returns 0, is this because each value is rounding down to 0?  How would this be done?
fields.DefineField(new EntityField2(ReportTotalsField.TotalBytesGB.ToString(), RawLogsFields.ScBytes / (1073741824.0), AggregateFunction.Sum), 4);

David Elizondo | LLBLGen Support Team