Dynamics AX
  RSS Feed  LinkedIn  Twitter
Want to turn you're data into a true asset? Ready to break free from the report factory?
Ready to gain true insights that are action focused for truly data informed decisions?
Want to do all of this across mutliple companies, instances of Dynamics and your other investments?
Hillstar Business Intelligence is the answer then! (www.HillstarBI.com)

Hillstar Business Intelligence for Microsoft Dynamics AX and NAV on Mobile, Desktop, Tablet

Let us prove to you how we can take the complexity out of the schema and truly enable users to answer the needed questions to run your business! Visit Hillstar Business Solutions at: www.HillstarBI.com

Friday, November 25, 2005

Some un-Real Precision?

Recently I was helping a fellow developer out on an issue. He was working with some basic code that added Real variables together. The precision for these variables was at the point 8th position of the decimal level. (ex: 99.99999999). When working with Real in X++ it is suppose to be able to handle a precision depth of 12 positions. Well as he took 99.99999999 and added it to 99.99999999 the result of the Real variable that stored the sum of these numbers showed up as: 100.

Very strange this was, and confusing, frustrating. So we took this a step further, and added another 99.99999999 to the given now 100 value variable. From this point the variable's value became 200?

After further investigation, it seems that X++ loses it's precision on Real variables after the 6th position. This can be a problem when trying to do exact percentages, but for what we were trying to do, shorting the precision to the 4th position made the sumaztion work correctly. So word to the wise, don't trust the precision of Real's in X++ past the 6th position. Maybe this will change with the next release, and until then, hopefully this will help another developer wasting hours on an issue that should not exist.

Find a job at: www.DynamicsAXJobs.com


Anonymous Anonymous said...

I have come across the rounding and precision issue before. I think the real problem is not the number of digits after the decimal sign, it’s the overall number of digits. Axapta handles precision up to 16 digits. I have posted a job here to demonstrate the point.

7:43 AM  
Blogger brandon said...

Thanks for your comment and posting axgeek!

If you would like I would love to invite you to join this blog, and let you post your code, etc. here. I like your work, it looks very good!

12:34 PM  

Post a Comment

<< Home

Copyright 2005-2011, J. Brandon George - All rights Reserved