Issue with the precision of double variables

Hello, I tried the following simple program on a Fez Panda II(SolutionReleaseInfo.solutionVersion:

using System;
using System.Threading;

using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;

using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.FEZ;
using GHIElectronics.NETMF.System;

namespace FEZ_Panda_II_Application1
    public class Program
        public static void Main()
            double t = 0.1;
            for (; t <= 1.0; t += 0.1)
                Debug.Print(t.ToString() + " ;;; "+(1.0/MathEx.Sin(60 * 2 * MathEx.PI * t)).ToString());




Its output is

0.10000000000000001 ;;; -680468306383140.38
0.20000000000000001 ;;; -340234153191570.19
0.30000000000000004 ;;; 102018682038146.86
0.40000000000000002 ;;; -170117076595785.09
0.5 ;;; -46384909705495.305
0.59999999999999998 ;;; -26853447514940.621
0.69999999999999996 ;;; -18896591278204.434
0.79999999999999993 ;;; -14577253497137.445
0.89999999999999991 ;;; -11865146580648.072
0.99999999999999989 ;;; -10003912542101.846

…look at the values of t.

At first I thought it was just a bug with ToString(), but then I tried it with matlab:


outputs is:

ans =


Is this a bug or am I expecting too much precision out of it?

Have you tried this on the emulator?

Hi Gus,
I can’t really test it with the emulator because it throws the following exception right after the first bracket.

'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.1\Assemblies\le\mscorlib.dll', Symbols loaded.
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.1\Assemblies\le\Microsoft.SPOT.Native.dll', Symbols loaded.
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.1\Assemblies\le\Microsoft.SPOT.Hardware.dll', Symbols loaded.
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.1 SDK\Assemblies\le\FEZPanda_II_GHIElectronics.NETMF.FEZ.dll'
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.1 SDK\Assemblies\le\GHIElectronics.NETMF.System.dll'
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.1 SDK\Assemblies\le\GHIElectronics.NETMF.Hardware.dll'
'Microsoft.SPOT.Emulator.Sample.SampleEmulator.exe' (Managed): Loaded 'C:\Users\geancarlo\Documents\Visual Studio 2010\Projects\FEZ Panda II Application1\FEZ Panda II Application1\bin\Debug\le\FEZ Panda II Application.exe', Symbols loaded.
    #### Exception System.InvalidOperationException - 0x00000000 (2) ####
    #### Message: FEZ Panda II not detected!
    #### GHIElectronics.NETMF.FEZ._Checker::.cctor [IP: 000c] ####
An unhandled exception of type 'System.InvalidOperationException' occurred in FEZPanda_II_GHIElectronics.NETMF.FEZ.dll

Additional information: FEZ Panda II not detected!

(I have set the transport to emulator)


Ok, I removed references to GHI :

double t = 0.1;
            for (; t <= 1.0; t += 0.1)
                //Debug.Print(t.ToString() + " ;;; "+(1.0/MathEx.Sin(60 * 2 * MathEx.PI * t)).ToString());

this outputs the same values for t

Try this:

float t = 0.1F;
            for (; t <= 1.0; t += 0.1F)
                Debug.Print(t.ToString());// + " ;;; " + (1.0 / MathEx.Sin(60 * 2 * MathEx.PI * t)).ToString());

Ok, emulator and panda both output the same…


This is interesting. I personally do not know the answer.

You can’t represent 0.1 with a float or double without having an error.

Check this site: IEEE-754 Analysis

Enter a value of “.1”, then copy the Binary64 string and paste it as value. You’ll see it will represent this number instead: “1.000000000000000055511151231257827021181583404541015625E-1”

Round to nearest value and you’ll get “1.0000000000000001E-1”

If you do not want that, use an integer and increment with 1 instead of .1, then do “6 * 2 * MathEx.PI * t” instead of “60 * 2 * MathEx.PI * t”

This is why the decimal type was invented. Not sure if it exists in .net mf.

Nope. VS will show a decimal type in Intellisense but it’s either barely implemented or not really at all. It’s useless. It doesn’t even support addition.

I studied & mentioned this several months ago. From as much as I can tell the precisions of float and double are the same and neither even comes close to the precision defined in the .NET docs. But, what I can’t figure out is if this is actually the case or if VS is just making it appear this way through the debugger. I’ve found several cases where people have converted code written to use doubles to then use floats and gained significant performance increases. So, my guess is that there is a difference and in fact doubles may actually be 64-bit floats. But, it’s hard to know for sure. I searched source and couldn’t find anything that defines the base types. So, I assume this is done at a lower level that isn’t quite open source. I’d really love to see someone at Microsoft write up something about this.

it looks like double is 64bit, and float 32bit.

On embedded device, I always try to avoid float or double and stick with integer math. Of course when calculating Sin and Cos, it is much easier to use float or double.

How did you determine this? Through the docs or experimentation? Through experimentation I haven’t been able to show anything close to 64bit precision.

The error of the “double” ggg showed us here can only occur if double is a 64bit floating point number.

If it was a 32 bit number the error would have been .000000001490116119384765625

My bad. According to the Microsoft docs a double has the range of -3.4 × 10^38 to +3.4 × 10^38 with 15-16 bits precision. I had forgotten that last part and was trying to get something closer to 38 decimal places. :-[

@ helsywarner - FYI, decimal is not supported with the NETMF.

Judging from his/her other postings on the forum on old subjects they are simply trying to promote a website. I’ve marked the posts as spam.