Missing tecnical data (max speed)

Archive
21

Well… Of course there is an answer. Either GHI allows the legal posting of such results or it doesn’t. However, in regards to whether there is actually any meaning to such results then there most likely is no such answer that really has meaning.

Something similar to following is the type of fine-print in a EULA I’m referring to…

[quote]Performance or Benchmark Testing. You may not disclose the results of any benchmark test of either the Server Software or Client Software for Microsoft SQL Server, Microsoft Exchange Server, Commerce Server, Host Integration Server, Internet Security and Acceleration Server, BizTalk Server, Application Center, Microsoft Transaction Server, Microsoft Message Queue Server, Microsoft Internet Information Server, or Microsoft Proxy Server to any third party without Microsoft’s prior written approval.
[/quote]

22

@ Elmue, ya i hear ya man. I dont know why we are not getting through. when people do embedded long enough what you are describing is one of the first test they do. I hope GHI does not misunderstand that we are not making any kind of accusation that “their” or “.netmf” is slow. we just simply want to see some performance regardless if its native or interpreted.

23

That should be closer to 10.000.000 / second… :wink:

24

With any testing technique you have to be very careful to ask yourself what the test is really telling you else you will jump to the wrong conclusions. Take a look at the large numbers of benchmarks used on PCs, all of them measure ‘something’ but what does the test really tell you. Hard drives have RPM ratings and ratings for the average random seek time (how long to get first byte of data read basically from random location on the platter(s)). I was buying a HD for an all-in-one touch screen PC recently and was not overly concerned with those ratings, I wanted a quite HD and ‘slower’ HDs are typically quieter.

But what does this test really tell you? I have a modern PC setting here in front of me and a test like this tells me nothing about it whatsoever. On a modern uC it tells you almost nothing of value either unless your application entails bit-banging something. Modern uCs have built in hardware for serial ports, memory control, USB, PWM, etc, etc; you can’t find out anything about how easy/fast your interaction with these peripherals might be by toggling a bit. So all the things I might have been worried about bit-toggling to do on a Z80 or 6502 are unimportant now as it is all done in HW. The interaction between the HW and SW is what is more important and the rate of toggling a pin won’t shed any light on that.

I’m not saying that tests/ratings are unimportant but rather you have to know what it is telling you to judge the result of such a test. I would submit that toggling a pin, while it may be a ‘classical’ test, does not provide an informative insight into a modern uC.

25

I have to respectfully disagree with you here.
It sheds a wealth of information. Many guys here seem to be using a comparison using the PC and hardware peripherals. That is not what we are talking about here.

Maybe if i try to explain it in a different way it will shed some light on what we are talking about.

Lets take a uP with a hardware SPI port. If i do something like WriteSPI(x) this will tell me nothing about how my code gets interpreted or the speed in which this happens. The code simply loads the value into the SPI register and the uP takes it from there. The uP will shift out the bits on its own and there is no “code” running that does this. so it does not reveal what we are looking for.

What we are talking about is how fast the JIT can perform the most mundane operations. Bit twiddling is of the slowest. when you do it this way you are actually running code line by line to toggle the pin high and low. This will give you a general idea of how fast the slowest operation can be.

Another example or maybe the one i should have used to start with would be comparing compiler A to compiler B. When doing this one would typically do something like this this right off the bat.

Create a while loop, and in that loop you start off by pulling a pin high. then start a for loop say 1 to 10k and in that lop you do some math operations. then pull pin low. Connect your scope to the port pin and see how long it takes compiler A do it, then try it on compiler B. What this will show you is how each compilers generated code will take. Sure, you could look at the assembly and count the cycles, but this is faster. One compilers output will toggle the pin faster than the other. I have done this may times over the years and you would be amazed at how well it weeds out the crappy compilers over the better ones.

int x = 10;
int y = -2345;
int z = 24059;
double q;
int t;

while (1)
{
Pin1(high);

for(t=0;t<10000;t++)
{
    q += t - ( (x + y) / (y + z) ) + 1 * t;
}

Pin1(Low);

}

Though this is straight line code as i call it. Its not in a RTOS environment. One would do something similar to test one RTOS to another. How efficient is one RTOS to another? doing bit banging test like this and thread swapping quickly will reveal all this to the coder.

Doing tests using hardware peripherals reveals nothing of what we are looking for.
Now that i have bored the crap out of you :wink: the last example might just bring this home.

Lets say some other company wanted to come out with something to compete with MS .NET.
the competitors is called .LINXMF.

As a coder how would you compare the throughput of these two platforms ?
The only way to do it is to force it to chug through code. Like bit twiddling and the example i gave above. To write an example say to the SPI port using .NETMF or .LINXMF the results would more than likely be the same as they are both writing to the hardware peripheral of the uP.

26

The problem here is that there are many many different benchmarks that will give vastly different results. There’s no single “number of source lines per second” or even “number of operations per second” number that can be given.

For example, you might get a vastly different “number of loops/sec” for code like this:


var cnt = 0;

for( var i = 0; i < 10000; i++ )
    cnt++;

… than you would for pin-toggling code. As for your nested-loops example, a REALLY clever compiler would realize that q isn’t used anywhere, and would optimize out the inner loop completely…

That’s why I said, the only way to know how fast NETMF is on your particular workload is to run your particular workload on NETMF and see. You can’t look at someone else’s workload and extrapolate your expected performance from that. This is the problem with microbenchmarks in general, and in particular when comparing systems with vastly different implementations.

27

I understand your point. Elmue is not looking for a precise number or a “benchmark”.
he was just trying to get a general idea. I am sure he knows that doing things one way or another will change the outcome.