Data logger help needed

Community,

I need your help. Probably I’m looking at this problem from the wrong angle.

What am I trying to accomplish:
Building an energy monitor for electricity, water and gas with a Fez Panda I.

How:
By counting the pulses I receive from 2 hall effect sensors and 1 open drain sensor.

Goals:
[ol]
[li]Store the real-time counters (as you see them on your utility meters) in the SRAM of a DS1306 (Done)[/li]
[li]Get real time instant consumption values (in Watt for electricity, in m3/hour for gas and water) (Done)[/li]
[li]Provide the data over serial on request(Done, but…)[/li]
[li]Display real-time data on a 3.6" SPI B/W Display (Done)[/li]
[li]Max resolution should be 1 hour. Meaning, I need the counter values every hour, though I will be polling them every 5 secs under normal circumstances to provide my other components (website) with real time data.[/li]
[/ol]
Problem:
The problem is goal number 4. As soon as the Panda code is running as it should be, it will never be updated anymore (neither code or firmware).
The Fez Panda will be connected over serial to a Fez Cobra. The Fez Cobra will be handling allot of other things like XBee, RF433, IO’s, etc.
So the cobra will be updated regularly with new features and here arises the problem. The time the Cobra will be off-line will be minimal, but I foresee that off-line time could exceed 2-3 hours or more.
Thus, the Panda should have some sort of buffer that in case I don’t poll for data, will keep the data updated for every hour (should be able to cover a new day (I could update the Cobra at 23:59)). When the Cobra comes back to life, it polls the Panda again, and the Panda should provide all the data since the last time The Cobra asked for it.
Initially I was thinking of using a Queue on the Panda, but with only 38kb free memory I suspect that I will run out of memory in no time. Besides that, there’s no way to update items already added to the Queue.
I also considered using an SD Card but the problem here is that my Panda refuses to work with the SD Card.

So, how would you guys solve this?

[quote]I also considered using an SD Card but the problem here is that my Panda refuses to work with the SD Card.
[/quote]

Have you see this? GHI Electronics – Where Hardware Meets Software

Yes, and verified everything 100 times, still doesn’t work. Added a cap as suggested in another thread, but still no go. I gave up on the Panda-SD Card combo.

Protoboard + Dataflash

http://www.sparkfun.com/products/301
http://www.sparkfun.com/products/494
http://www.sparkfun.com/products/7914

The queue is exactly what I should use. Don’t worry about memory, you only queue one measurement each hour. So if you limit the queue to 24 entries, you still cover a whole day…

Thanks Wouter,

I think I’ll go the Queue route for now.

Small problem. How do I prevent that the statement inside the “if” gets fired 1015 times:


		public static void HandleData()
		{
			int x = 0;
			while (true)
			{
				DateTime dt = DateTime.Now;
				if (dt.Second % 59 == 0 & dt.Minute % 59 == 0)
				{
                                        //Here I will be adding to the Queue
					Debug.Print("End of hour " + x++);
				}
			}
		}



public static void HandleData()
		{
			int x = 0;
			int lastAddedHour = -1;
			while (true)
			{
				DateTime dt = DateTime.Now;
				if (dt.Second % 59 == 0 && dt.Minute % 59 == 0 && lastAddedHour != dt.Hour)
				{
                                        //Here I will be adding to the Queue
					Debug.Print("End of hour " + x++);
					lastAddedHour = dt.Hour;
				}
			}
		}

By the way, adding a double && uses short-circuit evaluation so that if the first one isn’t true it won’t evaluate the second condition and so on. Might save some execution cycles as mod is an expensive check.

And why do we use % ?

Keep it simple


if (dt.Hour != lastAddedHour)
{
    // store value in queue
    lastAddedHour = dt.Hour;
}

Wouter,

that is brilliant, thanks.

Eric,

I am not sure I understand how your code is structured, but was wondering if you have considered using the ExtendedTimer class and separate threads for polling the data and writing the log?

Wouter’s suggested modification will certainly execute faster and is a simpler conditional statement, but I think the comparison operation itself will still be executed the same number of times as the original. (The number of times the comparison operation fires will be the number of times the function loops, which may be more or less than every second.) An ExtendedTimer class could be set up to fire the logging thread every hour and the CLR will take care of watching the clock. This will probably have much less overhead than having your application poll the time to check for whether it should write new data. For a discussion of the ExtendedTimer class, you can see Chapter 4 of Kuhlner’s book, which is conveniently available at: www.windowsfordevices.com/.../Kuhner.NETMicroFramework_Ch4_sample.pdf

gg47,

welcome an thanks for taking the time to reply.

I don’t need a timer firing every hour because in normal operation I don’t need the hourly data, but only when the cobra is not polling for data. So if I turn the cobra down, the panda knows and starts adding it’s data to the queue every hour. As soon as the cobra comes back to life, the panda knows this and sends over the contents of the queue.

Greetings.

The queue is a great idea, and if you’re not already doing this I’d suggest you make sure the Panda always adds data to the queue and you have a separate process for the handshake to the cobra that transmits and de-queues data. I think that should make the app logic easier, and really all you have to watch for is out of memory (or past your queue limit if oyu have one) when you’re queueing new data.