Some years ago I designed a messaging protocol on top of TCP/IP sockets. It stemmed from earlier work I did in London when working in the financial “City” of London.
The protocol was quite sophisticated and supported sending and receiving “messages” asynchronously (the code was all C#, back in 2009).
The developer never saw or dealt with low level sockets, but sent/received “messages” with optional forms of compression and ring buffering to facilitate minimizing IO (sending many messages at once resulted in a single network write with multiple messages in the write).
Messages were C# objects derived from a base and the IO interface dealt purely with these messages, all of the conversion from an object to a raw byte buffer was very efficient (very fast serialization) and hidden from the developer, the core goal of this was to deliver performance akin to SQL Server’s TDS protocol which is very fast and suited to transferring large volumes of table-like data very efficiently (TDS is closed source and written in C).
Anyway, is there anything like this around to today? Far too many examples I see of .Net networking code still deal with the raw sockets and byte buffers with little further abstraction. Connections were established and remained open as long as desired (one was not forced to adopt a stateless HTTP approach).
I’m interested in doing something with Endpoint where I can create rich message based systems and don’t want to re-invent the wheel so to speak.
Connectionless DTLS has been one of my goto’s lately. Secure, very performant (often used for secure voip), and with no TCP connection overhead. You can bridge to MQTT/Kafka/etc at the leading edge of the cloud.
The truth is, the answer to ‘which one’ is ‘it depends’. LoraWAN vs Sigfox vs wifi vs LTE-M/nb-IoT vs satellite are all going to drive different choices based on their cost, MTU size, latency, security, statefulness, and reliability.
Not heard of MQTT - its been like thirteen years since I did any kind of networking work. It looks quite interesting but of course has all that JSON baggage. You want to send a couple of bytes but end up sending rather more than that so not sure how it performs when data rates are important.
You mention RPC and of course Microsoft pushed “remoting” rather heavily in the early days of .Net but that never really became popular for IPC.
The work I did let us define classes all derived from a base that became a message header once serialized. That header had a numeric ID that identified the message “type” it was this that was used to identify how to serialize and deserialize, so a message consisting of (say) 512 raw bytes or multiple int/short/long/float fields contained only the raw data bytes. The ID was a key into a dictionary of Types.
So I could code stuff like:
var report_msg = new ReportMsg(account, quote, equity_class); // or whatever the message represnts.
channel.SendMessage(report_msg);
Under the hood the connection was persistent (until one wanted to close it) and there was internally generated heartbeat messages (invisible to the application coder) and this which was used to robustly detect sudden loss of socket connection or even cable’s being pulled out and so on.
The serialization was the key to high performance and we achieved a serialization rate of a million objects per second on a single core I7 3960X.
Granted, this was for a financial markets where very fast and efficient message densities were a core goal, this is typically not the case with IOT.
Hi, yes I stumbled upon this yesterday, very interesting, I don’t think that project was around in 2010 when I was working on the networking design. I was impressed at the performance of TDS, my managed design wasn’t quite a fast despite having very high data density and a minimum IO model.
The minimum IO was important, early research showed the huge cost of socket writes, on Windows anyway.
A single write of say 4K cost far less CPU time than four writes of 1K, this led to a design where we’d configure a max buffer storage time. As data was sent over a channel, it was written to a ring buffer and a timer started. Further messages were appended to the ring buffer and then if either the buffer was full or the time expired, whatever was in the buffer was sent in a single socket write.
The protocol therefore broke up and reassembled messages as needed, the application code only ever saw complete fully formed messages, the message’s length was in the common message header so this was a simple thing to do.
There was also a priority concept where an urgent message could be sent even if there was a lot of pending data about to be transmitted, the urgent message was sent immediately.
Yes it is very similar to Nagle at the packet level. We also provided a “now” bool argument in send functions which would force an immediate send irrespective of buffer free space or timeouts.
It looks quite interesting but of course has all that JSON baggage.
MQTT as a protocol has no JSON baggage. It is completely binary. Depending on how you use the protocol the payload could contain JSON. For example, AZURE IoT uses MQTT and the payloads are JSON. The MQTT protocol has no preset payload requirements, it is up to the user what is in the payload. The message headers for MQTT can be quite small if you choose small Topic and Subscription names.
It is my understanding the MQTT is probably one of the most popular Pub/Sub protocols out there.