Json Deserialized example

I try to play with JsonConverter to Deserialized object but I have got:
#### Message: You must provide an instance factory if you want to populate objects that have arrays in them
My string I want to deserialized is:
{"coord":{"lon":1.06,"lat":49.55},"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}]}
My object is:

class WeatherRoot
    {
        public coord coord { get; set; }
        public ArrayList weather { get; set; }

        WeatherRoot()
        {
            weather=new ArrayList();
        }
    }

I think problem is “weather” object which is an array but I can’t understand what I must do. I f someone has got a sample, it would be nice.

I can’t try as I do not have the board yet, but I would define a Weather Class for id , main, description icon and use

public List { get; set; }
instead of
public ArrayList weather { get; set; }

then WeatherRoot weather = JsonConvert.DeserializeObject(YourString);

Thanks for advice but List doesn’t exist.

Hi there @Bauland! Yes, this is a known issue and I’ll explain why it happens down below, but here’s what you need to do to get around it:

There’s a bug in array introspection in the interpreter. To work around it, given an object that includes arrays of scalars, strings, or classes :

public class TestClass
{
    public int[] intArray;
    public string[] stringArray;
}

You would deserialize it like this:

var obj = (TestClass)JsonConverter.DeserializeObject(stringValue, typeof(TestClass), CreateInstance);

Where CreateInstance is a function that will recognize the member name and allocate an array of the correct length:

private static object CreateInstance(string path, string name, int length)
{
    if (name == "intArray")
        return new int[length];
    else if (name == "stringArray")
        return new string[length];
    else
        return null;
}

Way too much detail:
The reason we have to do this is that there’s a bug in the type system for netmf-based interpreters. Normally, in netmf, for arrays, you can use introspection to find out what the type is for elements of the array. Unfortunately, on netmf, this information is missing. Netmf knows what the type is - it just won’t tell us. I looked at fixing the bug in the interpreter, but that would mean everyone running my code would have to run a custom interpreter. I might revisit that and publish the fix for GHI and other firmware providers to use, but that still means you’d either have to version-check the firmware at runtime, or include the above fix in order to run on downlevel firmware versions. Anyway, this hack seemed like the best way around it for now.

5 Likes

As this is now part of the core TinyCLR, we can add the necessary fixes. We should look into this in the future.

4 Likes

Sounds good - if I can work out the fix to the interpreter, I’ll happily share it.

4 Likes

That is a good news !

Thanks @mcalsyn, you give me the solution:

   class WeatherRoot
   {
       public coord coord { get; set; }
       public Weather[] weather { get; set; }
       public static object CreateInstance(string path, string name, int length)
       {
           if (name == "weather")
               return new Weather[length];
           if (path == "//weather")
               return new Weather();
           return null;
       }

       public static WeatherRoot FromJson(string content)
       {
           return (WeatherRoot)JsonConverter.DeserializeObject(content, typeof(WeatherRoot), CreateInstance);
       }

   }
2 Likes

Another issue: when a member is declare as double (ie. longitude), when json contains “longitude”:1, it returns an error: wrong_type. I think it read it as integer and not double.

So is Json the only serialization available now? I used to use Reflection.Serialize to save config files to an SD card. Seems there is no Reflection.Serialize in TinyCLR OS. I don’t really care, but it seems strange since the Serializable attributes are still there. Am I just missing it?

So this is actually a bit larger of a problem than I thought. My Config contains objects. It seems that the Json serializer can only handle basic types within the serialized object, so it fails. I can rebuild the config class with this in mind, but this would have jsut worked in 4.3 with Reflection.Serialize. Is there a solution for this?

We shouldn’t build features base on old code. We should build them based on what works best into the future. So, what we need from you an example real life application, and what we need to implement to make it work and why current options do not do the job for you. We all, as a community, can then decide what works best based on overall feedback, not individual cases.

Please share what you need and what purposed solutions you have and let the community share their thoughts.

@Phil_C why do not use sqlite instead your existing old config as new way of config

@Gus_Issa I totally agree. That is why I was asking for alternative solutions.

Here is a more complete example:

[Serializable]
public class SSConfiguration
{
    public Pump pump1;
    public Pump pump2;
    public Pump pump3;

    public Network network;

    public SSConfiguration()
    {
        pump1 = new Pump();
        pump2 = new Pump();
        pump3 = new Pump();

        network = new Network();
    }
}

[Serializable]
public class Pump
{
    public double SyringeVolume { get; set; } = 0.250;
    public double SyringeSteps { get; set; } = 6000;
}

[Serializable]
public class Network
{
    public byte[] IPAddress { get; set; } = { 172, 16, 0, 106 };
    public int Port { get; set; } = 64000;
    public byte[] SubnetMask { get; set; } = { 255, 255, 255, 0 };
    public byte[] DefaultGateway { get; set; } = { 172, 16, 0, 255 };
    public byte[] TlsEntropy { get; set; } = { 0, 1, 2, 3 };
    public byte[] DNSAddress1 { get; set; } = { 8, 8, 8, 8 };
    public byte[] DNSAddress2 { get; set; } = { 8, 8, 4, 4 };
    public byte[] MACAddress { get; set; } = { 0x9A, 0x81, 0x69, 0x3B, 0x13, 0x3F };
    public bool IsDHCP { get; set; } = false;
    public bool IsDynamicDNS { get; set; } = false;
}

Now, I used to be able to just call Reflection.Serialize on the SSConfiguration object and save it to disk. This allows users to modify connection and config settings from the PC and have them stored persistently on the device.

So far I see a couple options:
1.) Write a parser and just put it all in plain text
2.) Write a simple object with only value types and use Json
3.) @valon_hoti_gmail_com suggests to use sqlite

The parser is a pain. The simple object would work, but does chafe at my inner object-oriented programmer. sqLite is probably the best solution here, but does take space. I have also never used it, but, I am not hurting for space and have never shied away from learning something new. Any others to add to the list?

1 Like

I’m using the JSON serializer with deep complex object trees - can you share exactly what error or issue you are seeing? I’ll see if I can help out.

1 Like

I don’t know if it is applicable to me or to @Phil_C.
My problem is when a json string contains for example:

string content = "{\"coord\":{\"lon\":6,\"lat\":46.08}}";

and both lon and lat are declared to double, it raises an exception as in string lon is read as int.

Hi mcalsyn. Definitely appreciate you having a look if you are getting this stuff to work. Here is my configuration class. You can see that the config object (for now) consists of 3 pumps and 1 network object. I run into several issues depending on exactly what I try. If I just go for the gold on this sample, I get a System.Exception unsupported type at var bson = result.ToBson(); If think this has to do with the byte[]s in the network object? If I remove the network object, It creates the Jobject, but when I look at the string, it is a partial and has no values for any of the variables. I have never worked with Json before, so part of the problem is that I just don’t know what I can expect it to be capable of. Any insight would be appreciated as this would be the best way to do serialization for my purposes (vs an SQLite DB).

using System;
using System.IO;
using System.Reflection;
using GHIElectronics.TinyCLR.Data.Json;
using IPT;

namespace SampleStream_TinyCLR
{
[Serializable]
public class SSConfiguration
{
    public Pump pump1;
    public Pump pump2;
    public Pump pump3;

    public Network network;

    public SSConfiguration()
    {
        pump1 = new Pump();
        pump2 = new Pump();
        pump3 = new Pump();

        network = new Network();
    }
}
 
public static class ConfigurationUtilities
{
    public static void SaveConfiguration(string fileName, SSConfiguration config)
    {
        var result = JsonConverter.Serialize(config);
        //var jObject = JObject.Serialize(typeof(SSConfiguration), config);
        var bson = result.ToBson();

        var compare = (SSConfiguration)JsonConverter.FromBson(bson, typeof(SSConfiguration));
        //SDHelpers.WriteFile(fileName, bson);

    }

    public static SSConfiguration GetConfiguration(string configPath)
    {
        SSConfiguration config;
        string[] files = SDHelpers.GetFiles();

        foreach(string item in files)
        {
            if(string.Compare(configPath, item) == 0)
            {
                using(FileStream fs = new FileStream(configPath, FileMode.Open))
                {
                    
                    config = (SSConfiguration)JsonConverter.DeserializeObject(fs, typeof(SSConfiguration), CreateInstance);
                }
                return config;
            }
        }

        return null;
    }

    private static object CreateInstance(string path, string name, int length)
    {
        if (name == "intArray")
            return new int[length];
        else if (name == "stringArray")
            return new string[length];
        else if (name == "byteArray")
            return new byte[length];
        else
            return null;
    }
}


[Serializable]
public class Pump
{
    public double SyringeVolume { get; set; } = 0.250;
    public double SyringeSteps { get; set; } = 6000;
}

[Serializable]
public class Network
{
    public byte[] IPAddress { get; set; } = { 172, 16, 0, 106 };
    public int Port { get; set; } = 64000;
    public byte[] SubnetMask { get; set; } = { 255, 255, 255, 0 };
    public byte[] DefaultGateway { get; set; } = { 172, 16, 0, 255 };
    public byte[] TlsEntropy { get; set; } = { 0, 1, 2, 3 };
    public byte[] DNSAddress1 { get; set; } = { 8, 8, 8, 8 };
    public byte[] DNSAddress2 { get; set; } = { 8, 8, 4, 4 };
    public byte[] MACAddress { get; set; } = { 0x76, 0xCA, 0x75, 0x29, 0x57, 0x50 };
    public bool IsDHCP { get; set; } = false;
    public bool IsDynamicDNS { get; set; } = false;
}
}

Adding a bit, here is the string I get:

{{\r\n “pump2” : {\r\n “SyringeSteps” : {\r\n\r\n },\r\n “SyringeVolume” : {\r\n\r\n }\r\n },\r\n “network” : {\r\n "P}

It looks like it is getting truncated somehow.

Okay, I have done some more messing around with this now and I seem to have run into the same issue as @Bauland. I can save and restore objects that contain ints, but doubles are causing some issues. I have Reflector, so I can see where it broke. I can inspect type and it has a value and both the assembly full name and type full name are not null. I can’t inspect AppDomain and there are no other references to AppDomain in the file, so I am guessing that is the null ref here.

Also, File.WriteAllBytes does not flush the file if it is small (in this case, it was 74 bytes and was not writing to the SD Card). I had to implement it myself and add an FileStream.Flush() to get it to write. Might consider putting this into File.WriteAllBytes after the Stream.Write().

EDIT: I figured out the null reference. If you serialize an object that uses autoproperties like this:

  public int SyringeSteps { get; set; } = 6000;

it serializes the variable name but not the value. So then when you try to deserialize, there is obviously a null value. So, stay away from autoproperties, even though VS2019 lets you do it.

Thanks for helping and, by the way, the source code is available on the libraries repo if you want to see it. This code was developed by @mcalsyn so we have the expert here as well.

1 Like

Ah, thanks for the heads up. I hadn’t explored the repo yet.