How to load FluentNet<T> from json file?

Question

How to load FluentNet<T> from json file?

soloham opened this issue 6 years ago · comments

Hi, I am creating my model with FluentNet as:

myBrain.net = FluentNet<double>.Create(330, 200, 1)
                   .Conv(5, 5, 2).Stride(3).Pad(2)
                   .Relu()
                   .Pool(2, 2).Stride(2)
                   .Conv(5, 5, 2).Stride(1).Pad(2)
                   .Relu()
                   .Pool(2, 2).Stride(2)
                   .Conv(5, 5, 2).Stride(1).Pad(2)
                   .Relu()
                   .Pool(2, 2).Stride(2)
                   .Conv(5, 5, 2).Stride(1).Pad(2)
                   .Relu()
                   .Pool(2, 2).Stride(2)
                   .FullyConn(26)
                   .Softmax(26)
                   .Build();

And them I'm saving it as json using:
Mind.ToJson();

But There is no function to load the saved Fluentnet back, as there is in Net<T> using Net<T>.FromData(json) function.

Can you please add this feature or if it already exists please educate me on that. Thanks

Cédric Bovar · Answer 1 · Sun Jul 08 2018 13:42:51 GMT+0800 (China Standard Time)

Hi,
Have you tried this?

using ConvNetSharp.Core.Serialization;

[...]

// Serialize to json 
var json = net.ToJson();

// Deserialize from json
Net deserialized = SerializationExtensions.FromJson<double>(json);

Hamza · Answer 2 · Sun Jul 08 2018 19:28:14 GMT+0800 (China Standard Time)

Yes, I did try that but that method returns a Net<T> object so it gives me the following error:
error CS0029: Cannot implicitly convert type 'ConvNetSharp.Core.Net<double>' to 'ConvNetSharp.Core.Fluent.FluentNet<double>'

I am deserializing and loading it as follows:

private void LoadConvNet()
 {
     string json = SaveLoadManager.LoadConvNetFluent();
     Mind = SerializationExtensions.FromJson<double>(json); //<<< This line throws the error
 }

I am deserializing the json file as follows:

   public static string LoadConvNetFluent()
    {
        string bytesToString = "Couldn't Load Data";
        sceneName = SceneManager.GetActiveScene().name;
        string path = Application.persistentDataPath + "/" + sceneName + "/ConvNet/Fluent.txt";
        bool doesExist = File.Exists(path);
        if (doesExist)
        {
            //Byte[] writtenBytes = File.ReadAllBytes(Application.persistentDataPath + "/" + sceneName + "/ConvNet/Fluent.json");
            //bytesToString = Encoding.ASCII.GetString(writtenBytes);
            StreamReader reader = new StreamReader(path);
            bytesToString = reader.ReadToEnd();
            Debug.Log("DESERIALIZED: \n " + bytesToString);
        }
        return bytesToString;
    }

And after checking the log, it is clear that the json file is loading properly.

I really need the FluentNet<T> architecture for a project I'm working on for a company, and this part is the most crucial to the entire project so can you please implement this feature? or if there is a quick workaround, please inform me with that. Thanks.

Regards,
Hamza

Cédric Bovar · Answer 3 · Sun Jul 08 2018 23:25:41 GMT+0800 (China Standard Time)

Why do you need the deserialized network to be a FluentNet<T> ?

If you define Mind as INet<T>, does it break anything in your program ?

Hamza · Answer 4 · Mon Jul 09 2018 00:51:31 GMT+0800 (China Standard Time)

It doesn't necessarily break anything but I was testing which one works better, and FluentNet<T> works better in my case when I compared it with INet<T> on the same problem. But the main reason why I'm using FluentNet<T> is that I am not able to create as many Convolution Blocks(ConvLayer, ReLU Layer and PoolLayer) with INet<T> as I'm creating with FluentNet<T> on my problem, I dont know why.
Here is what I'm doing:

My INet<T> initialization:

myBrain.net = new Net<double>();
InputLayer inputLayer = new InputLayer(380, 1380, 1);
ConvLayer IConvLayer = new ConvLayer(5, 5, 8);
IConvLayer.Stride = 5;
IConvLayer.Pad = 2;
ConvLayer convLayer = new ConvLayer(5, 5, 2);
convLayer.Stride = 1;
convLayer.Pad = 2;
PoolLayer poollayer = new PoolLayer(2, 2);
poollayer.Stride = 2;

myBrain.net.AddLayer(inputLayer);
myBrain.net.AddLayer(IConvLayer);
myBrain.net.AddLayer(new ReluLayer());
myBrain.net.AddLayer(poollayer);
myBrain.net.AddLayer(convLayer);
myBrain.net.AddLayer(new ReluLayer());
myBrain.net.AddLayer(poollayer);
myBrain.net.AddLayer(convLayer);
myBrain.net.AddLayer(new ReluLayer());
myBrain.net.AddLayer(poollayer);
myBrain.net.AddLayer(convLayer);
myBrain.net.AddLayer(new ReluLayer());
myBrain.net.AddLayer(poollayer);
myBrain.net.AddLayer(new FullyConnLayer(26));
myBrain.net.AddLayer(new SoftmaxLayer(26));

When I fead an input forward through the network it breaks:

Inputs = BuilderInstance.Volume.From(inputs, networkData.inputShape);
Outputs = Mind.Forward(Inputs); //<<< This line gives the error

I get the following error:

IndexOutOfRangeException: Index was outside the bounds of the array.
ConvNetSharp.Volume.NcwhVolumeStorage`1[T].Get (System.Int32 i) (at Assets/DeepNet Library/src/ConvNetSharp.Volume/NcwhVolumeStorage.cs:76)
ConvNetSharp.Volume.VolumeStorage`1[T].Map (System.Func`3[T1,T2,TResult] f, ConvNetSharp.Volume.VolumeStorage`1[T] other, ConvNetSharp.Volume.VolumeStorage`1[T] result) (at Assets/DeepNet Library/src/ConvNetSharp.Volume/VolumeStorage.cs:32)
ConvNetSharp.Volume.Double.Volume.ReluGradient (ConvNetSharp.Volume.Volume`1[T] input, ConvNetSharp.Volume.Volume`1[T] outputGradient, ConvNetSharp.Volume.Volume`1[T] inputGradient) (at Assets/DeepNet Library/src/ConvNetSharp.Volume/Double/Volume.cs:590)
ConvNetSharp.Core.Layers.ReluLayer`1[T].Backward (ConvNetSharp.Volume.Volume`1[T] outputGradient) (at Assets/DeepNet Library/src/ConvNetSharp.Core/Layers/ReluLayer.cs:27)
ConvNetSharp.Core.Net`1[T].Backward (ConvNetSharp.Volume.Volume`1[T] y) (at Assets/DeepNet Library/src/ConvNetSharp.Core/Net.cs:52)
ConvNetSharp.Core.Training.TrainerBase`1[T].Backward (ConvNetSharp.Volume.Volume`1[T] y) (at Assets/DeepNet Library/src/ConvNetSharp.Core/Training/TrainerBase.cs:31)
ConvNetSharp.Core.Training.TrainerBase`1[T].Train (ConvNetSharp.Volume.Volume`1[T] x, ConvNetSharp.Volume.Volume`1[T] y) (at Assets/DeepNet Library/src/ConvNetSharp.Core/Training/TrainerBase.cs:47)
ConvNet_Model.Train (System.Single[] inputs, System.Single[] targets, System.Int32 sessions) (at Assets/Scripts/ConvNet_Model.cs:230)
ConvNet_Model+<ITrainNetwork>d__28.MoveNext () (at Assets/Scripts/ConvNet_Model.cs:249)

From the error it seems that there is something wrong with the dimensions of the shape that the input Volume<double> has but that's not the case as the same volume works just fine with FluentNet<T>, here is how the volumes are setup:

myBrain.inputShape = new ConvNetSharp.Volume.Shape(330, 200, 1);
myBrain.outputShape = new ConvNetSharp.Volume.Shape(26);

Now when I use FluentNet<T> it works just fine with the same Inputs Volume, and the network also has the same set of layers as the INet<T> I used, here is the initialization:

myBrain.net = FluentNet<double>.Create(380, 200, 1)
.Conv(5, 5, 8).Stride(5).Pad(2)
.Relu()
.Pool(2, 2).Stride(2)
.Conv(5, 5, 2).Stride(1).Pad(2)
.Relu()
.Pool(2, 2).Stride(2)
.Conv(5, 5, 2).Stride(1).Pad(2)
.Relu()
.Pool(2, 2).Stride(2)
.Conv(5, 5, 2).Stride(1).Pad(2)
.Relu()
.Pool(2, 2).Stride(2)
.FullyConn(26)
.Softmax(26)
.Build();

If there is something that I'm doing wrong here, please correct me otherwise If it's a bug with INet<T> please fix it or create the feature to load FluentNet<T> from json kindly. Thanks.

Regards,
Hamza

Cédric Bovar · Answer 5 · Mon Jul 09 2018 09:00:33 GMT+0800 (China Standard Time)

You should be able to:

Construct your network using the fluent way
Serialize it to a file
Deserialize it as a INet<T> and use it

FluentNet is merely some convenience methods that will create classic layers and link them together.

I see that you re-use some layers (poollayer, convLayer) when creating the network the non-fluent way. I think that's why you get an exception. Ideally an exception should be thrown when adding the same layer twice.

Hamza · Answer 6 · Mon Jul 09 2018 09:28:17 GMT+0800 (China Standard Time)

Oh great, that makes sense now...

The reason I stored ConvLayer and PoolLayer in variables is because I was not able to change their Stride and Pad parameters but as you said I can just create those layers using the fluent way, serialize the fluent network and deserialize as INet<T> ...right?

Cédric Bovar · Answer 7 · Mon Jul 09 2018 10:00:36 GMT+0800 (China Standard Time)

The reason I stored ConvLayer and PoolLayer in variables is because I was not able to change their Stride and Pad parameters

Why could'nt you change Stride and Pad parameters? Something like that would work:
myBrain.net.AddLayer(new ConvLayer(5, 5, 16) { Stride = 1, Pad = 2 });

I can just create those layers using the fluent way, serialize the fluent network and deserialize as INet ...right?

Yes

Hamza · Answer 8 · Mon Jul 09 2018 10:13:49 GMT+0800 (China Standard Time)

It turns out while I was importing the library to Unity3D I had to delete remove some code mostly the GPU code but i accidentally also deleted some other code which was causing some weird issues. Now after reimporting the library carefully it is working flawlessly.

Thanks a lot for helping me out and also for creating this awesome library and making our lives easier.