How to use 2d example with List<double[4]> array?

Question

How to use 2d example with List<double[4]> array?

galvanchi opened this issue 6 years ago · comments

Hi,

First of all thank you for your job.
I'm new in convolutional networks and investigating your project and take a look over https://github.com/cbovar/ConvNetSharp/blob/master/Examples/Classify2DDemo/Program.cs
this example suitable for my situation, but I don't understand what should to change to make it work. because of instead of array with 2 values I have an array with 4 values and project pushed me to crash on 85th line trainer.Train(netx, hotLabels); if I use following code.

Maybe I miss something?

private static void Classify2DUpdate(int n, List<double[]> data, TrainerBase<double> trainer, List<int> labels) {
            var avloss = 0.0;
            
            var netx = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));
            var hotLabels = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));

            for(var ix = 0; ix < n; ix++) {
                hotLabels.Set(0, 0, labels[ix], ix, 1.0);

                netx.Set(0, 0, 0, ix, data[ix][0]);
                netx.Set(0, 0, 1, ix, data[ix][1]);
                netx.Set(0, 0, 2, ix, data[ix][2]);
                netx.Set(0, 0, 3, ix, data[ix][3]);
                
            }

            for(var iters = 0; iters < 50; iters++) {
                trainer.Train(netx, hotLabels);
                avloss += trainer.Loss;
            }

            avloss /= 50.0;
            Console.WriteLine(" Loss:" + avloss);
        }

My initialization is follow

    public void Classify() {
        var net = new Net<double>();
        net.AddLayer(new InputLayer(1, 1, 4));
        net.AddLayer(new FullyConnLayer(6));
        net.AddLayer(new TanhLayer());
        net.AddLayer(new FullyConnLayer(2));
        net.AddLayer(new TanhLayer());
        net.AddLayer(new FullyConnLayer(2));
        net.AddLayer(new SoftmaxLayer(2));

        // Data
        var data = new List<double[]>();
        var labels = new List<int>();
        data.Add(new[] { -0.4326, 1.1909, 1.2305, 2.5345 });
        labels.Add(1);
        data.Add(new[] { 3.0, 4.0, 1.2805, 3.5234 });
        labels.Add(1);
        data.Add(new[] { 1.8133, 1.0139, 1.4305, 1.4535 });
        labels.Add(0);
        data.Add(new[] { 2.7258, 1.0668, 1.8705, 3.5243 });
        labels.Add(0);
        var n = labels.Count;

        var trainer = new SgdTrainer<double>(net) { LearningRate = 0.01, L2Decay = 0.001, BatchSize = n };

        // Training
        do {
            Classify2DUpdate(n, data, trainer, labels);
            nums++;
        } while(!Console.KeyAvailable);

        // Testing
        var netx = BuilderInstance.Volume.From(new double[2 * n], new Shape(1, 1, 4, n));
        for(var ix = 0; ix < n; ix++) {
            netx.Set(0, 0, 0, ix, data[ix][0]);
            netx.Set(0, 0, 1, ix, data[ix][1]);
            netx.Set(0, 0, 2, ix, data[ix][2]);
            netx.Set(0, 0, 3, ix, data[ix][3]);
        }

        var result = net.Forward(netx);
        var c = net.GetPrediction();
        var accurate = c[0] == labels[0];

        Console.ReadKey();
    }

Cédric Bovar · Answer 1 · Sat Mar 17 2018 11:37:34 GMT+0800 (China Standard Time)

Hi,

I think you dont allocate enough space on this line:

var netx = BuilderInstance.Volume.From(new double[2 * n], new Shape(1, 1, 4, n));

It should be

var netx = BuilderInstance.Volume.From(new double[4 * n], new Shape(1, 1, 4, n));

Cédric Bovar · Answer 2 · Sat Mar 17 2018 11:45:34 GMT+0800 (China Standard Time)

Actually you could use this instead so you dont have to provide an empty array:

var netx = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));

galvanchi · Answer 3 · Sun Mar 18 2018 15:58:47 GMT+0800 (China Standard Time)

Hi, thank you for the answer, but I still got out of bounds error inside Classify2DUpdate.
I changed code to

var netx = BuilderInstance.Volume.From(new double[4 * n], new Shape(1, 1, 4, n));
var hotLabels = BuilderInstance.Volume.From(new double[4 * n], new Shape(1, 1, 4, n));

from

var netx = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));
var hotLabels = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));

Any thoughts why is that happen?

Cédric Bovar · Answer 4 · Sun Mar 18 2018 20:47:31 GMT+0800 (China Standard Time)

There are only two classes in this example. So you should keep two classes for the hot labe;s :
var hotLabels = BuilderInstance.Volume.SameAs(new Shape(1, 1, 2, n));

galvanchi · Answer 5 · Mon Mar 19 2018 00:12:30 GMT+0800 (China Standard Time)

So if I understood correct for hotLabels should be 2 as third parameter for Shape and it's correspond to the number of classes, am I right?

Also I can see for 2D tutorial using trainer initialization as new SgdTrainer(net) but in 1d tutorial using simple new SgdTrainer(net) , is it correct?

Cédric Bovar · Answer 6 · Mon Mar 19 2018 09:10:45 GMT+0800 (China Standard Time)

So if I understood correct for hotLabels should be 2 as third parameter for Shape and it's correspond to the number of classes, am I right?

Correct. I have tried to explain that in the Wiki

Also I can see for 2D tutorial using trainer initialization as new SgdTrainer(net) but in 1d tutorial using simple new SgdTrainer(net) , is it correct?

I am not sure I understand the question

galvanchi · Answer 7 · Sun Apr 01 2018 19:22:33 GMT+0800 (China Standard Time)

Hi, still failing. please can you provide an example for Classify2DDemo, when you have three dimensional vector (x1, x2, x3) or fourdimensional vector?

Cédric Bovar · Answer 8 · Mon Apr 02 2018 23:04:14 GMT+0800 (China Standard Time)

This works for me (including the two modifications we talked about):

public static void Classify()
        {
            var net = new Net<double>();
            net.AddLayer(new InputLayer(1, 1, 4));
            net.AddLayer(new FullyConnLayer(6));
            net.AddLayer(new TanhLayer());
            net.AddLayer(new FullyConnLayer(2));
            net.AddLayer(new TanhLayer());
            net.AddLayer(new FullyConnLayer(2));
            net.AddLayer(new SoftmaxLayer(2));

            // Data
            var data = new List<double[]>();
            var labels = new List<int>();
            data.Add(new[] { -0.4326, 1.1909, 1.2305, 2.5345 });
            labels.Add(1);
            data.Add(new[] { 3.0, 4.0, 1.2805, 3.5234 });
            labels.Add(1);
            data.Add(new[] { 1.8133, 1.0139, 1.4305, 1.4535 });
            labels.Add(0);
            data.Add(new[] { 2.7258, 1.0668, 1.8705, 3.5243 });
            labels.Add(0);
            var n = labels.Count;

            var trainer = new SgdTrainer<double>(net) { LearningRate = 0.01, L2Decay = 0.001, BatchSize = n };

            // Training
            do
            {
                Classify2DUpdate(n, data, trainer, labels);
                //nums++;
            } while (!Console.KeyAvailable);

            // Testing
            var netx = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));
            for (var ix = 0; ix < n; ix++)
            {
                netx.Set(0, 0, 0, ix, data[ix][0]);
                netx.Set(0, 0, 1, ix, data[ix][1]);
                netx.Set(0, 0, 2, ix, data[ix][2]);
                netx.Set(0, 0, 3, ix, data[ix][3]);
            }

            var result = net.Forward(netx);
            var c = net.GetPrediction();
            var accurate = c[0] == labels[0];

            Console.ReadKey();
        }

        private static void Classify2DUpdate(int n, List<double[]> data, TrainerBase<double> trainer, List<int> labels)
        {
            var avloss = 0.0;

            var netx = BuilderInstance.Volume.SameAs(new Shape(1, 1, 4, n));
            var hotLabels = BuilderInstance.Volume.SameAs(new Shape(1, 1, 2, n));

            for (var ix = 0; ix < n; ix++)
            {
                hotLabels.Set(0, 0, labels[ix], ix, 1.0);

                netx.Set(0, 0, 0, ix, data[ix][0]);
                netx.Set(0, 0, 1, ix, data[ix][1]);
                netx.Set(0, 0, 2, ix, data[ix][2]);
                netx.Set(0, 0, 3, ix, data[ix][3]);
            }

            for (var iters = 0; iters < 50; iters++)
            {
                trainer.Train(netx, hotLabels);
                avloss += trainer.Loss;
            }

            avloss /= 50.0;
            Console.WriteLine(" Loss:" + avloss);
        }

        private static void Main(string[] args)
        {
            Classify();
        }

Cédric Bovar · Answer 9 · Mon Apr 02 2018 23:06:35 GMT+0800 (China Standard Time)

I'm not very satisfied with the current API (especially BuilderInstance.Volume.* stuff). If you have any ideas how to make it more intuitive please let me know.

galvanchi · Answer 10 · Tue Apr 03 2018 23:56:14 GMT+0800 (China Standard Time)

Yeah it's worked(for 3 labels also). From now on it's clear. Thank you a lot, you are rock.