my thoughts

[Nick Cullen, PhD]

Hi! I just had a quick look over Picard and had some thoughts + constructive criticism as someone who uses keras + hyperopt.

the json syntax you use is very cumbersome.. soo many levels of depth. It's also not clear why you do/dont use the built-in keras json model configs.. in fact your project seems to have no relation at all to keras/hyperopt (other than assumably using it in the backend) and instead you just invented a new configuration format. In general, it feels like it'd take longer to write those config files than it would be to just write a keras model and some for loops or learn hyperopt itself.

I guess i'm saying that for me personally, this syntax seems too complex. just my constructive opinion :)

for instance, this really scares me for having just two layers:

{
    "operators": {
        ...,
        "feedforward": {
          "#repeat": {
            "operator": {
              "#compose": [
                {
                  "layer": "Dense",
                  "config": {
                    "output_dim": 512,
                    "activation": {
                      "&choice": {
                        "options": [
                          "relu",
                          "sigmoid"
                        ]
                      }
                    }
                  }
                },
                {
                  "#optional": {
                    "layer": "Dropout",
                    "config": {
                      "p": {
                        "&uniform": {
                          "high": 0.6,
                          "low": 0.3
                        }
                      }
                    }
                  }
                }
              ]
            },
            "+times": {
              "&choice": {
                "options": [
                  2,
                  3,
                  4,
                  5
                ]
              }
            }
          }
        }
    }
}
}

[Jake Bian]

Hey thanks for the note.

The JSON tree is the more or less the minimal data structure for one to specify a finite graph with operator associated to edges. The graph spec itself is 2 or 3 levels deep.

You make your spec deeper by converting values to hyperopt variables - what you have above is not a 2 layer network - it specified hyperopt experiment that covers choices of number of layers, activation functions, dropout probability, whether to use dropout, and activation function. Building this by hand is about the same amount of code, would you not agree?

More generally, especially in cases like this where your config specifies expensive computation jobs, declarative configurations has benefits over sequential code. e.g.

• When running this in a distributed environments, this behaves more predictably than pickling.
• As you noted this is implementation agnostic, it's conceivable that we can swap out keras and hyperopt in the backend to improve the implementation in the future.
• With the declarative syntax you're free of local states in your code, you're free to strictly design the structure of the model without worrying about the correctness of your specific implementation.

Of course it's a personal preference. If working with hyperopt + keras on their own already solves all your problems, I encourage you to stick with that.

Going to close this for now as it's not a specific issue, feel free to continue this discussion either on here or through email!