Dimensionality when using FCNN and showing bias nodes

[Sven Anderzén]

Thanks for a great tool! 👍

When using the fully connected style and checking the "Show Bias Units" and "Show Layer Labels" the bias neurons are shown correctly. However, the dimensions of the labels are not changed.

I would expect the dimensions to drop by 1, since these neurons are not contributing to the dimensionality (but just moving the hyperplane in the X dimensional space). For example a 5 neuron hidden layer without bias would be in R ∈ 5 as shown, while the same hidden layer with the bias weight shown (where the connections are dropped in the illustration) would reside in R ∈ 4.

Am I interpreting the illustration correct or am I missing something? Or is it simply that the labels aren't updated correctly? 🙂

[Alexander Lenail]

Thanks for reaching out, @svenanderzen!

I think it could go either way -- although the bias doesn't contribute to the dimensionality of the underlying tensor when you're coding this up, I think it's still fair to say that the layer is in $R^5$. I think it's a little ambiguous which of these behaviors is expected, so I think I'll punt on this. Let me know whether you think there's strong evidence I'm wrong here, and everyone expects the layer to be in $R^4$

For your case, I would recommend downloading an SVG and editing the dimensionality to 4 (e.g. in inkscape), or even opening up chrome devtools and just changing the text in the label element.

[Sven Anderzén]

Hey, sorry totally forgot about answering this @alexlenail!

I would still consider the layer to be in R ∈ 4, since that's the dimensionality of the hypercube, but could be wrong here so I'll leave the decision to you. I went with the SVG approach like you suggested, just wanted to give you a heads up in case it was unintentional. 🙂

Anyway, this is a great tool. Big thanks! I'll close this issue, since you're aware of it and it was intentional 👍