ml-jku / hopfield-layers

Hopfield Networks is All You Need

Home Page:https://ml-jku.github.io/hopfield-layers/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how is it performance in seq2seq task?

af-74413592 opened this issue · comments

test in transformerlike-seq2seq task, frankly, it doesn't work.

Hi @af-74413592,

I cannot quite follow your issue, I'm afraid. If you want to deploy hopfield-layers in a transformer-like setting, the corresponding implementations

class HopfieldEncoderLayer(Module):
"""
Module with underlying Hopfield association to be used as an encoder in transformer-like architectures.
"""
and
class HopfieldDecoderLayer(Module):
"""
Module with underlying Hopfield associations to be used as a decoder in transformer-like architectures.
"""
are a drop-in replacement for TransformerEncoderLayer and TransformerDecoderLayer, respectively.