Default configuration for training
FahadEbrahim opened this issue · comments
Hi,
Just a quick question, for training an adapter, what are the default adapter configurations (Method or type)?
Thanks.
Hey, the default adapter configuration used when none is explicitly specified is defined in DEFAULT_ADAPTER_CONFIG
. It's a bottleneck adapter (BnConfig()
class) with one bottleneck layer per Transformer block, as proposed in this paper.
Hello @FahadEbrahim , the default configuration that is used if you call add_adapter
without specifying the config parameter is the seq_bn
config (previously named pfeiffer
config). To use a different configuration just pass the corresponding string as the config parameter (docs).