princeton-nlp / MeZO

[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

princeton-nlp/MeZO Issues

roberta-large zero shot
Updated 22 days ago
can not reproduce the the result of roberta large on dataste sst-2
Updated a month ago1
In which file is the code implemented by the algorithm？
Updated 2 months ago1
Maybe need a requirement.txt file to facilitate environment preparation？
Updated 2 months ago1
MeZO on ChatGLM6B
Closed 3 months ago2
Results of Trec dataset on Roberta-large(K=512) with MeZO(LoRA)
Updated 4 months ago8
Zero Order implementation does not converge in CIFAR-10 dataset.
Updated 5 months ago1
Standard FT does not work
Updated 6 months ago3
Getting a RuntimeError after training with mezo
Updated 6 months ago6
max_seq_length and max_seq_len confusion
Updated 6 months ago1
How to use MeZO in training a simple CIFAR-10 model
Updated 7 months ago3
Cannot reproduce some results of OPT
Updated 7 months ago3
Inconsistent results of MEZO for RoBERTa-large on SST-2
Updated 9 months ago
LoRA & p-tuning with multi-GPU
Updated 9 months ago3
Results on WSC and WIC datasets cannot be reproduced on OPT-13B with MeZO
Updated 10 months ago5
llama2 problem
Updated a year ago1
Cannot reproduce the results for RoBERTa on SST-2
Updated a year ago1
deepspeed reference on colab
Closed a year ago2
ValueError: The model did not return a loss from the inputs, only the following keys: logits,past_key_values. For reference, the inputs it received are input_ids,attention_mask.
Closed a year ago2
AttributeError: 'TrainingArguments' object has no attribute 'linear_probing'
Closed a year ago4
gpt_neo not supported
Closed a year ago8
Nanogpt implementation
Updated a year ago3
Cannot reproduce the results of OPT on SST2
Closed a year ago15
Can you provide more details about how to run the code?
Closed a year ago1
Not convergent in custom dataset.
Updated a year ago9
MeZO on continue pre-training
Updated a year ago1
About experimentical setting of 1000 examples
Closed a year ago2
Which trainer to use
Updated a year ago7
MeZO running script for roberta-large is not working
Closed a year ago1
MeZo can be used in NLG tasks?
Updated a year ago5
Best parameters found for datasets
Updated a year ago3
Impact of Dropout?
Closed a year ago1
Any benchmark on (MeZO) v.s. (ZeRO + CpuOffload + Grad checkpointing) ?
Closed a year ago2