kohjingyu / fromage

🧀 Code and models for the ICML 2023 paper "Grounding Language Models to Images for Multimodal Inputs and Outputs".

https://jykoh.com/fromage

kohjingyu/fromage Issues

Question about the frozen language model
Closed a year ago3
Unexpected key(s) in state_dict
Closed 2 months ago
Replacing OPT LLM with other LLMs
Updated 3 months ago
How to download dataset and prepare tsv file?
Updated 3 months ago
retrieval only mode
Updated 3 months ago1
Can I use the embedding for training
Closed 4 months ago2
Evaluation code of VQAv2
Updated 8 months ago
Freezing the final linear layer when adding new token [RET]
Closed 8 months ago1
torch.distributed.all_gather does not have grads
Closed 10 months ago2
=> no checkpoint found at '=/home/...
Closed 10 months ago
I got 'KeyError: 'best_score'' while trying to fine-tuning
Closed 10 months ago7
I got Unexpected key(s) in state_dict error
Closed 10 months ago2
can you give me a pre-trained weight file not pruning?
Closed a year ago5
What is CC3M Embeddings
Closed a year ago2
[RET] Embedding
Closed a year ago3
Dealing with Corrupted Images in CC3M
Closed a year ago4
The ability of in-context learning
Closed a year ago
Evaluation code for VQAv2
Updated a year ago4
Computing output likelihoods with the model
Closed a year ago7
Huggingface pipeline
Closed a year ago2
The reproduction of FROMAGe training
Closed a year ago6
The cross entropy loss in training stage
Closed a year ago2
The evaluation speed of IT2T on VisDial
Closed a year ago8
Evaluation for VisDial
Closed a year ago1
Weights of `lm_head` were not frozen during training?
Closed a year ago3
How are the inputs arranged for in-context retrieval evaluation?
Closed a year ago5
Choice of retrieval embedding dimension q = 256
Closed a year ago3
How does generate work?
Closed a year ago2
Do you think bigscience/bloom can be a replacement of facebook/opt model ?
Closed a year ago5
What is "fromage_vis4" model?
Closed a year ago3
How to load dataset?
Closed a year ago2
Failure in testing the demo
Closed a year ago1
Hello, I wanna konw the purpose of create_image_of_text
Closed a year ago2
Should the last_embedding_idx = caption - 2 ?
Closed a year ago2
Concatenating two captions in retrieval mode
Closed a year ago6
when the source codes can be released?
Closed 2 years ago3
Asking for roadmap with more details?
Closed 2 years ago1