CarperAI / cheese

Used for adaptive human in the loop evaluation of language and embedding models.

CarperAI/cheese Issues

"Getting started" guide not working (no client id/password provided)
Closed a year ago10
Issues with Running Examples
Updated a year ago
`instruct_hf_pipeline` example returns rankings of `None`
Updated a year ago1
device=0 leads to AssertionError: Torch not compiled with CUDA enabled
Closed a year ago3
`webdataset` not installed but required
Closed a year ago2
API Fails to connect to already running server sometimes
Closed a year ago1
Saving progress for datasets
Closed a year ago3
Make API accessible without running directly
Closed a year ago1
Prolific integration
Updated 2 years ago
Pipeline can't send directly to model
Closed 2 years ago1
Batched model input
Closed 2 years ago1
How about provide a NER annotation example?
Closed 2 years ago4
Ending data stream to user
Closed 2 years ago
Client stats
Closed 2 years ago
Pipeline can't send/receive data directly to/from model
Closed 2 years ago
Need to move communication with user to API
Closed 2 years ago1