Poor quality of text generation in Cloud Run compared to Colab
Oscerlot opened this issue · comments
First up, thanks for all the work you've put into all of the GPT-2-simple stuff. It's amazing!
But I've set up a generation with Cloud Run using the same model and same settings as in Colab, and the text outputs are significantly less cohesive with lines being constantly repeated. Any particular reason why this would be happening? Is it a limitation of the Cloud Run hardware vs the Colab hardware?
The model is intended to be a video game idea generator trained on ~15,000 posts from /r/gameideas. Here's an example of the same prefix in each context:
Colab
A game where you have to fight children or some shit. The children are easy to kill. You can run for cover or you can try to fight back but you're much slower. You can't run as fast as the children. You can hide, crawl, crawl out the door. There's also a lot of zombies.
If you're fast enough, you can jump off the roof and climb inside. The children are easier to kill. You can jump it too. The children can get stuck in the wall. You can jump to them, kill them and then climb up. There's a lot of enemies.
You can use the power of the house as a platform to jump in the first places. You can then jump to the roof where there's a bigger enemy. You can then crawl out the door to the other side to sneak in. There's a lot of zombies.
There's also a lot of fire. You can run into them. You can throw a torch at them. They'll die if you're not careful. Once they die, you can jump to the roof but the fire won't burn you if you're not careful.
I'm not sure if the game is multiplayer or not.
Cloud Run
A game where you fight children, and you can make them do anything you want, and you have a gun and you fight crime.
You can make people sick with drugs, and you can make people homeless, and you can make people commit crimes.
You can make the police and the military and the FBI and the CIA and the NSA and the CIA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and the NSA and you can make everyone in history a billionaire.
You can make President Trump a billionaire, and all the other billionaire games like they are a game, but you can only make a few people rich, and you can only make one type of person rich, and you can only make a certain amount of people rich, and you can only make a certain amount of people homeless, and you can only make them sick.
It's a consistent trait where the Cloud Run generation seems to almost ignore the context of the prefix and then gets stuck in a loop.
Did you set the same temperature? How do you generate your examples in both cases?
unlikely that this is an issue with the setup itself, rather with the code or parameters