GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation
Our code and evaluation benchmark will be out soon!
A demo figure using GPT-4V to shop on the Amazon app with an iphone:
If you find our work helpful to your research, please consider citing the paper:
@article{yan2023gpt,
title={GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation},
author={Yan, An and Yang, Zhengyuan and Zhu, Wanrong and Lin, Kevin and Li, Linjie and Wang, Jianfeng and Yang, Jianwei and Zhong, Yiwu and McAuley, Julian and Gao, Jianfeng and others},
journal={arXiv preprint arXiv:2311.07562},
year={2023}
}