vinayprabhu / X-is-all-you-need

A survey of all the 'X is all you need'

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

X is all you need

Co-creators: Vinay Uday Prabhu and Ryan Teehan

Citation:

@inproceedings{prabhu2020spices,
  title={SPICES: Survey Papers As Interactive Cheatsheet Embeddings},
  author={Prabhu, Uday Vinay and 
McAteer, Matthew and Teehan, Ryan },
  booktitle={Rethinking ML Papers - ICLR 2021 Workshop},
  howpublished = {\url{https://openreview.net/pdf?id=1sysg9hi3KS}},
  month = {April},
  year = {2020},
  note = {(Accessed on 04/03/2021)}
  year={2020}
}

The success enjoyed by the Attention is all you need spurred an avalanche of papers whose title or content banked off of the X-is all you need quip 😅.

In Figure below, we present a survey of all these papers that allows the reader to summarize where and why these quips occur. Landscape_of_papers

Authors Title Publication Volume Number Pages Year Publisher
Kidambi, Rahul; Chang, Jonathan; Sun, Wen; Optimism is All You Need: Model-Based Imitation Learning From Observation Alone arXiv preprint arXiv:2102.10769 2021.0
Shen, Chao; Wu, Hau-Tieng; Scalability and robustness of spectral embedding: landmark diffusion is all you need arXiv preprint arXiv:2001.00801 2020.0
Bertasius, Gedas; Wang, Heng; Torresani, Lorenzo; Is Space-Time Attention All You Need for Video Understanding? arXiv preprint arXiv:2102.05095 2021.0
Bach, Francis; The sum of a geometric series is all you need!
de Witt, Christian Schroeder; Gupta, Tarun; Makoviichuk, Denys; Makoviychuk, Viktor; Torr, Philip HS; Sun, Mingfei; Whiteson, Shimon; Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge? arXiv preprint arXiv:2011.09533 2020.0
Prinzing, Michael; Friendly superintelligent AI: all you need is love 3rd Conference on" Philosophy and Theory of Artificial Intelligence 288-301 2017.0 Springer
Javed, Khurram; Yao, Hengshuai; White, Martha; Is Fast Adaptation All You Need? arXiv preprint arXiv:1910.01705 2019.0
Abdu-Aguye, Mubarak G; Gomaa, Walid; Makihara, Yasushi; Yagi, Yasushi; Adaptive Pooling Is All You Need: An Empirical Study on Hyperparameter-insensitive Human Action Recognition Using Wearable Sensors 2020 International Joint Conference on Neural Networks (IJCNN) 1-6 2020.0 IEEE
Cui, Xuefeng; Attention is all you need for general-purpose protein structure embedding bioRxiv 2021.0 Cold Spring Harbor Laboratory
Xiao, Wencong; All You Need to Know about Scheduling Deep Learning Jobs
Roberts, Nicholas; Prabhu, Vinay Uday; McAteer, Matthew; Model weight theft with just noise inputs: The curious case of the petulant attacker arXiv preprint arXiv:1912.08987 2019.0
Gao, Yang; Meyer, Christian M; Mesgar, Mohsen; Gurevych, Iryna; Good Rewards Are All You Need: Reward Learning for Efficient Reinforcement Learning in Document Summarisation
Ramsey, Norman; All you need is the monad.. what monad was that again PPS Workshop 2016.0
Le, James; Meta-Learning Is All You Need MATH 789-Mathematics of Deep Learning 2020.0
Disambiguation, Cross-lingual Visual Verb Sense; Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need!
Isola, Phillip; Rethinking Few-Shot Image Classification: A Good Embedding is All You Need? Springer
Silva, Francisco Alejandro Gallegos; AlvaracĂ­n-Paula, Mario Eduardo; GonzĂĄlez, Hugo NicolĂĄs Acosta; Estimation of the causal effect of tourism advertising campaign All You Need is Ecuador on digital exposure Communications in Statistics: Case Studies, Data Analysis and Applications 6.0 2.0 247-269 2020.0 Taylor & Francis
Pedregosa, Fabian; Sufficient decrease is all you need
Meinert, Judith; All you need is a (heuristic) cue?
Joshi, Anirudh; QANet: Convolutions and Attention are All you Need
Ivanov, Andrei; Dryden, Nikoli; Ben-Nun, Tal; Li, Shigang; Hoefler, Torsten; Data Movement Is All You Need: A Case Study of Transformer Networks arXiv preprint arXiv:2007.00072 2020.0
Namazifar, Mahdi; Papangelis, Alexandros; Tur, Gokhan; Hakkani-TĂŒr, Dilek; Language Model is All You Need: Natural Language Understanding as Question Answering arXiv preprint arXiv:2011.03023 2020.0
Kostrikov, Ilya; Yarats, Denis; Fergus, Rob; Image augmentation is all you need: Regularizing deep reinforcement learning from pixels arXiv preprint arXiv:2004.13649 2020.0
Eikema, Bryan; Aziz, Wilker; Is map decoding all you need? the inadequacy of the mode in neural machine translation arXiv preprint arXiv:2005.10283 2020.0
Chen, Weijie; Xie, Di; Zhang, Yuan; Pu, Shiliang; All you need is a few shifts: Designing efficient convolutional neural networks for image classification Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 7241-7250 2019.0
TrÀuble, Frederik; Creager, Elliot; Kilbertus, Niki; Goyal, Anirudh; Locatello, Francesco; Schölkopf, Bernhard; Bauer, Stefan; Is independence all you need? on the generalization of representations learned from correlated data arXiv preprint arXiv:2006.07886 2020.0
Tamborrino, Alexandre; Pellicano, Nicola; Pannier, Baptiste; Voitot, Pascal; Naudin, Louise; Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning arXiv preprint arXiv:2004.14074 2020.0
Barnes, Jeff; Azure machine learning Microsoft Azure Essentials. 1st ed, Microsoft 2015.0
Uhlich, Stefan; Mauch, Lukas; Cardinaux, Fabien; Yoshiyama, Kazuki; Garcia, Javier Alonso; Tiedemann, Stephen; Kemp, Thomas; Nakamura, Akira; Mixed precision dnns: All you need is a good parametrization arXiv preprint arXiv:1905.11452 2019.0
Kalavri, Vasiliki; Liagouris, John; Hoffmann, Moritz; Dimitrova, Desislava; Forshaw, Matthew; Roscoe, Timothy; Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18) 783-798 2018.0
Le, James; Meta-Learning Is All You Need 2020.0
Wu, Junru; Dai, Xiyang; Chen, Dongdong; Chen, Yinpeng; Liu, Mengchen; Yu, Ye; Wang, Zhangyang; Liu, Zicheng; Chen, Mei; Yuan, Lu; Weak NAS Predictors Are All You Need arXiv preprint arXiv:2102.10490 2021.0
Zhu, Jing; Lu, Xingyu; Heimann, Mark; Koutra, Danai; Node Proximity Is All You Need: Unified Structural and Positional Node and Graph Embedding arXiv preprint arXiv:2102.13582 2021.0
Bilkhu, Manjot; Wang, Siyang; Dobhal, Tushar; Attention is all you need for Videos: Self-attention based Video Summarization using Universal Transformers arXiv preprint arXiv:1906.02792 2019.0
Prasad, Adarsh; Balakrishnan, Sivaraman; Ravikumar, Pradeep; A Robust Univariate Mean Estimator is All You Need International Conference on Artificial Intelligence and Statistics 4034-4044 2020.0 PMLR
Kumar, Saurabh; Kumar, Aviral; Levine, Sergey; Finn, Chelsea; One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL arXiv preprint arXiv:2010.14484 2020.0
Ivanov, Andrei; Dryden, Nikoli; Ben-Nun, Tal; Li, Shigang; Hoefler, Torsten; Data Movement Is All You Need: A Case Study on Optimizing Transformers arXiv e-prints arXiv: 2007.00072 2020.0
Ren, Xuanchi; Yang, Tao; Wang, Yuwang; Zeng, Wenjun; Do Generative Models Know Disentanglement? Contrastive Learning is All You Need arXiv preprint arXiv:2102.10543 2021.0
Boudiaf, Malik; Kervadec, Hoel; Masud, Ziko Imtiaz; Piantanida, Pablo; Ayed, Ismail Ben; Dolz, Jose; Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? arXiv preprint arXiv:2012.06166 2020.0
Blondé, Lionel; Strasser, Pablo; Kalousis, Alexandros; Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning arXiv preprint arXiv:2006.16785 2020.0
Bai, Shaojie; Kolter, J Zico; Koltun, Vladlen; Deep equilibrium models arXiv preprint arXiv:1909.01377 2019.0
Mishkin, Dmytro; Matas, Jiri; All you need is a good init arXiv preprint arXiv:1511.06422 2015.0
Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Lukasz; Polosukhin, Illia; Attention is all you need arXiv preprint arXiv:1706.03762 2017.0
Chollet, Francois; The limitations of deep learning Deep Learning With Python 2017.0 Manning Publications
Tran, Ba-Hien; Rossi, Simone; Milios, Dimitrios; Filippone, Maurizio; All You Need is a Good Functional Prior for Bayesian Deep Learning arXiv preprint arXiv:2011.12829 2020.0
Chen, Qiming; Wu, Ren; CNN is all you need arXiv preprint arXiv:1712.09662 2017.0
Bachlechner, Thomas; Majumder, Bodhisattwa Prasad; Mao, Huanru Henry; Cottrell, Garrison W; McAuley, Julian; Rezero is all you need: Fast convergence at large depth arXiv preprint arXiv:2003.04887 2020.0
Xie, Di; Xiong, Jiang; Pu, Shiliang; All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 6176-6185 2017.0
Guo, Yunhui; Li, Yandong; Wang, Liqiang; Rosing, Tajana; Depthwise convolution is all you need for learning multiple visual domains Proceedings of the AAAI Conference on Artificial Intelligence 33.0 1.0 8368-8375 2019.0
Li, Guohao; Xiong, Chenxin; Thabet, Ali; Ghanem, Bernard; Deepergcn: All you need to train deeper gcns arXiv preprint arXiv:2006.07739 2020.0
Jeddi, Ahmadreza; Shafiee, Mohammad Javad; Wong, Alexander; A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning arXiv preprint arXiv:2012.13628 2020.0
Gröndahl, Tommi; Pajola, Luca; Juuti, Mika; Conti, Mauro; Asokan, N; All You Need is" Love" Evading Hate Speech Detection Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security 2-12 2018.0
Hu, Ronghang; Singh, Amanpreet; Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer arXiv preprint arXiv:2102.10772 2021.0
Wang, Hao; Lu, Pu; Zhang, Hui; Yang, Mingkun; Bai, Xiang; Xu, Yongchao; He, Mengchao; Wang, Yongpan; Liu, Wenyu; All you need is boundary: Toward arbitrary-shaped text spotting Proceedings of the AAAI Conference on Artificial Intelligence 34.0 7.0 12160-12167 2020.0
Subakan, Cem; Ravanelli, Mirco; Cornell, Samuele; Bronzi, Mirko; Zhong, Jianyuan; Attention is All You Need in Speech Separation arXiv preprint arXiv:2010.13154 2020.0
Sharp, Nicholas; Attaiki, Souhaib; Crane, Keenan; Ovsjanikov, Maks; Diffusion is All You Need for Learning on Surfaces arXiv preprint arXiv:2012.00888 2020.0
O’Mahony, Niall; Campbell, Sean; Carvalho, Anderson; Harapanahalli, Suman; Hernandez, Gustavo Velasco; Krpalkova, Lenka; Riordan, Daniel; Walsh, Joseph; Deep learning vs. traditional computer vision Science and Information Conference 128-144 2019.0 Springer
Chaudhury, Ayan; Boudon, Frédéric; Godin, Christophe; 3D Plant Phenotyping: All You Need is Labelled Point Cloud Data European Conference on Computer Vision 244-260 2020.0 Springer
Lu, Amy X; Lu, Alex X; Moses, Alan; Evolution Is All You Need: Phylogenetic Augmentation for Contrastive Learning arXiv preprint arXiv:2012.13475 2020.0
Kato, Natsumi; Osone, Hiroyuki; Oomori, Kotaro; Ooi, Chun Wei; Ochiai, Yoichi; Gans-based clothes design: Pattern maker is all you need to design clothing Proceedings of the 10th Augmented Human International Conference 2019 1-7 2019.0
Klein, Tassilo; Nabi, Moin; Attention is (not) all you need for commonsense reasoning arXiv preprint arXiv:1905.13497 2019.0
Huang, Shaoli; Tao, Dacheng; All you need is a good representation: A multi-level and classifier-centric representation for few-shot learning arXiv preprint arXiv:1911.12476 2019.0
Wood-Doughty, Zach; Andrews, Nicholas; Dredze, Mark; Convolutions are all you need (for classifying character sequences) Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text 208-213 2018.0
Malach, Eran; Yehudai, Gilad; Shalev-Schwartz, Shai; Shamir, Ohad; Proving the lottery ticket hypothesis: Pruning is all you need International Conference on Machine Learning 6682-6691 2020.0 PMLR
Tian, Yonglong; Wang, Yue; Krishnan, Dilip; Tenenbaum, Joshua B; Isola, Phillip; Rethinking few-shot image classification: a good embedding is all you need? arXiv preprint arXiv:2003.11539 2020.0
Bojchevski, Aleksandar; Klicpera, Johannes; Perozzi, Bryan; Blais, Martin; Kapoor, Amol; Lukasik, Michal; GĂŒnnemann, Stephan; Is pagerank all you need for scalable graph neural networks? ACM KDD, MLG Workshop 2019.0
Zhao, Chen; Ghanem, Bernard; ThumbNet: One Thumbnail Image Contains All You Need for Recognition Proceedings of the 28th ACM International Conference on Multimedia 1506-1514 2020.0
Zisserman, AP; Afouras, T; Chung, JS; ASR is all you need: cross-modal distillation for lip reading Proceedings of the International Conference of Acoustics, Speech, and Signal Processing (ICASSP) 2020.0 IEEE
Nizan, Ori; Tal, Ayellet; Breaking the cycle-colleagues are all you need Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 7860-7869 2020.0
Prajwal, KR; Mukhopadhyay, Rudrabha; Namboodiri, Vinay P; Jawahar, CV; A lip sync expert is all you need for speech to lip generation in the wild Proceedings of the 28th ACM International Conference on Multimedia 484-492 2020.0
Wu, Tsung-Han; Hsieh, Chun-Chen; Chen, Yen-Hao; Chi, Po-Han; Lee, Hung-yi; Hand-crafted Attention is All You Need? A Study of Attention on Self-supervised Audio Transformer arXiv preprint arXiv:2006.05174 2020.0
Cao, Meng; Zou, Yuexian; All you need is a second look: Towards Tighter Arbitrary shape text detection ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2228-2232 2020.0 IEEE
Ayala, Daniel; Borrego, AgustĂ­n; HernĂĄndez, Inma; Rivero, Carlos R; Ruiz, David; AYNEC: all you need for evaluating completion techniques in knowledge graphs European Semantic Web Conference 397-411 2019.0 Springer
Snower, Michael; Kadav, Asim; Lai, Farley; Graf, Hans Peter; 15 Keypoints Is All You Need Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 6738-6748 2020.0
Cheng, Zehua; Wu, Yuxiang; Xu, Zhenghua; Lukasiewicz, Thomas; Wang, Weiyang; Segmentation is All You Need arXiv preprint arXiv:1904.13300 2019.0
Afouras, Triantafyllos; Chung, Joon Son; Zisserman, Andrew; Asr is all you need: Cross-modal distillation for lip reading ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2143-2147 2020.0 IEEE
Al-Dujaili, Abdullah; O'Reilly, Una-May; Sign bits are all you need for black-box attacks International Conference on Learning Representations 2019.0
Duan, Sufeng; Zhao, Hai; Attention Is All You Need for Chinese Word Segmentation arXiv preprint arXiv:1910.14537 2019.0
Gelly, Sylvain; Kurach, Karol; Michalski, Marcin; Zhai, Xiaohua; MemGEN: Memory is All You Need arXiv preprint arXiv:1803.11203 2018.0
Orseau, Laurent; Hutter, Marcus; Rivasplata, Omar; Logarithmic pruning is all you need Advances in Neural Information Processing Systems 33.0 2020.0
Komatsuzaki, Aran; One epoch is all you need arXiv preprint arXiv:1906.06669 2019.0
Entezari, Negin; Al-Sayouri, Saba A; Darvishzadeh, Amirali; Papalexakis, Evangelos E; All you need is low (rank) defending against adversarial attacks on graphs Proceedings of the 13th International Conference on Web Search and Data Mining 169-177 2020.0
Ramsauer, Hubert; SchĂ€fl, Bernhard; Lehner, Johannes; Seidl, Philipp; Widrich, Michael; Gruber, Lukas; Holzleitner, Markus; Pavlović, Milena; Sandve, Geir Kjetil; Greiff, Victor; Hopfield networks is all you need arXiv preprint arXiv:2008.02217 2020.0

About

A survey of all the 'X is all you need'


Languages

Language:TeX 100.0%