A niche collection of AI4Code papers and other resources (dataset, tutorial, etc.), this list will focus mostly on the papers that use pre-trained models and deep learning techniques for programming languages processing. There are also other collections that cover a wider range of AI4Code papers, such as:
- https://ml4code.github.io/
- https://github.com/src-d/awesome-machine-learning-on-source-code (Due to the lack of maintainers, this collection may be obsolete).
The emphasis is on combining program analysis and deep learning to solve novel software engineering/programming languages task. In most cases, strong empirical results are required. Typically, new datasets are usually curated.
- Interational Conference on Software Engineering (ICSE)
- Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)
- Automated Software Engineering (ASE).
- Programming Language Design and Implementation (PLDI)
The emphasis is on desigining novel neural network architectures to process code. Typically, new datasets are usually curated.
- Association for the Advancement of Artificial Intelligence (AAAI)
- Interational Conference in Machine Learning (ICML)
- International Conference on Neural Information Processing Systems(NeurIPS)
- International Conference on Learning Representation (ICLR)
The emphasis is on applying NLP techniques for code, and the evaluation is primarily on running the models on known benchmark datasets; unique tasks are rarely introduced.
- Empirical Methods in Natural Language Processing (EMNLP)
- North American Chapter of the Association for Computational Linguistics (NAACL)
- Annual Meeting of the Association for Computational Linguistics (ACL)
- Large Language Models of Code Fail at Completing Code with Potential Bugs - Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, George Karypis.
- Large Language Models Meet NL2Code: A Survey - Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, Jian-Guang Lou (EMNLP 2023)
- [SKCODER: A Sketch-based Approach for Automatic Code Generation)(https://arxiv.org/pdf/2302.06144.pdf) - Jia Allen Li, Yongmin Li, Ge Li, Zhi Jin, Yiyang Hao, Xing Hu
- RepoFusion: Training Code Models to Understand Your Repository - Disha Shrivastava, Denis Kocetkov, Harm de Vries, Dzmitry Bahdanau, Torsten Scholak
- XCODEEVAL: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval - Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, Shafiq Joty
- CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation - Yue Wang, Weishi Wang, Shafiq Joty, Steven C.H. Hoi (EMNLP 2021) (CodeT5).
- CodeBERT:A Pre-Trained Model for Programming and Natural Language - Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou (EMNLP 2020 Findings) (CodeBERT).
- Learning and Evaluating Contextual Embedding of Source Code - Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, Kensen Shi. (ICML 2020) (CuBERT).
- GraphCodeBERT: Pre-training Code Representations with Data Flow - Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, Ming Zhou (ICLR 2021) (GraphCodeBERT).
- InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees -Nghi D. Q. BUI, Yijun YU, Lingxiao JIANG (ICSE 2021) (InferCode).
- Self-Supervised Learning for Code Retrieval and Summarization through Semantic-Preserving Program Transformations - Nghi D. Q. BUI, Yijun YU, Lingxiao JIANG (SIGIR 2021) (Corder).
- Unsupervised Translation of Programming Languages - Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, Guillaume Lample (NeurIPS 2020) (Transcoder).
- Contrastive Code Representation Learning - Paras Jain, Ajay Jain, Tianjun Zhang, Pieter Abbeel, Joseph E. Gonzalez, Ion Stoica (arxiv 2020) (ContraCode).
- CoTexT: Multi-task Learning with Code-Text Transformer, Long Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James Anibal, Alec Peltekian, Yanfang Ye (arXiv 2021) (CoTexT)
- How could Neural Networks understand Programs?, Dinglan Peng, Shuxin Zheng, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu (ICML 2021) (OSCAR)
- Unified Pre-training for Program Understanding and Generation, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang (NAACL 2021) (PLBART).
- Exploring Software Naturalness through Neural Language Models, Luca Buratti, Saurabh Pujar, Mihaela Bornea, Scott McCarley, Yunhui Zheng, Gaetano Rossiello, Alessandro Morari, Jim Laredo, Veronika Thost, Yufan Zhuang, Giacomo Domeniconi, (arXiv 2020) (C-BERT).
- PYMT5: multi-mode translation of natural language and PYTHON code with transformers, Colin B. Clement, Dawn Drain, Jonathan Timcheck, Alexey Svyatkovskiy, Neel Sundaresan (EMNLP 2020) (PYMT5).
- DOBF: A Deobfuscation Pre-Training Objective for Programming Languages, Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec, Guillaume Lample, (arXiv 2021) (DOBF).
- Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks, , Antonio Mastropaolo, Simone Scalabrino, Nathan Cooper, David Nader Palacio, Denys Poshyvanyk, Rocco Oliveto, Gabriele Bavota (ICSE 2021).
- CodeTrans: Towards Cracking the Language of Silicone’s Code Through Self-Supervised Deep Learning and High Performance Computing, Ahmed Elnaggar, Wei Ding, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Silvia Severini, Florian Matthes, Burkhard Rost (arXiv 2021/04( (CodeTrans).
- Disentangled Code Representation Learning for Multiple Programming Languages,Jingfeng Zhang, Haiwen Hong, Yin Zhang, Yao Wan, Ye Liu, Yulei Sui (ACL-Fingings 2021) (CODEDISEN).
- SYNCOBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation, Xin Wang, Yasheng Wang, Fei Mi, Pingyi Zhou, Yao Wan, Xiao Liu, Li Li, Hao Wu, Jin Liu, Xin Jiang, (arXiv 2021) (SYNCOBERT).
- TreeBERT: A Tree-Based Pre-Trained Model for Programming Language, Xue Jiang, Zhuoran Zheng, Chen Lyu, Liang Li, Lei Lyu, (UAI 2021) (TreeBERT).
- Empirical Study of Transformers for Source Code, Nadezhda Chirkova, Sergey Troshin (ESEC/FSE 2021).