LIANGQINGYUAN / awesome_codegeneration

A list of papers and resources dedicated to code generation

• Code Generation
  • Datasets
    • Dataset for imperative programming language generation
    • Dataset for Text-to-SQL generation
  • Techniques
    • Generation architectures
    • Pretrained models
  • Tools
• Code Understanding
  • Datasets
  • Techniques
    • Variable representation
    • Code representation
    • Code and comment
    • Understanding with GNNs
    • Understanding with pre-trained models
• End

Code Generation

A list of papers and resources dedicated to code generation

Datasets

Dataset for imperative programming language generation

• Djongo: Learning to generate pseudo-code from source code using statistical machine translation, 2015
• Hearthstone: Latent Predictor Networks for Code Generation, 2016
• CoNaLa: Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow, 2018
• Concode: Mapping language to code in programmatic context, 2018
• APPS: Measuring coding challenge competence with apps, 2021
• HumanEval: Evaluating large language models trained on code, 2021
• CodeContest: Competition-Level Code Generation with AlphaCode, 2022
• Lyra: Lyra: A Benchmark for Turducken-Style Code Generation, 2022
  abstract

  This paper define a new code generation task: given a natural language comment, this task aims to generate a program in a base imperative language with an embedded declarative language. Then they present Lyra: a dataset in Python with embedded SQL. This dataset contains 2,000 carefully annotated database manip- ulation programs from real-world projects. Each program is paired with both a Chinese comment and an English comment.

Dataset for Text-to-SQL generation

• ATIS: Evaluation of spoken language systems: The ATIS domain,1990; Learning a neural semantic parser from user feedback, 2017

• GeoQuery: Learning to parse database queries using inductive logic programming, 1996; Towards a theory of natural language interfaces to databases, 2003

• Restaurants: Automated construction of database interfaces: Intergrating statistical and relational learning for semantic parsing, 2000

• Academic: Constructing an interactive natural language interface for relational databases, 2004

• Yelp&IMDB: SQLizer: query synthesis from natural language, 2017

• Scholar: Learning a neural semantic parser from user feedback, 2017

• Advising: Improving Text-to-SQL Evaluation Methodology, 2018

• WikiSQL: Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning, 2017

• Spider: Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task, 2018

• CSpider: A Pilot Study for Chinese SQL Semantic Parsing, 2019

• SQUALL: On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries, 2020

• TableQA: Tableqa: a large-scale chinese text-to-sql dataset for table-aware sql generation, 2020

• DuSQL: DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset, 2020

• KaggleDBQA: KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers, 2021

• SParC: Sparc: Cross-domain semantic parsing in context, 2019

• CoSQL: CoSQL: A conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases, 2019

• CHASE: Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL, 2021

Techniques

Generation architectures

• TreeGen: TreeGen: A Tree-Based Transformer Architecture for Code Generation, 2020

Pretrained models

• CodeBERT: CodeBERT: A Pre-Trained Model for Programming and Natural Languages, 2020
• GraphCodeBERT: GraphCodeBERT: Pre-training Code Representations with Data Flow, 2020
• CodeGPT: CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation, 2021
  abstract

  This paper introduce CodeXGLUE, a benchmark dataset and open challenge for code intelligence. It includes a collection of code intelligence tasks and a platform for model evaluation and comparison. CodeXGLUE stands for General Language Understanding Evaluation benchmark for CODE. It includes 14 datasets for 10 diversified code intelligence tasks covering the following scenarios: (1)code-code (clone detection, defect detection, cloze test, code completion, code repair, and code-to-code translation); (2)text-code (natural language code search, text-to-code generation); (3)code-text (code summarization); (4)text-text (documentation translation);
  
• CodeT5: CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation, EMNLP2021
• PLBART: Unified Pre-training for Program Understanding and Generation, NAACL2021
• CodeX: Evaluating large language models trained on code, 2021
• PANGU-CODER: PANGU-CODER: Program Synthesis with Function-Level Language Modeling, 2022
• CoditT5: CoditT5: Pretraining for Source Code and Natural Language Editing, ASE 2022
  abstract

  This paper propose a novel pretraining objective which explicitly models edits and use it to build CoditT5, a large language model for software-related editing tasks that is pretrained on large amounts of source code and natural language comments.
  
  They propose an edit-based output sequence representation: [Edit Plan] <s> [Target Sequence], where the model is trained to generate an edit plan ( 1 ) consisting of explicit edit operations that reconstruct the input sequence, followed by a separation token (<s>), and finally the target sequence ( 2 ) that matches the original input sequence.

Tools

• Gihub Copilot: GitHub Copilot uses the OpenAI Codex to suggest code and entire functions in real-time, right from your editor.
• Tabnine: Tabnine AI models are only trained on permissive open source licensed code, so your code will always remain yours.
• Amazon CodeWhisperer: Build applications faster with the ML-powered coding companion.
• Captain Stack: This feature is somewhat similar to Github Copilot's code suggestion. But instead of using AI, it sends your search query to Google, then retrieves StackOverflow and Github Gist answers and autocompletes them for you.

Code Understanding

Datasets

• BigCloneBench

  • provided by:Towards a Big Data Curated Benchmark of Inter-project Code Clones
  • filtered by: Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree
  abstract

  The dataset includes 901,028/415,416/415,416 pairs for training, validation and testing, respectively.

• POJ-104: Convolutional Neural Networks over Tree Structures for Programming Language Processing

  abstract

  POJ104 dataset is collected from an online judge platform, which consists of 104 program classes and includes 500 student-written C/C++ programs for each class.

Techniques

Variable representation

• IdBench: IdBench: Evaluating Semantic Representations of Identifier Names in Source Code, ICSE2021
• VarCLR: VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning, ICSE2022

Code representation

• IRGen: Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings, ICSE2022

Code and comment

• Associating Natural Language Comment and Source Code Entities, AAAI20

Understanding with GNNs

• Deep Just-In-Time Inconsistency Detection Between Comments and Source Cod, AAAI21

• Grape: Grape: Grammar-Preserving Rule Embedding, IJCAI2022

Understanding with pre-trained models

• Bridging pre-trained models and downstream tasks for source code understanding, ICSE2022

• ALERT: Natural Attack for Pre-trained Models of Code, ICSE2022

• Compressor: Compressing Pre-trained Models of Code into 3 MB, ASE2022

  abstract

  Compressor proposes a genetic algorithm (GA)-based strategy to guide the simplification process. They use the knowledge distillation technique to train the small model: unlabelled data is fed into the large model and the outputs are used as labels to train the small model.
  

End

Welcome to recommend code generation papers!

Please let me know if you have any interesting work related to code generation task!