NB: This repo is a WIP and I hope to keep refreshing and extending it as time allows
Welcome to awesome-ai-research, a personal collection of research papers that have piqued my interest in the expansive field of artificial intelligence, especially focusing on Large Language Models (LLMs) and their multifaceted applications.
Artificial intelligence is a dynamic field where innovation is constant. The papers included here reflect a range of work that I've found interesting - you may or may not too! :)
Here, you'll discover papers covering everything from enhancing the reasoning abilities of AI systems to innovative approaches for training and benchmarking. Each selection is driven by a personal interest and a belief in the importance of the work, rather than a comprehensive or systematic review.
I hope this collection serves as a springboard for your exploration and inspires a deeper understanding and appreciation of artificial intelligence.
This section presents papers exploring advanced reasoning capabilities of Large Language Models (LLMs). Topics include self-improvement techniques, multi-agent systems, ethical reasoning, mathematical reasoning, and approaches to enhance LLMs' ability to process and generate logically coherent and contextually relevant responses. Expect insights on enhancing code generation, debugging, and solving complex algorithmic challenges.
- General
- 2024
- Reason from Fallacy: Enhancing Large Language Models’ Logical Reasoning through Logical Fallacy Understanding (04/2024)
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (AlphaLLM) (04/2024)
- Stream of Search (SoS): Learning to Search in Language (04/2024)
- Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models (04/2024)
- DON’T TRUST: VERIFY – GROUNDING LLM QUANTITATIVE REASONING WITH AUTOFORMALIZATION (03/2024)
- Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking (03/2024)
- Self-Discover: Large Language Models Self-Compose Reasoning Structures (02/2024)
- 2023
- Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models (12/2023)
- Retrieval-Augmented Multi-Modal Chain-of-Thoughts Reasoning for Large Language Models (12/2023)
- MAMMOTH: BUILDING MATH GENERALIST MODELS THROUGH HYBRID INSTRUCTION TUNING (09/2023)
- Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models (09/2023)
- Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation (11/2023)
- HINT-ENHANCED IN-CONTEXT LEARNING WAKES LARGE LANGUAGE MODELS UP FOR KNOWLEDGE-INTENSIVE TASKS (11/2023)
- LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (10/2023)
- BRANCH-SOLVE-MERGE IMPROVES LARGE LANGUAGE MODEL EVALUATION AND GENERATION (10/2023)
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models (05/2023)
- Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (05/2023)
- Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework (05/2023)
- Causal Reasoning and Large Language Models: Opening a New Frontier for Causality (05/2023)
- Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models (06/2023)
- Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (04/2023)
- Progressive-Hint Prompting Improves Reasoning in Large Language Models (04/2023)
- 2024
- Coding
- 2024
- 2023
- AgentCoder: Multiagent-Code Generation with Iterative Testing and Optimisation (12/2023)
- INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair (11/2023)
- COFFEE: Boost Your Code LLMs by Fixing Bugs with Feedback (11/2023)
- MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning (11/2023)
- ClarifyGPT: Empowering LLM-based Code Generation with Intention Clarification (10/2023)
- Bridging Code Semantic and LLMs: Semantic Chain-of-Thought Prompting for Code Generation (10/2023)
- Logical reasoning
- Math
- Meta
Papers in this section address how expanding the contextual awareness of models can significantly enhance memory and attention mechanisms. This includes exploring architectures that handle long-term dependencies and infinite contexts to improve the working memory of transformers.
- Extension
- 2024
- Make Your LLM Fully Utilize the Context
- TransformerFAM: Feedback attention is working memory (04/2024)
- Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention (04/2024)
- InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences (02/2024)
- LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens (02/2024)
- LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning (01/2024)
- 2024
Focuses on optimizing the computational efficiency and practical deployment of LLMs. This includes reducing the computational load, increasing inference speed, improving resource allocation, and strategies to minimize costs without sacrificing performance.
- 2024
- LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding (04/2024)
- SnapKV : LLM Knows What You are Looking for Before Generation (04/2024)
- Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration (04/2024)
- ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference (04/2024)
- FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping (04/2024)
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models (04/2024)
- The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (02/2024)
- 2023
- Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy (12/2023)
- Exponential Faster Language Models (11/2023)
- Efficient LLM Inference on CPUs (11/2023)
- Cache me if you Can: an Online Cost-aware Teacher-Student Framework to Reduce the Calls to Large Language Models (10/2023)
- Herd: Using multiple, smaller LLMs to match the performances of proprietary, large LLMs via an intelligent composer (10/2023)
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills (08/2023)
- EdgeMoE: Fast On-Device Inference of MoE-based Large Language Models (08/2023)
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning (07/2023) 1
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (05/2023)
- Paper page - LLM in a flash: Efficient Large Language Model Inference with Limited Memory (12/2023)
This segment covers research on minimizing hallucinations—false or misleading information generated by LLMs. Papers propose various techniques for enhancing the fact-checking capabilities of models, ensuring more reliable and factual outputs.
- 2023
- CHAIN-OF-VERIFICATION REDUCES HALLUCINATION IN LARGE LANGUAGE MODELS (09/2023)
- Unveiling the Siren's Song: Towards Reliable Fact-Conflicting Hallucination Detection (10/2023)
- Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models (05/2023)
- “Oops, Did I Just Say That?” Testing and Repairing Unethical Suggestions of Large Language Models with Suggest-Critique-Reflect Process (05/2023)
- SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (03/2023)
Includes papers that apply LLMs to specialized fields such as data science, DevOps, and personalized learning. These studies explore how LLMs can be tailored to specific professional tasks, including automated code generation and data interpretation.
- Data Science
- Coding (and related)
- 2023
- Automated DevOps Pipeline Generation for Code Repositories using Large Language Models (12/2023)
- Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation (10/2023)
- Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agents (10/2023)
- Private-Library-Oriented Code Generation with Large Language Models (07/2023)
- Exploring the Robustness of Large Language Models for Solving Programming Problems (06/2023)
- Self-collaboration Code Generation via ChatGPT (04/2023)
- Constructing Effective In-Context Demonstration for Code Intelligence Tasks: An Empirical Study (04/2023)
- 2023
This section presents methodologies for assessing the performance of LLMs, particularly in handling extensive contexts and complex reasoning tasks. It includes innovative evaluation strategies that aim to provide more comprehensive and practical assessments of model capabilities.
Discusses methods and strategies for the continuous training and improvement of LLMs. Topics include token efficiency, scalable training strategies, and data autonomy to enhance the learning process without extensive resources.
-
General
- 2024
- 2023
-
"Continuous" learning
Explores how LLMs can be integrated with external tools or APIs to extend their functionality beyond pure text processing. This includes learning to interact with and manipulate external software tools or databases, thereby broadening the practical applications of LLMs.