henry-yeh / FM-COP

Recent research papers about [Foundation Models for Combinatorial Optimization]

Foundation Models for Combinatorial Optimization

FM4CO contains interesting research papers (1) using Existing Large Language Models for Combinatorial Optimization, and (2) building Domain Foundation Models for Combinatorial Optimization.

---

LLMs for Combinatorial Optimization

Most work uses existing LLMs to generate/improve solutions*, algorithms* (hyper-heuristic), or parameters* (hyper-network), achieving remarkable performance when combined with problem-specific heuristics or general meta-heuristics. Other work leverages LLMs to explore the interpretability* of COP solvers, or to achieve automation* of problem formulation or domain tools's usage through text prompts. This research direction may receive increasing attention due to the power of LLMs. Currently, we mainly focus on the below problems, and may include more variants (e.g., graph-based COPs) as the community grows:

• Traveling Salesman Problem
• Vehicle Routing Problem
• Scheduling Problem
• (Mixed) Integer Linear Programming

Date	Paper	Link	Problem	Venue	Remark*
2023.09	Can Language Models Solve Graph Problems in Natural Language?		Graph	NeurIPS 2023	Solution
2023.09	Large Language Models as Optimizers		TSP	ICLR 2024	Solution
2023.10	Chain-of-Experts: When LLMs Meet Complex Operations Research Problems		MILP	ICLR 2024	Automation
2023.10	OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models		MILP	ICML 2024	Automation
2023.10	AI-Copilot for Business Optimisation: A Framework and A Case Study in Production Scheduling		JSSP	ArXiv	Automation
2023.11	Large Language Models as Evolutionary Optimizers		TSP	ArXiv	Solution
2023.11	Algorithm Evolution Using Large Language Model		TSP	ArXiv	Algorithm
2023.12	Mathematical discoveries from program search with large language models		BPP	Nature	Algorithm
2024.02	Large Language Models as Hyper-Heuristics for Combinatorial Optimization		TSP,VRP,OP, MKP,BPP,EDA	ArXiv	Algorithm
2024.02	AutoSAT: Automatically Optimize SAT Solvers via Large Language Models		SAT	ArXiv	Algorithm
2024.02	From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto		MILP	ArXiv	Automation
2024.03	How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems		VRP	ArXiv	Solution
2024.03	RouteExplainer: An Explanation Framework for Vehicle Routing Problem		VRP	PAKDD 2024	Interpretability
2024.03	From Words to Routes: Applying Large Language Models to Vehicle Routing		VRP	ArXiv	Algorithm
2024.05	Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Mode		TSP,BPP, FJSSP	ICML 2024	Algorithm

---

Domain FMs for Combinatorial Optimization

Building a domain foundation model that can solve a wide range of COPs may be an interesting and challenging topic. Some recent work puts efforts towards this ambitious goal by devising a unified architecture* or representation* for different COPs.

Date	Paper	Link	Problem	Venue	Remark*	Paradigm
2023.05	Efficient Training of Multi-task Combinatorial Neural Solver with Multi-armed Bandits		TSP,VRP, OP,KP	ArXiv	Architecture
2024.02	Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalization		VRP	ArXiv	Architecture	Compositional Zero-Shot
2024.03	Towards a Generic Representation of Combinatorial Problems for Learning-Based Approaches		SAT,TSP, COL,KP	ArXiv	Representation
2024.04	Cross-Problem Learning for Solving Vehicle Routing Problems		TSP,OP, PCTSP	IJCAI 2024	Architecture	Efficient Fine-Tuning
2024.05	MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts		VRP	ICML 2024	Architecture