Giters

shaoyuyoung / DecryptPrompt

梳理Prompt范式相关模型，模型解读，AIGC应用

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

DecryptPrompt

持续更新以下内容，Star to keep updated~

  1. Prompt和LLM相关论文按细分方向梳理
  2. AIGC相关应用
  3. Prompt指南和教程
  4. ChatGPT及AGI相关解读
  5. 开源大模型
  6. ChatGPT相关商业应用 [WIP]

My blogs

Resources

paper List

Recommend Blog

Tools & Tutorial

  • ClickPrompt: 为各种prompt加持的工具生成指令包括Difussion，chatgpt
  • Prompt-Engineer-Guide: 如何写prompt的系列教学指南 ⭐
  • ChatGPT ShortCut：提供各式场景下的Prompt范例，支持搜索
  • PromptPerfect:用魔法打败魔法，输入关键词，模型创建条理清晰的最美提示词
  • openAI: ChatGPT出API啦, 价格下降10倍！
  • OpenAI Cookbook: 提供OpenAI模型使用示例 ⭐

AIGC playground

  • AI Topiah: 聆心智能AI角色聊天，和路飞唠了两句，多少有点中二之魂在燃烧
  • chatbase: 情感角色聊天，还没尝试
  • Vana: virtual DNA, 通过聊天创建虚拟自己！概念很炫
  • New Bing：需要连外网否则会重定向到bing**，需要申请waitlist
  • WriteSonic：AI写作，支持对话和定向创作如广告文案，商品描述, 支持Web检索是亮点，支持中文
  • copy.ai: WriteSonic竞品，亮点是像论文引用一样每句话都有对应网站链接，可以一键复制到右边的创作Markdown，超级好用！ :star1:
  • NotionAI：智能Markdown，适用真相！在创作中用command调用AI辅助润色，扩写，检索内容，给创意idea
  • Jasper: 同上，全是竞品哈哈
  • ChatExcel: 指令控制excel计算，对熟悉excel的有些鸡肋，对不熟悉的有点用
  • ChatPaper: 根据输入关键词，自动在arxiv上下载最新的论文，并对论文进行摘要总结，可以在huggingface上试用！
  • copy.down: 中文的营销文案生成，只能定向创作，支持关键词到文案的生成
  • Copilot: 要付费哟
  • Fauxpilot: copilot本地开源替代
  • CodeGex: 国内替代品，还没试过
  • dreamstudio.ai: 开创者，Stable Difussion， 有试用quota
  • midjourney: 开创者，艺术风格为主
  • Dall.E: 三巨头这就凑齐了
  • ControlNet: 为绘画创作加持可控性
  • GFPGAN: 照片修复

开源模型

国外

  • OPT-IML: Meta复刻GPT3，up to 175B, 不过效果并不及GPT3
  • Bloom：BigScience出品，规模最大176B, 感觉应该对标text-davinci-002
  • T0: BigScience出品，3B~11B的在T5进行指令微调的模型
  • LLaMA:Meta开源指令微调LLM，规模70 亿到 650 亿不等
  • ChatLLaMA: 基于RLHF微调了LLaMA
  • MetaLM: 微软开源的大规模自监督预训练模型
  • Alpaca: 斯坦福开源的使用52k数据在7B的LLaMA上微调得到，据说效果类似text-davinci-003, 模型不久后会发布

国内

  • 国内开源模型魔塔社区：https://www.modelscope.cn/home
  • PromptCLUE: 多任务Prompt语言模型
  • Chatyuan：基于PromptCLUE训练的对话模型
  • PLUG: 阿里达摩院发布的大模型，提交申请会给下载链接
  • CPM2.0: 智源发布CPM2.0
  • Moss: 复旦发布的大模型
  • GLM: 清华发布的中英双语130B大模型

Papers

Survey

  • Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing ⭐
  • Paradigm Shift in Natural Language Processing
  • Pre-Trained Models: Past, Present and Future

LLM Ability Analysis & Probing

  • How does in-context learning work? A framework for understanding the differences from traditional supervised learning
  • Why can GPT learn in-context? Language Model Secretly Perform Gradient Descent as Meta-Optimizers
  • Emerging Ability of Large Language Models
  • Rethinking the Role of Demonstrations What Makes incontext learning work?
  • Can Explanations Be Useful for Calibrating Black Box Models

Tunning Free Prompt

  • GPT2: Language Models are Unsupervised Multitask Learners
  • GPT3: Language Models are Few-Shot Learners ⭐
  • LAMA: Language Models as Knowledge Bases?
  • AutoPrompt: Eliciting Knowledge from Language Models

Fix-Prompt LM Tunning

  • T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  • PET-TC(a): Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference ⭐
  • PET-TC(b): PETSGLUE It’s Not Just Size That Matters Small Language Models are also few-shot learners
  • GenPET: Few-Shot Text Generation with Natural Language Instructions
  • LM-BFF: Making Pre-trained Language Models Better Few-shot Learners ⭐
  • ADEPT: Improving and Simplifying Pattern Exploiting Training

Fix-LM Prompt Tunning

  • Prefix-tuning: Optimizing continuous prompts for generation
  • Prompt-tunning: The power of scale for parameter-efficient prompt tuning ⭐
  • P-tunning: GPT Understands Too ⭐
  • WARP: Word-level Adversarial ReProgramming

LM + Prompt Tunning

  • P-tunning v2: Prompt Tuning Can Be Comparable to Fine-tunning Universally Across Scales and Tasks
  • PTR: Prompt Tuning with Rules for Text Classification
  • PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains

Instruction Tunning LLMs

  • Flan: FINETUNED LANGUAGE MODELS ARE ZERO-SHOT LEARNERS ⭐
  • Flan-T5: Scaling Instruction-Finetuned Language Models
  • Instruct-GPT: Training language models to follow instructions with human feedback star:
  • T0: MULTITASK PROMPTED TRAINING ENABLES ZERO-SHOT TASK GENERALIZATION
  • k-INSTRUCT: SUPER-NATURALINSTRUCTIONS: Generalization via Declarative Instructions on 1600+ NLP Tasks

Train for Dialogue

  • LaMDA: Language Models for Dialog Applications
  • Sparrow: Improving alignment of dialogue agents via targeted human judgements star:
  • BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
  • How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

Chain of Thought

  • Chain of Thought Prompting Elicits Reasoning in Large Language Models ⭐
  • COMPLEXITY-BASED PROMPTING FOR MULTI-STEP REASONING
  • SELF-CONSISTENCY IMPROVES CHAIN OF THOUGHT REASONING IN LANGUAGE MODELS
  • Large Language Models are Zero-Shot Reasoners
  • PaLM: Scaling Language Modeling with Pathways

RLHF

  • Deep reinforcement learning from human preferences
  • PPO: Proximal Policy Optimization Algorithms ⭐
  • InstrutGPT序作：learning to summarize from human feedback
  • InstructGPT: Training language models to follow instructions with human feedback ⭐
  • RL4LM：IS REINFORCEMENT LEARNING (NOT) FOR NATURAL LANGUAGE PROCESSING BENCHMARKS

About

梳理Prompt范式相关模型，模型解读，AIGC应用