prompt-injection

There are 20 repositories under prompt-injection topic.

CyberAlbSecOP / Awesome_GPT_Super_Prompting
ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.
adversarial-machine-learning agent ai assistant chatgpt gpt gpt-3 gpt-4 hacking jailbreak leaks llm prompt-engineering prompt-injection prompt-security prompts system-prompt
1942
protectai / llm-guard
The Security Toolkit for LLM Interactions
adversarial-machine-learning chatgpt large-language-models llm llm-security llmops prompt-engineering prompt-injection security-tools transformers
Language:Python 1485
protectai / rebuff
LLM Prompt Injection Detector
llm llmops prompt-engineering prompt-injection prompts security
Language:TypeScript 1204
abilzerian / LLM-Prompt-Library
Advanced Code and Text Manipulation Prompts for Various LLMs. Suitable for Deepseek, GPT o1, Claude, Llama3, Gemini, and other high-performance open-source LLMs.
ai apple-intelligence artificial-intelligence chatbot chatgpt chatgpt-api deepseek gpt gpt-4 machine-learning openai prompt prompt-engineering prompt-injection prompt-toolkit prompting prompts python siri text-analysis
995
whylabs / langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
large-language-models machine-learning nlg nlp observability prompt-engineering prompt-injection
Language:Jupyter Notebook 887
promptmap
utkusen / promptmap
a prompt injection scanner for custom LLM applications
ai-security chatgpt claude llm ollama prompt-engineering prompt-injection
Language:Python 755
brilliant-CV
yunanwg / brilliant-CV
💼 another CV template for your job application, yet powered by Typst and more
cv pdf prompt-injection resume resume-template typst typst-template
Language:Typst 532
zacfrulloni / Prompt-Engineering-Holy-Grail
# Prompt Engineering Hub ⭐️ lovable.dev no code builders: https://www.aidevelopers.tech/
ai ai-prompt-engineer ai-prompt-engineering artificial-intelligence generative-ai prompt prompt-engineer-course prompt-engineering prompt-engineering-certification prompt-engineering-course prompt-engineering-courses prompt-engineering-github prompt-engineering-jobs prompt-generator prompt-injection prompt-learning prompt-toolkit prompt-tuning prompts prompts-stable-diffusion
Language:HTML 420
tldrsec / prompt-injection-defenses
Every practical and proposed defense against prompt injection.
ai cybersecurity prompt-injection security
398
deadbits / vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
llm-security prompt-injection adversarial-attacks adversarial-machine-learning large-language-models llmops yara-scanner security-tools
Language:Python 355
automorphic-ai / aegis
Self-hardening firewall for large language models
adversarial-attacks llmops prompt-injection security large-language-models
Language:Python 263
langgptai / Awesome-Multimodal-Prompts
Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.
awesome awesome-list chatgpt dall-e dall-e3 dall-e3-prompts gpt4 gpt4v jailbreak-prompt multimodal multimodal-prompts newbing prompt-engineering prompt-injection prompts
246
dropbox / llm-security
Dropbox LLM Security research code and results
llm prompt-injection security
Language:Python 221
liu00222 / Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
llm-security llms prompt-injection prompt-injection-tool llm security-and-privacy
Language:Python 178
yunwei37 / prompt-hacker-collections
prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记
gpt gpt-4 prompt-engineering attack-defense prompt-injection
160
shell-nlp / gpt_server
gpt_server是一个用于生产级部署LLMs或Embedding的开源框架。
embedding gpt llama llm lmdeploy openai prompt-injection rerank text-moderation vllm tts fastchat infinity function-calling
Language:Python 158
lakeraai / pint-benchmark
A benchmark for prompt injection detection systems.
benchmark llm llm-benchmarking llm-security prompt-injection
Language:Jupyter Notebook 98
pasquini-dario / project_mantis
Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks
cybersecurity llms prompt-injection
Language:Python 65
HumanCompatibleAI / tensor-trust
A prompt injection game to collect data for robust ML research
game prompt-engineering prompt-injection security jailbreaks large-language-models llms llm django htmx prompting ctf
Language:Python 52
TrustAI-laboratory / Learn-Prompt-Hacking
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
jailbreak llm llm-security prompt-hack prompt-injection turorial llm-learning llm-tutorials prompt-engineering security-ctf
Language:Jupyter Notebook 47
gdalmau / lakera-gandalf-solutions
My inputs for the LLM Gandalf made by Lakera
ai ai-prompts gandalf lakera llm prompt-injection
41
PromptGuard
GPTSafe / PromptGuard
Build production ready apps for GPT using Node.js & TypeScript
chatgpt gpt gpt-2 gpt-3 openai prompt prompt-engineering prompt-attack prompt-injection prompt-caching prompt-hardening
Language:TypeScript 41
sinanw / llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
cybersecurity llm-prompting llm-security prompt-injection transformers-models
Language:Jupyter Notebook 37
jailbreakme-xyz / jailbreak
jailbreakme.xyz is an open-source decentralized app (dApp) where users are challenged to try and jailbreak pre-existing LLMs in order to find weaknesses and be rewarded. 🏆
ai bugbounty cryptocurrency cybersecurity prompt-engineering prompt-injection solana solana-program
Language:JavaScript 34
LostOxygen / llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
chatgpt confidentiality gpt llm llm-security machine-learning openai prompt-engineering prompt-injection deep-learning framework prompt-toolkit security systems-security transformers
Language:Python 34
gandalf_vs_gandalf
microsoft / gandalf_vs_gandalf
Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.
lakera llms prompt-injection
Language:Jupyter Notebook 29
grepstrength / WideOpenAI
Short list of indirect prompt injection attacks for OpenAI-based models.
azure-openai azure-openai-service ethical-hacking prompt-injection sql-query prompt-engin chatgpt gpt-4o large-language-models llm microsoft-copilot
25
MaxMLang / pytector
Easy to use LLM Prompt Injection Detection / Detector Python Package
huggingface huggingface-transformers llmops llms prompt-engineering prompt-injection python security
Language:Python 24
wearetyomsmnv / Awesome-LLMSecOps
LLM | Security | Operations in one github repo with good links and pictures.
ai awesome-list llm mlsecops mlsecurity owasp-top10 prompt-injection security llmsecops
Language:HTML 24
ZapDos7 / lakera-gandalf
My solutions for Lakera's Gandalf
ai-prompts artificial-intelligence llm password prompt-injection
24
peluche / deck-of-many-prompts
Manual Prompt Injection / Red Teaming Tool
jailbreak prompt-injection prompt-injection-tool red-team red-teaming-tools
Language:Python 22
SemanticBrainCorp / SemanticShield
The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).
chatgpt jailbreak llm moderation pii adversarial-attacks prompt-engineering prompt-injection security-tools
Language:Python 21
lakeraai / chainguard
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
langchain langchain-python llm llm-security prompt-injection
Language:Python 19
shaialon / ai-security-demos
🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:
ai aisec aisecurity data-poisoning llm prompt-injection
Language:JavaScript 18
CyberAlbSecOP / MINOTAUR_Impossible_GPT_Security_Challenge
MINOTAUR: The STRONGEST Secure Prompt EVER! Prompt Security Challenge, Impossible GPT Security, Prompts Cybersecurity, Prompting Vulnerabilities, FlowGPT, Secure Prompting, Secure LLMs, Prompt Hacker, Cutting-edge Ai Security, Unbreakable GPT Agent, Anti GPT Leak, System Prompt Security.
ai-jailbreak ai-jailbreak-prompts ai-security cyber-security gpt-security llm-security prompt-engineering prompt-injection prompt-security security-challenge system-prompt ai-cyber-security flow-gpt gpts-security prompt-security-challenge super-prompt
15
Safetorun / PromptDefender
A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.
ai ai-security prompt-injection security
Language:Go 15

prompt-injection

CyberAlbSecOP / Awesome_GPT_Super_Prompting

protectai / llm-guard

protectai / rebuff

abilzerian / LLM-Prompt-Library

whylabs / langkit

utkusen / promptmap

yunanwg / brilliant-CV

zacfrulloni / Prompt-Engineering-Holy-Grail

tldrsec / prompt-injection-defenses

deadbits / vigil-llm

automorphic-ai / aegis

langgptai / Awesome-Multimodal-Prompts

dropbox / llm-security

liu00222 / Open-Prompt-Injection

yunwei37 / prompt-hacker-collections

shell-nlp / gpt_server

lakeraai / pint-benchmark

pasquini-dario / project_mantis

HumanCompatibleAI / tensor-trust

TrustAI-laboratory / Learn-Prompt-Hacking

gdalmau / lakera-gandalf-solutions

GPTSafe / PromptGuard

sinanw / llm-security-prompt-injection

jailbreakme-xyz / jailbreak

LostOxygen / llm-confidentiality

microsoft / gandalf_vs_gandalf

grepstrength / WideOpenAI

MaxMLang / pytector

wearetyomsmnv / Awesome-LLMSecOps

ZapDos7 / lakera-gandalf

peluche / deck-of-many-prompts

SemanticBrainCorp / SemanticShield

lakeraai / chainguard

shaialon / ai-security-demos

CyberAlbSecOP / MINOTAUR_Impossible_GPT_Security_Challenge

Safetorun / PromptDefender