prompt-injection

There are 16 repositories under prompt-injection topic.

CyberAlbSecOP / Awesome_GPT_Super_Prompting
ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.
adversarial-machine-learning chatgpt gpt gpt-3 gpt-4 hacking jailbreak leaks llm prompt-engineering prompt-injection prompts agent ai assistant prompt-security system-prompt
1416
protectai / llm-guard
The Security Toolkit for LLM Interactions
large-language-models llm chatgpt security-tools adversarial-machine-learning prompt-engineering prompt-injection llmops llm-security transformers
Language:Python 1309
protectai / rebuff
LLM Prompt Injection Detector
llm llmops prompt-engineering prompts security prompt-injection
Language:TypeScript 1144
whylabs / langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
large-language-models machine-learning nlg nlp observability prompt-engineering prompt-injection
Language:Jupyter Notebook 862
abilzerian / LLM-Prompt-Library
Advanced Code and Text Manipulation Prompts for Various LLMs. Suitable for Siri, GPT-4o, Claude, Llama3, Gemini, and other high-performance open-source LLMs.
ai apple-intelligence artificial-intelligence chatbot chatgpt chatgpt-api gpt gpt-3 gpt-4 machine-learning openai prompt prompt-engineering prompt-injection prompt-toolkit prompting prompts python siri text-analysis
791
promptmap
utkusen / promptmap
automatically tests prompt injection attacks on ChatGPT instances
chatgpt prompt-engineering prompt-injection llm
Language:Python 667
brilliant-CV
mintyfrankie / brilliant-CV
💼 another CV template for your job application, yet powered by Typst and more
cv pdf typst typst-template resume resume-template prompt-injection
Language:Typst 479
tldrsec / prompt-injection-defenses
Every practical and proposed defense against prompt injection.
ai cybersecurity prompt-injection security
358
zacfrulloni / Prompt-Engineering-Holy-Grail
# Prompt Engineering Hub ⭐️ If you find this helpful, give it a star to show your support! This repository is a one-stop resource for prompt engineering. Also available on: https://promptengineeringhub.dev/
ai artificial-intelligence generative-ai prompt prompt-engineering prompt-generator prompt-injection prompt-learning prompt-toolkit prompt-tuning prompts prompts-stable-diffusion ai-prompt-engineer ai-prompt-engineering prompt-engineer-course prompt-engineering-certification prompt-engineering-course prompt-engineering-courses prompt-engineering-github prompt-engineering-jobs
Language:HTML 331
deadbits / vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
llm-security prompt-injection adversarial-attacks adversarial-machine-learning large-language-models llmops yara-scanner security-tools
Language:Python 330
automorphic-ai / aegis
Self-hardening firewall for large language models
adversarial-attacks llmops prompt-injection security large-language-models
Language:Python 259
langgptai / Awesome-Multimodal-Prompts
Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.
chatgpt gpt4 multimodal prompt-engineering prompts gpt4v newbing multimodal-prompts awesome awesome-list jailbreak-prompt prompt-injection dall-e dall-e3 dall-e3-prompts
227
dropbox / llm-security
Dropbox LLM Security research code and results
llm prompt-injection security
Language:Python 219
liu00222 / Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
llm-security llms prompt-injection prompt-injection-tool llm security-and-privacy
Language:Python 158
yunwei37 / prompt-hacker-collections
prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记
gpt gpt-4 prompt-engineering attack-defense prompt-injection
137
lakeraai / pint-benchmark
A benchmark for prompt injection detection systems.
benchmark llm llm-benchmarking llm-security prompt-injection
Language:Jupyter Notebook 94
pasquini-dario / project_mantis
Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks
cybersecurity llms prompt-injection
Language:Python 59
HumanCompatibleAI / tensor-trust
A prompt injection game to collect data for robust ML research
game prompt-engineering prompt-injection security jailbreaks large-language-models llms llm django htmx prompting ctf
Language:Python 49
gdalmau / lakera-gandalf-solutions
My inputs for the LLM Gandalf made by Lakera
ai ai-prompts gandalf lakera llm prompt-injection
40
PromptGuard
GPTSafe / PromptGuard
Build production ready apps for GPT using Node.js & TypeScript
chatgpt gpt gpt-2 gpt-3 openai prompt prompt-engineering prompt-attack prompt-injection prompt-caching prompt-hardening
Language:TypeScript 39
sinanw / llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
cybersecurity llm-prompting llm-security prompt-injection transformers-models
Language:Jupyter Notebook 35
LostOxygen / llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
chatgpt confidentiality gpt llm llm-security machine-learning openai prompt-engineering prompt-injection deep-learning framework prompt-toolkit security systems-security transformers
Language:Python 30
gandalf_vs_gandalf
microsoft / gandalf_vs_gandalf
Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.
lakera llms prompt-injection
Language:Jupyter Notebook 27
MaxMLang / pytector
A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.
huggingface huggingface-transformers llmops llms prompt-engineering prompt-injection python security
Language:Python 24
TrustAI-laboratory / Learn-Prompt-Hacking
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
jailbreak llm llm-security prompt-hack prompt-injection turorial llm-learning llm-tutorials prompt-engineering security-ctf
Language:Jupyter Notebook 24
probonodev / jailbreak
jailbreakme.xyz is an open-source decentralized app (dApp) where users are challenged to try and jailbreak pre-existing LLMs in order to find weaknesses and be rewarded. 🏆
ai bugbounty cryptocurrency cybersecurity prompt-engineering prompt-injection solana solana-program
Language:JavaScript 21
grepstrength / WideOpenAI
Short list of indirect prompt injection attacks for OpenAI-based models.
azure-openai azure-openai-service ethical-hacking prompt-injection sql-query prompt-engin chatgpt gpt-4o large-language-models llm microsoft-copilot
20
SemanticBrainCorp / SemanticShield
The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).
chatgpt jailbreak llm moderation pii adversarial-attacks prompt-engineering prompt-injection security-tools
Language:Python 20
wearetyomsmnv / Awesome-LLMSecOps
LLM | Security | Operations in one github repo with good links and pictures.
ai awesome-list llm mlsecops mlsecurity owasp-top10 prompt-injection security llmsecops
20
ZapDos7 / lakera-gandalf
My solutions for Lakera's Gandalf
ai-prompts artificial-intelligence llm password prompt-injection
20
lakeraai / chainguard
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
langchain langchain-python llm llm-security prompt-injection
Language:Python 18
shaialon / ai-security-demos
🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:
ai aisec aisecurity data-poisoning llm prompt-injection
Language:JavaScript 16
qroa / QROA
QROA: A Black-Box Query-Response Optimization Attack on LLMs
adversarial-attacks ai-safety black-box-attacks black-box-optimization jailbreak llm q-learning reinforcement-learning large-language-models prompt-injection
Language:Python 14
levitation-opensource / Manipulative-Expression-Recognition
MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.
benchmarking conversation-analysis conversation-analytics fraud-detection fraud-prevention llm llm-security llm-test llm-training manipulation prompt-engineering prompt-injection psychometrics misinformation expression-recognition sentiment-analysis sentiment-classification human-computer-interaction human-robot-interaction transparency
Language:HTML 13
peluche / deck-of-many-prompts
Manual Prompt Injection / Red Teaming Tool
jailbreak prompt-injection prompt-injection-tool red-team red-teaming-tools
Language:Python 11
Safetorun / PromptDefender
A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.
ai ai-security prompt-injection security
Language:Go 11

prompt-injection

CyberAlbSecOP / Awesome_GPT_Super_Prompting

protectai / llm-guard

protectai / rebuff

whylabs / langkit

abilzerian / LLM-Prompt-Library

utkusen / promptmap

mintyfrankie / brilliant-CV

tldrsec / prompt-injection-defenses

zacfrulloni / Prompt-Engineering-Holy-Grail

deadbits / vigil-llm

automorphic-ai / aegis

langgptai / Awesome-Multimodal-Prompts

dropbox / llm-security

liu00222 / Open-Prompt-Injection

yunwei37 / prompt-hacker-collections

lakeraai / pint-benchmark

pasquini-dario / project_mantis

HumanCompatibleAI / tensor-trust

gdalmau / lakera-gandalf-solutions

GPTSafe / PromptGuard

sinanw / llm-security-prompt-injection

LostOxygen / llm-confidentiality

microsoft / gandalf_vs_gandalf

MaxMLang / pytector

TrustAI-laboratory / Learn-Prompt-Hacking

probonodev / jailbreak

grepstrength / WideOpenAI

SemanticBrainCorp / SemanticShield

wearetyomsmnv / Awesome-LLMSecOps

ZapDos7 / lakera-gandalf

lakeraai / chainguard

shaialon / ai-security-demos

qroa / QROA

levitation-opensource / Manipulative-Expression-Recognition

peluche / deck-of-many-prompts

Safetorun / PromptDefender