prompt-injection

There are 17 repositories under prompt-injection topic.

CyberAlbSecOP / Awesome_GPT_Super_Prompting
ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.
adversarial-machine-learning chatgpt gpt gpt-3 gpt-4 hacking jailbreak leaks llm prompt-engineering prompt-injection prompts agent ai assistant prompt-security system-prompt
1723
protectai / llm-guard
The Security Toolkit for LLM Interactions
adversarial-machine-learning chatgpt large-language-models llm llm-security llmops prompt-engineering prompt-injection security-tools transformers
Language:Python 1376
protectai / rebuff
LLM Prompt Injection Detector
llm llmops prompt-engineering prompt-injection prompts security
Language:TypeScript 1168
abilzerian / LLM-Prompt-Library
Advanced Code and Text Manipulation Prompts for Various LLMs. Suitable for Siri, GPT-4o, Claude, Llama3, Gemini, and other high-performance open-source LLMs.
ai artificial-intelligence chatbot chatgpt chatgpt-api gpt gpt-3 gpt-4 machine-learning openai prompt prompt-engineering prompt-toolkit prompting prompts python text-analysis prompt-injection siri apple-intelligence
894
whylabs / langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
large-language-models machine-learning nlg nlp observability prompt-engineering prompt-injection
Language:Jupyter Notebook 871
promptmap
utkusen / promptmap
a prompt injection scanner for custom LLM applications
ai-security chatgpt claude llm ollama prompt-engineering prompt-injection
Language:Python 722
brilliant-CV
yunanwg / brilliant-CV
💼 another CV template for your job application, yet powered by Typst and more
cv pdf prompt-injection resume resume-template typst typst-template
Language:Typst 502
zacfrulloni / Prompt-Engineering-Holy-Grail
# Prompt Engineering Hub ⭐️ If you find this helpful, give it a star to show your support! This repository is a one-stop resource for prompt engineering. Also available on: https://promptengineeringhub.dev/
ai artificial-intelligence generative-ai prompt prompt-engineering prompt-generator prompt-injection prompt-learning prompt-toolkit prompt-tuning prompts prompts-stable-diffusion ai-prompt-engineer ai-prompt-engineering prompt-engineer-course prompt-engineering-certification prompt-engineering-course prompt-engineering-courses prompt-engineering-github prompt-engineering-jobs
Language:HTML 388
tldrsec / prompt-injection-defenses
Every practical and proposed defense against prompt injection.
ai cybersecurity prompt-injection security
381
deadbits / vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
adversarial-attacks adversarial-machine-learning large-language-models llm-security llmops prompt-injection security-tools yara-scanner
Language:Python 341
automorphic-ai / aegis
Self-hardening firewall for large language models
adversarial-attacks large-language-models llmops prompt-injection security
Language:Python 261
langgptai / Awesome-Multimodal-Prompts
Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.
awesome awesome-list chatgpt dall-e dall-e3 dall-e3-prompts gpt4 gpt4v jailbreak-prompt multimodal multimodal-prompts newbing prompt-engineering prompt-injection prompts
239
dropbox / llm-security
Dropbox LLM Security research code and results
llm prompt-injection security
Language:Python 220
liu00222 / Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
llm-security llms prompt-injection prompt-injection-tool llm security-and-privacy
Language:Python 167
yunwei37 / prompt-hacker-collections
prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记
gpt gpt-4 prompt-engineering attack-defense prompt-injection
156
shell-nlp / gpt_server
gpt_server是一个用于生产级部署LLMs或Embedding的开源框架。
embedding gpt llama llm lmdeploy openai prompt-injection rerank text-moderation vllm tts fastchat infinity function-calling
Language:Python 148
lakeraai / pint-benchmark
A benchmark for prompt injection detection systems.
benchmark llm llm-benchmarking llm-security prompt-injection
Language:Jupyter Notebook 95
pasquini-dario / project_mantis
Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks
cybersecurity llms prompt-injection
Language:Python 60
HumanCompatibleAI / tensor-trust
A prompt injection game to collect data for robust ML research
game prompt-engineering prompt-injection security jailbreaks large-language-models llms llm django htmx prompting ctf
Language:Python 50
gdalmau / lakera-gandalf-solutions
My inputs for the LLM Gandalf made by Lakera
ai ai-prompts gandalf lakera llm prompt-injection
41
PromptGuard
GPTSafe / PromptGuard
Build production ready apps for GPT using Node.js & TypeScript
chatgpt gpt gpt-2 gpt-3 openai prompt prompt-attack prompt-caching prompt-engineering prompt-hardening prompt-injection
Language:TypeScript 41
sinanw / llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
cybersecurity llm-prompting llm-security prompt-injection transformers-models
Language:Jupyter Notebook 36
TrustAI-laboratory / Learn-Prompt-Hacking
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
jailbreak llm llm-security prompt-hack prompt-injection turorial llm-learning llm-tutorials prompt-engineering security-ctf
Language:Jupyter Notebook 36
jailbreakme-xyz / jailbreak
jailbreakme.xyz is an open-source decentralized app (dApp) where users are challenged to try and jailbreak pre-existing LLMs in order to find weaknesses and be rewarded. 🏆
ai bugbounty cryptocurrency cybersecurity prompt-engineering prompt-injection solana solana-program
Language:JavaScript 32
LostOxygen / llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
chatgpt confidentiality gpt llm llm-security machine-learning openai prompt-engineering prompt-injection deep-learning framework prompt-toolkit security systems-security transformers
Language:Python 32
gandalf_vs_gandalf
microsoft / gandalf_vs_gandalf
Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.
lakera llms prompt-injection
Language:Jupyter Notebook 27
MaxMLang / pytector
A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.
huggingface huggingface-transformers llmops llms prompt-engineering prompt-injection python security
Language:Python 24
wearetyomsmnv / Awesome-LLMSecOps
LLM | Security | Operations in one github repo with good links and pictures.
ai awesome-list llm mlsecops mlsecurity owasp-top10 prompt-injection security llmsecops
Language:HTML 24
grepstrength / WideOpenAI
Short list of indirect prompt injection attacks for OpenAI-based models.
azure-openai azure-openai-service ethical-hacking prompt-injection sql-query prompt-engin chatgpt gpt-4o large-language-models llm microsoft-copilot
23
ZapDos7 / lakera-gandalf
My solutions for Lakera's Gandalf
ai-prompts artificial-intelligence llm password prompt-injection
21
SemanticBrainCorp / SemanticShield
The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).
chatgpt jailbreak llm moderation pii adversarial-attacks prompt-engineering prompt-injection security-tools
Language:Python 19
lakeraai / chainguard
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
langchain langchain-python llm llm-security prompt-injection
Language:Python 18
shaialon / ai-security-demos
🤯 AI Security EXPOSED! Live Demos Showing Hidden Risks of 🤖 Agentic AI Flows: 💉Prompt Injection, ☣️ Data Poisoning. Watch the recorded session:
ai aisec aisecurity data-poisoning llm prompt-injection
Language:JavaScript 17
peluche / deck-of-many-prompts
Manual Prompt Injection / Red Teaming Tool
jailbreak prompt-injection prompt-injection-tool red-team red-teaming-tools
Language:Python 15
qroa / QROA
QROA: A Black-Box Query-Response Optimization Attack on LLMs
adversarial-attacks ai-safety black-box-attacks black-box-optimization jailbreak llm q-learning reinforcement-learning large-language-models prompt-injection
Language:Python 14
levitation-opensource / Manipulative-Expression-Recognition
MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.
benchmarking conversation-analysis conversation-analytics expression-recognition fraud-detection fraud-prevention human-computer-interaction human-robot-interaction llm llm-security llm-test llm-training manipulation misinformation prompt-engineering prompt-injection psychometrics sentiment-analysis sentiment-classification transparency
Language:HTML 13

prompt-injection

CyberAlbSecOP / Awesome_GPT_Super_Prompting

protectai / llm-guard

protectai / rebuff

abilzerian / LLM-Prompt-Library

whylabs / langkit

utkusen / promptmap

yunanwg / brilliant-CV

zacfrulloni / Prompt-Engineering-Holy-Grail

tldrsec / prompt-injection-defenses

deadbits / vigil-llm

automorphic-ai / aegis

langgptai / Awesome-Multimodal-Prompts

dropbox / llm-security

liu00222 / Open-Prompt-Injection

yunwei37 / prompt-hacker-collections

shell-nlp / gpt_server

lakeraai / pint-benchmark

pasquini-dario / project_mantis

HumanCompatibleAI / tensor-trust

gdalmau / lakera-gandalf-solutions

GPTSafe / PromptGuard

sinanw / llm-security-prompt-injection

TrustAI-laboratory / Learn-Prompt-Hacking

jailbreakme-xyz / jailbreak

LostOxygen / llm-confidentiality

microsoft / gandalf_vs_gandalf

MaxMLang / pytector

wearetyomsmnv / Awesome-LLMSecOps

grepstrength / WideOpenAI

ZapDos7 / lakera-gandalf

SemanticBrainCorp / SemanticShield

lakeraai / chainguard

shaialon / ai-security-demos

peluche / deck-of-many-prompts

qroa / QROA

levitation-opensource / Manipulative-Expression-Recognition