prompt-injection
There are 15 repositories under prompt-injection topic.
CyberAlbSecOP / Awesome_GPT_Super_Prompting
ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.
- Language:Python 1251
- Language:TypeScript 1131
whylabs / langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
Language:Jupyter Notebook 852abilzerian / LLM-Prompt-Library
Advanced Code and Text Manipulation Prompts for Various LLMs. Suitable for Siri, GPT-4o, Claude, Llama3, Gemini, and other high-performance open-source LLMs.
- promptmapLanguage:Python 648
- brilliant-CV
mintyfrankie / brilliant-CV
💼 another CV template for your job application, yet powered by Typst and more
Language:Typst 465 tldrsec / prompt-injection-defenses
Every practical and proposed defense against prompt injection.
deadbits / vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
Language:Python 315zacfrulloni / Prompt-Engineering-Holy-Grail
# Prompt Engineering Hub ⭐️ If you find this helpful, give it a star to show your support! This repository is a one-stop resource for prompt engineering. Also available on: https://promptengineeringhub.dev/
aiartificial-intelligencegenerative-aipromptprompt-engineeringprompt-generatorprompt-injectionprompt-learningprompt-toolkitprompt-tuningpromptsprompts-stable-diffusionai-prompt-engineerai-prompt-engineeringprompt-engineer-courseprompt-engineering-certificationprompt-engineering-courseprompt-engineering-coursesprompt-engineering-githubprompt-engineering-jobsLanguage:HTML 263automorphic-ai / aegis
Self-hardening firewall for large language models
Language:Python 258langgptai / Awesome-Multimodal-Prompts
Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.
- Language:Python 217
liu00222 / Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
Language:Python 146yunwei37 / prompt-hacker-collections
prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记
lakeraai / pint-benchmark
A benchmark for prompt injection detection systems.
Language:Jupyter Notebook 87pasquini-dario / project_mantis
Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks
Language:Python 49HumanCompatibleAI / tensor-trust
A prompt injection game to collect data for robust ML research
Language:Python 44- PromptGuard
GPTSafe / PromptGuard
Build production ready apps for GPT using Node.js & TypeScript
Language:TypeScript 39 gdalmau / lakera-gandalf-solutions
My inputs for the LLM Gandalf made by Lakera
sinanw / llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
Language:Jupyter Notebook 34AnthenaMatrix / Website-Prompt-Injection
Website Prompt Injection is a concept that allows for the injection of prompts into an AI system via a website's. This technique exploits the interaction between users, websites, and AI systems to execute specific prompts that influence AI behavior.
Language:HTML 31LostOxygen / llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
Language:Python 29The Prompt Injection Testing Tool is a Python script designed to assess the security of your AI system's prompt handling against a predefined list of user prompts commonly used for injection attacks. This tool utilizes the OpenAI GPT-3.5 model to generate responses to system-user prompt pairs and outputs the results to a CSV file for analysis.
Language:Python 26- gandalf_vs_gandalf
microsoft / gandalf_vs_gandalf
Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.
Language:Jupyter Notebook 25 MaxMLang / pytector
A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.
Language:Python 23AI/LLM Prompt Injection List is a curated collection of prompts designed for testing AI or Large Language Models (LLMs) for prompt injection vulnerabilities. This list aims to provide a comprehensive set of prompts that can be used to evaluate the behavior of AI or LLM systems when exposed to different types of inputs.
Image Prompt Injection is a Python script that demonstrates how to embed a secret prompt within an image using steganography techniques. This hidden prompt can be later extracted by an AI system for analysis, enabling covert communication with AI models through images.
Language:Python 22TrustAI-laboratory / Learn-Prompt-Hacking
This is The most comprehensive prompt hacking course available, which record our progress on a prompt engineering and prompt hacking course.
Language:Jupyter Notebook 22SemanticBrainCorp / SemanticShield
The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).
Language:Python 19wearetyomsmnv / Awesome-LLMSecOps
LLM | Security | Operations in one github repo with good links and pictures.
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
Language:Python 18ZapDos7 / lakera-gandalf
My solutions for Lakera's Gandalf
grepstrength / WideOpenAI
Short list of indirect prompt injection attacks for OpenAI-based models.
ASCII Art Prompt Injection is a novel approach to hacking AI assistants using ASCII art. This project leverages the distracting nature of ASCII art to bypass security measures and inject prompts into large language models, such as GPT-4, leading them to provide unintended or harmful responses.
- Language:Python 14
