prompt-injection
There are 9 repositories under prompt-injection topic.
- Language:TypeScript 954
- Language:Python 893
whylabs / langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
Language:Jupyter Notebook 733- promptmapLanguage:Python 530
deadbits / vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
Language:Python 251automorphic-ai / aegis
Self-hardening firewall for large language models
Language:Python 244tldrsec / prompt-injection-defenses
Every practical and proposed defense against prompt injection.
langgptai / Awesome-Multimodal-Prompts
Prompts of GPT-4V & DALL-E3 to full utilize the multi-modal ability. GPT4V Prompts, DALL-E3 Prompts.
- Language:Python 186
CyberAlbSecOP / Awesome_GPT_Super_Prompting
ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.
yunwei37 / prompt-hacker-collections
prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记
liu00222 / Open-Prompt-Injection
Prompt injection attacks and defenses in LLM-integrated applications
Language:Python 86lakeraai / pint-benchmark
A benchmark for prompt injection detection systems.
Language:Jupyter Notebook 58HumanCompatibleAI / tensor-trust
A prompt injection game to collect data for robust ML research
Language:Python 36gdalmau / lakera-gandalf-solutions
My inputs for the LLM Gandalf made by Lakera
AnthenaMatrix / Website-Prompt-Injection
Website Prompt Injection is a concept that allows for the injection of prompts into an AI system via a website's. This technique exploits the interaction between users, websites, and AI systems to execute specific prompts that influence AI behavior.
Language:HTML 32- PromptGuard
GPTSafe / PromptGuard
Build production ready apps for GPT using Node.js & TypeScript
Language:TypeScript 30 The Prompt Injection Testing Tool is a Python script designed to assess the security of your AI system's prompt handling against a predefined list of user prompts commonly used for injection attacks. This tool utilizes the OpenAI GPT-3.5 model to generate responses to system-user prompt pairs and outputs the results to a CSV file for analysis.
Language:Python 20LostOxygen / llm-confidentiality
Whispers in the Machine: Confidentiality in LLM-integrated Systems
Language:Python 20MaxMLang / pytector
A Python package designed to detect prompt injection in text inputs utilizing state-of-the-art machine learning models from Hugging Face. The main focus is on ease of use, enabling developers to integrate security features into their applications with minimal effort.
Language:Python 20- gandalf_vs_gandalf
microsoft / gandalf_vs_gandalf
Turning Gandalf against itself. Use LLMs to automate playing Lakera Gandalf challenge without needing to set up an account with a platform provider.
Language:Jupyter Notebook 20 Image Prompt Injection is a Python script that demonstrates how to embed a secret prompt within an image using steganography techniques. This hidden prompt can be later extracted by an AI system for analysis, enabling covert communication with AI models through images.
Language:Python 18ASCII Art Prompt Injection is a novel approach to hacking AI assistants using ASCII art. This project leverages the distracting nature of ASCII art to bypass security measures and inject prompts into large language models, such as GPT-4, leading them to provide unintended or harmful responses.
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
Language:Python 13SemanticBrainCorp / SemanticShield
The Security Toolkit for managing Generative AI(especially LLMs) and Supervised Learning processes(Learning and Inference).
Language:Python 13sinanw / llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
Language:Jupyter Notebook 11AI/LLM Prompt Injection List is a curated collection of prompts designed for testing AI or Large Language Models (LLMs) for prompt injection vulnerabilities. This list aims to provide a comprehensive set of prompts that can be used to evaluate the behavior of AI or LLM systems when exposed to different types of inputs.
Safetorun / PromptDefender
A prompt defence is a multi-layer defence that can be used to protect your applications against prompt injection attacks.
Language:Go 8levitation-opensource / Manipulative-Expression-Recognition
MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.
benchmarkingconversation-analysisconversation-analyticsfraud-detectionfraud-preventionllmllm-securityllm-testllm-trainingmanipulationprompt-engineeringprompt-injectionpsychometricsmisinformationexpression-recognitionsentiment-analysissentiment-classificationhuman-computer-interactionhuman-robot-interactiontransparencyLanguage:HTML 7- Language:Python 7
- Image-Generator
CursedPrograms / Image-Generator
This project leverages the SDXL-Turbo model for versatile image processing tasks. Offering a simple command-line interface, it facilitates both Text-to-image and Image-to-image operations. Users select an operation, input prompts, and the script dynamically generates and executes code snippets.
Language:Python 6 velocitatem / raccoon
Let Raccoon sample the unknown, safeguarding your AI's home.
Language:Python 6rabbidave / Denzel-Crocker-Hunting-For-Fairly-Odd-Prompts
A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts
Language:Python 5- canaryLanguage:Python 3
genia-dev / vibraniumdome-sdk
Client SDK to send LLM interactions to Vibranium Dome
Language:Python 2rohilrg / CatchPromptInjection
This repo focus on how to deal with prompt injection problem faced by LLMs
Language:Python 2
