NVIDIA / garak

the LLM vulnerability scanner

https://discord.gg/uVch4puUCs

Repository from Github https://github.comNVIDIA/garak

NVIDIA/garak Issues

reporting: `aggregate_reports` should rebuild report digest jsonl
Closed a month ago
Whetstone integration
Updated 2 months ago1
validate test expectations are provided by fixtures
Closed 2 months ago3
bug: loading llava models fails
Updated 2 months ago5
reporting: unify command line interface for reporting analysis modules
Closed 2 months ago
probe: anamorpher image
Updated 2 months ago2
support helper command list filtering
Updated 2 months ago3
probe: echoleak / CurXecute
Closed 2 months ago1
When an invalid OpenAI API key is used with parallel_attempts > 1, the system throws Exception in thread Thread-4 (_handle_results) instead of failing gracefully.
Updated 2 months ago1
Race condition occurs when using parallel attempts while reading log file
Updated 2 months ago
Mistral AI Issue — When an incorrect API key is provided, the system should fail more gracefully.
Updated 2 months ago1
ggml generator crashes with AttributeError: 'GarakSubConfig' object has no attribute 'verbose'
Updated 2 months ago1
I want to know if this supports AMD devices like the MI210 just curious
Updated 2 months ago1
documentation: what is responsible for what
Updated 2 months ago2
Add probe tier to `--list_probes`
Updated 2 months ago1
Using openai's opensrc model
Closed 2 months ago3
reporting: add probe tags to digest
Closed 2 months ago1
Possible bug with detector `judge.Refusal` when set up with `nim` generator.
Closed 2 months ago
arch: remove support for accessing/instantiating prompt/`Message` as `str`
Updated 2 months ago
suffix.GCG fails due to fschat ABI changes
Closed 2 months ago5
rename `max_tokens` throughout to `max_generation_tokens`
Updated 3 months ago5
Build in ASCII smuggling
Closed 3 months ago1
Continue Cohere support updates
Updated 3 months ago1
Can i use modelscope to replace huggingface to run this code which is python -m garak --model_type huggingface --model_name gpt2 --probes lmrc.Profanity
Closed 3 months ago2
Scanning o3-mini in Azure Gov
Closed 3 months ago19
Bug Report: Rest Generator Grok in Azure - Persistent Expecting value: line 1 column 1 (char 0) Error with -G Flag
Closed 3 months ago2
dependency ecoji stale
Closed 3 months ago1
feature: stable, useful config capture
Updated 4 months ago3
Prompts into a file instead to a model
Closed 4 months ago2
reporting: aggregation should add a digest object
Closed 4 months ago1
HuggingFace InferenceAPI calls seem to no longer work
Updated 4 months ago1
bug: probes.phrasing.FutureTense poses past tense prompts
Updated 4 months ago
bug: probes.suffix.GCG - `nf.infty` deprecated
Closed 4 months ago1
feature: make llmaaj prompts switchable
Updated 4 months ago1
reporting: add option to suppress midway logging of incomplete attempts in `report.jsonl`
Updated 4 months ago1
Probes: rest probes error on topic
Updated 4 months ago1
bug: core cfg listed twice in report in default run
Updated 4 months ago
fixer: include ability to update config param names
Updated 4 months ago
arch: give exit codes on improper exit
Updated 4 months ago1
generator remove: octo
Closed 5 months ago
config: deprecation warnings
Updated 5 months ago
Inference Parameter Manipulation
Updated 5 months ago1
Calibration in v0.11.0 seems to have missed some name changes introduced in PR #1178 resulting in missing Z-scores
Closed 5 months ago3
probes: refactor leakreplay to use mixing structure
Updated 6 months ago
fixer: support moving plugins between modules
Updated 6 months ago
reporting: refactor calibration & reporting
Updated 6 months ago
reporting: log calibration in report.jsonl
Updated 6 months ago
generators: upgrade to cohere v2 api
Updated 6 months ago
migrate langdetect to langid.py; avoid needless loading
Updated 6 months ago
replace `fschat` with `Turn` & `Conversation`
Updated 6 months ago