Skip to content

Issues: NVIDIA/garak

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

313 Open 303 Closed
313 Open 303 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Estimate # prompts, # input tokens per probe architecture Architectural upgrades
#1071 opened Jan 11, 2025 by leondz 25.02 Efficiency
Enable stopping when call (/token) budget hit architecture Architectural upgrades
#1070 opened Jan 11, 2025 by leondz 25.02 Efficiency
Automate updating the calibration process housekeeping Repository and project maintenance
#1069 opened Jan 10, 2025 by leondz 25.02 Efficiency
Harness that orders probes for expected attack success rate (ASR) according to bag architecture Architectural upgrades
#1068 opened Jan 10, 2025 by leondz 25.02 Efficiency
2
Add “real” multi-turn support to chat-enabled generators architecture Architectural upgrades generators Interfaces with LLMs
#1067 opened Jan 10, 2025 by leondz 25.02 Efficiency
probe: Adapt sandwich attack to auto-find effective languages architecture Architectural upgrades probes Content & activity of LLM probes
#1066 opened Jan 10, 2025 by leondz
2 tasks
25.02 Efficiency
1
Bayesian optimisation base class architecture Architectural upgrades probes Content & activity of LLM probes
#1065 opened Jan 10, 2025 by leondz 25.02 Efficiency
Pattern for probes that can react to generator/detector responses architecture Architectural upgrades probes Content & activity of LLM probes
#1064 opened Jan 10, 2025 by leondz 25.02 Efficiency
docs: model type and name mixed up documentation Improvements or additions to documentation
#1063 opened Jan 10, 2025 by dennisrall
support for python 3.13 architecture Architectural upgrades
#1062 opened Jan 8, 2025 by leondz
7 tasks
Include information about why detectors are skipped detectors work on code that inherits from or manages Detector quality-accuracy This affects result quality/reliability reporting Reporting, analysis, and other per-run result functions ux Interface & interaction improvements
#1061 opened Jan 7, 2025 by erickgalinkin 25.02 Efficiency
@erickgalinkin
probe: text augmentations
#1056 opened Jan 3, 2025 by rafaelsandroni
Detector: Shields, to determine if an LLM moderator is functioning
#1055 opened Jan 2, 2025 by Eric-Hacker
StringDetector case sensitivity seems inverted. bug Something isn't working
#1054 opened Jan 2, 2025 by Eric-Hacker
@leondz
2
package garak for kali housekeeping Repository and project maintenance
#1036 opened Dec 5, 2024 by leondz
@jmartin-tech
3
error: 'latin-1' codec can't encode character '\u2026' bug Something isn't working
#1035 opened Dec 4, 2024 by wiktorw767
2
probes: fileformats probe fails when targeting a local modal path bug Something isn't working
#1034 opened Dec 3, 2024 by jmartin-tech
Fall back when online services can't be reached. bug Something isn't working
#1033 opened Dec 2, 2024 by erickgalinkin
1
align prompt count per probe architecture Architectural upgrades probes Content & activity of LLM probes
#1032 opened Dec 2, 2024 by leondz 25.02 Efficiency
reporting: enable logging & skipping of unrecognised probes, detectors in digest generation reporting Reporting, analysis, and other per-run result functions
#1024 opened Nov 25, 2024 by leondz
probe: Image Hijack new plugin Describes an entirely new probe, detector, generator or harness probes Content & activity of LLM probes
#1023 opened Nov 25, 2024 by leondz
probe: gritty pixy new plugin Describes an entirely new probe, detector, generator or harness probes Content & activity of LLM probes
#1020 opened Nov 22, 2024 by leondz
2
probe: more xss exfil probes probes Content & activity of LLM probes
#996 opened Nov 11, 2024 by leondz
4 tasks
24.12 Security focus
1
@leondz
atkgen: disinfo probe probes Content & activity of LLM probes
#994 opened Nov 11, 2024 by leondz
atkgen: illegal content probe probes Content & activity of LLM probes
#993 opened Nov 11, 2024 by leondz
ProTip! Adding no:label will show everything without a label.