Garak

Garak is an open-source LLM vulnerability scanner developed by NVIDIA with 6.9k GitHub stars and 775 forks.

Often called the “Nmap for LLMs,” it systematically probes language models for security weaknesses and safety issues.

GitHub: NVIDIA/garak | Latest Release: v0.14.0 (February 2026)

What is Garak?

Garak (named after the Star Trek character) is a framework for testing LLM security.

It automates the process of discovering vulnerabilities in language models including prompt injection susceptibility, data leakage risks, and content safety issues.

Developed by NVIDIA’s AI Red Team, Garak brings security testing rigor to the emerging field of LLM security.

Key Features

Vulnerability Scanning

Garak tests for multiple vulnerability categories:

• Prompt Injection - Override system instructions
• Data Leakage - Extract training data
• Jailbreaks - Bypass safety filters
• Hallucination - Generate false information
• Toxicity - Produce harmful content

Plugin Architecture

Extensible system of probes and detectors:

• Probes - Generate test inputs
• Detectors - Analyze model responses
• Generators - Interface with different LLMs
• Buffs - Modify probe behaviors

Multi-Model Support

Test various LLM providers:

• OpenAI (GPT-4, GPT-3.5)
• Hugging Face models
• Replicate models
• Custom endpoints

Installation

# Install via pip
pip install garak

# Or from source
git clone https://github.com/NVIDIA/garak.git
cd garak
pip install -e .

Usage

Basic Scan

# Scan OpenAI model
garak --model_type openai --model_name gpt-3.5-turbo

# Scan Hugging Face model
garak --model_type huggingface --model_name meta-llama/Llama-2-7b-chat-hf

# Scan with specific probes
garak --model_type openai --model_name gpt-4 --probes promptinject

Configuration

Create a config file for repeated scans:

# garak_config.yaml
run:
  generations: 10

plugins:
  probes:
    - promptinject
    - encoding
    - dan
  detectors:
    - always.Fail
    - toxicity.ToxicCommentModel

Probe Categories

Output and Reporting

Garak generates detailed reports:

# Generate HTML report
garak --model_type openai --model_name gpt-4 --report_prefix my_scan

# Output includes:
# - my_scan.report.html  (Human-readable report)
# - my_scan.report.jsonl (Machine-readable results)

Report Contents

• Vulnerability findings by category
• Pass/fail rates for each probe
• Specific prompts that succeeded
• Severity assessments

CI/CD Integration

Integrate Garak into security pipelines:

# GitHub Actions
- name: LLM Security Scan
  run: |
    pip install garak
    garak --model_type openai --model_name ${{ secrets.MODEL_NAME }} \
          --probes promptinject,dan \
          --report_prefix scan_results

- name: Upload Results
  uses: actions/upload-artifact@v3
  with:
    name: garak-report
    path: scan_results.report.html

Custom Probes

Create custom probes for specific testing:

# custom_probe.py
from garak.probes.base import Probe

class MyCustomProbe(Probe):
    name = "mycustom"
    description = "Custom security probe"

    prompts = [
        "Test prompt 1",
        "Test prompt 2",
    ]

Comparison with Other Tools

When to Use Garak

Garak is ideal for:

• Security teams evaluating LLM deployments
• Red team exercises for AI systems
• Pre-deployment security assessments
• Continuous LLM security monitoring