NVIDIA NeMo Guardrails

NVIDIA NeMo Guardrails is an open-source toolkit for adding programmable guardrails to LLM-based conversational applications, with 5.6k GitHub stars and 596 forks.

It provides a unique approach to AI safety by offering input rails, output rails, and dialog rails that control how conversations flow between users and language models.

GitHub: NVIDIA/NeMo-Guardrails | Latest Release: v0.20.0 (January 2026)

NeMo Guardrails is the only toolkit that combines safety controls with dialog management in a single framework.

What is NeMo Guardrails?

NeMo Guardrails addresses the challenge of making LLM-powered applications safe and controllable in production.

While other guardrail solutions focus on filtering inputs and outputs, NeMo Guardrails introduces dialog rails that model the intended flow of conversations, preventing the LLM from deviating into unwanted territory.

The toolkit uses Colang, a domain-specific language designed specifically for defining conversational flows and safety policies.

With Colang, developers declaratively specify what conversations should look like, what topics are allowed, and how the system should respond to various user intents.

Built by NVIDIA, NeMo Guardrails integrates seamlessly with NVIDIA AI Enterprise and leverages NVIDIA’s safety models for maximum protection.

The toolkit also supports third-party integrations with Trend Micro AI Guard, Cisco AI Defense, and other security providers.

Key Features

Input Rails

Input rails process user messages before they reach your LLM:

• Jailbreak Detection: Identifies attempts to bypass safety guidelines
• Prompt Injection: Detects malicious prompt manipulation
• Content Moderation: Filters inappropriate or harmful content
• Intent Classification: Routes messages based on detected user intent

Output Rails

Output rails validate and filter LLM responses:

• Fact-Checking: Verifies responses against knowledge bases
• Hallucination Detection: Identifies fabricated information
• Sensitive Data: Blocks responses containing PII or secrets
• Response Validation: Ensures outputs meet quality standards

Dialog Rails

Dialog rails are unique to NeMo Guardrails and control conversation flow:

• Topic Boundaries: Keep conversations within defined topics
• Conversation Flows: Model expected dialog patterns
• Canonical Forms: Standardize intents across variations
• Branching Logic: Handle complex multi-turn conversations

Colang Language

Define guardrails using intuitive Colang syntax:

define user ask about competitors
  "What do you think about [competitor name]?"
  "How do you compare to other products?"
  "Is [competitor] better than you?"

define flow handle competitor questions
  user ask about competitors
  bot inform topic not discussed
  "I focus on helping you with our products. How can I assist you today?"

Installation

Install NeMo Guardrails via pip:

pip install nemoguardrails

For development with all optional dependencies:

pip install nemoguardrails[all]

How to Use NeMo Guardrails

Basic Configuration

Create a guardrails configuration in your project:

# config.yml
models:
  - type: main
    engine: openai
    model: gpt-4

rails:
  input:
    flows:
      - self check input
  output:
    flows:
      - self check output

Python Integration

from nemoguardrails import RailsConfig, LLMRails

# Load configuration
config = RailsConfig.from_path("./config")

# Create rails instance
rails = LLMRails(config)

# Generate response with guardrails
response = await rails.generate(
    messages=[{"role": "user", "content": "Tell me about your product"}]
)
print(response["content"])

Custom Rails Definition

Create Colang files to define custom behaviors:

# rails/topics.co

define user ask off topic
  "Can you help me with something unrelated?"
  "Let's talk about something else"

define bot inform on topic only
  "I'm here to help with specific topics.
   What would you like to know about our services?"

define flow handle off topic
  user ask off topic
  bot inform on topic only

Fact-Checking Integration

from nemoguardrails import RailsConfig, LLMRails
from nemoguardrails.kb import KnowledgeBase

# Load knowledge base for fact-checking
kb = KnowledgeBase.from_documents(["./docs/product_info.md"])

config = RailsConfig.from_path("./config")
config.knowledge_base = kb

rails = LLMRails(config)

Integration

FastAPI Application

from fastapi import FastAPI
from nemoguardrails import RailsConfig, LLMRails

app = FastAPI()
rails = LLMRails(RailsConfig.from_path("./config"))

@app.post("/chat")
async def chat(message: str):
    response = await rails.generate(
        messages=[{"role": "user", "content": message}]
    )
    return {"response": response["content"]}

LangChain Integration

from langchain.llms import OpenAI
from nemoguardrails.integrations.langchain import RunnableRails

config = RailsConfig.from_path("./config")
guardrails = RunnableRails(config)

# Chain with guardrails
chain = guardrails | OpenAI() | guardrails
response = chain.invoke("User query here")

Docker Deployment

FROM python:3.11-slim

WORKDIR /app
COPY config/ ./config/
RUN pip install nemoguardrails

EXPOSE 8000
CMD ["nemoguardrails", "server", "--config", "./config"]

Integration with Security Providers

NeMo Guardrails integrates with external security services:

• NVIDIA AI Safety Models: Built-in support for NVIDIA’s moderation models
• Trend Micro AI Guard: Content filtering and threat detection
• Cisco AI Defense: Enterprise security integration
• Custom Providers: Extensible architecture for custom integrations

When to Use NeMo Guardrails

NeMo Guardrails is the right choice when you need fine-grained control over conversational AI behavior beyond simple input/output filtering.

It excels in applications where dialog flow matters, such as customer service bots, enterprise assistants, or any application where conversations should follow defined patterns.

Consider NeMo Guardrails if you need to keep conversations on topic, implement complex multi-turn dialog policies, or integrate fact-checking against knowledge bases.

The Colang language provides a declarative way to specify behavior that would require complex code in other frameworks.

For organizations already using NVIDIA AI Enterprise or seeking tight integration with NVIDIA’s ecosystem, NeMo Guardrails offers native compatibility.

Its unique dialog modeling capabilities make it particularly valuable for applications where the conversation flow is as important as individual message safety.