Skip to content

Comments

docs: Add hub guardrails#157

Open
nina-kollman wants to merge 7 commits intomainfrom
nk/hub_guardrails
Open

docs: Add hub guardrails#157
nina-kollman wants to merge 7 commits intomainfrom
nk/hub_guardrails

Conversation

@nina-kollman
Copy link
Contributor

@nina-kollman nina-kollman commented Feb 16, 2026

Important

Adds documentation for Traceloop Hub guardrails, including an overview, configuration guide, and evaluators reference, with updates to navigation in mint.json.

  • Documentation:
    • Adds hub/overview.mdx for real-time safety and quality checks overview.
    • Adds hub/guardrails-configuration.mdx for configuring guardrails in Traceloop Hub.
    • Adds hub/evaluators.mdx for a reference of 12 built-in evaluators.
  • Evaluators:
    • Introduces 12 evaluators in three categories: Safety, Validation, and Quality.
    • Each evaluator can run in pre_call, post_call, or both modes.
  • Configuration:
    • Describes YAML configuration structure for providers, guards, and pipelines.
    • Explains guard fields, modes, failure handling, and runtime control.
  • Navigation:
    • Updates mint.json to include new documentation pages under "Guardrails" group.

This description was created by Ellipsis for c044440. You can customize this summary. It will automatically update as commits are pushed.

Summary by CodeRabbit

  • New Features

    • Added Guardrails functionality with pre-call and post-call execution modes for request validation.
  • Documentation

    • Added new Guardrails documentation section covering overview, configuration, and evaluators.
    • Documented 12 built-in evaluators across Safety, Validation, and Quality categories.
    • Included configuration guides for providers, guards, and pipelines with YAML examples.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to c044440 in 12 seconds. Click for details.
  • Reviewed 996 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

Workflow ID: wflow_cREXtvCoTYytmTGt

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 16, 2026

📝 Walkthrough

Walkthrough

This PR introduces comprehensive guardrails documentation to the Traceloop Hub by adding a new "Guardrails" navigation section with three pages (overview, configuration, evaluators), updating existing guardrails documentation with implementation modes and YAML examples, and establishing detailed evaluator specifications.

Changes

Cohort / File(s) Summary
Navigation Configuration
mint.json
Added new "Guardrails" navigation group with three pages (overview, configuration, evaluators) positioned after Quick Start.
Guardrails Documentation
hub/guardrails/overview.mdx, hub/guardrails/configuration.mdx, hub/guardrails/evaluators.mdx
New documentation files covering guardrails overview (execution flow, concepts, observability), configuration schema (Providers, Guards, Pipelines), and 12 built-in evaluators with categorization, parameters, and usage examples.
Existing Guardrails Update
evaluators/guardrails.mdx
Added two implementation modes (Database Mode and Config Mode Guardrails v1) with YAML configuration examples, guards, pipelines, and multi-layer protection patterns; content includes some duplicated sections.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~18 minutes

Poem

🐰 A warren of guardrails now stands tall,
With configs and evaluators for all,
Protection flows in pre and post modes,
Safety guards line every road,
Hub's documentation now guards well!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'docs: Add hub guardrails' directly and clearly summarizes the main change—adding documentation for Traceloop Hub guardrails across multiple new pages and navigation updates.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch nk/hub_guardrails

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@hub/guardrails-configuration.mdx`:
- Around line 75-109: Update all broken evaluator links that point to
/hub/guardrails-evaluators to the correct slug /hub/evaluators; specifically
replace the link target on the "Evaluators Reference" anchor used in the YAML
examples and any other occurrences (e.g., the inline reference after
evaluator_slug and other docs blocks) so that references like "Evaluators
Reference" resolve to /hub/evaluators across the document (also fix the same
links in the repeated section covering lines 206-315).

In `@hub/overview.mdx`:
- Around line 213-219: The "Evaluators Reference" Card within the CardGroup uses
the wrong href; update the Card with title "Evaluators Reference" (the Card
component instance in the snippet) to use href="/hub/evaluators" instead of
href="/hub/guardrails-evaluators" so the link matches the navigation slug;
verify the CardGroup/Card markup is unchanged and only the href value is
corrected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the overview of the whole hub???

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be part of the existing configuration page

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@evaluators/guardrails.mdx`:
- Around line 68-74: The term "Database Mode" is misleading and should be
renamed to "Dashboard Mode": update the list item text and the section heading
"Database Mode" to "Dashboard Mode" in the content referencing evaluators
configured in the Traceloop dashboard, and update any nearby references (e.g.,
the paragraph that starts "Evaluators configured in Traceloop dashboard, applied
via SDK decorators..." and the "### Database Mode" heading) so they consistently
use "Dashboard Mode".

In `@hub/guardrails/overview.mdx`:
- Around line 226-229: Update the Card component's href to the new navigation
slug: change the href prop value on the Card inside CardGroup (the Card with
title "Configuration Guide") from "/hub/guardrails-configuration" to
"/hub/guardrails/configuration" so the link points to the correct page.

---

Duplicate comments:
In `@hub/guardrails/overview.mdx`:
- Around line 230-231: Update the Card component with title "Evaluators
Reference" to point to the new guardrails slug by changing its href value from
"/hub/evaluators" to "/hub/guardrails/evaluators" (locate the Card element in
overview.mdx and update the href attribute).

Comment on lines +68 to +74
Guardrails can be implemented in two modes:

1. **Database Mode** - Evaluators configured in Traceloop dashboard, applied via SDK decorators in your application code (shown below)
2. **Config Mode** - Available in Traceloop Hub v1, guardrails and evaluators fully defined in YAML (see [Config Mode Guardrails](#config-mode-guardrails-v1))

### Database Mode

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Rename “Database Mode” to “Dashboard Mode” for clarity.

Line 70 and Line 73 describe dashboard-configured evaluators, so “Database Mode” reads like a typo.

🛠️ Proposed fix
-1. **Database Mode** - Evaluators configured in Traceloop dashboard, applied via SDK decorators in your application code (shown below)
+1. **Dashboard Mode** - Evaluators configured in Traceloop dashboard, applied via SDK decorators in your application code (shown below)

-### Database Mode
+### Dashboard Mode
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@evaluators/guardrails.mdx` around lines 68 - 74, The term "Database Mode" is
misleading and should be renamed to "Dashboard Mode": update the list item text
and the section heading "Database Mode" to "Dashboard Mode" in the content
referencing evaluators configured in the Traceloop dashboard, and update any
nearby references (e.g., the paragraph that starts "Evaluators configured in
Traceloop dashboard, applied via SDK decorators..." and the "### Database Mode"
heading) so they consistently use "Dashboard Mode".

Comment on lines +226 to +229
<CardGroup cols={2}>
<Card title="Configuration Guide" icon="gear" href="/hub/guardrails-configuration">
Learn how to configure guardrails with complete YAML reference
</Card>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix Configuration Guide link path.

Line 227 points to /hub/guardrails-configuration, but the new navigation slug is /hub/guardrails/configuration. This will 404.

🛠️ Proposed fix
-  <Card title="Configuration Guide" icon="gear" href="/hub/guardrails-configuration">
+  <Card title="Configuration Guide" icon="gear" href="/hub/guardrails/configuration">
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@hub/guardrails/overview.mdx` around lines 226 - 229, Update the Card
component's href to the new navigation slug: change the href prop value on the
Card inside CardGroup (the Card with title "Configuration Guide") from
"/hub/guardrails-configuration" to "/hub/guardrails/configuration" so the link
points to the correct page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants