A Cisco-branded AI Defense reference architecture

Built for Cisco SE customer demos. Mirrors a real customer's stack: NetBox + Nemotron-driven chatbot, with Cisco AI Defense applied at three points.

This demo is a deployable reference architecture for customers building AI-assisted operations tooling over Cisco / network infrastructure. Every piece is something the customer can stand up themselves the same day, using accounts and entitlements they already have. The implementation total is roughly 600 lines of Python plus a small SVG-driven UI.

Every turn fires AI Defense three times

Input · Tool-args · Output — three different inspection contexts, one API, one policy.

A traditional content-filter sits in front of the LLM and stops bad prompts. That covers about a third of the OWASP LLM Top-10. The other two-thirds — destructive tool calls, sensitive-data egress, excessive agency — are caught after the model has already decided to act. Three-point gating moves inspection to the boundary where each risk class actually lives.

The Inspection API

Cisco AI Defense Inspection — runtime policy enforcement for AI applications.

The 13 input rules

Every prompt or tool-call gets scanned by 13 classifier rules in parallel. Each rule returns NONE_VIOLATION or a specific classification. Multi-violation responses are common — e.g. a prompt-injection that also contains harmful content fires both Prompt Injection (SECURITY_VIOLATION) and General Harms (SAFETY_VIOLATION).

Request shape

# Inspection request — same payload at every gate
{
  "messages": [{ "role": "user", "content": "<text being inspected>" }],
  "model":    "aidefense-demo",  # label only, not a real model
  "config":   { "enabled_rules": [] },  # empty = use policy default
  "metadata": {}
}

Response shape (a block)

{
  "is_safe": false,
  "action":  "Block",
  "severity": "NONE_SEVERITY",
  "classifications": ["SECURITY_VIOLATION", "SAFETY_VIOLATION"],
  "rules": [
    { "rule_name": "Prompt Injection", "classification": "SECURITY_VIOLATION", "entity_types": [] },
    { "rule_name": "General Harms",    "classification": "SAFETY_VIOLATION",   "entity_types": [] }
  ],
  "attack_technique": "NONE_ATTACK_TECHNIQUE",
  "event_id": "<uuid>",
  "processed_rules": [ /* all 13 with status — mostly NONE_VIOLATION */ ]
}

The reasoning engine

NVIDIA NIM cloud — OpenAI-compatible inference for the Nemotron family.

Why this specific Nemotron variant

Llama-3.3-Nemotron-Super-49B-v1 is a Llama-3.3 base fine-tuned by NVIDIA for instruction following and tool use. Two reasons we picked it over the 70B-Instruct variant:

• Free-tier accessible — 70B-Instruct returns HTTP 404 from this account; Super-49B-v1 returns clean 200s.
• Faster — 1-5 s response time vs. v1.5's 20-25 s on cold paths.
• Tool calling clean — emits proper tool_calls with no chain-of-thought leakage when prepended with detailed thinking off as the first system message.

Tool-call request shape

{
  "model": "nvidia/llama-3.3-nemotron-super-49b-v1",
  "messages": [
    { "role": "system", "content": "detailed thinking off\n\nYou are NetOps Assistant…" },
    { "role": "user",   "content": "List all firewalls in our fleet" }
  ],
  "tools": [ /* netbox_search, netbox_list_devices(role|model_contains), ...7 tools total */ ],
  "tool_choice": "auto",
  "temperature": 0.2,
  "max_tokens": 1024
}

The compute layer

Oracle Cloud Infrastructure — Always-Free Ampere ARM running the whole stack.

Docker Compose stack — 9 containers

Free-tier budget compliance

This tenancy currently runs 3 VMs on Always-Free: salient-web-1 (AMD Micro, serves salient-concepts.com), cisco-web-1 (AMD Micro, serves the CiscoPulse splash), and cisco-web-1-arm (this demo). At cap for AMD micros (2/2), using 2 OCPU / 12 GB of the 4 OCPU / 24 GB Ampere A1 budget. 2 reserved IPs of 2 allowed.

The name resolution layer

Cloudflare Free plan — DNS-only mode (gray cloud).

The customer's source of truth

NetBox 4.4 — IPAM/DCIM seeded with a realistic Cisco enterprise fleet.

NetBox is the de-facto IPAM/DCIM among enterprise NetOps. Customers building AI assistants for network operations almost always have it. This seed mirrors a mid-size enterprise: two production data centers with full Nexus 9000 fabric, four branch sites, a corporate HQ, and a security/compute layer.

Cisco device types in the seed

Intentional PII / credential fodder

For the output-gate demo to fire, NetBox needs data Cisco AI Defense will flag. The seed embeds two classes of sensitive content:

• Contacts with real-looking PII — names, emails (name@example-corp.com), phones in E.164 format. 13 contacts attached to sites as NOC / site-lead / on-call assignments.
• SNMP community strings in device comments — every device has a unique pseudo-credential like atl-c0re-r0!, nyc-fw-corp!. When the LLM tries to dump them, the output gate catches the dense credential pattern.

netbox-mcp — 7 tools

The only custom code in the stack

FastAPI + SSE chat loop · two MCP wrappers · ~600 lines of Python total.

SSE event types emitted per turn

turn_start            // session id + model
gate_start            // {where, content[:200]}
gate_result           // {action, severity, attack_technique, violations[], latency_ms}
llm_call_start        // {hop, model}
tool_call_proposed    // {hop, idx, name, arguments}
tool_executing        // {hop, idx, name}
tool_result           // {hop, idx, name, result}
assistant_message     // {content}
blocked               // {where, severity, attack_technique, violations[]}
model_declined        // {finish_reason, explanation} — defense-in-depth signal
turn_end              // {reason: ok | input_blocked | output_blocked | model_declined | error}
error

Production hygiene from day one

OpenBao for secrets · Let's Encrypt for TLS · no credentials on disk.

Secrets in OpenBao

Secrets live ONLY in OpenBao + the OCI VM's compose/.env file (chmod 600). They never appear in git, in container images, in CLAUDE memory files, or in this About page's source.

Cost breakdown

Every component is free-tier or self-hosted. Total monthly cost: $0.