My experiments with AI

Why Hallucinations Still Matter

Hallucinations are the paradox of generative AI. On one hand, they undermine trust when LLMs confidently invent facts. On the other, they power creativity — the very thing that makes generative AI useful for brainstorming, synthesis, and novel problem-solving.

Byron Cook (VP & Distinguished Scientist, AWS) put it well in Computerworld’s Today in Tech:

“In the press, you see hallucination as a bad thing, but hallucination is the creativity that we seek when using transformer-based models.”

The real challenge isn’t eliminating hallucinations, but governing them. That’s where guardrails — safety controls that define boundaries — come in.

The Four Types of Guardrails

When I think about AI safety, I picture it like a multilayered net. Each strand catches a different kind of risk — some keep the AI’s words appropriate, others keep its actions safe, others protect the data, and still others make sure we can see what’s going on. Together, these guardrails create defense-in-depth.

1. Behavioral Guardrails

The first layer is all about what the AI says and how it behaves. This is where we stop toxicity, bias, or just plain nonsense from slipping through. Imagine a banking chatbot — it should never tell you how to self-medicate, or slip into casual, off-brand commentary.

What they do: Control what the AI says, how it says it, and whether it’s aligned with ethics, policy, and truth.

Examples:

• Content safety filters (block hate speech, violence, sexual content).
• Policy alignment (e.g., no unqualified investment advice).
• Hallucination checks (ground answers in references, provide citations).
• Prompt adherence (refuse out-of-scope or malicious requests).

How to implement in AWS:

AWS handles this through Bedrock Guardrails content filters, denied topics, and word filters. These are the “bouncers at the door,” blocking unsafe or policy-violating outputs before they ever reach the user. Beyond that, contextual grounding checks and Automated Reasoning make sure the model’s answers are tied to real data, not hallucinations.

The goal here isn’t to muzzle creativity, but to keep it aligned with truth, ethics, and company policy.

📌 Implementation tip: Attach a guardrail to your Bedrock agent or Knowledge Base via the guardrailConfig. For third-party models, wrap them with the ApplyGuardrail API.

2. Workflow Guardrails

The next layer looks not at what the AI says, but how it acts. LLM agents don’t just chat — they plan, call APIs, and execute tasks. Without constraints, they could easily run wild.

Workflow guardrails are the traffic signals of an AI system. They decide: which tools can the agent use? What’s the order of operations? When must a human step in?

What they do: Regulate how agents plan, call tools, and execute multi-step tasks.

Examples:

• Restrict what tools/actions are allowed.
• Limit high-risk operations (e.g., >$10K transfer requires human approval).
• Prevent infinite reasoning loops.
• Require recovery paths when errors occur.

How to implement in AWS:

AWS provides a modern, agent-native approach through Human-in-the-loop confirmation in Bedrock Agents, which uses the Return of Control (ROC) feature to pause mid-execution, surface a proposed action to a human, and only proceed with explicit approval. This makes high-stakes decisions conversational, while still ensuring accountability.

But orchestration guardrails don’t stop at a single agent. In multi-agent systems, the Strands SDK adds an additional layer of protection. Strands integrates Bedrock Guardrails directly into its runtime, automatically intercepting unsafe inputs or outputs across agents and tools. It also supports “shadow mode” guardrails via hooks, letting developers monitor when interventions would occur before enforcing them — ideal for tuning policies without disrupting live workflows.

Together, these approaches ensure workflow safety at multiple levels:

• Bedrock Agents handle sensitive moments by returning control to the user.
• Strands SDK enforces cross-agent consistency, applying guardrails throughout an orchestrated workflow.

This combination transforms workflow guardrails from a single checkpoint into a mesh of protective logic that scales with the complexity of your system.

📌 Implementation tip: Use Return of Control (ROC) in Bedrock Agents to pause for human confirmation on sensitive actions, and layer in the Strands SDK to enforce guardrails across multi-agent workflows. Together they ensure high-risk steps are reviewed, and unsafe inputs/outputs are intercepted consistently throughout the orchestration.

3. Security & Data Guardrails

Then comes security — arguably the most sensitive layer. This is where you prevent the AI from exposing secrets, breaking compliance, or leaking customer data.

Think of it like building a vault around your most valuable information. Even if the AI wanted to misbehave, it physically cannot overstep.

What they do: Protect sensitive information, enforce access rules, and maintain compliance.

Examples:

• RBAC: limit agents’ access to only what’s necessary.
• Masking/redaction of PII.
• Preventing data leaks (e.g., no raw SSNs in outputs).
• Compliance with GDPR/HIPAA/PCI.

How to implement in AWS:

At the core is the principle of least privilege: agents should only ever have access to the data and tools they need. In AWS, this means attaching scoped IAM roles to your agents, using AWS Secrets Manager to prevent credentials from ever entering prompts, and applying guardrails at the Knowledge Base nodes so agents can only retrieve from approved sources.

But security doesn’t stop at access control. Once sensitive data like PII is in play, how it’s handled matters just as much. Amazon Bedrock Guardrails already provide sensitive information filters that can mask PII, redacting account numbers or emails before they reach the model. The latest advancement goes further: integrating tokenization with Bedrock Guardrails (Sept 2025). Unlike masking, which removes sensitive values entirely, tokenization replaces PII with reversible, format-preserving tokens. The LLM processes only the tokens, never the raw data. Later, when needed, a trusted service (such as Thales CipherTrust) detokenizes the values for downstream workflows.

This layer is the backbone of responsible enterprise AI. It ensures safety isn’t just a policy, but baked into the very permissions of the system.

📌 Implementation tip: Use Bedrock Guardrails with tokenization to ensure LLMs only ever handle tokenized data. Pair this with IAM-scoped roles and Guardrails at Knowledge Base nodes to keep unauthorized data completely out of reach. That way, you preserve privacy, comply with regulations, and still deliver functional, accurate responses.

4. Observability Guardrails

Finally, we come to observability — the rearview mirror of AI safety. Even the best-guarded system needs oversight. Without logs, traces, and monitoring, you’re essentially flying blind.

Observability guardrails give us the ability to see and learn. Every input, output, tool call, and guardrail trigger can be logged. This isn’t just about debugging. In regulated industries, it’s also about compliance — being able to show exactly what the AI did and why. In practice, observability turns an opaque black box into a glass box we can trust.

What they do: Provide visibility into how the AI operates — crucial for debugging, trust, and audits.

Examples:

• Log every input, output, and tool call.
• Monitor for anomalies (e.g., spikes in refusals or failed tool calls).
• Run automated evaluations (toxicity, groundedness, truthfulness).
• Red-team your own system regularly.

How to implement in AWS:

AWS prescribes a behavior-first approach to monitoring serverless and agentic AI systems. Unlike traditional apps, you don’t watch hosts or servers — you watch agent behavior, outputs, cost trends, and workflow health. This is where observability guardrails provide continuous accountability.

With AgentCore Observability:

• Enable OpenTelemetry (ADOT) to trace every step of a session (reasoning, memory lookups, tool invocations).
• Collect metrics and logs (latency, error rate, token usage) and stream them into CloudWatch dashboards.
• Use transaction search to debug full conversation flows and tie model outputs to downstream actions.
• Set alarms for anomalies — e.g., spikes in token consumption, degraded response times, or unexpected tool retries.

📌 Implementation tip: Treat each agent session as a transaction. Instrument it with AgentCore’s telemetry so you can trace prompt → reasoning → tool → output. Then configure CloudWatch alarms and dashboards to catch issues (hallucination spikes, runaway cost, degraded latency) before they impact users.

Observability guardrails make AI systems auditable and explainable: you may not stop every issue up front, but you’ll always know what happened, where, and why.

Putting It All Together

Think of guardrails as layers of defense:

• Behavioral filters stop toxic or fabricated answers.
• Workflow constraints limit what an agent can do.
• Security rules protect what it can access.
• Observability ensures you see everything it does.

📌 Implementation sample – Return of Control (ROC) + Strands enforcement, end-to-end

1. Gate high-risk actions with ROC (Bedrock Agents).
   Mark sensitive steps (e.g., funds transfer, prod change, PII export) as “confirmable.” Use Return of Control (ROC) so the agent pauses, surfaces the proposed action + rationale to the user, and only proceeds on explicit approval.
2. Enforce cross-agent guardrails in Strands.
   • Register a global guardrail provider (your Bedrock Guardrails ARN).
   • Add pre-tool and post-tool hooks to run ApplyGuardrail on inputs/outputs for every tool call—this catches unsafe prompts, tool arguments, and responses across the whole workflow.
   • Start in shadow mode (log would-block events without enforcing), then flip to enforce after tuning.
3. Bound autonomy at the orchestrator.
   • Set allowedTools per agent role and scope per task.
   • Add iteration guards (e.g., max_steps, per-tool timeouts, and circuit breakers on repeated failures) in the Strands run config.
   • Route any policy violation to the ROC path (human confirmation) instead of executing.
4. Wire observability for audits.
   • Propagate a correlation/trace ID from Strands into AgentCore observability so every ROC decision, guardrail trigger, and tool call is traceable in CloudWatch.
   • Alert on guardrail violations, ROC frequency spikes, max_steps hit, and tool-retry storms.

Sketch (pseudocode)

orchestrator:
  guardrails:
    provider: bedrock_guardrails_arn: "arn:aws:bedrock:...:guardrail/xyz"
    mode: shadow   # → switch to 'enforce' after tuning
    hooks:
      beforeToolCall: ApplyGuardrail(INPUT)
      afterToolCall:  ApplyGuardrail(OUTPUT)
  policy:
    allowedTools:
      payments-agent: [quoteService.read, paymentService.create?roc]
    limits:
      max_steps: 8
      tool_timeout_ms: 8000
      fail_fast_retries: 2
    onViolation: route_to_ROC   # human confirm path
  telemetry:
    propagate_trace_id: true
    emit_metrics: [latency, token_usage, violations, roc_approvals]

This gives you: conversational human-in-the-loop at the exact moment it matters (ROC), uniform guardrail enforcement across every tool hop (Strands), bounded autonomy to prevent runaway loops, and audit-grade traces for governance—all without repeating earlier guidance.

My Closing Thought

Hallucinations won’t vanish — they are baked into the probabilistic nature of LLMs. But enterprises can move from fear to trust by weaving together formal verification (Automated Reasoning) and layered guardrails across behavior, workflow, security, and observability.

AWS is building this foundation with Bedrock Guardrails and AgentCore runtime, giving developers consistent enforcement points. Yet real-world systems rarely stop at a single agent. This is where frameworks like the Strands SDK come in — orchestrating multi-agent workflows, embedding guardrails at the coordination layer, and ensuring that agents collaborate safely without bypassing policies.

The future of AI governance isn’t about silencing hallucinations or locking systems down. It’s about structured freedom: letting agents remain creative, but always within guardrails that catch risks before they fall through the net. With Bedrock, AgentCore, and Strands working together, we now have the building blocks to design agentic systems that are not just powerful, but also provably safe, auditable, and enterprise-ready.