Skip to content

SOP CS-01: AI Customer Service Triage (v2.1)

Status: Production Ready
Last Updated: 2026-01-19
Owner: Protocol Raw Operations
Platform: Make.com + Supabase Edge Functions + Claude Sonnet + Slack

Overview

Automated AI-powered customer service intake system that receives emails, applies code-enforced policy gates, classifies using Claude Sonnet, drafts responses in Protocol Raw's brand voice, and routes to human review via the Support Workstation.

Flow: Gmail → Customer Lookup → Find Open Ticket → Edge Function (creates ticket + triages) → Slack → Support Workstation

Key Achievement: - 80%+ of routine inquiries classified automatically - Policy gate catches health/safety/legal issues instantly (zero Claude tokens) - Single Edge Function handles all business logic - Make.com reduced to pure orchestration - Unified review workflow in Support Workstation

v2.1 Changes (from v2.0)

Component v2.0 v2.1
Review Interface Separate Agent Review tab Unified in Support Workstation
Draft Tracking None draft_usage field tracks how AI drafts were used
Policy Gate Display Hidden Visible in Support Workstation panel
Workflow Two queues (Support + Review) Single unified queue

Architecture

System Flow Diagram

┌─────────────────────────────┐
│  Gmail Trigger              │ ← Emails to hello@protocolraw.co.uk
│   (Module 1)                │   Filter: Unread only
└──────────┬──────────────────┘
           │ Filter: To contains hello@protocolraw.co.uk
           â–¼
┌─────────────────────────────┐
│ Customer Lookup             │ ← Supabase fn_get_customer_by_email
│   (Module 2)                │   Returns: customer_id, name, subscription, box_size
└──────────┬──────────────────┘
           │
           â–¼
┌─────────────────────────────┐
│ Find Open Ticket            │ ← Check for existing ticket (reply detection)
│   (Module 11)               │
└──────────┬──────────────────┘
           │
           â–¼
┌─────────────────────────────┐
│      Router                 │
│   (Module 12)               │
└──────────┬──────────────────┘
           │
    ┌──────┴──────┐
    â–¼             â–¼
┌───────────┐  ┌───────────┐
│ New       │  │ Existing  │
│ Ticket    │  │ Ticket    │
│ Path      │  │ Path      │
└─────┬─────┘  └─────┬─────┘
      │              │
      â–¼              â–¼
┌─────────────────────────────┐
│ Edge Function               │ ← cs-agent-triage
│ (Module 16)                 │   
│                             │   1. Check policy gates (code)
│                             │   2. Fetch AI prompt from KB
│                             │   3. Call Claude Sonnet
│                             │   4. Create/update ticket
│                             │   5. Store decision in cs_agent_decisions
└──────────┬──────────────────┘
           │
           â–¼
┌─────────────────────────────┐
│ Slack Notification          │ ← #cust-ops channel
│   (Module 4)                │   If escalate=true OR confidence<0.85
└──────────┬──────────────────┘
           │
           â–¼
┌─────────────────────────────┐
│ Support Workstation         │ ← Unified review interface
│   (Ops Portal)              │   Shows: message + context + AI draft + policy gate
└─────────────────────────────┘

Policy Gates

Code-enforced rules that trigger immediate escalation WITHOUT calling Claude. Zero tokens, ~150ms response.

Policy Gate Keywords

Gate Code Trigger Keywords Escalation Reason
health_vomiting vomit/vomiting/vomited/throwing up/puking + sick/ill/unwell Dog health concern - potential food reaction
health_diarrhea diarrhea/diarrhoea/loose stool/runny poo Dog health concern - potential food reaction
health_blood blood in stool/bloody stool/blood in poo Dog health concern - blood in stool
health_lethargy lethargic/lethargy/won't eat/not eating/refusing food Dog health concern - lethargy or appetite loss
legal_threat lawyer/solicitor/trading standards/legal action/sue Legal threat mentioned
media_threat newspaper/journalist/social media/going public Media/reputation threat
complaint_formal formal complaint/complaint/disgusted/outraged/furious Formal complaint or severe dissatisfaction
allergy_reaction allergic/allergy/reaction/swelling/hives Potential allergic reaction
foreign_object plastic/metal/glass/foreign object/found something Foreign object concern
social_media_threat post/share + facebook/twitter/instagram/review Social media threat
b2b_inquiry wholesale, retail partner, breeder program, bulk order B2B inquiry
repeat_contacter Customer contacted 3+ times in 7 days Repeat contacter
double_credit_request Has recent credit AND asking for more Double credit request

Policy Gate Outcomes

Outcome model_used confidence_score Claude Called?
Policy gate triggered policy_gate 0.00 No
AI classification claude-sonnet-4-20250514 0.00-1.00 Yes

Benefit: Health/safety escalations in ~150ms vs ~4000ms for Claude. Zero tokens spent.

Human Review Workflow

Unified Support Workstation (v2.1)

All tickets appear in the Support Workstation queue. Each ticket card displays:

  1. Customer Message - Full email content
  2. Customer Context - Subscription status, box size, order history, shipment status
  3. AI Triage Panel - Category, confidence score, recommended action
  4. Policy Gate Warning - If triggered, prominently displayed with gate code
  5. AI Draft Response - Pre-filled in editable textarea
  6. Action Buttons - Send Response, Send + Replacement, Close Without Sending

Operator Workflow

1. Open Support Workstation
2. Select ticket from queue (sorted by priority)
3. Review customer message + context
4. Review AI draft response
5. Either:
   a. Send as-is (AI was correct)
   b. Edit and send (AI needed adjustment)
   c. Replace entirely (AI was wrong)
   d. Close without sending (no response needed)
6. System tracks draft_usage for analytics

Priority Calculation

Condition Priority Sort Order
Policy gate triggered OR escalation_reason set 1 (High) First
Confidence < 0.70 2 (Medium) Second
Normal AI classification 3 (Normal) Third
Outbound/awaiting reply 4 (Awaiting) Last

Draft Usage Tracking (v2.1 NEW)

The system tracks how operators use AI-drafted responses:

draft_usage Meaning Indicates
sent_as_is Response sent without changes AI was correct
minor_edits >70% word overlap with original AI mostly correct
major_rewrite <70% word overlap AI needed significant changes
replaced Completely different response AI was wrong
no_draft No AI draft was provided Policy gate escalation

Calculation Logic

if (responseText === originalDraft) {
  draftUsage = 'sent_as_is';
} else {
  const similarity = matchingWords / originalWords.length;
  draftUsage = similarity > 0.7 ? 'minor_edits' : 'major_rewrite';
}

Analytics Queries

-- AI accuracy rate (last 30 days)
SELECT 
  draft_usage,
  COUNT(*) as count,
  ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER (), 1) as percentage
FROM raw_ops.support_tickets
WHERE resolved_at > NOW() - INTERVAL '30 days'
  AND draft_usage IS NOT NULL
GROUP BY draft_usage
ORDER BY count DESC;

-- Accuracy by category
SELECT 
  ai_category,
  COUNT(*) FILTER (WHERE draft_usage = 'sent_as_is') as sent_as_is,
  COUNT(*) FILTER (WHERE draft_usage IN ('minor_edits', 'major_rewrite', 'replaced')) as edited,
  ROUND(
    COUNT(*) FILTER (WHERE draft_usage = 'sent_as_is') * 100.0 / COUNT(*), 1
  ) as accuracy_pct
FROM raw_ops.support_tickets
WHERE draft_usage IS NOT NULL
GROUP BY ai_category
ORDER BY accuracy_pct DESC;

Database Schema

Table: support_tickets

Core ticket storage with AI fields:

Column Type Description
id UUID Primary key
customer_email TEXT Customer's email
subject TEXT Email subject
body TEXT Full email body
status TEXT open, in_progress, awaiting_reply, resolved, closed
ai_category TEXT AI classification
ai_confidence NUMERIC(3,2) 0.00-1.00
ai_draft_response TEXT AI-drafted response
ai_recommended_action TEXT Suggested action
ai_escalation_reason TEXT Why escalated (or null)
draft_usage TEXT How draft was used (v2.1)
... ... (other fields)

Table: cs_agent_decisions

Stores AI decisions separately for audit trail and analytics:

Column Type Description
id UUID Primary key
ticket_id UUID FK to support_tickets
category TEXT Classification result
confidence_score NUMERIC(3,2) 0.00-1.00
draft_response TEXT AI-drafted response
escalation_reason TEXT Why escalated
policy_gate_triggered BOOLEAN Whether policy gate fired
policy_gate_code TEXT Machine code for analytics
model_used TEXT 'claude-sonnet-4-20250514' or 'policy_gate'
execution_duration_ms INTEGER Processing time
dedupe_key TEXT Idempotency key
created_at TIMESTAMPTZ When processed

View: v_support_queue

Unified queue view for Support Workstation (updated v2.1):

SELECT 
  t.*,
  c.id AS customer_id,
  c.first_name,
  c.last_name,
  s.status AS subscription_status,
  s.box_size,
  s.frequency_weeks,
  s.seal_subscription_id,
  s.next_billing_date,
  d.policy_gate_triggered,
  d.policy_gate_code,
  d.id AS decision_id,
  CASE
    WHEN t.is_outbound THEN 4
    WHEN t.ai_escalation_reason IS NOT NULL THEN 1
    WHEN t.ai_confidence < 0.7 THEN 2
    ELSE 3
  END AS priority
FROM raw_ops.support_tickets t
LEFT JOIN raw_ops.customers c ON c.email = t.customer_email
LEFT JOIN raw_ops.subscriptions s ON s.customer_id = c.id
LEFT JOIN raw_ops.cs_agent_decisions d ON d.ticket_id = t.id
WHERE t.status NOT IN ('resolved', 'closed');

Module Configuration

Module 1: Gmail Trigger

Type: Gmail → Watch Emails
Configuration: - Criteria: Unread messages only - Mark as read: Yes - Limit: 10

Module 2: Customer Lookup

Type: HTTP → Make a request
URL: https://znfjpllsiuyezqlneqzr.supabase.co/rest/v1/rpc/fn_get_customer_by_email
Filter: To field contains hello@protocolraw.co.uk

Module 11: Find Open Ticket

Type: HTTP → Make a request
URL: https://znfjpllsiuyezqlneqzr.supabase.co/rest/v1/rpc/find_open_ticket_for_reply

Module 16: Edge Function (cs-agent-triage)

Type: HTTP → Make a request
URL: https://znfjpllsiuyezqlneqzr.supabase.co/functions/v1/cs-agent-triage
Method: POST
Timeout: 30 seconds

Response Fields:

Field Type Description
ticket_id UUID Created ticket ID
decision_id UUID AI decision ID
category string delivery / quality / feeding / subscription / escalation / other
confidence number 0.00-1.00 (0 for policy gate)
draft_response string AI-drafted response (empty if escalated)
escalate boolean Whether human review required
escalation_reason string Why escalated (or null)
policy_gate_triggered boolean Whether policy gate fired
policy_gate_code string Machine code for analytics

Module 4: Slack Notification

Type: Slack → Create a Message
Channel: #cust-ops

Filter (OR logic): - {{16.data.confidence}} < 0.85 - {{16.data.escalate}} = true

Operations Guide

Daily Monitoring

  1. Check #cust-ops Slack channel for escalated tickets
  2. Open Ops Portal → Support Workstation tab
  3. Process tickets by priority (high priority at top)
  4. Review AI draft, edit if needed, send response

Metrics to Track (Metabase)

Metric Query
Policy gate triggers by code GROUP BY policy_gate_code
Average confidence score AVG(confidence_score) WHERE model_used != 'policy_gate'
Draft acceptance rate COUNT(*) WHERE draft_usage = 'sent_as_is' / COUNT(*)
Escalation rate COUNT(*) WHERE escalate = true / COUNT(*)

Troubleshooting

Edge Function returning 401: - Check x-internal-secret header matches AGENT_INTERNAL_SECRET

Edge Function returning 500: - Check Supabase Edge Function logs - Common cause: ANTHROPIC_API_KEY not set

Policy gate not triggering: - Verify keywords match patterns in runPolicyGate() function - Check email body is being passed correctly

Duplicate tickets created: - Ensure dedupe_key is unique per email (uses Gmail message ID) - Check idempotency logic in Edge Function

Credentials & Access

Make.com

  • Scenario: CS-01 Support Intake (Phase 1)
  • Location: Protocol Raw workspace

Gmail

  • Account: anton@protocolraw.co.uk
  • Google Group: hello@protocolraw.co.uk forwards here

Anthropic (Claude)

  • Model: claude-sonnet-4-20250514
  • API Key: Stored in Supabase Edge Function secrets

Supabase

  • Project: znfjpllsiuyezqlneqzr
  • Schema: raw_ops
  • Edge Function: cs-agent-triage

Slack

  • Workspace: protocolraw.slack.com
  • Channel: #cust-ops

Future Enhancements (Not Yet Built)

Autonomous Mode

Currently all tickets require human review (shadow mode). Future autonomous mode will:

  1. Auto-send responses when confidence ≥ threshold (85%? 90%?)
  2. Require human review only for:
  3. Policy gate triggers
  4. Low confidence (<threshold)
  5. Categories marked for mandatory review

Requirements before enabling: - [ ] Define confidence threshold (recommend 90%) - [ ] Test email delivery end-to-end - [ ] Implement queue monitoring alerts - [ ] Achieve >80% draft acceptance rate in shadow mode

Queue Monitoring Alerts

Not yet implemented: - Slack alert if queue > 10 tickets - Slack alert if any ticket waiting > 2 hours - Daily summary of AI accuracy metrics

Version History

v2.1 (2026-01-19) - Unified Workflow

Changes: - Removed separate Agent Review tab - Unified review in Support Workstation - Added draft_usage tracking field - Policy gate info visible in Support Workstation - Updated v_support_queue view with policy gate fields

v2.0 (2026-01-19) - Edge Function Architecture

Major refactor: - Migrated from OpenAI GPT-4o to Claude Sonnet - Consolidated all business logic into single Edge Function - Added policy gate for zero-token escalations - Simplified Make.com to pure orchestration - Added idempotency via dedupe_key - Separated AI decisions into cs_agent_decisions table

v1.2 (2026-01-12) - Knowledge Base Integration

  • Added Module 15 for prompt fetching from AI Knowledge Base

v1.1 (2025-12-15) - Thread Detection

  • Added Module 11 for finding open tickets

v1.0 (2025-12-03) - Initial Production Release

  • Gmail trigger, OpenAI GPT-4o, Supabase storage, Slack notifications
  • SOP AI-KB-01: AI Knowledge Base (prompt content management)
  • SOP CS-00: Customer Operations System (master design)
  • SOP CS-02: Live Chat System
  • SOP CS-03: Autonomous Support Agent (future: auto-execute)
  • Ops Portal Documentation v3.5: Support Workstation reference

Document Owner: Protocol Raw Technical Team
Last Reviewed: 2026-01-19
Next Review: 2026-04-19
Version: 2.1
Status: Production Ready ✅