The Biggest Threat to Enterprise AI Is Not External
In Part 1, we covered the architecture blueprint for moving from PoC to production. Security is not something you think about after the blueprint is finished—it has to be built in from the design stage.
Three Core Security Threats in Enterprise AI
1. Prompt Injection
An attack in which a malicious user tries to bypass or overwrite the system prompt.
class PromptInjectionGuard:
INJECTION_PATTERNS = [
"ignore previous instructions",
"system: you are now",
"forget everything above",
" ",
" when when",
]
def check(self, user_input: str) -> bool:
lower = user_input.lower()
for pattern in self.INJECTION_PATTERNS:
if pattern in lower:
return False #
return True
def sanitize(self, user_input: str) -> str:
# User input when
return f"[USER INPUT START]\n{user_input}\n[USER INPUT END]"2. Data Exfiltration
The risk that the model exposes sensitive information from training data or the system prompt.
import re
class OutputFilter:
PII_PATTERNS = {
"email": r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
"phone": r'\b\d{2,3}[-.]?\d{3,4}[-.]?\d{4}\b',
"ssn": r'\b\d{6}[-]?\d{7}\b',
"card": r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b',
}
def filter(self, text: str) -> str:
for pii_type, pattern in self.PII_PATTERNS.items():
text = re.sub(pattern, f"[{pii_type.upper()} REDACTED]", text)
return textData Classification and AI Pipeline Isolation
from enum import Enum
class DataClassification(Enum):
PUBLIC = "public"
INTERNAL = "internal"
CONFIDENTIAL = "confidential"
RESTRICTED = "restricted"
class AIGateway:
""" """
MODEL_PERMISSIONS = {
DataClassification.PUBLIC: ["claude-sonnet-4-6", "gpt-4o"],
DataClassification.INTERNAL: ["claude-sonnet-4-6"], # external
DataClassification.CONFIDENTIAL: ["private-llm-on-prem"], #
DataClassification.RESTRICTED: [], # AI
}
def allowed_models(self, classification: DataClassification) -> list:
return self.MODEL_PERMISSIONS.get(classification, [])
async def query(self, prompt: str, classification: DataClassification) -> str:
models = self.allowed_models(classification)
if not models:
raise PermissionError(f"{classification.value} AI ")
return await llm_call(models[0], prompt)LLM Gateway Architecture
User request
↓
[Input filter]
- Prompt injection detection
- PII detection and masking
- Token limit check
↓
[Authentication/Authorization]
- JWT verification
- RBAC permission check
- Data classification decision
↓
[LLM call]
- Use only allowed models
- Write audit logs
↓
[Output filter]
- PII re-scan
- Harmful content filter
- Response logging
↓
User responseModel Access Control (RBAC)
from functools import wraps
ROLE_MODEL_ACCESS = {
"viewer": ["claude-haiku-4-5"],
"editor": ["claude-haiku-4-5", "claude-sonnet-4-6"],
"admin": ["claude-haiku-4-5", "claude-sonnet-4-6", "claude-opus-4-7"],
}
def require_model_access(model: str):
def decorator(fn):
@wraps(fn)
async def wrapper(user_role: str, *args, **kwargs):
allowed = ROLE_MODEL_ACCESS.get(user_role, [])
if model not in allowed:
raise PermissionError(f"{user_role} {model} ")
return await fn(*args, **kwargs)
return wrapper
return decorator
@require_model_access("claude-opus-4-7")
async def complex_analysis(text: str) -> str:
return await llm_call("claude-opus-4-7", text)GDPR/CCPA Compliance Checklist
| Requirement | Implementation |
|---|---|
| Data minimization | Pass only the necessary context to the LLM |
| Right to erasure | Personal data deletion API in the vector DB |
| Purpose specification | AI usage consent popup + logs |
| Cross-border transfer restrictions | On-premises or domestic-region only |
| Audit trail | Retain all LLM call logs for 2 years |
Part 3 covers an AI governance framework, including model lifecycle management and automated regulatory compliance.
Checking for Classification Labels Lost in the AI Pipeline
Even with a data classification policy, if labels drop off while documents pass through ingestion, chunking, and embedding, confidential material can end up in ordinary users' answers.
Causes
- The ingester reads only body text and ignores sensitivity labels from DLP or document management systems
- Unlabeled chunks default to "general"
- A document's label was raised later, but the index keeps the old one
Check 1: count unlabeled chunks
Regularly count chunks in the vector store with an empty classification field. Anything above zero means labels are being lost somewhere.
Check 2: leak test with a canary document
Index a test document marked confidential containing a unique string (e.g., CANARY-7F3A-CONF-TEST), then ask related questions from a general-permission account and check whether the string appears in answers or citations.
resp = ask(user="general-user", q="What is the budget of the confidential test project?")
assert "CANARY-7F3A" not in resp.text
assert all(c.metadata["classification"] != "confidential" for c in resp.citations)Fix
- Deny by default for unlabeled chunks (don't index them, or treat them as the highest class).
- Consume label-change events and update that document's chunk metadata immediately.
- Put the user's clearance filter in the retrieval query so content is blocked before generation.
Prevention
- Include the canary test in the deployment pipeline and weekly checks, and keep results as audit records.
Sources and checks vary by article. Only articles with a recorded check date, scope and sources below were checked, and only within that scope. For other articles, verify against the current docs for your version before applying anything, and email us about errors. Editorial policy · email us
This article has no recorded human check (date and scope) yet.
Comments
Be the first to comment.