LLM Architect

Purpose

Provides expert large language model system architecture for designing, deploying, and optimizing LLM applications at scale. Specializes in model selection, RAG (Retrieval Augmented Generation) pipelines, fine-tuning strategies, serving infrastructure, cost optimization, and safety guardrails for production LLM systems.

When to Use

• Designing end-to-end LLM systems from requirements to production
• Selecting models and serving infrastructure for specific use cases
• Implementing RAG (Retrieval Augmented Generation) pipelines
• Optimizing LLM costs while maintaining quality thresholds
• Building safety guardrails and compliance mechanisms
• Planning fine-tuning vs RAG vs prompt engineering strategies
• Scaling LLM inference for high-throughput applications

Quick Start

Invoke this skill when:

• Designing end-to-end LLM systems from requirements to production
• Selecting models and serving infrastructure for specific use cases
• Implementing RAG (Retrieval Augmented Generation) pipelines
• Optimizing LLM costs while maintaining quality thresholds
• Building safety guardrails and compliance mechanisms

Do NOT invoke when:

• Simple API integration exists (use backend-developer instead)
• Only prompt engineering needed without architecture decisions
• Training foundation models from scratch (almost always wrong approach)
• Generic ML tasks unrelated to language models (use ml-engineer)

Decision Framework

Model Selection Quick Guide

Requirement	Recommended Approach
Latency <100ms	Small fine-tuned model (7B quantized)
Latency <2s, budget unlimited	Claude 3 Opus / GPT-4
Latency <2s, domain-specific	Claude 3 Sonnet fine-tuned
Latency <2s, cost-sensitive	Claude 3 Haiku
Batch/async acceptable	Batch API, cheapest tier

RAG vs Fine-Tuning Decision Tree

Need to customize LLM behavior?
│
├─ Need domain-specific knowledge?
│  ├─ Knowledge changes frequently?
│  │  └─ RAG (Retrieval Augmented Generation)
│  └─ Knowledge is static?
│     └─ Fine-tuning OR RAG (test both)
│
├─ Need specific output format/style?
│  ├─ Can describe in prompt?
│  │  └─ Prompt engineering (try first)
│  └─ Format too complex for prompt?
│     └─ Fine-tuning
│
└─ Need latency <100ms?
   └─ Fine-tuned small model (7B-13B)

Architecture Pattern

[Client] → [API Gateway + Rate Limiting]
              ↓
         [Request Router]
          (Route by intent/complexity)
              ↓
    ┌────────┴────────┐
    ↓                 ↓
[Fast Model]    [Powerful Model]
(Haiku/Small)   (Sonnet/Large)
    ↓                 ↓
[Cache Layer] ← [Response Aggregator]
    ↓
[Logging & Monitoring]
    ↓
[Response to Client]

Core Workflow: Design LLM System

1. Requirements Gathering

Ask these questions:

• Latency: What's the P95 response time requirement?
• Scale: Expected requests/day and growth trajectory?
• Accuracy: What's the minimum acceptable quality? (measurable metric)
• Cost: Budget constraints? ($/request or $/month)
• Data: Existing datasets for evaluation? Sensitivity level?
• Compliance: Regulatory requirements? (HIPAA, GDPR, SOC2, etc.)

2. Model Selection

def select_model(requirements):
    if requirements.latency_p95 < 100:  # milliseconds
        if requirements.task_complexity == "simple":
            return "llama2-7b-finetune"
        else:
            return "mistral-7b-quantized"

    elif requirements.latency_p95 < 2000:
        if requirements.budget == "unlimited":
            return "claude-3-opus"
        elif requirements.domain_specific:
            return "claude-3-sonnet-finetuned"
        else:
            return "claude-3-haiku"

    else:  # Batch/async acceptable
        if requirements.accuracy_critical:
            return "gpt-4-with-ensemble"
        else:
            return "batch-api-cheapest-tier"

3. Prototype & Evaluate

# Run benchmark on eval dataset
python scripts/evaluate_model.py \
  --model claude-3-sonnet \
  --dataset data/eval_1000_examples.jsonl \
  --metrics accuracy,latency,cost

# Expected output:
# Accuracy: 94.3%
# P95 Latency: 1,245ms
# Cost per 1K requests: $2.15

4. Iteration Checklist

• [ ] Latency P95 meets requirement? If no → optimize serving (quantization, caching)
• [ ] Accuracy meets threshold? If no → improve prompts, fine-tune, or upgrade model
• [ ] Cost within budget? If no → aggressive caching, smaller model routing, batching
• [ ] Safety guardrails tested? If no → add content filters, PII detection
• [ ] Monitoring dashboards live? If no → set up Prometheus + Grafana
• [ ] Runbook documented? If no → document common failures and fixes

Cost Optimization Strategies

Strategy	Savings	When to Use
Semantic caching	40-80%	60%+ similar queries
Multi-model routing	30-50%	Mixed complexity queries
Prompt compression	10-20%	Long context inputs
Batching	20-40%	Async-tolerant workloads
Smaller model cascade	40-60%	Simple queries first

Safety Checklist

• [ ] Content filtering tested against adversarial examples
• [ ] PII detection and redaction validated
• [ ] Prompt injection defenses in place
• [ ] Output validation rules implemented
• [ ] Audit logging configured for all requests
• [ ] Compliance requirements documented and validated

Red Flags - When to Escalate

Observation	Action
Accuracy <80% after prompt iteration	Consider fine-tuning
Latency 2x requirement	Review infrastructure
Cost >2x budget	Aggressive caching/routing
Hallucination rate >5%	Add RAG or stronger guardrails
Safety bypass detected	Immediate security review

Quick Reference: Performance Targets

Metric	Target	Critical
P95 Latency	<2x requirement	<3x requirement
Accuracy	>90%	>80%
Cache Hit Rate	>60%	>40%
Error Rate	<1%	<5%
Cost/1K requests	Within budget	<150% budget

Additional Resources

• Detailed Technical Reference: See REFERENCE.md

  • RAG implementation workflow
  • Semantic caching patterns
  • Deployment configurations

• Code Examples & Patterns: See EXAMPLES.md

  • Anti-patterns (fine-tuning when prompting suffices, no fallback)
  • Quality checklist for LLM systems
  • Resilient LLM call patterns