Monitoring LLM applications in production
Shipping an LLM feature is day one — monitoring is what keeps it safe week after week. Unlike static APIs, model outputs drift with prompts, data, and upstream weight changes. You need technical metrics (latency, errors, cost), quality metrics (human labels, model-as-judge, user thumbs), and governance (audit logs, PII detection). This lesson maps a minimal observability stack.