Guardrails, Évaluation & Sécurité

Contrôler qualité, risques et conformité des réponses LLM : détection (tox, PII, hallucinations), policies explicites, observabilité et décisions ACCEPT/REJECT/WARN/SANITIZE.

Input/Output/Tool/Context guardrails Groundedness & factualité PII/Secrets • Toxicité • Injection P95, coûts, audit trail

Dimensions

Groundedness : citations présentes, alignement avec sources (RAG).
Factualité : vérif croisée (knowledge base/GraphRAG).
Cohérence/Complétude : claire, non contradictoire, couvre la question.
Format : JSON valide, schéma respecté.

Scoring (ex.)

# scores 0..1
quality = 0.35*grounded + 0.25*factual + 0.25*coherent + 0.15*format_ok
decision = "ACCEPT" if quality>=0.80 else "WARN" if quality>=0.60 else "REJECT"

Calculer séparément coverage (répond à toutes les sous-questions ?).

Auto-évaluation LLM

Tu es un juge. Évalue (0..1):
- groundedness (sources [1][2] ?)
- factualité (contradictions ?)
- cohérence / complétude
- format JSON valide ?

Exemples de règles

# PII (simples)
EMAIL = r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}"
IPV4  = r"\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b"
APIKEY = r"(sk|AKIA|ghp)_[A-Za-z0-9]{20,}"

Combiner règles + classifieurs + jugement LLM (robuste).

Hallucination detector (idée)

if requires_citation and not has_citations(answer):
    risk.add("no_citation")
if claims_not_in_context(answer, context):
    risk.add("unsupported_claim")

Policy (JSON)

{
 "input":{"forbid":["violence","malware"],"sanitize":["PII"]},
 "output":{"require_citation":true,"max_tokens":800,"json_schema":"ResultSchema"},
 "tools":{"allow_hosts":["api.acme.com"],"sql_mode":"readonly"},
 "context":{"strip_injections":true}
}

Trace

{"span":"guardrail.check","scores":{"grounded":0.82,"tox":0.00},
 "flags":["no-risk"],"decision":"ACCEPT","latency_ms":32}

Conserver audit trail : pourquoi REJECT/WARN/SANITIZE ?

Décision

ACCEPT : passe tel quel.
WARN : afficher + bannière/avertissement.
SANITIZE : e.g. [EMAIL_SUPPRIMÉ], [SECRET_MASKED].
REJECT : bloquer + raison.

KPIs

Groundedness score (0–1), JSON validity rate (%).
Toxicity rate (% bloquées), PII leak rate.
Latency P95, Cost/answer, User satisfaction.

Tests

Jeux adversariaux : injection prompt, toxicité.
QA : question + contexte + réponse attendue.
Benchmarks : TruthfulQA, ToxicityBench, HolisticBias.

Mesures (snippets)

# JSON validity
ok=sum([1 for y in outputs if is_json(y)]); rate=ok/len(outputs)
# Groundedness naïf
g = has_citations(y) * overlap_with_sources(y, context)

Input guard

if contains_forbidden(prompt): reject("topic_forbidden")
prompt = sanitize_pii(prompt)

Output guard

answer = enforce_json_schema(answer, ResultSchema)
if requires_citation and not has_citations(answer): reject("missing_citation")

Tool & Context guards

# Tool
assert host in ALLOW_LIST and sql_is_readonly(query)
# Context
context = strip_injections(context)

def decide(scores, flags, mode="strict"):
    if "tox" in flags or "pii_leak" in flags: return "REJECT"
    if mode=="strict" and (scores["grounded"]<0.5 or flags.get("missing_citation")):
        return "REJECT"
    if flags.get("minor_pii") or scores["format"]<0.7: return "SANITIZE"
    return "ACCEPT" if scores["quality"]>=0.8 else "WARN"

Documenter publiquement la politique (transparence + auditabilité).

def guardrails_pipeline(question, context):
    # 1) Input
    if forbidden(question): return reject("forbidden_topic")
    q = sanitize_pii(question)
    # 2) LLM
    draft = llm.generate(prompt(q, context))
    # 3) Risk detection
    flags = {}
    if regex_hit(draft, [EMAIL, IPV4, APIKEY]): flags["minor_pii"]=True
    if toxic(draft): flags["tox"]=True
    # 4) Groundedness & format
    scores = {
      "grounded": groundedness(draft, context),
      "factual": fact_check(draft),
      "coherent": coherence(draft),
      "format": is_json(draft)
    }
    scores["quality"] = 0.35*scores["grounded"]+0.25*scores["factual"]+0.25*scores["coherent"]+0.15*(1 if scores["format"] else 0)
    # 5) Decision
    decision = decide(scores, flags, mode=os.getenv("GUARD_MODE","strict"))
    if decision=="SANITIZE": draft = sanitize_output(draft)
    if decision in ("REJECT","WARN"): log_decision(decision, flags, scores)
    return decision, draft

Guardrails, Évaluation & Sécurité

Évaluation de la qualité

Détection de risques

Guardrails (policies explicites)

Observabilité & logs

Stratégies clés

Boucle de feedback

Qualité & Évaluation (KPIs & Tests)

Templates de guardrails

Décision & arbitrage

Exemple (pseudo-code Python)