1. Inference Engine
Exécute le modèle (forward pass) – latence & throughput.
⚙️ IA Engines – Cœurs exécutifs (ChatGPT, Gemini, Mistral, Grok)
Inference, Training, Tokenization/Embeddings, RAG, Tools/Function-Calling, Memory, Optimization/Compilation, Serving/Orchestration.
▼ Voir les 8 briques
2. Training Engine
Backprop + parallélismes (data/model/pipeline).
3. Tokenizer & Embedding Engine
Texte → tokens/embeddings ; impact direct sur coûts.
4. Retrieval & RAG Engine
Recherche vectorielle + fusion contextuelle.
5. Tool & Function Engine
LLM → actions/API : function calling & plugins.
6. Memory Engine
Mémoire session/long terme & personnalisation.
7. Optimization & Compilation Engine
Compilation graph → kernels optimisés.
8. Serving & Orchestration Engine
Scaling, routage multi-model, SLO/quotas.
1. Inference Engine
Cœur d’exécution (GPU/TPU) avec batching, KV-cache, paged attention, speculative decoding.
Exemples
- vLLM
- TensorRT-LLM
- DeepSpeed-Inference
- FasterTransformer
- Triton Server
- ONNX Runtime
- TorchServe
KPIs clés
- Token/s
- Latency p50/p95
- Cost/token
- Memory footprint
- Batch efficiency
2. Training Engine
Optimise les poids avec FSDP/ZeRO, checkpointing, mixed precision, gradient accumulation.
Exemples
- Megatron-LM
- DeepSpeed
- PyTorch FSDP
- JAX/XLA
- Alpa
- Ray Train
KPIs clés
- Time/itération
- GPU util.
- Samples/sec
- Convergence
- Stabilité num.
3. Tokenizer & Embedding Engine
BPE/SentencePiece, normalisation, vocab, dimensions d’embeddings, quantisation possible.
Exemples
- tiktoken
- SentencePiece
- BPE
- WordPiece
KPIs clés
- Tokens/input
- OOV rate
- Embedding dim
- Latency enc/dec
4. Retrieval & RAG Engine
Index densé (HNSW/IVF), reranking, chunking, freshness, guardrails et citations.
Exemples
- FAISS
- Milvus
- Pinecone
- Weaviate
- Qdrant
- OpenSearch Vector
- Vertex AI Search
KPIs clés
- Recall@k
- MMR
- Latency retrieval
- Index size
- Freshness
5. Tool & Function Engine
Schéma JSON, router d’outils, exécution sandboxée, observation-action-feedback.
Exemples
- OpenAI Function Calling
- LangChain Tool Router
- LlamaIndex Tool Executor
- Gemini Extensions
- Claude Function Gateway
KPIs clés
- Success rate
- Tool latency
- Safety violations
- Cost/call
6. Memory Engine
Résumé hiérarchique, vector store, PII scrub, TTL, embeddings utilisateur.
Exemples
- LangChain Memory
- Vector Memory
- OpenAI Memory API
- Gemini Context Manager
KPIs clés
- Recall quality
- Drift
- Storage cost
- Privacy score
7. Optimization & Compilation Engine
XLA, TensorRT, TorchInductor, TVM, quantization/flash-attn, CUDA graphs.
Exemples
- XLA Compiler
- TensorRT-LLM
- TorchInductor
- TVM
- Triton Compiler
- Apple MLX
KPIs clés
- Latency ↓
- Throughput ↑
- VRAM ↓
- Energy/token
8. Serving & Orchestration Engine
Autoscaling, load-balancing, A/B, canary, observabilité, multi-tenant.
Exemples
- Ray Serve
- BentoML
- Triton Server
- Vertex AI Endpoints
- SageMaker Endpoints
- Mistral Router
KPIs clés
- p50/p95
- Error rate
- QPS
- Cost/req
- SLA met %