POC of an AI agent for medical triage: SFT + DPO fine-tuning of an LLM (Qwen3-1.7B)

Python, LLM, Fine-Tuning, LoRA, DPO, Hugging Face, vLLM, MLOps, Healthcare AI, RGPD

POC of an AI Medical Triage Agent (Fine-Tuning SFT + DPO)

Overview

This project implements the complete specialization cycle of an LLM for a sensitive clinical use case: assisting hospital emergency department staff in the initial triage of patients. The objective is to transform a general-purpose base model (Qwen3-1.7B-Base) into an agent capable of evaluating a medical priority level, by following a rigorous methodology — from corpus creation to the deployment of a demonstration endpoint.

The project applies best practices of LLM fine-tuning and MLOps in a medical context, with particular attention paid to GDPR compliance, experiment traceability, and honesty regarding the system's clinical limitations.

Objectives

• Create a bilingual medical corpus (French/English), anonymized and GDPR compliant, for SFT training and DPO alignment.
• Specialize a base model through supervised fine-tuning (SFT) with LoRA adaptation.
• Align the model's behavior with validated clinical practices via Direct Preference Optimization (DPO).
• Deploy a demonstration endpoint optimized for inference (vLLM), integrated into a CI/CD pipeline.
• Objectively evaluate performance and formulate honest recommendations for scaling up.

Demonstrated Skills

Creation and Governance of a Medical Corpus

• Aggregation of bilingual medical corpora from heterogeneous sources (MediQA, FrenchMedMCQA, MedQuAD, UltraMedical-Preference).
• Production of approximately 5,000 instruction-response pairs for SFT and creation of a set of clinically validated preferential pairs for DPO.
• Anonymization of personal data with Presidio (AnalyzerEngine + AnonymizerEngine), testing of several masking strategies, and quality control of anonymization.
• Definition of a metadata schema (symptoms, medical history, vital signs, source, confidence level) and documentation of the GDPR process.
• Strict separation of train/val/test sets and clinical evaluation sets, with traceability of each transformation.

Supervised Fine-tuning (SFT) and LoRA Adaptation

• Fine-tuning of the Qwen3-1.7B-Base model on the created medical corpus, with LoRA adaptation to limit GPU footprint.
• Validation through small runs before scaling up, establishment of reproducible checkpoints.
• Systematic recording of hyperparameters and seeds for each trained version, tracking via MLflow / Weights & Biases.
• Active vigilance regarding the risk of overfitting on annotated examples.

Preference Alignment (DPO)

• DPO training of the SFT model using preferential pairs from the UltraMedical-Preference dataset.
• Clinical alignment objective: teach the model to distinguish a quality response from an incorrect or dangerous one.
• Dedicated safety controls (detection of hallucinations, potentially dangerous recommendations).

Deployment and Industrialization (MLOps)

• Containerization of the application (Docker) and exposure via a FastAPI API optimized by vLLM for inference.
• Automation of deployment via a CI/CD pipeline (GitHub Actions), including tests and deployment of new model versions.
• Latency and robustness tests under near real-world conditions, with full traceability of interactions for medical audits.
• Protection of secrets and endpoint access, documentation of usage limitations for users.

Evaluation and Strategic Reporting

• Critical analysis of performance metrics (latency, clinical relevance) with consistency between Go/No-Go verdicts and measured results.
• Drafting of a summary technical report, including a peer review of the conclusion to ensure consistency of the presented metrics.
• Formulation of honest recommendations on clinical performance gaps and a roadmap for scaling up (32B+ models, extended data).

Architecture

   Corpus médicaux bruts (MediQA, FrenchMedMCQA,
        MedQuAD, UltraMedical-Preference)
                       │
                       ▼
        Anonymisation RGPD (Presidio)
                       │
                       ▼
     Structuration : dataset SFT (~5000 paires)
        + dataset DPO (paires préférentielles)
                       │
                       ▼
       Fine-tuning SFT (Qwen3-1.7B-Base + LoRA)
                       │
                       ▼
        Alignement par préférences (DPO)
                       │
                       ▼
        Validation clinique & contrôles
           de sécurité (hallucinations)
                       │
                       ▼
      Conteneurisation (Docker) + API FastAPI
              Inférence optimisée (vLLM)
                       │
                       ▼
         Pipeline CI/CD (GitHub Actions)
                       │
                       ▼
     Endpoint pilote + rapport & recommandations

Technical Stack

Fine-tuning & Alignment

• PyTorch, Hugging Face Transformers
• PEFT (LoRA)
• DPO (Direct Preference Optimization)

Data & Compliance

• Hugging Face Datasets
• Presidio (GDPR anonymization)

Experiment Tracking

• MLflow / Weights & Biases

Deployment & Inference

• vLLM
• Docker, FastAPI

CI/CD

• GitHub Actions

Deliverables

• Anonymized and versioned bilingual medical dataset
• Qwen3-1.7B fine-tuned model (SFT + DPO), weights provided
• Deployed demonstration endpoint
• Automated CI/CD pipeline
• Technical report and strategic recommendations

Applied Best Practices

• Systematic and documented anonymization of personal data before any training
• Strict separation of training and evaluation data
• Full traceability of experiments (hyperparameters, seeds, checkpoints)
• Dedicated safety controls for clinical risk (hallucinations, dangerous recommendations)
• Rigorous consistency between measured metrics and verdicts formulated in the final report
• Explicit documentation of system usage limitations for end-users
• Progressive approach (conceptual validation → targeted optimization → industrial projection)

Results

Upon completion of this project, the system allows for:

• demonstrating the technical feasibility of an AI medical triage agent on a compact model;
• quantifying the gain provided by SFT fine-tuning and DPO alignment compared to the base model;
• exposing a pilot endpoint optimized for inference under near real-world conditions;
• ensuring the traceability of each interaction for medical audits;
• providing an honest evaluation of clinical performance gaps and a credible trajectory towards production.

Skills Acquired

• LLM Fine-tuning (SFT) and LoRA adaptation
• Model alignment through human preferences (DPO)
• Anonymization and governance of sensitive medical data (GDPR)
• Tracking and traceability of ML experiments (MLflow / W&B)
• Deployment of optimized inference (vLLM) and MLOps industrialization
• Implementation of CI/CD pipelines for language models
• Critical evaluation of clinical performance and communication of results to business stakeholders