LabVLA 🔬🤖
Vision-Language-Action Foundation Model for Scientific Laboratories
Vision-Language-Action
for Scientific Discovery
LabVLA is the first multimodal foundation model purpose-built for scientific laboratory scenarios. It unifies visual perception, natural language understanding, and robotic action generation to enable autonomous experimentation.
Input Modalities
📸 Experimental Images / Video – microscope streams, reaction vessels, TLC plates, crystallization dynamics.
📝 Natural Language Instructions – “Synthesize target molecule X”, “Monitor pH every 2 min”.
📊 Instrument State – temperature, pressure, spectrometer readings, robotic joint angles.
Output: Executable Actions + Safety
🔧 Action sequences – low-level robot primitives (pipetting, gripping, heating) or high-level experiment plans.
⚠️ Safety constraint checks – embedded compliance with hazard limits, protocol bounds, and equipment specs. Outputs are verified before physical execution.
Scientific Lab VLA — first of its kind. Unlike generic VLAs (RT-2, etc.), LabVLA incorporates domain-specific inductive biases: chemical spectra, biological imaging, instrument semantics, and mechanistic constraints, enabling zero-shot generalization to unseen lab tasks.
📷 Scientific Image Understanding
Spectrum parsing (NMR, IR, MS), microscopic image analysis (cell morphology, crystal habits), real-time experiment phenotyping (color change, precipitation, turbidity).
🗣️ Experimental Instruction Generation
From high-level goal “Synthesize target molecule” → detailed step-by-step protocol: reagent volumes, temperature ramps, purification steps, with conditional branching based on visual feedback.
🔗 Multimodal Alignment
Align visual features (micrograph structures) ↔ scientific concepts (crystal polymorphs, protein aggregates) ↔ robotic instructions (dispense, mix, transfer). Enables joint embedding space for perception-to-action.
Scientific Experiment Video Library
> 2M annotated clips of chemical synthesis, biological assays, materials processing. Includes multi-view camera, thermal imaging, and instrument telemetry.
Instrument Operation Logs
Hundreds of thousands of robotic and semi-automated runs: liquid handlers, reactors, analyzers. Action traces paired with sensor readings.
Knowledge Graph-Augmented Instruction-Action Pairs
Sci-KG enhanced triplets: (instruction context, visual state, action sequence). Enables semantic generalization and compositional reasoning.
LabVLA-7B
7B parameters · 32k context · 4-bit quantizable
Best for rapid prototyping, edge lab devices, and real-time inference. Balances performance and throughput.
LabVLA-13B
13B parameters · multimodal alignment · tool use
Full scientific image+state understanding, complex reasoning, and long-horizon planning. Recommended for most autonomous labs.
LabVLA-34B
34B parameters · advanced safety guardrails
State-of-the-art scientific reasoning, zero-shot generalization to novel instruments, and highest safety compliance. For large-scale discovery platforms.
📄 Full model cards include benchmark scores (LabGym-Bench atomic/composite), safety evaluation, and fine-tuning recipes. Pre-trained checkpoints will be available on HuggingFace 🤗.
Built for the Embodied Science Ecosystem
LabVLA natively integrates with PLAD agents (Perception-Language-Action-Discovery) and LabGym digital twin for sim-to-real transfer. It serves as the core cognitive engine that translates scientific intent into reliable, safe physical actions across robotic platforms.