Foundation Models & Vision-Language-Action (VLA) Models for Robotics: The Ultimate Guide

For most of robotics history a policy did one thing. You trained a network to insert one connector, or to pick one part off one conveyor, and if the part changed or the lighting shifted or someone asked for a different task, you trained again. Each skill was a fresh data-collection campaign, a fresh reward, a fresh model. The knowledge did not accumulate. A robot that could fold a towel knew nothing about folding a shirt, and a robot that could stack blocks in a lab could not stack them in a different lab across the hall.

Vision-language-action models are the attempt to break that pattern by copying what worked in language and vision: pretrain one large network on an enormous, diverse pile of data, then adapt it to specific tasks with a little more. A VLA takes a camera image (or several) and a natural-language instruction like "pick up the empty can and put it in the recycling bin," and it emits robot actions directly, joint or end-effector commands, at a few hertz to a few tens of hertz. The bet is that a single model trained across hundreds of tasks, dozens of robot bodies, and internet-scale images and text will generalize to objects, phrasings, and situations it never saw, the way a language model answers questions it was never explicitly trained on.

As of 2026 this is the most active area in robot learning, and it is genuinely early. The results are real: models that follow novel instructions, generalize to unseen objects, and transfer across robot bodies. The limits are also real: data is scarce, evaluation is unreliable, latency fights against high-frequency control, and nobody has yet produced the robotics equivalent of the moment large language models became broadly dependable. This guide is the practitioner's version: what a VLA actually is, how actions get turned into tokens the model can predict, where the training data comes from, the named systems that define the field, and the honest set of things that do not work yet.

The take: A VLA is a pretrained vision-language model with its output space repurposed to emit robot actions instead of words, fine-tuned on robot demonstrations. The architecture is mostly settled and borrowed from multimodal LLMs. The hard, unsolved problem is data: robotics has no internet-scale corpus of embodied experience, so every serious effort is really a bet on how to manufacture, pool, or substitute for that missing data. Cross-embodiment pooling (Open X-Embodiment), teleoperated demonstration factories, and simulation are the three answers on the table, and the winner is not yet decided. Treat 2026 VLAs as a powerful research substrate that generalizes impressively within its training distribution and remains brittle at the edges, more capable of one-off demos than of unattended production reliability.

Companion reading: imitation learning for robotics, reinforcement learning for robotics, robot simulation & digital twins, humanoid robot hardware, machine vision, and sim-to-real transfer.

Table of contents

1. Key takeaways
2. From task-specific policies to generalist models
3. What a VLA actually is
4. Action representation: tokenization and decoding
5. The three ingredients: pretraining, robot data, cross-embodiment
6. Open X-Embodiment and the data question
7. The systems that define the field
8. Architecture patterns and design choices
9. Compute, latency, and running one in real time
10. Generalization, evaluation, and current limits
11. Where VLAs fit next to RL and classical stacks
12. Outlook
13. Frequently asked questions

From task-specific policies to generalist models

The old way of building a robot skill was a pipeline of narrow specialists. A perception module detected the object, a planner computed a grasp, a controller executed a trajectory, and each piece was engineered or trained for its specific job. When learning entered the picture, it usually replaced one block: a learned grasp detector, or a policy trained by imitation or reinforcement learning for one task in one setting. The policy was a specialist. It had no concept of language, no knowledge of objects it had not been trained on, and no way to benefit from data collected for a different task.

The shift VLAs represent is the same shift that transformed natural language processing between roughly 2013 and 2020. NLP moved from per-task models (one classifier for sentiment, another for translation) to a single pretrained model adapted to many tasks. The engine of that shift was scale plus self-supervision on a giant, cheap, diverse corpus. A model that had read a large fraction of the internet acquired broad competence that transferred to tasks it was never explicitly trained on.

Robotics wants the same deal. The dream is one robot foundation model: pretrain once on a broad distribution of tasks, objects, environments, and even robot bodies, then adapt cheaply to any new job. Generalization would come from breadth of training rather than from clever engineering per task. A model that has seen thousands of manipulation behaviors across hundreds of settings should, in principle, pick up a new object or a rephrased instruction without a fresh data campaign.

Two things make robotics harder than language. First, there is no internet of robot actions. The web contains text and images in essentially unlimited quantity, but it does not contain the paired (observation, action) trajectories a robot policy needs to learn from. Every trajectory has to be produced, usually by a human teleoperating a real robot in real time. Second, robots are physically diverse. A sentence is a sentence, but a Franka arm, a UR5, a bimanual mobile manipulator, and a humanoid have different kinematics, different grippers, different action spaces, and different cameras. Pooling their data is not as simple as concatenating text. VLAs are the current best attempt to get the foundation-model payoff despite both obstacles.

What a VLA actually is

Concretely, a vision-language-action model is a function that maps an observation and an instruction to an action:

a_t  =  f_theta( image(s)_t ,  language instruction ,  [proprioception, history] )

The inputs are one or more camera images at time t, a natural-language instruction ("move the yellow block onto the plate"), and often the robot's own state (joint angles, gripper width) and a short history of recent observations or actions. The output is an action: an end-effector pose delta, a set of joint targets, and a gripper command, typically a 7-dimensional vector for a single arm (three translation, three rotation, one gripper) or more for bimanual and mobile systems.

The internals are almost always a transformer built on top of a pretrained vision-language model (VLM). A VLM is a multimodal model (in the lineage of CLIP, PaLI, LLaVA, PaLM-E, or the open-weight Prismatic and Llava-style stacks) that has already learned to encode images and text into a shared token sequence and reason over them. The VLA reuses that machinery. Images pass through a vision encoder (often a ViT, sometimes with SigLIP or DINO features) to become a sequence of visual tokens; the instruction is tokenized into text tokens; the two streams are concatenated and fed to the transformer backbone. Everything up to this point is inherited from vision-language pretraining, which is why the model already knows what a "banana" is, what "left" means, and that a "bowl" is the concave thing you put food in.

The one robotics-specific change is the output. Instead of decoding text tokens, the model decodes actions. How it does that is the central design decision, covered next. The reason this whole approach works at all is that the semantic grounding a VLM brings (objects, spatial relations, affordances, instruction meaning) is exactly the knowledge a robot needs to generalize, and it is knowledge you could never collect from robot demonstrations alone. Internet pretraining hands the robot a world model for free; the robot data only has to teach it how to move.

Rule of thumb: The vision-language backbone gives a VLA its generalization; the robot data gives it its competence. If a VLA generalizes to a novel object, thank the internet pretraining. If it executes a smooth grasp, thank the demonstrations.

Action representation: tokenization and decoding

The trick that made VLAs possible is turning continuous robot actions into something a language-model transformer can predict. There are two dominant approaches, and the field is mid-transition from the first to the second.

Discrete action tokenization. Take each dimension of the action vector and discretize its continuous range into a fixed number of bins, commonly 256. Bin 0 is the most negative value that dimension can take, bin 255 the most positive. A 7-dimensional action becomes 7 integers in [0, 255], and each integer is treated as a token drawn from a vocabulary of 256 action symbols. The transformer then predicts these tokens autoregressively, one dimension at a time, using the identical next-token machinery it uses for words:

# One action, tokenized (7-DoF end effector)
raw:     [ 0.021, -0.005,  0.014,  0.10, -0.02,  0.00,  1.0 ]   # dx dy dz  droll dpitch dyaw  grip
binned:  [  164 ,   131 ,   150 ,  178 ,  126 , 128 ,  255 ]    # each in [0,255]

# The transformer predicts these 7 tokens the same way it predicts text tokens.
# Decoding maps each bin back to a continuous value at the bin center.

This is elegant because it lets you reuse a pretrained language model with almost no architectural surgery. RT-2's key move was to overload some of the least-used tokens in the language model's existing vocabulary as action bins, so the same softmax that predicts words predicts actions. The cost is that autoregressive decoding of one token per dimension is slow, the binning quantizes away fine control, and independent per-dimension prediction ignores correlations between action dimensions.

Continuous action heads. The newer approach keeps the vision-language backbone but replaces the discrete token head with a continuous action decoder, usually a small diffusion or flow-matching network conditioned on the transformer's output. Instead of predicting bins, the model learns to generate a continuous action vector (or a short chunk of them) by iteratively denoising from noise, the same generative recipe behind image diffusion models. This gives smoother actions, higher effective resolution, and native support for multimodal action distributions (when several different actions are all valid, a diffusion head can represent that, where a single Gaussian or an argmax over bins cannot). Physical Intelligence's pi0 uses flow matching over action chunks; diffusion-policy heads appear across the field. The tradeoff is a more complex training objective and an iterative sampling step at inference, though few-step flow matching keeps that cheap.

Action chunking cuts across both. Rather than predicting a single action per forward pass, the model predicts a short horizon of future actions at once (say 8 to 50 steps), and the robot executes them open-loop before querying the model again. This amortizes the expensive forward pass over many control steps, which is often the difference between a big VLA being fast enough to run and being unusably slow. It also improves temporal consistency and reduces the compounding-error problem that plagues single-step imitation learning. The cost is reactivity: while executing a chunk the robot is not looking at fresh observations, so long chunks hurt on dynamic or contact-sensitive tasks. Typical systems balance this by re-planning partway through a chunk or blending overlapping predictions.

Action representation	How it decodes	Strengths	Weaknesses
Discrete tokens (256 bins/dim)	Autoregressive, per-dimension	Reuses LM head directly; simple	Slow decoding; quantized; ignores dim correlation
Diffusion head	Iterative denoising of continuous vector	Smooth, multimodal, high-resolution	Multiple sampling steps; heavier training
Flow matching head	Few-step continuous generation	Smooth, fast sampling, chunk-friendly	Newer, more complex objective
Action chunking (any head)	Predict N steps, execute open-loop	Amortizes compute; temporal consistency	Less reactive within a chunk

The three ingredients: pretraining, robot data, cross-embodiment

Every VLA is a mixture of three ingredients, and the design of a given model is mostly a choice about how much of each and how to combine them.

Internet-scale pretraining. The vision-language backbone is pretrained on web-scale image-text data before it ever sees a robot. This is where the model learns language, objects, and visual semantics. It is the cheapest and most abundant ingredient because the data already exists. A VLA that skips this and trains a vision-action model from scratch on robot data alone learns to execute but generalizes poorly, because a few hundred thousand trajectories cannot teach the breadth of visual and linguistic concepts the web teaches for free. RT-2's central finding was exactly this: co-training on web vision-language data plus robot data produced emergent generalization (following instructions about objects and concepts never seen in the robot data) that robot-only training did not.

Large robot demonstration datasets. The model learns to act from paired (observation, action) trajectories, almost always collected by human teleoperation. This is the scarce, expensive ingredient. A single demonstration might take 10 to 60 seconds of a person's time to produce, and a model might train on hundreds of thousands to a few million of them. Datasets like RT-1's 130,000 episodes, the Bridge datasets, DROID (76,000 trajectories across many scenes), and the aggregate Open X-Embodiment collection are the reference corpora. The economics are brutal: producing a million high-quality trajectories is a serious operational undertaking involving fleets of robots and teams of teleoperators.

Cross-embodiment. Because no single robot has enough data, pool across robot bodies. Train one model on demonstrations from many different robots (different arms, grippers, cameras, action spaces) so that skills learned on one body transfer to another and the total data pool is large enough to matter. This requires reconciling heterogeneous action and observation spaces, usually by normalizing to a common representation or by conditioning the model on which embodiment it is controlling. Cross-embodiment is the strategy that turns many small, incompatible datasets into one large useful one.

Rule of thumb: If your VLA generalizes to new objects and instructions, that came from internet pretraining. If it works across different robots, that came from cross-embodiment pooling. If it executes competently at all, that came from demonstrations. Weakness in any one ingredient shows up as a specific, predictable failure.

Open X-Embodiment and the data question

Open X-Embodiment (OXE), released in 2023 by a collaboration of more than 30 labs, is the clearest attempt to solve the data problem by pooling. It aggregated over 60 existing robot datasets covering 22 different robot embodiments into a single, format-unified collection of roughly a million trajectories, spanning arms, bimanual systems, and mobile manipulators doing hundreds of distinct skills. The datasets were converted to a common format (RLDS) so a model could train on all of them at once.

The headline result, from the RT-X models trained on OXE, was that a policy trained on the pooled multi-robot data outperformed policies trained on any single robot's data, including on that robot's own tasks. Cross-embodiment training was a net positive: skills and representations learned on one body helped on others. This validated the core bet that pooling diverse robot data buys generalization you cannot get from a single source, and it made OXE the default pretraining substrate for open VLAs like OpenVLA and Octo.

But OXE also makes the scale gap vivid. A million trajectories sounds large until you compare it to language. A single trajectory contains maybe a few hundred timesteps, so the whole corpus is on the order of hundreds of millions of state-action pairs. A large language model trains on trillions of tokens. Robotics is three to four orders of magnitude short of the data volume that made language models work, and the robot data is far more expensive per unit. The corpus is also unbalanced (dominated by a few large contributors and a few common tasks) and heavy on tabletop pick-and-place, so diversity of behavior is narrower than the trajectory count suggests.

Three responses to the data gap are in play, and every serious lab is betting on some mix:

Strategy	What it does	Who leans on it	Limitation
Pooling (OXE-style)	Merge many labs' real robot data	OpenVLA, Octo, RT-X	Still orders of magnitude short of web scale
Teleoperation factories	Operate robot fleets at scale to manufacture demonstrations	Physical Intelligence, Tesla, others	Expensive, slow, real-time-bound
Simulation	Generate synthetic trajectories cheaply	NVIDIA GR00T, sim-heavy stacks	Sim-to-real gap; hard for contact and deformables
Human/web video	Learn from videos of humans doing tasks	Research efforts, GR00T's "neural trajectories"	No action labels; embodiment mismatch

The last row is the wildcard. The web is full of videos of humans manipulating objects, and if a model could extract usable action knowledge from that (despite the missing action labels and the human-robot embodiment mismatch) the data ceiling would lift dramatically. This is an active research frontier, and NVIDIA's GR00T explicitly incorporates synthetic and human-video-derived data alongside real demonstrations. Nobody has fully cracked learning actions from action-free video, but the prize is large enough that the effort is intense.

The systems that define the field

The named efforts below are the reference points a practitioner should know. They illustrate the recipe rather than rank it, and the field moves fast enough that specific numbers date quickly.

RT-1 (Google, 2022). The proof of concept that a single transformer could absorb a large, diverse robot dataset (~130,000 episodes, 700+ tasks) and produce a general-purpose manipulation policy conditioned on language. RT-1 tokenized images and instructions and output discrete action tokens. It established that scale and diversity of robot data translated into generalization, and it produced the dataset that later work built on.

RT-2 (Google, 2023). The model that made "VLA" a category. RT-2 took a large pretrained vision-language model (PaLI-X / PaLM-E scale) and co-trained it on web vision-language data and RT-1 robot data simultaneously, emitting actions as text tokens from the model's existing vocabulary. The result showed emergent semantic generalization: the robot could follow instructions involving objects, categories, and reasoning that appeared only in the web data, never in the robot demonstrations. This was the demonstration that internet knowledge could flow into physical action.

Octo (2024). An open-source transformer generalist trained on OXE, designed to be flexible: it accepted varying camera setups, used a diffusion action head, and could be fine-tuned to new robots and action spaces cheaply. A deliberately small, research-friendly generalist.

OpenVLA (2024). A 7-billion-parameter open-weight VLA built on a Llama-2 backbone with SigLIP and DINOv2 vision features, trained on ~970,000 OXE trajectories with discrete action tokens. It became a widely used open baseline: competitive with much larger closed models on manipulation benchmarks while fully open and fine-tunable on a single workstation with parameter-efficient methods.

Physical Intelligence pi0 (2024). A VLA with a flow-matching action head, trained on a large mix of the company's own teleoperated data plus open datasets. pi0 emphasized dexterous, high-frequency, multi-stage tasks (folding laundry, assembling boxes, bussing tables) and showed that continuous action generation over chunks could drive smooth, fast bimanual manipulation. A prominent example of both the flow-matching head and the teleoperation-factory data strategy.

NVIDIA GR00T (2024-2025). A foundation-model effort aimed at humanoids, with the Isaac GR00T N-series open models, combining real demonstrations, simulation, and human video in its training mix and targeting the humanoid form factor. NVIDIA leans on its Isaac simulation stack to manufacture synthetic data and stretch scarce real data further.

Google DeepMind Gemini Robotics (2025). A VLA line built on the Gemini multimodal foundation model, with a companion "embodied reasoning" model for spatial and physical understanding. It represents the trend of building VLAs directly on frontier general-purpose multimodal models rather than smaller specialized backbones, emphasizing stronger generalization and interactive, reasoning-driven behavior.

The common thread: all of them are a pretrained vision-language backbone plus a robot-action output, trained on demonstrations. They differ in backbone scale, action head (discrete tokens vs diffusion vs flow matching), data mix (pooled real vs teleoperation-factory vs simulation-heavy), and target embodiment (tabletop arm vs bimanual vs humanoid).

Architecture patterns and design choices

Stepping back from individual systems, a handful of design axes describe the whole space.

Backbone scale. VLAs run from a few hundred million parameters (Octo-scale, research-friendly, fast) to 7 billion (OpenVLA) and up to frontier-model scale (Gemini Robotics). Bigger backbones bring stronger language and visual reasoning and better generalization, at the cost of inference latency and hardware. There is a genuine tension: the same scale that makes a model smart makes it slow enough to threaten real-time control.

Action head. Discrete tokenization (simple, reuses the LM head, slower, quantized) versus continuous generation via diffusion or flow matching (smoother, higher-frequency, multimodal, more complex). The field is trending toward continuous heads for anything requiring dexterity or speed, while discrete heads persist for simplicity and for tasks where coarse actions suffice.

Observation and history. Single image versus multiple cameras (wrist plus overhead is common), whether proprioception is fed in, and how much history the model sees. More cameras and history help with occlusion and partial observability but add tokens and cost.

Co-training ratio. How much web vision-language data to mix with robot data during training. Too little and generalization suffers; too much and the model may not commit enough capacity to acting. RT-2 showed co-training matters; the exact ratio is a tuning problem.

Language conditioning depth. Whether language is a simple task selector or is deeply integrated so the model can follow compositional, novel, or reasoning-heavy instructions. Frontier-backbone VLAs push toward the latter, where the model can chain steps and respond to corrections mid-task.

War story: A common early mistake is to fine-tune a VLA on a narrow in-house dataset and watch its benchmark numbers on that task climb while its generalization quietly collapses. The model overfits the new distribution and forgets the breadth the pretraining gave it, the robotics version of catastrophic forgetting. Teams that keep a slice of the original pretraining mix in the fine-tuning data, or freeze the backbone and adapt with low-rank updates, keep the generalization they paid for. Watching only the target-task success rate hides the loss until you test on anything else.

Compute, latency, and running one in real time

A VLA has to run inside a control loop on hardware that fits on a robot, and this is where the foundation-model dream meets the real-time reality of robotics.

Inference latency. A 7-billion-parameter transformer doing an autoregressive decode of 7 action tokens is not free. On a workstation GPU a single OpenVLA-class forward pass lands in the range of tens of milliseconds; on smaller onboard compute it is worse. That translates to control rates of only a few hertz to low tens of hertz for the big models, far below the 50-1000 Hz a stiff joint controller wants. This is why the architecture leans so hard on action chunking: predict 8 to 50 actions per forward pass and execute them open-loop, so a 10 Hz model produces effective control at a much higher rate. A fast joint-level controller runs underneath to track the predicted targets, the same two-rate structure used across robot learning.

# Two-rate control with a chunking VLA
VLA forward pass:     ~5-10 Hz   -> predicts a chunk of N=20 end-effector targets
Chunk execution:      open-loop, interpolated
Joint PD controller:  200-1000 Hz -> tracks the interpolated targets
# The big model sets intent slowly; the small controller does the fast tracking.

Onboard versus offboard. Big VLAs often run on a workstation or server GPU with the robot streaming observations and receiving actions over a link, which is fine in a lab but adds network latency and a reliability dependency for deployment. Running fully onboard needs either a smaller model or a capable edge accelerator (Jetson Thor / Orin-class), and it pushes toward distillation and quantization. Smaller distilled VLAs and quantized weights (INT8/INT4) are how you fit a usable model onto the robot itself, at some cost in capability. See edge AI robot compute for the hardware side.

Training compute. Pretraining or fine-tuning a VLA is far cheaper than pretraining a frontier language model, because the robot data is small and the backbone is often already pretrained. Fine-tuning OpenVLA with parameter-efficient methods (LoRA) fits on a single high-memory GPU. Full pretraining of a large VLA on OXE is a multi-GPU job measured in GPU-days to GPU-weeks, modest by LLM standards. The expensive resource, as in the rest of robot learning, is the demonstration data; the FLOPs are cheap by comparison.

Rule of thumb: Latency blocks a big VLA from a real robot more often than accuracy does. Reach for action chunking and a continuous head before you reach for a bigger model; a smaller model that runs at 10 Hz with 20-step chunks often beats a larger one that runs at 3 Hz.

Generalization, evaluation, and current limits

The honest assessment of 2026 VLAs separates what genuinely works from what is still demo-grade.

What generalizes well. Novel objects within familiar categories (a mug it has not seen, given many mugs in training). Rephrased and compositional instructions, thanks to the language backbone. Moderate changes in scene, lighting, and clutter. Transfer of a skill to a related object or location. These are the wins that internet pretraining plus diverse robot data deliver, and they are real: a good VLA follows "put the fruit in the bowl" for a fruit it never manipulated.

What does not. Long-horizon tasks where per-step error compounds into failure. Fine force control and contact-rich precision (tight insertions, delicate manipulation) that demonstrations rarely capture and quantized or averaged actions execute poorly. Genuinely out-of-distribution environments, deformable and transparent objects, and anything where the visual or physical distribution is far from the data. Recovery from novel failures the model has no demonstration for. And reliability at the tail: a VLA that succeeds 85% of the time is impressive in a demo and unusable in a factory that needs 99.9%.

The evaluation problem is structural. Success is measured by running the policy on a real robot and counting completions: slow (seconds to minutes per trial), expensive (hardware and people), noisy (physical conditions vary), and hard to reproduce (another lab's robot, lighting, and objects differ). There is no cheap, standardized, high-signal benchmark like the ones that drove language-model progress. Simulation benchmarks (SIMPLER, LIBERO) make evaluation cheap and reproducible but carry their own sim-to-real gap. Reported success rates rarely transfer across labs, which makes progress hard to measure. You cannot reliably improve what you cannot cheaply and comparably measure.

Reproducibility and robustness. Like the rest of robot learning, VLA results are sensitive to seeds, data details, and evaluation setup. A model that works in one lab's conditions may degrade in another's. Reliability, the ability to run unattended for hours without a failure that needs a human, remains the gap between impressive demonstrations and deployable products.

Rule of thumb: Judge a VLA claim by the diversity and realism of the evaluation. The peak success number tells you little on its own. "90% on 5 in-distribution tabletop tasks" and "60% across 50 tasks in unseen homes" describe very different capabilities, and the second is the harder, more meaningful result.

Where VLAs fit next to RL and classical stacks

VLAs do not erase the rest of the robot-learning toolkit; they occupy a specific niche and lean on the others.

Compared to task-specific reinforcement learning, a VLA trades peak performance and robustness on one task for breadth across many. An RL policy trained in simulation for one quadruped gait will out-walk any generalist on that gait, with tiny latency and strong disturbance rejection. A VLA cannot match that on a single task, but it can follow a spoken instruction to do dozens of loosely related manipulation tasks, which no single RL policy can. The two are complementary: RL excels at high-frequency, contact-rich, single-skill control learned in sim; VLAs excel at language-conditioned, multi-task, semantically general manipulation learned from demonstrations.

The training substrate under a VLA is imitation learning: behavior cloning on demonstrations is the core objective, which means VLAs inherit imitation's strengths (stable, simple training) and weaknesses (compounding error, only as good as the demonstrations, no notion of reward or improvement beyond the data). A promising direction is to combine them: pretrain a VLA on demonstrations for breadth, then fine-tune specific skills with RL for robustness and performance the demonstrations never reached. This mirrors the language-model recipe of pretraining followed by reinforcement-learning fine-tuning, and it is an active research direction for robotics.

VLAs also increasingly sit inside a hierarchy. A large VLA or a general multimodal model handles high-level reasoning and language ("figure out the steps to clear the table"), while lower-level skills (a precise grasp, a stable walk) are executed by specialized controllers or RL policies. The generalist provides semantic understanding and task decomposition; the specialists provide the fast, reliable, precise execution. Treating the VLA as the whole stack is a mistake for anything requiring precision or safety; treating it as the reasoning and generalization layer over a foundation of reliable low-level control is where it is strongest.

Outlook

The direction of travel is clear even though the destination is not. Data is the axis everything turns on, so expect the biggest moves there: larger teleoperation operations manufacturing demonstrations at scale, heavier use of simulation and synthetic-trajectory generation to stretch scarce real data, and continued attempts to learn from human video despite the missing action labels. Whoever solves data at the scale language solved it will likely define the field, and it is not yet obvious whether that is a pooling consortium, a well-funded teleoperation factory, or a simulation-plus-video breakthrough.

On architecture, continuous action heads (flow matching and diffusion) and action chunking are becoming standard for dexterity and speed, and backbones are trending toward frontier-scale multimodal models for stronger reasoning, offset by distillation and quantization to fit real hardware. Expect the reasoning-then-acting split to formalize: a large model that plans and decomposes, feeding smaller fast policies that execute. Expect RL fine-tuning on top of VLA pretraining to mature, giving generalists the robustness that pure imitation cannot. And expect evaluation to get serious attention, because the field cannot progress reliably on rollout counts alone; standardized, cheap, trustworthy benchmarks are a prerequisite for the next phase.

The realistic framing for a practitioner in 2026: VLAs are the most promising path to general-purpose robots and simultaneously not yet reliable enough for unattended deployment on hard tasks. They generalize impressively within their training distribution and remain brittle outside it. The gap between a compelling demonstration and a dependable product is the whole game, and closing it is mostly a data and evaluation problem, with architecture a secondary and increasingly settled concern. Whether this decade produces the "ChatGPT moment for robotics" depends less on a clever model and more on whether someone manufactures embodied data at a scale nobody has managed yet. For where this sits in the broader trajectory, see the next 10 years of robotics.

Frequently asked questions

What does VLA stand for, and how is it different from a VLM? Vision-Language-Action. A VLM (vision-language model) takes images and text and outputs text. A VLA takes images and text and outputs robot actions. A VLA is typically built by taking a pretrained VLM and replacing or extending its output so it emits actions instead of words, then fine-tuning on robot demonstrations. The vision-language understanding is inherited; the acting is added.

How does a language-model transformer output continuous robot actions? Two ways. The classic approach discretizes each action dimension into bins (commonly 256), treats those bins as vocabulary tokens, and predicts them autoregressively like words. The newer approach attaches a continuous action head (diffusion or flow matching) that generates the action vector directly. Continuous heads give smoother, higher-frequency control and handle multimodal action distributions; discrete tokens are simpler and reuse the language-model head directly.

Why is data such a bottleneck for VLAs? Because there is no internet of robot actions. Language models train on trillions of freely available web tokens; robot policies need paired (observation, action) trajectories that mostly have to be produced by humans teleoperating real robots in real time. The largest pooled datasets (Open X-Embodiment, roughly a million trajectories) are still three to four orders of magnitude short of the data volume that made language models work, and the data is far more expensive per unit.

What is Open X-Embodiment and why does it matter? It is a 2023 collaboration that merged over 60 robot datasets across 22 robot bodies into one unified collection of about a million trajectories. It matters because models trained on the pooled multi-robot data (RT-X) outperformed models trained on any single robot's data, proving that cross-embodiment pooling buys generalization. It became the default pretraining substrate for open VLAs like OpenVLA and Octo.

What is cross-embodiment and why is it hard? Cross-embodiment means training one model on data from many different robot bodies so skills transfer between them and the pooled dataset is large enough to be useful. It is hard because robots differ in kinematics, grippers, action spaces, and cameras, so their data cannot simply be concatenated the way text can. Models reconcile this by normalizing to a common action representation or by conditioning on which embodiment is being controlled.

Can a VLA run in real time on a robot? The big models (billions of parameters) run at only a few to low tens of hertz per forward pass, below what stiff control needs. Action chunking (predict a horizon of actions per forward pass and execute them open-loop under a fast joint controller) is the standard fix, effectively multiplying the control rate. Large models often run offboard on a workstation GPU with the robot streaming observations; running fully onboard needs a smaller, distilled, or quantized model on edge hardware.

How do VLAs relate to reinforcement learning and imitation learning? VLAs are trained by imitation learning (behavior cloning on demonstrations), so they inherit its stability and its weaknesses (compounding error, limited by demonstration quality, no reward signal). RL is complementary: an emerging recipe pretrains a VLA on demonstrations for breadth, then fine-tunes specific skills with RL for robustness the demonstrations never showed, mirroring the pretrain-then-RL recipe from language models.

Are VLAs ready for production? For narrow, controlled tasks with human oversight, some are usable. For general, unattended, high-reliability deployment on hard tasks, not yet. VLAs generalize well within their training distribution and remain brittle outside it, struggle with long-horizon tasks, fine force control, and out-of-distribution scenes, and the tail reliability that factories need (99.9%+) is not there. Evaluation is also hard, so claims should be judged by the diversity and realism of testing; the peak success number means little on its own.

Which VLA should I start experimenting with? For open, hands-on work, OpenVLA (7B, open weights, fine-tunable with LoRA on a single high-memory GPU) and Octo (smaller, flexible, diffusion head) are the common open baselines, both trained on Open X-Embodiment. NVIDIA's open GR00T N-series targets humanoids. Closed frontier efforts (Physical Intelligence pi0, Google DeepMind Gemini Robotics) define the capability edge but are less openly accessible. Start with an open model on a simulation benchmark (LIBERO, SIMPLER) before committing to hardware.

Why is evaluating VLAs so difficult? Success is measured by physical rollouts on real robots: slow, expensive, noisy, and hard to reproduce across labs with different hardware, lighting, and objects. There is no cheap standardized benchmark like the ones that drove language-model progress. Simulation benchmarks make evaluation cheap and reproducible but carry a sim-to-real gap. The result is that reported numbers rarely transfer between labs, which makes progress hard to measure and is itself a serious bottleneck.