Robot Teleoperation: The Ultimate Guide

A surgeon in one operating room moves a pair of finger loops, and eight feet away four robotic arms echo the motion inside a patient, with the tremor of the human hand filtered out and the scale of the motion shrunk five to one. A pilot in Nevada flies an aircraft over another continent through a two-second round trip of satellite delay. An operator on a support ship watches a murky camera feed and works a manipulator on a wellhead three kilometers down, where the tether that carries the video also carries the only thing keeping a fifteen-million-dollar vehicle from being abandoned. All three are teleoperation: a human in the control loop of a machine they are not standing next to, closing perception and action across a link that adds delay, loses information, and sometimes lies.

This guide is about the engineering of putting a person inside a robot's control loop across distance. We cover why teleoperation still matters when autonomy is improving fast, the interface hardware (joysticks, six-DOF haptic devices, VR headsets, and leader-follower rigs like ALOHA), the kinematic problem of mapping a human hand onto a robot that has a different body, the latency problem and the predictive-display and move-and-wait strategies that fight it, bilateral teleoperation with force feedback and the control architectures that carry it, the passivity theory that keeps a force-reflecting loop from exploding, shared and supervised autonomy that blend human intent with machine competence, and the applications that pay for all of it: surgery, subsea, space, explosive-ordnance disposal, and the newest one, collecting demonstration data to train robots that will eventually not need a human at all.

The take: teleoperation is the bridge robots cross while their autonomy is still too brittle to trust alone, and it is also the tool that builds that autonomy by generating demonstration data. The two hard problems are latency and force. Delay turns a stable hand-eye loop into an oscillator, and the fix is either to hide the delay behind a predictive local model or to wrap the whole link in a passivity guarantee that trades transparency for stability. Force feedback makes contact tasks feel real, but a force-reflecting loop across a network is a closed loop with a human, a robot, and an unknown environment all inside it, so you design it against a stability proof, not against a demo. Every serious system in 2026 is sliding from raw direct control toward shared autonomy, where the human supplies intent and the robot supplies the fast, precise, delay-tolerant execution.

Companion reading: surgical & medical robots, imitation learning for robotics, underwater robots: AUVs & ROVs, exoskeletons, and real-time control systems.

Table of contents

1. Key takeaways
2. Why teleoperation matters
3. The interface: from joysticks to leader-follower rigs
4. Mapping a human onto a robot
5. The latency problem
6. Predictive display and move-and-wait
7. Bilateral teleoperation and force reflection
8. Passivity: the stability backbone
9. Shared and supervised autonomy
10. Teleoperation as a data engine
11. Applications
12. Metrics, ergonomics, and failure modes
13. Where it is heading
14. Frequently asked questions

Why teleoperation matters

Autonomy keeps improving, and a fair question is why anyone still builds a control station with a human in it. The answer is that teleoperation and autonomy solve different problems, and the places teleoperation wins share a shape.

The task is dangerous and an error is catastrophic. Explosive-ordnance disposal is the clean example: a robot approaches a suspected device, and a human decides, frame by frame, whether to cut, pull, or disrupt. No one wants a learned policy making that call in 2026. Nuclear decommissioning, chemical response, and firefighting reconnaissance are the same: the robot is a proxy body that keeps the human out of the blast, the radiation, or the heat.

The place is remote and the environment is unmodeled. Three kilometers underwater or on the surface of Mars, you cannot pop out to fix a stuck gripper, and you cannot pre-build a simulator faithful enough to trust an autonomous stack against the unknown. A human brings general-purpose reasoning to a novel scene that no training distribution covered.

The task is dexterous and the consequences are high. Surgery demands sub-millimeter manipulation inside a deformable, bleeding, patient-specific anatomy, with a human liable for the outcome. The robot supplies tremor filtering, motion scaling, and extra wrists inside a small port. The human supplies the judgment.

You need demonstrations. This one is newer and it changed the field. To train a robot to do a task autonomously by imitation learning, you need examples of the task done correctly, and the fastest way to generate them is to teleoperate the actual robot through the task hundreds of times. Teleoperation became the front end of the autonomy pipeline it is supposedly being replaced by.

Rule of thumb: reach for teleoperation when the cost of an autonomy mistake is high, the environment is open-ended, or you need demonstration data. Reach for autonomy when the task is repetitive, well-modeled, or must run faster or longer than a human can supervise. Most mature systems end up blending the two.

The honest framing is a spectrum from direct manual control, where the human moves every joint, through shared and supervised autonomy, to full autonomy, and a given robot slides along it as its competence grows. Teleoperation is the left end of that axis and the on-ramp to the rest of it.

The interface: from joysticks to leader-follower rigs

The interface is where the human's intent becomes signal. Four families cover almost everything in service.

Joysticks and gamepads are the workhorses of mobile and field robotics. A two- or three-axis stick maps to robot velocity, a second stick to a camera or a manipulator, and buttons to modes. They are cheap, rugged, learnable in minutes, and low-bandwidth in the human-effort sense, which is exactly why bomb-disposal robots, ROVs, and ground vehicles use them. Their limit is degrees of freedom: driving a six-DOF arm through two thumbsticks is slow and unintuitive, and there is no force feedback.

Six-DOF haptic devices are precision input tools that also render force back to the hand. A stylus or handle on a small articulated linkage or parallel mechanism reads position and orientation and drives small motors to push back. Devices in this class (the 3D Systems Touch, the Force Dimension omega and sigma series, Haption's arms) are the standard for research bilateral teleoperation and are the lineage behind surgical master consoles. They give clean six-DOF input and real force reflection over a small workspace, which is why they suit fine manipulation and are wrong for driving across a room.

VR headsets with tracked controllers exploded as an interface because the tracking is cheap and the immersion is high. The operator sees a stereo view, often from the robot's head cameras or a reconstructed scene, and the controllers track their hands in full six-DOF at sub-millimeter resolution. This is how many humanoid and mobile-manipulator teleoperation stacks work now: the human reaches, the robot's arm follows, and the depth and spatial intuition that a flat screen destroys come back. The weakness is that consumer VR controllers give no force feedback, so contact is felt only through the eyes.

Leader-follower rigs are the most direct interface of all: a physical replica of the robot's arm that the human backdrives by hand, with the follower arm copying the joint angles in real time. The operator's own kinesthetic sense of where their hand is becomes the input, so the mapping is one-to-one and needs no learning. The ALOHA system (Stanford, 2023) built low-cost leader-follower pairs of manipulator arms specifically for cheap, high-quality demonstration collection, and Mobile ALOHA put the rig on a wheeled base. When the leader and follower are identical arms, the correspondence is exact and even bimanual coordination feels natural. The cost is that you need a whole extra robot as the input device, and unless you add torque sensing the operator feels no force from the far end.

Interface	DOF	Force feedback	Intuitiveness	Typical use
Joystick / gamepad	2-6 (multiplexed)	No	Moderate	Mobile robots, ROVs, EOD, drones
6-DOF haptic device	6 (+grip)	Yes	High for fine work	Surgery masters, bilateral research
VR headset + controllers	6 per hand	Rarely	Very high spatial	Humanoids, mobile manipulation, data collection
Leader-follower rig	Matches robot	Optional (torque)	Highest, one-to-one	Demonstration collection, dexterous manipulation
Exoskeleton / wearable	Many	Yes	High for whole-arm	Whole-body avatars, heavy manipulation

Wearable exoskeleton interfaces sit at the immersive extreme, reading and reflecting force across the whole arm or body for avatar-style teleoperation. They overlap heavily with the exoskeleton hardware world, and they are the most transparent interface and the most expensive.

Mapping a human onto a robot

An interface produces motion in the human's frame. Turning that into robot motion is a kinematic design problem with a few recurring decisions.

Position control versus rate control. In position (or position-position) control, the robot's end-effector pose tracks the operator's hand pose directly: move your hand ten centimeters right, the tool goes ten centimeters right (times a scale factor). It is intuitive and precise for tasks that fit inside the operator's reach. In rate (velocity) control, the operator's displacement commands a velocity: hold the stick right and the robot keeps moving right until you release. Rate control suits large or unbounded workspaces (driving a vehicle, slewing a crane) where position control would run out of arm. Many systems mix them: position control for the fine manipulation, rate control for gross positioning.

Motion scaling. Surgical systems scale the master motion down, commonly by factors of two to five, so a centimeter of surgeon hand becomes a couple of millimeters of instrument. Scaling below one magnifies precision and filters tremor. Scaling above one is used when a small operator input should cover a large workspace. The scale factor s simply multiplies the mapped displacement: x_robot = s * x_hand.

Indexing (clutching). A position-controlled hand runs out of workspace before the robot does. Clutching solves this the way lifting and repositioning a mouse solves running off the mousepad: the operator presses a clutch, decouples the mapping, repositions their hand to a comfortable spot, releases, and the robot stays put while the human re-centers. Every position-control surgical console has a clutch pedal for exactly this reason.

Body correspondence. When the robot's kinematics differ from the human's, the mapping gets interesting. A seven-DOF arm has a redundant elbow the human arm does not obviously correspond to. A robot hand with a different number of fingers cannot copy human finger poses one-to-one, so retargeting solves an optimization that matches fingertip positions or contact intent while respecting the robot's joint limits. For humanoid teleoperation, whole-body retargeting maps the operator's tracked joints onto the robot's while keeping it balanced, which is a constrained inverse-kinematics problem solved every control cycle.

Rule of thumb: use position control with scaling and a clutch for fine manipulation inside a bounded workspace, and rate control for gross motion over a large or unbounded one. The clutch is not optional the moment you choose position control.

The latency problem

Latency is the single hardest thing in teleoperation, and it is worth understanding why it is so corrosive.

A human driving a robot by looking at a video feed and moving a controller is a feedback loop: the human is a controller, the robot and environment are the plant, and the communication link inserts a pure time delay in both the forward (command) and return (video and force) paths. A pure transport delay of T seconds contributes phase lag that grows linearly with frequency, phi(omega) = -omega * T, while leaving the amplitude untouched. That last part is the trap. The loop gives no warning that it is approaching instability, because the magnitude response is flat; only the phase is quietly eaten away. Push the loop gain up (a motivated operator moving quickly) and the accumulated phase lag crosses the point where the feedback becomes positive, and the whole hand-eye system oscillates. In force-reflecting systems the effect is violent: the reflected force and the operator's response chase each other into a growing buzz.

The delays add up from several sources. Speed-of-light transport dominates over long links: geostationary satellite adds roughly 240-280 ms each way, so a round trip through one hop is over half a second, and Earth-to-Moon is about 1.3 seconds each way. Earth-to-Mars ranges from about 3 to 22 minutes each way, which puts real-time teleoperation off the table entirely. On top of transport sit codec and buffering delays in the video path (often 100-300 ms for compressed video), network jitter, and the control-loop periods at each end. A subsea tether or a fiber link can be low-latency, but a compressed 4K video feed over it may not be.

Link	One-way transport	Real-time teleop?
Local wired / LAN	< 1 ms	Yes, force reflection viable
Terrestrial internet (regional)	10-50 ms	Yes, with care
Geostationary satellite (1 hop)	240-280 ms	Marginal, needs predictive display
Earth-Moon	~1.3 s	Predictive display / supervisory only
Earth-Mars	3-22 min	No; command sequences only

The classic human adaptation to delay, documented since the 1960s teleoperation literature (Ferrell, Sheridan), is move-and-wait: the operator makes a small open-loop move, stops, waits for the delayed feedback to confirm the result, then moves again. It is stable because the human takes themselves out of the closed loop during each wait, and it is slow, with completion time growing roughly linearly with delay. Move-and-wait is what an unaided human does across a second of delay, and it is the baseline that every latency-mitigation technique is trying to beat.

Predictive display and move-and-wait

If you cannot remove the delay, hide it. Predictive display puts a fast, local model of the robot and environment in front of the operator so they can close a tight loop against the prediction instead of the delayed reality.

The mechanism: the control station maintains a model (a kinematic or dynamic simulation, or a rendered 3D reconstruction) of the robot and the known parts of the scene. When the operator moves, the local model responds immediately, with no round-trip delay, and the operator drives against that responsive predicted robot. The command also goes out over the link to the real robot, which executes it T seconds later, and the returning telemetry corrects the model to keep the prediction from drifting away from reality. The operator experiences a locally responsive system and only sees the model-versus-reality error, which is far more tolerable than the raw delay.

Predictive display was central to space telerobotics and to deep-sea work, where seconds of delay would otherwise force pure move-and-wait. A common form overlays a wireframe or shaded prediction of where the arm will be onto the delayed camera image, so the operator sees both the predicted pose (responsive) and the real pose (lagging) and drives the prediction into the target. The quality of the prediction is everything: for the robot's own kinematics the model is excellent because the robot's forward kinematics are known exactly, so the predicted end-effector pose is accurate the instant the operator moves. For the environment the model is only as good as the scene reconstruction, so contact and interaction remain the hard part, and predictive display helps most with free-space positioning.

The related supervisory control idea (Sheridan) goes further: instead of sending continuous low-level commands across the delay, the operator sends higher-level goals ("move to this waypoint," "close the gripper on that object") that the robot executes autonomously using its own fast local loops, and reports back. This trades away moment-to-moment control for delay tolerance, and it is the only workable mode across very long links. Mars rover operation is the extreme case: operators plan and validate a command sequence in simulation, uplink it once, and the rover executes it over hours with onboard hazard avoidance.

War story: an early undersea manipulation trial ran fine on the bench and then oscillated the moment it went through the vehicle's real video chain. Nothing in the controller had changed. The compressed video feed had added about 250 ms that the bench setup never had, and the operator, driving harder because the task was slow, pushed the hand-eye loop past its now much smaller phase margin. The fix came from a wireframe predictive overlay that gave the operator an instant-response arm to drive, with the laggy video demoted to a correction reference. A better controller would have missed the point.

Bilateral teleoperation and force reflection

Unilateral teleoperation sends commands one way and returns video. Bilateral teleoperation closes a second loop: force from the remote environment is reflected back to the operator's hand, so they feel contact, stiffness, and weight. Feeling the far end transforms contact tasks. Inserting a peg, mating a connector, palpating tissue, or judging a grip all get dramatically easier when the operator feels resistance instead of inferring it from a camera.

The architecture question is which signals cross the link. The standard framing uses the two-port model: the master (operator side) and slave (robot side) are each a port, and the communication channel connects them. Several channel architectures exist.

Position-position (position-error). Both devices exchange positions, and each runs a controller that drives its position toward the other's. If the slave is blocked by the environment, its position lags the master, the position error grows, and that error produces a restoring force the operator feels. It is simple and robust and needs no force sensor, but the felt force is only a proxy (the position error times a stiffness), so free-space motion feels sluggish (the operator drags the slave) and stiff contact feels mushy.

Position-force. The master commands the slave's position, and a force sensor at the slave measures the real contact force and sends it back to be displayed on the master. This gives accurate, crisp force feedback because you reflect the measured force directly, but it is the least stable architecture, because a delay in that direct force loop is exactly what drives the oscillation described earlier, and a stiff environment makes it worse.

Four-channel. Both position and force are exchanged in both directions. Lawrence's four-channel architecture (1993) showed that transmitting all four signals lets you achieve, in principle, perfect transparency, meaning the impedance the operator feels equals the true environment impedance, so a wall feels like a wall and free space feels like nothing. The catch is that perfect transparency and robust stability pull in opposite directions, and the four-channel design assumes you can measure and transmit force cleanly, which delay and noise spoil.

Transparency is the formal name for how faithfully the operator feels the true environment. A perfectly transparent system displays the exact environment impedance: the operator cannot tell they are teleoperating. A perfectly stable but opaque system might feel like pushing through molasses regardless of what the robot touches. Every bilateral design lives on the tradeoff between the two, and delay pushes the achievable frontier the wrong way. This is why surgical masters, which run over a rigid short link with negligible delay, can afford high transparency, while a satellite-hop system cannot.

Passivity: the stability backbone

The reason a force-reflecting loop across a delay explodes, and the reason it can be tamed, both come from energy. The whole field's stability toolkit rests on passivity.

A system is passive if it cannot produce more energy than was put into it, up to the energy it started with. Formally, for a system with input f (force) and output v (velocity) at its port, passivity requires that the energy absorbed over any time never falls below a fixed bound:

integral_0^t  f(tau) * v(tau)  d(tau)  >=  -E_0    for all t

where E_0 is the initial stored energy. A passive system can store and dissipate energy but never generate it. The key theorem is that a feedback interconnection of passive subsystems is stable. A human arm is passive, a physical environment (a wall, tissue, water) is passive, and a well-designed robot controller can be made passive. So if you can also make the communication channel passive, the entire chain of operator, master, channel, slave, and environment is a cascade of passive elements and is guaranteed stable, for any delay and any passive environment.

The problem is that a plain communication delay is not passive. Sending a force one way and a velocity the other way across a delay T can create energy: the delayed signals arrive out of phase, and the channel acts like it is pumping energy into the loop, which is precisely the oscillation. Two families of fix dominate.

Wave variables (scattering transformation). Instead of transmitting force and velocity directly, encode them into wave variables before they cross the link (Niemeyer and Slotine, 1991). Define, with a characteristic impedance b:

u = (b*v + f) / sqrt(2b)      (right-moving wave)
w = (b*v - f) / sqrt(2b)      (left-moving wave)

Transmit u and w across the delay instead of f and v, and decode them back on the far side. The algebra guarantees that the delayed channel, expressed in wave variables, is passive for any constant delay, because the wave encoding makes the transmitted power a clean difference of squared incoming and outgoing waves. You buy unconditional stability against delay. The price is a wave-reflection artifact: fast motions produce reflections that feel like a spring or added drag, and the tuning of b trades stiff-contact fidelity against free-space lightness. Time-varying delay and packet loss need extra reconstruction.

Time-domain passivity control (TDPC). Rather than encode everything, monitor energy at runtime. A passivity observer tracks the net energy flowing through the port in real time, and a passivity controller, a variable damper, switches on to dissipate exactly the excess energy the moment the observer detects the port producing energy it should not (Hannaford and Ryu, 2002). It is adaptive and only intervenes when needed, so it costs less transparency than always-on wave damping, and it handles variable delay gracefully because it reacts to measured energy rather than assuming a fixed T. The cost is that it needs reliable energy measurement and can produce small artifacts when it kicks in.

Rule of thumb: if the delay is fixed and you want a clean stability proof, use wave variables. If the delay is variable or you want to preserve transparency and only pay for stability when contact demands it, use a passivity observer with a passivity controller. Either way, design against the passivity condition, not against how the demo felt.

The deep point is that passivity buys robust stability at the cost of transparency, the same tradeoff as everywhere else in the field. A wave-variable system will never destabilize, and it will also never feel perfectly like the real environment, because the same damping that absorbs the dangerous energy also softens the genuine contact.

Shared and supervised autonomy

Direct teleoperation asks the human to control everything, which is tiring, delay-sensitive, and only as precise as the interface. Shared autonomy splits the work: the human supplies intent and high-level decisions, and the robot supplies fast, precise, locally-closed execution that tolerates delay because it runs onboard. The blends form a spectrum.

Assisted teleoperation and virtual fixtures. The robot constrains or nudges the operator's commands to help. A virtual fixture is a software constraint that acts like a ruler or a jig: a guidance fixture gently pulls the tool toward a desired path, and a forbidden-region fixture stops it from entering a no-go zone (near a nerve, a critical structure, a fragile surface). The operator still drives, and the assistance filters out the parts of their command that would violate the constraint. Surgical systems use forbidden-region fixtures to protect anatomy, and assembly systems use guidance fixtures to speed up insertion.

Intent prediction and blending. The system infers what the operator is trying to do (which object they are reaching for, which of a few likely goals) from the partial trajectory, and blends increasing autonomous assistance as its confidence grows. Early in a reach the human dominates, and as the target becomes clear the robot takes over the fine approach and grasp. This is the mainstream research formulation of assistive teleoperation, and it shines for operators with limited input bandwidth, including assistive robotic arms for people with motor impairments.

Traded control. Control is handed back and forth: the human positions the arm near a bolt, presses a button, and the robot autonomously runs a learned or scripted insert-and-tighten skill, then returns control. Each side does what it is best at, and the handoff points are explicit.

Supervisory control over a fleet. One operator oversees many robots that are mostly autonomous, intervening only when a robot flags uncertainty or gets stuck. This is how warehouse and delivery fleets are run at scale, and how remote-driving companies staff their operations centers: the ratio of robots to humans is the business model, and every increment of autonomy raises it.

Rule of thumb: give the human the decisions that need judgment and context, and give the robot the sub-loops that need speed and precision. The right division of labor beats a better interface, and it is the main lever that makes teleoperation scale past one-human-per-robot.

Shared autonomy is also how you defeat latency without predictive display: if the fast contact loop runs onboard the robot, the human's delayed commands only need to set goals, and the delay stops mattering for the parts of the task that are delay-sensitive.

Teleoperation as a data engine

The newest reason to build good teleoperation is to teach robots. Imitation learning trains a policy to reproduce demonstrated behavior, and the demonstrations have to come from somewhere. Teleoperating the actual robot through the task, and recording the synchronized observations and actions, is the highest-quality source, because the data is collected on the exact embodiment the policy will run on, with the exact sensors, so there is no cross-embodiment gap to bridge.

This is why cheap, high-fidelity teleoperation rigs became a research priority. ALOHA (Zhao et al., Stanford, 2023) is a low-cost bimanual leader-follower setup built specifically to collect fine-manipulation demonstrations, where the operator backdrives two leader arms and two follower arms mimic them while cameras and joint states are logged. Mobile ALOHA extended it to a wheeled base for whole-body tasks like cooking and cleaning. The pattern spread fast, because the demonstration data is the bottleneck for imitation learning and teleoperation is the cheapest way to produce it at quality. VR-based teleoperation is used the same way for humanoids and mobile manipulators, letting a human's tracked hands generate reach-and-manipulate demonstrations.

A few properties make teleoperated data good. It is on-policy for the eventual deployment embodiment, so the action space matches. It captures human strategy including recovery from small errors, which scripted trajectories miss. And it can be scaled by many operators in parallel, turning demonstration collection into an operations problem. The weaknesses are that human teleoperators are inconsistent, that the interface's own limits (no force feedback in most VR rigs) leave gaps in the data, and that collecting enough demonstrations for a robust policy is expensive in human hours, which is the current frontier problem.

The loop closes in a satisfying way: teleoperation collects the data that trains the autonomy that reduces the need for teleoperation, and the residual teleoperation shifts up to supervisory control over the now-more-autonomous fleet. The interface that once only drove the robot now also teaches it, and then supervises it.

Applications

The domains that pay for teleoperation each stress a different part of the problem.

Surgery. Master-slave surgical systems are the most commercially mature teleoperation on Earth. The surgeon sits at a console, views a stereo endoscope, and moves master handles whose motion is scaled down, tremor-filtered, and mapped onto wristed instruments inside the patient. The link is short and rigid, so delay is negligible and the design can chase transparency and precision rather than fighting latency. The da Vinci platform (Intuitive Surgical) is the dominant example, with a large installed base and millions of procedures. Force feedback has historically been limited on these systems, and adding reliable haptics is an active area. The surgical and medical robots guide covers the clinical side in depth. A newer thread is remote surgery over a network (telesurgery), where the link is long and delay returns as the central problem, revived by low-latency 5G and dedicated fiber demonstrations.

Subsea. Work-class ROVs are teleoperated by pilots on a surface vessel through a tether that carries power, video, and control down to depths of several kilometers. The pilot flies the vehicle and works one or two manipulators against currents, poor visibility, and the crushing practicalities of the deep. Delay is usually modest over the tether itself but the compressed video and the difficulty of judging distance and force underwater make it hard, and force feedback and predictive display both help. The underwater robots guide covers the vehicles. Offshore energy and subsea cable work are the economic base.

Space. Astronauts teleoperate manipulators like the station's robotic arms for capture and berthing, and ground operators drive planetary rovers under delays that force supervisory control and command sequencing rather than continuous teleoperation. Orbital servicing and lunar surface operations, where the delay is seconds rather than minutes, are the sweet spot for predictive display and shared autonomy.

Explosive-ordnance disposal and hazardous response. EOD robots are tracked or wheeled platforms with a manipulator and multiple cameras, driven by an operator at standoff distance over a radio or fiber link. The task is inherently human-judgment-bound, so these stay firmly on the teleoperation end of the spectrum, though autonomy creeps in for the driving. Nuclear decommissioning, disaster response, and hazardous-material handling share the profile: the robot is a disposable proxy body.

Remote driving and mobile fleets. Teleoperation is the fallback and enabler for autonomous vehicles and delivery robots. When an autonomous stack gives up, a remote operator takes over to resolve the situation, and the whole operation is designed so that one operator supervises many vehicles. The economics live in that ratio.

Metrics, ergonomics, and failure modes

You evaluate a teleoperation system on more than whether it works in a demo.

Transparency measures how faithfully the operator perceives the remote environment, formalized as the match between displayed impedance and true environment impedance. Stability margin measures how much delay, gain, or environment stiffness the loop tolerates before it oscillates, and passivity gives a conservative guarantee of it. These two trade off directly, and a good design states where on that frontier it chose to sit and why.

Task performance is the practical scorecard: completion time, error and retry rate, and force overshoot on contact. Situational awareness captures whether the operator understands the remote scene, which camera placement, field of view, and depth cues drive, and which a single narrow camera destroys. Operator workload matters because teleoperation is fatiguing, and high workload causes errors and limits how long a shift can run, which is why shared autonomy that offloads sub-tasks is not a luxury.

Fitts's law frames the fundamental speed-accuracy limit of pointing: the time to move to a target of width W at distance D scales with the index of difficulty, MT = a + b * log2(2D / W). Motion scaling, latency, and a jittery interface all inflate the effective difficulty, so a system that shrinks W (through scaling and tremor filtering) or the effective distance speeds every reach.

The failure modes are specific and recurring:

• Latency-induced oscillation, the master failure, when accumulated phase lag crosses the loop's margin. Predictive display or passivity is the fix.
• Loss of situational awareness from a narrow or poorly-placed camera, so the operator collides with something just out of frame. More or better-placed cameras, or a reconstructed 3D view, fix it.
• Kinematic singularities and joint limits on the follower that the operator does not feel coming, so the arm stalls or jerks. Signaling the limit through the interface, or retargeting away from it, helps.
• Operator fatigue and habituation, where a tired operator misses a cue. Workload reduction through autonomy is the real answer.
• Link dropout, where the network stalls mid-motion. The robot must fail safe (stop, hold, or retract) rather than continue the last command blindly, which is a real-time control and safety-design requirement.

Rule of thumb: put at least as much engineering into the return path (video placement, depth cues, force display, latency handling) as into the command path. Operators lose tasks far more often from not perceiving the remote scene than from imprecise commands.

Where it is heading

Three currents are reshaping teleoperation.

The slide toward shared autonomy is accelerating. As onboard perception and control improve, more of the fast loop moves onto the robot and the human moves up to intent and supervision. The operator-to-robot ratio climbs, which is the whole economic point, and direct low-level teleoperation contracts toward the tasks that genuinely need a human hand in the loop.

Teleoperation and learning are fusing. Demonstration collection made teleoperation a first-class part of the autonomy pipeline, and the two now co-evolve: better teleoperation produces better data, which produces better policies, which reduce the teleoperation load to supervision. Expect teleoperation rigs to be designed as data-collection instruments as much as control stations, with force and tactile channels added specifically so the demonstrations capture contact.

Immersion and telepresence keep improving. Higher-fidelity VR and mixed-reality displays, 3D scene reconstruction on the fly, and better wearable haptics push transparency up, and low-latency networks (5G and dedicated fiber) shrink the delay budget for links that used to be hopeless. Full-body avatar teleoperation, where a human's whole body drives a humanoid with force reflected back, is the ambitious end of this, and it borrows directly from exoskeleton hardware.

The durable core will not change. Latency will always add phase lag with no amplitude warning, so predictive local models and supervisory control will always be how you cross a long link. Force reflection will always be a closed loop with a human and an unknown environment inside it, so passivity will always be how you guarantee it does not explode. And the division of labor between human judgment and machine execution will always be the lever that decides how well the system scales. The interfaces and the networks will keep getting better, and those three ideas will still be running underneath.

Frequently asked questions

What is the difference between teleoperation, telerobotics, and telepresence? Teleoperation is operating a machine at a distance. Telerobotics usually implies the remote machine has some autonomy of its own, so the human supervises and directs rather than driving every joint. Telepresence emphasizes the operator's sense of being present at the remote site, through immersive video, audio, and haptics. In practice the terms overlap, and a modern system is often all three at once.

Why does latency make a teleoperation loop unstable? A pure time delay adds phase lag that grows with frequency but leaves signal amplitude unchanged, so the loop gets no amplitude warning as it approaches instability. When the operator drives the hand-eye or force loop harder, the accumulated phase lag eventually turns the feedback positive and the system oscillates. Force-reflecting loops are especially prone because the reflected force and the operator's response chase each other.

What is bilateral teleoperation? It is teleoperation where force is reflected back to the operator, so they feel the remote contact, stiffness, and weight through the hand as well as seeing it. It closes a second loop (force from the environment to the hand) on top of the command loop, which makes contact tasks far easier and also makes the system prone to instability, which is why passivity theory matters.

What is passivity and why does everyone use it? Passivity means a system cannot generate energy, only store and dissipate it. A feedback connection of passive systems is stable, and a human arm and a physical environment are both passive, so if you make the controllers and the communication channel passive too, the whole chain is guaranteed stable for any delay and any passive environment. Wave variables and time-domain passivity control are the two standard ways to make a delayed link passive.

What are wave variables? A change of variables that encodes force and velocity into two waves before transmitting them across the delay, and decodes them on the far side. The encoding makes a delayed channel passive for any constant delay, guaranteeing stability. The cost is wave-reflection artifacts that feel like added springiness or drag, tuned through the characteristic impedance.

What is predictive display? A control-station technique that shows the operator a fast local model or 3D reconstruction of the robot, so they drive against an instantly-responsive prediction while the real robot catches up over the delay. It makes teleoperation usable across seconds of delay by hiding the lag behind an accurate local model, and it works best for free-space positioning where the robot's own kinematics make the prediction exact.

What is shared autonomy? A division of labor where the human supplies high-level intent and the robot supplies fast, precise, delay-tolerant local execution. It ranges from virtual fixtures and assisted teleoperation through traded control to one operator supervising a fleet. It reduces operator workload, defeats latency for the onboard loops, and is the main way teleoperation scales past one human per robot.

Why is teleoperation used to collect training data? Imitation learning needs demonstrations of a task done correctly, and teleoperating the actual robot through the task, while recording synchronized observations and actions, produces the highest-quality demonstrations because they are on the exact embodiment and sensors the policy will use. Leader-follower rigs like ALOHA were built specifically for cheap, high-fidelity demonstration collection.

What is a leader-follower rig? A physical replica of the robot arm that the operator backdrives by hand, with the follower arm copying the joint angles in real time. When the leader and follower are identical, the mapping is exact and needs no learning, which makes it excellent for dexterous manipulation and for recording demonstrations. Adding torque sensing gives the operator force feedback from the far end.

Can teleoperation work across very long delays like Earth to Mars? Not as continuous real-time control. With one-way delays of minutes, operators shift to supervisory control: they plan and validate a sequence of high-level commands, uplink it once, and the robot executes it autonomously with onboard hazard handling, then reports back. Continuous teleoperation is only viable up to delays of roughly a second, and even then it needs predictive display.