eRLT · Action-Relevant Token Routing

Vision-Language-Action (VLA) models provide strong behavioral priors for robotic manipulation, yet efficiently adapting them to downstream tasks remains challenging. Recent work addresses this challenge by adapting frozen VLAs through online reinforcement learning (RL), whose sample efficiency depends on the quality of the state representation used by the actor and critic. Existing methods construct such representations either with VLA-independent visual encoders or through fixed compression of internal VLA representations. Neither design explicitly extracts the task-specific action-relevant VLA features most useful for downstream action refinement and action-value estimation, therefore limiting sample efficiency.

To address this limitation, we introduce eRLT, which constructs an effective state representation by routing task-specific action-relevant information across both tokens and layers of the frozen VLA. Specifically, learned routing tokens dynamically aggregate visual-language features at multiple depths, while a lightweight layer router combines these summaries into a fixed-dimensional RL token. The routing module is initialized using expert demonstrations to capture features predictive of expert actions and then refined using critic feedback from online interactions for action-value estimation.

Across seven LIBERO and RoboTwin tasks, eRLT improves mean normalized learning-curve AUC by up to 23.7% over representative baselines. Real-robot experiments on USB connector insertion and motherboard ribbon-cable insertion further show AUC improvements of 108.9% and 46.7%, respectively, over the strongest baseline.

Overview

The right features make online learning more efficient.

eRLT routes task-specific, action-relevant information across the tokens and layers of a frozen VLA. The resulting RL token gives a lightweight actor-critic a more useful state representation for action refinement and value estimation.

Motivation: fixed layer and token choices can miss useful information, while eRLT routes action-relevant features across both dimensions. — Useful information is distributed across VLA tokens and layers. eRLT learns what to retain for the downstream task.

+23.7%Mean AUC · 7 simulation tasksrelative to RLT*

+108.9%AUC · USB insertionrelative to RLT

+46.7%AUC · Ribbon-cable insertionrelative to RLT

AUC measures average success over a fixed interaction budget. Higher AUC means learning successful behavior earlier and sustaining it.

Method

A compact RL token, informed by actions and refined through experience.

eRLT architecture showing token and layer routing, offline expert-action prediction, and online critic feedback.

Route useful information

Learned routing tokens aggregate visual-language features at multiple depths. A layer router combines these summaries into one RL token.

Initialize with actions

Expert-action prediction on existing demonstrations gives the representation an action-relevant starting point.

Refine with experience

Online critic feedback adapts the routing module for action-value estimation. The pretrained VLA stays frozen throughout.

Real-world Experiments

Learning precise alignment and contact-rich insertion.

We evaluate on two precision manipulation tasks using the left arm of an RB-Y1 robot and two wrist-mounted cameras. Within each task, all adapted policies share the frozen VLA and online RL procedure; only the state representation changes.

Experimental protocol & comparison methods

Frozen VLA is the task-specific policy before online adaptation. DSRL* uses an independent visual encoder; the asterisk denotes the use of this representation in the shared RL pipeline. RLT uses the original RLT representation. eRLT learns action-relevant token routing.

USB insertion uses 80 trajectories and ribbon-cable insertion uses 90, including warm-up and human-assisted episodes. All methods follow the same human-in-the-loop protocol. AUC summarizes the full learning curve. Adapted-policy success rates use the final 10-episode window; the frozen reference uses 10 evaluation trials. Edited video counters describe the shown rollouts, not the complete statistical evaluation.

Simulation Results

Consistent improvements across tasks and robot embodiments.

Across four LIBERO tasks and three RoboTwin tasks, eRLT achieves the strongest mean normalized AUC. It outperforms RLT* on all seven tasks and DSRL* on six of seven.

Four LIBERO tasks and three RoboTwin manipulation tasks.

Mean normalized learning-curve AUC across 7 tasks ↑ Higher is better
Method	Mean AUC	State representation
DSRL*	0.585	VLA-independent visual encoder
RLT*	0.506	Fixed compression of final-layer VLA features
eRLT Ours	0.626	Action-relevant routing across tokens and layers

Success-rate learning curves on three LIBERO and three RoboTwin tasks, comparing eRLT with RLT and DSRL representations. — Learning curves on six representative tasks. Dark curves show smoothed trends; light curves show raw evaluations. Original paper colors are retained.

View results for all seven tasks

Task	DSRL*	RLT*	eRLT
LIBERO · Drawer Opening	0.856	0.709	0.881
LIBERO · Bowl-to-Plate	0.801	0.707	0.818
LIBERO · Cabinet-Bowl-to-Plate	0.130	0.171	0.247
LIBERO · Two-Moka-Pots	0.244	0.207	0.264
RoboTwin · Adjust Bottle	0.804	0.797	0.897
RoboTwin · Move Can–Pot	0.608	0.461	0.523
RoboTwin · Place Container Plate	0.649	0.490	0.752

What Matters in eRLT?

Action-relevant routing and a useful starting point.

Component ablations · Bowl-to-Plate
Variant	AUC ↑
eRLT	0.818
Without offline initialization	0.762
Without online adaptation	0.803
Without token-wise routing	0.742
Without layer-wise routing	0.797

Each component contributes.

Removing token-wise routing causes the largest drop in AUC. An action-predictive initialization also provides a strong starting point, while layer-wise routing and online adaptation offer complementary gains.

A compact representation is enough.

Increasing the number of routing tokens does not consistently improve performance. eRLT uses one routing token by default.

Citation

@article{huang2026erlt,
  title={eRLT: Efficient VLA Reinforcement Learning via Action-Relevant Token Routing},
  author={Huang, Dehao and Liu, Jianbang and Gao, Jianpan and Tang, Chao and Cen, Zilang and Dan, Zedong and Wang, Jiaheng and Li, Tingguang and Wang, Yue and Zhang, Hong},
  journal={arXiv preprint arXiv:2610.00913},
  year={2026}
}