NVIDIA-NeMo/RL #4245
Β· all RL links pin head fc77827, Gym links pin 267305e.
_ng_task_index the dataset row had, or none. The same prompt seen again (next epoch, or a re-run) sent the same number again."{task}-{rollout}". Dotted words have a short
explanation: hover or tap them.Gym builds the key in
maybe_rollout_id_from_run_body,
and its docstring warns: βReused task and rollout indices produce a repeated capture keyβ
(L48).
With observability on, Gym files /verify calls under that key
(server_utils.py:514).
The Harbor agent names its job folder t{task}-r{rollout} and returns any finished trial
already in it
(app.py:139,
L518-L540).
So a task number that repeats makes a new rollout look like an old one.
Where the fix lands: RL only.
rollout_manager.py hands out the numbers;
single_controller.py,
grpo.py,
rollout_checkpoint.py and
setup.py save and restore the counter.
No Gym change: Gym already uses whatever number RL sends.
@@ rollout_manager.py L983-L986 Β· new method on AsyncNemoGymRolloutImpl + def _take_nemo_gym_task_index(self) -> int: + task_index = self._next_nemo_gym_task_index + self._next_nemo_gym_task_index += 1 + return task_index @@ rollout_manager.py L1013-L1016 Β· run_rollout β one number per attempt rollout_inputs = self._build_inputs( input_sample, + task_index=self._take_nemo_gym_task_index(), rollout_ids=rollout_ids, @@ rollout_manager.py L1090-L1094 Β· _build_inputs β every sibling row shares it template_row: dict = copy.deepcopy(input_sample["extra_env_info"]) # type: ignore + # One fresh ID per run_rollout attempt, shared by all sibling rows. + template_row[NEMO_GYM_TASK_INDEX_KEY] = task_index @@ single_controller.py L4538-L4541 Β· _save_checkpoint_impl β save next value save_state.sampler_dispatch_index = self._sampler.dispatch_index + save_state.next_nemo_gym_task_index = ( + self._rollout_manager.get_next_nemo_gym_task_index() + ) @@ single_controller.py L425-L428 Β· __init__ β restore it on resume self._rollout_manager = actor_args.rollout_manager + self._rollout_manager.set_next_nemo_gym_task_index( + actor_args.save_state.next_nemo_gym_task_index + )
In order: each call to run_rollout takes the next number (40, then 53 in the
example). _build_inputs writes it on a copy of the row, so the dataset sample keeps its
own 7. A retry of failed rows inside one attempt reuses the built rows, so it keeps 40
(tested at
test_rollout_generation_failures.py).
The saved value, 54, also goes into rollout-only snapshots
(L4116).
Old checkpoints load as 0, and old snapshots keep the trainer's value.
The older async path uses the same key with a different meaning: there,
_ng_task_index is the prompt's place in the dataloader
(trajectory_collector.py:617).
The two paths never share rows, so nothing breaks. The review asks for the comment in
_build_inputs to say why Single Controller needs a fresh number, and that it is not a
dataloader position.
π‘ review comment β rollout_manager.py:1091