Why the Gym version bump in this PR renames four finalize metrics into junk
NVIDIA-NeMo/RL #4264
(Gym checkpoint participants, 1/3 of a stack). RL permalinks pin head ad3c9cb;
Gym permalinks pin the old pin 267305e and the new pin f4fcf8c.
Before
Every token-capture run published four per-method counts, one per way Gym picks a rollout's final model call:
finalize/terminal_selection_{declared,response_id,content,heuristic}_count (plus a fraction for each).
This PR
Moves the Gym submodule forward 145 commits. One of them makes the field those names come from optional (… | None).
Status
Open — not fixed in this PR, or in #4265 or #4266. Two tests in the nemo_gym CI lane fail. Raised in the review.
Nine pages on this stack — this one: #4264: a side effect of the Gym submodule bump.
The RL finalizer does not hard-code the metric names. It reads them from the type of Gym's
RolloutReceipt.terminal_selection field with typing.get_args
(rollout_reassembler.py:449-464).
That works only while the type is a bare Literal. At the old pin it was
(records.py:373 @ 267305e).
At the new pin it is Literal[…] | None
(records.py:407 @ f4fcf8c),
so get_args now returns the whole Literal and NoneType, not the four names.
This file is not in the PR's diff. The bump alone changes what it does.
Same code, two Gym versions
The second failure has the same cause. The new Gym also adds a check that an unfailed receipt must name
its final call and how it was picked
(records.py:421-441).
The test's "empty manifest" receipt sets terminal_model_call_id=None, so it is now rejected as
invalid_receipt before the empty-manifest check runs, and the assert at
test_rollout_reassembler.py:149-153
that expects "empty_manifest" fails.
What would fix it — proposed, not in the PR
Nothing below is implemented. Skip None, then take the args of what is left:
# rollout_reassembler.py, in place of the get_args(...) call at :449-451
annotation = RolloutReceipt.model_fields["terminal_selection"].annotation
# old pin: Literal["declared", …, "heuristic"]# new pin: Literal["declared", …, "heuristic"] | None
terminal_selection_methods = tuple(
method
for arg in get_args(annotation) # new pin → (Literal[...], NoneType)
if arg is not type(None) # drop NoneType → (Literal[...],)
for method in (get_args(arg) or (arg,))
)
# both pins → ('declared', 'response_id', 'content', 'heuristic')
Checked on Python 3.12 against both type shapes. The test at
:149
also needs its "empty" receipt to pass the new Gym check (or the finalizer should test for an
empty manifest before it validates the receipt).
So what. This is not behind the new opt-in flag. Every token-capture run on this Gym pin
stops reporting which method chose each rollout's final call, and the nemo_gym test lane goes red. The fix is a
few lines in rollout_reassembler.py plus one test fixture, and it belongs in this PR, because this PR's bump is what breaks it.