train_on_responses_only

[Zuozhuo]

I saw the following code snippet in your qwen2.5 fine-tuning tutorial:

trainer = train_on_responses_only(
    trainer,
    instruction_part = "<|im_start|>user\n",
    response_part = "<|im_start|>assistant\n",
)

Here, trainer is an instance of SFTTrainer.

My question is, when I directly use the instantiated SFTTrainer to execute trainer.predict, the predictions in the result contains normal logits. However, after processing trainer with train_on_responses_only and then executing trainer.predict, I was surprised to find that the predictions in the result is an empty tuple.

Why does this happen? How can I make it return logits as expected?

[danielhanchen]

Sorry on the delay - try doing at the very beginning before importing unsloth:

import os
os.environ["UNSLOTH_RETURN_LOGITS"] = "1"