[Feature Request] Support for Function Calling Fine-tuning

[jc-ryan]

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

Function calling has become a mainstream approach for fine-tuning LLMs to enhance their tool usage capabilities. I have two related questions:

1. Is there any plan to add support for function calling fine-tuning in this repository?
2. If I want to implement function calling fine-tuning myself, what modifications would be needed in the codebase?

Any clarification would be greatly appreciated. Thank you!

Solution

No response

Alternatives

No response

Additional context

No response

[Gaiejj]

I am very sorry for the late reply. In fact, we did not include the fine-tuning of function calling in our development plan. However, I carefully read your requirements and found that this can be easily integrated into the align-anything framework. You can refer to the implementation of align_anything/configs/format_dataset.py and the getting started documentation, and achieve this functionality by customizing the template for the function calling fine-tuning dataset.

Sure, you can also specify the model and dataset you require; we are willing to support you and provide runnable scripts directly.