Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

删除yaml文件的影响 #79

Open
feifei-111 opened this issue Jun 20, 2023 · 0 comments
Open

删除yaml文件的影响 #79

feifei-111 opened this issue Jun 20, 2023 · 0 comments

Comments

@feifei-111
Copy link
Collaborator

feifei-111 commented Jun 20, 2023

一、问题描述 📚

PaDiff 仓库中有一份 yaml 文件(datas文件夹下),该文件标注了paddle与torch提供的组件权重之间的差异,例如linear的weight需要转置才能对齐。此外,在进行对齐检查时还有一个actions机制(在checker文件夹下),根据当前传入的类型名来选取不同的比较函数。目前这两个机制是独立的,actions实际上只有一种没有起作用。

二、 任务目标 🚀

修改checker模块下,关于模型权重以及梯度的对齐逻辑,剔除yamls的影响,改为使用actions机制(同时可以优化一下get_action()接口)

P.S. 该 yaml 文件在权重初始化功能中仍被使用,权重初始化是一个独立的模块。就对齐工具而言只处理 checker 模块下的部分即可。

三、 TIPS

在获取actions时,需要区分当前的对齐目标:针对模型输出 or 模型权重。不同的对齐目标应当影响返回的 action 类型,为此,可能需要为 dump 下来的文件添加额外信息

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant