Non-default argument follows default argument

[DubiousCactus]

I am running more and more into this TypeError when defining several experiments that override the hydra_defaults and add arguments to Partials using make_config. Here is an example:

Experiment = builds(
    launch_experiment,
    populate_full_signature=True,
    hydra_defaults=[
        "_self_",
        {"model": "lnp"},
        {"optimizer": "sgd"},
        {"scheduler": "step"},
        {"training": "default"},
        {"dataset": "contactpose"},
        {"trainer": "base"},
    ],
    data_loader=pbuilds(TaskLoader, builds_bases=(DataloaderConf,)),
)
...
experiment_store = store(group="experiment", package="_global_")
experiment_store(
    make_config(
        hydra_defaults=[
            "_self_",
        ],
        model=dict(
            y_dim=21 * 3, x_dim=store["dataset", "contactpose"].obj_ptcld_size * 3
        ),  # TODO: Better way to handle this ? It breaks if I override the dataset config.
        dataset=dict(
            pair=("object", "hand"),
            scaling="hand_object",
            use_mano=False,
            positive_unit_cube=True,
        ),
        bases=(Experiment,),
    ),
    name="lnp_keypoints_obj_to_hand",
)
experiment_store(
    make_config(
        hydra_defaults=[
            "_self_",
            {"override /trainer": "mano"}, 
            {"override /model": "megalnp"},
        ],
        bases=(Experiment,),
        model=dict(  # 18 MANO pose parameters and a 4x4 Sim(3) group transformation
            mano_dim=18, x_dim=store["dataset", "contactpose"].obj_ptcld_size * 3
        ),
        dataset=dict(
            pair=("object", "hand"),
            scaling="hand_object",
            use_mano=True,
            positive_unit_cube=False,
        ),
    ),
    name="megalnp_mano_obj_to_hand",
)

Now for some reason, I get this error: TypeError: non-default argument 'trainer' follows default argument.
I can usually fix it by changing the order of the non-default argument in the launch_experiment function such that it does not follow a default argument. The problem is that there can be a situation where I'm in a deadlock due to other configurations overriding a different argument. It is very confusing; is there a better way to do this? Or can you improve make_config such that it handles argument ordering automatically?

Thanks for the help

[jgbos]

@DubiousCactus Interesting question. Unfortunately I cannot figure out what is going on with the information you provided. Is it possible to create a simple example recreating the error?

At first glance I would say it's odd to use builds to create your experiment config. Not saying it's wrong (many ways to do this), but it might help to update so that the Experiment is a standard dataclass using make_config.

[rsokl]

I'd have to see more of the code to repro and to try to figure out a solution. One user fix could be to specify hydra_zen.MISSING as the default values for those fields in the dataclass. Or maybe None (@jgbos thoughts?).

We may be able to automate the above solution for users.

[jgbos]

@rsokl yep. @DubiousCactus might already be setting trainer correctly in launch_experiments, but we don't see the whole config because of the use of populate_full_signature.

[DubiousCactus]

Hi @jgbos and @rsokl , did you get a chance to look at my example?

[jgbos]

Apologies, I've been swamped at work. I'll see if I can take a look at your example today.

[DubiousCactus]

Thanks for getting back to me so quickly. I am working on a PyTorch template with hydra-zen, so you can test the code if you clone my repo and run ./train.py +experiment=experiment_a. It works fine until you uncomment this line.

I followed your tutorial that shows how to use builds to configure multiple experiments. It's very convenient for me to be able to populate defaults with store entries so that I can override config groups in the CLI with something like dataset=my_dataset. Can that be done with make_config?

I don't see how I would set hydra_zen.MISSING fields for some config groups that are Partials. Also, I want them to have default values, but I want to be able to specify a new config that overrides these defaults.

If you have suggestions for improving my architecture I would love to hear them, thanks!

[jgbos]

@DubiousCactus ok. I can confirm that there is something weird going on here. The solution is to use hydra_zen.MISSING as None does not work. Here is my working example:

from hydra_zen import zen, store, MISSING, make_config


def launch_experiment(trainer, model):
    print(trainer, model)


store(dict(foo=1), group="trainer", name="default")
store(dict(foo=3), group="trainer", name="mano")

store(dict(bar=2), group="model", name="default")
store(dict(bar=4), group="model", name="megalnp")

Experiment = make_config(
    hydra_defaults=["_self_", {"trainer": "default"}, {"model": "default"}],
    trainer=MISSING,   # normally I would use None, but that does not work with experiment configs
    model=MISSING,  # normally I would use None, but that does not work with experiment configs
)
store(Experiment, name="config")

experiment_store = store(group="experiment", package="_global_")
experiment_store(
    make_config(
        hydra_defaults=[
            "_self_",
            {"override /trainer": "mano"},
            {"override /model": "megalnp"},
        ],
        bases=(Experiment,),
    ),
    name="exp1",
)

if __name__ == "__main__":
    store.add_to_hydra_store()
    zen(launch_experiment).hydra_main(
        config_path=None, config_name="config", version_base="1.1"
    )

$ python myapp.py
{'foo': 1} {'bar': 2}

$ python myapp.py +experiment=exp1
{'foo': 3} {'bar': 4}

Note, because you use defaults setting MISSING does not cause any issues as the configuration gets set by the defaults.

[rsokl]

Thanks for looking into this @jgbos .

In the (hopefully near) future I am hoping that make_config/builds can handle this automatically by looking at the base classes and filling in MISSING when necessary.

[DubiousCactus]

Thank you! I'll keep an eye out for when this automatic feature is implemented.

[rsokl]

@DubiousCactus sorry for the late follow up on this. I finally found time to dig in. Bad news and good news:

Bad news: I am not going to make hydra-zen handle this automatically. It is really really hairy and will probably break other, more common patterns.

Good news: This problem goes away if enable keyword-only fields in your configs (dataclasses)! Note that this requires Python 3.10+.

from hydra_zen import make_config

A = make_config("a", "b")

B = make_config(a=1, bases=(A,))  # raises TypeError due to non-default arg following default arg

C = make_config(a=1, bases=(A,), zen_dataclass={'kw_only': True})  # OK!

So I think you can just do

Experiment = builds(
    launch_experiment,
    populate_full_signature=True,
    hydra_defaults=[
        "_self_",
        {"model": "lnp"},
        {"optimizer": "sgd"},
        {"scheduler": "step"},
        {"training": "default"},
        {"dataset": "contactpose"},
        {"trainer": "base"},
    ],
    data_loader=pbuilds(TaskLoader, builds_bases=(DataloaderConf,)),
    zen_dataclass={'kw_only': True}
)

[DubiousCactus]

That's great. thanks for looking into this!