Textual Inversion training with negative example by altering loss function?

[indoorshark]

Let say I want to train a bad/good hand textual inversion embedding,
can I use both positive and negative example (good/bad hand) instead of using only one?

I imagine preparing pairs of images that are identical, except for the hand (via inpainting manually as data preparation),
instead of calculating loss as loss = mse(target), I calculate it as loss = mse(positive_target) - mse(negative_target). or even loss = mse(positive_target) - max(mse(target) for target in negative_targets)

would that work? would that actually be inefficient and better off just training with positive example?

[indoorshark]

I realize the problem/possible space of "bad hand" might be too large, or perhaps just scope it down to too many fingers for the sake of discussion.