softplus transform does not properly rescale parameters

[michaeldeistler]

Problem

The ParamTransform transforms params to be unbounded and (as a nice side effect) roughly z-scores them. However, if one an upper or lower bound are passed, then it does not z-score.

Solution

We should generally find a good solution to z-score bounded, unbounded, and half-bounded parameters.

Maybe sth like the following:

tf: Dict = {}

# Some parameters are bounded from both sides.
tf = tf | {"HH_gNa": jx.SigmoidTransform(lower=0.0, upper=2.0)}
tf = tf | {"HH_gK": jx.SigmoidTransform(lower=0.0, upper=1.0)}

# Some parameters are bounded from above.
tf = tf | {"length": jx.SoftplusTransform(upper=100.0, scale=30.0)}

# Some parameters are bounded from below.
tf = tf | {"radius": jx.SoftplusTransform(lower=0.0, scale=6.0)}

# Some parameters are unbounded but should be z-scored.
tf = tf | {"IonotropicSynapse_gS": jx.AffineTransform(scale=5.0, offset=2.0)}

transform = jx.ParamTransform(tf)

# Apply transform
params = net.get_parameters()
opt_params = transform.inverse(params)

[michaeldeistler]

@manuelgloeckler you might be interested in this?