Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug with the sequential domain reduction #476

Closed
bwkong opened this issue May 10, 2024 · 9 comments
Closed

Bug with the sequential domain reduction #476

bwkong opened this issue May 10, 2024 · 9 comments

Comments

@bwkong
Copy link

bwkong commented May 10, 2024

Bug1
In domain_reduction.py, line 105-108:
for i, entry in enumerate(new_bounds):
if entry[0] > entry[1]:
new_bounds[i, 0] = entry[1]
new_bounds[i, 1] = entry[0]
I guess it aims to swap the values of two entries but it does not work in that way. When new_bounds[i, 0] is rewritten, entry[0] will also be rewritten and new_bounds[i, 1] cannot be designated the appropriate value.

Bug2
new_bounds in the function transform in domain_reduction.py is never trimmed:
line 147: self._trim(new_bounds, self.original_bounds) is doing nothing.
Instead, it should be like: new_bounds = self._trim(new_bounds, self.original_bounds)
These have caused new bounds not within the original bounds.

@till-m
Copy link
Member

till-m commented May 11, 2024

Hey @bwkong,

thanks for reporting this bug. I think this might be fixed already via #441. We have just not pushed a release since then.

@pawlissimo
Copy link

Hey @till-m ,
I faced with the same issue few months ago. Do you have any ideas on when it could be released (since it seems to be fixed more than 1 year ago)?
Thx

@till-m
Copy link
Member

till-m commented Jun 24, 2024

Hey @pawlissimo,

my apologies, I know this is annoying. I'm looking into getting this done over the next two weeks (possibly actually today, but no promises).

In the meantime, you can always install directly from github. If you don't know how to do that, let me know how you usually install packages and I can help you.

@till-m
Copy link
Member

till-m commented Jun 24, 2024

Hey @pawlissimo @bwkong,

I've pushed release 1.5.0 (big thanks to @Panaetius). Could you download the latest version and let me know if it fixes the problem for you?

@pawlissimo
Copy link

Hi @till-m ,
Thanks for so fast reply! I think, you saved a lot of time not only for me;)

Btw, just updated my local conda env and verified that the latest bayesian-optimization=1.5.0 installed. But looks like the main issue still exists or I'm doing something wrong.

In shorts:

  1. I use SequentialDomainReductionTransformer and subscribed JSONLogger to store my results.
  2. Since there is a bounds_transformer (there're no dumps for it in JSONLogger's log), I dump its content as pickle.dump(bounds_transformer, 'path/to/bounds_transformer.log')
  3. At the next day I create new loaded_optimizer and load_logs(loaded_optimizer, logs='path/to/optimizer.log.json) and loaded_bounds_transformer = pickle.load(f).
  4. If I call maximize() for both original and loaded optimizers, then I see, that last one starts optimization without any care about its own loaded bounds_transformer So, no way to make a "checkpoint" and continue optimization later?.
    I've also tried to loaded_bounds_transformer._update(loaded_optimizer.space)

Hope, it makes sense and you could point me out on the mistake in my approach

P.S. I carefully read whole documentation and followed most of examples., but was not able to find the proper way.
P.P.S. I'm so interesting in this fix because each single function evaluation costs approx. 1..5$ and may take up to 1 hour (!!!) And there are ~20 func. args. So, trying to prepare proper framework to continue my optimization after sporadic failures.

@till-m
Copy link
Member

till-m commented Jun 25, 2024

Hi @pawlissimo

This sounds like a slightly different problem to me, in all likelihood related to the serialization of the optimizer/transformer. If I am reading correctly, the problem is not related to the trim or clamping logic of the bounds transformer, but the transformer doesn't work at all?

If yes, could you please open a separate issue, ideally with a code snippet that allows us to reproduce the problem?

@pawlissimo
Copy link

pawlissimo commented Jun 25, 2024

It could be a separate issue or slightly related to this one (based on my understanding ot the code and logic).
I've opened a new bug request. Could you pls take a look - #486 ?
Thx a lot!

@pawlissimo
Copy link

Hi @till-m ,
looks like I forgot to mention you yesterday in my response.
I've opened a new bug request. Could you pls take a look - #486 ?
Thx a lot!

@bwheelz36
Copy link
Collaborator

Closing in facor of #468

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants