Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added jupyter notebook to make parameter specification and running of the fusion easier #4

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

max-brambach
Copy link
Contributor

@max-brambach max-brambach commented Nov 17, 2020

Changed mvregfus_run.py such that now fusion parameters and paths are specified in a separate jupyter notebook, where they are exported as config file (.json). mvregfus_run.py now only contains functions that run the fusion using this config file.

Added run_fusion.ipynb which contains a description of the fusion parameters, together with some functionality to generate a config file and to run the fusion.
Added some convenience functionality:

  • Parameter filepaths now also takes a string. If so, glob.glob(string + '/*.czi') is used to find files in the specified folder (string).
  • Added parameter generate_file_order. If True, a file_order.txt file is generated in the out_dir. It contains the contents of the filepaths variable. This is useful, since MVRegfus currently renames the output files to mv_000_.... from which it is not possible to reconstruct which file was processed.
  • Parameter registration_pairs can be set to None.
  • If registration_pairs == None, the new parameters n_volumes and n_views are used to generate the registration pairs.

…re specified in a separate jupyter notebook, where they are exported as config file (.json). `mvregfus_run.py` now only contains functions that _run_ the fusion using this config file.

Added `run_fusion.ipynb` which contains a description of the fusion parameters, together with some functionality to generate a config file and to run the fusion.
Added some convenience functionality:
* Parameter `filepaths` now also takes a string. If so, glob.glob(string + '/*.czi') is used to find files in the specified folder (string).
* Added parameter generate_file_order. If True, a `file_order.txt` file is generated in the `out_dir`. It contains the contents of the `filepaths` variable. This is useful, since MVRegfus currently renames the output files to mv_000_.... from which it is not possible to reconstruct which file was processed.
* Parameter `registration_pairs` can be set to None.
* If `registration_pairs == None`, the new parameters `n_volumes` and `n_views` are used to generate the registration pairs.
@max-brambach
Copy link
Contributor Author

Ciao Marvin,
I've recently made a little jupyter notebook to run my fusions a bit more conveniently via remote. Thought, I should share that with you:)
Basically, I've rewritten the mvregfus_run.py file to only contain functions that relate to running the fusion. The parameter handling I've outsourced to the notebook run_fusion.ipynb, that is used to: 1. Specify the parameters, 2. Create a config file, 3. Start the fusion.
The neat thing (at least in my opinion:) ) is that one can use this notebook to start multiple fusions without the need to create copies of the mvregfus_run.py file. Since the config file is saved together with the output files, the parameters stay reproducible.

@max-brambach max-brambach changed the title Changed mvregfus_run.py such that now fusion parameters and paths a… Added jupyter notebook to make parameter specification and running of the fusion easier Nov 17, 2020
@m-albert
Copy link
Owner

That's super cool! Thanks a lot for sharing.

You're completely right that it makes sense to separate execution and configuration. And it's been on my todo list forever 😂.

Exactly as you propose, I think it makes sense to define a parameter dictionary which can then be saved as a json config file that can be (re-)used in future runs. And I like the idea of having a jupyter notebook that guides the setting of the parameters, that's really nice!

Regarding the implementation, probably the run_fusion function should live in a new module such as mvregfus/execution.py and it would make sense that all currently available parameters can be set in the config dict and have associated default parameters (otherwise execution and configuration would still not be properly isolated from each other).

Copy link
Owner

@m-albert m-albert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some comments to the files you shared (also as a reminder to myself!).

Comment on lines +44 to +47
if generate_file_order:
with open(out_dir + '/file_order.txt', 'w') as f:
for item in filepaths:
f.write("%s\n" % item)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely the output filenames should reference the input files.. The easiest way will be to simply change this in mv_graph.py, which I'll do.

Comment on lines +64 to +107
# ----- other parameters ----
# options for DCT image quality metric for fusion
# setting None automatically calculates good values
# size of the cubic volume blocks on which to calc quality
dct_size = None
# size of maximum filter kernel
dct_max_kernel = None
# size of gaussian kernel
dct_gaussian_kernel = None

# weight normalisation parameters
# normalise such that approx. <dct_cumulative_weight_best_views> weight is
# contained in the <dct_how_many_best_views> best views
dct_how_many_best_views = 2
dct_cumulative_weight_best_views = 0.9

# options for weighted Lucy Richardson multi-view deconvolution
# maximum number of iterations
LR_niter = 25 # iters
# convergence criterion
LR_tol = 5e-5 # tol
# gaussian PSF sigmas
LR_sigma_z = 4 # sigma z
LR_sigma_xy = 0.5 # sigma xy


# how to calculate final fusion volume
# 'sample': takes best quality z plane of every view to define the volume
# 'union': takes the union of all view volumes
final_volume_mode = 'union'
# whether to perform an affine chromatic correction
# and which channel to use as reference
perform_chromatic_correction = False
ref_channel_chrom = 0


# ----- derivatives of the set parameters -----
channelss = [channels]*len(filepaths)
reg_channels = [reg_channel] *len(filepaths)
ref_views = [ref_view] *len(filepaths)
registration_pairss = [registration_pairs] *len(filepaths)
view_dict = {i:{'view':i, 'ill':2} for i in list(range(16))}
# where elastix can be found (top folder) ## urrently not used
elastix_dir = '/data/shared/elastix'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these would probably need default values in the function definition (or default values in a json file for default values).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've only selected the ones I'm using normally, but of course it makes sense to have the other parameters available as well.

Comment on lines +32 to +33
n_volumes = parameters['n_volumes']
n_views = parameters['n_views']
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can probably be automatically determined in mv_graph.build_multiview_graph.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants