Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What's meaning that notation x_s,mix_t? #1

Open
racyjelly opened this issue Mar 2, 2023 · 1 comment
Open

What's meaning that notation x_s,mix_t? #1

racyjelly opened this issue Mar 2, 2023 · 1 comment

Comments

@racyjelly
Copy link

image

Hello, I'm Jennie who is studying Diffusion.
I was curious about the number 6~8 line code.
Why you make x_s,mix_t? also, I think you regard x_s,mix_t as x_0.
I'm not good enough to understand to this sampling code.
Please make sure this code! Thank you

@fallenshock
Copy link
Owner

Hi Jenny, the lines 6-8 can be interpreted as follows:
line 6: takes the noisy and blurry image $x^s_t$ from the previous step and outputs a denoised yet still blurry image $x^{s, mix}_t$.
line 7: removes the blurry part of the image which is approximated by upsampling the resulting image from the previous scale (line 14 in alg.2) depending on the gamma schedule (see paper) to obtain a denoised and deblurred $\hat{x}^s_0$.
line 8: mixes back the blurry image from the previous scale with the clean image to obtain a slightly less blurry image
x^{s,mix}_t-1.
In the next lines 9-10 we proceed to re-add noise to the mixed image to prepare for the next iteration.

The reason for the blurring is as follows. During sampling, we obtain a clean image at the end of the reverse diffusion process in each scale. This clean image is upsampled and contaminated by noise, to serve as the initial image for the reverse diffusion of the next scale. Now, when we up-sample the image between scales, we naturally add blur. This implies that our model needs to remove not only noise, but also blur. That’s why during the forward process we also gradually blur the image in addition to adding noise (for every scale s>0). This forces the model to learn to remove both noise and blur from the initial image.
This is also explained thoroughly in App. D.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants