suggest your papers here!

[lucidrains]

1. Trans-Unet https://arxiv.org/abs/2102.04306

2. U^2-Net https://arxiv.org/abs/2005.09007

3. Restormer https://arxiv.org/abs/2111.09881

[lucidrains]

it will also come complete with conditioning, so it can be used straightforwardly in DDPMs

[MicPie]

"iUNets - Fully invertible U-Nets with Learnable Up- and Downsampling", https://arxiv.org/abs/2005.05220, https://github.com/cetmann/iunets

“Mask2Former - Masked-attention Mask Transformer for Universal Image Segmentation", https://bowenc0221.github.io/mask2former/

Looking forward to the code! :-)

[lucidrains]

Hi Micpie! 👋

Is Maskformer considered a unet?? I guess it does have downsamples and then upsample layers

[MicPie]

Hi Phil,

yeah, Mask2Former is a little bit different from the standard UNet setup, which very likely makes it tricky to implement it in one abstraction (but I wanted to bring it up, because of the good results they achieved).

This is another interesting approach:
"CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation", https://arxiv.org/abs/2103.03024, https://github.com/YtongXie/CoTr

I'm pretty excited of mixing convs + attn for UNets due to https://github.com/xxxnell/how-do-vits-work

I also recently rewrote your UNet code from https://github.com/lucidrains/denoising-diffusion-pytorch/blob/master/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py#L180 to use it as "normal" UNet, but then had to postpone testing it due to other work coming up, but this is why I'm looking forward to your x-unet even more. :-)

Cheers,
Michael

[lucidrains]

@MicPie ok, let me figure out the main inductive bias behind maskformer and get back to you :)

[lucidrains]

@MicPie are you using any u-nets for your AI medical imaging work?

[MicPie]

Yeah, I would say UNets are still the standard there.
If I remember it correctly one of the nnUnet authors (https://github.com/MIC-DKFZ/nnUNet) once mentioned that a well trained UNet is hard to beat. (But training them properly can be of course tricky. ;-) )

[lucidrains]

@MicPie yea, even the transformers side seems to converge on the same structure (https://arxiv.org/abs/2110.13711 and https://arxiv.org/abs/2202.10890)

i'll see what i can do to push things forward :)

edit: you are so well read, as usual!

[lucidrains]

U-Net_and_Its_Variants_for_Medical_Image_Segmentation_A_Review_of_Theory_and_Applications.pdf

👀

[MicPie]

Thank you for sharing, I didn't know the "Hierarchical Perceiver" paper and the one about UNet variants. 🙏

UNet++ looks really fancy, the corresponding paper has also some nice figures with the setup: https://arxiv.org/abs/1807.10165
(There is also a PyTorch version out, but with a lot of hard coded connections in the forward: https://github.com/hellopipu/unet_plus/blob/master/model.py)

[lucidrains]

https://arxiv.org/abs/1906.06148

[vztu]

Hi @lucidrains, please check out our [CVPR2022-Oral] "MAXIM: Multi-Axis MLP for Image Processing", a multi-stage UNet with (standalone) multi-axis MLP layers. It is concurrent work with Restormer with similar performances on multiple low-level tasks.

We provide Jax code here: https://github.com/google-research/maxim
Paper: https://arxiv.org/abs/2201.02973

[vvvm23]

Imagen (needs no introduction) proposes some interesting improvements as a so-called "Efficient U-Net". Might be worth checking out Appendix B.1 for a summary.

[gunesevitan]

UNet++: A Nested U-Net Architecture for Medical Image Segmentation
Attention U-Net: Learning Where to Look for the Pancreas
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
UNETR: Transformers for 3D Medical Image Segmentation
Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

[lucidrains]

Imagen (needs no introduction) proposes some interesting improvements as a so-called "Efficient U-Net". Might be worth checking out Appendix B.1 for a summary.

yup, i have it built over at https://github.com/lucidrains/imagen-pytorch and @nousr has successfully used it for his medical segmentation school work

[lucidrains]

@vvvm23 i'll see if i can port over the logic in a clean manner

[lucidrains]

UNet++: A Nested U-Net Architecture for Medical Image Segmentation Attention U-Net: Learning Where to Look for the Pancreas TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation UNETR: Transformers for 3D Medical Image Segmentation Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

great list! 🙏

[a11to1n3]

I think this paper is worth looking at

• UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise
  Perspective with Transformer

[lalalune]

https://github.com/3dim-paper/website
Talks about X-Unet, based on imagen