Roadmap Request for Feedback

[darcymason]

Open source projects need to occasionally step back and consider where they need to go, and how to get there.

We are looking for input from users of pydicom (the library/Python package) and the other libraries in the pydicom github organization (Python in Dicom), to help guide development of a roadmap.

Any input is welcome, whether it be technical/source code related, community, documentation, examples, tutorials, etc.

Examples questions to prompt discussion:

• What do we do well?
• What can we do better?
• If you chose another library for some project, what was the situation and why was the other library preferred?
• If you chose pydicom or other Python in DICOM libraries for a project, what were some of the key reasons?
• What libraries do you currently use that have pydicom as a dependency?
• Which libraries can we integrate with better, and what is missing for that to happen?
• How welcoming/inclusive/helpful is our community of maintainers and contributors? How can we do better?
• Do you have anything you would like to contribute but have perhaps thought it might not "fit" the current codebases? Please let us know what that was and why you think that piece is important.

How to respond:

• by posting a reply here in this discussion
• email to [email protected], if for any reason you do not want to post publicly

If you are able to, please supply some information about yourself and your use of the tools - e.g. a user in academia or industry, a maintainer/contributor of a specific open source library, etc.

Thanks in advance for any help you can give us in this process!

[pieper]

@darcymason thanks for opening this discussion, and for all your work building this technology and community 👍

We use pydicom in the 3D Slicer project and appreciate the reliable and pythonic access to individual dicom datasets. For our application's use cases, we rely on other code, like gdcm or dcmtk in various C++ wrappers in ITK or CTK to expose higher level functionality that is also exposed in python, e.g. through PythonQt. That is, ITK provides some code to map sets of instances to itk::Image for volumetric image processing, or for use as a vtkImageData for VTK rendering. CTK exposes dcmtk networking and other features. Slicer also has a plug-in architecture for converting dicom instances to higher level constructs like longitudinal collections of studies. Much of this functionality either pre-dated pydicom or developed in parallel.

I don't think we'll ever get to the point where a single python implementation would be able to replace all these use cases, but as Slicer developers we've often talked about factoring out as much of this functionality as possible into python packages that could be more widely used (and debugged and maintained) and perhaps pydicom could be a home for such an effort.

Even being able to agree on how particular dicom study types should be interpreted is complex, e.g. an MR study that includes structural, diffusion, and perfusion series is very difficult to sort out and there are very few tools that can do this reliably. Multiply this by the number of modalities and vendor permutations, plus the derived objects being addressed in highdicom, and, well, it's a very big job to say the least.

I hope this is a reasonable response to your posting. I would definitely like to see pydicom grow and improve. I guess my suggestion would be for pydicom to describe the scope of future development, in the context of these various use cases I've alluded to. That is, will this effort remain at the instance level with some proof of concept examples for higher level use cases or do you see tackling some of these other topics more natively in pydicom?

[darcymason]

Thanks, Steve, that is great input - I knew that pydicom was used somewhat in 3D Slicer but this really helps understand where and why.

will this effort remain at the instance level with some proof of concept examples for higher level use cases

That is a key question we will have to consider. The highdicom folks have also raised that and we hope to try to coordinate with them and other projects to cover the spectrum of use cases in the best way.

[ZviBaratz]

Thank you, @darcymason! I am Zvi, I've been using pydicom extensively throughout my PhD to help my lab build a research database management application for its DICOM data.

I very much enjoyed pydicom's stability and reliable extraction of raw DICOM header information.
To summarize our recent discussion in #1757, here are a few things I think could be beneficial for general use:

• Make native Python values more accessible – while it is technically possible to return IS as int, DS as float, DA as datime.date, etc., having a consistent DataElement method to return these “translated” values would, IMO, be much more convenient and intuitive.
• Creating a Series class – enabling users to read entire directories and summarize metadata.
• Pixel data interpretation – while it will probably be more of a continuous project rather than a single effort, I think it would be great if pydicom (or another DICOM in Python repo) centralized and formalized the interpretation of various data encoding schemes. In neuroimaging, dcm2niix has done an incredible job in doing that for the purposes of conversion to the NIfTI format, but having a more general-purpose implementation in Python could be more accessible to researchers in other domains and in general.

In addition, I'd like to suggest:

• Creating a pydicom TUI – this is another potential project I've been thinking about for some time. It would be great to able to pydicom dicom_archive/ and interactively explore a DICOM archive. Adding Patient and Study classes on top of the Series class could make this a fantastic tool for exploring directories of DICOM files, extracting metadata, organizing them, etc. It might be a little challenging in terms of performance, but I think it's worth experimenting with.
• Refactoring pydicom modules to reduce size and complexity – some modules in the pydicom package are considerably large and complex (e.g., valuerep is over 2,000 lines long). To make the code more accessible, I think it would be beneficial to split it up in a way that is clearer for potential contributors to navigate.

Hope this makes sense, thank you again for all the great work and for your initiative in reaching out to the community.

[darcymason]

Hi Zvi,

Thanks so much for taking the time to give feedback - it really helps us understand what people are using and why, and, of course, helps to understand what future directions are worth pursuing.

I'll only give a brief reply right now, we'll go through your suggestions in more detail, but one question and one thought:

• is a 'TUI' a text UI? just wanted to be sure I know what you mean here.
• on the Series question, I often forget to mention FileSets. The docs linked there start with a DICOMDIR, but one can also add instances and the class can organize them by Series etc (and even write the DICOMDIR and the files out in a consistently named way). I didn't write the class myself, nor have I really used it yet, so I'm not clear on how well that "fits" with what most people want. If someone has a chance to use it perhaps we can see if that can be developed further to meet the Study/Series needs

Again, thanks for your comments. Hopefully we can get a number more and start to build the roadmap.

[ZviBaratz]

@darcymason, thank you!

• Yes. I think textual has recently really pushed the world of Python text-based UI forward, so the difference between TUI and GUI from a user perspective is minimal.
• Thanks for the reference – this definitely could be a possible basis for something like that. However, I think it is important to give users access to something that requires a little less configuration and manual tinkering, at least for the majority of use-cases.

In the context of the TUI application, I thought it would be great to combine textual with django_dicom to create a DB of choice (probably SQLite) in the first iteration of the archive. django_dicom is currently unmaintained, and suffers from some foundational mistakes (perhaps the worst one being the wrong relationship between Patient and Study), but I've been meaning to get back to it soon and fix as much as I can.

[pieper]

@ZviBaratz what would you think about standardizing on the SQLite schema developed for CommonTK? It's been in use for several years and is very robust. We use dcmtk-based C++ code to populate and query it, but It wouldn't be a big task to implement the same features using pydicom.

[ZviBaratz]

Sounds like a great idea. Do you think it would make sense to recreate the schema with Django's ORM? Or better to keep it separate? I am more familiar with Django than I am with SQLAlchemy, peewee, or any other Python ORM solution, so that's my natural tendency.

[pieper]

I'm not familiar with those tools, but I would think it should be possible to develop something compatible. As I understand it those tools would automatically expose a nice API from the schema but I've never tried.

[CPBridge]

Hi @darcymason,

My use of pydicom comes from inter-connected directions:

• My primary role is a researcher in ML applied to medical images (mostly radiology). In this I use pydicom day in day out for "bread and butter" tasks to load and inspect DICOM files, preprocess them, and pass them to AI models for inference.
• As a part of this work, I have a general interest in trying to promote and enable use of standards for communication and display of AI results. This has led to me being a core developer of the highdicom library, which builds entirely on top of pydicom (most highdicom objects are subclasses of pydicom ones). The library's focus is on creation and parsing of particular IODs for communicating image derived information (we don't limit ourselves to AI results but that is our motivation), such as DICOM Segmentations, Structured Reports, presentation states, parametric maps, etc
• I also participate in other open source communities and projects, most notably the monai project, which also develop on top of pydicom extensively to provide general tools for training and deploying AI models in medical imaging with Python.

For the "bread and butter" work, pydicom is great: very stable and mature and dependable.

From my perspective of a highdicom developer, I am hugely grateful to be building on the solid foundation pydicom offers but nonetheless there are a few directions that it would be nice to improve upon. Generally speaking, pydicom focuses on parsing/reading DICOM files and to a lesser extent on creation of new files. Generally speaking this feels like a big gap in the ecosystem right now and the solution will likely extend beyond the pydicom library itself. Hopefully highdicom has a role here at least for some types of file, and probably the actual implementation of IODs for other files (CT, MR, CR) exists at another level but would be hugely useful. However, some parts of the solution probably do exist at the pydicom level. I know there has been work on some of this over the last few years (a few small bits from me), and I may be a bit out of date, but it would be great to reach the point where pydicom comprehensively checks all values satisfy VR constraints, and conveniently compresses/encodes pixel data when creating new files, for example.

Another direction, as we have discussed directly, is that the performance of pydicom for long, nested sequences is quite slow. This is typically not an issue for most use cases, but becomes quite a bottleneck when working with IODs such as structured reports and segmentations as they make use of these types of structures extensively. Especially when these may encode, for example, a sequence containing AI generated annotations of every single cell nucleus in a whole slide microscopy image or a large number of organs/structures in a CT.

Finally, and partly echoing what @ZviBaratz said, there are a bunch of higher level utilities I find myself constantly writing and rewriting and have not yet found a good place for. Things like grouping files into series, collecting attributes on the series level (where this makes sense), sorting according to e.g. spatial position. There are some efforts in the monai community to build a package with these sorts of utils, so maybe that is the location.

Hope this is helpful, and thanks to you and all the pydicom contributors for all your work on this!

Tagging a few other people who may be interested in responding: @MMelQin @hackermd @cgorman @fedorov @vikashg @ericspod @GreySeaWolf @seandoyle

[ericspod]

Hi @darcymason @ZviBaratz @mrbean-bremen,
Thanks for the good discussion. We definitely should have a good think about what to do to pull in a lot of the experience and knowledge that people have accumulated into one place that can be maintained. Not in pydicom directly but yes perhaps in a separate repo. It just seems that everyone picks up techniques and little bits of info on how non-standard things are done in DICOM by various vendors, having one place to park this and save people from relearning/reimplementing it would be helpful.

Maintainability is definitely a problem and I've seen a number of "contrib" directories or subprojects have issues keeping code current. Perhaps instead of code we need guides on best practices and learned experiences then let people write their own simple code to solve the problem. These would be simple recipes that would be easy to write or adapt to one's own situation but still capture that "folklore".

[ZviBaratz]

Ok, I created an elementary awesome-dicom repo, but there is surely a lot to add and improve. I'll do my best to update and maintain it, any help would be appreciated 🙏🏼

[mrbean-bremen]

This already looks quite good, thanks! 👏🏼

[darcymason]

This already looks quite good, thanks! 👏🏼

Yes, very nice!

[ericspod]

@ZviBaratz that's a great resource already!

[vikashg]

Hi all,
First of all Thanks for maintaining and writing the pydicom library. I will echo @CPBridge's words in conveying our gratitude. I will try to answer some of the questions from OP here as follows

• What do we do well?

I think pydicom does an excellent job in providing access to the dicom tags and this is what I use mostly for to see what dicom tags are available and read dicom files. I think I do not have any complains in reading 2D dicom images. I just use the dcmread function. I am not sure if there is a dicom series reader which can read 3D images. Correct me if I am wrong but dcmread is only used to read an individual dicom file right?
I like the stop_before_pixels functionality in dcmread. Especially if we are searching for specific tags in a whole bunch of images and reading the whole image can be expensive.

• What can we do better?

If it is not already implemented maybe implementing a reader for a 3D dicom reader. I know we can read individual images and sort it which we usually do but I was wondering if there is any reason for not implementing an equivalent of SimpleITK's GDCMSeriesReader
I will again echo @ericspod 's words that feature creep can be a thing so it is not essential to have that functionality. But maybe why not ?

• If you chose another library for some project, what was the situation and why was the other library preferred?

I often used SimpleITK when I have to read 3D dicom images just because you can read a dicom series in 3 lines. Only when the output is not as desired I go back to read individual dicom files and sort it using pydicom

• If you chose pydicom or other Python in DICOM libraries for a project, what were some of the key reasons?

The main reason was that the dicom data was non standard and so pydicom gave me access to the dicom tags which I could use to sort the images and build by workflow

• What libraries do you currently use that have pydicom as a dependency?

MONAI, highdicom

• Which libraries can we integrate with better, and what is missing for that to happen?

I was wondering if it is possible to write something so that we can directly load the dicom data (or pixel array) directly in the GPU. Because reading dicom images during training is quite time-consuming so I convert dicom to numpy and then use the numpy for training.

One of the things that can also be incorporated is the ability to collect (make a list or something similar) all dicom images with some common dicom tag. For example, all the dicom images from a study are contained in a folder and I want to collect the same slice from multiple series(es), there can be functionality to give a list of dicom images based on a dicom tag-based search criteria.

I always use Open CV to burn some text on the pixel array and save it using pydicom. There can be a function, which always writes some text on the image, instead of giving a location where to write the text, it will just detect background (which is not difficult to detect based on intensity thresholding) and just write the text there.

• How welcoming/inclusive/helpful is our community of maintainers and contributors? How can we do better?

I was always able to find anything I was looking for on stack overflow or github so the community was very helpful.

• Do you have anything you would like to contribute but have perhaps thought it might not "fit" the current codebases? Please let us know what that was and why you think that piece is important.

I wrote some tools for reading and writing DICOM RT and I will be happy to contribute that code to the repo if you feel that it can go there.

Hope this helps. Thanks once again.

[mrbean-bremen]

@hackermd - I just wanted to disagree, arguing that the series should in this case only represent a DICOM series - having in mind that pydicom should reflect the basic DICOM terms. Loading images would not be a part of this in my mind.

However, after thinking about the possible use cases I only see the case of a DICOMDIR, which is already handled, while in any other case we either don't know which images comprise a series, or we do know and need no special handling for this. So, I now tend to agree with you that we may not need a series reader, except for the already existing FileSet.

Would like to hear @darcymason's opinion on this - I may have just overlooked some use cases.

[dclunie]

I agree that it is important not to misuse DICOM terminology - there is no question that a "Series" is set of things with the same SeriesInstanceUID attribute value, no more and no less, and DICOM specifies various rules for what can be in it.

However, there is obviously value in identifying the equivalence of a set of things that may be represented differently depending on the IOD or SOP Class (or other scenario) - the example of a set of reconstructed slides through space that may be single instances in one series or one multi-frame instance (or split into multiple instances of a concatenation) being semantically the same thing.

We introduced the term "FrameSet" in the IHE Basic Image Review (BIR) integration profile for this, in order to dictate the same behavior in the user interface regardless of the DICOM SOP Class used or the number of Instances involved.

See also com.pixelmed.dicom.FrameSet and com.pixelmed.dicom.SetOfFrameSets for some of my crude experiments with this concept.

Note that a FrameSet might span multiple DICOM Series.

Obviously, a FrameSet is different from a "File-set" (the latter also being a standard DICOM term), and it would be unfortunate if pydicom's FileSet class were any different than what DICOM defines.

[mrbean-bremen]

Thank you - a frame set sounds like a helpful (and still sufficiently low-level) construct that may be used, though I'm not entirely sure where it would fit in pydicom. It is certainly usefull as a base for subsequent sorting and combining images into 3D/4D data sets.

Note that a FrameSet might span multiple DICOM Series.

This is not what the supplement says in my understanding:

there will never be fewer FrameSets than Series (i.e., Series will not be combined)

Obviously, a FrameSet is different from a "File-set" (the latter also being a standard DICOM term), and it would be unfortunate if pydicom's FileSet class were any different than what DICOM defines.

Agreed. A FileSet currently indeed represents a "File-set" and shall continue to do so.

[darcymason]

Thanks for the comments, @dclunie.

Agreed. A FileSet currently indeed represents a "File-set" and shall continue to do so.

I also agree here. I was just thinking some of the behavior could be borrowed or wrapped - separating files into Patient, Study, Series, etc.

o, I now tend to agree with you that we may not need a series reader, except for the already existing FileSet.

Would like to hear @darcymason's opinion on this - I may have just overlooked some use cases.

Well, I still think something very simple would be good - I've written the code in the past (okay, as I think about it, maybe not quite so simple) that can be pointed to a directory (without a DICOMDIR file) and read all files, separating them into classes by Patient, Study and Series by the patient id, Study, Series UIDs. But again, this is much like FileSet already does, at least it prints out in that format. Probably the different viewpoints arise (as @mrbean-bremen noted at one point) is in handling image (or other data) - I would still see this as just collecting individual datasets for the user, but being a neutral container as much as possible, not cross-linking data elements for any purpose (much like pydicom usually sticks to) -- really, effectively just providing dicts of datasets sharing a patient ID or DICOM UID. Pydicom's Fileset is not very well known, I think, so I'm just trying to find a way to make it fit into the pythonic/pydicom style a little better and promote it more.

I think we will have to keep discussing this. I'd like to review the standard again, pydicom's Fileset, and the Frameset examples @dclunie pointed to (just for contrast, not to consider implementing), to get this all straight. And as maintainers of pydicom, we will eventually produce a roadmap, which again the community can comment on. But that process will have to include some reasoning and clear principles to stick to.

[mrbean-bremen]

I've written the code in the past (okay, as I think about it, maybe not quite so simple) that can be pointed to a directory (without a DICOMDIR file) and read all files, separating them into classes by Patient, Study and Series by the patient id, Study, Series UIDs.

That actually makes sense, if you include Patient and Study here, and it is a common use case to collect the data and for example feed part of it into a database with Patient, Study and Series tables.
I was only arguing against having solely a Series class that would be used as some kind of image container - what I understood others were talking about (I may have misunderstood that part).

[seandoyle]

Hi - I've mostly been using pydicom in industry or in industry-like applications at hospitals. I've also had a lot of experience introducing new users to DICOM.

In general - I think that pydicom is excellent. It just works; it's documented; there is a community of people that help. I agree with all of the praise in the comments above :-).

The main issues I see with it are largely independent of pydicom itself. I think that the three largest ones that I run into are

1. DICOM files are modified but not in a way consistent with the standard which leads to unfortunate results (updating demographics but not the DICOM UIDs, updating the pixel format without changing the pixel representation). I'm not sure that there is a fix for this at the level of pydicom. The root issue from my point of view is that the standard is difficult for new users to follow - particularly users who are focused other domains (image processing, ...). It's a complex standard.
2. Objects which have not been validated. Perhaps some text could be added to https://pydicom.github.io/pydicom/stable/old/best_practices.html that described different validation tools and libraries. I often use more than one tool (dciodfy, DicomSRValidator.sh). I see that there is a pydicom validator https://github.com/pydicom/dicom-validator - I've never tried it but I will soon.
3. People don't know where to start. I've found that a great teaching tool for showing people how to build a DICOM object is to take an existing DICOM object similar to what they want to build and run it through your codify utility. Some users find it easier to read code than the DICOM spec - I think reading through the generated code and the standard together give users a good way to see how objects should be put together.

[darcymason]

Thanks, @seandoyle for taking the time to give this feedback. I'm really glad to hear of the connection to teaching/helping new users.

Your points about validation are great feedback - I really like the idea of giving documentation pointers to validators. In addition to the Best Practices section, probably even as a warning in the introductory materials, and maybe a mention in the writing routines (dcmwrite or save_as).