DALI 2022 roadmap
JanuszL opened this issue · comments
The following represents a high-level overview of our 2022 plan. You should be aware that this roadmap may change at any time and the order below does not reflect any type of priority.
We strongly encourage you to comment on our roadmap and provide us feedback on this issue here.
Some of the items mentioned below are the continuation of the 2021 effort (#2978)
Improving Usability:
- eager mode - introducing DALI operators callable as standalone entities to simplify debugging, prototyping, and improve adoption: #3648, #3734, #4016
- conditional execution - ability to conditionally apply each operation, providing auto Augmentor style capabilities - #3701, #4359, #4405, #4358
- intra-pipeline batch size variability - providing an ability to change batch size from operator to operator inside the execution graph
- support for Hopper GPU architecture - #4308
Extending input format support:
Extending support of formats and containers with variable frame rate videos: #3615, #3668, #4184, #4296, #4302, #4351, #4354, #4327, #4424
Image decoding operators with support for the following higher dynamic ranges - #4223
Performance:
- reducing memory consumption by utilizing fast, pool based, memory allocator: #3728, #3667, #3670, #3678, #3754, #3759
- operators performance optimizations:
New transformations:
We are constantly extending the set of operations supported by DALI. Currently, this section lists the most notable additions to our areas of interest that we plan to do this year. This list is not exhaustive and we plan on expanding the set of operators as the needs or requests arise.
- new transformations for general data processing
- new transformations for image processing
- new transformations for video processing - extending support of existing operator for video sequences (including temporal - per frame parametrized, augmentations)
Hi @JanuszL I was interested in a more seamless integration between DALI and torchvision for better end to end model training and inference time
Relevant PRs
- In DALI: #2569
- In torch/vision: pytorch/vision#608
In particular we don't necessarily need to integrate everything including the data loader but at the very least I think the accelerated image decoding and specialized preprocessing kernels will be of huge value gated behind another vision backend https://github.com/pytorch/vision#image-backend
The integration would probably look similar to the one made with accimage
- https://github.com/pytorch/vision/blob/main/torchvision/transforms/functional.py
- https://github.com/pytorch/vision/blob/main/torchvision/transforms/functional_pil.py
- https://github.com/pytorch/vision/blob/main/torchvision/datasets/folder.py
I'm guessing torchaudio facebookresearch/mmf (multimodal) would also be similar
Hi @msaroufim,
Thank you for your feedback regarding our 2022 roadmap.
It would be nice to accelerate existing TorchVision pipelines, however, I'm not convinced if the way that DALI works can be combined in the suggested fashion. DALI relies on the processing graph, we plan to extract some of its operators into callable entities but this will not match the performance of the pipeline execution model. Also, it will lead to less efficient GPU memory utilization.
I would wait until this effort is at least partially completed and let the TorchVision community experiment with it and see what works best.
Hi, @JanuszL,
Is there a timeline for a stable version of DALI with support of python-3.10, as I saw this warning as:
Warning: DALI support for Python 3.10 is experimental and some functionalities may not work.
deprecation_warning("DALI support for Python 3.10 is experimental and some functionalities "
Your answer and guide will be appreciated!
Hi @songyuc,
This warning is there mostly because we don't have a full test coverage for Python 3.10, although it should work fine.
I cannot commit to a particular timeline but we hope to do it sooner than latter.
Hi @csheaff,
Thank you for reaching out. We don't have a short-term plan to support DICOM. As I understand the usual workflow is the conversion from DICOM to NumPy (which includes offline preprocessing, like normalization), and then the NumPy files are used for the training. The conversion is done only once while NumPy files are reused multiple times. That is why we have this item low on our priority list.
Still, can you describe your workflow so we can reprioritize it if needed?
Thanks for the response @JanuszL. I've just discovered DALI and I'm sort of new to MLOps, so perhaps my ideas are off here. If you have better suggestions on workflow I'm happy to hear them.
My priority is low latency from end-to-end in medical imaging applications. Triton is great for inference, but i'm looking for ways to handle data loading and pre-processing as well on the GPU, to be used in tandem with an inference model served by Triton.
It's true that there can be a heavy amount of pre-processing and metadata extraction with DICOMs. The metadata extraction for purposes other than the main processing pipeline will likely always be there. Perhaps this means it doesn't make sense for DALI to handle DICOMs directly.
As for the normalization, my understanding is that DALI would be able to handle such tasks. Perhaps I'm mistaken.
Hi @csheaff,
As for the normalization, my understanding is that DALI would be able to handle such tasks.
DALI is mostly useful for online augmentation, and in general things, you need to do each iteration of your training/inference process. In the case of DICOM the conversion to NumPy can be done only once as a part of offline preparation, doing it every iteration wouldn't yield any value and would be wasteful from the resource point of view.
Nevertheless, I agree that it would be nice to seemingly handle that in DALI.
It looks like dali at least partially supports dicom by way of nvjpeg2k now and a bit of a hack. eg: https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371534 notebook here - https://www.kaggle.com/code/tivfrvqhs5/decode-jpeg2000-dicom-with-dali?scriptVersionId=113466193
Kaggle isn't releasing a decoded dataset for the code competition so folks are having to decode on each run (train about 10 minutes, inference about 7 hrs!). The dali speedup is likely to be a huge win, but as noted it's only for dcmfile.file_meta.TransferSyntaxUID .90 standard, and .70 makes up about 1/2 of the other images. Heres the breakdown:
1.2.840.10008.1.2.4.70 29519
1.2.840.10008.1.2.4.90 25187
Any thoughts on how we might be able to get .70 in there as well? Are there fundamental limitations as to why it can't be supported?
Hi @blazespinnaker,
I'm glad to see that the community made that work. I think that it should be possible to use the external operator to extract DICOM data and pass it directly to the decoder instead of writing it to the disk.
Can you tell me more about .70 standard? How it is encoded (it may just happen that DALI doesn't support such a format yet)?
Looks like .70 is a rarely used Jpeg Lossless standard .. https://crnl.readthedocs.io/jpeg_formats/index.html
If I had to guess, I'd say the question is whether nvjpeg can support it.
Maybe if we can get the pydicom folks to help support a pipeline to nvjpeg / nvjpeg2000 this can be done.
I just checked with the nvJPEG team and this format is not supported yet. In this case, DALI should fall back to the CPU libjpeg-turbo decoder. I'm sorry but I don't think we can do much now.
Hmm, I think libjpeg might be a better fall back for 1.2.840.10008.1.2.4.70? libjpeg-turbo does not yet support lossless jpeg I believe.
Hi @blazespinnaker,
Currently, we fully rely on libjpeg-turbo for JPEG decoding. If it fails DALI cannot decode it.
What we can do it to try to fallback to the OpenCV (although I believe it may use libjpeg-turbo as well) here https://github.com/NVIDIA/DALI/blob/main/dali/image/jpeg.cc#L80. We would be more than happy to accept PR adding such functionality.
Please check #4578 for the 2023 roadmap.