Tell us what you are using TotalSegmentator for
wasserth opened this issue · comments
It would be really helpful for us to now who is using TotalSegmentator and what they are using it for. Therefore, we invite you to comment on this issue: What are you using TotalSegmentator for? What classes are you using? Are there any classes you are missing (we can try to add them in future releases)? Who are you (only if you want to disclose this)?
@wasserth
HI sorry to disappoint, I'm am just an enthusiast amateur in medecine, I have no serious use for this software currently, still it is a potentially life saving advance in medecine.
Noob question:
I want to use the online version and give it as input a CT scan of a leg.
So my question is: what are the supported formats for the CT scan single file?
I know it must be a zip but once uncompressed, can it be a regular image (JPEG, PNG) or does the image needs to conform to a specific format?
E.g can I input this?
https://www.researchgate.net/figure/Patient-Abdominal-CT-scan-Normal-diameter-appendix-with-no-evidence-of-inflammation-of_fig2_323371468/amp
Hi,
the input needs to be a DICOM image (compressed inside of a zip) or a NIFTI image. JPEG or PNG does not work. You can find a example CT file here: https://github.com/wasserth/TotalSegmentator/tree/master/tests/reference_files
Thank you
Hi Jakob @wasserth ,
Thank you for letting us try this great program. We will be using this for lung, lobe, and rib cage segmentation and add these as additional components for our surgical planning.
Our project is to further improve our lung segmentation and analysis platform within 3D Slicer.
Are there any plans to include segments, more detailed airways, or lung vessels in totalsegmentator?
Best
Rudolf
Great to hear that our tool is working well for you. We are currently working on adding airways and vessel segmentation for the lung as well as adding pneumothorax segmentation. What are you meaning by "segments"?
That is great news. If you want us to test any airways or vessels please let me know. We have a well-working airway segmentation tool at hand within 3D Slicer.
In addition, we plan to set up an ML active learning platform using IDC data soon. This will be based on DICOMWeb, 3D Slicer, MONIALabel, and lungmask. If we could also work with TotalSegmentator in this context it would be awesome.
The lung segments (10 on the right, 8 on the left side) would be the next and final substructures of lobes - they are not clearly divided by structures like fissures but usually pyramids of tissues originating from the branches of segment bronchi and pulmonary arteries (not veins). The segments are clinically important because in lung surgery we have more and more switched to segmentectomies in recent years for small lung cancers. For that one needs to know whether or not the tumor is located centrally in one or more of these segments. Segment borders are always approximations on CT and can later be intraoperatively defined by lack of perfusion (if you cut the vessels) or lack of segmental ventilation (if you cut the bronchi).
(wikipedia)
Thanks for the explanation. We have no plans so far to add those segments. It seems that it would be quite a bit of effort to manually draw those segments since the borders are not clearly visible in the image (but are only defined by the vessels).
Understand. How many labeled datasets would you need for a sufficient segment training process with TotalSegmentator?
In the beginning I would roughly need 20 labeled datasets. Then I would train a model and predict the 1200 subjects from the training dataset, then those predictions would need to be corrected where necessary.
Could you provide 20 labeled datasets?
I'll discuss it with my team, but we could probably include this plan in our current 3D Slicer lung segmentation project and provide those complicated labels of the 20 datasets within 6-8 weeks.
On which data would you like to work on here for the initial training? Could we choose the data or would you define them?
Sound good. Ideally you would pick 20 subjects from the TotalSegmentator dataset: https://zenodo.org/record/6802614
I does not really matter which subjects as long as they are from the training set (which subjects belong to which set can be seen in the meta.csv file which comes with the dataset).
I'll contact you about this on Thursday. Which communication do you prefer? Email or are you on Discord?
Email would be best: jakob.wasserthal@usb.ch
What a phenomenal tool, thank you for making it publicly available. I am very interested in some of the additional models you have added after publication of the original paper, especially the vessel segmentation models since there's very little public data out there for this problem!
Will you be posting metrics for these additional models, as you have for the original 104 classes? Was the same training set used? Will you eventually be releasing the additional annotations?
In the future we will also do evaluation for those additional models and release the labels. However, for now this is not very polished yet. For this reason I did not add them in the beginning, but after realizing that those models might already be quite helpful for other people I added them.
Hi, I found the solutioin very interessoing. I am evaluating the code. We want to use it in a project for planning cardiac interventions.
Hi, thank you for this amazing work, I was wondering if you thought about segmenting fat tissue (and differentiating perivisceral fat tissue from sub cutaneous fat tissue). we can consider making some annotation if you have interests in this project. Salim
I am currently working on adding sub cutaneous fat and visceral fat segmentations. But it will take a few more weeks until it is done.
That's cool, thank you for all the job done, really interesting project in which we hope to contribute in the future.
I'm currently working on a pre-clinical project at the University Hospital in Copenhagen, where we are interested in total tumor segmentation. We published an article about using the nnUnet for this task, with great results in improving time needed for physicians to do segmentations. Link to article: https://ejnmmires.springeropen.com/articles/10.1186/s13550-022-00901-2
Now we would like to combine the information of tumor segmentation with their location in the body, and I'll try to implement your organ segmentation model, to gain the knowledge of which organs the tumors are located in. This will allow physicians to quickly create an overview of the total tumor volume, as well as the location and volume of individual lesions.
Thanks for the great work!! It's highly appreciated.
Kristian
I am going to test its results for segmenting ~150 whole-body CT scans of pediatric patients. My current interests are segmenting the lungs and possibly the bronchi and trachea. Any plans to train and validate this tool for segmenting CT scans of pediatric patients?
@owensca I would love to read a blog about such real world uses.
Lung lobe segmentation has recently become a reliable 2-minute process within 3D Slicer with Lung CT Segmenter calls of the TotalSegmentator AI extension.
Consequently, the free and open-source 3D Slicer Lung CT Analyzer extension (V 2.62) now includes lung lobe analysis of pulmonary infiltrates, lung collapse, and emphysema. A special emphysema result table and display are available which will be helpful in lung volume reduction surgery (LVRS) and the surveillance of COPD.
In this COPD case, 35% of the right upper lobe is affected, so this anatomical structure could be a primary target for LVRS.
The pathology is well detected with conventional thresholding in the TotalSegmentator rendered right upper lobe.
Copd.mp4
Very excited to try out this tool, but I haven't yet had the chance to install the code.
Can you please comment on the output format of TotalSegmentator? NIFTII? Volumetric or 2D?
Thank you!
The output files are 3D volumetric nifti files.
This is truly an impressive project, I can't even begin to fathom the amount of time and effort your team must have put into this, AND its all open source! 🙏🎉🎉🎉
On top of my mind, I already know a few PIs that have existing research/cro projects that needed this YESTERDAY. Given that contouring of organs are done manually in many projects (typically oncology, with different organ focuses, liver, kidney...), any contour from CAD would be a productivity booster/tremendous time saver for the image annotation team. Obviously, for clinical work, this will drop the turn around time for report generation/surgical planning, thus bringing quantitative image analysis one huge step towards clinical practice.
Below, I am sharing the dice computed from TotalSegmentator outputs versus manual contours in AMOS2022 (n=357) and Pediatric-CT-SEG datasets (n=~359, age ranging from 0 to 16) (** disclaimer, there could still be bugs in my assessment scripts, further no other datasets have 104 labels, thus i had to merge the 104 labels to the dataset's label set ** )
AMOS2022:
Pediatric-CT-SEG (with and without --fast flag):
Dice of liver in Pediatric-CT-SEG grouped by age (cc @owensca, since you mentioned you are interested using this for pediatric patients):
- scripts to generate the above available here.
- you can view other organs plotted by age here.
Happy New year, congratulations on your amazing work. I am working as a doctor in the craniofacial area; it is already very useful to me that totalsegmentator give automatically the brain volume. I am more interested in the face, which appears as a global area. Is it a chance that it could be segmented in the future? I am not a geek, and I hardly understand the technical names, but I am a good clinician, and I wonder how I could help with the development of this part of the process. The skeletal part is easily done, but the segmentation of other parts: eyeball, masticatory muscles, pharyngeal muscles, would be useful in the clinical research on CFM malformations etc...
I would like to use this software to segment the spine and vertebra out of dicom images, but sadly the software is not working correctly (either with docker or normal build).
I would appreciate any help!
Please open a new issue with your specific error message so we can try to fix your problem.
Happy New year, congratulations on your amazing work. I am working as a doctor in the craniofacial area; it is already very useful to me that totalsegmentator give automatically the brain volume. I am more interested in the face, which appears as a global area. Is it a chance that it could be segmented in the future? I am not a geek, and I hardly understand the technical names, but I am a good clinician, and I wonder how I could help with the development of this part of the process. The skeletal part is easily done, but the segmentation of other parts: eyeball, masticatory muscles, pharyngeal muscles, would be useful in the clinical research on CFM malformations etc...
Thanks for pointing to those structures. We will see if we can add the eyeball, masticatory muscles and pharyngeal muscles in a future release.
If you send me a direct message (jakob.wasserthal@usb.ch) then we can discuss ways how you might be able to help.
Hello,
I was wondering, can we specify the organs that we want segmented using Total Segmentator, or does it always have to perform total body segmentation.
@wasserth and team – this work is really amazing. I applaud you for open-sourcing all of this and doing such a great job of making it easy to install and use. Well done.
We've done some early testing on data from our hospital and the results are very promising. We are developing an open-source AI deployment platform for the NHS in the UK, called AIDE. Link here to recent press release. Would love to show it to you and discuss in the context of TotalSegmentator. I'll drop you an email. Thanks again!
@wasserth and team – thanks for this truly amazing work!
Is there any chance shoulder rotator cuff will be included in future updates of the algorithm?
There are several papers in literature referring to this structure as potentially really relevant on a correct shoulder pathology diagnosis.
We will put the shoulder rotator cuff on our list for future releases.
hey @wasserth and team,
thanks for this amazing tool. Yesterday a radiologist showed me your tool and I am fascinated by the results. I would like to use it for indexing CT images in terms of CBIR. I wonder if there is a way to output only (!) the statistics without writing the actual segmentations. Sure, i can just delete them later, but in terms of disk access this would be a desirable feature.
Have you thought about something like this?
I just added an argument --skip_saving. You have to install from master branch to use it. Statistics calculation will be a lot faster if you use the --fast option.
Hi @wasserth and team,
thanks a lot for your fantastic work. I'm an application manager at Bayer, working with Chinese radiologists on their academic projects. TotalSegmentator is a well-trained tool for segmentation and has helped me a lot in dealing with the weights of different nnunet models. And I've sent you an e-mail inquiring about the licenses. Looking forward to your reply.
Besides, I'm interested in segmenting the liver into healthy and diseased regions based on dual-energy CT images. What do you think of that idea?
Great that our work is helpful for you. So far we have focused on segmenting only anatomical structures. Segmenting pathologys and diseased regions would open another big field. This is further in the future on our roadmap.
Hi @wasserth and team,
Thanks for the time and the expertise dedicated to this project. I would like to use TotalSegmentator in a radiation therapy project as a contouring quality control tool. However, I noticed some common critical structures are not available in the current structure set. Do you have plans for adding prostate, seminal vesicles, breasts, and rectum?
We have these on our agenda, but further in the future. They are not part of the next release.
That is great news. If you want us to test any airways or vessels please let me know. We have a well-working airway segmentation tool at hand within 3D Slicer.
In addition, we plan to set up an ML active learning platform using IDC data soon. This will be based on DICOMWeb, 3D Slicer, MONIALabel, and lungmask. If we could also work with TotalSegmentator in this context it would be awesome.
The lung segments (10 on the right, 8 on the left side) would be the next and final substructures of lobes - they are not clearly divided by structures like fissures but usually pyramids of tissues originating from the branches of segment bronchi and pulmonary arteries (not veins). The segments are clinically important because in lung surgery we have more and more switched to segmentectomies in recent years for small lung cancers. For that one needs to know whether or not the tumor is located centrally in one or more of these segments. Segment borders are always approximations on CT and can later be intraoperatively defined by lack of perfusion (if you cut the vessels) or lack of segmental ventilation (if you cut the bronchi).
(wikipedia)
We would be interested in the 18 lung segmentation model, as well. Did you guys try segment some sample 20 training patients and try this already? Thanks!
We did not further pursue this. So unfortunately we do not have such a model yet.
Excellent software! Any progress on the adipose tissue segmentations? I noticed a post from November 2022.
Hi @wasserth and all,
thank you again for all your efforts, and I highly appreciate your great work. For an ongoing project we are using the vertebral body as a relative localization measure (e.g. at height / z-axis of L2 etc.). In this scenario, only the vertebral body matters but not the arch (consisting of procc. spinosi / procc. transversi).
As a workaround, one can segment the spine (using task=total), and remove the vertebral arch (by setting everything outside of the mask generated by task=vertebrae_body to 0). However, this requires an additional segmentation. I was wondering whether you might consider to create a new task like 'vertebrae_body_detailed'. This might not even require new segmentations but the training data could be generated by fusing the masks as described.
Even if you do not plan to do this, thank you and congratulations to TotalSegementator once again!
Best, Felix
Thanks for your feedback. There is a task vertebrae_body. This only returns the vertebral body. Does that help?
vertebrae_body returns the vertebral bodies of the complete spine, without differentiating between individual vertebrae. as described one can use the vertebrae segmentations from total and combine that with vertebrae body, however this requires two segmentation jobs.
Excatly, you have to combine the results from vertebrae_body and total. This will require two segmentation jobs, but I think this is not too bad.
I am using TotalSegmentator to see my old medical imaging pictures without contrast and investigate my body
I would like to segment lungs,heart, breast/chest wall ctv/tissue and internal mammary nodes ctv for radiotherapy project. Any idea if those ctvs can be done or in future?thanks
Hello,
I am a Master student at University of Applied Science of Zurich (ZHAW) and using Total Segmentator to segment the brain from head-neck-torso CT images and further train nnunet on these images cropped to the ROI (brain) to segment intracranial aneurysms. The tool is working well, thank you very much !
I have a question, when I visualize the images in ITK-Snap before and after applying TS + cropping they look quite different wrt the intensity range, i.e I see less details/nuances of grey. Is it normal ? Thanks in advance for any insights !
Hi team,
We used TotalSegmentator to create a novel dataset called SARAMIS (Simulation Assets for Robotic Assisted and Minimally Invasive Surgery): https://github.com/NMontanaBrown/saramis and https://openreview.net/forum?id=SEU9m9NReo¬eId=SEU9m9NReo. It was presented at NeurIPS 23 Datasets and Benchmarks Track.
Thanks for making such a great tool!
Hello,
I am a Master student at University of Applied Science of Zurich (ZHAW) and using Total Segmentator to segment the brain from head-neck-torso CT images and further train nnunet on these images cropped to the ROI (brain) to segment intracranial aneurysms. The tool is working well, thank you very much !
I have a question, when I visualize the images in ITK-Snap before and after applying TS + cropping they look quite different wrt the intensity range, i.e I see less details/nuances of grey. Is it normal ? Thanks in advance for any insights !
Probably the autowindowing results in differnet contrast. Try to set a fixed window which is the same for both images.
I would like to segment lungs,heart, breast/chest wall ctv/tissue and internal mammary nodes ctv for radiotherapy project. Any idea if those ctvs can be done or in future?thanks
We are working on adding those. But it will take some more time.
Could you please add model weights for vertebrae class to do single part like vertebrae_C1 L1 T1 segementation? Thank you.
I know vertebrae_body could provide a whole spine segementation, but it could not give the subparts of C1 L1 T1 etc.