prototype-multi-modality-assistant repository on the MIDA organization.
MIDA is a Research Project that deals with the use of a recently proposed technique in literature: Deep Convolutional Neural Networks (CNN). These deep networks will incorporate information from several different modes by a User Interface (UI) implemented based on our Prototype Breast Screening repository. The hereby repository is a mirror of our Prototype Breast Screening repository which is an Open Source Project with the goal to deliver an example of web based medical imaging platform for the breast cancer diagnosis. We also have several demos to see in our YouTube Channel, please follow us.
We kindly ask scientific works and studies that make use of the repository to cite it in their associated publications. Similarly, we ask open-source and closed-source works that make use of the repository to warn us about this use.
You can cite our work using the following BibTeX entry:
@article{CALISTO2021102607,
title = {Introduction of Human-Centric AI Assistant to Aid Radiologists for Multimodal Breast Image Classification},
journal = {International Journal of Human-Computer Studies},
pages = {102607},
year = {2021},
issn = {1071-5819},
doi = {https://doi.org/10.1016/j.ijhcs.2021.102607},
url = {https://www.sciencedirect.com/science/article/pii/S1071581921000252},
author = {Francisco Maria Calisto and Carlos Santiago and Nuno Nunes and Jacinto C. Nascimento},
keywords = {Human-Computer Interaction, Artificial Intelligence, Healthcare, Medical Imaging, Breast Cancer},
abstract = {In this research, we take an HCI perspective on the opportunities provided by AI techniques in medical imaging, focusing on workflow efficiency and quality, preventing errors and variability of diagnosis in Breast Cancer. Starting from a holistic understanding of the clinical context, we developed BreastScreening to support Multimodality and integrate AI techniques (using a deep neural network to support automatic and reliable classification) in the medical diagnosis workflow. This was assessed by using a significant number of clinical settings and radiologists. Here we present: i) user study findings of 45 physicians comprising nine clinical institutions; ii) list of design recommendations for visualization to support breast screening radiomics; iii) evaluation results of a proof-of-concept BreastScreening prototype for two conditions Current (without AI assistant) and AI-Assisted; and iv) evidence from the impact of a Multimodality and AI-Assisted strategy in diagnosing and severity classification of lesions. The above strategies will allow us to conclude about the behaviour of clinicians when an AI module is present in a diagnostic system. This behaviour will have a direct impact in the clinicians workflow that is thoroughly addressed herein. Our results show a high level of acceptance of AI techniques from radiologists and point to a significant reduction of cognitive workload and improvement in diagnosis execution.}
}
First of all, you will need NodeJS installed locally on your machine. This project needs both npm and http-server dependencies to install and run the core project. If you do not have those installed please follow the INSTALL instructions.
The following assumes you will be using a git version control for this repository, storing thanks to GitHub. First, Download and Install git. Our system needs to be integrated with WADO-URI servers, DICOMWeb servers or any HTTP based server that returns a DICOM P10 instances. We suggest you to use an Orthanc server, since it is a simple and powerful standalone DICOM server by providing a RESTful API.
You can download a latest version or you can use our own sample of an Orthanc version with our examples of patient images. The instructions to use our solution are as follows.
1.1.1. Clone the DICOM Server repository:
git clone git@github.com:MIMBCD-UI/dicom-server.git
1.1.2. Go inside the project folder:
cd dicom-server/
1.2.1. Install the local dependencies:
npm install
1.2.2. You can now Run the project, just follow the next section.
1.3.1. Inside the project folder:
cd dicom-server
1.3.2. Start the DICOM Server (Orthanc) for MacOS:
npm run start:multi
OR
cd ..
cd dicom-server/orthancAndPluginsOSX.stable/
./startOrthanc.command
NOTE: If you are not using MacOS, for instance, if you are using Windows or Debian you have a documentation for that. Just follow the Windows or Debian documentations. You also have several other options.
1.3.3. Open the link:
localhost:8248
NOTE: If you need some help see the Demo.
Our main server uses NodeJS and has several dependencies. For the following steps you must have already installed both NodeJS and npm in your machine.
2.1.1. Clone the project repository:
git clone git@github.com:MIMBCD-UI/prototype-heatmap.git
2.1.2. Go inside the project folder:
cd prototype-heatmap/
2.2.1. Install the local dependencies:
npm install
2.2.2. You can now Run the project, just follow the next section.
2.3.1. Inside the project folder:
cd prototype-heatmap/
2.3.2. If you have already run the DICOM Server on a previous section, please jump to the 2.3.3. point, otherwise do:
npm run dicom-server
2.3.3. Run the code:
npm run build:multi
2.3.4. Start the project:
npm run start:multi
2.3.5. Open the link:
localhost:8286/src/public/index.html
Access-Control-Allow-Origin is a CORS (Cross-Origin Resource Sharing) header. If you want to know How does Access-Control-Allow-Origin header work? follow the link.
- To deal with the CORS issue it is necessary to open Google Chrome with
--disable-web-securityflag on:
open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
- Or install the CORS plugin for Google Chrome.
For more information about the MIDA Project just follow the link. Pieces of information about details of this repository are also in a wiki. This prototype was developed using several libraries and dependencies. Despite that all libraries had their importance and supported the development, one of it was of chief importance. The CornerstoneJS library and secondary libraries, respectively, are supporting this prototype. We Acknowledge all people involved in the path.
Copyright © 2017 Instituto Superior Técnico (IST)
The prototype-heatmap repository is distributed under the terms of both Academic License and Commercial License, for academic and commercial purpose, respectively. For more information regarding the License of the hereby repository, just follow both ACADEMIC and COMMERCIAL files.
A special thanks to Chris Hafey, the propelling person of CornerstoneJS, who also developed the cornerstoneDemo. Not forgetting the three supporters of the CornerstoneJS library, Aloïs Dreyfus, Danny Brown and Erik Ziegler. We also would like to give a special thanks to Erik Ziegler who support several issues during this path.