OncoScan AI

Author: Charles R. Clark
Started: February 27, 2024
Finished: April 19, 2024

Table of Contents

• Description
• How to Use
• Documentation
  • Custom CNN Architecture
  • Frontend Webpage
  • Deployment
  • Project Process Architecture
• Where's the Rest?
• For Grading
• References

Description

OncoScan AI is an app which takes as input MRI brain scans and classifies it using one of three different labels: normal (no tumors present), glioma (glioma tumor present), and meningioma (meningioma tumor present). At this time, the classification of pituitary tumors is not supported. It is meant to address the issue of cancer misdiagnosis by serving as a tool both patients and medical professionals can use as a sort of "indicator" or "second opinion." Ultimately, it's my hope that OncoScan AI can prevent at least some cancer misdiagnoses by inspiring the user to either ask for (in the case of a patient) or prescribe (in the case of a medical professional) additional testing and avoiding potentially fatal tumors from not being diagnoses and treated.

How to Use

The "Choose File" button opens the user's file system and allows them to upload an image to the application. The uploaded image file will appear in the center window (where the black brain clipart is in the image above) upon upload, allowing the user to make sure they've selected the correct file before sending it to the backend for classification. The uploaded image must be a JPEG with file extension ".jpeg" or ".jpg," otherwise the application will display an error message in the window on the bottom. However, the image can be rotated by any increment of 90 degrees and the model will still be accurate, as its training data was augmented using rotations, among other transformations.

Once the user has uploaded a JPEG MRI image file and are ready to classify it, they should click the "Predict" button to send the image to the backend for classification. Once there, the image will be passed through a custom Convolutional Neural Network (CNN) constructed using PyTorch; a diagram representation of this custom CNN can be found in the first figure in the Documentation section. The output of this model is passed to a Softmax activation function in order to obtain probabilities for each label, and then the maximum-probability label is selected as the predicted label. Lastly, the backend returns both the predicted label and its associated probability to the frotend, which then displays these results in the bottom window. The images at the bottom of this section show examples in which the model correctly classifies each of the three possible labels using the sample data included in this repository.

OncoScan AI's backend model was trained using M.H. Hashemi's "Crystal Clean: Brain Tumors MRI Dataset" [2] and evaluated using a combined sample set made up of specific data from J. Cheng et al.'s "Brain Tumor Retrieval Image Data" [3] and A. Hamada's "Br35H :: Brain Tumor Detection 2020" [4] data set. This sample set can be found here, should you want to test the app yourself. Although it's not recommended to evaluate the app using the training data, given it already knows it well, you can find the full training set in the backend's separate repo here.

Documentation

Custom CNN Architecture

The diagram below depicts the custom CNN's model architecture. The first main component of the model is made up of two 2D convolution layers, each followed by batch normalization, the ReLU activation function, and then 2D max pooling. Then, the output of this sequence of convolutional layers is passed to a fully-connected component made up of two linear layers with batch normalization and the ReLU activation function between them.

During training and validation, the Cross-Entropy Loss Function with builtin log-softmax activation was used in updating the model's weights. During prediction, the output of the model is passed through a softmax activation function to obtain class probabilities, which are then used to determine the most-likely class; both the selected class and its softmax probability are returned to the frontend.

Frontend Webpage

The frontend was developed using React and CSS. The specifics of how to use it are discussed in the previous section: How to Use.

Deployment

The custom CNN model was connected to the frontend by means of Flask. More specifically, Flask was used to implement a backend API which itself calls the model saved during training to generate predictions for posted image files. This backend API was deployed using Heroku; you can find the backend source code here in its separate GitHub repo.

Project Process Architecture

The overall project architecture can be found in the diagram below. The full training set was separated into training and validation subsets by way of a 80-20 training-validation split. The custom CNN was iteratively developed until I deemed its performance sufficient using these subsets, and then this final model was evaluated (after deployment) using a separate testing/sample subset (see How to Use section). The bottom part of the diagram depicts how the frontend and backend interact with each other: the frontend posts an image file to the backend API, where the custom CNN model generates a prediction and returns it to the frontend.

Where's the Rest?

Although most of OncoScan AI's documentation is here in this README, some backend-specific information can be found here in the backend repo's README.

For Grading

The documentation to be graded can be found here.

References

1. Z. Kira et al. (2024). Georgia Institute of Technology, CS7643: Deep Learning.*
2. M.H. Hashemi, "Crystal Clean: Brain Tumors MRI Dataset," kaggle.com. [Online]. Available: https://www.kaggle.com/datasets/mohammadhossein77/brain-tumors-dataset.
3. J. Cheng et al., "Brain Tumor Retrieval Image Data," figureshare.com. [Online]. Available: https://figshare.com/articles/dataset/brain_tumor_dataset/1512427.
4. A. Hamada, "Br35H :: Brain Tumor Detection 2020," kaggle.com. [Online]. Available: https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection.
5. "Datasets and DataLoaders." pytorch.org. https://pytorch.org/tutorials/beginner/basics/data_tutorial.html.
6. "Optimizing Model Parameters." pytorch.org. https://pytorch.org/tutorials/beginner/basics/optimization_tutorial.html.
7. "Save and Load the Model." pytorch.org. https://pytorch.org/tutorials/beginner/basics/saveloadrun_tutorial.html.
8. "Installation." flask.palletsprojects.com. https://flask.palletsprojects.com/en/2.3.x/installation/.
9. "Deploying PyTorch in Python via a REST API with Flask." pytorch.org. https://pytorch.org/tutorials/intermediate/flask_rest_api_tutorial.html.
10. A. Pandey and A. Wadehra, "File Upload with React & Flask." medium.com. https://medium.com/excited-developers/file-upload-with-react-flask-e115e6f2bf99.
11. M. Grinberg, Create a React + Flask Project. (Feb. 21, 2020). [Online Video]. Available: https://youtu.be/Q2eafQYgglM?si=-t1Uzh1Bj9y0ugiY.
12. A. Neupane, How to Create a Flask + React Project | Python Backend + React Frontend. (Jun. 26, 2021). [Online Video]. Available: https://youtu.be/7LNl2JlZKHA?si=taVGLzTTJYfe4tbb.
13. M. Sharrard, "A Student's Guide to Deploying React and Python Projects." medium.com. https://medium.com/@meaghan.sharrard89/a-students-guide-to-deploying-react-and-python-projects-c5c1fffbe701.
14. Dev Sense, Deploying Deep Learning Flask + React Application to Heroku. (Aug. 15, 2020). [Online Video]. Available: https://youtu.be/TESsM66YRKU?si=5qQfPj6Bj8qExzl2.
15. "Slug Compiler." devcenter.heroku.com. https://devcenter.heroku.com/articles/slug-compiler.
16. A. Singh, "Create-react-app files/folders structure explained." medium.com. https://medium.com/@abesingh1/create-react-app-files-folders-structure-explained-df24770f8562.
17. "Flask-CORS - API Docs." flask-cors.corydolphin.com. https://flask-cors.readthedocs.io/en/latest/api.html.
18. "Deployment of React Application using GitHub Pages." geeksforgeeks.com. https://www.geeksforgeeks.org/deployment-of-react-application-using-github-pages/.
19. Wikimedia Commons, "Red Silhouette - Brain.png Preview." commons.wikimedia.org. https://commons.wikimedia.org/wiki/File:Red_Silhouette_-_Brain.svg.
20. Online Png Tools, Png Color Replacer. Browserling. https://onlinepngtools.com/change-png-color.
21. Online Png Tools, Png Transparency Filler. Browserling. https://onlinepngtools.com/fill-transparent-png-pixelsr.
22. "Converting .mat file extension image to .jpg via python." stackoverflow.com. https://stackoverflow.com/questions/59208896/converting-mat-file-extension-image-to-jpg-via-python

*: This reference is used with permission from CS 7643's instructor and is intentionally vague to maintain academic integrity for future semesters.