research paper on brain tumor

• Open access
• Published: 23 January 2023

MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques

• Soheila Saeedi 1 ,
• Sorayya Rezayi 1 ,
• Hamidreza Keshavarz 2 &
• Sharareh R. Niakan Kalhori 1 , 3  

BMC Medical Informatics and Decision Making volume  23 , Article number:  16 ( 2023 ) Cite this article

41k Accesses

104 Citations

1 Altmetric

Metrics details

Detecting brain tumors in their early stages is crucial. Brain tumors are classified by biopsy, which can only be performed through definitive brain surgery. Computational intelligence-oriented techniques can help physicians identify and classify brain tumors. Herein, we proposed two deep learning methods and several machine learning approaches for diagnosing three types of tumor, i.e., glioma, meningioma, and pituitary gland tumors, as well as healthy brains without tumors, using magnetic resonance brain images to enable physicians to detect with high accuracy tumors in early stages.

Materials and Methods

A dataset containing 3264 Magnetic Resonance Imaging (MRI) brain images comprising images of glioma, meningioma, pituitary gland tumors, and healthy brains were used in this study. First, preprocessing and augmentation algorithms were applied to MRI brain images. Next, we developed a new 2D Convolutional Neural Network (CNN) and a convolutional auto-encoder network, both of which were already trained by our assigned hyperparameters. Then 2D CNN includes several convolution layers; all layers in this hierarchical network have a 2*2 kernel function. This network consists of eight convolutional and four pooling layers, and after all convolution layers, batch-normalization layers were applied. The modified auto-encoder network includes a convolutional auto-encoder network and a convolutional network for classification that uses the last output encoder layer of the first part. Furthermore, six machine-learning techniques that were applied to classify brain tumors were also compared in this study.

The training accuracy of the proposed 2D CNN and that of the proposed auto-encoder network were found to be 96.47% and 95.63%, respectively. The average recall values for the 2D CNN and auto-encoder networks were 95% and 94%, respectively. The areas under the ROC curve for both networks were 0.99 or 1. Among applied machine learning methods, Multilayer Perceptron (MLP) (28%) and K-Nearest Neighbors (KNN) (86%) achieved the lowest and highest accuracy rates, respectively. Statistical tests showed a significant difference between the means of the two methods developed in this study and several machine learning methods ( p -value < 0.05).

The present study shows that the proposed 2D CNN has optimal accuracy in classifying brain tumors. Comparing the performance of various CNNs and machine learning methods in diagnosing three types of brain tumors revealed that the 2D CNN achieved exemplary performance and optimal execution time without latency. This proposed network is less complex than the auto-encoder network and can be employed by radiologists and physicians in clinical systems for brain tumor detection.

Peer Review reports

Introduction

In medical terms, tumors are known as malignant or benign neoplasms, of which there are more than 200 diverse varieties that may affect humans [ 1 ]. According to the American Cancer Society, a brain tumor is a severe disease in which irregular brain tissue growth impairs brain function. The National Brain Tumor Foundation (NBTF) reported that the number of people who have lost their lives due to brain tumors has increased by 300% in the last three decades [ 2 ]. Brain tumors can lead to death if left untreated [ 3 ]. The complexity of brain tumors poses challenges for healthcare providers in diagnosing and caring for affected patients. Early detection of brain tumors and initiation of treatment play vital roles in the survival rate of these patients [ 4 ]. Brain tumor biopsy is not as easy as biopsy of other parts of the body, as it must be performed with surgery. Therefore, the need for another method for accurate diagnosis without surgery is crucial. Magnetic Resonance Imaging (MRI) is the best and most commonly used option for diagnosing brain tumors [ 5 ].

Recent advances in machine learning, particularly in deep learning, have led to the identification and classification of medical imaging patterns. Successes in this area include the possibility of retrieving and extracting knowledge from data instead of learning from experts or scientific texts. Machine learning is rapidly becoming a helpful tool for improving performance in various medical applications in various fields, including the prognosis and diagnosis of diseases, identification of molecular and cellular structures, tissue segmentation, and the classification of images [ 6 , 7 , 8 ]. In image processing, the most successful techniques currently used are Convolutional Neural Networks (CNNs), as they have many layers and high diagnostic accuracy if the number of input images is high [ 9 , 10 ]. Autoencoders are an unsupervised learning method in which neural networks are leveraged for representation learning. Remarkably, various deep learning and machine learning algorithms have been used to identify tumors (such as lung tumors) and detect cardiovascular stenosis. Moreover, performance evaluations have shown that they have high diagnostic accuracy [ 11 , 12 , 13 , 14 ].

Many studies have been conducted on the detection of brain tumors by various methods and models [ 5 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. However, some of these studies have had a number of limitations, such as a lack of a performance comparison between the proposed model and traditional machine learning methods [ 5 , 22 , 23 ]. The proposed model in one study required complex computations [ 24 ]. The majority of relevant studies have provided models for classifying three types of brain tumors without including healthy subjects [ 22 , 23 , 24 , 25 ].

Speaking scientifically, tumor diagnosis by medical images is erroneous and depends heavily on the radiologist's experience. Because of widespread pathology variation and the possible fatigue of human specialists, researchers and physicians can benefit from computer-assisted interventions [ 6 ], and computational intelligence-oriented techniques can assist physicians in identifying and classifying brain tumors [ 5 ]. Machine learning approaches, especially deep learning, can also play a vital role in the analysis, segmentation, and classification of cancer images, especially brain tumors [ 26 ]. Furthermore, the use of such methods paves the way for accurate and error-free identification of tumors to recognize and distinguish them from other similar diseases. In the present study, we have tried to propose models that consider the suggestions and limitations presented in studies and suggest suitable solutions for them. Eight modeling methods have been compared to determine whether a significant difference exists between these methods in terms of performance.

Contributions of this work

The significant contributions of this work are detailed below:

Our networks are performed on an extensive dataset of 3264 T1-weighted contrast-enhanced MRI images, which are desirable for the training and testing phases.

The internal architecture of the modified 2D CNN and convolutional auto-encoder neural network are adjusted in terms of the number of layers, how the layers are positioned next to each other, the type of parameters and hyperparameters, and their values that can be varied to fine-tune our models to enhance accuracy.

Extracted essential features are utilized to classify three types of brain tumors and healthy brains (no tumor) by 2D CNN, auto-encoder network, and six common machine learning techniques.

In the modified 2D CNN, several convolution layers are considered; all layers in this hierarchical network have a 2*2 kernel function. This network consists of eight convolutional layers and four pooling layers; after all convolution layers, batch-normalization layers were applied. The training process was accomplished over 100 training epochs, and the batch size was 16. Each epoch last 7 s.

The auto-encoder network includes a convolutional auto-encoder network and a convolutional network for classification that uses the last output encoder layer of the first part. The encoder part has a convolutional layer of 32-filter length, two continuous convolutional layers with a filter length of 128, and two continuous convolutional layers with a filter length of 64. The decoding part of the network consists of a convolutional layer with a filter length of 32, two continuous convolutional layers with a filter length of 64, and two continuous convolutional layers with a filter length of 128 as well as a convolutional layer with a filter length of 128. For all convolution layers, the 2*2 kernel function was applied.

The developed networks achieved optimal accuracy of approximately 95% to 96%, and areas under the receiver operating characteristics curves (AUROC) are 0.99 or 1. Performance analysis proposes a renovation of our proposed techniques by comparing related papers.

One-way ANOVA for three parameters of precision, recall, and F-measure in eight modeling methods showed a statistically significant difference between the methods ( p -value < 0.001).

Our architectures attain competitive undertakings analogized with other state-of-the-art approaches on the MRI dataset and demonstrate a heightened generalization.

Related works

In recent years, many methodologies for classifying brain tumors by MRI images have been developed (Table 1 ).

A study conducted by Badža and Barjaktarovic´ in 2020 used a CNN to classify glioma, meningioma, and pituitary tumors. The network architecture applied in this study consisted of an input layer, two blocks “A,” two blocks “B,” a classification block, and an output layer, with 22 layers in total. Network performance was evaluated by employing the k-fold cross-validation method. The best value for the tenfold cross-validation method, which was obtained in this study, was 96.56%. The image dataset used in this study comprised 3064 T1-weighted contrast-enhanced MRI images from the Nanfang Hospital, General Hospital, and Tianjin Medical University in China [ 5 ].

In 2018 [ 24 ] developed capsule algorithms networks (DCNet) and diverse capsule networks (DCNet++). DCNet essentially adds a deeper convolutional network, leading to learning distinctive feature maps. DCNet++ uses a hierarchical architecture for learning, which makes it more efficient for learning complex data. They used a dataset comprising 3064 MRI images of 233 brain tumor patients for classification and considered only images of three types of brain tumors; a dataset of healthy people was not considered for classification. The DCNet model was developed by changing the eight initial convolutional layers to four layers with 16 kernels and was trained with eightfold cross-validation. The accuracy of the DCNet algorithm test was 93.04%, and the accuracy of the DCNet++ algorithm was 95.03%.

Gumaei et al. [ 23 ] introduced an automated approach to assist radiologists and physicians in identifying different types of brain tumors. The study was conducted in three steps: brain image preprocessing, brain feature extraction, and brain tumor classification. In the preprocessing step, brain images were converted into intensity brain images in the range of [0, 1], using a min–max normalization rule. In the next step, the PCA-NGIST method (a combination of normalized GIST descriptor with PCA) was adopted to extract features from MRI images. In the final step, Regularized Extreme Learning Machine (RELM) classification was applied to identify and classify the tumor types. The dataset provided by Cheng was used by the researchers in their study and consisted of 3064 MRI images from 233 patients divided into two subsets, 70% was used for training and 30% for classifier testing; a fivefold cross-validation method was utilized. The results reported 94.23% accuracy. The study, however, performed no comparative evaluation with other techniques, which can be considered as a study limitation [ 23 ].

Pashaei et al. [ 27 ] developed different methods to identify meningioma, glioma, and pituitary tumors. In their model, a CNN was used to extract hidden features from images and select features. The proposed model consisted of four convolutional layers, four pooling layers, one fully connected layer, and four batch normalization layers. The authors used ten epochs, 16 iterations per epoch, and the learning rate in this model was 0.01. The dataset provided by Cheng was also used in this study. The performance of the proposed model was evaluated using a tenfold cross-validation method, and 70% and 30% of the data was applied for training and system testing, respectively. The study compared the proposed method with MLP, Stacking, XGBoost, SVM, and RBF, and the results showed the high accuracy of the proposed method (93.68%) [ 27 ].

A CNN was also used by Abiwinanda in 2018 to diagnose the three most common types of brain tumors. In the learning process, the “adam” optimizer was used, which is a method for stochastic optimization using the stochastic gradient descent principle. In the study, the CNN was trained by 3064 T-1 weighted CE-MRI from brain tumor images provided by Cheng. The dataset included 1426 images of meningiomas, 708 images of gliomas, and 930 images of pituitary tumors. Of all the available images, 700 images from each class were applied, of which 500 were used for the training phase, and another 200 images were considered for the validation phase. In this model, all convolutional layers in the architectures used 32 filters, ReLu was used as an activation function, the maxpool kernel size was 2 × 2, and all the fully connected layers used 64 neurons. There were three neurons in the output layer, and the softmax activation function was employed at the output layer. The best-reported accuracy rates for training and validation were 98.51% and 84.19%, respectively [ 28 ].

In another study (2018), CNNs were applied to diagnose brain tumors using magnetic resonance images automatically. This study aimed to differentiate between healthy brains and brain tumor images. A two-stage multi-model system made the diagnosis. In the first stage, preprocessing and feature selection were performed by a CNN, and in the second stage, classification was done by an Error-Correcting Output Codes Support Vector Machine (ECOC-SVM). In the first stage, three algorithms, namely AlexNet, VGG-16, and VGG-19, were employed, among which AlexNet had the best performance with 99.55% accuracy. BraTS (2013 dataset) was used for the brain tumor localization phase, and images extracted from the standard Reference Image Database to Evaluate Response (RIDER) neuro MRI database were used for performance evaluation in the first phase [ 15 ].

Rehman et al. [ 22 ] studied three CNNs, namely AlexNet, GoogLeNet, and VGGNet. The study's primary purpose was to differentiate three brain tumor types, meningioma, glioma, and pituitary, using deep learning techniques and MRI images processing. Automated features were classified in the last phase using a linear classifier. Data augmentation techniques were applied to increase the sample size and reduce the possibility of over-fitting. The evaluation results showed that the VGG16 technique had the highest accuracy (98.69%) compared to other methods [ 22 ].

Mittal et al. [ 29 ] used the combination of Stationary Wavelet Transform (SWT) and a new Growing CNN (GCNN) to automate the segmentation process. In fact, they utilized these effective methods to identify brain tumors by MRI images. The evaluation results showed that the technique proposed in the study had the highest accuracy compared to the genetic algorithm; K-NN, SVM, and CNN [ 29 , 30 ].

Paul et al. [ 25 ] used deep learning methods to classify brain images related to meningioma, glioma, and pituitary tumors. In this research, the same dataset, i.e., 3064 T1-weighted contrast-enhanced MRI brain images of 233 patients, was applied; two types of neural networks, i.e., fully connected and CNNs, were designed. Moreover, a fivefold cross-validation technique showed that the general methods, with an accuracy of 91.43%, worked better than the specific methods, which required image dilation [ 25 ].

Material and methods

The methodology of the present study is illustrated in Fig.  1 . Major steps in the present study comprise brain tumor dataset selection, pre-processing MRI images, feature extraction, and classification by various classifiers.

Stages of the proposed methodology

The applied image-based dataset comprised 3264 T1-weighted contrast-enhanced MRI images [ 31 ]. There were four types of images in this dataset: glioma (926 images), meningioma (937 images), pituitary gland tumor (901 images), and healthy brain (500 images). All images were in sagittal, axial, and coronal planes. Figure  2 presents examples of the various types of tumors and different planes. The segment of tumors has been branded with a red outline. The number of images is different for each patient.

Description of normalized MRI images presenting diverse varieties of tumor in a different plane

Data augmentation and image pre-processing

Magnetic resonance images from this dataset had distinct sizes. These images represented the networks' input layer, so they were resized to 80*80 pixels. Each image was converted in two directions to augment the dataset. The first change included image rotation by 90°, and the second was flipping images vertically. Our chosen dataset was augmented three times, which resulted in 9792 images.

Proposed solutions

Figure  3 shows the proposed architecture for the two-dimensional CNN. A set of 9792 data was used in this study, 90% (8812) of which was employed as the training data and 10% (980) as the testing data. The proposed network had several layers, including convolution, which possessed two convolutional layers with 64 filters. Moreover, two convolution layers included 32 filters, and the others have 16. The final two convolutional layers make the desired network filters with a length of 8. The layers in this network have a 2*2 kernel function.

The architecture of the 2D convolution network

The convolutional network, which is also referred to as a neural network, has a hierarchical structure. This network creates a link between convolution layers, alternate pooling layers, and fully connected layers. One factor that should be noted here is that there is no need to use a pooling layer after each convolution layer. Figure  3 shows that the network has eight convolutional and four pooling layers. The final pooling layer with 2D output is changed to a 1D layer by flattened layers so it can be sent to the fully connected layers. Also, a type of padding is needed to manage and control the convolutional layer's output size. This study showed that the padding in adjacent cells is used for all networks to manage the edges of input data with the same values. To classify the data into categories by softmax activation function, a total of 1024 fully connected layer and a 4 fully connected layer were used. In this process, the batch-normalization layers were used to prevent overfitting. A dropout layer with a rate of 0.1 was also used following the max-pooling and fully connected layers.

For the activation function, the ReLU function was used in all layers apart from the last fully connected layer. To increase the efficiency, the Adam was used as an optimizing function. Different values, including 0.01, 0.001 and 0.0001, were used to test the learning rate parameter. Also, the best value with minimum learning error was found to be 0.001.

After 100 epochs, the training process was confirmed. The batch size was determined to be 16, and each epoch lasted about 7 s. The features extracted from the convolutional layer included input from the first layer fully connected to Ufc = 1024 hidden layers. The number of weights (Wconv) depended on the output size of the prior convolution layer (y1*y2), the number of filters (k), and the number of hidden layers in fully connected layers. Thus, the convolutional layer's weight was determined as follows [ 32 ]: Wconv = y1*y2*k*Ufc = 5*5*8*1024 = 204,800, where the number of existing parameters to the first fully connected layer equals 204,800 + 1024 (biases) = 205,824.

A summary of learning parameters for the proposed network can be seen in Table 2 . As seen in this table, the value of all parameters used to determine the four categories of this network are calculated by summing up the values in cited in the param column in Table 2 . The consequent value is 243,924, where all parameters are trainable.

Convolutional auto-encoder neural network

This study was conducted to design the architecture of a convolutional auto-encoder network. In this network, in order to predict the target value (Y) for the input (X), an auto-encoder was trained to predict the input (X) rather than training the network. The auto-encoder network was used to train and classify the data set instead of creating input images. Figure  4 shows the designed architecture of the convolutional auto-encoder network.

Convolutional auto-encoder network classification part

The network architecture designed in this study had two main parts. The first part included the convolutional auto-encoder network for data training, and the second part contained a convolutional network for classification, which utilizes the last output encoder layer of the first part. The first part of the architecture also consisted of 2D multilayer convolutional networks for both the encoder and the decoder. A total data set of 9792 was used in this study, 90% (8812) of which was used as the training data and 10% (980) as the test data.

The encoder part included a convolutional layer with a 32-filter length, two continuous convolutional layers with a 128-filter length, and two continuous convolutional layers with a 64-filter length. In the encoder, no pooling layer existed after each convolutional layer, but a second stage 2*2 max-pooling layer was considered after a sequence of two convolutional layers. The network’s decoder also included a convolutional layer with a 32-filter length, two continuous convolutional layers with a 64-filter length, two continuous convolutional layers with a 128-filter length, and a convolutional layer with a 128-filter length. A 2*2 kernel function was used for all convolutional layers, and there was no up-sampling layer after each convolutional layer. However, after a sequence of two convolutional layers, a 2*2 up-sampling layer was applied. The same padding was used for this network, and a batch normalization layer after each convolutional layer was considered. A dropout of 0.1 was also operated after each max-pooling layer, apart from the last layer, to prevent overfitting. Values of 0.01, 0.001 and 0.0001 were used to examine the learning rates, and the best value with minimum learning error was found to be 0. 001. The designed network was trained after 100 training epochs, and data was transmitted to the network in batches of 16 (batch-size), while each epoch ran in 14 s.

The critical features of the input data were removed by the automatic encoder network, and the output of the encoder layer was used for the classification (Fig.  5 ). For accurate classification, the output of the encoder layer was trained by two continuous convolution layers with 64-filter length, a 2*2-kernel function, and a 2*2-max-pooling layer with step 2. Batch-normalization and 0.1 dropout layers were also used to prevent overfitting [ 33 ]. To forward the output of the max-pooling layer to a 4-fully connected layer, the flattened layer was used [ 34 ], and the ReLU activation function was used for all layers.

Architecture of the proposed convolutional auto-encoder network

The vital factors for training and classification in the auto-encoder convolutional network include the encoder and classifier parameters. The extracted features of the encoder's last layer are trained by several convolutional layers, and the final extracted features would turn into the input of the first layer fully connected to the hidden layer of Ufc = 4. The number of weights (Wconv) depends on the number of hidden layers in the fully connected layer and the output size of the flattened layer. The flattened layer's output equals 5*5*64 = 1,600. Hence, the number of weights equals Wconv = out-flatten*Ufc = 1600*4 = 6,400, and the number of existing parameters to the second fully connected layer equals 6400 + 4 (biases) = 6404.

The learning parameters of this network are presented in Table 3 . The value of all modified parameters can be calculated by summing up the values in the param column (Table 3 ). The value of all modified parameters is 158,760, of which 1,569,000 are related to learning, and 960 are related to non-learning parameters.

Whole process in the present study was carried out in Keras with the Tensorflow backend. The networks in this study were designed in the Python environment and then, ran by cross-library in the Google Collaboratory (Colab) environment. Colab supplies a platform for running Python codes, especially machine learning, deep learning, and data analysis. The details of Colab hardware technical characteristics are given in Table 4 .

Performance evaluation metrics

The main objective of the current study was to classify MRI images into glioma, meningioma, pituitary gland tumor, and healthy brain classes. Metrics for performance evaluation included accuracy, precision, recall, and F-measure.

Accuracy refers to the proximity of a measured value to a standard or actual value. In other words, it is the ability of the tool to measure the exact value, whose accuracy can be measured.

In machine learning, precision results from dividing actual cases into sums of true and false cases. Recall is also the result of dividing the true items by all the items in that class. The weighting value for F-measure can be computed based on the precision and recall measures. F-measure is a good measure in evaluating the quality of classification and describing the weighted average between the quantities of precision and recall. The value of this measure is between 0 and 1, with 0 being the worst circumstance and 1 the best condition. This parameter was calculated by the following Eq. ( 4 ):

For organizing and evaluating classifiers and visualizing their performance, drawing receiver operating characteristics (ROC) plots can be useful in describing the results. ROC plots are commonly applied in medical decision-making and have recently been noticed in machine learning and data mining. The ROC curve is constructed by plotting the true positive rate (TPR) versus the false positive rate (FPR) in various threshold sets. Therefore, maximizing TPR while minimizing FPR are ideal achievements. This means that the upper left corner of the plot is the ideal point (FPR = 0 and TPR = 1).

Experimental results

Table 5 outlines the results of our proposed 2D CNN and convolutional auto-encoder neural network. The training accuracy of the proposed 2D CNN was found to be 96.4752%, whereas its validation accuracy was 93.4489%. The training accuracy of the proposed convolutional auto-encoder was found to be 95.6371%, and its validation accuracy was 90.9255%. The precision, recall, and F-measure of the four classes obtained from 2D CNN and the convolutional auto-encoder neural network are summarized in Tables 6 and 7 , respectively. Figure  6 shows the training, validation accuracy, and loss analyses of the proposed models concerning the number of epochs.

Training and validation analysis over 100 epochs for (1) 2D CNN: a training and testing accuracy analysis, and b training and testing loss analysis. (2) Convolutional auto-encoder neural network: c training and testing accuracy analysis, and d training and testing loss analysis

In the field of artificial intelligence, a confusion matrix is a matrix in which the performance of relevant algorithms is visualized. Each matrix column represents the predicted value of instances, and each row represents the actual (true) value of instances. This matrix justifies its appellation that allows us to see whether there are confusing results or overlaps between the classes. In medical research, it is significantly important to reduce the false positive and false negative outcomes in the modeling process. The impact of false positive and false negative rates is shown in Fig.  7 .

Confusion matrix analyses of the proposed model representing TP, TN, FP, and FN ratio obtained from the testing dataset of the a 2D CNN, and b convolutional auto-encoder neural network

Figure  8 presents the ROC curves of the proposed models along with classes 0, 1, 2, and 4 of the brain tumor classification models. The ideal point is observable for both class 0 and class 1.

Roc plots of the a 2D CNN, and b convolutional auto-encoder neural network

The outcomes of classical machine learning classifiers like Support Vector Machine (SVM), Logical Regression (LR), Random Forest (RF), Nearest Neighbor (NN), Stochastic Gradient Descent (SGD), and Multilayer Perceptron (MLP) were compared and classified into four classes. The obtained accuracy rates were 86% for NN, 82% for RF, 80% for SVM, 62% for LR, 52% for SGD, and 28% for MLP. Figure  9 shows the comparison of these results. The precision, recall, and F-measure for each set of glioma, meningioma, pituitary gland tumor, and healthy brain images were calculated by these methods and are summarized in Table 8 . For glioma tumor images, the highest precision was obtained by MLP (100%), the highest recall by KNN (90%), and the highest F-measure by KNN (87%). For meningioma tumors, the highest precision was obtained by KNN (93%), the highest recall by MLP (81%), and the highest F-measure by KNN (86%). For pituitary gland tumors, the highest precision was obtained by KNN (91%), the highest recall by RF (95%), and the highest F-measure by KNN (91%). For healthy brains, the highest precision was obtained by RF and SVM (83%), the highest recall by KNN (88%), and the highest F-measure by KNN (82%).

Comparison of classification accuracy rates of machine learning classifiers

The results of one-way ANOVA for the three parameters of precision, recall, and F-measure in eight modeling methods showed a statistically significant difference between the methods ( p -value < 0.001) (Table 9 ). LSD post hoc test results showed a significant difference between the means of precision, recall, and F-measure in the two methods presented in this study (2D CNN and convolutional auto-encoder) and the means of the three methods LR, SGD, and MLP ( p -value < 0.05). The mean F-measure parameter of the 2D CNN method, in addition to the three methods mentioned, was also significantly different from SVM ( p -value < 0.05) (Table 10 ).

The main objective of the current study was to develop two various deep learning networks and six machine learning techniques to classify MRI images into three classes of brain tumors (glioma, meningioma and pituitary gland tumor) and one class of healthy brain. The applied image dataset was publicly available at GitHub with 3264 T1-weighted contrast-enhanced magnetic resonance imaging (MRI) images.

According to the literature, some studies have used the famous T1-weighted contrast-enhanced MRI dataset (Figshare dataset), which contained 3064 MRI images of the human brain for tumor detection with computational approaches like neural networks. Studies using this dataset for the classification of brain tumors are listed in Table 11 . It should be noted that the study employed another dataset that included 3264 MRI images. This dataset contained four categories of MRI images, namely glioma, meningioma, and pituitary gland tumors and healthy brains (no tumors). Badža and Barjaktarović [ 5 ] conducted brain tumor detection using a CNN developed in MATLAB R2018a. Their proposed CNN had two convolutional layers of 64- and 16-filter lengths. The classification block had two fully connected layers: the first representing the flattened output of the max-pooling layer and the second having an equal number of hidden units to the number of tumor classes. The best result was reported as 95.40% for record-wise cross-validation for augmented images. Nonetheless, the highest accuracy obtained from the mentioned study (95.40%) with a value of 1.07 is less than our proposed 2D CNN. The execution time of our study was longer because of the complexity and high frequency of layers in the network, which justified the good accuracy we obtained. The longer execution time in the current study can be explained by the number of hidden layers, the pooling layers, and the batch sizes. It should be noted that the training of deeper networks requires extra time than the training of shallower or simpler networks [ 35 ].

In another research, a CNN and an extreme learning machine were applied to diagnose brain tumors. The proposed model utilized four convolution layers and batch-normalization layers with 16-, 32-, 64- and 128-filter lengths (3*3). Four ReLU layers and three max-pooling layers were used in the proposed CNN with stride size [ 2 , 2 ]. The model only had one fully connected layer with three types of classes. Feature vectors extracted by the mentioned convolution and layers were used as the input of KE-CNN (kernel CNN). The KE-CNN had 91.28% accuracy for classifying brain tumors [ 27 ]. However, our proposed 2D CNN and auto-encoder network achieved 96.47% and 95.63% accuracy, respectively. We used several layers of convolution for both networks and created complex networks, which can be justified by the large volume of data we used to increase classification accuracy. In comparison, other studies used networks with a small number of layers or a small amount of data [ 36 ].

In general, by comparing the two networks used in the current study, it can be concluded that the 2D CNN operated with 1% more accuracy than the auto-encoder network. Although the 2D CNN is more straightforward than the auto-encoder network, it performed better in feature extraction and learning, and according to what was previously mentioned, it uses all the parameters for learning [ 37 ]. The execution time (the duration of each epoch) or the runtime of the proposed convolutional network is less than that of the auto-encoder network. Therefore, the use of ordinary hardware and memory can be enough to run our proposed 2D CNN. One of the most notable differences between the current study and others is the use of six machine learning bribes to classify brain tumor images. SVM, NN, RF, SGD, LR, and MLP were developed for diagnosing brain tumors accurately.

In [ 23 ], researchers used a hybrid feature extraction approach with regularized extreme learning machine to classify types of brain tumors. Their method works by extracting the main features of brain images, and then applying principal component analysis to compute a covariance matrix of features. In the last step, a RELM is developed for diagnosing brain tumors into three classes (meningioma, pituitary, glioma). This method achieved 94.23% accuracy, which is not optimal compared to the results obtained from the networks designed in the current study.

A capsule network was employed in another study to classify brain tumors. Due to its good performance, the segmented tumor regions were applied as the inputs of the proposed capsule net. This method was implemented on Python 2.7, based on the Keras Library, using the Adam optimizer. Capsule net reached 86.56% accuracy for classifying segmented tumor regions and 78% accuracy for whole-brain tissue as input [ 21 ]. The researchers varied the feature maps in the convolutional layer of CapsNet in order to enhance accuracy; however, they achieved the highest accuracy of 86.56% using 64 feature maps with one convolutional layer of CapsNet. The network that employs only the tumor region or some other segmented part as input performs better in terms of execution speed. It also demands segmentation methods or a dedicated specialist to sign those parts [ 3 , 29 , 30 ]. The most favorable outcome in the research utilizing the segmented image parts as inputs has been presented by Tripathi and Bag [ 38 ], with 94.64% accuracy. They used features as inputs of classifiers extracted from the segmented brain tissue in the image. They checked their proposed method employing a fivefold cross-validation technique.

Like the approaches proposed in the current study, Rehman et al. performed the preprocessing of images with contrast improvement and dataset augmentation to reduce the occurrence of over-fitting and increase the database samples. Three types of CNNs (AlexNet, VGGNet, and GoogleNet) with an SVM model were utilized to diagnose brain tumors. The fine-tuned VGG16 network obtained the highest accuracy of 98.69% for the classification target [ 22 ]. In comparison, our developed methods 2% and 3% less accurate, respectively, than VGG16. This network is an intense and very deep network with 138 million weights, requiring complex hardware for calculating real-time performance [ 39 ]. Notably in this study, similar to ours, the researchers utilized various data augmentation methods to increase the size of the training dataset, such as rotating and flipping with raw MRI images. Data augmentation aims to enhance network performance by intentionally creating more training data from the original data.

In another study [ 28 ], CNNs were molded to determine the three most common types of brain tumors (i.e., glioma, meningioma, and pituitary gland tumors). In this study, researchers applied five different architectures of CNN for the classification of brain tumors and reported the highest accuracy for architecture 2. This architecture's training and validation accuracies were 98.51% and 84.19%, respectively. Architecture 2 is comprised of two convolutional layers, ReLU layer, and max-pooling with 64 hidden neurons. The testing accuracies of the developed networks in the current study were more significant than the accuracy of this architecture; nevertheless, this architecture has the capacity to overfit using a small learning rate or lower amount of training and testing data.

The current work is a pioneer study to develop two deep CNNs with optimal learning parameters and high accuracy. We also compared six machine learning techniques applied to classify brain tumors and healthy brains (no tumor). To better compare the previous studies conducted in this area, some key results are given in Table 11 .

A limitation of medical image classification is the small size of medical image databases. This limitation, in turn, restricts the availability of medical images for training deep neural networks. One way to deal with this challenge in our study is to apply data augmentation techniques to create new brain tumor lesions through scaling and rotation, which may cause class imbalance. In addition, in this study, our primary plan was to train the networks using local images of a hospital, but the problem of labeling the images prevented this from being implemented. Labeling cancer images is not only time-consuming but also requires a high level of expertise that is challenging in brain tumor analysis. In future works, considering the importance of rapid and accurate diagnosis of brain tumors without latency, we will investigate the constructions of other robust deep neural networks for brain tumor classification with less execution time and more simplicity. Hence, full machine learning and deep learning algorithms can be implemented as future enhancements. Furthermore, the proposed techniques can be used to detect different forms of cancers in MRI or Computed Tomography (CT) scan.

One of the areas of use for artificial intelligence and machine learning is the health domain. Deep networks are currently being designed and developed to detect diseases based on imaging. In order to do this, we have proposed computational-oriented methods to classify brain tumors. In our study, a novel 2D CNN architecture, a convolutional auto-encoder network, and six common machine-learning techniques were developed for brain tumor detection. This classification was conducted using a T1-weighted, contrast-enhanced MRI dataset, which includes three types of tumors and a healthy brain with no tumors.

According to the results and output shown in Figs. 6 , 7 and 8 , the proposed neural networks showed significant improvement over previous ones in detecting brain MRI image features and classifying them into three types of tumors and one class of healthy brain. The training accuracy of the proposed 2D CNN was found to be 96.47%, and the training accuracy of the proposed auto-encoder network was found to be 95.63%. In addition to the two-deep networks used in our study, six machine-learning techniques were also developed to classify brain tumors. The highest accuracies of 86%, 82% and 80% were attained for KNN, RF, and SVM, respectively. Comparing our networks with similar state-of-the-art methods shows that our proposed networks performed somewhat better with optimal execution time (maximum 15 min for 2D network and 25 min for auto-encoder network). The results of this study demonstrate that our proposed networks have an immeasurable generalization and high execution speed; therefore, they can be applied as effective decision-support agents for radiologists in medical diagnostics.

Availability of data and materials

All data generated or analyzed during this study are included in this published article. The link of the public MRI dataset that is used in this study is: https://www.kaggle.com/sartajbhuvaji/brain-tumor-classification-mri/ .

Prabukumar M, Agilandeeswari L, Ganesan K. An intelligent lung cancer diagnosis system using cuckoo search optimization and support vector machine classifier. J Ambient Intell Humaniz Comput. 2019;10(1):267–93.

Article   Google Scholar  

El-Dahshan ESA, Mohsen HM, Revett K, Salem ABM. Computer-aided diagnosis of human brain tumor through MRI: A survey and a new algorithm. Expert Syst Appl. 2014;41(11):5526–45.

Meng Y, Tang C, Yu J, Meng S, Zhang W. Exposure to lead increases the risk of meningioma and brain cancer: a meta-analysis. J Trace Elem Med Biol. 2020;60:126474.

Article   CAS   Google Scholar  

McFaline-Figueroa JR, Lee EQ. Brain tumors. Am J Med. 2018;131(8):874–82.

Badža MM, Barjaktarović MČ. Classification of brain tumors from MRI images using a convolutional neural network. Appl Sci. 2020;10(6):1999.

Shen D, Wu G, Suk H-I. Deep learning in medical image analysis. Annu Rev Biomed Eng. 2017;19(1):221–48.

Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.

Suzuki K. Overview of deep learning in medical imaging. Radiol Phys Technol. 2017;10(3):257–73.

Hijazi S, Kumar R, Rowen C. Using convolutional neural networks for image recognition. San Jose: Cadence Design Systems Inc; 2015. p. 1–12.

Google Scholar  

O’Shea K, Nash R. An introduction to convolutional neural networks. 2015. arXiv:1511.08458 .

Wong KKL, Fortino G, Abbott D. Deep learning-based cardiovascular image diagnosis: a promising challenge. Futur Gener Comput Syst. 2020;110:802–11.

Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Ther. 2015;8:2015–22.

CAS   Google Scholar  

Işın A, Direkoğlu C, Şah M. Review of MRI-based brain tumor image segmentation using deep learning methods. Proc Comput Sci. 2016;102:317–24.

Sarker P, Shuvo MMH, Hossain Z, Hasan S, editors. Segmentation and classification of lung tumor from 3D CT image using K-means clustering algorithm. In: 2017 4th international conference on advances in electrical engineering (ICAEE); 2017 28–30 Sept. 2017.

Abd-Ellah MK, Awad AI, Khalaf AAM, Hamed HFA. Two-phase multi-model automatic brain tumour diagnosis system from magnetic resonance images using convolutional neural networks. EURASIP J Image Video Process. 2018;2018(1):97.

Malathi M, Sinthia P. Brain tumour segmentation using convolutional neural network with tensor flow. Asian Pac J Cancer Prevent APJCP. 2019;20(7):2095.

Nalbalwar R, Majhi U, Patil R, Gonge S. Detection of brain tumor by using ANN. Image. 2014;2(3):7.

Amin J, Sharif M, Raza M, Yasmin M. Detection of brain tumor based on features fusion and machine learning. J Ambient Intell Humaniz Comput 2018;1–17.

Usman K, Rajpoot K. Brain tumor classification from multi-modality MRI using wavelets and machine learning. Pattern Anal Appl. 2017;20(3):871–81.

Farhi L, Zia R, Ali ZA. Performance analysis of machine learning classifiers for brain tumor MR images. Sir Syed Univ Res J Eng Technol. 2018;8(1):23–8.

Afshar P, Mohammadi A, Plataniotis KN. Brain tumor type classification via capsule networks. In: 2018 25th IEEE international conference on image processing (ICIP). IEEE; 2018 Oct 7. p. 3129–33.

Rehman A, Naz S, Razzak MI, Akram F, Imran M. A deep learning-based framework for automatic brain tumors classification using transfer learning. Circuits Syst Signal Process. 2020;39(2):757–75.

Gumaei A, Hassan MM, Hassan MR, Alelaiwi A, Fortino G. A Hybrid feature extraction method with regularized extreme learning machine for brain tumor classification. IEEE Access. 2019;7:36266–73.

Phaye SS, Sikka A, Dhall A, Bathula D. Dense and diverse capsule networks: making the capsules learn better. arXiv:1805.04001 . 2018 May 10.

Paul JS, Plassard AJ, Landman BA, Fabbri D. Deep learning for brain tumor classification. In: Paper presented at the Medical Imaging 2017: Biomedical Applications in Molecular, Structural, and Functional Imaging. 2017.

Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process. 2017;26(9):4509–22.

Pashaei A, Sajedi H, Jazayeri N. Brain tumor classification via convolutional neural network and extreme learning machines. In: 2018 8th international conference on computer and knowledge engineering (ICCKE). IEEE; 2018 Oct 25. p. 314–9.

Abiwinanda N, Hanif M, Hesaputra ST, Handayani A, Mengko TR. Brain tumor classification using convolutional neural network. In: World congress on medical physics and biomedical engineering 2018. Singapore: Springer; 2019. p. 183–9.

Mittal M, Goyal LM, Kaur S, Kaur I, Verma A, Jude HD. Deep learning based enhanced tumor segmentation approach for MR brain images. Appl Soft Comput. 2019;78:346–54.

Mohammed MA, Abd Ghani MK, Hamed RI, Ibrahim DA, Abdullah MK. Artificial neural networks for automatic segmentation and identification of nasopharyngeal carcinoma. J Comput Sci. 2017;21:263–74.

Lee JY, Lee KS, Seo BK, Cho KR, Woo OH, Song SE, et al. Radiomic machine learning for predicting prognostic biomarkers and molecular subtypes of breast cancer using tumor heterogeneity and angiogenesis properties on MRI. Eur Radiol. 2022;32(1):650–60.

Riazi R, Asrardel M, Shafaee M, Vakilipour S, Zare H, Veisi H. A data mining study on combustion dynamics and NOx emission of a swirl stabilised combustor with secondary fuel injection. Int J Heavy Veh Syst. 2017;24(3):215–38.

Fouladi S, Safaei AA, Mammone N, Ghaderi F, Ebadi M. Efficient deep neural networks for classification of Alzheimer’s disease and mild cognitive impairment from scalp EEG recordings. Cogn Comput. 2022; 1–22.

Fouladi S, Ebadi M, Safaei AA, Bajuri MY, Ahmadian A. Efficient deep neural networks for classification of COVID-19 based on CT images: virtualization via software defined radio. Comput Commun. 2021;176:234–48.

Wang Y, Wang L, Wang H, Li P. End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access. 2019;7:31959–70.

Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing. 2018;282:248–61.

Khalid S, Khalil T, Nasreen S. A survey of feature selection and feature extraction techniques in machine learning. In: 2014 science and information conference. IEEE; 2014 Aug 27. p. 372–8.

Tripathi PC, Bag S. Non-invasively grading of brain tumor through noise robust textural and intensity based features. In: Computational intelligence in pattern recognition. Springer; 2020. p. 531–9.

Han S, Pool J, Tran J, Dally W. Learning both weights and connections for efficient neural network. Adv Neural Inf Process Syst. 2015;28:1135–43.

Download references

Acknowledgements

Not applicable.

In this paper, we didn't have any financial sponsor.

Author information

Authors and affiliations.

Medical Informatics and Health Information Management Department, School of Allied Medical Sciences, Tehran University of Medical Sciences, 3rd Floor, No #17, Farredanesh Alley, Ghods St, Enghelab Ave, Tehran, 14177-44361, Iran

Soheila Saeedi, Sorayya Rezayi & Sharareh R. Niakan Kalhori

Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran

Hamidreza Keshavarz

Peter L. Reichertz Institute for Medical Informatics, TU Braunschweig and Hannover Medical School, 38106, Brunswick, Germany

Sharareh R. Niakan Kalhori

You can also search for this author in PubMed   Google Scholar

Contributions

SS, SR, HK, SRNK: Conceptualization, Methodology, Software, SS, SR, HK, SRNK: Data curation, Writing-Original draft preparation. SS, SR, HK, SRNK: Visualization, Investigation. HK, SRNK: Supervision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sorayya Rezayi .

Ethics declarations

Ethics approval and consent to participate.

All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Competing interests.

The authors declare no competing interests regarding the publication of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Saeedi, S., Rezayi, S., Keshavarz, H. et al. MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques. BMC Med Inform Decis Mak 23 , 16 (2023). https://doi.org/10.1186/s12911-023-02114-6

Download citation

Received : 04 June 2022

Accepted : 16 January 2023

Published : 23 January 2023

DOI : https://doi.org/10.1186/s12911-023-02114-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Convolutional neural network
• Brain tumor
• Machine learning
• Medical imaging

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

Brain and Spinal Cord Tumor Research Results and Study Updates

See Advances in Brain and Spinal Cord Tumor Research for an overview of recent findings and progress, plus ongoing projects supported by NCI.

Recent results from several small clinical trials have suggested it may be possible to develop an effective immunotherapy for glioblastoma. Among them are findings from a four-patient trial testing a unique type of mRNA cancer vaccine.

FDA has granted an accelerated approval to tovorafenib (Ojemda) for kids and teens who have low-grade glioma with changes in the BRAF gene. In a small clinical trial, the drug shrank or completely eliminated tumors in nearly 70% of patients.

The activity of 34 genes can accurately predict the aggressiveness of meningiomas, a new study shows. This gene expression signature may help oncologists select the best treatments for people with this common type of brain cancer than they can with current methods.

An NCI-supported study called OPTIMUM, part of the Cancer Moonshot, was launched to improve the care of people with brain tumors called low-grade glioma in part by bringing them into glioma-related research.

Treating craniopharyngioma often requires surgery, radiation therapy, or both. But results of a study suggest that, for many, combining the targeted therapies vemurafenib (Zelboraf) and cobimetinib (Cotellic) may substantially delay, or even eliminate, the need for these treatments.

In a large clinical trial, vorasidenib slowed the growth of low-grade gliomas that had mutations in the IDH1 or IDH2 genes. Vorasidenib is the first targeted drug developed specifically to treat brain tumors.

Researchers have found that the aggressive brain cancer glioblastoma can co-opt the formation of new synapses to fuel its own growth. This neural redirection also appears to play a role in the devastating cognitive decline seen in many people with glioblastoma.

Two companion studies have found different forms of some brain tumors, diffuse midline glioma and IDH-mutant glioma, become dependent for their survival on the production of chemicals called pyrimidines. Clinical trials are planned to test a drug that blocks pyrimidine synthesis in patients with gliomas.

The combination of dabrafenib (Tafinlar) and trametinib (Mekinist) shrank more brain tumors, kept the tumors at bay for longer, and caused fewer side effects than chemotherapy, trial results showed. The children all had glioma with a BRAF V600 mutation that could not be surgically removed or came back after surgery.

Two separate but complementary studies have identified a new way to classify meningioma, the most common type of brain tumor. The grouping system may help predict whether a patient’s tumor will grow back after treatment and identify new treatments.

A nanoparticle coating may help cancer drugs reach medulloblastoma tumors in the brain and make the treatment less toxic. Mice treated with nanoparticles containing palbociclib (Ibrance) and sapanisertib lived substantially longer than those treated with either drug alone.

A new test could potentially be used to identify children treated for medulloblastoma who are at high risk of their cancer returning. The test detects evidence of remaining cancer in DNA shed from medulloblastoma tumor cells into cerebrospinal fluid.

Standard radiation for medulloblastoma can cause long-term damage to a child’s developing brain. A new clinical trial suggests that the volume and dose of radiation could be safely tailored based on genetic features in the patient’s tumor.

In people with glioblastoma and other brain cancers, steroids appear to limit the effectiveness of immunotherapy drugs, a new study shows. The findings should influence how steroids are used to manage brain tumor symptoms, researchers said.

Results from two studies show that a liquid biopsy that analyzes DNA in blood accurately detected kidney cancer at early and more advanced stages and identified and classified different types of brain tumors.

A method that combines artificial intelligence with an advanced imaging technology can accurately diagnose brain tumors in fewer than 3 minutes during surgery, a new study shows. The approach can also accurately distinguish tumor from healthy tissue.

Glioblastoma cells sneak many copies of a key oncogene into circular pieces of DNA. In a new NCI-funded study, scientists found that the cells also slip several different genetic “on switches” into these DNA circles, helping to fuel the cancer’s growth.

Men and women with glioblastoma appear to respond differently to standard treatment. A new study identifies biological factors that might contribute to this sex difference.

A liquid biopsy blood test can detect DNA from brain tumors called diffuse midline gliomas, researchers have found. This minimally invasive test could be used to identify and follow molecular changes in children with these highly lethal brain tumors.

Despite continued efforts to develop new therapies for glioblastoma, none have been able to improve how long patients live appreciably. Despite some setbacks, researchers are hopeful that immunotherapy might be able to succeed where other therapies have not.

Central Nervous System (CNS) Tumors

• First Online: 30 August 2024

Cite this chapter

• Aliasgar Moiyadi 5 ,
• Vikas Singh 5 ,
• Raees Tonse 6 &
• Rakesh Jalali 6  

The term central nervous system tumors includes both primary and metastatic tumors of the brain and spine. The etiology of CNS tumors remains largely elusive with certain genetic syndromes having increased risk. With advances in imaging technology and patient awareness, the incidence of these tumors is increasing worldwide. The histological spectrum of tumors is variable with both benign and malignant tumors seen across all the age groups. Meningiomas and gliomas are among the common primary tumors in adults while medulloblastomas and gliomas are common in children. Surgery is the mainstay of treatment in the majority of tumours and involves the use of multiple adjuncts for achieving maximal safe resection. Unlike other solid tumors, oncological resections (margin negative resections) are difficult (due to eloquent location) and adjuvant treatment (radiotherapy and chemotherapy) plays an important role in the management of the disease, even for benign tumors. Advances in radiation and particle therapy have greatly improved outcomes. Therapy-related sequelae may be associated with significant morbidity. Rehabilitation is vital in improving acute as well as long-term patient outcomes. Even after complete treatment, the risk of disease progression/recurrence always remains and close clinical follow-up is essential.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

Atypical teratoid/rhabdoid tumors

Blood–brain barrier

Central nervous system

Computed tomography

Diffusion tractography imaging

Epidermal growth factor receptor

Embryonal tumor with multi-layered rosettes

Germ cell tumor

Generalized tonic-clonic convulsion

Isocitrate dehydrogenase

Image-guided radiation therapy

Intensity-modulated radiation therapy

Medulloblastoma

Magnetic resonance imaging

MR spectroscopy

Primary central nervous system lymphoma

Procarbazine, lomustine, and vincristine

Positron emission tomography

Primitive neuro-ectodermal tumor

Proton beam radiation therapy

Pleomorphic xanthoastrocytoma

Stereotactic radiosurgery SVZ sub-ventricular zone

Volumetric modulated arc therapy

Whole brain radiation therapy

AIHW (Australian Institute of Health and Welfare) and AACR (Australasian Association of Cancer Registries). Cancer in Australia: an overview, vol. CAN 46. Canberra: AIHW; 2008.

Google Scholar  

Aldape K, Burger PC, Perry A. Clinicopathologic aspects of 1p/19q loss and the diagnosis of oligodendroglioma. Arch Pathol Lab Med. 2007;131:242.

Article   CAS   PubMed   Google Scholar  

Kopelson G, Linggood RM, Kleinman GM, Doucette J, Wang CC. Management of intramedullary spinal cord tumors. Radiology. 1980;135:473–9.

Adams RD, Victor M. Intraspinal tumors. In: Principles of neurology. 2nd ed. New York: McGraw-Hill; 1984;638–41.

Mintz AP, Cairncross JG. Treatment of a single brain metastasis. The role of radiation following surgical excision. JAMA. 1998;280:1527–9.

Nussbaum ES, Djalilian HR, Cho KH, Hall WA, Brain metastases. Histology, multiplicity, surgery, and survival. Cancer. 1996;78:1781–8.

Lacayo A, Farmer PM. Brain tumors in children: a review. Ann Clin Lab Sci. 1991;21(1):26–35.

CAS   PubMed   Google Scholar  

Nair M, Varghese C, Swaminathan R. Cancer: current scenario, intervention strategies and projections for 2015. NCMH Background Papers; 2015.

Dasgupta A, Gupta T, Jalali R. Indian data on central nervous tumors: a summary of published work. South Asian J Cancer. 2016;5(3):147–53.

Article   PubMed   PubMed Central   Google Scholar  

Jain A, Sharma MC, Suri V, Kale SS, Mahapatra AK, Tatke M, et al. Spectrum of pediatric brain tumors in India: a multi-institutional study. Neurol India. 2011;59(2):208–11.

Article   PubMed   Google Scholar  

Jalali R, Datta D. Prospective analysis of incidence of central nervous tumors presenting in a tertiary cancer hospital from India. J Neuro-Oncol. 2008;87:111–4.

Article   Google Scholar  

Jaiswal J, Shastry AH, Ramesh A, Chickabasaviah YT, Arimappamagan A, Santosh V. Spectrum of primary intracranial tumors at a tertiary care neurological institute: a hospital-based brain tumor registry. Neurol India. 2016;64(3):494–501.

MacMahon B. The ethnic distribution of cancer mortality in New York City, 1955. Acta Unio Int Contra Cancrum. 1960;16(53):57.

Newill VA. Distribution of cancer mortality among ethnic subgroups of the white population of New York City, 1953–1958. J Natl Cancer Inst. 1961;26:405–17.

Choi NW, Schuman LM, Gullen WH. Epidemiology of primary central nervous system neoplasms. I. Mortality from primary central nervous system neoplasms in Minnesota. Am J Epidemiol. 1970;91(3):238–59.

Musicco M, Filippini G, Bordo BM, Melotto A, Morello G, Berrino F. Gliomas and occupational exposure to carcinogens: case–control study. Am J Epidemiol. 1982;116(5):782–90.

Farrell CJ, Plotkin SR. Genetic causes of brain tumors: neurofibromatosis, tuberous sclerosis, von Hippel-Lindau, and other syndromes. Neurol Clin. 2007;25(4):925–46, viii.

Blatt J, Jaffe R, Deutsch M, Adkins JC. Neurofibromatosis and childhood tumors. Cancer. 1986;57(6):1225–9.

Lynch HT, Marcus JM, Watson P. Genetic epidemiology of breast cancer. In: Lynch HT, Hirayama T, editors. Genetic epidemiology of cancer. Boca Raton: CRC Press; 1989. p. 289–332.

Swift M, Morrell D, Cromartie E, Chamberlin AR, Skolnick MH, Bishop DT. The incidence and gene frequency of ataxia-telangiectasia in the United States. Am J Hum Genet. 1986;39(5):573–83.

CAS   PubMed   PubMed Central   Google Scholar  

Bolande R. Teratogenesis and oncogenesis: a developmental spectrum. In: Lynch HT, editor. Genetic epidemiology of cancer. Boca Raton: CRC Press; 1989. p. 58–68.

Bansal K, Liang ML, Rutka JT. Molecular biology of human gliomas. Technol Cancer Res Treat. 2006;5(3):185–94.

Tomkova K, Tomka M, Zajac V. Contribution of p53, p63, and p73 to the developmental diseases and cancer. Neoplasma. 2008;55(3):177–81.

Zang KD, Singer H. Chromosomal constitution of meningiomas. Nature. 1967;216(5110):84–5.

Bigner SH, Mark J, Mahaley MS, Bigner DD. Patterns of the early, gross chromosomal changes in malignant human gliomas. Hereditas. 1984;101(1):103–13.

Bolger GB, Stamberg J, Kirsch IR, Hollis GF, Schwarz DF, Thomas GH. Chromosome translocation t(14:22) and oncogene (c-sis) variant in a pedigree with familial meningioma. N Engl J Med. 1985;312(9):564–7.

Seizinger BR, Martuza RL, Gusella JF. Loss of genes on chromosome 22 in tumorigenesis of human acoustic neuroma. Nature. 1986;322(6080):644–7.

Bithell JF, Stewart AM. Pre-natal irradiation and childhood malignancy: a review of British data from the Oxford survey. Br J Cancer. 1975;31(3):271–87.

Article   CAS   PubMed   PubMed Central   Google Scholar  

MacMahon B. Phenobarbital: epidemiological evidence. In: Wald NJ, Doll R, editors. Interpretation of negative epidemiological evidence for carcinogenicity, vol. 65. Lyon: IARC; 1985. p. 153–8.

Ron E, Modan B, Boice JD Jr, Alfandary E, Stovall M, Chetrit A, et al. Tumors of the brain and nervous system after radiotherapy in childhood. N Engl J Med. 1988;319(16):1033–9.

Preston-Martin S, Yu MC, Henderson BE, Roberts C. Risk factors for meningiomas in men in Los Angeles County. J Natl Cancer Inst. 1983;70(5):863–6.

Krewski D, Byus CV, Glickman BW, Lotz WG, Mandeville R, McBride ML, et al. Recent advances in research on radiofrequency fields and health. J Toxicol Environ Health B Crit Rev. 2001;4(1):145–59.

Boice JD, McLaughlin JK. Concerning mobile phone use and risk of acoustic neuroma. Br J Cancer. 2006;95(1):130.

Gold E, Gordis L, Tonascia J, Szklo M. Risk factors for brain tumors in children. Am J Epidemiol. 1979;109(3):309–19.

Preston-Martin S, Henderson BE, Pike MC. Descriptive epidemiology of cancers of the upper respiratory tract in Los Angeles. Cancer. 1982;49(10):2201–7.

Hoover R, Fraumeni JF Jr. Risk of cancer in renal-transplant recipients. Lancet. 1973;2(7820):55–7.

Charman HP, Lowenstein DH, Cho KG, DeArmond SJ, Wilson CB. Primary cerebral angiosarcoma. Case report. J Neurosurg. 1988;68(5):806–10.

Mahaley MS Jr, Mettlin C, Natarajan N, Laws ER Jr, Peace BB. National survey of patterns of care for brain-tumor patients. J Neurosurg. 1989;71:826–36.

Forsyth PA, Posner JB. Headaches in patients with brain tumors: a study of 111 patients. Neurology. 1993;43:1678–83.

Lohmann P, Kocher M, Steger J, Galldiks N. Radiomic derived from amino acid PET and conventional MRI in patients with high grade gliomas. Q J Nucl Med Mol Imaging. 2018;62(3):272–80.

Louis DN, Perry A, Wesseling P, Brat DJ, Cree IA, Figarella-Branger D, Hawkins C, Ng HK, Pfister SM, Reifenberger G, Soffietti R, von Deimling A, Ellison DW. The 2021 WHO Classification of Tumors of the central nervous system: a summary. Neuro-Oncology. 2021;23(8):1231–51. https://doi.org/10.1093/neuonc/noab106 .

Kubben PL, ter Meulen KJ, Schijns OE, ter Laak-Poort MP, van Overbeeke JJ, van Santbrink H. Intraoperative MRI-guided resection of glioblastoma multiforme: a systematic review. Lancet Oncol. 2011;12(11):1062–70.

Moiyadi AV. Intraoperative ultrasound technology in neuro-oncology practice-current role and future applications. World Neurosurg. 2016;93:81–93.

Moiyadi AV, Shetty PM, Mahajan A, Udare A, Sridhar E. Usefulness of three-dimensional navigable intraoperative ultrasound in resection of brain tumors with a special emphasis on malignant gliomas. Acta Neurochir (Wien). 2013;155(12):2217–25.

Moiyadi AV, Shetty P. Direct navigated 3D ultrasound for resection of brain tumors: a useful tool for intraoperative image guidance. Neurosurg Focus. 2016;40(3):E5.

Ferraro N, Barbarite E, Albert TR, Berchmans E, Shah AH, Bregy A, et al. The role of 5-aminolevulinic acid in brain tumor surgery: a systematic review. Neurosurg Rev. 2016;39(4):545–55.

De Witt Hamer PC, Robles SG, Zwinderman AH, Duffau H, Berger MS. Impact of intraoperative stimulation brain mapping on glioma surgery outcome: a meta-analysis. J Clin Oncol. 2012;30:2559–65.

Yan Y, Guo Z, Zhang H, Wang N, Xu Y. Precision radiotherapy for brain tumors: a 10-year bibliometric analysis. Neural Regener Res. 2012;7(22):1752–9.

Swisher-McClure S, Hahn SM, Bekelman J. Proton beam therapy: the next disruptive innovation in healthcare? PMJ. 2015;91(1075):241–3.

Mizumoto M, Oshiro Y, Yamamoto T, Kohzuki H, Sakurai H. Proton beam therapy for pediatric brain tumor. Neurol Med Chir. 2017;57(7):343–55.

Eaton BR, MacDonald SM, Yock TI, Tarbell NJ. Secondary malignancy risk following proton radiation therapy. Front Oncol. 2015;5:261.

Mizumoto M, Okumura T, Ishikawa E, Yamamoto T, Takano S, Matsumura A, et al. Reirradiation for recurrent malignant brain tumor with radiotherapy or proton beam therapy. Technical considerations based on experience at a single institution. Strahlenther Onkol. 2013;189(8):656–63.

Tivnan A, Heilinger T, Lavelle EC, Prehn JHM. Advances in immunotherapy for the treatment of glioblastoma. J Neuro-Oncol. 2017;131(1):1–9.

Article   CAS   Google Scholar  

Tonse R, Gupta T, Epari S, Shastri JG, Gurav M, Bano N, Jalali R. Impact of WHO 2016 update of brain tumor classification, molecular markers and clinical outcomes in pleomorphic xanthoastrocytoma. J Neuro-Oncol. 2018;136(2):343–50.

Pignatti F, van den Bent M, Curran D, et al., European Organization for Research and Treatment of Cancer Brain Tumor Cooperative Group, European Organization for Research and Treatment of Cancer Radiotherapy Cooperative Group. Prognostic factors for survival in adult patients with cerebral low-grade glioma. J Clin Oncol. 2002;20(8):2076–84.

Shaw EG, Wang M, Coons SW, Brachman DG, Buckner JC, Stelzer KJ, et al. Randomized trial of radiation therapy plus procarbazine, lomustine, and vincristine chemotherapy for supratentorial adult low-grade glioma: initial results of RTOG 9802. J Clin Oncol. 2012;30(25):3065–70.

Cairncross G, Wang M, Shaw E, Jenkins R, Brachman D, Buckner J, et al. Phase III trial of chemoradiotherapy for anaplastic oligodendroglioma: long-term results of RTOG 9402. J Clin Oncol. 2013;31(3):337–43.

Fisher BJ, Hu C, Macdonald DR, Lesser GJ, Coons SW, Brachman DG, et al. Phase 2 study of temozolomide-based chemoradiation therapy for high-risk low-grade gliomas: preliminary results of Radiation Therapy Oncology Group 0424. Int J Radiat Oncol Biol Phys. 2015;91(3):497–504.

Van den Bent MJ, Baumert B, Erridge SC, Vogelbaum MA, et al. Interim results from the CATNON trial (EORTC study 26053-22054) of treatment with concurrent and adjuvant temozolomide for 1p/19q non-co-deleted anaplastic glioma: a phase 3, randomised, open-label intergroup study. Lancet. 2017;390(10103):1645–53. https://doi.org/10.1016/S0140-6736(17)31442-3 . Epub 2017 Aug 8. Erratum in: Lancet. 2017 Oct 7;390(10103):1644. PMID: 28801186; PMCID: PMC5806535.

Hasegawa M, Niibe H, Mitsuhashi N, Yamakawa M, Kato S, Furuta M, et al. Hyperfractionated and hypofractionated radiation therapy for human malignant glioma xenograft in nude mice. Jpn J Cancer Res. 1995;86(9):879–84.

Stupp R, Hegi ME, Mason WP, van den Bent MJ, Taphoorn MJ, Janzer RC, et al. Effects of radiotherapy with concomitant and adjuvant temozolomide versus radiotherapy alone on survival in glioblastoma in a randomised phase III study: 5-year analysis of the EORTC-NCIC trial. Lancet Oncol. 2009;10:459–66.

Perry JR, Laperriere N, O’Callaghan CJ, Brandes AA, Menten J, Phillips C, et al. Short-course radiation plus temozolomide in elderly patients with glioblastoma. N Engl J Med. 2017;376(11):1027–37.

Stupp R, Taillibert S, Kanner A, Read W, Steinberg D, Lhermitte B, et al. Effect of tumor treating fields plus maintenance temozolomide vs maintenance temozolomide alone on survival in patients with glioblastoma: a randomized clinical trial. JAMA. 2017;318(23):2306–16.

Gupta T, Nair V, Paul SN, Kannan S, Moiyadi A, Epari S, Jalali R. Can irradiation of potential cancer stem-cell niche in the subventricular zone influence survival in patients with newly diagnosed glioblastoma. J Neuro-Oncol. 2012;109(1):195–203.

Huang J, Liu F, Liu Z, Tang H, Wu H, Gong Q, et al. Immune checkpoint in glioblastoma: promising and challenging. Front Pharmacol. 2017;8:242.

Monje M, Mitra SS, Freret ME, Raveh TB, Kim J, Masek M, et al. Hedgehog-responsive candidate cell of origin for diffuse intrinsic pontine glioma. Proc Natl Acad Sci U S A. 2011;108:4453.

Kunder R, Jalali R, Sridhar E, Moiyadi A, Goel N, Goel A, et al. Real-time PCR assay based on the differential expression of microRNAs and protein-coding genes for molecular classification of formalin-fixed paraffin embedded medulloblastomas. Neuro-Oncology. 2013;15(12):1644–51.

Ramaswamy V, Remke M, Bouffet E, Bailey S. Risk stratification of childhood medulloblastoma in the molecular era: the current consensus. Acta Neuropathol. 2016;131(6):821–31.

Wiemels J, Wrensch M, Claus EB. Epidemiology and etiology of meningioma. J Neuro-Oncol. 2010;99(3):307–14.

Gousias K, Schramm J, Simon M. The Simpson grading revisited: aggressive surgery and its place in modern meningioma management. J Neurosurg. 2016;125(3):551–60.

Sahm F, Schrimpf D, Olar A, Koelsche C, Reuss D, Bissel J, et al. TERT promoter mutations and risk of recurrence in meningioma. JNCI. 2016;108(5):djv377.

Chang EF, Zada G, Kim S, Lamborn KR, Quinones-Hinojosa A, Tyrrell JB, et al. Long-term recurrence and mortality after surgery and adjuvant radiotherapy for nonfunctional pituitary adenomas. J Neurosurg. 2008;108:736.

Erridge SC, Conkey DS, Stockton D, Strachan MW, Statham PF, Whittle IR, et al. Radiotherapy for pituitary adenomas: long-term efficacy and toxicity. Radiother Oncol. 2009;93:597.

Patt H, Jalali R, Yerawar C, Khare S, Gupta T, Goel A, et al. High-precision conformal fractionated radiotherapy is effective in achieving remission in patients with acromegaly after failed transsphenoidal surgery. Endocr Pract. 2016;22(2):162–72.

Sughrue ME, Kaur R, Rutkowski MJ, Kane AJ, Kaur G, Yang I, et al. Extent of resection and the long-term durability of vestibular schwannoma surgery. J Neurosurg. 2011;114:1218.

Meijer OW, Vandertop WP, Baayen JC, Slotman BJ. Single-fraction vs. fractionated linac-based stereotactic radiosurgery for vestibular schwannoma: a single-institution study. Int J Radiat Oncol Biol Phys. 2003;56(5):1390–6.

Driven L, Armstrong TS, Blakeley JO, Brown PD, Grant R, Jalali R, et al. Working plan for the use of patient-reported outcome measures in adults with brain tumours: a response assessment in neuro-oncology (RANO) initiative. Lancet Oncol. 2018;19(3):e173–80.

Mulhern RK, Merchant TE, Gajjar A, Reddick WE, Kun LE. Late neurocognitive sequelae in survivors of brain tumours in childhood. Lancet Oncol. 2004;5(7):399–408.

Livesey EA, Hindmarsh PC, Brook CG, Whitton AC, Bloom HJ, Tobias JS, et al. Endocrine disorders following treatment of childhood brain tumours. Br J Cancer. 1990;61(4):622–5.

Hua C, Bass JK, Khan R, Kun LE, Merchant TE. Hearing loss after radiotherapy for pediatric brain tumors: effect of cochlear dose. Int J Radiat Oncol Biol Phys. 2008;72(3):892–9.

Jalali R, Gupta T, Goda JS, Goswami S, Shah N, Dutta D, et al. Efficacy of stereotactic conformal radiotherapy vs conventional radiotherapy on benign and low-grade brain tumors: a randomized clinical trial. JAMA Oncol. 2017;3(10):1368–76.

Farjam R, Pramanik P, Aryal MP, Srinivasan A, Chapman CH, Tsien CI, et al. A radiation-induced hippocampal vascular injury surrogate marker predicts late neurocognitive dysfunction. Int J Radiat Oncol Biol Phys. 2015;93(4):908–15.

Download references

Acknowledgments

The authors acknowledge input and contributions from colleagues from the Neuro Oncology Disease Management Group, especially Dr. Prakash Shetty, Dr. Tejpal Gupta, Dr. Girish Chinnaswamy, Dr. Jayant Goda, Dr. Epari Sridhar, and Ms. Nazia Bano.

Author information

Authors and affiliations.

Department of Surgical Oncology, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Aliasgar Moiyadi & Vikas Singh

Department of Radiation Oncology, Apollo Proton Cancer Center, Chennai, Tamil Nadu, India

Raees Tonse & Rakesh Jalali

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Aliasgar Moiyadi .

Editor information

Editors and affiliations.

Emeritus Professor of Surgical Oncology, Former Director, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Rajendra A. Badwe

Professor of Medical Oncology, Director, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Sudeep Gupta

Professor & Head, Cancer Surgery Chief, Gastrointestinal and HPB Service Deputy Director, Tata Memorial Hospital, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Shailesh V. Shrikhande

Professor of Radiation Oncology Deputy Director Academics Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Siddhartha Laskar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 Springer Nature Singapore Pte Ltd.

About this chapter

Moiyadi, A., Singh, V., Tonse, R., Jalali, R. (2024). Central Nervous System (CNS) Tumors. In: Badwe, R.A., Gupta, S., Shrikhande, S.V., Laskar, S. (eds) Tata Memorial Centre Textbook of Oncology. Springer, Singapore. https://doi.org/10.1007/978-981-99-3378-5_29

Download citation

DOI : https://doi.org/10.1007/978-981-99-3378-5_29

Published : 30 August 2024

Publisher Name : Springer, Singapore

Print ISBN : 978-981-99-3377-8

Online ISBN : 978-981-99-3378-5

eBook Packages : Medicine Medicine (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Advanced imaging technique-based brain tumor segmentation using ResNET-50 CNN

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Reprints and Permissions
• Cite Icon Cite
• Search Site

S. M. Hussain , T. Naz , M. Shakeel , F. Akram , J. S. U. Rahman , K. S. Sathish; Advanced imaging technique-based brain tumor segmentation using ResNET-50 CNN. AIP Conf. Proc. 30 August 2024; 3161 (1): 020156. https://doi.org/10.1063/5.0229438

Download citation file:

• Ris (Zotero)
• Reference Manager

The brain is a vital and complex organ of the human system. It is responsible for controlling the overall activity of the human body. Normal functionality of the organs depends on the healthy brain and any disorder in the brain may cause a critical and life-threatening condition. There are several disorders related to the brain like Alzheimer’s, Tumors, Cancer, Epilepsy, and Stroke. The severity of the brain disorder depends on the region and its intensity. Among all these disorder brain Tumors is the most common which may occur in any age and gender. There are several modalities used to acquire the image of the brain to diagnose a brain tumor but identifying the tumor from the image requires profound knowledge and expertise. Physically identification of the tumor from the radiologic image is a common practice of medical professionals; however, the procedure has the probability of human error. Computer-aided diagnoses of Brain Tumors assist the medical professional to estimate the proper region of the tumor in an image in a better way. Several approaches have been adopted by the researcher to identify the brain tumor from a radiologic image. This article proposed a MATLAB-based graphical user interface to assist medical professionals to estimate Brain Tumors from radiologic Images. The algorithm is based on the acquisition of the radiological image, preprocessing, Image segmentation, and algorithm implementation for Tumor detection. Further, the Convolutional Neural Network techniques have been applied to compare the accuracy of the six different pre-trained models. Significant results have been achieved via the proposed algorithm. The algorithm reduces the process time and rate of human error cost-effectively. The ResNET50 Approach-1 is identified as the better approach among the six approaches with a Training and Validation Accuracy of 99 % and Test Accuracy of 93%.

Citing articles via

Publish with us - request a quote.

Sign up for alerts

• Online ISSN 1551-7616
• Print ISSN 0094-243X
• For Researchers
• For Librarians
• For Advertisers
• Our Publishing Partners  
• Physics Today
• Conference Proceedings
• Special Topics

pubs.aip.org

• Privacy Policy
• Terms of Use

Connect with AIP Publishing

This feature is available to subscribers only.

Sign In or Create an Account

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Ensemble Deep Learning Technique for Detecting MRI Brain Tumor

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, index terms.

Applied computing

Life and medical sciences

Health care information systems

Computing methodologies

Artificial intelligence

Computer vision

Computer vision problems

Image segmentation

Interest point and salient region detections

Video segmentation

Machine learning

Machine learning approaches

Neural networks

Recommendations

Otsu’s thresholding technique for mri image brain tumor segmentation.

MRI image segmentation is very challenging area in medical image processing. It is implemented with the low contract of MRI scan. In terms of certain input features or expert information, the major objective of medical image segmentation is to ...

Multiclass Brain Tumor Classification using Region Growing based Tumor Segmentation and Ensemble Wavelet Features

In this research, an automated method is proposed for Brain tumor classification into four different types which is an important step in brain tumor diagnosis. Most of the recent research studies focus on binomial classification of brain MR image into ...

Efficient Framework for Identifying, Locating, Detecting and Classifying MRI Brain Tumor in MRI Images

Image processing has plays vital role in today's technological world. It can be applied in numerous application areas such as medical, remote sensing, computer vision etc. Brain tumor is caused due to formation of abnormal tissues within human brain. ...

Information

Published in.

Hindawi Limited

London, United Kingdom

Publication History

• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 12 September 2022

DACBT: deep learning approach for classification of brain tumors using MRI data in IoT healthcare environment

• Amin ul Haq 1 ,
• Jian Ping Li 1 ,
• Shakir Khan 2 ,
• Mohammed Ali Alshara 2 ,
• Reemiah Muneer Alotaibi 2 &
• CobbinahBernard Mawuli 1  

Scientific Reports volume  12 , Article number:  15331 ( 2022 ) Cite this article

10k Accesses

57 Citations

10 Altmetric

Metrics details

• Computational biology and bioinformatics
• Engineering
• Mathematics and computing

The classification of brain tumors (BT) is significantly essential for the diagnosis of Brian cancer (BC) in IoT-healthcare systems. Artificial intelligence (AI) techniques based on Computer aided diagnostic systems (CADS) are mostly used for the accurate detection of brain cancer. However, due to the inaccuracy of artificial diagnostic systems, medical professionals are not effectively incorporating them into the diagnosis process of Brain Cancer. In this research study, we proposed a robust brain tumor classification method using Deep Learning (DL) techniques to address the lack of accuracy issue in existing artificial diagnosis systems. In the design of the proposed approach, an improved convolution neural network (CNN) is used to classify brain tumors employing brain magnetic resonance (MR) image data. The model classification performance has improved by incorporating data augmentation and transfer learning methods. The results confirmed that the model obtained high accuracy compared to the baseline models. Based on high predictive results we suggest the proposed model for brain cancer diagnosis in IoT-healthcare systems.

Similar content being viewed by others

Brain tumor detection from images and comparison with transfer learning methods and 3-layer CNN

Automatic brain-tumor diagnosis using cascaded deep convolutional neural networks with symmetric U-Net and asymmetric residual-blocks

Detection and classification of brain tumor using hybrid deep learning models

Introduction.

Brain tumor (BT) is a series medical problem, and many people are suffering from it globally 1 . Because of its critical nature, brain tumours are one of the most dangerous types of brain cancer. Compared to other cancers from brain cancer less number of people are suffering 2 . Meningioma, Glioma, Pituitary, and Acoustic Neuroma are examples of brain tumors. In medical observation, the rates of Meningioma, GLioma, and Pituitary tumours in all brain tumors are 15%, 45%, and 15%, respectively 3 . A brain tumor has long-term and psychological consequences for the patient. Brain tumors are caused by tissue abnormalities that develop within the brain or the central spine, interfering with normal brain function. There are two types of brain tumors: benign and malignant. Benign brain tumors are not cancerous and grow slowly. They do not spread and are not common. Malignant brain tumours contain cancer cells and grow rapidly in one region of the brain and spread to other parts of the brain and spine.

The diagnosis of brain cancer is significantly necessary for early stage for effective treatment and recovery. In this regards to classify brain tumors and identify brain cancer, different non-invasive are developed in literature by researchers and medical experts in Internet of Things (IoT) healthcare industries. In the deigning of computer automatic diagnostic systems (CADS) for brain cancer detection Machine Learning (ML) and Deep Learning (DL) models are commonly used. The diagnosis of brain cancer using images data using the DL Convolution neural network (CNN) model has grown in popularity, and the CNN model is commonly used for image classification and analysis, particularly for medical image data analysis 4 . The CNNs model can extract more related features from data for accurate image classification 2 , 5 , 6 . Furthermore, data augmentation and transfer learning techniques can also improve the predictive capability of deep learning models to effective classify the brain tumors and diagnosis brain cancer in IoT healthcare industries 6 , 7 .

In the literature, various methods have been proposed for brain cancer diagnosis using ML and DL learning approaches by different scholars. Zacharaki et al. 8 designed a brain cancer diagnosis system to classify various grades of Glioma employing SVM and KNN machine learning model and respectively achieved 85% and 88% classification accuracy. Cheng et al. 9 proposed a classification approach for brain tumor classification and augmented the tumor region for improving the classification performance. They employed three techniques for feature extraction such as Gray level co-occurrence matrix, a bag of words, and an intensity histogram. Their proposed method obtained 91.28% classification accuracy.

Haq et al. 6 proposes an AI-based intelligent integrated framework (CNN-LSTM) for brain tumors classification and diagnosis in the IoT healthcare industry. In the integrated framework design, they have incorporated the CNN model to extract features from medical MRI data automatically. The extracted features are passed to Long short-term memory (LSTM) model to learn the dependencies in the features and finally predict the class for the tumor. Further they applied brain MRI data sets for the assessment of the proposed integrated model. Massive data is one requirement for an effective deep learning model. Since the size of our original data set is small, they utilized data augmentation approaches to increase the data set size, thereby improving the model result during training. Also used the train-test splits Cross-validation approach for hyperparameter tuning and best model selection to ensure proper model fitting. For model assessment, used well-known evaluation measures. They compared the predictive outputs of the proposed CNN-LSTM model with previous methods in the Medical Internet of Things (MIoT) healthcare industry and the model obtained high predictive performance.

Paul et al. 4 employed axial brain tumor images for convolution neural network training. In the proposed method they used two convolution layers, two max-pooling layers, and lastly, two fully connected layers for the final classification process. The proposed approach obtained 91.43% classification accuracy. El-dahshan et al. 10 designed a brain tumors classification method for 80 brain images MRI classification. They used discrete wavelet transform and PCA algorithms for reducing dimensions of data. To classify the normal and abnormal tumors, they used ANN and KNN machine learning classifiers. The classifiers ANN and KNN, achieved 97% and 98% classification accuracy respectively.

In another study, Afshar et al. 11 proposed a brain tumor classification method employing a capsule network that combined MRI images of the brain and coarse tumor boundaries and 90.89% accuracy achieved by the proposed method. Anaraki et al. 12 developed an integrated framework for brain tumor classification, and in the proposed technique, they integrated CNN and GA, and designed GA-CNN framework and obtained 94.2% accuracy. Khan et al. 13 proposed brain tumors classification method employing transfer learning techniques (CNN-Transfer learning) and achieved 94.82% accuracy 14 . The proposed multi-classification method employing ensemble of deep features and ML algorithms and obtained high performance.

According to the review of the literature, current brain cancer diagnosis techniques still lack a robust predictive capability in terms of accuracy to correctly diagnose brain cancer for proper treatment and recovery. To address this issue, a novel robust method for accurately diagnosing brain cancer for proper treatment and recovery in IoT healthcare industries is required. Furthermore, the artificial intelligence based brain cancer diagnosis systems also reduce the financial costs of healthcare department.

In this study, we created an improved CNN model for the classification of brain MR images to diagnosis brain cancer in IoT healthcare industries. In the development of the proposed model, we used Convolution neural network model to classify brain tumors types (Meningioma, Glioma and Pituitary) employing MR images data. The CNN model is more suitable for the Meningioma, Glioma, and pituitary classification using brain tumors images data and its extract more deep features from images data for final classification. To further improve the CNN model predictive capability, we have incorporated a transfer-learning (TL) techniques for proper training of the CNN architecture, the brain MR images data is insufficient. In transfer learning, we used the well-known pre-trained models ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet. The weights generated of these pre trained models individually transferred to CNN architecture for effective training o CNN model. For the fine-tuning process, the model was trained with brain MR images data set. The generated weights of pre trained models improving CNN model final predictive performance. Additionally, the data augmentation technique is incorporated to increase the data set size for effective training of the model. We also used held-out cross-validation (CV) and performance evaluation metrics. The performance of the model compared with base lines models. The experimental results confirmed that the proposed model generated higher predictive results and it could be applied in IoT-healthcare systems easily.

Innovations of this study summarized as follows:

In IoT healthcare systems, an improved model based on CNN and TL for classifying brain tumors using MR image data is proposed for diagnosis of brain cancer.

To increase the predictive accuracy of the CNN model, TL techniques are used because the brain tumor image data is insufficient for effective training of the CNN model. Pre-trained models ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet are used to train with the well-known ImageNet data set for generating trained parameters (weights). The weights of these pre tained models are individually transfer to CNN model effective training. Fine-tuning the model CNN with brain tumor images data along with transferred weights final classification.

To improve model performance, the data augmentation technique is used to increase the size of the data set for effective model training.

When compared to baseline methods, our model has a high predictive performance.

The rest of the paper is organized as follows: In “ Materials and method ” section data set and proposed model methodology have explored. In “ Experiments ” section the experiments are reported. In “ Discussion ” section, we discussed the significance of the work. The conclusion and research direction of future work are reported in “ Conclusion ” section.

Materials and method

We used a brain tumor data set (BTDS) from China’s Nanfang hospital and general hospital, as well as Tianjing medical university, in this study (2005 to 2010) 9 , and new versions in 2017 have been published. T1-Weighted Contrast-Enhanced images (TWCEI) of 233 subjects with meningioma, glioma, and pituitary tumours are included in this data set. The data set is freely accessible via the Kaggle repository 15 . We also used the Brain MRI Images Data Set (BMIDS) for cross dataset validation, which contains 253 MRI brain images. The tumor class in the data set has 155 images, while the non-tumor class has 98 images 16 .

Background of convolutional neural network (CNN) architecture

Deep Learning model convolutional neural networks is a kind of Feed-Forward Neural Network 17 . Convolutions can capture translation invariance, which means that the filter is independent of position that significantly reduces the number of parameters. The CNN model have Convolutional, Pooling, and fully connected layers. Different functions are accomplished by these layers, such as dimensionality reduction, feature extractors, and classification. During the convolution operation of the forward pass, the filter is slide on the input shape and compute the map of activation, which computing the point-wise value of each output. Further add these output to achieve the activation of that point. Designed a Sliding Filter (SF) using convolution as a linear operator, and expressed as a dot product for fast deployment. Let consider x and w are input and the kernel function, the convolution process \((x*w)(a)\) on time index t can be mathematically expressed in Eq. ( 1 ).

In Eq. ( 1 ) a is in \(\text{ R}^n\) for any \(n \ge 1\) . While Parameter t is discrete. In this case, the discrete convolution can be expressed as in Eq. ( 2 ):

However, usually use 2 or 3-dimensional convolutions in CNN model. In case of 2-dimensional image I as input and K is a two dimensional kernel and the convolution can be mathematically expressed as in Eq. ( 3 ):

If the case is 3 dimensional data image, then the convolution process can be written mathematically in Eq. ( 4 ) as follow:

In addition to gain non-linearities, two activation functions can be incorporate suc as Sigmoid and ReLU. The sigmoid activation fumction non-linearity is expressed mathematically in Eq. ( 5 ):

The sigmoid non-linearity activation function is suitable when need the output to be include in the range of [0,1]. Furthermore, the sigmoid function is monotone growing which means \(\lim \limits _{n \rightarrow +\infty } \theta (x)=1\) , and \(\lim \limits _{n \rightarrow +\infty } \theta (x)=0\) . However, this fact may be cause vanishing gradients, when the input x is not near to 0, the neuron will be more and the gradient of \(\theta (x)\) will nearly to zero and will make successive optimization difficult.

The second activation function is relu which is mathematically defined in Eq. ( 6 ):

The gradient of of \(relu(x)=1\) for \(x>0\) and \(relu^-(x)=0\) for \(x<0\) . The relu convergence capability of is good then sigmoid non-linearities.

The CNN model Pooling layers are utilized to produce a statistics summary of its inputs and deduced the dimensionality without missing important information. There are different types of pooling. In the layer of Max-Pooling generate the extreme values in individually rectangular neighborhood of individual point i.e i, j, k for data of three dimensional of individual feature of input respectively, while the average values generated by the average pooling layer.

The last layer is fully connected with n and m respectively input and output sizes. The output layer is expressed by the parameters such as a weight matrix i.e \(W \in M_{m, n}\) with m rows, and n columns and a bias vector \(b \in {\textbf {R}}^m\) . The input vector \(x \in {\textbf {R}}^n\) , the fully connected output layer FC along function of activation f is expressed mathematically in Eq. ( 7 ) as:

In Eq. ( 7 ) Wx is the product matrix while the function f is used component wise.

The last layers fully connected employed for classification of problems. The CNN model architecture last layer is fully connected layers and CNN output is flattened and showed as a single vector.

Convolution neural network for brain tumors classification

Recently, CNN models generated significant outcomes in numerous domains, such as NLP, image classification 18 , and diagnosis systems. In contrast to MLPs, CNN reduces the number of neurons and parameters, which results in lower complexity and faster adaptation.

The CNN model has significant applications in the classification of medical images 18 , 19 . In this paper we developed the CNN networks architecture with 4 alternating convolutional layers and max-pooling layers and a dropout layer after each Conv/pooling pair. The last pooling layer connected fully layer with 256 neurons, ReLU activation function, dropout layer, and sigmoid activation function are employed for classification of brain MR images (Meningioma, Glioma, and Pituitary). In addition, we have used the optimization algorithm Stochastic Gradient Descend (SGD) 20 . The CNN architecture is given in Fig. 1 .

CNN model architecture for classification of Brain tumors.

Improve CNN model for brain tumors classification

To improve CNN model predictive accuracy, we employed Data augmentation (DA) and Transfer learning (TL) techniques. The data augmentation can resolve the problem of insufficient data for model training. To expand the data amount, the zooming technique is used on original image data to produce images data with the similar label. The new created data set is used for fine tuning of the model. Th The transfer learning (TL) techniques widely used in image classification tasks 21 , cancer sub-type recognition 22 and medical images filtering 23 . In this work, we used the transfer learning ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet models to enhanced the predictive performance of the proposed CNN model. The ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet pre-train models were trained on imageNet data set and transferred the trained parameters weights of these models individually to CNN model for effective training, and fine-tuned the model using the brain tumor augmented MR images data set for final classification of the CNN model.

Model cross validation and evaluation criteria

The holdout cross-validation 6 , 24 , 25 mechanism was used for training and validation of the model. In hold out CV data is randomly assign to two sets \(d_0\) and \(d_1\) . The \(d_0\) and \(d_1\) use for training and testing of the model respectively. In hold out CV the training data set is usually large as compare to testing data set. The is train on \(d_0\) and testing on \(d_1\) . The holdout CV is suitable validation method in case when the data set is very plenty. In this study brain tumor MRI Images data set was divided into 70% for training and \(30\%\) for teasing of the model. The performance evaluation metrics Accuracy (Acc), Sensitivity (Sn), Specificity (Sp), Precision (Pr), F1-Score (F1-S), and Matthews Correlation Coefficient (MCC) 26 , 27 , 28 , 29 are used for model evaluation.

Proposed brain tumors classification model

NCNN models are now popular for image classification problems. A large image data set is more suitable for the CNN model’s effective training, as it allows the model to extract more related features during the training process for accurate image classification. The CNN model’s performance suffers as a result of the scarcity of large image data sets, particularly in the medical domain. However, to enhance the proposed CNN classifier performance, data augmentation and transfer learning 6 , 21 , 30 , 31 techniques are incorporated. We have used transfer learning pre-trained models ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet along with data augmentation technique zooming. The imagesNet data set has been employed for pre-trained of ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet models, and the generated weights (trained parameters) of these models were transferred for the effective training of the CNN model individually. Brain tumor MRI data set was used for fine-tuning of CNN model and for final classification of the model in IoT healthcare system.

Furthermore, the proposed CNN model was trained and tested on a data set of brain tumour MR images, and its performance was compared to that of the transfer learning technique. A heldout cross-validation mechanism is used in the proposed method for model training and testing, with 70% used for training and 30% for model validation. The data augmentation 20 technique was used to augment the original dataset by using the zooming method, which improves the model generalisation capability. The integration of data augmentation and transfer learning greatly enhanced the predictive accuracy of the CNN model. The evaluation criteria of the model different assessment metrics have used.

The data set X ( i ,  i ) embedded into the CNN classifier,We used data transformations to increase the size of the data set so that we could train the model. Furthermore, the number of epochs E , model parameters w , Learning Rate (LR) \(\eta\) , size of batch b , and the number of layers in both CNN were configured accordingly. For the optimization of our model parameters, we have used the stochastic gradient descent algorithm (SGD). The pseudo-code of the proposed model is given in algorithm 1 and flow chart in Fig. 2 .

Flow chart of proposed tumor classification framework in IoT healthcare systems. The pre trained CNN models (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) are trained with image-net dataset and the generated weights of these pre trained models are individually transferred to proposed CNN model for effective training. While the augmented data set is used for fine-tuning of the ResNet-CNN model for final classification of brain tumors.

Experiments

Experimental setup.

We conducted various experiments to test the feasibility of our proposed model in IoT healthcare system. The proposed model was tested using a brain tumour image data set in this study. To improve the proposed CNN model predictive performance, we have employed (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) CNN pre trained models with imagenet dataset to produce high trained parameters (weights) and then transferred trained parameters weights of pre trained models to the CNN model individually for effective training of the model. For fine-tuning the CNN model, the brain tumor images data set was employed for final classification. The brain tumor data have 233 subjects and 3064 slices, which belong to three classes, i.e., Meningioma, Glioma, and Pituitary. This data set is very Small for effective training of the CNN model. In addition to tackle the problem of small brain tumor data method of data augmentation 20 has used to augment the original data set. Data augmentation technique (zooming) is used, and all three types of images (Meningioma, Glioma, and Pituitary) are zoomed horizontally and vertically and added with existing images. The new augmented data set image size of three kinds of images is 6128. Held out technique is used for model training and validation, and respectively 70% and 30% data are employed for training and validation of the model for all experiments. To effectively optimize the model SGD Optimization algorithm is used 20 . In addition, other parameters such as learning rate (LR) \(\alpha\) , SGD = 0.0001, epochs = 100, batch size = 120, outer and inner activation function = ReLu is used in all experiments. It is worth noting that for the final prediction layer our CNN model, the softmax activation function was used. Evaluation metrics are incorporated to evaluate the model performance.

All experiments used a laptop and a Google collaborator with GPU. All experiments required Python v3.7, and the CNN model was created using Keras framework v2.2.4 as a high-level API and Tensor flow v1.12 as the back end. All experiments were repeated numerous times to obtain consistent results. All experiment results were tabulated and graphed.

Results and analysis

Results of data pre-processing.

The brain tumor data set (BTDS) is obtained from the Kaggle repository 15 . T1-weighted contrast-enhanced images of 233 meningioma, glioma, and pituitary tumour patients are included in this data set. The Brain Tumor data contains 233 subjects and 3064 slices, with meningioma subjects accounting for 82 with slices 708, glioma subjects accounting for 91 with slices 1426, and pituitary subjects accounting for 60 with slices 930. Thus, the total number of subjects in the data is 233, and the total number of slices is 3064. In order to reduce the dimension of \(512\times 512\times 1\) into \(224\times 224\times 1\) for effective training of model.

To handle imbalance problem in data set because Brain tumor data set has the different number of three subjects slices. The distribution of the data is different, and it creates a problem of over fitting the model. To balance the meningioma, glioma, and pictutitary in the data set, we incorporate the data augmentation 20 method to augment the original dataset by using random zooming. All slices are being zoomed, and a new data set with 6128 slices has been created. The ratio of samples in an original data set is shown in Fig. 3 . The data set has three subfolders for meningioma, glioma, and pictutitary images. Held out techniques is used for model training and validation because the new data set is very big and heldout validation is suitable in case of plenty dataset. The data set has splitted into 70% and 30% for training and validation of the model respectively. The cross-validation method has also been employed for an augmented data set.

We also used the Brain MRI Images Data Set (BMIDS) for cross dataset validation, which contains 253 MRI brain images. The tumor class in the data set has 155 images, while the non-tumor class has 98 images.

Ratio of samples in data set.

Results of the proposed CNN model, on original and augmented data sets

The performance of the proposed CNN model is evaluated using the original and augmented brain tumour MR image data sets. The CNN model is configured with essential hyper-parameters such as optimizer SGD with a Learning Rate (LR) of 00.0001, epochs 100, and size of batch was 120. The 70% data for training and 30% for the testing of the model is used. Different evaluation matrices were used for model performance evaluation. The input image size \(264\times 264\times 1\) is used for training and evaluation of the proposed CNN model. All these hyper-parameters values and the output of the experimental results have been reported in Table 1 .

Table 1 presented the proposed CNN model obtained 97.40% accuracy, 98.03% specificity, 95.10% sensitivity, 99.02% Precision, 97.75% MCC, and 97.26% F1-score respectively on original brain tumor MR images data set. The 97.40% accuracy demonstrated that our CNN architecture accurately classifies the three classes of brain tumors (meningioma, glioma, and pictutitary). The 98.03% specificity shows that the Proposed CNN model is a highly suitable detecting model for healthy subjects recognition, while 95.10% sensitivity presents that the model significantly detected the affected subjects. The MCC value was 97.75%, which gives confusion metrics a good summary.

On the other hand, the CNN model gained very excellent performance when trained and evaluated on an augmented data set. The CNN model obtained 98.56% accuracy, 100.00% specificity, 98.09% sensitivity, and 98.00% MCC when trained and evaluated on an augmented data set. The accuracy of the model improved from 97.40 to 98.56% which demonstrated the importance of the data augmentation process. Also, it illustrated that model needs more data for effective training of the CNN model.

From the experimental results, we concluded that the proposed CNN model effectively classified the brain tumor types, and the augmentation process further improved the model CNN performance because the CNN model more data for extract more related features for classification. The high accuracy of the proposed CNN model might be due to the suitable architecture of the CNN model and proper fitting of essential parameters of the model and data augmentation.

CNN model performance evaluation with cross dataset

We have evaluated the predictive performance of CNN model with independent cross dataset. We trained the proposed CNN model with original and augmented brain tumor data set and validated with independent Brain MRI Images Data Set (BMIDS). The model is configured with essential hyper-parameters such as optimizer SGD with a Learning Rate (LR) of 00.0001, epochs 100, and size of batch was 120. Different evaluation matrices were used for model performance evaluation. The input image size \(264\times 264\times 1\) is used for training and evaluation of the proposed CNN model. The experimental results of model with cross data are reported in Table 2 .

Table 2 presented that the proposed CNN model obtained 97.96% accuracy, 99.00% specificity, 97.30% sensitivity, 98.18% Precision, 98.00% MCC, and 99.02% F1-score when trained on original brain tumor MR images data set (BTDS) and validated with independent data set (BMIDS).

Other other side the model achieved 98.97% accuracy, 99.89% specificity, 99.39% sensitivity, 98.89% Precision, 99.40% MCC, and 99.30% F1-score when trained with augmented data set (BTDS) and validated with independent data set (BMIDS). Hence, from experimental results we observed that model predictive and generalization capability improved when trained and validated with independent data sets.

Results of the transfer learning models (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) on original and augmented data sets

The performances of transfer learning (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) models have checked on original and augmented data sets. Theses models have configured other essential hyper-parameters such as optimizer SGD with learning rate 0.0001, the number of epoch 100, batch size 120. The input image size \(264\times 264\times 1\) is used for training and evaluation of the proposed model. The (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) models are evaluated using different performance evaluation metric. The (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) models hyper-parameters values and the output of the experimental results have reported in Table 3 .

Table 3 show that the ResNet-50 model obtained 97.03% accuracy, 97.04% specificity, 93.10% sensitivity, 94.21% Precision, 93.23% MCC, and 95.00% F1-score respectively on original brain tumor data set. The 95.30% accuracy show that the ResNet-50 model accurately classifies the three classes of brain tumors (meningioma, glioma, and pictutitary). The 97.04% specificity shows that the ResNet-50 model is a highly suitable detecting model for healthy subjects recognition, while 93.10% sensitivity show that the model accurately detected the affected subjects.

The predictive Performance of transfer learning model ResNet-50 very high when model trained and evaluated with augmented data set. According to Table 3 the transfer learning model ResNet-50 obtained 98.07% accuracy, 99.30% specificity, 100.00% sensitivity, 96.07% precision, 96.00% MCC, and 97.00% F1-S, when trained and evaluated on augmented data set.

The VGG-16 model with original and augmented data sets obtained 94.77% accuracy, 96.30% specificity, 94.67% sensitivity, 93.43% precision, 91.90% MCC, 96.61% F1-S, and 95.97% accuracy, 96.95% specificity, 99.40% sensitivity, 96.84% precision, 92.98% MCC, and 96.80% F1-S respectively.

Inception V3 obtained 93.23% accuracy, 96.89% specificity, 95.00% sensitivity, 96.08% precision, 95.56% MCC, and 97.87% F1-s, with original data set. While on augmented data set Inception V3 obtained 96.03% accuracy, 97.03% specificity, 97.00% sensitivity, 97.01% precision, 96.05% MCC, 98.00% F1-S. DenseNet201 model obtained 96.76% accuracy on original data set and increase it 97.43% accuracy with augmented data set.

The Xception model with original data set achieved 93.00% accuracy, 97.03% specificity, 98.00% sensitivity, 97.09% precision, 99.32% MCC, 97.23% F1-S and obtained 95.60% accuracy, 98.98% specificity, 96.00% sensitivity, 98.04% precision, 99.98% MCC, and 98.00% F1-S with augmented data set. MobilleNet model obtained 96.76% accuracy with original data set and 97.87% with augmented data set. Among all models the ResNet-50 model performance in terms of accuracy is high with augmented data set. The model improved accuracy from 95.30 to 98.07% with data augmentation. The other evaluation metrics values also improved with data augmentation. From the experimental results, we concluded that the data augmentation process increased the training of ResNet-50 and model effectively classified the brain tumor types.

Results of the integrated frameworks (ResNet-50-CNN, VGG-16-CNN, Inception V3-CNN, DenseNet201-CNN, Xception-CNN, and MobilleNet-CNN) on original and augmented data sets

The integrated frameworks (ResNet-50-CNN, VGG-16-CNN, Inception V3-CNN, DenseNet201-CNN, Xception-CNN, and MobilleNet-CNN) performances have checked on original and augmented data sets. Furthermore, we have incorporated the TL ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet CNN architectures with imageNet data set to generate high weights and then transferred trained parameters weights of these pre trained models to the CNN model individually for effective training of CNN model. For fine-tuning of the CNN model, the brain tumors original and augmented data sets have used for final classification. The models have configured with concern hyper-parameters such as optimizer SGD with learning rate 0.0001, the number of epoch 100, batch size 120. The proposed framework performance has been evaluated employing various matrices. The input image size \(264\times 264\times 1\) has been used for training and evaluation of the proposed model. All these hyper-parameters values and the output of the experimental results of (ResNet-50-CNN, VGG-16-CNN, Inception V3-CNN, DenseNet201-CNN, Xception-CNN, and MobilleNet-CNN) models have reported in Table 4 .

Table 4 presented that the ResNet50-CNN model obtained 99.10% accuracy, 100.00% specificity, 89.60% sensitivity, 98.75% Precision, 98.66% MCC, and 99.5% F1-score respectively on original brain tumor data set. The 99.10% accuracy demonstrated that the architecture accurately classifies the three classes of brain tumors (meningioma, glioma, and pictutitary). The 100% specificity shows that the Proposed model is a highly suitable detecting model for healthy subjects recognition, while 89.60% sensitivity presents that the model significantly detected the affected subjects.

On the other hand, the model obtained very high performance when it trained and evaluated on the augmented data set. The integrated CNN and transfer learning model (ResNet-50-CNN) obtained 99.90% accuracy, 99.08% specificity, 96.13% sensitivity, and 99.10% MCC when trained and evaluated on augmented data set.

The VGG-16-CNN model with original and augmented data sets obtained 96.78% accuracy, 99.23% specificity, 95.00% sensitivity, 96.99% precision, 98.93% MCC, 97.98% F1-S, and 97.88% accuracy, 98.00% specificity, 100.00% sensitivity, 96.98% precision, 98.79% MCC, and 99.00% F1-S respectively.

Inception V3-CNN model obtained 97.00% accuracy, 99.00% specificity, 99.87% sensitivity, 98.92% precision, 95.76% MCC, 98.09% F1-S with original data set. While on augmented data set Inception V3 obtained 98.02% accuracy, 100.00% specificity, 98.67% sensitivity, 97.56% precision, 99.00% MCC, and 97.30% F1-S.

DenseNet201-CNN model obtained 97.00% accuracy on original data set and increase it 97.90% accuracy with augmented data set. Hence, the integrated model DenseNet201-CNN improved accuracy 97.00–97.90% = 0.90% with data augmentation process.

The Xception-CNN model with original data set achieved 98.20% accuracy, 98.88% specificity, 97.40% sensitivity, 99.00% precision, 99.10% MCC, 98.65% F1-S, and obtained 98.97% accuracy, 99.00% specificity, 98.60% sensitivity, 97.24% precision, 97.99% MCC, 99.30% F1-S with augmented data set. MobilleNet-CNN model obtained 98.08% accuracy with original data set and 98.56% with augmented data set. The improved accuracy 98.08% to 98.56% when model fine tuned with augmented data set.

From above anlaysis we conculded that among all the ResNet-50-CNN, VGG-16-CNN, Inception V3-CNN, DenseNet201-CNN, Xception-CNN, and MobilleNet-CNN, the predictive performance of ResNet-50-CNN model is high in terms of accuracy. The accuracy of the model improved from 99.10 to 99.90% which is illustrated the importance of the data augmentation and transfer learning process. Hence we concluded that the ResNet-50-CNN model effectively classify the brain tumor types. The high accuracy of the proposed integrated diagnosis framework might be due to the suitable architecture of the model and proper fitting of essential parameters of the model and data augmentation. In addition, the proposed integrated model (ResNet-50-CNN) accuracy has compared with CNN model and transfer learning ResNet-50 model in Table 5 on augmented data set and graphically shown in Fig. 4 .

CNN, ResNet-50 and Integrated (ResNet-50-CNN) models accuracy comparison with augmented Brain tumor data set. The CNN model obtained accuracy’s with augmented data is 98.97%, while ResNet-50 obtained 96.07% and Integrated model ResNet-CNN obtained high predictive accuracy 99.90% with augmented data. Thus, the proposed integrated ResNet-CNN model is suitable for effective classification of brain tumors and could assist clinical professionals to diagnosis brain cancer accurately and efficiently. Due to the high performance of proposed ResNet-CNN method we recommend it for diagnosis of brain cancer in IoT-healthcare.

Accuracy comparison of the proposed (ResNet-CNN) model with state of-the-art models

We have compared our ResNet-50-CNN (ResNet-CNN) model performance in terms of accuracy with state-of-the-art methods in Table 6 . Table 6 and Fig. 5 presented that proposed model obtained 99.89% accuracy, which is high as compared to state-of-the-art techniques. The high performance of the proposed method demonstrated that it is correctly classified brain tumors (meningioma, glioma, and pictutitary), and it can easily be deployed in IoT-health care for the classification of brain tumors.

ResNet-CNN model performance comparison with baseline models show that our model predictive performance in terms of accuracy is high from baseline models. The ResNet-CNN model cloud accurately and efficiently classify the brain tumors and assist medical experts to interpret the images of brain tumors to diagnosis brain cancer.

Space and time complexity

Also, in Tables 3 , 4 , and 6 , we present both the models space and complexity of the various proposed methods used in the prediction of Brain cancer. Since the proposed models are convolutional deep learning methods, the space complexities are analyzed in terms of the each model’s trainable parameters. For the time complexity, the model’s training time is used. It could be deduced from Table 3 that VGG-16 has the worst space complexity since its trainable parameter is 138.4 million, whiles MobileNet has the best space time complexity. Moreover for the time complexity, the Xception model has the worst time complexity because its training time is 4.3 h. Because of the difficulty of accessing the models of the competing methods in Table 4 , we could not experimentally analyze the complexity of the models in terms of algorithmic run-time. It is more likely that almost all the methods with the deep learning techniques, the convolutional neural networks will have a worse space and time complexity because of the significant number of parameters and matrix computation that come with the models’ architecture. Irrespective of the worst case time and space complexity, our proposed model has an accuracy performance gain as compared to all competing methods. The time complexity is the training time (in hours) of the models as reported in Tables 3 , 4 , and 6 . The space and time complexity of our model are \(\mathscr {O}(cwh + 1)f\) and \(\mathscr {O}(f*u*m)\) respectively.

Brain Tumor Classification using MR images are critical in the detection of brain cancer In IoT healthcare systems. Artificial intelligence (AI) based computer automatic diagnostic systems (CAD) can effectively different diagnose diseases in IoT healthcare system. Deep learning techniques are widely used in CAD systems to diagnose critical diseases 32 , especially convolutional neural networks. The CNN model is mostly used for medical image classification 18 , 19 . The CNN model extracts deep features from image data, and these features played an important role in final image classification. For the diagnosis of brain cancer, various methods have been proposed by researchers using brain MR image data and deep learning models. However, these existing methods have lack of accuracy of diagnosis. In order to tackle this problem, a new method is necessary to diagnose the disease accurately and efficiently IoT healthcare systems.

In this study, we have proposed a CNN model for the accurate classification of brain tumor using Brain MR images. In the design of the proposed method, we have applied the deep learning CNN model for the classification of tumors meningioma, gLioma, and pituitary. The CNN model extracts more deep features from image data for final classification. To further improve the CNN model predictive capability, we have incorporated a transfer learning mechanism because, for proper training of the CNN architecture, the brain MR images data is insufficient. In transfer learning, we have used the well-known pre-trained models (ResNet-50, VGG-16, Inception V3, DenseNet201, Xception, and MobilleNet) with big imageNet data set to generate high parameters (weights). These generated weights of models individually transferred to CNN model for effective training. For the fine-tuning process, the model was trained with brain MR images data set. Also, the data augmentation method is employed to increase the data set size for effective training of the model. Furthermore, we have used held-out cross-validation and performance evaluation metrics. We also used cross data set for cehcking the propoed CNN model predictice performance.

According to Tables 2 , 3 , 4 and 6 the proposed method obtained high results as compared to baseline methods. The high performance of the proposed ResNet-CNN model might be due to the proper setting of model parameters such as learning rate, batch size, number of the epoch, and pre-processing, and data augmentation. We recommend the proposed method for meningioma, gLioma, and pituitary classification. Furthermore, the proposed method would be applied for diagnosis of a brain cancer in IoT-Healthcare systems easily.

For accurate medical image classification, the CNN model is played a significant role, and in most CAD systems CNN model is used for the analysis of medical image data. In research study, we have proposed a deep learning-based diagnosis approach for brain tumor classification. In the proposed method, we have used a deep CNN model for the classification of tumor types Meningioma, Glioma, and Pituitary employing brain tumor MR images data. To enhance the predictive capability of the CNN model, we have incorporated transfer learning and data augmentation techniques. The experimental results show that the proposed integrated diagnosis framework ResNet-CNN has obtained 99.90% accuracy as compared to baseline methods. The high predictive outcomes of the proposed method might be due to the effective pre-processing of data and the adjustment of other parameters of the model such as numbers of layers, optimizer and activation functions, transfer learning, and data augmentation. Due to the high performance of the proposed ResNet-CNN model, it could be applicable for the classification of brain tumors and diagnosis of brain cancer in IoT-Healthcare. In the future, we will use other brain tumors datasets and other deep learning techniques to diagnose brain tumors.

Data availibility

The data sets we used in this study are available on the kaggle machine learning repository at linked below: (1) Brain tumor dataset ( https://www.kaggle.com/datasets/awsaf49/brain-tumor ), and (2) Brain MRI Images for Brain Tumor Detection data set ( https://www.kaggle.com/datasets/navoneel/brain-mri-images-for-brain-tumor-detection ). All methods were performed in accordance with the relevant guidelines and regulations.

Roser, M. & Ritchie, H. Cancer. Our World in Data (2015). https://ourworldindata.org/cancer .

Khan, H. A., Jue, W., Mushtaq, M. & Mushtaq, M. U. Brain tumor classification in MRI image using convolutional neural network. Math. Biosci. Eng 17 , 6203–6216 (2020).

Article   MathSciNet   Google Scholar  

Swati, Z. N. K. et al. Content-based brain tumor retrieval for MR images using transfer learning. IEEE Access 7 , 17809–17822 (2019).

Article   Google Scholar  

Sultan, H. H., Salem, N. M. & Al-Atabany, W. Multi-classification of brain tumor images using deep neural network. IEEE Access 7 , 69215–69225 (2019).

Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

MATH   Google Scholar  

Haq, A. U. H. et al. IIMFCBM: Intelligent integrated model for feature extraction and classification of brain tumors using MRI clinical imaging data in IoT-healthcare. IEEE J. Biomed. Health Inform. https://doi.org/10.1109/JBHI.2022.3171663 (2022).

Article   PubMed   Google Scholar  

Haq, A. U. & Li, J. P. Stacking approach for accurate invasive ductal carcinoma classification. Comput. Electr. Eng. 100 , 107937 (2022).

Zacharaki, E. I. et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn. Reson. Med. 62 , 1609–1618 (2009).

Cheng, J. et al. Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLoS ONE 10 , e0140381 (2015).

El-Dahshan, E.-S.A., Hosny, T. & Salem, A.-B.M. Hybrid intelligent techniques for MRI brain images classification. Digit. Signal Process. 20 , 433–441 (2010).

Afshar, P., Plataniotis, K. N. & Mohammadi, A. Capsule networks for brain tumor classification based on mri images and coarse tumor boundaries. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 1368–1372 (IEEE, 2019).

Anaraki, A. K., Ayati, M. & Kazemi, F. Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms. Biocybern. Biomed. Eng. 39 , 63–74 (2019).

Swati, Z. N. K. et al. Brain tumor classification for MR images using transfer learning and fine-tuning. Comput. Med. Imaging Graph. 75 , 34–46 (2019).

Kang, J., Ullah, Z. & Gwak, J. MRI-based brain tumor classification using ensemble of deep features and machine learning classifiers. Sensors 21 , 2222 (2021).

Article   ADS   Google Scholar  

BT. Brain tumor. Kaggle: Machine Learning Repository. https://www.kaggle.com/awsaf49/brain-tumor (Accessed 12 Mar 2021) (2022).

BT. Brain tumor. Kaggle: Machine Learning Repository https://www.kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection (Accessed 10 Mar 2022) (2022).

Zhang, S., Yao, L., Sun, A. & Tay, Y. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. 52 , 5 (2019).

Google Scholar  

Cai, J., Lu, L., Xie, Y., Xing, F. & Yang, L. Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function. arXiv preprint arXiv:1707.04912 (2017).

Wolz, R. et al. Automated abdominal multi-organ segmentation with subject-specific atlas generation. IEEE Trans. Med. Imaging 32 , 1723–1730 (2013).

Goodfellow, I., Bengio, Y., Courville, A. & Bengio, Y. Deep Learning Vol. 1 (MIT Press, 2016).

Schwarz, M., Schulz, H. & Behnke, S. RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features. In 2015 IEEE International Conference on Robotics and Automation (ICRA) , 1329–1335 (IEEE, 2015).

Hajiramezanali, E., Dadaneh, S. Z., Karbalayghareh, A., Zhou, M. & Qian, X. Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data. arXiv preprint arXiv:1810.09433 (2018).

Bickel, S. ECML-PKDD discovery challenge 2006 overview. In ECML-PKDD Discovery Challenge Workshop , 1–9 (2006).

Haq, A. U., Li, J. P., Memon, M. H., Nazir, S. & Sun, R. A hybrid intelligent system framework for the prediction of heart disease using machine learning algorithms. Mob. Inf. Syst. 2018 (2018).

Li, J. P. et al. Heart disease identification method using machine learning classification in e-healthcare. IEEE Access 8 , 107562–107582 (2020).

Yurttakal, A. H., Erbay, H., İkizceli, T., Karaçavus, S. & Çinarer, G. A comparative study on segmentation and classification in breast MRI imaging. IIOAB J. 9 , 23–33 (2018).

Gallego-Ortiz, C. & Martel, A. L. Improving the accuracy of computer-aided diagnosis for breast MR imaging by differentiating between mass and nonmass lesions. Radiology 278 , 679–688 (2016).

Yang, Q., Li, L., Zhang, J., Shao, G. & Zheng, B. A new quantitative image analysis method for improving breast cancer diagnosis using DCE-MRI examinations. Med. Phys. 42 , 103–109 (2015).

Haq, A. U. et al. Detection of breast cancer through clinical data using supervised and unsupervised feature selection techniques. IEEE Access 9 , 22090–22105 (2021).

Khan, M. A., Javed, M. Y., Sharif, M., Saba, T. & Rehman, A. Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) , 1–7 (IEEE, 2019).

Shin, H.-C. et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35 , 1285–1298 (2016).

Haq, A. U. & Li, J. P. A survey of deep learning techniques based Parkinson’s disease recognition methods employing clinical data. Expert Syst. Appl. 208 , 118045 (2022).

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61370073), the National High Technology Research and Development Program of China, the project of Science and Technology Department of Sichuan Province (Grant No. 2021YFG0322).

Author information

Authors and affiliations.

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

Amin ul Haq, Jian Ping Li & CobbinahBernard Mawuli

College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh, 11432, Saudi Arabia

Shakir Khan, Mohammed Ali Alshara & Reemiah Muneer Alotaibi

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, A.U.H., and J.P.L.; methodology, A.U.H. and J.P.L. C.M.; software, A.U.H.; validation, A.U.H., and J.P.L.; C.M. formal analysis, A.U.H., and J.P.L.; investigation, A.U.H., and S.K.; resources, A.U.H., and J.P.L., S.K.; data curation, A.U.H., Mohammed Ali Alshara, C.M. and R.M.A.; writing-original draft preparation, A.U.H.; writing-review and editing, A.U.H., M.A.A., and S.K.; visualization, A.U.H.; supervision, J.P.L.; project administration, A.U.H.; funding acquisition, J.P.L. All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Amin ul Haq or Jian Ping Li .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Haq, A.u., Li, J.P., Khan, S. et al. DACBT: deep learning approach for classification of brain tumors using MRI data in IoT healthcare environment. Sci Rep 12 , 15331 (2022). https://doi.org/10.1038/s41598-022-19465-1

Download citation

Received : 04 November 2021

Accepted : 30 August 2022

Published : 12 September 2022

DOI : https://doi.org/10.1038/s41598-022-19465-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Advanced ai-driven approach for enhanced brain tumor detection from mri images utilizing efficientnetb2 with equalization and homomorphic filtering.

• A. M. J. Zubair Rahman
• Muskan Gupta
• Suresh Guluwadi

BMC Medical Informatics and Decision Making (2024)

Integrated approach of federated learning with transfer learning for classification and diagnosis of brain tumor

• Eid Albalawi
• Mahesh T.R.
• Ahlam Almusharraf

BMC Medical Imaging (2024)

Classification of brain tumor types through MRIs using parallel CNNs and firefly optimization

• Faxue Zhang

Scientific Reports (2024)

DDFC: deep learning approach for deep feature extraction and classification of brain tumors using magnetic resonance imaging in E-healthcare system

• Abdus Saboor
• Jian Ping Li
• Saad Abdullah Alajlan

Revolutionizing heart disease prediction with quantum-enhanced machine learning

• S. Venkatesh Babu
• Jeffin Gracewell

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Alzheimer's disease & dementia
• Arthritis & Rheumatism
• Attention deficit disorders
• Autism spectrum disorders
• Biomedical technology
• Diseases, Conditions, Syndromes
• Endocrinology & Metabolism
• Gastroenterology
• Gerontology & Geriatrics
• Health informatics
• Inflammatory disorders
• Medical economics
• Medical research
• Medications
• Neuroscience
• Obstetrics & gynaecology
• Oncology & Cancer
• Ophthalmology
• Overweight & Obesity
• Parkinson's & Movement disorders
• Psychology & Psychiatry
• Radiology & Imaging
• Sleep disorders
• Sports medicine & Kinesiology
• Vaccination
• Breast cancer
• Cardiovascular disease
• Chronic obstructive pulmonary disease
• Colon cancer
• Coronary artery disease
• Heart attack
• Heart disease
• High blood pressure
• Kidney disease
• Lung cancer
• Multiple sclerosis
• Myocardial infarction
• Ovarian cancer
• Post traumatic stress disorder
• Rheumatoid arthritis
• Schizophrenia
• Skin cancer
• Type 2 diabetes
• Full List »

share this!

August 29, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

High-resolution brain tumor mapping reveals possible reason why some patients don't respond to new drug

by Weizmann Institute of Science

The cells that make up cancerous brain tumors are extremely varied and sometimes create unique three-dimensional shapes. As far back as 1932, American neurosurgeon Percival Bailey attempted to label these cells and discovered that they can be divided into several families of cells with similar properties.

More than 90 years later, we still know precious little about the identities of cell groups that make up different kinds of brain tumors , these groups' organization and how they affect the course of the disease and the outcome of treatment. This is why the success rate for treatment of most brain cancers is typically not high.

Over the past decade, genetic sequencing technology that works at the single-cell level has made it possible to examine, in minute detail and in one fell swoop, thousands of cells in the same tissue, to understand which genes they express and to then categorize them and study the role of each group.

Scientists in Dr. Itay Tirosh's research group in the Weizmann Institute of Science's Molecular Cell Biology Department, in collaboration with Prof. Mario L. Suvà's lab at Massachusetts General Hospital, harnessed this technology in order to reexamine some of the unanswered questions in the field of brain tumors.

The most common type of primary brain tumor is the glioma, which originates from the support cells that assist our nerve cells . There are two main types of glioma tumors: those that are usually less aggressive and have a mutation in the gene encoding an enzyme called IDH, and those without this mutation, which are highly aggressive and known in medical terminology as glioblastoma.

In the past few years, researchers from Tirosh's lab have been using single-cell RNA sequencing to analyze the cellular composition of both kinds of tumors. They revealed that the tumor cells are divided into groups, each of which expresses a unique genetic program that determines the biological "state" of the cancer cells in this group.

Among other findings, the researchers discovered groups of cells that use their unique genetic programs to mimic normal brain cells.

In a study published in Cell , researchers from Tirosh's lab—led by Dr. Alissa Greenwald, Noam Galili Darnell and Dr. Rouven Hoefflin—harnessed technologies that make it possible to not only sequence the RNA on the single-cell level but also spatially map its expression.

This allowed them, for the first time, to identify which genes are uniquely expressed in each of the thousands of areas within a brain tumor. As a result, they were able to precisely map how glioblastoma and glioma tumors are organized. To conduct the study, they took biopsies from 13 patients with glioblastomas and from six patients with gliomas that had the IDH mutation.

The researchers' first discovery was that the groups of various cells within a glioma tumor are not distributed evenly across the tumor; rather, they are concentrated in various environments inside the growth. These microenvironments are not entirely homogenous. Cells from other groups were always found in proximity to other types of cells.

In the next stage of the study, the researchers checked whether there were groups of tumor cells that usually exist in proximity to each other. They discovered that the cells not only had preferred neighbors but also that these good-neighbor couplings were consistent in different patients.

Certain neighboring pairs imitated the natural behavior of brain tissue. For example, cells that imitate the parent cells of the oligodendrocyte support cell were found close to endothelial cells, which line the walls of blood vessels. This coupling also occurs in healthy tissue, since endothelial cells release substances that are vital for the survival and proliferation of oligodendrocyte precursor cells.

Similarly, cells that imitate neuron progenitor cells were found in the parts of the tumor that penetrated healthy brain tissue, just as progenitor cells in healthy tissue migrate when the tissue is regenerated.

Taking an overview to gain a fuller understanding of these couplings, the researchers realized that the cells created five distinct layers by organizing themselves into separate environments within the tumor. The innermost layer—the core of the tumor—is made up of necrotic cells, which do not receive enough oxygen to survive.

In the layer surrounding the necrotic core, the researchers found cells similar to embryonic connective tissue, as well as additional cells, including immune system cells responsible for causing inflammation. The third layer was primarily made up of blood vessels, endothelial cells forming blood vessel walls and additional immune system cells.

Cells in the two outer layers of the tumor don't suffer from a lack of oxygen. This enables groups of tumor cells that mimic healthy brain tissue—progenitors of neurons and support cells—to develop in the fourth layer.

The fifth, outermost layer contains healthy brain tissue, into which the tumor penetrates. These findings about the different layers of a tumor indicate that the driving force behind the tumor's layered structure is the lack of oxygen, which is exacerbated as the disease progresses and the tumor develops.

Based on these findings, the researchers noticed a much more chaotic structure in less aggressive tumors—which are also usually smaller—and in areas of the tumor with a plentiful supply of oxygen.

In most glioma tumors with the IDH mutation, for example, there was usually no necrotic tissue, and the structure of the tumor was disorganized; in the rare cases when there was necrotic tissue, the biopsies also showed a relatively well-ordered structure.

"We discovered that an organized spatial structure is characteristic of the more aggressive tumors," Tirosh explains.

"The lack of oxygen in the tumor cells ' environment influences the gene program that they express and therefore affects their state. As the tumor grows, distinct layers are formed, some of which may be less accessible to drugs and to cells from the immune system, and these could make the tumor more resilient."

The changing status of cancer cells

Researchers from Tirosh's lab used the information they collected on the cellular composition of glioma tumors to work out how a new, promising drug helped some of the patients with this type of cancer.

To do so, they used biopsies from tumors of three patients who had participated in a clinical trial of the new drug and who had responded to the treatment, as well as biopsies from six patients who had not undergone any treatment. To complete the picture, they also used data from biopsies taken from an additional 23 patients who had taken the drug and 134 patients who had not.

The research team, led by Dr. Avishay Spitzer, found that the drug , which works by inhibiting the mutant IDH enzyme, caused the cells to alter the gene program that they expressed. In fact, the treatment encourages the cancerous stem cells to differentiate into mature cells, thereby undermining their ability to divide rapidly, blocking the disease's progress.

The researchers postulated that if the drug works by causing cancerous cells to differentiate into mature cells, the mutation attacking the gene that is critical to the differentiation process could explain those cases in which the drug does not work.

In the biopsies taken from patients who did not receive the drug, they identified a certain gene that is linked to low levels of mature cancerous cells. When they silenced that gene in a mouse model of cancer, they found, as expected, that the drug did not work.

"This indicates that the gene mutation we identified could be a biological marker allowing us to determine in advance which patients will benefit from the treatment and which will not," Tirosh explains. These new findings could also help find a course of treatment that combines IDH inhibitors with another drug that encourages the differentiation process and increases the treatment's impact on the tumor.

"Our two most recent studies revealed the forces that shape the character of cancerous cells in a tumor, be that in their untouched environment or in one resulting from a therapy that alters the cells' genetic program," Tirosh says.

"These findings pave the way for a new approach to cancer treatment, since once we are familiar with the cell groups that populate every area of the tumor and we know how a cell can move from one state to another, we might be able to develop new targeted treatments that will alter the course of the disease.

"The understanding that both the composition of cells within the tumor and its three-dimensional structure are linked to the level of the tumor's aggressiveness could also lead to new diagnostic methods that do not rely solely on the volume of the tumor and the mutations it contains."

Explore further

Feedback to editors

How hunger influences aversive learning in fruit flies

20 hours ago

Study outlines an activity-regulated genetic program underlying the formation of synapses during development

Aug 31, 2024

Novel low-dose 3-in-1 blood pressure pill significantly outperforms standard care, study shows

Digital consultations found to improve the rate at which heart failure patients receive optimal medication

Researchers find 60% of infant and toddler foods sold in US do not meet desired nutritional standards

Single blood test predicts 30-year cardiovascular disease risks for women

Not just a 'bad guy': Researchers discover neuroprotective function of Tau protein

Aug 30, 2024

Study shows how common genetic variants in Black Americans increase Alzheimer's risk

Silicon exoskeletons for blood cells: Engineered blood cells successfully transfused between species

Neuroscientists explore the intersection of music and memory

Related stories.

Mapping cell behaviors in high-grade glioma to improve treatment

Nov 13, 2023

Study offers new insights into earliest stages of brain tumor development

Mar 12, 2024

Tumor heterogeneity and immune-evasive environment in malignant lymphoma at single-cell resolution

Dec 5, 2023

Using spatial single-cell transcriptomics to visualize diffuse midline glioma

Dec 5, 2022

A cerebrospinal fluid test to track tumor progression in some gliomas

Jan 24, 2019

Brain tumors in children: Cancer cells become less aggressive as they migrate within the tumor, finds study

Jan 8, 2024

Recommended for you

Immunotherapy research offers new hope for children with primary liver carcinomas

New anti-cancer 'degrader' targets protein essential to infant leukemia

Navigating the digestive tract: Study offers first detailed map of the small intestine

Aug 29, 2024

Gene therapy gets a turbo boost from researchers

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Medical Xpress in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Diagnostics (Basel)

Convolutional Neural Network Techniques for Brain Tumor Classification (from 2015 to 2022): Review, Challenges, and Future Perspectives

1 Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; [email protected] (Y.X.); [email protected] (F.Z.); [email protected] (R.L.); [email protected] (C.T.)

Fulvio Zaccagna

2 Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy; [email protected]

Leonardo Rundo

3 Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, 84084 Fisciano, Italy; ti.asinu@odnurl

Claudia Testa

4 Department of Physics and Astronomy, University of Bologna, 40127 Bologna, Italy

Raffaele Agati

5 Programma Neuroradiologia con Tecniche ad elevata complessità, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy; [email protected]

Raffaele Lodi

6 IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy

David Neil Manners

Caterina tonon, associated data.

Not applicable.

Convolutional neural networks (CNNs) constitute a widely used deep learning approach that has frequently been applied to the problem of brain tumor diagnosis. Such techniques still face some critical challenges in moving towards clinic application. The main objective of this work is to present a comprehensive review of studies using CNN architectures to classify brain tumors using MR images with the aim of identifying useful strategies for and possible impediments in the development of this technology. Relevant articles were identified using a predefined, systematic procedure. For each article, data were extracted regarding training data, target problems, the network architecture, validation methods, and the reported quantitative performance criteria. The clinical relevance of the studies was then evaluated to identify limitations by considering the merits of convolutional neural networks and the remaining challenges that need to be solved to promote the clinical application and development of CNN algorithms. Finally, possible directions for future research are discussed for researchers in the biomedical and machine learning communities. A total of 83 studies were identified and reviewed. They differed in terms of the precise classification problem targeted and the strategies used to construct and train the chosen CNN. Consequently, the reported performance varied widely, with accuracies of 91.63–100% in differentiating meningiomas, gliomas, and pituitary tumors (26 articles) and of 60.0–99.46% in distinguishing low-grade from high-grade gliomas (13 articles). The review provides a survey of the state of the art in CNN-based deep learning methods for brain tumor classification. Many networks demonstrated good performance, and it is not evident that any specific methodological choice greatly outperforms the alternatives, especially given the inconsistencies in the reporting of validation methods, performance metrics, and training data encountered. Few studies have focused on clinical usability.

1. Introduction

Brain tumors are a heterogenous group of common intracranial tumors that cause significant mortality and morbidity [ 1 , 2 ]. Malignant brain tumors are among the most aggressive and deadly neoplasms in people of all ages, with mortality rates of 5.4/100,000 men and 3.6/100,000 women per year being reported between 2014 and 2018 [ 3 ]. According to the 2021 World Health Organization (WHO) Classification of Tumors of the Central Nervous System, brain tumors are classified into four grades (I to IV) of increasingly aggressive malignancy and worsening prognosis. Indeed, in clinical practice, tumor type and grade influence treatment choice. Within WHO Grade IV tumors, glioblastoma is the most aggressive primary brain tumor, with a median survival after diagnosis of just 12–15 months [ 4 ].

The pathological assessment of tissue samples is the reference standard for tumor diagnosis and grading. However, a non-invasive tool capable of accurately classifying tumor type and of inferring grade would be highly desirable [ 5 ]. Although there are several non-invasive imaging modalities that can visualize brain tumors, i.e., Computed Tomography (CT), Positron Emission Tomography (PET), and Magnetic Resonance Imaging (MRI), the last of these remains the standard of care in clinical practice [ 6 ]. MRI conveys information on the lesion location, size, extent, features, relationship with the surrounding structures, and associated mass effect [ 6 ]. Beyond structural information, MRI can also assess microstructural features such as lesion cellularity [ 7 ], microvascular architecture [ 8 ], and perfusion [ 9 ]. Advanced imaging techniques may demonstrate many aspects of tumor heterogeneity related to type, aggressiveness, and grade; however, they are limited in assessing the mesoscopic changes that predate macroscopic ones [ 10 ]. Many molecular imaging techniques have recently been developed to better reveal and quantify heterogeneity, permitting a more accurate characterization of brain tumors. However, in order to make use of this wealth of new information, more sophisticated and potentially partially automated tools for image analysis may be useful [ 10 ].

Computer-aided detection and diagnosis (CADe and CADx, respectively), which refer to software that combines artificial intelligence and computer vision to analyze radiological and pathology images, have been developed to help radiologists diagnose human disease in several body districts, including in applications for colorectal polyp detection and segmentation [ 11 , 12 ] and lung cancer classification [ 13 , 14 , 15 ].

Machine learning has vigorously accelerated the development of CAD systems [ 16 ]. One of the most recent applications of machine learning in CAD is classifying objects of interest, such as lesions, into specific classes based on input features [ 17 , 18 , 19 , 20 ]. In machine learning, various image analysis tasks can be performed by finding or learning informative features that successfully describe the regularities or patterns in data. However, conventionally, meaningful or task-relevant features are mainly designed by human experts based on their knowledge of the target domain, making it challenging for those without domain expertise to leverage machine learning techniques. Furthermore, traditional machine learning methods can only detect superficial linear relationships, while the biology underpinning living organisms is several orders of magnitude more complex [ 21 ].

Deep learning [ 22 ], which is inspired by an understanding of the neural networks within the human brain, has achieved unprecedented success in facing the challenges mentioned above by incorporating the feature extraction and selection steps into the training process [ 23 ]. Generically, deep learning models are represented by a series of layers, and each is formed by a weighted sum of elements in the previous layer. The first layer represents the data, and the last layer represents the output or solution. Multiple layers enable complicated mapping functions to be reproduced, allowing deep learning models to solve very challenging problems while typically needing less human intervention than traditional machine learning methods. Deep learning currently outperforms alternative machine learning approaches [ 24 ] and, for the past few years, has been widely used for a variety of tasks in medical image analysis [ 25 ].

A convolutional neural network (CNN) is a deep learning approach that has frequently been applied to medical imaging problems. It overcomes the limitations of previous deep learning approaches because its architecture allows it to automatically learn the features that are important for a problem using a training corpus of sufficient variety and quality [ 26 ]. Recently, CNNs have gained popularity for brain tumor classification due to their outstanding performance with very high accuracy in a research context [ 27 , 28 , 29 , 30 , 31 ].

Despite the growing interest in CNN-based CADx within the research community, translation into daily clinical practice has yet to be achieved due to obstacles such as the lack of an adequate amount of reliable data for training algorithms and imbalances within the datasets used for multi-class classification [ 32 , 33 ], among others. Several reviews [ 31 , 32 , 33 , 34 , 35 , 36 ] have been published in this regard, summarizing the classification methods and key achievements and pointing out some of the limitations in previous studies, but as of yet, none of them have focused on the deficiencies regarding clinical adoption or have attempted to determine the future research directions required to promote the application of deep learning models in clinical practice. For these reasons, the current review considers the key limitations and obstacles regarding the clinical applicability of studies in brain tumor classification using CNN algorithms and how to translate CNN-based CADx technology into better clinical decision making.

In this review, we explore the current studies on using CNN-based deep learning techniques for brain tumor classification published between 2015 and 2022. We decided to focus on CNN architectures, as alternative deep-learning techniques, such as Deep Belief Networks or Restricted Boltzmann Machines, are much less represented in the current literature.

The objectives of the review were three-fold: to (1) review and analyze article characteristics and the impact of CNN methods applied to MRI for glioma classification, (2) explore the limitations of current research and the gaps in bench-to-bedside translation, and (3) find directions for future research in this field. This review was designed to answer the following research questions: How has deep learning been applied to process MR images for glioma classification? What level of impact have papers in this field achieved? How can the translational gap be bridged to deploy deep learning algorithms in clinical practice?

The review is organized as follows: Section 2 introduces the methods used to search and select literature related to the focus of the review. Section 3 presents the general steps of CNN-based deep learning methods for brain tumor classification, and Section 4 introduces relevant primary studies, with an overview of their datasets, preprocessing techniques, and computational methods for brain tumor classification, and presents a quantitative analysis of the covered studies. Furthermore, we introduce the factors that may directly or indirectly degrade the performance and the clinical applicability of CNN-based CADx systems and provide an overview of the included studies with reference to the degrading factors. Section 5 presents a comparison between the selected studies and suggests directions for further improvements, and finally, Section 6 summarizes the work and findings of this study.

2. Materials and Methods

2.1. article identification.

In this review, we identified preliminary sources using two online databases, PubMed and Scopus. The search queries used to interrogate each database are described in Table 1 . The filter option for the publication year (2015–2022) was selected so that only papers in the chosen period were fed into the screening process ( Supplementary Materials ). Searches were conducted on 30 June 2022. PubMed generated 212 results, and Scopus yielded 328 results.

The search queries used to interrogate the PubMed and Scopus databases.

2.2. Article Selection

Articles were selected for final review using a three-stage screening process ( Supplementary Materials ) based on a series of inclusion and exclusion criteria. After removing duplicate records that were generated from using two databases, articles were first screened based on the title alone. The abstract was then assessed, and finally, the full articles were checked to confirm eligibility. The entire screening process ( Supplementary Materials ) was conducted by one author (Y.T.X). In cases of doubt, records were reviewed by other authors (D.N.M, C.T), and the decision regarding inclusion was arrived at by consensus.

The meet the inclusion criteria, articles had to:

• Be original research articles published in a peer-reviewed journal with full-text access offered by the University of Bologna;
• Involve the use of any kind of MR images;
• Be published in English;
• Be concerned with the application of CNN deep learning techniques for brain tumor classification.

Included articles were limited to those published from 2015 to 2022 to focus on deep learning methodologies. Here, a study was defined as work that employed a CNN-based deep learning algorithm to classify brain tumors and that involved the use of one or more of the following performance metrics: accuracy, the area under the receiver operating characteristics curve, sensitivity, specificity, or F 1 score.

Exclusion criteria were:

• Review articles;
• Book or book chapters;
• Conference papers or abstracts;
• Short communications or case reports;
• Unclear descriptions of data;
• No validation performed.

If a study involved the use of a CNN model for feature extraction but traditional machine learning techniques for the classification task, it was excluded. Studies that used other deep learning networks, for example, artificial neural networks (ANNs), generative adversarial networks (GANs), or autoencoders (AEs), instead of CNN models were excluded. Studies using multiple deep learning techniques as well as CNNs were included in this study, but only the performance of the CNNs will be reviewed.

Figure 1 reports the numbers of articles screened after exclusion at each stage as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 37 ]. A review of 83 selected papers is presented in this paper. All of the articles cover the classification of brain tumors using CNN-based deep learning techniques.

The PRISMA flowchart of this review. n : number of articles.

3. Literature Review

This section presents a detailed overview of the research papers dealing with brain tumor classification using CNN-based deep learning techniques published during the period from 2015 to 2022. This section is formulated as follows: Section 3.1 presents a brief overview of the general methodology adopted in the majority of the papers for the classification of brain MRI images using CNN algorithms. Section 3.2 presents a description of the popular publicly available datasets that have been used in the research papers reviewed in the form of a table. Section 3.3 introduces the commonly applied preprocessing methods used in the reviewed studies. Section 3.4 provides an introduction of widely used data augmentation methods. Finally, Section 3.5 provides a brief overview of the performance metrics that provide evidence about the credibility of a specific classification algorithm model.

3.1. Basic Architecture of CNN-Based Methods

Recently, deep learning has shown outstanding performance in medical image analysis, especially in brain tumor classification. Deep learning networks have achieved higher accuracy than classical machine learning approaches [ 24 ]. In deep learning, CNNs have achieved significant recognition for their capacity to automatically extract deep features by adapting to small changes in the images [ 26 ]. Deep features are those that are derived from other features that are relevant to the final model output.

The architecture of a typical deep CNN-based brain tumor classification frame is described in Figure 2 . To train a CNN-based deep learning model with tens of thousands of parameters, a general rule of thumb is to have at least about 10 times the number of samples as parameters in the network for the effective generalization of the problem [ 38 ]. Overfitting may occur during the training process if the training dataset is not sufficiently large [ 39 ]. Therefore, many studies [ 40 , 41 , 42 , 43 , 44 ] use 2D brain image slices extracted from 3D brain MRI volumes to solve this problem, which increases the number of examples within the initial dataset and mitigates the class imbalance problem. In addition, it has the advantage of reducing the input data dimension and reducing the computational burden of training the network.

The basic workflow of a typical CNN-based brain tumor classification study with four high-level steps: Step 1. Input Image: 2D or 3D Brain MR samples are fed into the classification model; Step 2. Preprocessing: several preprocessing techniques are used to remove the skull, normalize the images, resize the images, and augment the number of training examples; Step 3. CNN Classification: the preprocessed dataset is propagated into the CNN model and is involved in training, validation, and testing processes; Step 4. Performance Evaluation: evaluation of the classification performance of a CNN algorithm with accuracy, specificity, F 1 score, area under the curve, and sensitivity metrics.

Data augmentation is another effective technique for increasing both the amount and the diversity of the training data by adding modified copies of existing data with commonly used morphological techniques, such as rotation, reflection (also referred to as flipping or mirroring), scaling, translation, and cropping [ 44 , 45 ]. Such strategies are based on the assumption that the size and orientation of image patches do not yield robust features for tumor classification.

In deep learning, overfitting is also a common problem that occurs when the learning capacity is so large that the network will learn spurious features instead of meaningful patterns [ 39 ]. A validation set can be used in the training process to avoid overfitting and to obtain the stable performance of the brain tumor classification system on future unseen data in clinical practice. The validation set provides an unbiased evaluation of a classification model using multiple subsets of the training dataset while tuning the model’s hyperparameters during the training process [ 46 ]. In addition, validation datasets can be used for regularization by early stopping when the error on the validation dataset increases, which is a sign of overfitting to the training data [ 39 , 47 ]. Therefore, in the article selection process, we excluded the articles that omitted validation during the training process.

Evaluating the classification performance of a CNN algorithm is an essential part of a research study. The accuracy, specificity, F 1 score (also known as the Dice similarity coefficient) [ 48 ], the area under the curve, and sensitivity are important metrics to assess the classification model’s performance and to compare it to similar works in the field.

3.2. Datasets

A large training dataset is required to create an accurate and trustworthy deep learning-based classification system for brain tumor classification. In the current instance, this usually comprises a set of MR image volumes, and for each, a classification label is generated by a domain expert such as a neuroradiologist. In the reviewed literature, several datasets were used for brain tumor classification, targeting both binary tasks [ 27 , 40 , 41 , 45 ] and multiclass classification tasks [ 24 , 30 , 49 , 50 , 51 ]. Table 2 briefly lists some of the publicly accessible databases that have been used in the studies reviewed in this paper, including the MRI sequences as well as the size, classes, unbiased Gini Coefficient, and the web address of the online repository for the specific dataset.

An overview of publicly available datasets.

The Gini coefficient (G) [ 52 ] is a property of distribution that measures its difference using uniformity. It can be applied to categorical data in which classes are sorted by prevalence. Its minimum value is zero if all of the classes are equally represented, and its maximum values varies between 0.5 for a two-class distribution to an asymptote of 1 for many classes. The unbiased Gini coefficient divides G by the maximum value of the number of classes present and takes values in the range of 0–1. The maximum value for a distribution with n classes is (n − 1)/n. The values of the unbiased Gini coefficient were calculated using R package DescTools [ 52 ]. Table 2 shows the characteristics of public datasets in terms of balancing the samples of the available classes of tumors (unbiased Gini coefficient) while considering the total number of samples in the datasets (“Size” column).

Among the public datasets, the dataset from Figshare provided by Cheng [ 55 ] is the most popular dataset and has been widely used for brain tumor classification. BraTS, which refers to the Multimodal Brain Tumor Segmentation Challenge (a well-known challenge that has taken place every year since 2012), is another dataset that is often used for testing brain tumor classification methods. The provided data are pre-processed, co-registered to the same anatomical template, interpolated to the exact resolution (1 mm 3 ), and skull stripped [ 55 ].

Most MR techniques can generate high-resolution images, while different imaging techniques show distinct contrast, are sensitive to specific tissues or fluid regions, and highlight relevant metabolic or biophysical properties of brain tumors [ 64 ]. The datasets listed in Table 2 collect one or more MRI sequences, including T 1 -weighted (T 1 w), T 2 -weighted (T 2 w), contrast-enhanced T 1 -weighted (ceT 1 w), fluid-attenuated inversion recovery (FLAIR), diffusion-weighted imaging (DWI), and dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) sequences. Among these, the T 1 w, T 2 w, ceT 1 w, and FLAIR sequences are widely used for brain tumor classification in both research and in clinical practice. Each sequence is distinguished by a particular series of radiofrequency pulses and magnetic field gradients, resulting in images with a characteristic appearance [ 64 ]. Table 3 lists the imaging configurations and the main clinical distinctions of T 1 w, T 2 w, ceT 1 w, and FLAIR with information retrieved from [ 64 , 65 , 66 , 67 ].

The imaging configurations and main clinical distinctions of T 1 w, T 2 w, ceT 1 w, and FLAIR.

* Pictures from [ 68 ]. TR, repetition time. TE, echo time.

3.3. Preprocessing

Preprocessing is used mainly to remove extraneous variance from the input data and to simplify the model training task. Other steps, such as resizing, are needed to work around the limitations of neural network models.

3.3.1. Normalization

The dataset fed into CNN models may be collected with different clinical protocols and various scanners from multiple institutions. The dataset may consist of MR images with different intensities because the intensities of MR image are not consistent across different MR scanners [ 69 ]. In addition, the intensity values of MR images are sensitive to the acquisition condition [ 70 ]. Therefore, input data should be normalized to minimize the influence of differences between the scanners and scanning parameters. Otherwise, any CNN network that is created will be ill-conditioned.

There are many methods for data normalization, including min-max normalization, z-score normalization, and normalization by decimal scaling [ 71 ]. Min-max normalization is one of the most common ways to normalize MR images found in the included articles [ 27 , 36 , 40 ]. In that approach, the intensity values of the input MR images are rescaled into the range of (0, 1) or (−1, 1).

Z-score normalization refers to the process of normalizing every intensity value found in MR images such that the mean of all of the values is 0 and the standard deviation is 1 [ 71 ].

3.3.2. Skull Stripping

MRI images of the brain also normally contain non-brain regions such as the dura mater, skull, meninges, and scalp. Including these parts in the model typically deteriorates its performance during classification tasks. Therefore, in the studies on brain MRI datasets that retain regions of the skull and vertebral column, skull stripping is widely applied as a preprocessing step in brain tumor classification problems to improve performance [ 24 , 72 , 73 ].

3.3.3. Resizing

Since deep neural networks require inputs of a fixed size, all of the images need to be resized before being fed into CNN classification models [ 74 ]. Images larger than the required size can be downsized by either cropping the background pixels or by downscaling using interpolation [ 74 , 75 ].

3.3.4. Image Registration

Image registration is defined as a process that spatially transforms different images into one coordinate system. In brain tumor classification, it is often necessary to analyze multiple images of a patient to improve the treatment plan, but the images may be acquired from different scanners, at different times, and from different viewpoints [ 76 ]. Registration is necessary to be able to integrate the data obtained from these different measurements.

Rigid image registration is one of the most widely utilized registration methods in the reviewed studies [ 77 , 78 ]. Rigid registration means that the distance between any two points in an MR image remains unchanged before and after transformation. This approach only allows translation and rotation transformations.

3.3.5. Bias Field Correction

In medical images, the bias field is an undesirable artifact caused by factors such as the scan position and instrument used as well as by other unknown issues [ 79 ]. This artifact is characterized by differences in brightness across the image and can significantly degrade the performance of many medical image analysis techniques. Therefore, a preprocessing step is needed to correct the bias field signal before submitting corrupted MR images to a CNN classification model.

The N4 bias field correction algorithm and the Statistical Parametric Mapping (SPM) module are common approaches for correcting the inhomogeneity in the intensity of MR images. The N4 bias field correction algorithm is a popular method for correcting the low-frequency-intensity non-uniformity present in MR image data [ 80 ]. SPM contains several software packages that are used for brain segmentation. These packages usually contain a set for skull stripping, intensity non-uniformity (bias) correction, and segmentation routines [ 81 ].

3.4. Data Augmentation

CNN-based classification requires a large number of data. A general rule of thumb is to have at least about 10 times the number of samples set as parameters in the network for the effective generalization of the problem [ 38 ]. If the database is significantly smaller, overfitting might occur. Data augmentation is one of the foremost data techniques to subside imbalanced distribution and data scarcity problems. It has been used in many studies focusing brain tumor classification [ 24 , 45 , 49 , 50 ] and involves geometrical transformation operations such as rotation, reflection (also referred to as flipping or mirroring), scaling, translation, and cropping ( Figure 3 ).

Data augmentation: ( a ) original image; ( b ) 18° rotation. When rotating by an arbitrary number of degrees (non-modulo 90), rotation will result in the image being padded in each corner. Then, a crop is taken from the center of the newly rotated image to retain the largest crop possible while maintaining the image’s aspect ratio; ( c ) left–right flipping; ( d ) top–bottom flipping; ( e ) scaling by 1.5 times; ( f ) cropping by center cropping to the size 150 × 150; ( g ) random brightness enhancement; ( h ) random contrast enhancement.

Data augmentation techniques can be divided into two classes: position augmentation and color augmentation. Some of the most popular position augmentation methods include rotation, reflection (also referred to as flipping or mirroring), scaling, translation, and cropping, and they have been commonly used to enlarge MR datasets in studies focusing on brain tumor classification [ 45 , 51 , 72 , 77 ]. Color augmentation methods such as contrast enhancement and brightness enhancement have also been applied in the included studies [ 28 , 43 ].

Recently, well-established data augmentation techniques have begun to be supplemented by automatic methods that use deep learning approaches. For example, the authors in [ 44 ] proposed a progressively growing generative adversarial network (PGGAN) augmentation model to help overcome the shortage of images needed for CNN classification models. However, such methods are rare in the literature reviewed.

3.5. Performance Measures

Evaluating the classification performance of a CNN algorithm is an essential part of a research study. Here, we outline the evaluation metrics that are the most commonly encountered in the brain tumor classification literature, namely accuracy, precision, sensitivity, F1 score, and the area under the curve.

In classification tasks, true positive ( TP ) represents an image that is correctly classified into the positive class according to the ground truth. Similarly, true negative is an outcome in which the model correctly classifies an imagine into the negative class. On the other hand, false positive ( FP ) is an outcome in which the model incorrectly classifies an image into the positive class when the ground truth is negative. False negative ( FN ) is an outcome in which the model incorrectly classifies an image that should be placed in the positive class.

3.5.1. Accuracy

Accuracy ( ACC ) is a metric that measures the performance of a model in correctly classifying the classes in a given dataset and is given as the percentage of total correct classifications divided by the total number of images.

3.5.2. Specificity

Specificity ( SPE ) represents the proportion of correctly classified negative samples to all of the negative samples identified in the data.

3.5.3. Precision

Precision ( PRE ) represents the ratio of true positives to all of the identified positives.

3.5.4. Sensitivity

Sensitivity ( SEN ) measures the ability of a classification model to identify positive samples. It represents the ratio of true positives to the total number of (actual) positives in the data.

3.5.5. F 1 Score

The F 1 score [ 48 ] is one of the most popular metrics and considers both precision and recall. It can be used to assess the performance of classification models with class imbalance problems [ 82 ] and considers the number of prediction errors that a model makes and looks at the type of errors that are made. It is higher if there is a balance between PRE and SEN .

3.5.6. Area under the Curve

The area under the curve (AUC) measures the entire two-dimensional area underneath the ROC curve from (0, 0) to (1, 1). It measures the ability of a classifier to distinguish between classes.

Clinicians and software developers need to understand how performance metrics can measure the properties of CNN models for different medical problems. In research studies, several metrics are typically used to evaluate a model’s performance.

Accuracy is among the most commonly used metric to evaluate a classification model but is also known for being misleading in cases when the classes have different distributions in the data [ 83 , 84 ]. Precision is an important metric in cases when the occurrence of false positives is unacceptable/intolerable [ 84 ]. Specificity measures the ability of a model to correctly identify people without the disease in question. Sensitivity, also known as recall, is an important metric in cases where identifying the number of positives is crucial and when the occurrence of false negatives is unacceptable/intolerable [ 83 , 84 ]. It must be interpreted with care in cases with strongly imbalanced classes.

It is important to recognize that there is always a tradeoff between sensitivity and specificity. Balancing between two metrics has to be based on the medical use case and the associated requirements [ 83 ]. Precision and sensitivity are both proportional to TP but have an inverse relationship. Whether to maximize recall or precision depends on the application: Is it more important to only identify relevant instances, or to make sure that all relevant instances are identified? The balance between precision and sensitivity has to be considered in medical use cases in which some false positives are tolerable; for example, in cancer detection, it is crucial to identify all positive cases. On the other hand, for a less severe disease with high prevalence, it is important to achieve the highest possible precision [ 83 ].

This section provides an overview of the research papers focusing on brain tumor classification using CNN techniques. Section 4.1 presents a quantitative analysis of the number of articles published from 2015 to 2022 on deep learning and CNN in brain tumor classification and the usage of the different CNN algorithms applied in the studies covered. Then, Section 4.2 introduces the factors that may directly or indirectly degrade the performance and the clinical applicability of CNN-based CADx systems. Finally, in Section 4.3 , an overview of the included studies will be provided with reference to the degrading factors introduced in Section 4.2 .

4.1. Quantitative Analysis

As mentioned in the introduction, many CNN models have been used to classify the MR images of brain tumor patients. They overcome the limitations of earlier deep learning approaches and have gained popularity among researchers for brain tumor classification tasks. Figure 4 shows the number of research articles on brain tumor classification using deep learning methods and CNN-based deep learning techniques published on PubMed and Scopus in the years from 2015 to June 2022; the number of papers related to brain tumor classification using CNN techniques grows rapidly from 2019 onwards and accounts for the majority of the total number of studies published in 2020, 2021, and 2022. This is because of the high generalizability, stability, and accuracy rate of CNN algorithms.

Number of articles published from 2015 to 2022.

Figure 5 shows the usage of the most commonly used preprocessing techniques for addressing problems in brain tumor classification, including data augmentation, normalization, resizing, skull stripping, bias field correction, and registration. In this figure, only data from 2017 to 2022 are visualized, as no articles using the preprocessing methods mentioned were published in 2015 or 2016. Since 2020, data augmentation has been used in the majority of studies to ease data scarcity and overfitting problems. However, the bias field problem has yet to be taken seriously, and few studies have included bias field correction in the preprocessing process.

Usage of preprocessing techniques from 2017 to 2022.

Figure 6 breaks down the usage of the publicly available CNN architectures used in the articles included in this review, including custom CNN models, VGG, AlexNet, ResNet, GoogLeNet, DenseNet, and EfficientNet.

Usage of state-of-the-art CNN models from 2015 and 2022.

AlexNet [ 85 ] came out in 2012 and was a revolutionary advancement in deep learning; it improved traditional CNNs by introducing a composition of consecutively stacked convolutional layers and became one of the best models for image classification. VGG, which refers to the Visual Geometry Group, was a breakthrough in the world of convolutional neural networks after AlexNet. It is a type of deep CNN architecture with multiple layers that was originally proposed by K. Simonyan and A. Zisserman in [ 86 ] and was developed to improve model performance by increasing the depth of such CNNs.

GoogLeNet is a deep convolutional neural network with 22 layers based on the Inception architecture; it was developed by researchers at Google [ 87 ]. GoogLeNet addresses most of the problems that large networks face, such as computational expense and overfitting, by employing the Inception module. This module can use max pooling and three varied sizes of filters (1 × 1, 3 × 3, 5 × 5) for convolution in a single image block; such blocks are then concatenated and passed onto the next layer. An extra 1 × 1 convolution can be added to the neural network before the 3 × 3 and 5 × 5 layers to make the process even less computationally expensive [ 87 ]. ResNet stands for Deep Residual Network. It is an innovative convolutional neural network that was originally proposed in [ 88 ]. ResNet makes use of residual blocks to improve the accuracy of models. A residual block is a skip-connection block that typically has double- or triple-layer skips that contain nonlinearities (ReLU) and batch normalization in between; it can help to reduce the problem of vanishing gradients or can help to mitigate accuracy saturation problems [ 88 ]. DenseNet, which stands for Dense Convolutional Network, is a type of convolutional neural network that utilizes dense connections between layers. DenseNet was mainly developed to improve the decreased accuracy caused by the vanishing gradient in neural networks [ 89 ]. Additionally, those CNNs take in images with a pixel resolution of 224 × 224. Therefore, for brain tumor classification, the authors need to center crop a 224 × 224 patch in each image to keep the input image size consistent.

Convolutional neural networks are commonly built using a fixed resource budget. When more resources are available, the depth, width, and resolution of the model need to be scaled up for better accuracy and efficiency [ 90 ]. Unlike previous CNNs, EfficientNet is a novel baseline network that uses a different model-scaling technique based on a compound coefficient and neural architecture search methods that can carefully balance network depth, width, and resolution [ 90 ].

4.2. Clinical Applicability Degrading Factors

This section introduces the factors that hinder the adoption and development of CNN-based brain tumor classification CADx systems into clinic practice, including data quality, data scarcity, data mismatch, data imbalance, classification performance, research value towards clinic needs, and the Black-Box characteristics of CNN models.

4.2.1. Data Quality

During the MR image acquisition process, both the scanner and external sources may produce electrical noise in the receiver coil, generating image artifacts in the brain MR volumes [ 69 ]. In addition, the MR image reconstruction process is sensitive to acquisition conditions, and further artifacts are introduced if the subject under examination moves during the acquisition of a single image [ 69 ]. These errors are inevitable and reduce the quality of the MR images used to train networks. As a result, the quality of the training data degrades the sensitivity/specificity of CNN models, thus compromising their applicability in a clinic setting.

4.2.2. Data Scarcity

Big data is one of the biggest challenges that CNN-based CADx systems face today. A large number of high-quality annotated data is required to build high-performance CNN classification models, while it is a challenge to label a large number of medical images due to the complexity of medical data. When a CNN classification system does not have enough data, overfitting can occur—as classification is based on extraneous variance in the training set—affecting the capacity of the network to generalize new data [ 91 ].

4.2.3. Data Mismatch

Data mismatch refers to a situation in which a model that has been well-trained in a lab environment fails to generalize real-world clinical data. It might be caused by overfitting of the training set or due to mismatch between research images and clinic ones [ 82 ]. Studies are at high risk of generalization failure if they omit a validation step or if the test set does not reflect the characteristics of the clinical data.

4.2.4. Class Imbalance

In brain MRI datasets such as the BraTS 2019 dataset [ 92 ], which consists of 210 HGG and 75 LGG patients (unbiased Gini coefficient 0.546, as shown in Table 2 ), HGG is represented by a much higher percentage of samples than LGG, leading to so-called class imbalance problems, in which inputting all of the data into the CNN classifier to build up the learning model will usually lead to a learning bias to the majority class [ 93 ]. When an unbalanced training set is used, it is important to assess model performance using several performance measures ( Section 3.5 ).

4.2.5. Research Value towards Clinical Needs

Different brain tumor classification tasks were studied using CNN-based deep learning techniques during the period from 2015 to 2022, including clinically relevant two-class classification (normal vs. tumorous [ 29 , 41 , 94 , 95 ], HGG vs. LGG [ 27 , 40 , 45 , 73 ], LGG-II vs. LGG-III [ 96 ], etc.); three-class classification (normal vs. LGG vs. HGG [ 24 ], meningioma (MEN) vs. pituitary tumor (PT) vs. glioma [ 39 , 42 , 49 , 50 ], glioblastoma multiforme (GBM) vs. astrocytoma (AST) vs. oligodendroglioma (OLI) [ 30 ], etc.); four-class classification (LGG vs. OLI vs. anaplastic glioma (AG) vs. GBM [ 72 ], normal vs. AST-II vs. OLI-III vs. GBM-IV [ 24 ], normal vs. MEN vs. PT vs. glioma [ 97 ], etc.); five-class classification (AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM-IV [ 24 ]); and six-class classification (normal vs. AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM-IV [ 24 ]).

Not all classification tasks are equally difficult, and this is the case for the deep learning research community and clinical practice. The authors in [ 24 ] used AlexNet for multi-class classification tasks, including two-class classification: normal vs. tumor, three-class classification: normal vs. LGG vs. HGG; four-class classification: normal vs. AST vs. OLI vs. GBM; five-class classification: AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM-IV, and six-class classification: normal vs. AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM-IV. The results reported 100% accuracy for the normal vs. tumorous classification. The accuracy for the five-class classification (AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM-IV) was only 87.14%. Similarly, in a recent publication [ 98 ], the authors utilized the same CNN model for multi-class brain tumor classification. The overall accuracy obtained for normal vs. tumorous classification reached 100% compared to the lower accuracy of 90.35% obtained for the four-class classification task (Grade I vs. Grade II vs. Grade III vs. Grade IV) and 86.08% for the five-class classification of AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM.

The goal of research in the field of CADx is to help address existing unmet clinical needs and to provide assistance methods and tools for the difficult tasks that human professionals cannot easily handle in clinical practice. It is observed that CNN-based models have achieved quite high accuracies for normal/tumorous image classification, while more research is needed to improve the classification performance of more difficult tasks, especially in five-class classification (e.g., AST-II vs. AST-III vs. OLI-II vs. OLI-III vs. GBM) and four-class classification (e.g., Grade I vs. Grade II vs. Grade III vs. Grade IV) tasks. Therefore, studies that use normal vs. tumorous as their target problem have little clinical value.

4.2.6. Classification Performance

Classification performance, which indicates the reliability and trustworthiness of CADx systems, is one of the most important factors to be considered when translating research findings into clinical practice. It has been shown that CNN techniques perform well in most of brain tumor classification tasks, such as in two-class classification (normal and tumorous [ 94 , 95 ] and HGG and LGG [ 45 , 73 ]) and three-class classification (normal vs. LGG vs. HGG [ 24 ] and MEN vs. PT vs. glioma [ 49 , 50 ]) tasks. However, the classification performance obtained for more difficult classification tasks, such as a five-class classification between AST-II, AST-III, OLI-II, OLI-III, and GBM, remains poor [ 24 , 98 ] and justifies further research.

4.2.7. Black-Box Characteristics of CNN Models

The brain tumor classification performance of some of the CNN-based deep learning techniques reviewed here is remarkable. Still, their clinical application is also limited by another factor: the “Black-Box” problem. Even the designers of a CNN model cannot usually explain the internal workings of the model or why it arrived at a specific decision. The features used to decide the classification of any given image are not an output of the system. This lack of explainability reduces the confidence of clinicians in the results of the techniques and impedes the adoption and development of deep learning tools into clinical practice [ 99 ].

4.3. Overview of Included Studies

Many research papers have emerged following the wave of enthusiasm for CNN-based deep learning techniques from 2015 to present day. In this review, 83 research papers are assessed to summarize the effectiveness of CNN algorithms in brain tumor classification and to suggest directions for future research in this field.

Among the articles included, twenty-five use normal/tumorous as their classification target. However, as mentioned in Section 4.2.5 , the differentiation between normal and tumorous images is not a difficult task. It has been well-solved both in research and clinic practice and thus has little value for clinical application. Therefore, studies that use normal vs. tumorous as their target problem will not be reviewed further in the following assessment steps.

Table 4 a provides an overview of the included studies that focus on CNN-based deep learning methods for brain tumor classification but does not include studies working with a normal vs. tumorous classification. The datasets, MRI sequences, size of the datasets, and the preprocessing methods are summarized. Table 4 b summarizes the classification tasks, classification architecture, validation methods, and performance metrics of the reviewed articles.

(a) Overview of included studies that focus on CNN-based deep learning methods for brain tumor classification, with the exception of studies focusing on normal vs. tumorous classification. Datasets, MRI sequences, size of the datasets, and preprocessing methods are summarized. (b) Overview of included studies that focus on CNN-based deep learning methods for brain tumor classification, with the exception of study focusing on normal vs. tumorous classification. Classification tasks, classification architecture, validation methods, and performance metrics are summarized.

Notes: 1 Rigid registration unless otherwise notes; 2 translation also referred to as shifting; 3 scaling also referred to as zooming; 4 reflection also referred to as flipping or mirroring; ** The Cancer Imaging Archive, https://www.cancerimagingarchive.net/ (accessed on 27 July 2022). 5 Referring to overall accuracy, mean accuracy, or highest accuracy depending on the information provided by the paper or the highest accuracy when multiple models are used.

As introduced in Section 4.2 , the major challenge confronting brain tumor classification using CNN techniques in MR images lies in the training data, including the challenges caused by data quality, data scarcity, data mismatch, and data imbalance, which hinder the adoption and development of CNN-based brain tumor classification CADx systems into clinic practice. Here, we assess several recently published studies to provide a convenient collection of the state-of-the-art techniques that have been used to address these issues and the problems that have not been solved in those studies.

Currently, data augmentation is recognized as the best solution to the problem caused by data scarcity and has been widely utilized in brain tumor classification studies.

The authors in [ 100 ] used different data augmentation methods, including rotation, flipping, Gaussian blur, sharpening, edge detection, embossing, skewing, and shearing, to increase the size of the dataset. The proposed system aims to classify between Grade I, Grade II, Grade III, and Grade IV, and the original data consist of 121 images (36 Grade I images, 32 Grade II images, 25 Grade III images, and 28 Grade IV images), and by using data augmentation techniques, 30 new images are generated from each MR image. The proposed model is experimentally evaluated using both augmented and original data. The results show that the overall accuracy after data augmentation reaches 90.67%, which is greater than the accuracy of 87.38% obtained without augmentation.

While most data augmentation techniques aim to increase extraneous variance in the training set, deep learning can be used by itself, at least in theory, to increase meaningful variance. In a recent publication by Allah et al. [ 44 ], a novel data augmentation method called a progressive growing generative adversarial network (PGGAN) was proposed and combined with rotation and flipping methods. The method involves an incremental increase of the size of the model during the training to produce MR images of brain tumors and to help overcome the shortage of images for deep learning training. The brain tumor images were classified using a VGG19 feature extractor coupled with a CNN classifier. The accuracy of the combined VGG19 + CNN and PGGAN data augmentation framework achieved an accuracy of 98.54%.

Another approach that helps overcome the problem of data scarcity and that can also reduce computational costs and training time is transfer learning. Transfer learning is a hot research topic in machine learning; previously learned knowledge can be transferred for the performance of a new task by fine-tuning a previously generated model with a smaller dataset that is more specific to the aim of the study. Transfer learning is usually expressed using pre-trained models such as VGG, GoogLeNet, and AlexNet that have been trained on the large benchmark dataset ImageNet [ 101 ].

Many attempts have been made to investigate the value of transfer learning techniques for brain tumor classification [ 39 , 45 , 50 , 102 , 104 , 108 , 116 , 121 ]. Deepak and Ameer [ 39 ] used the GoogLeNet with the transfer learning technique to differentiate between glioma, MEN, and PT from the dataset provided by Cheng [ 55 ]. This proposed system achieved a mean classification accuracy of 98%.

In a study conducted by Yang et al. [ 45 ], AlexNet and GoogLeNet were both trained from scratch and fine-tuned from pre-trained models from the ImageNet database for HGG and LGG classification. The dataset used in this method consisted of ceT 1 w images from 113 patients (52 LGG, 61 HGG) with pathologically proven gliomas. The results show that GoogLeNet proved superior to AlexNet for the task. The performance measures, including validation accuracy, test accuracy, and test AUC of GoogLeNet trained from scratch, were 0.867, 0.909, and 0.939, respectively. With fine-tuning, the pre-trained GoogLeNet obtained performed better during glioma grading, with a validation accuracy of 0.867, a test accuracy of 0.945, and a test AUC 0.968.

The authors in [ 50 ] proposed a block-wise fine-tuning strategy using a pre-trained VGG19 for brain tumor classification. The dataset consisted of 3064 images (708 MEN, 1426 glioma, and 930 PT) from 233 patients (82 MEN, 89 glioma, and 62 PT). The authors achieved an overall accuracy of 94.82% under five-fold cross-validation. In another study by Bulla et al. [ 108 ], classification was performed in a pre-trained InceptionV3 CNN model using data from the same dataset. Several validation methods, including holdout validation, 10-fold cross-validation, stratified 10-fold cross-validation, and group 10-fold cross-validation, were used during the training process. The best classification accuracy of 99.82% for patient-level classification was obtained under group 10-fold cross-validation.

The authors in [ 104 ] used InceptionResNetV2, DenseNet121, MobileNet, InceptionV3, Xception, VGG16, and VGG19, which have already been pre-trained on the ImageNet dataset, to classify HGG and LGG brain images. The MR images used in this research were collected from the BraTS 2019 database, which contains 285 patients (210 HGG, 75 LGG). The 3D MRI volumes from the dataset were then converted into 2D slices, generating 26,532 LGG images and 94,284 HGG images. The authors selected 26,532 images from HGG to balance these two classes to reduce the impact on classification performance due to class imbalance. The average precision, f1-score, and sensitivity for the test dataset were 98.67%, 98.62%, and 98.33%, respectively.

Lo et al. [ 116 ] used transfer learning with fine-tuned AlexNet and data augmentation to classify Grade II, Grade III, and Grade IV brain tumor images from a small dataset comprising 130 patients (30 Grade II, 43 Grade III, 57 Grade IV). The results demonstrate much higher accuracy when using the pre-trained AlexNet. The proposed transferred DCNN CADx system achieved a mean accuracy of 97.9% and a mean AUC of 0.9991, while the DCNN without pre-trained features only achieved a mean accuracy of 61.42% and a mean AUC of 0.8222.

Kulkarni and Sundari [ 121 ] utilized five transfer learning architectures, AlexNet, VGG16, ResNet18, ResNet50, and GoogLeNet, to classify benign and malignant brain tumors from the private dataset collected by the authors, which only contained 200 images (100 benign and 100 malignant). In addition, data augmentation techniques, including scaling, translation, rotation, translation, shearing, and reflection, were performed to generalize the model and to reduce the possibility of overfitting. The results show that the fine-tuned AlexNet architecture achieved the highest accuracy and sensitivity values of 93.7% and 100%.

Despite many studies on CADx systems demonstrating inspiring classification performance, the validation of their algorithms for clinical practice has hardly been carried out. External validation is an efficient approach to overcome the problems caused by data mismatch and to improve the generalization, stability, and robustness of classification algorithms. It is the action of evaluating the classification model in a new independent dataset to determine whether the model performs well. However, we only found two studies that used an external clinical dataset to evaluate the effectiveness and generalization capability of the proposed scheme, which is described in below.

Decuyper et al. [ 73 ] proposed a 3D CNN model to classify brain MR volumes collected from the TCGA-LGG, TCGA-GBM, and BraTS 2019 databases into HGG and LGG. Multiple MRI sequences, including T 1 w, ceT 1 w, T 2 w, and FLAIR, were used in this research. All of the MR data were co-registered to the same anatomical template and interpolated to 1 mm 3 voxel sizes. Additionally, a completely independent dataset of 110 patients acquired at the Ghent University Hospital (GUH) was used as an external dataset to validate the efficiency and generalization of the proposed model. The resulting validation accuracy, sensitivity, specificity, and AUC for the GUH dataset were 90.00%, 90.16%, 89.80%, and 0.9398.

In [ 120 ], Gilanie et al. presented an automatic method using a CNN architecture for astrocytoma grading between AST-I, AST-II, AST-III, and AST-IV. The dataset consisted of MR slices from 180 subjects, including 50 AST-I cases, 40 AST-II cases, 40 AST-III cases, and 50 AST-IV cases. T1w, T2w, and FLAIR were used in the experiments. In addition, the N4ITK method [ 80 ] was used in the preprocessing stage to correct the bias field distortion present in the MR images. The results were validated on a locally developed dataset to evaluate the effectiveness and generalization capabilities of the proposed scheme. The proposed method obtained an overall accuracy of 96.56% for the external validation dataset.

In brain tumor classification, it is often necessary to use image co-registration to preprocess input data when images are collected from different sequences or different scanners. However, we found that this problem has not yet been taken seriously. In the surveyed articles, six studies [ 73 , 76 , 98 , 118 , 135 , 136 ] used data from multiple datasets for one classification target, while only two studies [ 73 , 76 ] performed image co-registration during the image preprocessing process.

The authors in [ 76 ] proposed a 2D Mask RCNN model and a 3DConvNet model to distinguish between LGG (Grades II and Grade III) and HGG (Grade IV) on multiple MR sequences, including T 1 w, ceT 1 w, T 2 w, and FLAIR. The TCIA-LGG and BraTS 2018 databases were used to train and validate these two CNN models in this research work. In the 2D Mask RCNN model, all of the input MR images were first preprocessed by rigid image registration and intensity inhomogeneity correction. In addition, data augmentation was also implemented to increase the size and the diversity of the training data. The performance measures accuracy, sensitivity, and specificity achieved values of 96.3%, 93.5%, and 97.2% using the proposed 2D Mask RCNN-based method and 97.1%, 94.7%, and 96.8% with the 3DConvNet method, respectively.

In the study conducted by Ayadi [ 98 ], the researchers built a custom CNN model for multiple classification tasks. They collected data from three online databases, Radiopaedia, the dataset provided by Cheng, and REMBRANDT, for brain tumor classification, but no image co-registration was performed to minimize shift between images and to reduce its impact on the classification performance. The overall accuracy obtained for tumorous and normal classification reached 100%; for normal, LGG, and HGG classification, it reached 95%; for MEN, glioma, and PT classification, it reached 94.74%; for normal, AST, OLI, and GBM classification, it reached 94.41%; for Grade I, Grade II, Grade III, and Grade IV classification, it reached 90.35%; for AST-II, AST-III, OLI-II, OLI-III, and GBM classification, it reached 86.08%; and for normal, AST-II, AST-III, OLI-II, OLI-III, and GBM classification, it reached 92.09%.

The authors in [ 118 ] proposed a 3D CNN model for brain tumor classification between GBM, AST, and OLI. A merged dataset comprising data from the CPM-RadPath 2019 and BraTS 2019 databases was used to train and validate the proposed model, but the authors did not perform image co-registration. The results show that the classification model has very poor performance during brain tumor classification, with an accuracy of 74.9%.

In [ 135 ], the researchers presented a CNN-PSO method for two classification tasks: normal vs. Grade II vs. Grade III vs. Grade IV and MEN vs. glioma vs. PA. The MR images used for the first task were collected from four publicly available datasets: the IXI dataset, REMBRANDT, TCGA-GBM, and TCGA-LGG. The overall accuracy obtained was 96.77% for classification between normal, Grade II, Grade III, and Grade IV and 98.16% for MEN, glioma, and PA classification.

Similar to the work conducted in [ 135 ], Anaraki et al. [ 136 ] used MR data merged from four online databases: the IXI dataset, REMBRANDT, TCGA-GBM, and TCGA-LGG, and from one private dataset collected by the authors for normal, Grade II, Grade III, and Grade IV classification. They also used the dataset proposed by Cheng [ 55 ] for MEN, glioma, and PA classification. Different data augmentation methods were performed to further enlarge the size of the training set. The authors in these studies did not co-register the MR images from different sequences from different institutions for the four-class classification task. The results show that 93.1% accuracy was achieved for normal, Grade II, Grade III, and Grade IV classification, and 94.2% accuracy was achieved for MEN, glioma, and PA classification.

Despite the high accuracy levels reported in most studies using CNN techniques, we found that in several studies [ 102 , 117 , 118 , 137 ], the models demonstrated very poor performance during brain tumor classification tasks.

The authors in [ 102 ] explored transfer learning techniques for brain tumor classification. The experiments were performed on the BraTS 2019 dataset, which consists of 335 patients diagnosed with brain tumors (259 patients with HGG and 76 patients with LGG). The model achieved a classification AUC of 82.89% on a separate test dataset of 66 patients. The classification performance obtained by transfer learning in this study is relatively low, hindering its development and application in clinical practice. The authors of [ 117 ] presented a 3D CNN model developed to categorize adult diffuse glioma cases into the OLI and AST classes. The dataset used in the experiment consisted of 32 patients (16 patients with OLI and 16 patients with AST). The model achieved accuracy values of 80%. The main reason for the poor performance probably lies in the small dataset, with only 32 patients being used for model training. That is far from enough to train a 3D model.

In another study [ 137 ], two brain tumor classification tasks were studied using the Lenet, AlexNet, and U-net CNN architectures. In the experiments, MR images from 11 patients (two metastasis, six glioma, and three MEN) obtained from Radiopaedia were utilized to classify metastasis, glioma, and MEN; the data of 20 patients collected from BraTS 2017 were used for HGG and LGG classification. The results show poor classification performance by the three CNN architectures on the two tasks, with an accuracy of 75% obtained by AlexNet and an accuracy of 48% obtained by Lenet for the first task and an accuracy of 62% obtained by AlexNet and an accuracy of 60% obtained by U-net for the second task. The poor performance of Lenet is probably due to its simple architecture, which is not capable of high-resolution image classification. On the other hand, the U-net CNN performs well in segmentation tasks but is not the most commonly used network for classification.

Even though CNNs have demonstrated remarkable performance in brain tumor classification tasks in the majority of the reviewed studies, their level of trustworthiness and transparency must be evaluated in a clinic context. Of the included articles, only two studies, conducted by Artzi et al. [ 122 ] and Gaur et al. [ 127 ], investigated the Black-Box nature of CNN models for brain tumor classification to ensure that the model is looking in the correct place rather than at noise or unrelated artifacts.

The authors in [ 122 ] proposed a pre-trained ResNet-50 CNN architecture to classify three posterior fossa tumors from a private dataset and explained the classification decision by using gradient-weighted class activation mapping (Grad-CAM). The dataset consisted of 158 MRI scans of 22 healthy controls and 63 PA, 57 MB, and 16 EP patients. In this study, several preprocessing methods were used to reduce the influence of MRI data on the classification performance of the proposed CNN model. Image co-registration was performed to ensure that the images become spatially aligned. Bias field correction was also conducted to remove the intensity gradient from the image. Data augmentation methods, including flipping, reflection, rotation, and zooming, were used to increase the size and diversity of the dataset. However, class imbalance within the dataset, particularly the under-representation of EP, was not addressed. The proposed architecture achieved a mean validation accuracy of 88% and 87% for the test dataset. The results demonstrate that the proposed network using Grad-CAM can identify the area of interest and train the classification model based on pathology-related features.

Gaur et al. [ 127 ] proposed a CNN-based model integrated with local interpretable model-agnostic explanation (LIME) and Shapley additive explanation (SHAP) for the classification and explanation of meningioma, glioma, pituitary, and normal images using an MRI dataset of 2870 MR images. For better classification results, Gaussian noise was introduced in the pre-processing step to improve the learning for the CNN, with mean = 0 and a standard deviation of 10 0.5 . The proposed CNN architecture achieved an accuracy of 94.64% for the MRI dataset. The proposed model also provided a locally model-agnostic explanation to describe the results for ordinary people more qualitatively.

5. Discussion

Many of the articles included in this review demonstrate that CNN-based architectures can be powerful and effective when applied to different brain tumor classification tasks. Table 4 b shows that the classification of HGG and LGG images and the differentiation of MEN, glioma, and PT images were the most frequently studied applications. The popularity of these applications is likely linked to the availability of well-known and easily accessible public databases, such as the BraTS datasets and the dataset made available by Cheng [ 55 ]. Figure 7 reveals that there is an increase in the overall accuracy achieved by CNN architectures for brain tumor classification from 2018 to 2022. It is observed that from 2019 onwards, the overall classification accuracy achieved in most studies reached 90%, with only few works obtaining lower accuracies, and in 2020, the extreme outlier accuracy was 48% [ 137 ]. It is also apparent from this figure that the proportion of papers with an accuracy higher than 95% increases after 2020.

Classification accuracy by publication year.

In order to discuss the technical differences and points of similarity between the papers included in the present review, we decided to proceed thematically. Wherever possible, it is more useful to make comparisons between studies containing as few differences as possible. The most commonly reported metric, and the only one that will be employed here, is the accuracy. There are several studies that allow us to make such comparisons across only one factor. In other cases, several studies employ a similar methodology, and we can perform across-study comparisons. Finally, accuracy data can be plotted for single factors to allow for a simple visual comparison without attempting to separate confounding factors.

5.1. The Importance of the Classification Task

Three papers [ 24 , 97 , 98 ] investigated the effect of splitting a dataset into different numbers of categories. They all showed the expected monotonic decrease in accuracy as the number of classes increased, with the caveat that the “normal” image category is relatively easy to distinguish from the others and does not decrease accuracy when added as an additional category. The pattern is also apparent in Figure 8 —the maximum accuracy for two-class problems was 100%; for four-class problems, it was 98.8%; and for six-class problems, it was 93.7%.

Classification accuracy by classification task.

Two papers employed a single architecture to perform different classification tasks [ 30 , 138 ] while keeping the number of classes constant. The results in [ 30 ] showed little difference between the accuracy obtained for two different problems, which could be explained by differences in the datasets. The results of [ 138 ] showed slightly larger variation between four two-class problems. Curiously, nets trained on larger datasets yielded worse accuracy values, suggesting that results obtained from smaller samples have an inflated accuracy (100% for a problem based on 219 images, 96.1% for a problem based on 2156 images). With reference to Figure 8 , the classification task seems to have a larger effect than the class number on the accuracy. Note that the categories that group various specific tasks (two-class, three-class) together show much greater heterogeneity than those with the same number of classes for specific comparisons.

Further evidence regarding the importance of the task comes from a comparison of the accuracy in the papers comparing tumor grade (LGC vs. HGC) and those seeking to differentiate different types of tumors (MEN vs. glioma vs. PT); although the latter task involves more classes, the median accuracy is 97.6 (against 94.4 for the former). We compared the articles that studied the classification of HGG and LGG and found that the classification performance varies widely, even between the articles published in 2021 that utilized state-of-the-art CNN techniques. One of the key factors that significantly affects the performance of CNN models for brain tumor classification lies in the size of the datasets. The authors of [ 40 , 78 ] both proposed custom CNN models to classify HGG and LGG images of 285 MRI scans from the BraTS 2017 dataset. The overall accuracy values were 90.7% and 94.28%, respectively. The authors of [ 137 ] utilized AlexNet for the same task, but MRI data of only 20 patients from the same dataset were studied. The model in this study yielded a poor classification accuracy of 62%, the lowest value among the articles on this classification task.

Figure 8 presents the overall accuracies achieved by the reviewed studies that worked on different classification tasks. What stands out in the figure is that with the exception of the five-class tasks, which achieved accuracies lower than 90%, the CNNs achieved promising accuracies on different brain tumor classification tasks, especially in three-class classification tasks distinguishing between MEN, glioma, and PT. We also noticed that the accuracies of the three-class classification tasks fluctuated widely, with the lowest accuracy being 48% in [ 137 ] for the metastasis vs. glioma vs. MEN classification. More research attention should be paid to improving the accuracies of these classification tasks.

5.2. The Effect of the Dataset

A few studies applied the same network architecture to two different datasets. For He et al. [ 78 ], the results demonstrating a higher accuracy (94.4% against 92.9%) were based on a training set that was both larger and more unbalanced. The first factor would have improved the training process, while the latter made the classification task easier. Several papers derive different subgroups from different datasets (for example, healthy subject data from IXI and tumors from other sets). This is poor practice, as there are likely to be non-pathological differences between the sets acquired from different centres, and this can artificially inflate classification accuracy [ 139 ].

As was mentioned in the Results section, dataset size is considered a critical factor in determining the classification performance of a CNN architecture. Some studies report the dataset size in terms of the number of subjects included, and others report it in terms of the number of images. Typically, several images are included from each subject, but this number is not specified.

Figure 9 and Figure 10 sum up the classification accuracies obtained according to each of the factors; Figure 9 shows that there is a marked increase in the overall accuracy achieved with more training subjects The improvement gained by increasing the image number seems more modest.

Classification accuracy by number of patients.

Classification accuracy by number of images.

Another interesting aspect of the datasets used is the choice of MRI sequence. This may provide a hint as to the features being used for classification. Comparing the articles that focused on the same classification task, of the sequences listed in Table 3 , only ceT 1 w was associated with studies showing a higher classification accuracy than those that excluded it for MEN vs. Glioma vs. PT classification, while all of the sequences contributed to an improvement in LGG vs. HGG classification. As a consequence, studies using multiple sequences were associated with higher accuracy in the LGG vs. HGG task but not in MEN vs. Glioma vs. PT classification.

5.3. The Effect of CNN Architecture

Three studies present comparisons of different architectures trained on the same problems (Yang et al. [ 45 ], Kulkarni et al. [ 121 ], Wahling et al. [ 137 ]).

In a study conducted by Yang et al. [ 45 ], GoogLeNet and AlexNet were both trained from scratch and fine-tuned from pre-trained models from the ImageNet database for HGG and LGG classification. When both were trained from scratch, GoogLeNet proved superior to AlexNet for the task. The test accuracies were 0.909 and 0.855, respectively. Fine-tuning pre-existing nets resulted in better performance in both cases, with accuracies on the test set of 0.945 and 0.927, respectively. In [ 121 ], five nets were used to distinguish benign from malignant tumors. The reported accuracies were surprisingly variable; from worst to best, the results were VGG16 (0.5) and ResNet50 (0.68). In [ 137 ], AlexNet and LeNet were both used to distinguish three classes.

The overall accuracies achieved by the different CNN architectures that have been used extensively for brain tumor classification are summarized in Figure 11 . It shows that the majority of CNN models have achieved high performance for brain tumor classification tasks, in which transfer learning with ResNet, VGG, and GoogleNet showed more stable performance than other models, such as 3D CNN. Among the reviewed articles, five articles utilized 3D CNN for brain tumor classification, and the classification accuracy of those studies fluctuates wildly. The highest accuracy was 97.1%, achieved by Zhuge et al. [ 77 ], who trained a 3D CNN architecture with a dataset of 315 patients (210 HGG, 105 LGG). The lowest accuracy of 75% was obtained by Pei et al. [ 118 ], who used 398 brain MR image volumes for GBM vs. AST vs. OLI classification. In another study [ 117 ], the authors explored a 3D CNN model for OLI and AST classification using a very small dataset of 32 patients (16 OLI, 16 AST) and obtained a low accuracy of 80%. It seems that 3D CNN is a promising technique for realizing patient-wise diagnosis, and the accessibility of a large MRI dataset can hopefully improve the performance of 3D CNNs on brain tumor classification tasks.

Classification accuracy by CNN architecture.

5.4. The Effect of Pre-Processing and Data Augmentation Methods

Researchers have paid increasing amounts of attention to enhancing input image quality by conducting different preprocessing steps on brain MRI datasets before propagating them into CNN architectures. No studies have systematically tested the number and combination of operations that optimize classification accuracy. Figure 12 presents the overall accuracy obtained with different numbers of preprocessing operations. It shows that the studies that pre-processed input MR images collectively obtained higher classification accuracies than the studies that performed no preprocessing methods. However, it is not obvious that more steps led to better performance.

Classification accuracy by number of preprocessing operations.

As previously stated, data augmentation can create variations in the images that can improve the generalization capability of the models to new images, and different data augmentation techniques have been widely explored and applied to increase both the amount and the diversity of training data. Figure 13 illustrates the overall accuracy obtained with different numbers of data augmentation operations. It can be seen that studies that performed five data augmentation techniques achieved higher and more stable classification performance than the studies that performed fewer operations.

Classification accuracy by number of data augmentation operations.

The accuracy data do not support the use of any single data augmentation method. It is interesting to ask whether data augmentation techniques were implemented specifically in those studies that lacked training data. However, on average, there is little difference between the 59 studies including or the 27 omitting a data augmentation step. On average, the former included 233 cases or 4743 images, and the latter included 269 cases or 7517 images. Curiously, the number of studies employing data augmentation has fallen as a proportion among those published in 2022, both compared to the total and compared to those using pre-processing methods.

Figure 14 indicates the cumulative impact of factors that are not fully reported or considered in the studies reported in Table 4 . Articles with multiple analyses for which factors differed were scored 1 (i.e., missing). Data are derived from Table 4 , with the following exceptions: “Explainability considered” means that there was some analysis within the article on the information used to come to a diagnosis. Out-of-cohort testing occurred when CNN testing was performed on a cohort that was not used in the training/validation phase (i.e., different hospital or scanner). Author affiliations were derived from the author information in the DOI/CrossRef listed in the bibliography. An author was considered to have a clinical affiliation if their listed affiliations included a department of radiology, clinical neurology, neurosurgery, or oncology.

Histogram (left scale) and cumulative distribution (right scale) of factors not fully reported or considered in the studies reported in Table 4 .

From the figure, the category other performance criteria performed means that performance criteria other than accuracy were reported. Validation was considered to be not properly reported if it was not performed or if the methods used in the validation step were not clearly described. Training patients/images properly reported means that the number of patients/images in each category used for training/validation is explicitly defined. Both factors are relevant as separate images from the same patient and are not fully independent. Public data used means that the data used are available to other researchers. In practice, all of the public data used were gathered in other studies, and no non-public data were made available by any of the studies identified.

5.5. The Effect of Other Factors

Beyond showing accuracy gains, the surveyed articles rarely examined their generalization capability and interpretability. Only very few studies [ 73 , 120 ] tested their classification models on an independent dataset, and only one study [ 122 ] investigated the Black-Box characteristic of CNN models for brain tumor classification to ensure that the model they obtained was looking in the correct place for decision-making rather than at noise or unrelated artifacts.

A limitation of this survey arises from the challenge of making comparisons in an objective manner between studies to analyze how each degrading factor affects the classification performance. One reason is that some studies worked on the same classification task but utilized different datasets, preprocessing methods, or classification techniques. Another reason lies in the variety of performance metrics reported. While accuracy was the most popular performance metric, it was not universally reported. Based on the difficulties encountered in the preparation of the present review, we suggest that at the very least, all deep learning studies for classification clearly report the classification accuracy of the models constructed and the numbers of images/subjects of each class used for training, validation, and testing purposes.

5.6. Future Directions

It is clear from the comparative analysis presented in Table 4 b that CNN techniques and algorithms have great power and ability to handle medical MR data, but so far, but none of them are at the point of clinical usability. The challenges we have identified here must be appropriately addressed if CNN research is to be translated into clinic practice. This review has identified some common performance-degrading factors and potential solutions.

5.6.1. The Training Data Problem

An exorbitant number of training cases are required to train a deep learning algorithm from scratch. With a limited number of training data, transfer learning with fine-tuning on pre-trained CNNs was demonstrated to yield better results for brain tumor classification than training such CNNs from scratch [ 45 , 116 ]. This is an efficient method for training networks when training data are expensive or difficult to collect in medical fields. In addition, high hardware requirements and long training times are also challenges that CNN-based CADx brain tumor classification systems face in clinical applications today. The continued development of state-of-the-art CNN architectures has resulted with a voracious appetite for computing power. Since the cost of training a deep learning model scales with the number of parameters and the amount of input data, this implies that computational requirements grow at the rate of at least the square of the number of training data [ 140 ]. With pre-trained models, transfer learning is also promising to address the difficulties caused by high hardware requirements and long training times when adopting CNN-based CADx systems for brain tumor classification in clinical practice. There are many issues related to optimizing transfer learning that remain to be studied.

5.6.2. The Evaluation Problem

CADx systems are mainly used for educational and training purposes but not in clinical practice. Clinics still hesitate to use CADx-based systems. One reason for this is the lack of standardized methods for evaluating CADx systems in a realistic setting. The performance measures described in Section 4.2 are a useful and necessary baseline to compare algorithms, but they are all highly sensitive to the training set used, and more sophisticated tools are needed. It would be useful to define a pathway towards in-use performance evaluation, such as what was recently proposed for quantitative neuroradiology [ 141 ]. It is notable that many of the papers reviewed did not include any authors with a clinical background and that the image formats used to train the models were those typical of the AI research community (PNG) and not those of the radiology community (DICOM, NIfTI).

5.6.3. Explainability and Trust

The Black-Box nature of deep CNNs has greatly limited their application outside of a research context. To trust systems powered by CNN models, clinicians need to know how they make predictions. However, among the articles surveyed, very few addressed this problem. The authors in [ 142 ] proposed a prototypical part network (ProtoPNet) that can highlight the image regions used for decision-making and can explain the reasoning process for the classification target by comparing the representative patches of the test image with the prototypes learned from a large number of data. To date, several studies have tested the explanation model proposed in [ 142 ] that was able to highlight image regions used for decision making in medical imaging fields, such as for mass lesion classification [ 143 ], lung disease detection [ 144 , 145 ], and Alzheimer’s diseases classification [ 146 ]. Future research in the brain tumor classification field will need to test how explainable models influence the attitudes and decision-making processes of radiologists or other clinicians.

The lack of physician training on how to interact with CADx systems and how to interpret their results to make diagnostic decisions is a separate but related technical challenge that can reduce the performance of CADx systems in practice, something that is not addressed in any of the papers included in the review. A greater role for physicians in the research process may bring benefits both in terms of the relevance of research projects and the acceptance of their results.

In summary, the future of CNN-based brain tumor classification studies is very promising and focusing on the right direction with references to the challenges mentioned above would advance these studies from research labs to hospitals. We believe that our review provides researchers in the biomedical and machine learning communities with indicators for useful future directions for this purpose.

6. Conclusions

CADx systems may play an important role in assisting physicians in making decisions. This paper surveyed 83 articles that adopted CNNs for brain MRI classification and analyzed the challenges and barriers that CNN-based CADx brain tumor classification systems face today in clinical application and development. A detailed analysis of the potential factors that affect classification accuracy is provided in this study. From the comparative analysis in Table 4 b, it is clear that CNN techniques and algorithms have great power and ability to handle medical MR data. However, many of the CNN classification models that have been developed so far still are still lacking in one way or another in terms of clinical application and development. Research oriented towards appropriately addressing the challenges noted here can help drive the translation of CNN research into clinical practice for brain tumor classification. In this review, some performance degrading factors and their solutions are also discussed to provide researchers in the biomedical and machine learning communities with indicators for developing optimized CADx systems for brain tumor classification.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12081850/s1 , Table S1: Article Screening Recording.

Funding Statement

This research was funded by China Scholarship Council (grant number: 202008320283). And The APC was funded by a voucher belonging to author L.R.

Author Contributions

Conceptualization, C.T. (Claudia Testa), D.N.M., F.Z., L.R., Y.X.; methodology, C.T. (Claudia Testa), D.N.M., F.Z., L.R., Y.X.; formal analysis, C.T. (Caterina Tonon), C.T. (Claudia Testa), D.N.M., F.Z., L.R.; investigation, C.T. (Claudia Testa), D.N.M., F.Z., L.R.; re-sources, C.T. (Caterina Tonon), R.A., R.L.; data curation, D.N.M., Y.X.; writing—original draft preparation, Y.X.; writing—review and editing, C.T. (Caterina Tonon), C.T. (Claudia Testa), D.N.M., F.Z., L.R.; supervision, C.T. (Caterina Tonon), C.T. (Claudia Testa), D.N.M.; funding acquisition, C.T. (Caterina Tonon), R.A., R.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Informed consent statement, data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.