Deep Learning Model for Detecting COVID-19 on Chest X-ray using MATLAB

Let’s first understand human learning vs. machine learning.

Human Learning vs. Machine Learning

Let’s say, you have some data, and I am asking you to take a decision based on the data. What will you do? You will check the data and you will do some calculations on the data. Then you will tell me the decision. This is called human learning.

Data analytics is since beginning

Our survival or homo sapiens (humankind) survival is based on learning and identifying patterns from data, or you may say, data analytics. Like:

The four points above were the key factors for our ancestors to survive, What kind of food is edible and which not. Identifying different kinds of natural herbs for the treatment of different diseases. Protection from threatening animals or forecasting weather. In any case they identified the patterns between good and bad for survival. Without identifying a pattern, any kind of learning is not possible. And based on the pattern we take decisions. This is called human learning. Without pattern learning is not possible.

Now imagine that I am giving the same facilities to a machine. The machine will take a decision for us, then it is called Machine Learning. And machine learning is nothing else but an algorithm.

How Learning Works

Let’s take an example of the cause of day and night.

As all we know, day and night occur because of Earth rotation around the sun. Just forget this theory for a moment. And think of the causes of day and night. There may be three possible scenarios:

• Maybe there is a switch to On and Off the sun.
• Sun is rotating around Earth.
• Earth is rotating around Sun.

As we understand, in the above-mentioned scenarios only the last one is correct. And how did our ancestors identify the pattern of day and night? They used some calculations and analysis to get a pattern between the sunrise and the sunset.

Now think if you have similar kinds of scenarios for your data. What if you will choose the first scenario “switch off and on”? You can choose the first option but in that case your model can’t work if data keep changing. For example, in the above case summer has different timings for sunrise and sunset.

Next, if you choose the second scenario, you may get a high accuracy but your algorithm will be completely wrong.
If you chose the third one, you rock.

What I want to say is that you have to choose an appropriate machine learning algorithm based on your data. MATLAB provides you an app to select the best algorithm for your data. It is called classification app learner for classification and regression app learner for regression, in the case of machine learning.

How Deep Learning is different from Machine Learning

Deep learning is a part of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.

In deep learning, a computer model learns to perform classification tasks directly from images, texts, or numeric data.

In the case of machine learning, if I have to detect different kinds of vehicles, then first I will extract features of the vehicles and these features will be input for my machine learning algorithm.

In the case of deep learning there are hidden layers, which will extract features from different kinds of vehicles automatically and based on these features it will learn about types of cars.

Deep Learning success enabled by:

What Happens During Training in Deep Learning

• There are 10s to 100s of layers in deep learning that each learns to detect different features of an image, text, sound or time series data.
• Filters are applied to each training data at different resolutions, and the output of each feature extraction layer is used as the input to the next layer.
• Filters can start as very simple features extraction. Then it goes much deeper to extract features. In the case of an image, filters can start as very simple features, such as brightness and edges, and increase in complexity to features that uniquely define the object.

COVID-19 Detection Based on Chest X-ray Images

Dataset

We have total 798 sample images, 399 for COVID-19 and 399 normal X-ray images.

Why X-ray (Radiography)

It usually takes less than 15 minutes for an entire X-ray procedure.

X-ray images are digital, so a doctor can see them on a screen within minutes.

About ResNet-50

• ResNet-50 is a convolutional neural network that is 50 layers deep.
• ResNet, short for Residual Networks is a classic neural network used as a backbone for many computer vision tasks.
• This model was the winner of ImageNet challenge in 2015.
• You can load a pretrained version of the network trained on more than a million images
• The pretrained network can classify images into 1000 object categories

Workflow

We will follow the workflow below for a deep learning model on COVID-19.

• First we will create an image data store. We will save images in two different sub-folders by class name. In covid sub-folder infected X-ray images will be saved and in normal sub-folder uninfected X-ray images will be saved.

• We will split data for training and testing. To get the accuracy, testing is important.
• Next we will modify ResNet-50 network according to our data. Because in ResNet-50 millions of images are trained with 1000 of classes. In our case we have 798 images with 2 classes.
• Next step is to define training parameters, like what will be initial learning rate, maximum number of epochs and batch size etc.
• Further we will train our model and optimize hyper parameters if it is required.
• When the model is trained we will test the model on testing dataset to check its accuracy.
• If everything works fine you can further deploy your model on a hardware for real time application.

Explanation with Algorithm.

Download and Load the Dataset

To download the dataset check the references at the end of the article.

Let’s create an image database using imageDatastore. It will automatically label all images.

% Clear workspace
clear,clc, close all
% Images Datapath – You can modify your path accordingly 
datapath='dataset';
 
% Image Datastore
imds=imageDatastore(datapath, ...  
    'IncludeSubfolders',true, ...
    'LabelSource','foldernames'); %Datastore for image data

Count samples for every class(label)

All_samples=countEachLabel(imds) %Count files in ImageDatastore labels

Visualize some Images

Visualize the images and see how the images differ for each class. We will read random 6 images by applying random permutation.

% Number of Images
num_images=length(imds.Labels);

% Visualize random images
randper=randperm(num_images,6); %Random permutation of integers
figure;
for idx=1:length(randper)
    
    subplot(2,3,idx); %Create axes in tiled positions
    imshow(imread(imds.Files{randper(idx)}));
    title((imds.Labels(randper(idx))))
    
end

Split Data for Training and Testing

[imdsTrain,imdsTest] = splitEachLabel(imds,.8,'randomized'); %Split ImageDatastore labels by proportions

Load resnet50

net=resnet50;
    lgraph = layerGraph(net) % Extract all layers
        lgraph.Layers % visualize all layers
    lgraph.Layers(175).OutputSize % get number of classes in resnet-50
    clear net;

Count number of categories, in our case we have normal and covid

numClasses = numel(categories(imdsTrain.Labels)) %Number of array elements

Create a new layer

A fully connected layer

  newLearnableLayer = fullyConnectedLayer(numClasses, ...
        'Name','new_fc', ...
        'WeightLearnRateFactor',10, ...
        'BiasLearnRateFactor',10);

We require to replace the last 3 layers with new layers in ResNet-50

   lgraph = replaceLayer(lgraph,'fc1000',newLearnableLayer);
    %A softmax layer applies a softmax function to the input.
    newsoftmaxLayer = softmaxLayer('Name','new_softmax');
    lgraph = replaceLayer(lgraph,'fc1000_softmax',newsoftmaxLayer);
    %A classification layer computes the cross entropy loss for multi-class
   %  classification problems with mutually exclusive classes.
    newClassLayer = classificationLayer('Name','new_classoutput');
    lgraph = replaceLayer(lgraph,'ClassificationLayer_fc1000',newClassLayer);

View the Layers Architecture

figure,
    plot(lgraph)

Preprocessing on dataset

The function which reads data, is specified as a function handle. During training, a preprocess function will be applied automatically. We require this preprocess because images can be grey or colour, means a single matrix image for grey and 3 matrices for colour images. While training we require same dimensions of matrices, otherwise it will throw out an error.

Preprocess function is given at the end of the article.

imdsTrain.ReadFcn = @(filename)preprocess_images(filename);
    imdsTest.ReadFcn = @(filename)preprocess_images(filename);

Training options,

You can choose a small mini-batch size if you have less memory in GPU or if you don’t have supported GPU.

Training options for Adam optimizer (adaptive moment estimation)
Other solvers which you can choose:
‘sgdm’ — Use the stochastic gradient descent with momentum (SGDM) optimizer. You can specify the momentum value using the ‘Momentum’ name-value pair argument.
‘rmsprop’— Use the RMSProp optimizer. You can specify the decay rate of the squared gradient moving average using the ‘SquaredGradientDecayFactor’name-value pair argument.

options = trainingOptions('adam',...
        'MaxEpochs',5,'MiniBatchSize',20,...
        'Shuffle','every-epoch', ...
        'InitialLearnRate',1e-4, ...
        'Verbose',false, ...
        'Plots','training-progress'); %Options for training deep learning neural network
  

Data Augmentation

An image data augmenter configures multiple preprocessing options for image augmentation, such as resizing, rotation, and reflection. If you have similar kind of images, then you can apply data augmentation for better classification on any kind of chest X-ray images.

augmenter = imageDataAugmenter( ...
        'RandRotation',[-5 5],'RandXReflection',1,...
        'RandYReflection',1,'RandXShear',[-0.05 0.05],'RandYShear',[-0.05 0.05]);

Resizing all training images to [224 224] for ResNet architecture

Because in ResNet image size is set to [224 224].

augmentedImageDatastore Transform batches to augment image data

    auimds = augmentedImageDatastore([224 224],imdsTrain,'DataAugmentation',augmenter); 

Now training part

In my case training took 36 minutes 28 sec.

We use trainNetwork function to train the deep learning model. It requires three input arguments: image data, layers and a training option. We’ve created them all in the above steps.

    netTransfer = trainNetwork(auimds,lgraph,options);

Testing

Resize all testing images to [224 224] for ResNet architecture

  augtestimds = augmentedImageDatastore([224 224],imdsTest);
predicted_labels = classify(netTransfer,augtestimds);

Predicted classes are saved in predicted_labels.

Accuracy

    sum(predicted_labels==imdsTest.Labels)/numel(predicted_labels)*100

As you see, the accuracy is 100 percent on the testing data, not a single image is miss-classified. It means we have developed a perfect model to classify chest X-ray images for COVID-19.

Confusion Matrix

Measure the performance of our algorithm on the testing dataset. – This matrix also lets us know the correctly classified images and miss-classified images. By observing the confusion matrix we can check the performance of our deep learning model.

    % Actual Labels
actual_labels=imdsTest.Labels;

% Confusion Matrix
figure;
plotconfusion(actual_labels,predicted_labels)
title('Confusion Matrix: ResNet50');

Check the activation of a linear rectified unit

We will check the feature activation from activation_2_relu (second repeated linear rectified unit). Here we are checking for the 5th image from the image data store.

IMout = preprocess_images(imdsTest.Files{5});

    imResized = imresize(IMout,[[224 224]]);
    imageActivations = activations(netTransfer,imResized,'activation_2_relu');
   figure
    montage(imageActivations)

Visualize network features using deep dream

If you want to check what features are extracted from the CNN layer you can use deepDreamImage function.

   layer = 'bn_conv1';
channels = 1:25;

I = deepDreamImage(netTransfer,layer,channels, ...
    'PyramidLevels',1, ...
    'Verbose',0);

figure
for i = 1:25
    subplot(5,5,i)
    imshow(I(:,:,:,i))
end

Read one image from the test dataset and classify

im = preprocess_images(imdsTest.Files{15});

    imResized = imresize(im,[224 224]);
    [ class, score]=classify(netTransfer,imResized);
    figure
    imshow(imResized)
    title([ 'Predclass=' char(string(class)),', ','score=',num2str(max(score)),', ','origClass=', char(string(imdsTest.Labels(1)))])

Preprocessing functions

Resize matrix dimension of all images in m*n*3

function IMout = preprocess_images(filename)

I = imread(filename);

if ~ismatrix(I) % if matrix is not in form m*n
    I=rgb2gray(I); 
end
IMout = cat(3,I,I,I); % creating matrix in the form of m*n*3
end

Extract COVID-19 images from downloaded X-ray images

%Read csv file
clear,clc
aa=readtable('metadata.csv')

%extract Finding column and index of COVID-19 X-ray images
covid19=categorical(aa.finding);
filename=categorical(aa.filename);
idx=find(covid19=='COVID-19');


%Total Covid-19 X-ray images
nnz(idx)
covidimage=filename(idx);

% Extract COVID-19 infected X-ray images and write in a folder covid
for i=1:length(idx)
    im=imread(char(covidimage(i)));
 
    imwrite(im,['covid' '\', char(covidimage(i))])
end

For recorded webinar on detecting COVID-19 on Chest X-ray using MATLAB click here

References:
Download infected covid19 dataset
Download uninfected covid-19 dataset
Inspired by MathWorks blog here

Note:- Keep in mind that the COVID-19 analysis in this article is presented for educational purposes only and not allowed for publications. The goal is to spread awareness among faculties, researchers and students, on how deep learning and machine learning can make a big impact for such kind of analysis.

Download COVID-19 MATLAB app (.exe file) here.