Professional Documents
Culture Documents
ISSN No:-2456-2165
Abstract:- Knee Osteoarthritis (OA) is an extremely To automatically localise the knee joints, we present a
common and degenerative musculoskeletal disease fully-convolutional neural network (FCN)-based method. A
worldwide which creates a significant burden on patients FCN is an end-to-end network that has been trained to make
with reduced quality of life and also on society because pixel-by-pixel predictions [3]. To automatically classify the
of its financial impact. Therefore, technical try and localized knee joints, we propose two methods: 1) training a
efforts to reduce the burden of the disease could help CNN from scratch for multi-class classification of knee OA
both patients and society. In this paper, an automated images, and 2) training a CNN to optimize a weighted ratio
novel method is proposed with a supported combination of two loss functions: categorical cross-entropy for multi-
of joint shape and modified Fully connected neural class classification and mean-squared error for
network (FCNN) based bone texture features, to reclassification. We compare the results of these methods to
differentiate between the knee radiographs with and those of WND-CHARM [4] and our previous research. We
without osteoarthritis. Moreover, an endeavor is formed also compare the classification results to both manual and
to explain the bone texture using CNN. Knee automated methods.We propose a novel pipeline for
radiographs from Osteoarthritis Initiative (OAI) and automatically quantifying knee OA severity, which
Multicenter Osteoarthritis (MOST) datasets are utilized includes an FCN for localising knee joints and a CNN
in this paper. The proposed models were trained on 8000 jointly trained for knee joint classification and
knee radiographs from OAI and evaluated on 3500 knee regression. The main contributions of this work
radiographs from MOST. The results demonstrate that include a fully-convolutional network (FCN)-based
fusing the proposed shape and texture parameters method for automatically localising knee joints, as
achieves the state-of-the art performance in well as training a network (CNN) from scratch that
radiographic OA detection yielding area under the ROC optimises a weighted ratio of both categorical cross-
curve (AUC) of 98.75% accuracy. entropy and mean-squared error for knee joint
regression. This multi-objective convolutional learning
Keywords:- Knee Osteoarthritis, KL grades, Automatic improves overall quantification while also providing
Detection, Fully Convolutional Neural Networks, multi-class classification and regression outputs at the
Classification and Regression. same time.
I. INTRODUCTION II. LITERATURE SURVEY
Knee Osteoarthritis (OA) may be a debilitating joint Classifying the severity of knee OA can be
disorder that mainly degrades the knee articular cartilage. accomplished by detecting variations in joint space width
Clinically, the most important pathological features of knee and the formation of osteophytes in the knee joints [5]. Yoo
OA include joint space narrowing, osteophyte formation, et al., in a recent approach, developed a scoring system to
and sclerosis. Knee OA has a high incidence among the predict radiographic and symptomatic knee OA risks using
elderly, obese, and people with a sedentary lifestyle. In its artificial neural networks (ANN) and KNHANES V-1 data.
severe stages, it causes excruciating pain and ends up in Then, the Shamir et al., proposed a WND-CHARM method ,
total joint arthroplasty. In the Early diagnosis period , it is a which is a multipurpose bio-medical image classifier [6]
crucial for clinical treatments and pathology [1]. Despite the was used to classify knee OA radiographs [7] and to detect
introduction of several imaging modalities like, Optical knee OA early using computer aided analysis. WND-
Coherence Tomography,MRI and radiography (X-ray) , an CHARM extracts handcrafted features from raw images and
ultrasound for augmented OA diagnosis, has been image transforms [8].
traditionally preferredand “gold standard” for preliminary
diagnosis of knee OA [2]. During this work, we train CNNs Convolutional neural networks (CNNs) have
from scratch to automatically quantify knee OA severity recently outperformed many methods based on hand-
using X-ray images. This involves two main steps: 1) crafted features in many computer vision tasks such as
automatically detecting and extracting the region of interest image recognition, automatic detection and segmentation,
(ROI) and localizing the knee joints, and 2) classifying the content-based image retrieval, and video classification.
localized knee joints. CNNs learn effective feature representations that are
The network is composed of four stages of The network's input is [256 256], and its output is the
convolutions, each followed by a max-pooling layer, same size.
and the final stage of convolutions is followed by an
up-sampling and a fully-convolutional layer. The first C. FCN Training
and second layer of convolution employ 32 filters The network was trained from scratch using training
each, the third layer consist of 64 filters, and the fourth samples of knee OA radiographs from the OAI and MOST
layer with 96 filters. The network employs uniform [3 datasets. Binary images with masks specifying the ROI: the
3] convolution and max pooling [2 2]. Following each knee joints are used to train the network. Figure 4 depicts an
convolution layer is a batch normalization and a example of a binary mask: the ground truth. The binary
rectified linear unit activation layer (ReLU). As the masks were generated from manual annotations of knee OA
network employs three stages of [2 2] max pooling, a radiographs using a fast annotation tool that we created. The
[8 8] up-sampling is performed after the final network was trained to minimise the total binary cross
convolution layer. Up-sampling is required for end-to- entropy between predicted and true pixels. We used the
end learning via back propagation from pixel-wise loss adaptive moment estimation (Adam) optimizer [20] with
and to obtain pixel-dense results. For pixel-based default parameters and discovered that it provided faster
classification, the final layer is a fully convolutional convergence than standard SGD. Figure 3 shows that the
layer with a kernel size of [1 1] and sigmoid activation. ROI (Region of Interest) regions are marked and segmented.
Fig. 4: The Fully Convolutional Network for automatically detecting knee joints
In the figure 4, the knee joints are extracted and Following the final pooling layer is a fully
deduce the bounding boxes of the knee joints using connected layer and a softmax dense layer. We
simple contour detection from the output predictions of include a drop out layer to avoid over-fitting. In the
FCN. n the proposed architecture, four convolutional final two convolutional layers (conv3 and conv4), as
layers are added , then four Max-pooling layers are added well as the fully connected layer, we apply an L2-norm
to reduce the dimensionality of the datasets. The fully weight regularization penalty of 0.01. (fc5). Applying a
connected layers are added at the end of the architecture. regularization penalty to other layers lengthens training
The output of the max-pooling layers is given as an input time while introducing no significant variation in
to the fully connected layers. The output from this layers learning curves. Using the Adam optimizer, the
are flattened, the weights are compared with the flattened network was trained to minimize categorical cross-
value to predict the Normal and OA images, and extract entropy loss [20]. The network's inputs are knee images
the knee joints from knee OA radiographs using the with dimensions of [200 300]. Based on the mean
bounding boxes. We upscale the bounding boxes from aspect ratio (1.6) of all extracted knee joints, we chose
the FCN output of size [256 256] to the original size of this size to roughly preserve the aspect ratio.
each knee OA radiograph before extracting the knee
joints to preserve the aspect ratio of the knee joints. E. Jointly training CNN for Classification and Regression
In general, assessing the severity of knee OA is
D. Quantifying knee OA severity using Convolutional based on multi-classifying knee images and assigning a
Neural Network KL grade to each distinct category [10]. We argued in
Convolutional neural networks are trained from our previous paper [1] that assigning a continuous
scratch using knee OA data, and networks are jointly grade (0-4) to knee images through regression is a
trained to minimize classification and regression losses, better approach for quantifying knee OA severity
allowing for a more accurate assessment of knee OA because the disease is progressive. However, there is no
severity. ground truth of KL grades on a continuous scale to
train a network directly for regression output with this
Training CNN for Classification: The network is approach. As a result, we use multi-objective
made up of five learned weight layers: four convolutional learning [8] to train networks to optimize
convolutional layers and one fully connected layer. a weighted-ratio of two loss functions: categorical
The network architecture is depicted in Figure 5. cross-entropy and mean-squared error. Mean squared
Due to the scarcity of training data, we considered a error informs the network about grade ordering, while
lightweight architecture with few layers, and the cross entropy informs it about grade quantization.
network has 5.4 million free parameters in total. We Optimizing a network with two loss functions
discovered that this architecture is the best for intuitively provides a stronger error signal and is a step
classifying knee images after experimenting with toward improving overall quantification, taking both
the number of convolutional layers and other classification and regression results into account. We
parameters. The network's convolutional layers are arrived at the final architecture shown in Figure6 after
followed by batch normalization and a rectified much experimentation. This network has six learned
linear unit activation layer (ReLU). There is a max weight layers: five convolutional layers and one fully
pooling layer after each convolutional stage.
Fig. 5: The proposed network architecture for simultaneous classification and regression
120
100
80 WIND-CHARM method
60 Fractal dimension
Fine-Tuned BVLC Caffe Net
40
Proposed Model
20
0
Sensitivity Specificity Accuracy F1-Score
Fig. 6: Comparison of proposed work Results with the state of art methods
Fig. 7: (a) Training (Tr) and validation (Val) accuracy (acc), (b) Training and validation loss for joint
classification(Clsf) and regression(Reg) training.
According to the results in Tables 2 and 3, the 0.898. Table 5 compares the precision, recall, F1 score,
network trained jointly for classification and and area under curve (AUC) of the networks trained
regression has a higher multi-class classification for classification and regression. These findings
accuracy of 98.75% and a lower mean-squared error of indicate that the network jointly trained for
0.661 than the previous network trained only for classification and regression learns a better
classification, which has a multi-class classification representation than the previous network trained only
accuracy of 98.75% and a mean-squared error of for classification.
From the Figure 10 shows some examples of because the individual categories are not equidistant from
various classifications: knee joints predicted as Normal, each other [3,4]. This could explain why the automatic
Doubtful, Moderate, Mild and Severe. These images show quantification has a low multi-class classification
minimal variations in terms of joint space width and accuracy. Because knee OA features such as joint space
osteophytes formation, making them challenging to narrowing, osteophytes formation, and sclerosis are
distinguish. Even though the KL grades are used in separately graded, using OARSI readings instead of KL
clinical settings to assess knee OA severity, there has grades may provide better results for automatic
been ongoing investigation and criticism of their use quantification.
Fig. 9: Proposed model predicted the given input image as a) Doubtful b) Moderate
Fig. 10: The obtained Confusion matrix for the proposed model