In this project, we propose a two-stage workflow used for the segmentation and scoring of lung diseases. The workflow inherits quantification, qualification, and visual assessment of lung diseases on X-ray images estimated by radiologists and clinicians. It requires the fulfillment of two core stages devoted to lung and disease segmentation as well as an additional post-processing stage devoted to scoring. The latter integrated block is utilized, mainly, for the estimation of segment scores and computes the overall severity score of a patient. The models of the proposed workflow were trained and tested on four publicly available X-ray datasets of COVID-19 patients and two X-ray datasets of patients with no pulmonary pathology. Based on a combined dataset consisting of 580 COVID-19 patients and 784 patients with no disorders, our best-performing algorithm is based on a combination of DeepLabV3+, for lung segmentation, and MA-Net, for disease segmentation. The proposed algorithms’ mean absolute error (MAE) of 0.30 is significantly reduced in comparison to established COVID-19 algorithms, BS-net and COVID-Net-S, possessing MAEs of 2.52 and 1.83 respectively. Moreover, the proposed two-stage workflow was not only more accurate but also computationally efficient, it was approximately 11 times faster than the mentioned methods. In summary, we proposed an accurate, time-efficient, and versatile approach for segmentation and scoring of lung diseases illustrated for COVID-19 and with broader future applications for pneumonia, tuberculosis, pneumothorax, amongst others.
In the proposed workflow, Stage I focuses on lung segmentation, where we curated and pre-processed three publicly available datasets: Darwin, Montgomery, and Shenzhen (Table 1). These datasets contain chest X-rays from patients diagnosed with COVID-19, pneumonia, or tuberculosis, serving as valuable resources for model training. Moving to Stage II, dedicated to disease segmentation and scoring, we collected and pre-processed four COVID-19 datasets, namely Actualmed COVID-19 Chest X-ray Dataset (ACCD), COVID-19 Radiography Database (CRD), COVID Chest X-Ray Dataset (CCXD), and Fig. 1 COVID Chest X-ray Dataset (FCXD) (Table 2). These datasets encompass CXRs from individuals diagnosed with COVID-19, sourced from over 40 medical institutions and hospitals. Additionally, to enhance network generalization, we included subjects with no pathological findings, ensuring a comprehensive training regime.
Table 1. Description of the datasets used for lung segmentation.
| Dataset | Training | Validation | Testing | Total |
|---|---|---|---|---|
| Darwin | 4884 | 611 | 611 | 6106 / 90% |
| Montgomery | 110 | 14 | 14 | 138 / 2% |
| Shenzhen | 452 | 57 | 57 | 566 / 8% |
| Total | 5446 / 80% | 682 / 10% | 682 / 10% | 6810 / 100% |
Table 2. Description of the datasets used for COVID-19 segmentation and scoring.
| Dataset | COVID-19 | Normal | Training | Validation | Testing | Total |
|---|---|---|---|---|---|---|
| ACCD | 49 | 0 | 39 | 5 | 5 | 49 / 4% |
| CRD | 104 | 0 | 83 | 10 | 11 | 104 / 8% |
| CCXD | 399 | 0 | 319 | 40 | 40 | 399 / 29% |
| FCXD | 28 | 0 | 22 | 3 | 3 | 28 / 2% |
| CXN | 0 | 431 | 344 | 43 | 44 | 431 / 31% |
| RSNA | 0 | 353 | 282 | 35 | 36 | 353 / 26% |
| Total | 580 / 43% | 784 / 57% | 1089 / 80% | 136 / 10% | 139 / 10% | 1364 / 100% |
The proposed workflow inherits the quantification and qualification of lung diseases (scoring and decision-making) from expert radiologists, and fulfills the following processing steps, as shown in Figure 1:
- Lung segmentation: pixel-level localization of the lungs and removal of unnecessary areas;
- Disease segmentation: pixel-level localization of the infected area of the lungs;
- Severity scoring: quantification and qualification of the infected area of the lungs.
Figure 1. Schematic illustration of the proposed workflow.
For the overall comparison of the proposed solutions, we showcase MAE estimated on the testing subset, the frame rate (FPS), the number of overall parameters, and MAC in Figure 2. The Y-axes “Parameters” and “MAC” refer to the overall number of parameters and the theoretical amount of multiply-accumulate operations for both stages of the proposed workflow. Similar to the accuracy estimation, we choose DeepLabV3+ as the core network of Stage I. In Stage II we tested nine networks. All networks were tested in the evaluation mode meaning that (a) normalization or dropout layers work in evaluation mode instead of training; (b) the automatic differentiation engine is deactivated. Adoption of the evaluation mode reduces memory usage and speeds up computations turning the back-propagation over the network. The main GPU used for testing is NVIDIA RTX 2080 Ti 11 Gb. The best performance (12.5 images/s) resulted in a proposed pipeline consisting of DeepLabV3+ (Stage I) and PSPNet (Stage II) whilst ranking sixth by MAE of the severity score. The most accurate solution consisted of DeepLabV3+ (Stage I) and MA-Net (Stage II), ranking eighth in the level of performance (7.9 images/s). On the other hand, the prediction speed of the tailor-made solutions, BS-net and COVID-Net-S, turned out to be the lowest making up 0.7 and 0.6 images/s respectively.
Figure 2. Overall comparison of the obtained solutions.
 |
 |
 |
| (a) U-net Severity score: 4 |
(b) U-net++ Severity score: 5 |
(c) DeepLabV3 Severity score: 0 |
 |
 |
 |
| (d) DeepLabV3+ Severity score: 3 |
(e) FPN Severity score: 4 |
(f) Linknet Severity score: 3 |
 |
 |
 |
| (g) PSPNet Severity score: 3 |
(h) PAN Severity score: 5 |
(i) MA-Net Severity score: 5 |
Figure 3. Comparison of the segmentation and severity score estimation of a COVID-19 subject from the ACCD dataset. A cyan delineation refers to the lung segmentation obtained by Stage I; a red mask is a disease mask obtained by Stage II; a yellow mask refers to the ground-truth segmentation of the disease.
In this study, we present a workflow for scoring and segmenting lung diseases, inspired by clinical practices for assessing lung infections from X-ray images. Our approach involves two core stages: lung and disease mask generation, followed by severity score estimation. We evaluated nine neural networks and found DeepLabV3+ for lung segmentation and MA-Net for disease segmentation to be the most accurate. Compared to BS-net and COVID-Net-S, our approach offers greater stability and faster prediction times.
- Operating System
- macOS
- Linux
- Windows (limited testing carried out)
- Python 3.8.x
- Required core packages: requirements.txt
Step 1: Download and install Miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_22.11.1-1-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.shStep 2: Clone the repository
git clone https://github.com/ViacheslavDanilov/covid_scoring.gitStep 3: Create a conda environment and install the requirements
cd covid_scoring
conda create --name scoring python=3.8 --no-default-packages --yes
conda activate scoring
pip install -r requirements.txt --no-cache-dirAll essential components of the study, including the curated dataset and trained models, have been made publicly available:
- Dataset 1 (lung segmentation): https://data.mendeley.com/datasets/8gf9vpkhgy/2.
- Dataset 2 (COVID-19 segmentation and scoring): https://data.mendeley.com/datasets/36fjrg9s69/1.
- Models: https://zenodo.org/doi/10.5281/zenodo.8393555.
Please cite our paper if you found our data, methods, or results helpful for your research:
Danilov V.V., Litmanovich D., Proutski A., Kirpich A., Nefaridze D., Karpovsky A., Gankin Y. (2022). Automatic scoring of COVID-19 severity in X-ray imaging based on a novel deep learning workflow. Scientific Reports, 12(1), 12791. DOI: https://doi.org/10.1038/s41598-022-15013-z