Automatic Report Generation for Medical Images

Abstract

In this work, we propose an encoder-decoder-based automatic report generation system capable of generating radiology reports for chest x-rays. We tested five backbone Convolutional Neural Networks, namely VGG16, InceptionV3, Resnet50, MobileNet and NasNet mobile, to extract visual features and used Long Short-Term Long Memory (LSTM) to extract the text features from the reports. Both features are concatenated and given to a deep network for report generation. We performed experiments on publicly available Indiana University’s NLMCXR dataset. We evaluated our system against different backbones and evaluated accuracy and BLEU score. The result showed that our method achieved reliable and convincing results.