Social Media Image Caption Generation Using Deep Learning

  Omkar Nitin Shinde,  Rishikesh Gawde,  Anurag Paradkar

Image captioning for Social Media—the task of providing caption of the content within an image—lies at the intersection of Computer Vision (CV) and Natural Language Processing (NLP). In this paper, we present a model based on a recurrent architecture that combines the recent advances in computer vision and machine translation and thus it can be used to generate captions for an image. It is an integral task which requires semantic understanding of images and the ability of generating Captions with proper relevant meaning and structure. The main aim of the paper is to train Convolutional Neural Network (CNN) and Long Short term memory (LSTM) model with many hyper parameters which extract features from the image and map these features to their appropriate descriptive keywords and apply it on a large dataset of pictures and combine the results with a Recurrent Neural Network (RNN) to generate dynamic and suitable captions and hash tags for social media for the classified image. Using the Flickr8K, Flickr30K datasets and also we had designed our own dataset of different categories of image in order to show more accuracy in generating captions.

Keywords- Image Captioning, Computer Vision, Convolutional Neural Network, Recurrent Neural Network, Long Short term memory

Unique Identification Number - IJEDR2004036
Page Number(s) - 222-228
Pubished in - Volume 8 | Issue 4 | December 2020
DOI (Digital Object Identifier) -    http://doi.one/10.1729/Journal.25198
Publisher - IJEDR (ISSN - 2321-9939)

  Omkar Nitin Shinde,  Rishikesh Gawde,  Anurag Paradkar,   "Social Media Image Caption Generation Using Deep Learning", International Journal of Engineering Development and Research (IJEDR), ISSN:2321-9939, Volume.8, Issue 4, pp.222-228, December 2020, Available at :http://www.ijedr.org/papers/IJEDR2004036.pdf