Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset

Edy Mulyanto; Esther Irawati Setiawan; Eko Mulyanto Yuniarno; Mauridhi Hery Purnomo

doi:10.1109/CIVEMSA45640.2019.9071632

Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset

Edy Mulyanto, Esther Irawati Setiawan, Eko Mulyanto Yuniarno, Mauridhi Hery Purnomo

Research Center for Artificial Intelligence and Health Technology (KATK)

Institut Teknologi Sepuluh Nopember

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Citations (Scopus)

Abstract

Image captioning is a challenge in computer vision research. This paper extends research on automatic image captioning generation in the Indonesian dimension. Description in Indonesian sentences is generated for unlabeled images. The dataset used is FEEH-ID, this is the first Indonesian image captioning dataset. This research is crucial due to unavailability of a corpus for image captioning in Indonesian. This paper will compare the experimental results in the FEEH-ID dataset with English, Chinese and Japanese datasets using the CNN and LSTM models. The performance of the model proposed in the test set provides promising results of 50.0 for the BLEU-1 score and 23.9 for BLEU-3, which is above average of the Bleu evaluation results in other language datasets. The merging model between CNN and LSTM displays pretty good results for the FEEH-ID dataset. The experimental results will be better with a larger dataset.

Original language	English
Title of host publication	2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781538683446
DOIs	https://doi.org/10.1109/CIVEMSA45640.2019.9071632
Publication status	Published - Jun 2019
Event	24th Annual IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Tianjin, China Duration: 14 Jun 2019 → 16 Jun 2019

Publication series

Name	2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings

Conference

Conference	24th Annual IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019
Country/Territory	China
City	Tianjin
Period	14/06/19 → 16/06/19

Keywords

CNN
FEEH-ID
LSTM
image captioning

Access to Document

10.1109/CIVEMSA45640.2019.9071632

Cite this

Mulyanto, E., Setiawan, E. I., Yuniarno, E. M., & Purnomo, M. H. (2019). Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset. In 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings Article 9071632 (2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CIVEMSA45640.2019.9071632

Mulyanto, Edy ; Setiawan, Esther Irawati ; Yuniarno, Eko Mulyanto et al. / Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset. 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. (2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings).

@inproceedings{99d5ee9444374dbca64e9660f9a18302,

title = "Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset",

abstract = "Image captioning is a challenge in computer vision research. This paper extends research on automatic image captioning generation in the Indonesian dimension. Description in Indonesian sentences is generated for unlabeled images. The dataset used is FEEH-ID, this is the first Indonesian image captioning dataset. This research is crucial due to unavailability of a corpus for image captioning in Indonesian. This paper will compare the experimental results in the FEEH-ID dataset with English, Chinese and Japanese datasets using the CNN and LSTM models. The performance of the model proposed in the test set provides promising results of 50.0 for the BLEU-1 score and 23.9 for BLEU-3, which is above average of the Bleu evaluation results in other language datasets. The merging model between CNN and LSTM displays pretty good results for the FEEH-ID dataset. The experimental results will be better with a larger dataset.",

keywords = "CNN, FEEH-ID, LSTM, image captioning",

author = "Edy Mulyanto and Setiawan, {Esther Irawati} and Yuniarno, {Eko Mulyanto} and Purnomo, {Mauridhi Hery}",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 24th Annual IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 ; Conference date: 14-06-2019 Through 16-06-2019",

year = "2019",

month = jun,

doi = "10.1109/CIVEMSA45640.2019.9071632",

language = "English",

series = "2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings",

address = "United States",

}

Mulyanto, E, Setiawan, EI, Yuniarno, EM & Purnomo, MH 2019, Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset. in 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings., 9071632, 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 24th Annual IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019, Tianjin, China, 14/06/19. https://doi.org/10.1109/CIVEMSA45640.2019.9071632

Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset. / Mulyanto, Edy; Setiawan, Esther Irawati; Yuniarno, Eko Mulyanto et al.
2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. 9071632 (2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset

AU - Mulyanto, Edy

AU - Setiawan, Esther Irawati

AU - Yuniarno, Eko Mulyanto

AU - Purnomo, Mauridhi Hery

PY - 2019/6

Y1 - 2019/6

N2 - Image captioning is a challenge in computer vision research. This paper extends research on automatic image captioning generation in the Indonesian dimension. Description in Indonesian sentences is generated for unlabeled images. The dataset used is FEEH-ID, this is the first Indonesian image captioning dataset. This research is crucial due to unavailability of a corpus for image captioning in Indonesian. This paper will compare the experimental results in the FEEH-ID dataset with English, Chinese and Japanese datasets using the CNN and LSTM models. The performance of the model proposed in the test set provides promising results of 50.0 for the BLEU-1 score and 23.9 for BLEU-3, which is above average of the Bleu evaluation results in other language datasets. The merging model between CNN and LSTM displays pretty good results for the FEEH-ID dataset. The experimental results will be better with a larger dataset.

AB - Image captioning is a challenge in computer vision research. This paper extends research on automatic image captioning generation in the Indonesian dimension. Description in Indonesian sentences is generated for unlabeled images. The dataset used is FEEH-ID, this is the first Indonesian image captioning dataset. This research is crucial due to unavailability of a corpus for image captioning in Indonesian. This paper will compare the experimental results in the FEEH-ID dataset with English, Chinese and Japanese datasets using the CNN and LSTM models. The performance of the model proposed in the test set provides promising results of 50.0 for the BLEU-1 score and 23.9 for BLEU-3, which is above average of the Bleu evaluation results in other language datasets. The merging model between CNN and LSTM displays pretty good results for the FEEH-ID dataset. The experimental results will be better with a larger dataset.

KW - CNN

KW - FEEH-ID

KW - LSTM

KW - image captioning

UR - http://www.scopus.com/inward/record.url?scp=85084646761&partnerID=8YFLogxK

U2 - 10.1109/CIVEMSA45640.2019.9071632

DO - 10.1109/CIVEMSA45640.2019.9071632

M3 - Conference contribution

AN - SCOPUS:85084646761

T3 - 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings

BT - 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 24th Annual IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019

Y2 - 14 June 2019 through 16 June 2019

ER -

Mulyanto E, Setiawan EI, Yuniarno EM, Purnomo MH. Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset. In 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. 9071632. (2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2019 - Proceedings). doi: 10.1109/CIVEMSA45640.2019.9071632

Automatic Indonesian Image Caption Generation using CNN-LSTM Model and FEEH-ID Dataset

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this