• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Zeng, C. (Zeng, C..) [1] | Kwong, S. (Kwong, S..) [2] | Zhao, T. (Zhao, T..) [3] | Wang, H. (Wang, H..) [4]

Indexed by:

Scopus

Abstract:

Automatically evaluating the quality of image captions can be very challenging since human language is quite flexible that there can be various expressions for the same meaning. Most current captioning metrics rely on token-level matching between candidate caption and the ground truth label sentences. It usually neglects the sentence-level information. Motivated by the auto-encoder mechanism and contrastive representation learning advances, we propose a learning-based metric I2CE (Intrinsic Image Captioning Evaluation). For learning the evaluation metric, we develop three progressive model structures capturing the sentence level representations–single branch model, dual branches model, and triple branches model. For evaluation of the proposed metric, we select one automatic captioning model and collect human scores on the quality of the generated captions. We introduce a statistical test on the correlation between human scores and metric scores. Our proposed metric I2CE achieves the Spearman correlation value of 51.42, which is better than the score of 41.95 achieved by one recently proposed BERT-based metric. The result is also better than the conventional rule-based metrics. Extensive results on the Composite-coco dataset and PASCAL-50S also validate the effectiveness of our proposed metric. The proposed metric could serve as a novel indicator of the intrinsic information between captions, which complements the existing ones. © 2022

Keyword:

Auto-encoder Contrastive learning Image captioning evaluation Sentence representations

Community:

  • [ 1 ] [Zeng, C.]Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong
  • [ 2 ] [Kwong, S.]Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong
  • [ 3 ] [Zhao, T.]College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
  • [ 4 ] [Wang, H.]Department of Computer Science and Technology, Tongji University, Shanghai, China

Reprint 's Address:

  • [Kwong, S.]Department of Computer Science, Hong Kong

Show more details

Related Keywords:

Source :

Information Sciences

ISSN: 0020-0255

Year: 2022

Volume: 609

Page: 913-930

8 . 1

JCR@2022

0 . 0 0 0

JCR@2023

ESI HC Threshold:61

JCR Journal Grade:1

CAS Journal Grade:1

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count: 7

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 2

Affiliated Colleges:

Online/Total:190/10016179
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1