Exploring image specific structured loss for image annotation with incomplete labelling

Xing Xu, Atsushi Shimada, Rin Ichiro Taniguch

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we address the problem of image annotation with incomplete labelling, where the multiple objects in each training image are not fully labeled. The conventional one-versus-all SVM (OVA-SVM) that performs fairly well on full labelling decays drastically under the incomplete setting. Recently, structured learning method termed OVA-SSVM is proposed to boost the performance of OVA-SVM by modeling the structured associations of labels and show efficiency under incomplete setting. The OVA-SSVM assumes that each training sample includes a single label and adopts an loss measure of classification style that as long as one of the predicted label is correct, the overall prediction should be considered correct. However, this may not be appropriate for the multi-label annotation task. In this paper, we extend the OVA-SSVM method to the multi-label situation and design a novel image specific structured loss measure to account for the dependencies between predicted labels relying on the image-label associations. Then we develop an efficient optimization algorithm to learn the model parameters. Finally, we present extensive empirical results on two benchmark datasets with various degree of incompletion, and show that proposed method outperforms OVA-SSVM and achieves competitive performance compared with other state-of-the-art methods which are also designed for the issue of incomplete labelling.

Original languageEnglish
Title of host publicationComputer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers
PublisherSpringer Verlag
Pages704-719
Number of pages16
ISBN (Electronic)9783319168647
DOIs
Publication statusPublished - Jan 1 2015
Event12th Asian Conference on Computer Vision, ACCV 2014 - Singapore, Singapore
Duration: Nov 1 2014Nov 5 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9003
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other12th Asian Conference on Computer Vision, ACCV 2014
CountrySingapore
CitySingapore
Period11/1/1411/5/14

Fingerprint

Image Annotation
Labeling
Labels
Training Samples
Annotation
Optimization Algorithm
Efficient Algorithms
Decay
Benchmark
Prediction
Modeling

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Xu, X., Shimada, A., & Taniguch, R. I. (2015). Exploring image specific structured loss for image annotation with incomplete labelling. In Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers (pp. 704-719). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9003). Springer Verlag. https://doi.org/10.1007/978-3-319-16865-4_46

Exploring image specific structured loss for image annotation with incomplete labelling. / Xu, Xing; Shimada, Atsushi; Taniguch, Rin Ichiro.

Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag, 2015. p. 704-719 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9003).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Xu, X, Shimada, A & Taniguch, RI 2015, Exploring image specific structured loss for image annotation with incomplete labelling. in Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9003, Springer Verlag, pp. 704-719, 12th Asian Conference on Computer Vision, ACCV 2014, Singapore, Singapore, 11/1/14. https://doi.org/10.1007/978-3-319-16865-4_46
Xu X, Shimada A, Taniguch RI. Exploring image specific structured loss for image annotation with incomplete labelling. In Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag. 2015. p. 704-719. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-16865-4_46
Xu, Xing ; Shimada, Atsushi ; Taniguch, Rin Ichiro. / Exploring image specific structured loss for image annotation with incomplete labelling. Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag, 2015. pp. 704-719 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{5bc6616a7c614b5c9790168db8bdee59,
title = "Exploring image specific structured loss for image annotation with incomplete labelling",
abstract = "In this paper, we address the problem of image annotation with incomplete labelling, where the multiple objects in each training image are not fully labeled. The conventional one-versus-all SVM (OVA-SVM) that performs fairly well on full labelling decays drastically under the incomplete setting. Recently, structured learning method termed OVA-SSVM is proposed to boost the performance of OVA-SVM by modeling the structured associations of labels and show efficiency under incomplete setting. The OVA-SSVM assumes that each training sample includes a single label and adopts an loss measure of classification style that as long as one of the predicted label is correct, the overall prediction should be considered correct. However, this may not be appropriate for the multi-label annotation task. In this paper, we extend the OVA-SSVM method to the multi-label situation and design a novel image specific structured loss measure to account for the dependencies between predicted labels relying on the image-label associations. Then we develop an efficient optimization algorithm to learn the model parameters. Finally, we present extensive empirical results on two benchmark datasets with various degree of incompletion, and show that proposed method outperforms OVA-SSVM and achieves competitive performance compared with other state-of-the-art methods which are also designed for the issue of incomplete labelling.",
author = "Xing Xu and Atsushi Shimada and Taniguch, {Rin Ichiro}",
year = "2015",
month = "1",
day = "1",
doi = "10.1007/978-3-319-16865-4_46",
language = "English",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "704--719",
booktitle = "Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers",
address = "Germany",

}

TY - GEN

T1 - Exploring image specific structured loss for image annotation with incomplete labelling

AU - Xu, Xing

AU - Shimada, Atsushi

AU - Taniguch, Rin Ichiro

PY - 2015/1/1

Y1 - 2015/1/1

N2 - In this paper, we address the problem of image annotation with incomplete labelling, where the multiple objects in each training image are not fully labeled. The conventional one-versus-all SVM (OVA-SVM) that performs fairly well on full labelling decays drastically under the incomplete setting. Recently, structured learning method termed OVA-SSVM is proposed to boost the performance of OVA-SVM by modeling the structured associations of labels and show efficiency under incomplete setting. The OVA-SSVM assumes that each training sample includes a single label and adopts an loss measure of classification style that as long as one of the predicted label is correct, the overall prediction should be considered correct. However, this may not be appropriate for the multi-label annotation task. In this paper, we extend the OVA-SSVM method to the multi-label situation and design a novel image specific structured loss measure to account for the dependencies between predicted labels relying on the image-label associations. Then we develop an efficient optimization algorithm to learn the model parameters. Finally, we present extensive empirical results on two benchmark datasets with various degree of incompletion, and show that proposed method outperforms OVA-SSVM and achieves competitive performance compared with other state-of-the-art methods which are also designed for the issue of incomplete labelling.

AB - In this paper, we address the problem of image annotation with incomplete labelling, where the multiple objects in each training image are not fully labeled. The conventional one-versus-all SVM (OVA-SVM) that performs fairly well on full labelling decays drastically under the incomplete setting. Recently, structured learning method termed OVA-SSVM is proposed to boost the performance of OVA-SVM by modeling the structured associations of labels and show efficiency under incomplete setting. The OVA-SSVM assumes that each training sample includes a single label and adopts an loss measure of classification style that as long as one of the predicted label is correct, the overall prediction should be considered correct. However, this may not be appropriate for the multi-label annotation task. In this paper, we extend the OVA-SSVM method to the multi-label situation and design a novel image specific structured loss measure to account for the dependencies between predicted labels relying on the image-label associations. Then we develop an efficient optimization algorithm to learn the model parameters. Finally, we present extensive empirical results on two benchmark datasets with various degree of incompletion, and show that proposed method outperforms OVA-SSVM and achieves competitive performance compared with other state-of-the-art methods which are also designed for the issue of incomplete labelling.

UR - http://www.scopus.com/inward/record.url?scp=84938897005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938897005&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-16865-4_46

DO - 10.1007/978-3-319-16865-4_46

M3 - Conference contribution

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 704

EP - 719

BT - Computer Vision - ACCV 2014 - 12th Asian Conference on Computer Vision, Revised Selected Papers

PB - Springer Verlag

ER -