Investigating multi-label classification for human values

Emi Ishita, Douglas W. Oard, Kenneth R. Fleischmann, An Shou Cheng, Thomas Clay Templeton

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

This paper describes the development of a scalable process for people and machines working together to identify sections of text that reflect specific human values. A total of 2,005 sentences from 28 prepared testimonies presented before hearings on Net neutrality were manually annotated for one or more of ten human values using an annotation frame based on experience annotating similar content using the Schwartz Values Inventory. Moderately good agreement suggests that meaningful distinctions can often be drawn by human annotators. Several k-Nearest-Neighbor classifiers were compared in this preliminary study, yielding results that appear promising and that clearly point to productive directions for future work.

Original languageEnglish
JournalProceedings of the ASIST Annual Meeting
Volume47
DOIs
Publication statusPublished - Nov 1 2010

Fingerprint

Audition
Labels
Classifiers
Values
neutrality
testimony
experience

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Library and Information Sciences

Cite this

Investigating multi-label classification for human values. / Ishita, Emi; Oard, Douglas W.; Fleischmann, Kenneth R.; Cheng, An Shou; Templeton, Thomas Clay.

In: Proceedings of the ASIST Annual Meeting, Vol. 47, 01.11.2010.

Research output: Contribution to journalArticle

Ishita, Emi ; Oard, Douglas W. ; Fleischmann, Kenneth R. ; Cheng, An Shou ; Templeton, Thomas Clay. / Investigating multi-label classification for human values. In: Proceedings of the ASIST Annual Meeting. 2010 ; Vol. 47.
@article{04cdb0cd474943c9ae430540af83f308,
title = "Investigating multi-label classification for human values",
abstract = "This paper describes the development of a scalable process for people and machines working together to identify sections of text that reflect specific human values. A total of 2,005 sentences from 28 prepared testimonies presented before hearings on Net neutrality were manually annotated for one or more of ten human values using an annotation frame based on experience annotating similar content using the Schwartz Values Inventory. Moderately good agreement suggests that meaningful distinctions can often be drawn by human annotators. Several k-Nearest-Neighbor classifiers were compared in this preliminary study, yielding results that appear promising and that clearly point to productive directions for future work.",
author = "Emi Ishita and Oard, {Douglas W.} and Fleischmann, {Kenneth R.} and Cheng, {An Shou} and Templeton, {Thomas Clay}",
year = "2010",
month = "11",
day = "1",
doi = "10.1002/meet.14504701116",
language = "English",
volume = "47",
journal = "Proceedings of the ASIST Annual Meeting",
issn = "1550-8390",
publisher = "Learned Information",

}

TY - JOUR

T1 - Investigating multi-label classification for human values

AU - Ishita, Emi

AU - Oard, Douglas W.

AU - Fleischmann, Kenneth R.

AU - Cheng, An Shou

AU - Templeton, Thomas Clay

PY - 2010/11/1

Y1 - 2010/11/1

N2 - This paper describes the development of a scalable process for people and machines working together to identify sections of text that reflect specific human values. A total of 2,005 sentences from 28 prepared testimonies presented before hearings on Net neutrality were manually annotated for one or more of ten human values using an annotation frame based on experience annotating similar content using the Schwartz Values Inventory. Moderately good agreement suggests that meaningful distinctions can often be drawn by human annotators. Several k-Nearest-Neighbor classifiers were compared in this preliminary study, yielding results that appear promising and that clearly point to productive directions for future work.

AB - This paper describes the development of a scalable process for people and machines working together to identify sections of text that reflect specific human values. A total of 2,005 sentences from 28 prepared testimonies presented before hearings on Net neutrality were manually annotated for one or more of ten human values using an annotation frame based on experience annotating similar content using the Schwartz Values Inventory. Moderately good agreement suggests that meaningful distinctions can often be drawn by human annotators. Several k-Nearest-Neighbor classifiers were compared in this preliminary study, yielding results that appear promising and that clearly point to productive directions for future work.

UR - http://www.scopus.com/inward/record.url?scp=84861430823&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84861430823&partnerID=8YFLogxK

U2 - 10.1002/meet.14504701116

DO - 10.1002/meet.14504701116

M3 - Article

AN - SCOPUS:84861430823

VL - 47

JO - Proceedings of the ASIST Annual Meeting

JF - Proceedings of the ASIST Annual Meeting

SN - 1550-8390

ER -