Challenges in classifying privacy policies by machine learning with word-based features

Keishiro Fukushima, Daisuke Ikeda, Toru Nakamura, Shinsaku Kiyomoto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we discuss challenges when we try to automatically classify privacy policies using machine learning with words as the features. Since it is difficult for general public to understand privacy policies, it is necessary to support them to do that. To this end, the authors believe that machine learning is one of the promising ways because users can grasp the meaning of policies through outputs by a machine learning algorithm. Our final goal is to develop a system which automatically translates privacy policies into privacy labels [1]. Toward this goal, we classify sentences in privacy policies with category labels, using popular machine learning algorithms, such as a naive Bayes classifier. We choose these algorithms because we could use trained classifiers to evaluate keywords appropriate for privacy labels. Therefore, we adopt words as the features of those algorithms. Experimental results show about 85% accuracy. We think that much higher accuracy is necessary to achieve our final goal. By changing learning settings, we identified one reason of low accuracies such that privacy policies include many sentences which are not direct description of information about categories. It seems that such sentences are redundant but maybe they are essential in case of legal documents in order to prevent misinterpreting. Thus, it is important for machine learning algorithms to handle these redundant sentences appropriately.

Original languageEnglish
Title of host publicationProceedings of 2018 the 2nd International Conference on Cryptography, Security and Privacy, ICCSP 2018
PublisherAssociation for Computing Machinery
Pages62-66
Number of pages5
ISBN (Electronic)9781450363617
DOIs
Publication statusPublished - Mar 16 2018
Event2nd International Conference on Cryptography, Security and Privacy, ICCSP 2018 - Guiyang, China
Duration: Mar 16 2018Mar 18 2018

Publication series

NameACM International Conference Proceeding Series

Other

Other2nd International Conference on Cryptography, Security and Privacy, ICCSP 2018
CountryChina
CityGuiyang
Period3/16/183/18/18

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this

Fukushima, K., Ikeda, D., Nakamura, T., & Kiyomoto, S. (2018). Challenges in classifying privacy policies by machine learning with word-based features. In Proceedings of 2018 the 2nd International Conference on Cryptography, Security and Privacy, ICCSP 2018 (pp. 62-66). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3199478.3199486