TY - JOUR
T1 - Few-Shot Text Style Transfer via Deep Feature Similarity
AU - Zhu, Anna
AU - Lu, Xiongbo
AU - Bai, Xiang
AU - Uchida, Seiichi
AU - Iwana, Brian Kenji
AU - Xiong, Shengwu
N1 - Funding Information:
Manuscript received October 25, 2018; revised December 11, 2019 and January 29, 2020; accepted May 6, 2020. Date of publication May 21, 2020; date of current version July 8, 2020. This work was supported in part by the National Natural Science Foundation of China under Grant No. 61703316, the National Major Program under Grant No. 2017YFB1402203, the Defense Industrial Technology Development Program under Grant No. 201910GC01 and Major Technological Innovation Projects in Hubei Province under Grant No. 2019AAA024. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Catarina Brites. (Corresponding author: Shengwu Xiong.) Anna Zhu, Xiongbo Lu, and Shengwu Xiong are with the School of Computer Science and Technology, Wuhan University of Technology, Wuhan 430070, China (e-mail: annakkk@live.com; luxiongbo01@gmail.com; xiongsw@whut.edu.cn).
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2020
Y1 - 2020
N2 - Generating text to have a consistent style with only a few observed highly-stylized text samples is a difficult task for image processing. The text style involving the typography, i.e., font, stroke, color, decoration, effects, etc., should be considered for transfer. In this paper, we propose a novel approach to stylize target text by decoding weighted deep features from only a few referenced samples. The deep features, including content and style features of each referenced text, are extracted from a Convolutional Neural Network (CNN) that is optimized for character recognition. Then, we calculate the similarity scores of the target text and the referenced samples by measuring the distance along the corresponding channels from the content features of the CNN when considering only the content, and assign them as the weights for aggregating the deep features. To enforce the stylized text to be realistic, a discriminative network with adversarial loss is employed. We demonstrate the effectiveness of our network by conducting experiments on three different datasets which have various styles, fonts, languages, etc. Additionally, the coefficients for character style transfer, including the character content, the effect of similarity matrix, the number of referenced characters, the similarity between characters, and performance evaluation by a new protocol are analyzed for better understanding our proposed framework.
AB - Generating text to have a consistent style with only a few observed highly-stylized text samples is a difficult task for image processing. The text style involving the typography, i.e., font, stroke, color, decoration, effects, etc., should be considered for transfer. In this paper, we propose a novel approach to stylize target text by decoding weighted deep features from only a few referenced samples. The deep features, including content and style features of each referenced text, are extracted from a Convolutional Neural Network (CNN) that is optimized for character recognition. Then, we calculate the similarity scores of the target text and the referenced samples by measuring the distance along the corresponding channels from the content features of the CNN when considering only the content, and assign them as the weights for aggregating the deep features. To enforce the stylized text to be realistic, a discriminative network with adversarial loss is employed. We demonstrate the effectiveness of our network by conducting experiments on three different datasets which have various styles, fonts, languages, etc. Additionally, the coefficients for character style transfer, including the character content, the effect of similarity matrix, the number of referenced characters, the similarity between characters, and performance evaluation by a new protocol are analyzed for better understanding our proposed framework.
UR - http://www.scopus.com/inward/record.url?scp=85088036814&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85088036814&partnerID=8YFLogxK
U2 - 10.1109/TIP.2020.2995062
DO - 10.1109/TIP.2020.2995062
M3 - Article
AN - SCOPUS:85088036814
VL - 29
SP - 6932
EP - 6946
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
SN - 1057-7149
M1 - 9098082
ER -