Face completion with pyramid semantic attention and latent codes

Shilei Cao, Kouichi Sakurai

研究成果: Chapter in Book/Report/Conference proceedingConference contribution

抄録

—Face completion, which is to reproduce the missing region of an incomplete face image, has achieved promising performance due to the adoption of generative adversarial network (GANs) and the development of GPU. However, current network frameworks always ignore that face images usually have strong semantic correlation and symmetry, failing to recover some semantically plausible details especially for filling in large contiguous holes. To produce visually realistic and semantically correct results, we propose a two-stage adversarial framework, the first stage is to produce coarse images and works as a prior for searching the most similar latent codes in reference sets, which are combined by composing their intermediate feature maps. Besides, the second stage captures texture information with our novel pyramid semantic attention block for fully using semantic information and embed the learned structure features into the inpainting process. Our attention layer considers not only the known contents but also our reconstructed parts, so that we can improve the realism of reconstructing parts, then apply this attention layer into a novel pyramid structure. In addition, we add weights in the loss function around the predicted boundary for encouraging our model to generate clearer contour lines and better interpolation properties. Empirically, the experiment on CelebA dataset shows our proposed method is effective to fill in large contiguous holes on the face images. Especially, the SSIM score of our model is nearly higher 0.1 than context encoder model.

本文言語英語
ホスト出版物のタイトルProceedings - 2020 8th International Symposium on Computing and Networking, CANDAR 2020
出版社Institute of Electrical and Electronics Engineers Inc.
ページ1-8
ページ数8
ISBN(電子版)9781728182216
DOI
出版ステータス出版済み - 11 2020
イベント8th International Symposium on Computing and Networking, CANDAR 2020 - Virtual, Naha, 日本
継続期間: 11 24 202011 27 2020

出版物シリーズ

名前Proceedings - 2020 8th International Symposium on Computing and Networking, CANDAR 2020

会議

会議8th International Symposium on Computing and Networking, CANDAR 2020
国/地域日本
CityVirtual, Naha
Period11/24/2011/27/20

All Science Journal Classification (ASJC) codes

  • 人工知能
  • 計算理論と計算数学
  • コンピュータ ネットワークおよび通信
  • コンピュータ サイエンスの応用
  • ソフトウェア

フィンガープリント

「Face completion with pyramid semantic attention and latent codes」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル