Home>Results

  • Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

[期刊论文]

STD-Net: Spatio-Temporal Decomposition Network for Video Demoiring With Sparse Transformers

Share
Edit Delete 报错

author:

Niu, Yuzhen (Niu, Yuzhen.) [1] (Scholars:牛玉贞) | Xu, Rui (Xu, Rui.) [2] | Lin, Zhihua (Lin, Zhihua.) [3] | Unfold

Indexed by:

EI Scopus SCIE

Abstract:

The problem of video demoireing is a new challenge in video restoration. Unlike image demoireing, which involves removing static and uniform patterns, video demoireing requires tackling dynamic and varied moire patterns while maintaining video details, colors, and temporal consistency. It is particularly challenging to model moire patterns for videos with camera or object motions, where separating moire from the original video content across frames is extremely difficult. Nonetheless, we observe that the spatial distribution of moire patterns is often sparse on each frame, and their long-range temporal correlation is not significant. To fully leverage this phenomenon, a sparsity-constrained spatial self-attention scheme is proposed to concentrate on removing sparse moire efficiently for each frame without being distracted by dynamic video content. The frame-wise spatial features are then correlated and aggregated via the local temporal cross-frame-attention module to produce temporal-consistent high-quality moire-free videos. The above decoupled spatial and temporal transformers constitute the Spatio-Temporal Decomposition Network, dubbed STD-Net. For evaluation, we present a large-scale video demoireing benchmark featuring various real-life scenes, camera motions, and object motions. We demonstrate that our proposed model can effectively and efficiently achieve superior performance on video demoireing and single image demoireing tasks. The proposed dataset is released at https://github.com/FZU-N/LVDM.

Keyword:

Image restoration sparse transformer spatio-temporal network video demoireing video restoration

Community:

  • [ 1 ] [Niu, Yuzhen]Fuzhou Univ, Coll Comp & Data Sci, Fujian Key Lab Network Comp & Intelligent Informa, Fuzhou 350108, Peoples R China
  • [ 2 ] [Xu, Rui]Fuzhou Univ, Coll Comp & Data Sci, Fujian Key Lab Network Comp & Intelligent Informa, Fuzhou 350108, Peoples R China
  • [ 3 ] [Liu, Wenxi]Fuzhou Univ, Coll Comp & Data Sci, Fujian Key Lab Network Comp & Intelligent Informa, Fuzhou 350108, Peoples R China
  • [ 4 ] [Niu, Yuzhen]Minist Educ, Engn Res Ctr Bigdata Intelligence, Fuzhou 350108, Peoples R China
  • [ 5 ] [Xu, Rui]Minist Educ, Engn Res Ctr Bigdata Intelligence, Fuzhou 350108, Peoples R China
  • [ 6 ] [Liu, Wenxi]Minist Educ, Engn Res Ctr Bigdata Intelligence, Fuzhou 350108, Peoples R China
  • [ 7 ] [Lin, Zhihua]Res Inst Alipay Informat Technol Co Ltd, Hangzhou 310000, Peoples R China

Reprint 's Address:

  • 刘文犀

    [Liu, Wenxi]Fuzhou Univ, Coll Comp & Data Sci, Fujian Key Lab Network Comp & Intelligent Informa, Fuzhou 350108, Peoples R China

Show more details

Version:

Source :

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY

ISSN: 1051-8215

Year: 2024

Issue: 9

Volume: 34

Page: 8562-8575

8 . 3 0 0

JCR@2023

CAS Journal Grade:1

Cited Count:

WoS CC Cited Count:

30 Days PV: 2

Online/Total:253/10741754
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1