Cascaded Temporal and Spatial Attention Network for solar adaptive optics image restoration

¹ University of Electronic Science and Technology of China, Xiyuan Avenue, West Hi-tech Zone, Chengdu, Sichuan 611731, PR China
e-mail: wangshuai0601@uestc.edu.cn
² Yangtze Delta Region Institute of University of Electronic Science and Technology of China, Chengdian Road, Kecheng, Quzhou, Zhejiang 324003, PR China
³ Institute of Optics and Electronics Chinese Academy of Sciences, Box 350, Shuangliu, Chengdu, Sichuan 610209, PR China
e-mail: chrao@ioe.ac.cn
⁴ University of Chinese Academy of Sciences, Yanqi Lake East Road, Huairou, Beijing 100049, PR China
⁵ Key Laboratory on Adaptive Optics, Chinese Academy of Sciences, Box 350, Shuangliu, Chengdu, Sichuan 610209, PR China

Received: 7 September 2022
Accepted: 13 March 2023

Abstract

Context. Atmospheric turbulence severely degrades the quality of images observed through a ground-based telescope. An adaptive optics (AO) system only partially improves the image quality by correcting certain level wavefronts, making post-facto image processing necessary. Several deep learning-based methods have recently been applied in solar AO image post-processing. However, further research is still needed to get better images while enhancing model robustness and using inter-frame and intra-frame information.

Aims. We propose an end-to-end network that can better handle solar adaptive image anisoplanatism by leveraging attention mechanisms, pixel-wise filters, and cascaded architecture.

Methods. We developed a cascaded attention-based deep neural network named Cascaded Temporal and Spatial Attention Network (CTSAN) for solar AO image restoration. CTSAN consists of four modules: optical flow estimation PWC-Net for inter-frame explicit alignment, temporal and spatial attention for dynamic feature fusion, temporal sharpness prior for sharp feature extraction, and encoder-decoder architecture for feature reconstruction. We also used a hard example mining strategy to create a loss function in order to focus on the regions that are difficult to restore, and a cascaded architecture to further improve model stability.

Results. CTSAN and the other two state-of-the-art (SOTA) supervised learning methods for solar AO image restoration are trained on real 705 nm photospheric and 656 nm chromospheric AO images supervised by corresponding Speckle images. Then all the methods are quantitatively and qualitatively evaluated on five real testing sets. Compared to the other two SOTA methods, CTSAN can restore clearer solar images, and shows better stability and generalization performance when restoring the lowest contrast AO image.