Image Transforms

Locally-Structured Unitary Network and Nonseparable Lapped Transforms

Locally-Structured Unitary Network (LSUN)

We propose a design method and theoretical framework for a learnable, self-supervised linear autoencoder that incorporates locally adaptive structures into non-separable overlapping orthogonal transforms, achieving locality.

The proposed Locally-Structured Unitary Network (LSUN) is theoretically applicable to signals of any dimension (tensors of any order).

Furthermore, it can capture the tangent spaces of manifolds embedded in high-dimensional data through its local structure.

Due to the global orthogonality of the network, a decoder can be easily obtained as an adjoint network of the encoder. The no-DC leakage property (analogous to the admissibility in wavelet transforms) is structurally guaranteed, making LSUN well-suited for hierarchical modeling. Structurally, it enables dimensionality reduction without requiring deep structures that combine nonlinear functions like neural networks or iterative algorithms such as those found in sparse optimization problems. It can also be integrated as a linear layer within other models.

LSUN enables example-based design and can be used as a replacement for or a preprocessing step before Principal Component Analysis (PCA) or Proper Orthogonal Decomposition (POD) to map high-dimensional data into a lower-dimensional feature space.

The ease of incorporating overlapping and hierarchical structures is a key advantage of this approach.

With the increasing adoption of data-driven modeling techniques, expectations are growing for applications in disaster prevention and the elucidation of complex physical phenomena.

In addition, data-driven modeling techniques are becoming increasingly important for designing digital twins in Cyber-Physical Systems (CPS).

LSUN is also highly compatible with deep neural networks and is expected to serve as a new type of linear layer replacing convolutional layers. Moreover, incorporating LSUN into sparse modeling frameworks may contribute to advancements in existing algorithms.

MATLAB/PyTorch Source Code

Research Achievements

Journal Papers

International Conferences

  • Godage Yasas, Shogo Muramatsu, Tangent Space Sampling of Video Sequence with Locally Structured Unitary Network, Proc. of 2023 IEEE International Conference on Visual Communications and Image Processing, DOI: 10.1109/VCIP59821.2023.10402611, Dec. 2023.

Acknowledgment

This research was supported by KAKENHI Grant-in-Aid for Scientific Research (A) (22H00512).

Nonseparable Lapped Orthognal Transform

MATLAB Source Code

Publications

Journal Papers

  • Chen Zhiyu and Shogo Muramatsu, Multi-focus Image Fusion based on Multiple Directional LOTs, IEICE Trans. on Fundamentals, vol.E98-A, no.11, pp.2360-2365, Nov. 2015.
  • Chen Zhiyu and Shogo Muramatsu, SURE-LET Poisson Denoising with Multiple Directional LOTs, IEICE Trans. on Fundamentals, vol.E98-A, no.8, pp. 1820-1828, Aug. 2015.
  • Natsuki Aizawa, Shogo Muramatsu and Masahiro Yukawa, Image Restoration with Multiple DirLOTs, IEICE Trans. on Fundamentals, Vol.E96-A,No.10,pp.1954-1961,DOI: 10.1587/transfun.E96.A.1954, Oct. 2013.
  • Shogo Muramatsu, Dandan Han, Tomoya Kobayashi and Hisakazu Kikuchi: Directional Lapped Orthogonal Transform: Theory and Design, IEEE Trans. on Image Processing, Vol.21, No.5, pp.2434-2448, DOI: 10.1109/TIP.2011.2182055, May 2012.
  • Shogo Muramatsu, Tomoya Kobayashi, Minoru Hiki and Hisakazu Kikuchi: Boundary Operation of 2-D Non-separable Linear-phase Paraunitary Filter Banks, IEEE Trans. on Image Processing, Vol.21, No.4, pp.2314-2318, DOI: 10.1109/TIP.2011.2181527, April 2012.
  • Atsuyuki Adachi, Shogo Muramatsu, Hisakazu Kikuchi, Constraints of Second-Order Vanishing Moments on Lattice Structures for Non-separable Orthogonal Symmetric Wavelets, IEICE Trans. on Fundamentals, Vol. E92-A, No. 3, pp.788-797, Mar. 2009. (Summary)
  • Shogo Muramatsu, Akihiko Yamada and Hitoshi Kiya, A Design Method of Multidimensional Linear-phase Paraunitary Filter Banks with a Lattice Structure, IEEE Transactions on Signal Processing, vol. 47, no. 3, pp. 690-700, DOI: 10.1109/78.747776, Mar. 1999.

International Conferences

  • Zhiyu Chen and Shogo Muramatsu, Poisson denoising with multiple Directional LOTs, Proc. of 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1234-1238, May 2014.
  • Natsuki Aizawa and Shogo Muramatsu, FISTA-Based Image Restoration with Multiple DirLOTs, Proc. of IWAIT 2013, Jan. 2013.
  • Shogo Muramatsu, Natsuki Aizawa and Masahiro Yukawa, Image Restoration with Union of Directional Orthonormal DWTs, Proc. of APSIPA ASC 2012, Dec. 2012.
  • Shogo Muramatsu: SURE-LET Image Denoising with Multiple Directional LOTs, Proc. of 2012 Picture Coding Symposium (PCS2012), May 2012.
  • Shogo Muramatsu and Dandan Han: Image Denoising with Union of Directional Orthonormal DWTs, IEEE Proc. of 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.1089-1092, Mar. 2012.
  • Shogo Muramatsu, Dandan Han and Hisakazu Kikuchi, ”SURE-LET Image Denoising with Directional LOTs, Proc. of APSIPA ASC 2011, Thu-PM.PS1.9, Xi’an, China, Oct. 18 – 21, 2011.
  • Shogo Muramatsu, Tomoya Kobayashi, Dandan Han and Hisakazu Kikuchi, Design Method of Directional GenLOT with Trend Vanishing Moments, Proc. of APSIPA ASC 2010, pp.692-701, Biopolis, Singapore, Dec. 14 – 17, 2010,
  • Shogo Muramatsu, Dandan Han, Tomoya Kobayashi and Hisakazu Kikuchi, Theoretical Analysis of Trend Vanishing Moments for Directional Orthogonal Transforms, Proc. of PCS2010, pp.130-133, Nagoya, Japan, Dec. 7-9, 2010.
  • Tomoya Kobayashi, Shogo Muramatsu and Hisakazu Kikuchi, 2-D Nonseparable GenLOT with Trend Vanishing Moments, IEEE Proc. of International Conf. on Image Proc. (ICIP2010), Hong Kong, pp.385-388, Sep. 2010.
  • Tomoya Kobayashi, Shogo Muramatsu, Hisakazu Kikuchi, Two-Degree Vanishing Moments on 2-D Non-separable GenLOT, IEEE Proc. of 2009 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS2009), pp.248-251, Kanazawa, Japan, Dec. 7-9, 2009.
  • Shogo Muramatsu and Minoru Hiki, Block-Wise Implementation of Directional GenLOT‘, IEEE Proc. of International Conference on Image Processing (ICIP2009), pp.3977-3980, Cairo, Egypt, Nov. 7-11 2009.

Nonseparable Lapped Orthogonal Transforms

MATLAB Source Code

Publications

Journal Papers

  • Shogo Muramatsu, Kosuke Furuya and Naotaka Yuki, Multidimensional Nonseparable Oversampled Lapped Transforms: Theory and Design，IEEE Trans. on Signal Process.，Vol. 65, No. 5,  pp.1251 -1264, DOI:  10.1109/TSP.2016.2633240, Mar. 2017.
  • Kosuke Furuya, Shintaro Hara, Kenta Seino and Shogo Muramatsu, Boundary Operation of 2-D Non-separable Oversampled Lapped transforms,  APSIPA Transactions on Signal and Information Processing, Vol. 5, pp.1-9, DOI:10.1017/ATSIP.2016.3, April 2016.

International Conferences

  • Shogo Muramatsu, Masaki Ishii and Zhiyu Chen, Efficient Parameter Optimization for Example-Based Design of Non-separable Oversampled Lapped Transform, Proc. of 2016 IEEE Intl. Conf. on Image Processing (ICIP), Sept. 2016.
  • Shogo Muramatsu, Structured Dictionary Learning with 2-D Non-separable Oversampled Lapped Transform, Proc. of 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2643-2647, May 2014
  • Kousuke Furuya, Shintaro Hara and Shogo Muramatsu, Boundary Operation of 2-D non-separable Oversampled Lapped Transforms, Proc. of Asia Pacific Signal and Information Proc. Assoc. Annual Summit and Conf. (APSIPA ASC), Kaohsiung, Taiwan, Nov. 2013
  • Shogo Muramatsu and Natsuki Aizawa, Image Restoration with 2-D Non-separable Oversampled Lapped Transforms, Proc. of 2013 IEEE International Conference on Image Processing(ICIP), Sep. 2013
  • Shogo Muramatsu and Natsuki Aizawa, Lattice Structures for 2-D Non-separable Oversampled Lapped Transforms, Proc. of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2013

Acknowlegement

These works are supported by JSPS KAKENHI JP23560443, JP26420347.

Related Works

Links