On the Discrete-Time Dynamics of Cross-Coupled Hebbian Algorithm

Principal/minor component analysis (PCA/MCA), generalized principal/minor component analysis (GPCA/GMCA), and singular value decomposition (SVD) algorithms are important techniques for feature extraction. In the convergence analysis of these algorithms, the deterministic discrete-time (DDT) method can reveal the dynamic behavior of PCA/MCA and GPCA/GMCA algorithms effectively. However, the dynamic behavior of SVD algorithms has not been studied quantitatively because of their special structure. In this paper, for the first time, we utilize the advantages of the DDT method in PCA algorithms analysis to study the dynamics of SVD algorithms. First, taking the cross-coupled Hebbian algorithm as an example, by concatenating the two cross-coupled variables into a single vector, we successfully get a PCA-like DDT system. Second, we analyze the discrete-time dynamic behavior and stability of the PCA-like DDT system in detail based on the DDT method, and obtain the boundedness of the weight vectors and learning rate. Moreover, further discussion shows the universality of the proposed method for analyzing other SVD algorithms. As a result, the proposed method provides a new way to study the dynamical convergence properties of SVD algorithms.