1: \begin{abstract}

2: In deep learning, stochastic gradient descent (SGD) and its momentum-based variants are widely used for optimization, but they typically suffer from slow convergence. Conversely, existing adaptive learning rate optimizers speed up convergence but often compromise generalization. To resolve this issue, we propose a novel optimization method designed to accelerate SGD's convergence without sacrificing generalization. Our approach reduces the variance of the historical gradient, improves first-order moment estimation of SGD by applying Wiener filter theory, and introduces a time-varying adaptive gain. Empirical results demonstrate that SGDF (SGD with Filter) effectively balances convergence and generalization compared to state-of-the-art optimizers. The code is available at \textcolor{magenta}{https://github.com/LilYau350/SGDF-Optimizer}.

3: % In Deep Learning, stochastic gradient descent (SGD) and its momentum-based variants are widely used in optimization, but they usually face the problem of slow convergence. Meanwhile, existing adaptive learning rate optimizers accelerate convergence but often at the expense of generalization. To address this contradiction, we propose a novel optimization method that aims to accelerate the convergence of SGD without loss of generalization. This approach is based on the idea of reducing the variance of the historical gradient, enhancing the first-order moment estimation of the SGD by applying Wiener Filter theory, and introducing a time-varying adaptive gain. Experimental results show that SGDF (SGD with Filter) achieves a balance between convergence and generalization compared to state-of-the-art optimizers. The code is available at \textcolor{magenta}{https://anonymous.4open.science/r/SGDF-Optimizer-27E5}.

4: \end{abstract}

5: