1: \begin{abstract}

2: The variational quantum eigensolver (VQE) is a promising algorithm to compute eigenstates and eigenenergies of a given quantum system that can be performed on a near-term quantum computer.

3: Obtaining eigenstates and eigenenergies in a specific symmetry sector of the system is often necessary for practical applications of the VQE in various fields ranging from high energy physics to quantum chemistry. 

4: It is common to add a penalty term in the cost function of the VQE to calculate such a symmetry-resolving energy spectrum; however, systematic analysis of the effect of the penalty term has been lacking, and the use of the penalty term in the VQE has not been justified rigorously.

5: In this work, we investigate two major types of penalty terms for the VQE that were proposed in previous studies.

6: We show that a penalty term of one of the two types works properly in that eigenstates obtained by the VQE with the penalty term reside in the desired symmetry sector.

7: We further give a convenient formula to determine the magnitude of the penalty term, which may lead to faster convergence of the VQE.

8: Meanwhile, we prove that the other type of penalty term does not work for obtaining the target state with the desired symmetry in a rigorous sense and even gives completely wrong results in some cases.

9: We finally provide numerical simulations to validate our analysis.

10: Our results apply to general quantum systems and lay the theoretical foundation for the use of the VQE with the penalty terms to obtain the symmetry-resolving energy spectrum of the system, which fuels the application of a near-term quantum computer.

11: \end{abstract}

12: