1: \begin{abstract}

2:             This study investigates imposing inequality constraints on the outputs of CNNs, with application to weakly supervised segmentation. In the context of deep networks, constraints are commonly handled with {\em penalties} for their simplicity, and despite their well-known limitations. Lagrangian-dual optimization has been largely avoided, mainly due to the computational complexity and stability/convergence issues caused by alternating {\em explicit} dual updates/projections and stochastic optimization. Several studies showed that, for deep CNNs, the theoretical and practical advantages of Lagrangian optimization over penalties do not materialize in practice. We propose {\em log-barrier extensions}, which approximate Lagrangian optimization of constrained-CNN problems with a sequence of unconstrained losses. Unlike standard interior-point and log-barrier methods, our formulation does not need an initial feasible solution. Furthermore, we provide a new technical result, which shows that the proposed extensions yield an upper bound on the duality gap. This generalizes the duality-gap result of standard log-barriers, yielding sub-optimality certificates for feasible solutions. While sub-optimality is not guaranteed for non-convex problems, our result shows that log-barrier extensions are a principled way to approximate Lagrangian optimization for constrained CNNs via {\em implicit} dual variables. We report comprehensive constrained-CNN experiments, showing that our formulation outperforms several penalty-based methods, in terms of accuracy and training stability.

3:             

4:         \end{abstract}

5: