Domain Generalization (DG) aims to train models that perform well not only on the training (source) domains but also on novel, unseen target data distributions. A key challenge in DG is preventing overfitting to source domains, which can be mitigated by finding flatter minima in the loss landscape. In this work, we propose Quantization-aware Training for Domain Generalization (QT-DoG) and demonstrate that weight quantization effectively leads to flatter minima in the loss landscape, thereby enhancing domain generalization. Unlike traditional quantization methods focused on model compression, QT-DoG exploits quantization as an implicit regularizer by inducing noise in model weights, guiding the optimization process toward flatter minima that are less sensitive to perturbations and overfitting. We provide both theoretical insights and empirical evidence demonstrating that quantization inherently encourages flatter minima, leading to better generalization across domains. Moreover, with the benefit of reducing the model size through quantization, we demonstrate that an ensemble of multiple quantized models further yields superior accuracy than the state-of-the-art DG approaches with no computational or memory overheads. Our extensive experiments demonstrate that QT-DoG generalizes across various datasets, architectures, and quantization algorithms, and can be combined with other DG methods, establishing its versatility and robustness.

The figure below showcases GradCAM results from the PACS dataset, illustrating four experiments where each run targets a different domain while using others for training. These visualizations evidence that quantization focuses on better regions than ERM, and with a much larger receptive field. In certain cases, ERM does not even focus on the correct image region. It is quite evident that quantization pushes the model to learn more generalized patterns, leading to a model that is less sensitive to the specific details of the training set. These qualitative results confirm the quantitative evidence provided in the Table above.

We demonstrates that incorporating quantization in the Empirical Risk Minimization (ERM) process leads to finding flatter minima. Our analysis indicates that QT-DoG achieves flatter minima compared to both ERM and existing methods such as SAM and SWA.

Following the approach of previous work, we quantify local flatness \( \mathcal{F}_\gamma (\mathbf{w}) \) by measuring the expected change in loss values between a model with parameters \( \mathbf{w} \) and a perturbed model \( \mathbf{w}' \) on a sphere of radius \( \gamma \) centered at \( \mathbf{w} \):

\[ \mathcal{F}_\gamma (\mathbf{w})=\mathbb{E}_{\|\mathbf{w}'\|} \left[\mathcal{E}(\mathbf{w}') - \mathcal{E}(\mathbf{w})\right], \]

where \( \mathcal{E}(\mathbf{w}) \) represents the accumulated loss over samples from either source or target domains, depending on the evaluation context.

We thank Soumava Roy for assisting with the GradCAM visualizations across different datasets. We also thank Chen Zhao for his helpful comments and feedback. Moreover, we are grateful to the creators of the SWAD and LSQ GitHub repositories, as our code was built upon their work.