The association between heterozygosity and fitness is positive but weak on average and varies between studies. Inbreeding has been invoked as the driving force between the positive heterozygosity-fitness associations, yet in spatio-temporally stable environments a negative correlation is expected. Furthermore, different patterns can arise because of the effects of natural selection on different loci and variation can be expected among groups of individuals that experience different levels of stress. In this paper we report on fitness-heterozygosity associations in the winter moth for six allozyme loci. The relationship is estimated for males and females separately, in four areas differing in their degree of fragmentation, and variation among loci is modelled. We introduce a linear mixed model framework to achieve this analysis. This approach differs from more traditional (multiple) regression analyses and allows testing specific interactions. We show that fitness, as estimated by body size, is negatively correlated with heterozygosity, but only so in females. This association does not vary significantly among loci and the four areas. We speculate that a trade-off between fitness-consequences of inbreeding and outbreeding at different stages of the winter moth life cycle could explain the observed patterns.