To cope with the uncertainty and variability of wind power, it is important for the power system to maintain adequate reserve capacity. The energy storage ability of the district heating system (DHS) provides considerable flexibility for the combined heat and power (CHP) units, and hence CHP can also participate in reserve capacity service. However, the CHP operation is restricted not only by its power condition, but also by the DHS heating condition, which brings difficulty in quantifying the CHP reserve flexibility. This paper focuses on properly assessing and utilizing the available reserve capacity of the CHP units as well as addressing the DHS heating constraints. The particularity for CHP reserve assessment is first analyzed and summarized as the component-coupling feature and the heating-dependency feature. Based on this, the regulating region method is proposed to describe the heating-restricted reserve capacity of the CHP units. Furthermore, an integrated power and heat dispatch approach is developed which utilizes the regulating region to formulate the available CHP reserve capacity. Finally, the numerical simulation is performed to demonstrate the necessity of precisely assessing the available reserve capacity of the CHP units and the benefit of utilizing this potential reserve flexibility for wind accommodation.