Abstract: Building heights are necessary for urban informatics, providing a significant basis for the planning and early warning of risks for urban construction. The shadow method, which can measure the heights of urban buildings on a large scale at a low cost, faces challenges such as low efficiency, accuracy, and robustness in building height inversion in complex shadow scenes. This study proposed a measurement method for these scenes. First, the shadows were measured and delineated using the fishing net method combined with multiple constraints. Second, the shadow lengths of all the zones divided were obtained, and the optimal values were determined using the quartile method and the bidirectional approximation strategy. Third, the shadow lengths were determined through a comprehensive assessment of the optimal values of all zones. The results show that 90.6% of building heights calculated using the new method exhibited absolute errors ranging from 0 to 5 m. Therefore, this method features elevated accuracy of building height inversion for various complex shadow scenes, laying a basis for research into the inversion and expansion of urban building heights.