Enhancing Winter Wheat Yield Estimation With a CNN-Transformer Hybrid Framework Utilizing Multiple Remotely Sensed Parameters

To address insufficient feature extraction due to individual deep learning models' limitations in capturing local and global features, and the tendency to underestimate high yields and overestimate low yields, a new deep learning model called CNN-Transformer with serial connection (CNN-Transformer_s) was introduced to estimate winter wheat yield by combining the local feature extraction strengths of Convolutional Neural Networks (CNN) with the global information extraction abilities of Transformer networks utilizing self-attention mechanisms. The remote sensing technology included temperature and spectral response indicators; the Vegetation Temperature Condition Index (VTCI), Leaf Area Index (LAI), and Fraction of Photosynthetically Active Radiation (FPAR) aggregated over 10-day periods. Compared to the CNN model, the Transformer model, the CNN-Transformer model with parallel connection (CNN-Transformer_p), and the Transformer-CNN model with serial connection (Transformer-CNN_s), the CNN-Transformer_s achieved a higher accuracy in estimating winter wheat yield (R2=0.70, RMSE=420.39 kg/ha, MAPE=7.65%), which was capable of extracting more information related to yield from various remotely sensed parameters and addressing the problems of high yield underestimation and low yield overestimation observed. The robustness and generalization of the CNN-Transformer_s was further assessed through the 5-fold cross-validation and the leave-one-year-out methods. Additionally, utilizing the CNN-Transformer_s, the study uncovered the cumulative impact of the winter wheat growth period, examined how incrementally adding data at 10-day intervals affects yield estimation, and assessed the model's proficiency in depicting growth accumulation during the growth process. The findings indicated that the model well identified the crucial growth phase of winter wheat between late March and early May.