To realize the fast and accurate counting of winter wheat at the seedling stage in fields, we propose a recognition method based on an improved YOLOv4 model. Firstly, we employed a simplified MobileNetv3 neural network instead of the standard CSPDarknet53 network structure. Besides, we added an adaptive image scaling layer in front of the MobileNetv3 network. Finally, we utilized the coyote optimization algorithm (COA) to optimize the learning rate and convolution kernel size. Results showed that the average precision (AP) values of the improved network model for 2-leaf and 3-leaf winter wheat were 96.46% and 93.87%, respectively. The mean average accuracy (mAP) was 95.15% and the average recognition speed was 0.07 s. These indicators were the best, compared with the YOLOv4, YOLOv3, and Faster-RCNN models. Also, the mAP was 12.28% higher than the standard YOLOv4model, and the average recognition speed was 1.49 times faster. Therefore, this method can achieve the fast and accurate counting of winter wheat during the seedling stage in the field.