利用改进YOLOv5s模型检测番茄果实成熟度及外观品质

doi:10.16420/j.issn.0513-353x.2023-0859

摘要/Abstract

摘要：

以粉果番茄为试验材料，基于深度学习方法开展了番茄果实成熟度和外观品质的检测研究。试验中共采集番茄图片数据2 036张，通过处理扩增至5 316张，然后将数据进行标注和文件转换，构建了试验用数据集；通过在YOLOv5s模型中加入CA注意力机制、替换Stem block结构、结合识别需求优化检测层尺度、替换K-means++聚类算法来实现SC-YOLOv5s识别精度提升，提高模型的特征表达能力；通过在SC-YOLOv5s模型中加入Fire module结构进行轻量化卷积、降低Bottleneck模块的参数量来实现SC-YOLOv5s-lite轻量化设计，提升模型在硬件上的检测速度；将SC-YOLOv5s-lite模型在训练集上进行训练优化、消融试验和性能对比，结果表明，SC-YOLOv5s-lite模型内存大小为7.73 M，其准确率为89.04%，召回率83.35%，平均精度91.34%，检测时间为143 ms，相比于YOLOv5s，模型参数量降低了45.57%，模型大小压缩了44.86%，平均精度提升3.98%，检测时间减少20.99%，优势明显，更适合于硬件上部署。

关键词: 番茄, 成熟度, 外观品质, 检测, 深度学习, 计算机视觉

Abstract:

The study was conducted on the detection of tomato fruit maturity and appearance quality based on deep learning methods using pink tomato as the experimental material. Two thousand and thirty-six tomato image data were collected and amplified to 5 316 through preprocessing. Then，the data was annotated and converted into files to construct an experimental dataset. The experiment improves the accuracy of SC-YOLOv5s by adding CA attention mechanism，replacing the Stem block structure，optimizing the detection layer scale based on recognition requirements，and replacing the K-means++ clustering algorithm to improve the model’s feature expression ability. By adding a fire module structure to SC-YOLOv5s for lightweight convolution and reducing the parameter count of the Bottleneck module，the SC-YOLOv5s-lite lightweight design is achieved，improving the detection speed of the model on hardware；Train and optimize the SC-YOLOv5s-lite model on the training set. The results showed that the memory usage of the SC-YOLOv5s-lite model was 7.73 M，with an accuracy rate of 89.04%，a recall rate of 83.35%，an average accuracy of 91.34%，and a detection time of 143 ms. Compared to YOLOv5s，the model parameter quantity is reduced by 54.57%，model size is compressed by 44.86%，with an average accuracy improvement of 3.98%，and the detection time is reduced by 20.99%. It has obvious advantages and is more suitable for hardware deployment.

Key words: tomato, maturity, appearance quality, detection, deep learning, computer vision

孙宇朝, 李守豪, 夏秀波, 杨玮, 李民赞, 张焕春. 利用改进YOLOv5s模型检测番茄果实成熟度及外观品质[J]. 园艺学报, 2024, 51(2): 396-410.

SUN Yuchao, LI Shouhao, XIA Xiubo, YANG Wei, LI Minzan, ZHANG Huanchun. Detecting Tomato Fruit Ripeness and Appearance Quality Based on Improved YOLOv5s[J]. Acta Horticulturae Sinica, 2024, 51(2): 396-410.

图/表 19

参考文献 31

[1]	Chen Qiao-song, Chen Peng-chang, Li Pei, Zhang Ya-ling, Deng Xin, Sun Kai-wei, Wang Jin. 2023. Multi-scale feature selection and fusion for object detection. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 35 (2):227-234. (in Chinese)
	陈乔松, 陈鹏昌, 李佩, 张亚玲, 邓欣, 孙开伟, 王进. 2023. 基于多尺度特征选择与融合的目标检测方法. 重庆邮电大学学报(自然科学版), 35 (2):227-234.
[2]	Du Ming-yang, Zhang Tian-zhu. 2022. Beautiful unmanned farms in the digital landscape of agriculture. Vegetables,(9):1-9. (in Chinese)
	杜名扬, 张天柱. 2022. 农业数字化风景下的靓点无人农场. 蔬菜,(9):1-9.
[3]	Feng Fan. 2022. The application of internet of things technology in modern agriculture under the background of smart agriculture. The Farmers Consultant,(21):25-27. (in Chinese)
	冯帆. 2022. 智慧农业背景下物联网技术在现代农业中的应用. 农家参谋,(21):25-27.
[4]	Feng Jun-hui, Li Zhi-wei, Rong You-li, Sun Zhi-li. 2021. Identification of mature tomatoes based on an algorithm of modified circular hough transform. Journal of Chinese Agricultural Mechanization, 42 (4):190-196. (in Chinese)
	冯俊惠, 李志伟, 戎有丽, 孙志立. 2021. 基于改进Hough圆变换算法的成熟番茄果实识别. 中国农机化学报, 42 (4):190-196. doi: 10.13733/j.jcam.issn.2095-5553.2021.04.27
[5]	Gao Yan-zhao, Tao Chang-yong. 2023. Hardware-accelerated consistent computing structure for signal processing and deep learning. Journal of National University of Defense Technology, 45 (2):112-120. (in Chinese)
	高彦钊, 陶常勇. 2023. 信号处理与深度学习硬件加速的一致性计算结构. 国防科技大学学报, 45 (2):112-120.
[6]	He Jie, Ma Ting-juan, Zhang Yong-liang, Zhang Ya-juan, 2022. Construction of evaluation model and index system of blended learning in higher vocational education aiming at deep learning. Vocational and Technical Education, 43 (26):64-70. (in Chinese)
	贺杰, 马婷娟, 张永良, 张亚娟. 2022. 指向深度学习的高职混合式学习评价模型及指标体系构建. 职业技术教育, 43 (26):64-70.
[7]	Huang Bo. 2022. Countermeasures and analysis of talent loss in the background of rural revitalization. Practical Rural Technology,(4):46-48. (in Chinese)
	黄波. 2022. 乡村振兴背景下的人才流失问题对策与分析. 农村实用技术,(4):46-48.
[8]	Huang Tong-bin, Huang He-qing, Li Zhen, Lü Shi-lei, Xue Xiu-yun, Dai Qiu-fang, Wen Wei. 2022. Citrus fruit recognition method based on the improved model of YOLOv5. Journal of Huazhong Agricultural University, 41 (4):170-177. (in Chinese)
	黄彤镔, 黄河清, 李震, 吕石磊, 薛秀云, 代秋芳, 温威. 2022. 基于YOLOv5改进模型的柑橘果实识别方法. 华中农业大学学报, 41 (4):170-177.
[9]	Kang S. 2022. Research on intelligent video detection of small targets based on deep learning intelligent algorithm. Computational Intelligence and Neuroscience, 16(5):67-69.
[10]	Li Yan-zhou, He Yan-zhou, Qin Feng, Qian Wan-qiang, Wu Mei, Qiao Xi. 2023. Identification of spartina alterniflora based on convolutional neural network. Journal of Chinese Agricultural Mechanization, 44 (4):159-166. (in Chinese) doi: 10.13733/j.jcam.issn.20955553.2023.04.022
	李岩舟, 何艳洲, 覃锋, 钱万强, 吴媚, 乔曦. 2023. 基于卷积神经网络的互花米草识别研究. 中国农机化学报, 44 (4):159-166.
[11]	Liu Fang, Liu Yu-kun, Lin Sen, Guo Wen-zhong, Xu Fan, Zhang Bai. 2020. Fast recognition method for tomatoes under complex environments based on improved YOLO. Transactions of the Chinese Society for Agricultural Machinery, 51 (6):229-237. (in Chinese)
	刘芳, 刘玉坤, 林森, 郭文忠, 徐凡, 张白. 2020. 基于改进型YOLO的复杂环境下番茄果实快速识别方法. 农业机械学报, 51 (6):229-237.
[12]	Liu Zheng, Wei Chuan-cheng. 2021. Key points,difficulties and countermeasures for promoting the comprehensive revitalization of rural areas. Economic Review Journal,(10):122-128. (in Chinese)
	刘铮, 魏传成. 2021. 推进乡村全面振兴的重点、难点及对策. 经济纵横,(10):122-128.
[13]	Luo Xi-wen, Liao Juan, Zang Ying, Ou Ying-gang, Wang Pei. 2022. Developing from mechanized to smart agricultural production in China. Strategic Study of CAE, 24 (1):46-54. (in Chinese) doi: 10.15302/J-SSCAE-2022.01.005
	罗锡文, 廖娟, 臧英, 区颖刚, 汪沛. 2022. 我国农业生产的发展方向:从机械化到智慧化. 中国工程科学, 24 (1):46-54. doi: 10.15302/J-SSCAE-2022.01.005
[14]	Ma Jin-lin, Zhang Yu, Ma Zi-ping, Mao Kai-ji. 2022. Research progress of lightweight neural network convolution design. Journal of Frontiers of Computer Science and Technology, 16 (3):512-528. (in Chinese)
	马金林, 张裕, 马自萍, 毛凯绩. 2022. 轻量化神经网络卷积设计研究进展. 计算机科学与探索, 16 (3):512-528. doi: 10.3778/j.issn.1673-9418.2107056
[15]	Song Ling, Cao Mian, Hu Xiao-chun, Jia Pei-yuan, Chen Yan, Chen Ning-jiang. 2023. Detection of cassava leaf diseases under complicated background based on YOLOX. Transactions of the Chinese Society for Agricultural Machinery, 54 (3):301-307. (in Chinese)
	宋玲, 曹勉, 胡小春, 贾沛沅, 陈燕, 陈宁江. 2023. 基于YOLOX的复杂背景下木薯叶病害检测方法. 农业机械学报, 54 (3):301-307.
[16]	Sun Feng-gang, Wang Yun-lu, Lan Peng, Zhang Xu-dong, Chen Xiu-de, Wang Zhi-jun. 2022. Identification of apple fruit diseases using improved YOLOv5s and transfer learning. Transactions of the Chinese Society of Agricultural Engineering, 38 (11):171-179. (in Chinese)
	孙丰刚, 王云露, 兰鹏, 张旭东, 陈修德, 王志军. 2022. 基于改进YOLOv5s和迁移学习的苹果果实病害识别方法. 农业工程学报, 38 (11):171-179.
[17]	Wang Hai-qun, Zhang Cheng-jun, Zhang Yi. 2023. A lightweight flame detection algorithm based on YOLOv4. Journal of Shandong University of Science and Technology(Natural Science), 42 (1):91-99. (in Chinese)
	王海群, 张成君, 张怡. 2023. 基于YOLOv4的轻量级火焰检测算法. 山东科技大学学报(自然科学版), 42 (1):91-99.
[18]	Wang J J, Feng C, Wang L Y, Li G L, He B. 2022. Detection of weak and small targets in forward-looking sonar image using multi-branch shuttle neural network. IEEE Sensors Journal, 22 (7):6772-6783.
[19]	Wang Liang, Zhang Chao. 2023. A lightweight pedestrian detection method based on YOLOv5. Industrial Control Computer, 36 (4):84-86,89. (in Chinese)
	王亮, 张超. 2023. 一种基于YOLOv5的轻量型行人检测方法. 工业控制计算机, 36 (4):84-86,89.
[20]	Wu Di, Li Ming-hui, Ma Wen-kai, Li Rui-tong, Li Yan. 2023. Surface defect detection of steel based on improved Yolov5. Journal of Shaanxi University of Science & Technology, 41 (2):162-169. (in Chinese)
	吴敌, 李明辉, 马文凯, 李睿童, 李艳. 2023. 基于改进YOLOv5的钢材表面缺陷检测. 陕西科技大学学报, 41 (2):162-169.
[21]	Xi Hou-yin, Zhang Dong, Zhou Tao, Yang Yun-xiao. 2021. Research on the method of picking up overlapping tomato fruits by picking robot. Journal of Agricultural Mechanization Research, 43 (12):17-23,50. (in Chinese)
	郗厚印, 张栋, 周涛, 杨云霄. 2021. 采摘机器人识别抓取重叠番茄果实的方法研究. 农机化研究, 43 (12):17-23,50.
[22]	Xiong Jun-tao, Zheng Zhen-hui, Liang Jia-en, Zhong Zhuo, Liu Bo-lin, Sun Bao-xia. 2020. Citrus detection method in night environment based on improved YOLO v3 network. Transactions of the Chinese Society for Agricultural Machinery, 51 (4):199-206. (in Chinese)
	熊俊涛, 郑镇辉, 梁嘉恩, 钟灼, 刘柏林, 孙宝霞. 2020. 基于改进YOLO v3网络的夜间环境柑橘识别方法. 农业机械学报, 51 (4):199-206.
[23]	Yang Qian-shan, Zhou De-bing, Tu Wan-ying, Liu Du-yang. 2023. Concrete crack detection research based on offline data enhancement and YOLOv5. Shanxi Architecture, 49 (8):63-66. (in Chinese)
	阳茜汕, 卓德兵, 涂婉莹, 刘杜洋. 2023. 基于离线数据增强与YOLOv5的混凝土裂缝检测研究. 山西建筑, 49 (8):63-66.
[24]	Yue You-jun, Sun Bi-yu, Wang Hong-jun, Zhao Hui. 2021. Object detection of tomato fruit based on cascade RCNN. Science Technology and Engineering, 21 (6):2387-2391. (in Chinese)
	岳有军, 孙碧玉, 王红君, 赵辉. 2021. 基于级联卷积神经网络的番茄果实目标检测. 科学技术与工程, 21 (6):2387-2391.
[25]	Zha Wen-wen, Pan Wei-hao, Chen Cheng-peng, Chen Wei, Gu Li-chuan, Jiao Jun. 2023. Study on pose recognition and tracking of herd-raised pigs based on edge computing and improved YOLOv5. Journal of Northeast Agricultural University, 54 (3):83-96. (in Chinese)
	查文文, 潘伟豪, 陈成鹏, 陈威, 辜丽川, 焦俊. 2023. 基于边缘计算与改进YOLOv5的群养生猪姿态识别及跟踪研究. 东北农业大学学报, 54 (3):83-96.
[26]	Zhang Huai-feng, He Xiang-jian, Wu Qiang. 2006. Generic object detection:a survey. Journal of Yunnan Minzu University(Natural Sciences Edition), 15 (4):261-267. (in Chinese)
	张淮峰, 何祥健, 吴强. 2006. 通用目标检测算法研究进展与评述. 云南民族大学学报(自然科学版), 15 (4):261-267.
[27]	Zhang Hui-feng, Xing Yu-tong, Zhang Miao, Li Wen-xu, Li Cong. 2022. Comparison of epiphyseal localization accuracy by different deep learning models. Chinese Journal of Forensic Medicine, 37 (3):267-271. (in Chinese)
	张会丰, 邢雨彤, 张淼, 李文旭, 李聪. 2022. 不同深度学习模型的骨骺定位准确率比较. 中国法医学杂志, 37 (3):267-271.
[28]	Zhang Lei, Liu Qi-fang, Nie Hong-mei, Wang Chen, Niu Fan. 2022. Tomato fruit dection based on improved YOLOV4 network model. Journal of Chinese Agricultural Mechanization, 43 (12):162-169. (in Chinese)
	张磊, 刘琪芳, 聂红玫, 王晨, 牛帆. 2022. 基于改进YOLOV4网络模型的番茄果实检测. 中国农机化学报, 43 (12):162-169.
[29]	Zhou Shen-he, Shao Wu-chang, Wang Qiang. 2023. Overview of small object detection algorithms based on deep learning. Digital Technology & Application, 41 (2):156-158. (in Chinese)
	周升和, 邵武长, 王强. 2023. 基于深度学习的小目标检测算法综述数字技术与应用, 41 (2):156-158.
[30]	Zhu De-li, Wen Rui, Xiong Jun-yi. 2023. Lightweight corn silk detection network incorporating with coordinate attention mechanism. Transactions of the Chinese Society of Agricultural Engineering, 39 (3):145-153. (in Chinese)
	朱德利, 文瑞, 熊俊逸. 2023. 融合坐标注意力机制的轻量级玉米花丝检测. 农业工程学报, 39 (3):145-153.
[31]	Zu Linlu. 2023. Research on the data-driven model for predicting tomato growth in solar greenhouse[Ph. D. Dissertation]. Tai’an: Shandong Agricultural University. (in Chinese)
	祖林禄. 2023. 数据驱动的日光温室番茄果实生长预测模型研究[博士论文]. 泰安: 山东农业大学.

成熟度分类 Maturity classification	外观品质分级 Appearance quality grading	主要特点 Main features
成熟果 Ripe fruit	优等 Superior	商品果，果实周正，无瑕疵。着色率 ≥ 80% Commodity fruit，and the fruit is round and flawless. Coloring rate ≥ 80%
	二等 Second-class	非商品果，但可食用，有瑕疵，裂纹，轻微日灼。着色率 ≥70% Non commercial fruit，but edible，with defects，cracks，and slight sunburn. Coloring rate ≥ 70%
	劣等 Inferior	非商品果，有明显的腐烂，挤压伤，裂果，表面有虫。着色率 ≥ 70% Non commercial fruits，with obvious decay，crushing damage，cracking，and insect infestation on the surface. Coloring rate ≥ 70%
半熟果 Semi-ripe fruit	不分等级 Not graded	白粉色，或者着色率 < 70% White and pink，or coloring rate < 70%
未熟果 Unripe fruit	不分等级 Not graded	白果、绿果 White and green fruits

成熟度分类 Maturity classification	外观品质分级 Appearance quality grading	主要特点 Main features
成熟果 Ripe fruit	优等 Superior	商品果，果实周正，无瑕疵。着色率 ≥ 80% Commodity fruit，and the fruit is round and flawless. Coloring rate ≥ 80%
	二等 Second-class	非商品果，但可食用，有瑕疵，裂纹，轻微日灼。着色率 ≥70% Non commercial fruit，but edible，with defects，cracks，and slight sunburn. Coloring rate ≥ 70%
	劣等 Inferior	非商品果，有明显的腐烂，挤压伤，裂果，表面有虫。着色率 ≥ 70% Non commercial fruits，with obvious decay，crushing damage，cracking，and insect infestation on the surface. Coloring rate ≥ 70%
半熟果 Semi-ripe fruit	不分等级 Not graded	白粉色，或者着色率 < 70% White and pink，or coloring rate < 70%
未熟果 Unripe fruit	不分等级 Not graded	白果、绿果 White and green fruits

配置名称 Configuration name	详细信息 Detailed information
CPU	Intel（R）Core i7-12700H
GPU	NVIDIA GEFORCE RTX 3060
显存Video storage	16GB
操作系统Operating system	Windows11
深度学习框架 Deep learning framework	Pytorch1.9.0
开发语言Development language	Python3.9
开发IDE Develop IDE	Pycharm
虚拟环境 Virtual environment	Anaconda

配置名称 Configuration name	详细信息 Detailed information
CPU	Intel（R）Core i7-12700H
GPU	NVIDIA GEFORCE RTX 3060
显存Video storage	16GB
操作系统Operating system	Windows11
深度学习框架 Deep learning framework	Pytorch1.9.0
开发语言Development language	Python3.9
开发IDE Develop IDE	Pycharm
虚拟环境 Virtual environment	Anaconda

模型 Model	精度提升策略 Precision improvement strategy				参数量 Parameter quantity	模型大小/M Model memory	平均精度/% mAP	检测时间/ms Detection time
模型 Model	CA注意力机制 CA attention mechanism	Stem Block 结构 Stem Block structure	优化检测层尺度 Optimize detection layer scale	替换K-means++聚类 Replace K-means++ clustering	参数量 Parameter quantity	模型大小/M Model memory	平均精度/% mAP	检测时间/ms Detection time
YOLOv5s	—	—	—	—	7 103 997	14.02	81.19	181
YOLOv5s	√	—	—	—	7 115 587	14.13	82.12	185
YOLOv5s	√	√	—	—	7 098 934	14.17	83.23	193
YOLOv5s	√	√	√	—	5 287 521	10.65	83.34	188
SC-YOLOv5s	√	√	√	√	5 287 521	10.62	83.97	188