Supervision 结合 YOLO V8 玩爆各种计算机视觉处理任务 一、Supervision 介绍Supervision是一个针对于计算机视觉各种任务的框架工具为用户了提供便捷高效的视觉处理方法可以轻松处理数据集或直观地展现检测结果。另外他还提供了多种绘制检测结果的方式并且还提供了统计特定区域内检测跟踪、越线数量统计、切片推理、轨迹平滑等不同计算机视觉任务的方法封装。可谓是在CV领域有了它你可以少写很多展现和计算代码。Github 地址https://github.com/roboflow/supervision官方介绍及文档地址https://supervision.roboflow.com/latest/而YOLO V8支持全方位的视觉AI任务包括检测、分割、姿态估计、跟踪和分类。并且在速度和准确性方面具有无与伦比的性能。能够应用在各种对速度和精度要求较高的应用和领域。关于YOLO V8的使用可以参考下面这篇文章基于 YOLO V8 Fine-Tuning 训练自定义的目标检测模型下面将实验使用YOLO V8作为算法处理层提供CV检测任务使用Supervision进行上层的展现和不同任务的处理。安装 Supervision前提Supervision需要Python3.8环境中安装所以注意你环境中的Python版本是否符合。如果需要更加轻量级在服务器端应用程序运行则可以直接Supervisionpipinstallsupervision-ihttps://pypi.tuna.tsinghua.edu.cn/simple如果需要带有GUI支持的完整版本可以指定为supervision[desktop]此版本会包含OpenCV组件pipinstallsupervision[desktop]-ihttps://pypi.tuna.tsinghua.edu.cn/simple二、目标追踪检测任务目标追踪指在视频序列中连续地识别和定位一个或多个特定目标以及目标的行动轨迹。Supervision提供了ByteTrack类可以帮助我们快速的实现该功能。首先下载任务视频Supervision提供了不同的示例任务视频以方便我们快速实验效果fromsupervision.assetsimportdownload_assets,VideoAssets video_namedownload_assets(VideoAssets.PEOPLE_WALKING)print(video_name)执行后可以看到下载后的视频实现过程如下importcv2importsupervisionassvfromultralyticsimportYOLOfromsupervision.assetsimportdownload_assets,VideoAssets# 如果不存在则下载视频video_namedownload_assets(VideoAssets.PEOPLE_WALKING)# 加载 YoloV8n 模型如果不存在会自动下载modelYOLO(yolov8n.pt)# 追踪器trackersv.ByteTrack()# 初始化展现对象corner_annotatorsv.BoxCornerAnnotator(corner_length15,thickness2,colorsv.Color(r255,g255,b0))label_annotatorsv.LabelAnnotator()trace_annotatorsv.TraceAnnotator(trace_length100,colorsv.Color(r255,g0,b0))# 读取视频capcv2.VideoCapture(video_name)# 初始化计时器prev_tickcv2.getTickCount()whileTrue:# 循环读取每一帧ret,framecap.read()# 由于原视频帧比较大方便处理和后面展现缩小一些framecv2.resize(frame,(1280,720))ifnotret:breakresultmodel(frame,conf0.5,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)# 目标跟踪detectionstracker.update_with_detections(detections)labels[f{tracker_id}{result.names[class_id]}forclass_id,tracker_idinzip(detections.class_id,detections.tracker_id)]# 绘制边框framecorner_annotator.annotate(frame,detectionsdetections)# 绘制标签framelabel_annotator.annotate(frame,detectionsdetections,labelslabels)# 绘制轨迹frametrace_annotator.annotate(frame,detectionsdetections)# 计算帧率current_tickcv2.getTickCount()fpscv2.getTickFrequency()/(current_tick-prev_tick)prev_tickcurrent_tick cv2.putText(frame,FPS: {:.2f}.format(fps),(10,30),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),2)cv2.imshow(video,frame)cv2.waitKey(1)运行效果三、追踪检测轨迹平滑直接从追踪算法输出的结果往往包含噪声和不连续性可能会导致轨迹不平滑、不自然在Supervision中为我们提供了DetectionsSmoother类可以帮助我们快速的实现该功能。实现过程如下importcv2importsupervisionassvfromultralyticsimportYOLOfromsupervision.assetsimportdownload_assets,VideoAssets# 如果不存在则下载视频video_namedownload_assets(VideoAssets.PEOPLE_WALKING)# 加载 YoloV8n 模型如果不存在会自动下载modelYOLO(yolov8n.pt)# 追踪器trackersv.ByteTrack()# 轨迹平滑器smoothersv.DetectionsSmoother(length4)# 初始化展现对象corner_annotatorsv.BoxCornerAnnotator(corner_length15,thickness2,colorsv.Color(r255,g255,b0))label_annotatorsv.LabelAnnotator()trace_annotatorsv.TraceAnnotator(trace_length100,colorsv.Color(r255,g0,b0))# 读取视频capcv2.VideoCapture(video_name)# 初始化计时器prev_tickcv2.getTickCount()whileTrue:# 循环读取每一帧ret,framecap.read()# 由于原视频帧比较大方便处理和后面展现缩小一些framecv2.resize(frame,(1280,720))ifnotret:breakresultmodel(frame,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)# 目标跟踪detectionstracker.update_with_detections(detections)# 轨迹平滑detectionssmoother.update_with_detections(detections)labels[f{tracker_id}{result.names[class_id]}forclass_id,tracker_idinzip(detections.class_id,detections.tracker_id)]# 绘制边框framecorner_annotator.annotate(frame,detectionsdetections)# 绘制标签framelabel_annotator.annotate(frame,detectionsdetections,labelslabels)# 绘制轨迹frametrace_annotator.annotate(frame,detectionsdetections)# 计算帧率current_tickcv2.getTickCount()fpscv2.getTickFrequency()/(current_tick-prev_tick)prev_tickcurrent_tick cv2.putText(frame,FPS: {:.2f}.format(fps),(10,30),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),2)cv2.imshow(video,frame)cv2.waitKey(1)运行效果四、越线数据统计任务通过在视频中预设界限对越线的目标进行1统计常用在交通监控、安防监控等领域Supervision提供了LineZone类可以帮助我们快速的实现该功能。实现过程如下importcv2importsupervisionassvfromultralyticsimportYOLOfromsupervision.assetsimportdownload_assets,VideoAssets# 如果不存在则下载视频video_namedownload_assets(VideoAssets.VEHICLES)# 加载 YoloV8n 模型如果不存在会自动下载modelYOLO(yolov8n.pt)# 预设界限startsv.Point(0,400)endsv.Point(1280,400)# 初始预线检测器line_zonesv.LineZone(startstart,endend)# 追踪器trackersv.ByteTrack()# 初始化展现对象trace_annotatorsv.TraceAnnotator()label_annotatorsv.LabelAnnotator(text_scale1)line_zone_annotatorsv.LineZoneAnnotator(thickness1,text_thickness1,text_scale1)# 读取视频capcv2.VideoCapture(video_name)# 初始化计时器prev_tickcv2.getTickCount()whileTrue:# 循环读取每一帧ret,framecap.read()ifnotret:break# 由于原视频帧比较大方便处理和后面展现缩小一些framecv2.resize(frame,(1280,720))resultmodel(frame,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)# 目标跟踪detectionstracker.update_with_detections(detections)# 更新预线检测器crossed_in,crossed_outline_zone.trigger(detections)print(fin:{line_zone.in_count},fout:{line_zone.out_count})# 获得各边界框的标签labels[f{result.names[class_id]}forclass_id,tracker_idinzip(detections.class_id,detections.tracker_id)]# 绘制轨迹frametrace_annotator.annotate(frame,detectionsdetections)# 绘制标签framelabel_annotator.annotate(frame,detectionsdetections,labelslabels)# 绘制预制线frameline_zone_annotator.annotate(frame,line_counterline_zone)# 计算帧率current_tickcv2.getTickCount()fpscv2.getTickFrequency()/(current_tick-prev_tick)prev_tickcurrent_tick cv2.putText(frame,FPS: {:.2f}.format(fps),(10,30),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),2)cv2.imshow(video,frame)cv2.waitKey(1)运行效果五、切片检测任务当图像过于庞大或者目标非常小的情况是不利于目标的捕捉的一种方式就是对图像进行却片分割成若干区域分别进行检测然后再对结果合并和NMS过滤这种方式Supervision也为我们封装好了提供了InferenceSlicer类可以快速实现该功能。实现过程importcv2importsupervisionassvfromultralyticsimportYOLOimportnumpyasnp# 加载 yolov8n 模型modelYOLO(yolov8n.pt)# 切片之后的处理defcallback(image_slice:np.ndarray)-sv.Detections:resultmodel(image_slice,verboseFalse)[0]returnsv.Detections.from_ultralytics(result)# 初始化切片器slicersv.InferenceSlicer(callbackcallback,# 切片后每张子图进行处理的回调函数slice_wh(320,320),# 切片后子图的大小overlap_ratio_wh(0.3,0.3),# 连续切片之间的重叠率iou_threshold0.5,# 子图合并nms的iou阈值thread_workers4# 处理线程数)# 展现对象box_annotatorsv.BoxAnnotator()label_annotatorsv.LabelAnnotator()# 读取图像imagecv2.imread(img/3.png)resultmodel(image,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)image1box_annotator.annotate(image.copy(),detectionsdetections)# 绘制标签labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]image1label_annotator.annotate(image1,detectionsdetections,labelslabels)# 展现未切片的结果cv2.imshow(img,image1)cv2.waitKey(0)# 切片检测detectionsslicer(image)image2box_annotator.annotate(image.copy(),detectionsdetections)# 绘制标签labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]image2label_annotator.annotate(image2,detectionsdetections,labelslabels)# 展现切片后的结果cv2.imshow(img,image2)cv2.waitKey(0)未切片前的结果切片后的结果可以看到右上角的汽车被检测出来了但是左边也出现了个误检可以通过置信度做一层过滤。六、辅助展现任务从上面具体任务可可以感觉出来Supervision已经将效果展现的操作封装好了这里再介绍几种展现的方式。6.1 目标检测边框展现importcv2importsupervisionassvfromultralyticsimportYOLOimportnumpyasnp# 加载 yolov8n 模型modelYOLO(yolov8n.pt)# 方框展现对象box_annotatorsv.BoxAnnotator()label_annotatorsv.LabelAnnotator()# 读取图像imagecv2.imread(img/img.png)resultmodel(image,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)imagebox_annotator.annotate(image,detectionsdetections)# 绘制标签labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]imagelabel_annotator.annotate(image,detectionsdetections,labelslabels)cv2.imshow(img,image)cv2.waitKey(0)角点边框展现importcv2importsupervisionassvfromultralyticsimportYOLOimportnumpyasnp# 加载 yolov8n 模型modelYOLO(yolov8n.pt)# 角点边框展现对象corner_annotatorsv.BoxCornerAnnotator(corner_length15,thickness2,colorsv.Color(r255,g0,b0))label_annotatorsv.LabelAnnotator()# 读取图像imagecv2.imread(img/img.png)resultmodel(image,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)imagecorner_annotator.annotate(image,detectionsdetections)# 绘制标签labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]imagelabel_annotator.annotate(image,detectionsdetections,labelslabels)cv2.imshow(img,image)cv2.waitKey(0)三角形投标展现importcv2importsupervisionassvfromultralyticsimportYOLOimportnumpyasnp# 加载 yolov8n 模型modelYOLO(yolov8n.pt)# 三角展现对象triangle_annotatorsv.TriangleAnnotator(base30,height30,positionsv.Position[TOP_CENTER],colorsv.Color(r0,g255,b0))label_annotatorsv.LabelAnnotator()# 读取图像imagecv2.imread(img/img.png)resultmodel(image,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]imagelabel_annotator.annotate(image,detectionsdetections,labelslabels)imagetriangle_annotator.annotate(image,detectionsdetections)cv2.imshow(img,image)cv2.waitKey(0)6.2 语义分割importcv2importsupervisionassvfromultralyticsimportYOLO# 加载 yolov8n-seg 模型modelYOLO(yolov8n-seg.pt)# 语义分割展现对象mask_annotatorsv.MaskAnnotator()label_annotatorsv.LabelAnnotator()# 读取图像imagecv2.imread(img/img.png)resultmodel(image,device[0]# 如果是 cpu 则是 devicecpu)[0]detectionssv.Detections.from_ultralytics(result)labels[f{class_name}{confidence:.2f}forclass_name,confidenceinzip(detections[class_name],detections.confidence)]imagelabel_annotator.annotate(image,detectionsdetections,labelslabels)imagemask_annotator.annotate(image,detectionsdetections)cv2.imshow(img,image)cv2.waitKey(0)