A simple baseline for one-shot multi-object tracking一1. 背景介绍作者来自华中科技大和微软亚亚研院之前LZ是看过一点多目标跟踪的算法的当时采用的是centernetdeepsort的的方法速度呢在LZ的小笔记本上970m在12fps左右但是这篇论文中说速度已经是30fps速度提升了一倍多并且在2DMOT15MOT16MOT17MOT20都是rank first是个非常厉害的工作并且速度和精度的提升使得工业界也能够使用对应的思想恩为AI算法的落地又贡献了一步。LZ也并不是想单纯的阅读对应的论文做个翻译对于数据处理代码理解包括一些实验也会包含进来还是想好好理解这个算法而不仅仅是跑个demo。算法的概要是这样的目前夺目标跟踪主要分为两个大类一个是Two-Step MOT Methods一个是One-Shot MOT Methods这两个在之前的算法中是各有优缺点。a. Two-Step MOT Methods两步的算法主要是把多目标跟踪的任务分成两个独立的任务一个是目标检测第二个就是Re-ID主要流程是先对图像进行目标检测把检测框进行crop和resize后送到identity embedding network来提取Re-ID的特征最后进行box linking操作怎么进行box linking呢首先是通过Re-ID features和IoU计算一个对应的算是矩阵 然后通过Kalman Filter和Hungarian algorithm来完成linking的任务优点因为检测和Re-ID feature提取任务是分开的所以我们可以找到各自任务比较适合的网络结构对于得到的框送到identity embedding network是需要进行resize操作的这样也可以解决对应的尺度变化的问题缺点效果也不错但就是慢b. One-Shot MOT Methods这个方法主要的目的是利用同一个网络同时完成目标检测和indentity embeddingRe-ID features来减少推断的时间如果做项目的小伙伴就会深有感触crop和resize操作在cpu比较弱的情况下还是会比较耗时的能省一点时间是一点吧优点只要做一次推断就可以得到对应的Re-ID features速度肯定会快一些缺点速度快精度就下降了。。。鱼和熊掌不可兼得这篇文章其实主要使用的是One-Shot MOT的方式不仅速度快精度也很高所以恩很牛2. 算法框架和代码初探算法的主要流程其实真的非常简洁就是先用一个特征提取器提取对应特征然后对应不同的任务分出两个分支一个分支是目标检测分支另外一个分支是获得对应维度的Re-ID特征非常的清晰。下面就是单纯是LZ的理解方式了所以顺序可能会和paper顺序不一样了建议还是要自己读一遍paper的2.1 环境配置LZ是不想再重新装笔记本了所以开发环境基本上都放在服务器上或者docker镜像里面了而且LZ是2016年11月份买的笔记本快四年了折腾不动了哭唧唧。。。按照论文给的参考地址在github下载对应代码参考对应的README.md文档先配置环境为啥晚上装呢人都下班了网速快之前也有一个教程是换源这个真的亲测可以加速感兴趣的小伙伴可以看一下conda create-nFairMOT conda activate FairMOT condainstallpytorch1.2.0torchvision0.4.0cudatoolkit10.0-cpytorch // LZ使用的这个版本的pytorch condainstallpytorch torchvisioncudatoolkit10.2-cpytorchcd${FAIRMOT_ROOT}pipinstall-rrequirements.txtcdsrc/lib/models/networks/DCNv2_newshmake.shDCNv2的编译细节请参考这里2000 years later2. 2 数据准备论文中说训练的时候用来6个公共数据集ETH和 CityPerson只提供了bounding box的标注所以我们只用这部分来训练检测的分支剩余的四个CalTechMOT17MCUHK-SYSU和PRW不仅提供了bounding box也提供了identity annotation所以我们可以用这四个数据集训练对应的detection和identity embedding这两个分支用了六个数据集LZ下数据要下哭了。。。训练数据肯定要知道数据集是啥也就借这个机会了解一下各个数据集。数据集的一些简单介绍在这里所有的上述数据集都要整理成对应的格式Caltech |——————images | └——————00001.jpg | |—————— ... | └——————0000N.jpg └——————labels_with_ids └——————00001.txt |—————— ... └——————0000N.txt每个图片都有对应的txt文件给定一个图片的路径标注文件地址可以通过将路径中images替换成labels_with_ids将.jpg替换成.txt的后缀在标注文件中每一行代表一个bounding box并且按照如下的格式[class] [identity] [x_center] [y_center] [width] [height]The field [class] should be 0. Only single-class multi-object tracking is supported in this version.The field [identity] is an integer from 0 to num_identities - 1, or -1 if this box has no identity annotation.Notethat the values of [x_center] [y_center] [width] [height] are normalized by the width/height of the image, so they are floating point numbers ranging from 0 to 1.下载的话这个地址都给出了对应的下载链接可以直接下载处理好的数据2.3 初步训练按照官网上的教程我们需要先修改数据集的路径在 src/lib/cfg/data.json和 ‘data_dir’ in src/lib/opts.py中进行修改data.json的源文件如下可以看到训练集就是我们之前介绍的6个数据集测试集是mot15{ root:/data/yfzhang/MOT/JDE, train: { mot17:./data/mot17.train, caltech:./data/caltech.train, citypersons:./data/citypersons.train, cuhksysu:./data/cuhksysu.train, prw:./data/prw.train, eth:./data/eth.train }, test_emb: { mot15:./data/mot15.val }, test: { mot15:./data/mot15.val } }opts.py的具体内容后续再分析目前关注的是data_dir# motself.parser.add_argument(--data_cfg,typestr,default../src/lib/cfg/data.json,helpload data from cfg)self.parser.add_argument(--data_dir,typestr,default)如果实在不知道改哪就先运行这个代码sh experiments/all_dla34.sh肯定会报路径的错误然后按照实际的情况进行修改就行了OSError: /data/yfzhang/MOT/JDE/PRW/labels_with_ids/c1s1_000151.txt not found.然后按照实际情况修改data.json即可在terminal中出现显示如下恭喜小伙伴你已经成功运行代码啦当然这只是第一步代码都没好好看这怎么好意思呢。。。2.4 TrackingTacking部分其实就是inference阶段了LZ就使用作者给的bashline尝试一下直接运行对应的track.py文件其实会报一个错误具体错误如下File track.py, line 102, in main dataloader datasets.LoadImages(osp.join(data_root, seq, img1), opt.img_size) File FairMOT\src\lib\datasets\dataset\jde.py, line 31, in __init__ self.nF len(self.files) # number of image files AttributeError: LoadImages object has no attribute files后来在github上这里找到对应问题是数据集的位置没有写正确写正确后运行如下命令cd src python track.py mot --load_model ../models/all_hrnet_v2_w18.pth --conf_thres 0.6 --arch hrnet_18 --reid_dim 128 --data_dir /your/path/to/data这个结果是MOT15 TRAIN的测试结果后期还会仔细分析这些结果的具体含义因为LZ没有安装ffmpeg所以把源码中对应ffmpeg代码全都改成OpenCV的接口来保存对应的视频了2.5 Demopython demo.py mot --load_model ../models/all_hrnet_v2_w18.pth --arch hrnet_18 --reid_dim 128 --conf_thres 0.4最后放上一张跟踪的结果图吧后面再开一篇专门对着论文和代码讲吧不然一篇博客太长了。。。参考地址代码地址https://github.com/ifzhang/FairMOT环境安装补充(FairMOT) root2d6362a3620d:FairMOT-master# pip install -r requirements.txt Collecting yacs Downloading yacs-0.1.8-py3-none-any.whl (14 kB) Collecting opencv-python Downloading opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB) |████████████████████████████████| 49.4 MB 2.1 MB/s Collecting cython-bbox Downloading cython_bbox-0.1.3.tar.gz (41 kB) |████████████████████████████████| 41 kB 372 kB/s ERROR: Command errored out with exit status 1: command: /root/anaconda3/envs/FairMOT/bin/python -c import sys, setuptools, tokenize; sys.argv[0] /tmp/pip-install-x2aapxs_/cython-bbox/setup.py; __file__/tmp/pip-install-x2aapxs_/cython-bbox/setup.py;fgetattr(tokenize, open, open)(__file__);codef.read().replace(\r\n, \n);f.close();exec(compile(code, __file__, exec)) egg_info --egg-base /tmp/pip-pip-egg-info-3sptlang cwd: /tmp/pip-install-x2aapxs_/cython-bbox/ Complete output (5 lines): Traceback (most recent call last): File string, line 1, in module File /tmp/pip-install-x2aapxs_/cython-bbox/setup.py, line 10, in module from Cython.Build import cythonize ModuleNotFoundError: No module named Cython ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. (FairMOT) root2d6362a3620d:FairMOT-master# conda install Cython Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /root/anaconda3/envs/FairMOT added / updated specs: - cython The following packages will be downloaded: package | build ---------------------------|----------------- cython-0.29.21 | py38he6710b0_0 2.0 MB defaults ------------------------------------------------------------ Total: 2.0 MB The following NEW packages will be INSTALLED: cython pkgs/main/linux-64::cython-0.29.21-py38he6710b0_0 Proceed ([y]/n)? y Downloading and Extracting Packages cython-0.29.21 | 2.0 MB | ############################################################################################################################################################ | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done (FairMOT) root2d6362a3620d:FairMOT-master# pip install -r requirements.txt Collecting yacs Using cached yacs-0.1.8-py3-none-any.whl (14 kB) Collecting opencv-python Using cached opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB) Collecting cython-bbox Using cached cython_bbox-0.1.3.tar.gz (41 kB) Collecting scipy Downloading scipy-1.5.2-cp38-cp38-manylinux1_x86_64.whl (25.7 MB) |████████████████████████████████| 25.7 MB 2.5 MB/s Collecting numba Downloading numba-0.51.1-cp38-cp38-manylinux2014_x86_64.whl (3.1 MB) |████████████████████████████████| 3.1 MB 2.8 MB/s Collecting progress Downloading progress-1.5.tar.gz (5.8 kB) Collecting motmetrics Downloading motmetrics-1.2.0-py3-none-any.whl (151 kB) |████████████████████████████████| 151 kB 2.7 MB/s Collecting matplotlib Downloading matplotlib-3.3.1-cp38-cp38-manylinux1_x86_64.whl (11.6 MB) |████████████████████████████████| 11.6 MB 2.4 MB/s Collecting lap Downloading lap-0.4.0.tar.gz (1.5 MB) |████████████████████████████████| 1.5 MB 2.4 MB/s Collecting openpyxl Downloading openpyxl-3.0.5-py2.py3-none-any.whl (242 kB) |████████████████████████████████| 242 kB 2.8 MB/s Requirement already satisfied: Pillow in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (7.2.0) Collecting tensorboardX Downloading tensorboardX-2.1-py2.py3-none-any.whl (308 kB) |████████████████████████████████| 308 kB 2.7 MB/s Collecting PyYAML Downloading PyYAML-5.3.1.tar.gz (269 kB) |████████████████████████████████| 269 kB 2.8 MB/s Requirement already satisfied: numpy1.17.3 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from opencv-python--r requirements.txt (line 2)) (1.19.1) Collecting llvmlite0.35,0.34.0.dev0 Downloading llvmlite-0.34.0-cp38-cp38-manylinux2010_x86_64.whl (24.6 MB) |████████████████████████████████| 24.6 MB 2.2 MB/s Requirement already satisfied: setuptools in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from numba--r requirements.txt (line 5)) (49.6.0.post20200814) Collecting pytest-benchmark Downloading pytest_benchmark-3.2.3-py2.py3-none-any.whl (49 kB) |████████████████████████████████| 49 kB 3.6 MB/s Collecting xmltodict0.12.0 Downloading xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB) Collecting pandas0.23.1 Downloading pandas-1.1.1-cp38-cp38-manylinux1_x86_64.whl (10.4 MB) |████████████████████████████████| 10.4 MB 2.5 MB/s Collecting flake8-import-order Downloading flake8_import_order-0.18.1-py2.py3-none-any.whl (15 kB) Collecting flake8 Downloading flake8-3.8.3-py2.py3-none-any.whl (72 kB) |████████████████████████████████| 72 kB 2.5 MB/s Collecting pytest Downloading pytest-6.0.1-py3-none-any.whl (270 kB) |████████████████████████████████| 270 kB 2.8 MB/s Collecting python-dateutil2.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) |████████████████████████████████| 227 kB 2.8 MB/s Collecting pyparsing!2.0.4,!2.1.2,!2.1.6,2.0.3 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) |████████████████████████████████| 67 kB 3.1 MB/s Collecting cycler0.10 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Collecting kiwisolver1.0.1 Downloading kiwisolver-1.2.0-cp38-cp38-manylinux1_x86_64.whl (92 kB) |████████████████████████████████| 92 kB 147 kB/s Requirement already satisfied: certifi2020.06.20 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from matplotlib--r requirements.txt (line 9)) (2020.6.20) Collecting et-xmlfile Downloading et_xmlfile-1.0.1.tar.gz (8.4 kB) Collecting jdcal Downloading jdcal-1.4.1-py2.py3-none-any.whl (9.5 kB) Requirement already satisfied: six in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from tensorboardX--r requirements.txt (line 13)) (1.15.0) Collecting protobuf3.8.0 Downloading protobuf-3.13.0-cp38-cp38-manylinux1_x86_64.whl (1.3 MB) |████████████████████████████████| 1.3 MB 2.8 MB/s Collecting py-cpuinfo Downloading py-cpuinfo-7.0.0.tar.gz (95 kB) |████████████████████████████████| 95 kB 2.3 MB/s Collecting pytz2017.2 Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB) |████████████████████████████████| 510 kB 2.8 MB/s Collecting pycodestyle Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB) |████████████████████████████████| 41 kB 1.3 MB/s Collecting mccabe0.7.0,0.6.0 Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB) Collecting pyflakes2.3.0,2.2.0 Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB) |████████████████████████████████| 66 kB 3.1 MB/s Collecting py1.8.2 Downloading py-1.9.0-py2.py3-none-any.whl (99 kB) |████████████████████████████████| 99 kB 2.9 MB/s Collecting more-itertools4.0.0 Downloading more_itertools-8.5.0-py3-none-any.whl (44 kB) |████████████████████████████████| 44 kB 4.5 MB/s Collecting packaging Downloading packaging-20.4-py2.py3-none-any.whl (37 kB) Collecting iniconfig Downloading iniconfig-1.0.1-py3-none-any.whl (4.2 kB) Collecting toml Downloading toml-0.10.1-py2.py3-none-any.whl (19 kB) Collecting attrs17.4.0 Downloading attrs-20.1.0-py2.py3-none-any.whl (49 kB) |████████████████████████████████| 49 kB 4.2 MB/s Collecting pluggy1.0,0.12 Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Building wheels for collected packages: cython-bbox, progress, lap, PyYAML, et-xmlfile, py-cpuinfo Building wheel for cython-bbox (setup.py) ... done Created wheel for cython-bbox: filenamecython_bbox-0.1.3-cp38-cp38-linux_x86_64.whl size58568 sha2563ebedb34e95b49afd367c1b62c1970e877f11136625c31e1e17c98b7be4bc15d Stored in directory: /root/.cache/pip/wheels/56/8d/2a/c7f38dc46bdfdee9aee8676cf3adf70d6e6e28b076cdb17963 Building wheel for progress (setup.py) ... done Created wheel for progress: filenameprogress-1.5-py3-none-any.whl size8074 sha256b4de011b051274a03887d309d45ca5e7d91c835e6d32cb7fda6e8c450e29a1fa Stored in directory: /root/.cache/pip/wheels/f1/5b/38/115b213dfbf5562108ea22df17c063f6378350205a819795aa Building wheel for lap (setup.py) ... done Created wheel for lap: filenamelap-0.4.0-cp38-cp38-linux_x86_64.whl size1617987 sha2562236f75be33da4b03820a1e420807f2394bf2e1ab4184302adbf3ab845049b12 Stored in directory: /root/.cache/pip/wheels/86/ec/8a/2baa6fb3ea3a3bdc40a9c2c4b8957b5b544f207531f7a78a89 Building wheel for PyYAML (setup.py) ... done Created wheel for PyYAML: filenamePyYAML-5.3.1-cp38-cp38-linux_x86_64.whl size44619 sha256369e890ee55fad482eec15f5ec8fe09d90aa23b399c829454ea96f2a5283f453 Stored in directory: /root/.cache/pip/wheels/13/90/db/290ab3a34f2ef0b5a0f89235dc2d40fea83e77de84ed2dc05c Building wheel for et-xmlfile (setup.py) ... done Created wheel for et-xmlfile: filenameet_xmlfile-1.0.1-py3-none-any.whl size8917 sha2560265a2f3639bc7e9d3226db4ca8c4b9a4f7ce4fd1d90fadc3fcff49593e541b1 Stored in directory: /root/.cache/pip/wheels/6e/df/38/abda47b884e3e25f9f9b6430e5ce44c47670758a50c0c51759 Building wheel for py-cpuinfo (setup.py) ... done Created wheel for py-cpuinfo: filenamepy_cpuinfo-7.0.0-py3-none-any.whl size20070 sha2562c40aeb8553a10a4257cbd1a02786903e4426b179c06dabec08350097795b8e6 Stored in directory: /root/.cache/pip/wheels/b4/99/9f/8eb77fdf759c1380719071722f2c37dd0fa1f6aa477c51cb6c Successfully built cython-bbox progress lap PyYAML et-xmlfile py-cpuinfo Installing collected packages: PyYAML, yacs, opencv-python, cython-bbox, scipy, llvmlite, numba, progress, py, more-itertools, pyparsing, packaging, iniconfig, toml, attrs, pluggy, pytest, py-cpuinfo, pytest-benchmark, xmltodict, pytz, python-dateutil, pandas, pycodestyle, flake8-import-order, mccabe, pyflakes, flake8, motmetrics, cycler, kiwisolver, matplotlib, lap, et-xmlfile, jdcal, openpyxl, protobuf, tensorboardX Successfully installed PyYAML-5.3.1 attrs-20.1.0 cycler-0.10.0 cython-bbox-0.1.3 et-xmlfile-1.0.1 flake8-3.8.3 flake8-import-order-0.18.1 iniconfig-1.0.1 jdcal-1.4.1 kiwisolver-1.2.0 lap-0.4.0 llvmlite-0.34.0 matplotlib-3.3.1 mccabe-0.6.1 more-itertools-8.5.0 motmetrics-1.2.0 numba-0.51.1 opencv-python-4.4.0.42 openpyxl-3.0.5 packaging-20.4 pandas-1.1.1 pluggy-0.13.1 progress-1.5 protobuf-3.13.0 py-1.9.0 py-cpuinfo-7.0.0 pycodestyle-2.6.0 pyflakes-2.2.0 pyparsing-2.4.7 pytest-6.0.1 pytest-benchmark-3.2.3 python-dateutil-2.8.1 pytz-2020.1 scipy-1.5.2 tensorboardX-2.1 toml-0.10.1 xmltodict-0.12.0 yacs-0.1.8 (FairMOT) root2d6362a3620d 
长河君 · 2026/10/22 1:13:48 · 阅读 1.2k 【Pytorch】Paper reading：A simple baseline for one-shot multi-object tracking（一）
A simple baseline for one-shot multi-object tracking一1. 背景介绍作者来自华中科技大和微软亚亚研院之前LZ是看过一点多目标跟踪的算法的当时采用的是centernetdeepsort的的方法速度呢在LZ的小笔记本上970m在12fps左右但是这篇论文中说速度已经是30fps速度提升了一倍多并且在2DMOT15MOT16MOT17MOT20都是rank first是个非常厉害的工作并且速度和精度的提升使得工业界也能够使用对应的思想恩为AI算法的落地又贡献了一步。LZ也并不是想单纯的阅读对应的论文做个翻译对于数据处理代码理解包括一些实验也会包含进来还是想好好理解这个算法而不仅仅是跑个demo。算法的概要是这样的目前夺目标跟踪主要分为两个大类一个是Two-Step MOT Methods一个是One-Shot MOT Methods这两个在之前的算法中是各有优缺点。a. Two-Step MOT Methods两步的算法主要是把多目标跟踪的任务分成两个独立的任务一个是目标检测第二个就是Re-ID主要流程是先对图像进行目标检测把检测框进行crop和resize后送到identity embedding network来提取Re-ID的特征最后进行box linking操作怎么进行box linking呢首先是通过Re-ID features和IoU计算一个对应的算是矩阵 然后通过Kalman Filter和Hungarian algorithm来完成linking的任务优点因为检测和Re-ID feature提取任务是分开的所以我们可以找到各自任务比较适合的网络结构对于得到的框送到identity embedding network是需要进行resize操作的这样也可以解决对应的尺度变化的问题缺点效果也不错但就是慢b. One-Shot MOT Methods这个方法主要的目的是利用同一个网络同时完成目标检测和indentity embeddingRe-ID features来减少推断的时间如果做项目的小伙伴就会深有感触crop和resize操作在cpu比较弱的情况下还是会比较耗时的能省一点时间是一点吧优点只要做一次推断就可以得到对应的Re-ID features速度肯定会快一些缺点速度快精度就下降了。。。鱼和熊掌不可兼得这篇文章其实主要使用的是One-Shot MOT的方式不仅速度快精度也很高所以恩很牛2. 算法框架和代码初探算法的主要流程其实真的非常简洁就是先用一个特征提取器提取对应特征然后对应不同的任务分出两个分支一个分支是目标检测分支另外一个分支是获得对应维度的Re-ID特征非常的清晰。下面就是单纯是LZ的理解方式了所以顺序可能会和paper顺序不一样了建议还是要自己读一遍paper的2.1 环境配置LZ是不想再重新装笔记本了所以开发环境基本上都放在服务器上或者docker镜像里面了而且LZ是2016年11月份买的笔记本快四年了折腾不动了哭唧唧。。。按照论文给的参考地址在github下载对应代码参考对应的README.md文档先配置环境为啥晚上装呢人都下班了网速快之前也有一个教程是换源这个真的亲测可以加速感兴趣的小伙伴可以看一下conda create-nFairMOT conda activate FairMOT condainstallpytorch1.2.0torchvision0.4.0cudatoolkit10.0-cpytorch // LZ使用的这个版本的pytorch condainstallpytorch torchvisioncudatoolkit10.2-cpytorchcd${FAIRMOT_ROOT}pipinstall-rrequirements.txtcdsrc/lib/models/networks/DCNv2_newshmake.shDCNv2的编译细节请参考这里2000 years later2. 2 数据准备论文中说训练的时候用来6个公共数据集ETH和 CityPerson只提供了bounding box的标注所以我们只用这部分来训练检测的分支剩余的四个CalTechMOT17MCUHK-SYSU和PRW不仅提供了bounding box也提供了identity annotation所以我们可以用这四个数据集训练对应的detection和identity embedding这两个分支用了六个数据集LZ下数据要下哭了。。。训练数据肯定要知道数据集是啥也就借这个机会了解一下各个数据集。数据集的一些简单介绍在这里所有的上述数据集都要整理成对应的格式Caltech |——————images | └——————00001.jpg | |—————— ... | └——————0000N.jpg └——————labels_with_ids └——————00001.txt |—————— ... └——————0000N.txt每个图片都有对应的txt文件给定一个图片的路径标注文件地址可以通过将路径中images替换成labels_with_ids将.jpg替换成.txt的后缀在标注文件中每一行代表一个bounding box并且按照如下的格式[class] [identity] [x_center] [y_center] [width] [height]The field [class] should be 0. Only single-class multi-object tracking is supported in this version.The field [identity] is an integer from 0 to num_identities - 1, or -1 if this box has no identity annotation.Notethat the values of [x_center] [y_center] [width] [height] are normalized by the width/height of the image, so they are floating point numbers ranging from 0 to 1.下载的话这个地址都给出了对应的下载链接可以直接下载处理好的数据2.3 初步训练按照官网上的教程我们需要先修改数据集的路径在 src/lib/cfg/data.json和 ‘data_dir’ in src/lib/opts.py中进行修改data.json的源文件如下可以看到训练集就是我们之前介绍的6个数据集测试集是mot15{ root:/data/yfzhang/MOT/JDE, train: { mot17:./data/mot17.train, caltech:./data/caltech.train, citypersons:./data/citypersons.train, cuhksysu:./data/cuhksysu.train, prw:./data/prw.train, eth:./data/eth.train }, test_emb: { mot15:./data/mot15.val }, test: { mot15:./data/mot15.val } }opts.py的具体内容后续再分析目前关注的是data_dir# motself.parser.add_argument(--data_cfg,typestr,default../src/lib/cfg/data.json,helpload data from cfg)self.parser.add_argument(--data_dir,typestr,default)如果实在不知道改哪就先运行这个代码sh experiments/all_dla34.sh肯定会报路径的错误然后按照实际的情况进行修改就行了OSError: /data/yfzhang/MOT/JDE/PRW/labels_with_ids/c1s1_000151.txt not found.然后按照实际情况修改data.json即可在terminal中出现显示如下恭喜小伙伴你已经成功运行代码啦当然这只是第一步代码都没好好看这怎么好意思呢。。。2.4 TrackingTacking部分其实就是inference阶段了LZ就使用作者给的bashline尝试一下直接运行对应的track.py文件其实会报一个错误具体错误如下File track.py, line 102, in main dataloader datasets.LoadImages(osp.join(data_root, seq, img1), opt.img_size) File FairMOT\src\lib\datasets\dataset\jde.py, line 31, in __init__ self.nF len(self.files) # number of image files AttributeError: LoadImages object has no attribute files后来在github上这里找到对应问题是数据集的位置没有写正确写正确后运行如下命令cd src python track.py mot --load_model ../models/all_hrnet_v2_w18.pth --conf_thres 0.6 --arch hrnet_18 --reid_dim 128 --data_dir /your/path/to/data这个结果是MOT15 TRAIN的测试结果后期还会仔细分析这些结果的具体含义因为LZ没有安装ffmpeg所以把源码中对应ffmpeg代码全都改成OpenCV的接口来保存对应的视频了2.5 Demopython demo.py mot --load_model ../models/all_hrnet_v2_w18.pth --arch hrnet_18 --reid_dim 128 --conf_thres 0.4最后放上一张跟踪的结果图吧后面再开一篇专门对着论文和代码讲吧不然一篇博客太长了。。。参考地址代码地址https://github.com/ifzhang/FairMOT环境安装补充(FairMOT) root2d6362a3620d:FairMOT-master# pip install -r requirements.txt Collecting yacs Downloading yacs-0.1.8-py3-none-any.whl (14 kB) Collecting opencv-python Downloading opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB) |████████████████████████████████| 49.4 MB 2.1 MB/s Collecting cython-bbox Downloading cython_bbox-0.1.3.tar.gz (41 kB) |████████████████████████████████| 41 kB 372 kB/s ERROR: Command errored out with exit status 1: command: /root/anaconda3/envs/FairMOT/bin/python -c import sys, setuptools, tokenize; sys.argv[0] /tmp/pip-install-x2aapxs_/cython-bbox/setup.py; __file__/tmp/pip-install-x2aapxs_/cython-bbox/setup.py;fgetattr(tokenize, open, open)(__file__);codef.read().replace(\r\n, \n);f.close();exec(compile(code, __file__, exec)) egg_info --egg-base /tmp/pip-pip-egg-info-3sptlang cwd: /tmp/pip-install-x2aapxs_/cython-bbox/ Complete output (5 lines): Traceback (most recent call last): File string, line 1, in module File /tmp/pip-install-x2aapxs_/cython-bbox/setup.py, line 10, in module from Cython.Build import cythonize ModuleNotFoundError: No module named Cython ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. (FairMOT) root2d6362a3620d:FairMOT-master# conda install Cython Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /root/anaconda3/envs/FairMOT added / updated specs: - cython The following packages will be downloaded: package | build ---------------------------|----------------- cython-0.29.21 | py38he6710b0_0 2.0 MB defaults ------------------------------------------------------------ Total: 2.0 MB The following NEW packages will be INSTALLED: cython pkgs/main/linux-64::cython-0.29.21-py38he6710b0_0 Proceed ([y]/n)? y Downloading and Extracting Packages cython-0.29.21 | 2.0 MB | ############################################################################################################################################################ | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done (FairMOT) root2d6362a3620d:FairMOT-master# pip install -r requirements.txt Collecting yacs Using cached yacs-0.1.8-py3-none-any.whl (14 kB) Collecting opencv-python Using cached opencv_python-4.4.0.42-cp38-cp38-manylinux2014_x86_64.whl (49.4 MB) Collecting cython-bbox Using cached cython_bbox-0.1.3.tar.gz (41 kB) Collecting scipy Downloading scipy-1.5.2-cp38-cp38-manylinux1_x86_64.whl (25.7 MB) |████████████████████████████████| 25.7 MB 2.5 MB/s Collecting numba Downloading numba-0.51.1-cp38-cp38-manylinux2014_x86_64.whl (3.1 MB) |████████████████████████████████| 3.1 MB 2.8 MB/s Collecting progress Downloading progress-1.5.tar.gz (5.8 kB) Collecting motmetrics Downloading motmetrics-1.2.0-py3-none-any.whl (151 kB) |████████████████████████████████| 151 kB 2.7 MB/s Collecting matplotlib Downloading matplotlib-3.3.1-cp38-cp38-manylinux1_x86_64.whl (11.6 MB) |████████████████████████████████| 11.6 MB 2.4 MB/s Collecting lap Downloading lap-0.4.0.tar.gz (1.5 MB) |████████████████████████████████| 1.5 MB 2.4 MB/s Collecting openpyxl Downloading openpyxl-3.0.5-py2.py3-none-any.whl (242 kB) |████████████████████████████████| 242 kB 2.8 MB/s Requirement already satisfied: Pillow in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (7.2.0) Collecting tensorboardX Downloading tensorboardX-2.1-py2.py3-none-any.whl (308 kB) |████████████████████████████████| 308 kB 2.7 MB/s Collecting PyYAML Downloading PyYAML-5.3.1.tar.gz (269 kB) |████████████████████████████████| 269 kB 2.8 MB/s Requirement already satisfied: numpy1.17.3 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from opencv-python--r requirements.txt (line 2)) (1.19.1) Collecting llvmlite0.35,0.34.0.dev0 Downloading llvmlite-0.34.0-cp38-cp38-manylinux2010_x86_64.whl (24.6 MB) |████████████████████████████████| 24.6 MB 2.2 MB/s Requirement already satisfied: setuptools in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from numba--r requirements.txt (line 5)) (49.6.0.post20200814) Collecting pytest-benchmark Downloading pytest_benchmark-3.2.3-py2.py3-none-any.whl (49 kB) |████████████████████████████████| 49 kB 3.6 MB/s Collecting xmltodict0.12.0 Downloading xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB) Collecting pandas0.23.1 Downloading pandas-1.1.1-cp38-cp38-manylinux1_x86_64.whl (10.4 MB) |████████████████████████████████| 10.4 MB 2.5 MB/s Collecting flake8-import-order Downloading flake8_import_order-0.18.1-py2.py3-none-any.whl (15 kB) Collecting flake8 Downloading flake8-3.8.3-py2.py3-none-any.whl (72 kB) |████████████████████████████████| 72 kB 2.5 MB/s Collecting pytest Downloading pytest-6.0.1-py3-none-any.whl (270 kB) |████████████████████████████████| 270 kB 2.8 MB/s Collecting python-dateutil2.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) |████████████████████████████████| 227 kB 2.8 MB/s Collecting pyparsing!2.0.4,!2.1.2,!2.1.6,2.0.3 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) |████████████████████████████████| 67 kB 3.1 MB/s Collecting cycler0.10 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Collecting kiwisolver1.0.1 Downloading kiwisolver-1.2.0-cp38-cp38-manylinux1_x86_64.whl (92 kB) |████████████████████████████████| 92 kB 147 kB/s Requirement already satisfied: certifi2020.06.20 in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from matplotlib--r requirements.txt (line 9)) (2020.6.20) Collecting et-xmlfile Downloading et_xmlfile-1.0.1.tar.gz (8.4 kB) Collecting jdcal Downloading jdcal-1.4.1-py2.py3-none-any.whl (9.5 kB) Requirement already satisfied: six in /root/anaconda3/envs/FairMOT/lib/python3.8/site-packages (from tensorboardX--r requirements.txt (line 13)) (1.15.0) Collecting protobuf3.8.0 Downloading protobuf-3.13.0-cp38-cp38-manylinux1_x86_64.whl (1.3 MB) |████████████████████████████████| 1.3 MB 2.8 MB/s Collecting py-cpuinfo Downloading py-cpuinfo-7.0.0.tar.gz (95 kB) |████████████████████████████████| 95 kB 2.3 MB/s Collecting pytz2017.2 Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB) |████████████████████████████████| 510 kB 2.8 MB/s Collecting pycodestyle Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB) |████████████████████████████████| 41 kB 1.3 MB/s Collecting mccabe0.7.0,0.6.0 Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB) Collecting pyflakes2.3.0,2.2.0 Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB) |████████████████████████████████| 66 kB 3.1 MB/s Collecting py1.8.2 Downloading py-1.9.0-py2.py3-none-any.whl (99 kB) |████████████████████████████████| 99 kB 2.9 MB/s Collecting more-itertools4.0.0 Downloading more_itertools-8.5.0-py3-none-any.whl (44 kB) |████████████████████████████████| 44 kB 4.5 MB/s Collecting packaging Downloading packaging-20.4-py2.py3-none-any.whl (37 kB) Collecting iniconfig Downloading iniconfig-1.0.1-py3-none-any.whl (4.2 kB) Collecting toml Downloading toml-0.10.1-py2.py3-none-any.whl (19 kB) Collecting attrs17.4.0 Downloading attrs-20.1.0-py2.py3-none-any.whl (49 kB) |████████████████████████████████| 49 kB 4.2 MB/s Collecting pluggy1.0,0.12 Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Building wheels for collected packages: cython-bbox, progress, lap, PyYAML, et-xmlfile, py-cpuinfo Building wheel for cython-bbox (setup.py) ... done Created wheel for cython-bbox: filenamecython_bbox-0.1.3-cp38-cp38-linux_x86_64.whl size58568 sha2563ebedb34e95b49afd367c1b62c1970e877f11136625c31e1e17c98b7be4bc15d Stored in directory: /root/.cache/pip/wheels/56/8d/2a/c7f38dc46bdfdee9aee8676cf3adf70d6e6e28b076cdb17963 Building wheel for progress (setup.py) ... done Created wheel for progress: filenameprogress-1.5-py3-none-any.whl size8074 sha256b4de011b051274a03887d309d45ca5e7d91c835e6d32cb7fda6e8c450e29a1fa Stored in directory: /root/.cache/pip/wheels/f1/5b/38/115b213dfbf5562108ea22df17c063f6378350205a819795aa Building wheel for lap (setup.py) ... done Created wheel for lap: filenamelap-0.4.0-cp38-cp38-linux_x86_64.whl size1617987 sha2562236f75be33da4b03820a1e420807f2394bf2e1ab4184302adbf3ab845049b12 Stored in directory: /root/.cache/pip/wheels/86/ec/8a/2baa6fb3ea3a3bdc40a9c2c4b8957b5b544f207531f7a78a89 Building wheel for PyYAML (setup.py) ... done Created wheel for PyYAML: filenamePyYAML-5.3.1-cp38-cp38-linux_x86_64.whl size44619 sha256369e890ee55fad482eec15f5ec8fe09d90aa23b399c829454ea96f2a5283f453 Stored in directory: /root/.cache/pip/wheels/13/90/db/290ab3a34f2ef0b5a0f89235dc2d40fea83e77de84ed2dc05c Building wheel for et-xmlfile (setup.py) ... done Created wheel for et-xmlfile: filenameet_xmlfile-1.0.1-py3-none-any.whl size8917 sha2560265a2f3639bc7e9d3226db4ca8c4b9a4f7ce4fd1d90fadc3fcff49593e541b1 Stored in directory: /root/.cache/pip/wheels/6e/df/38/abda47b884e3e25f9f9b6430e5ce44c47670758a50c0c51759 Building wheel for py-cpuinfo (setup.py) ... done Created wheel for py-cpuinfo: filenamepy_cpuinfo-7.0.0-py3-none-any.whl size20070 sha2562c40aeb8553a10a4257cbd1a02786903e4426b179c06dabec08350097795b8e6 Stored in directory: /root/.cache/pip/wheels/b4/99/9f/8eb77fdf759c1380719071722f2c37dd0fa1f6aa477c51cb6c Successfully built cython-bbox progress lap PyYAML et-xmlfile py-cpuinfo Installing collected packages: PyYAML, yacs, opencv-python, cython-bbox, scipy, llvmlite, numba, progress, py, more-itertools, pyparsing, packaging, iniconfig, toml, attrs, pluggy, pytest, py-cpuinfo, pytest-benchmark, xmltodict, pytz, python-dateutil, pandas, pycodestyle, flake8-import-order, mccabe, pyflakes, flake8, motmetrics, cycler, kiwisolver, matplotlib, lap, et-xmlfile, jdcal, openpyxl, protobuf, tensorboardX Successfully installed PyYAML-5.3.1 attrs-20.1.0 cycler-0.10.0 cython-bbox-0.1.3 et-xmlfile-1.0.1 flake8-3.8.3 flake8-import-order-0.18.1 iniconfig-1.0.1 jdcal-1.4.1 kiwisolver-1.2.0 lap-0.4.0 llvmlite-0.34.0 matplotlib-3.3.1 mccabe-0.6.1 more-itertools-8.5.0 motmetrics-1.2.0 numba-0.51.1 opencv-python-4.4.0.42 openpyxl-3.0.5 packaging-20.4 pandas-1.1.1 pluggy-0.13.1 progress-1.5 protobuf-3.13.0 py-1.9.0 py-cpuinfo-7.0.0 pycodestyle-2.6.0 pyflakes-2.2.0 pyparsing-2.4.7 pytest-6.0.1 pytest-benchmark-3.2.3 python-dateutil-2.8.1 pytz-2020.1 scipy-1.5.2 tensorboardX-2.1 toml-0.10.1 xmltodict-0.12.0 yacs-0.1.8 (FairMOT) root2d6362a3620d