1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
------------
status: published
title: Duke MTMC Dataset
desc: <span class="dataset-name">Duke MTMC</span> is a dataset of surveillance camera footage of students on Duke University campus
subdesc: Duke MTMC contains over 2 million video frames and 2,700 unique identities collected from 8 HD cameras at Duke University campus in March 2014
slug: duke_mtmc
cssclass: dataset
caption: A still frame from the Duke MTMC (Multi-Target-Multi-Camera) CCTV dataset captured on Duke University campus in 2014. The dataset has now been terminated by the author in response to this report.
image: assets/background.jpg
published: 2019-4-18
updated: 2019-05-22
authors: Adam Harvey
------------
# Duke MTMC
Update: In response to this report and an [investigation](https://www.ft.com/content/cf19b956-60a2-11e9-b285-3acd5d43599e) by the Financial Times, Duke University has terminated the Duke MTMC dataset.
### sidebar
### end sidebar
Duke MTMC (Multi-Target, Multi-Camera) is a dataset of surveillance video footage taken on Duke University's campus in 2014 and is used for research and development of video tracking systems, person re-identification, and low-resolution facial recognition.
The dataset contains over 14 hours of synchronized surveillance video from 8 cameras at 1080p and 60 FPS, with over 2 million frames of 2,000 students walking to and from classes. The 8 surveillance cameras deployed on campus were specifically setup to capture students "during periods between lectures, when pedestrian traffic is heavy".[^duke_mtmc_orig]
For this analysis of the Duke MTMC dataset over 100 publicly available research papers that used the dataset were analyzed to find out who's using the dataset and where it's being used. The results show that the Duke MTMC dataset has spread far beyond its origins and intentions in academic research projects at Duke University. Since its publication in 2016, more than twice as many research citations originated in China as in the United States. Among these citations were papers links to the Chinese military and several of the companies known to provide Chinese authorities with the oppressive surveillance technology used to monitor millions of Uighur Muslims.
In one 2018 [paper](http://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_Attention-Aware_Compositional_Network_CVPR_2018_paper.pdf) jointly published by researchers from SenseNets and SenseTime (and funded by SenseTime Group Limited) entitled [Attention-Aware Compositional Network for Person Re-identification](https://www.semanticscholar.org/paper/Attention-Aware-Compositional-Network-for-Person-Xu-Zhao/14ce502bc19b225466126b256511f9c05cadcb6e), the Duke MTMC dataset was used for "extensive experiments" on improving person re-identification across multiple surveillance cameras with important applications in suspect tracking. Both SenseNets and SenseTime have been linked to the providing surveillance technology to monitor Uighur Muslims in China. [^xinjiang_nyt][^sensetime_qz][^sensenets_uyghurs]
Despite [repeated](https://www.hrw.org/news/2017/11/19/china-police-big-data-systems-violate-privacy-target-dissent) [warnings](https://www.hrw.org/news/2018/02/26/china-big-data-fuels-crackdown-minority-region) by Human Rights Watch that the authoritarian surveillance used in China represents a humanitarian crisis, researchers at Duke University continued to provide open access to their dataset for anyone to use for any project. As the surveillance crisis in China grew, so did the number of citations with links to organizations complicit in the crisis. In 2018 alone there were over 90 research projects happening in China that publicly acknowledged using the Duke MTMC dataset. Amongst these were projects from CloudWalk, Hikvision, Megvii (Face++), SenseNets, SenseTime, Beihang University, China's National University of Defense Technology, and the PLA's Army Engineering University.
| Organization | Paper | Link | Year | Used Duke MTMC |
|---|---|---|---|
| Army Engineering University of PLA | Ensemble Feature for Person Re-Identification | [arxiv.org](https://arxiv.org/abs/1901.05798) | 2019 | ✔ |
| Beihang University | Orientation-Guided Similarity Learning for Person Re-identification | [ieee.org](https://ieeexplore.ieee.org/document/8545620) | 2018 | ✔ |
| Beihang University | Online Inter-Camera Trajectory Association Exploiting Person Re-Identification and Camera Topology | [acm.org](https://dl.acm.org/citation.cfm?id=3240663) | 2018 | ✔ |
| CloudWalk | CloudWalk re-identification technology extends facial biometric tracking with improved accuracy | [BiometricUpdate.com](https://www.biometricupdate.com/201903/cloudwalk-re-identification-technology-extends-facial-biometric-tracking-with-improved-accuracy) | 2019 | ✔ |
|CloudWalk| Horizontal Pyramid Matching for Person Re-identification | [arxiv.org](https://arxiv.org/pdf/1804.05275.pdf) | 2018 | ✔ |
| Hikvision | Learning Incremental Triplet Margin for Person Re-identification | [arxiv.org](https://arxiv.org/abs/1812.06576) | 2018 | ✔ |
| Megvii (Face++) | Person Re-Identification (slides) | [github.io](https://zsc.github.io/megvii-pku-dl-course/slides/Lecture%2011,%20Human%20Understanding_%20ReID%20and%20Pose%20and%20Attributes%20and%20Activity%20.pdf) | 2017 | ✔ |
| Megvii (Face++) | Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project | [SemanticScholar](https://www.semanticscholar.org/paper/Multi-Target%2C-Multi-Camera-Tracking-by-Hierarchical-Zhang-Wu/10c20cf47d61063032dce4af73a4b8e350bf1128) | 2018 | ✔ |
| Megvii (Face++) | SCPNet: Spatial-Channel Parallelism Network for Joint Holistic and Partial PersonRe-Identification | [arxiv.org](https://arxiv.org/abs/1810.06996) | 2018 | ✔ |
| National University of Defense Technology | Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers | [SemanticScholar.org](https://www.semanticscholar.org/paper/Tracking-by-Animation%3A-Unsupervised-Learning-of-He-Liu/e90816e1a0e14ea1e7039e0b2782260999aef786) | 2018 | ✔ |
| National University of Defense Technology | Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based Recurrent Attention Networks | [SemanticScholar.org](https://www.semanticscholar.org/paper/Unsupervised-Multi-Object-Detection-for-Video-Using-He-He/59f357015054bab43fb8cbfd3f3dbf17b1d1f881) | 2018 | ✔ |
| SenseNets, SenseTime | Attention-Aware Compositional Network for Person Re-identification | [SemanticScholar](https://www.semanticscholar.org/paper/Attention-Aware-Compositional-Network-for-Person-Xu-Zhao/14ce502bc19b225466126b256511f9c05cadcb6e) | 2018 | ✔ |
|SenseTime| End-to-End Deep Kronecker-Product Matching for Person Re-identification | [thcvf.com](http://openaccess.thecvf.com/content_cvpr_2018/papers/Shen_End-to-End_Deep_Kronecker-Product_CVPR_2018_paper.pdf) | 2018| ✔ |
The reasons that companies in China use the Duke MTMC dataset for research are technically no different than the reasons it is used in the United States and Europe. In fact, the original creators of the dataset published a follow up report in 2017 titled "[Tracking Social Groups Within and Across Cameras](https://www.semanticscholar.org/paper/Tracking-Social-Groups-Within-and-Across-Cameras-Solera-Calderara/9e644b1e33dd9367be167eb9d832174004840400)" with specific applications to "automated analysis of crowds and social gatherings for surveillance and security applications". Their work, as well as the creation of the original dataset in 2014 were both supported in part by the United States Army Research Laboratory.
Citations from the United States and Europe show a similar trend to that in China, including publicly acknowledged and verified usage of the Duke MTMC dataset supported or carried out by the United States Department of Homeland Security, IARPA, IBM, Microsoft (who has provided surveillance to ICE), and Vision Semantics (who has worked with the UK Ministry of Defence). One [paper](https://pdfs.semanticscholar.org/59f3/57015054bab43fb8cbfd3f3dbf17b1d1f881.pdf) is even jointly published by researchers affiliated with both the University College of London and the National University of Defense Technology in China.
| Organization | Paper | Link | Year | Used Duke MTMC |
|---|---|---|---|
| IARPA, IBM | Horizontal Pyramid Matching for Person Re-identification | [arxiv.org](https://arxiv.org/abs/1804.05275) | 2018 | ✔ |
| Microsoft | ReXCam: Resource-Efficient, Cross-CameraVideo Analytics at Enterprise Scale | [arxiv.org](https://arxiv.org/abs/1811.01268) | 2018 | ✔ |
| Microsoft | Scaling Video Analytics Systems to Large Camera Deployments | [arxiv.org](https://arxiv.org/pdf/1809.02318.pdf) | 2018 | ✔ |
| University College of London | Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based RecurrentAttention Networks | [SemanticScholar.org](https://pdfs.semanticscholar.org/59f3/57015054bab43fb8cbfd3f3dbf17b1d1f881.pdf) | 2018 | ✔ |
| US Dept. of Homeland Security | Re-Identification with Consistent Attentive Siamese Networks | [arxiv.org](https://arxiv.org/abs/1811.07487/) | 2019 | ✔ |
| Vision Semantics Ltd. | Unsupervised Person Re-identification by Deep Learning Tracklet Association | [arxiv.org](https://arxiv.org/abs/1809.02874) | 2018 | ✔ |
By some metrics the dataset is considered a huge success. It is regarded as highly influential research and has contributed to hundreds, if not thousands, of projects to advance artificial intelligence for person tracking and monitoring. All the above citations, regardless of which country is using it, align perfectly with the original [intent](http://vision.cs.duke.edu/DukeMTMC/) of the Duke MTMC dataset: "to accelerate advances in multi-target multi-camera tracking".
The same logic applies for all the new extensions of the Duke MTMC dataset including [Duke MTMC Re-ID](https://github.com/layumi/DukeMTMC-reID_evaluation), [Duke MTMC Video Re-ID](https://github.com/Yu-Wu/DukeMTMC-VideoReID), Duke MTMC Groups, and [Duke MTMC Attribute](https://github.com/vana77/DukeMTMC-attribute). And it also applies to all the new specialized datasets that will be created from Duke MTMC, such as the low-resolution face recognition dataset called [QMUL-SurvFace](https://qmul-survface.github.io/), which was funded in part by [SeeQuestor](https://seequestor.com), a computer vision provider to law enforcement agencies including Scotland Yards and Queensland Police. From the perspective of academic researchers, security contractors, and defense agencies using these datasets to advance their organization's work, Duke MTMC provides significant value regardless of who else is using it, so long as it advances their own interests in artificial intelligence.
But this perspective comes at significant cost to civil rights, human rights, and privacy. The creation and distribution of the Duke MTMC dataset illustrates an egregious prioritization of surveillance technologies over individual rights, where the simple act of going to class or a place of worship (students were filmed going into the university's chapel) could implicate your face in a surveillance training dataset, perhaps even used by foreign defense agencies.
For the approximately 2,000 students in Duke MTMC dataset there may be no escape. It's not impossible to remove oneself from all copies of the dataset downloaded around the world. Instead, over 2,000 students and visitors who happened to be walking to class in 2014 will forever remain in all downloaded copies of the Duke MTMC dataset and all its extensions, contributing to a global supply chain of data that powers governmental and commercial expansion of biometric surveillance technologies.
### Updates
- June 2, 2019: Duke University seems to have shutdown the [Duke MTMC dataset project](http://vision.cs.duke.edu/DukeMTMC/)
- June 2, 2019: A computer vision surveillance workshop ([https://reid-mct.github.io/2019/](https://reid-mct.github.io/2019/)) using the Duke MTMC dataset has been cancelled. "Due to some unforeseen circumstances, the test data has not been available. The multi-target multi-camera tracking and person re-identification challenge is cancelled. We sincerely apologize for any inconvenience caused."
{% include 'dashboard.html' %}
{% include 'supplementary_header.html' %}
#### Video Timestamps
The video timestamps contain the likely, but not yet confirmed, date and times the video recorded. Because the video timestamps align with the start and stop [time sync data](http://vision.cs.duke.edu/DukeMTMC/details.html#time-sync) provided by the researchers, it at least confirms the relative timing. The [precipitous weather](https://www.wunderground.com/history/daily/KIGX/date/2014-3-19?req_city=Durham&req_state=NC&req_statename=North%20Carolina&reqdb.zip=27708&reqdb.magic=1&reqdb.wmo=99999) on March 14, 2014 in Durham, North Carolina supports, but does not confirm, that this day is the likely capture date.
=== columns 2
| Camera | Date | Start | End |
| --- | --- | --- | --- |
| Camera 1 | March 14, 2014 | 4:14PM | 5:43PM |
| Camera 2 | March 14, 2014 | 4:13PM | 4:43PM |
| Camera 3 | March 14, 2014 | 4:20PM | 5:48PM |
| Camera 4 | March 14, 2014 | 4:21PM | 5:54PM |
===========
| Camera | Date | Start | End |
| --- | --- | --- | --- |
| Camera 5 | March 14, 2014 | 4:12PM | 5:43PM |
| Camera 6 | March 14, 2014 | 4:18PM | 5:43PM |
| Camera 7 | March 14, 2014 | 4:16PM | 5:40PM |
| Camera 8 | March 14, 2014 | 4:25PM | 5:42PM |
=== end columns
#### Errata
The original Duke MTMC dataset paper mentions 2,700 identities, but their ground truth file only lists annotations for 1,812, and their own research typically mentions 2,000. For this writeup we used 2,000 to describe the approximate number of students.
#### Ethics
Please direct any questions about the ethics of the dataset to Duke University's [Institutional Ethics & Compliance Office](https://hr.duke.edu/policies/expectations/compliance/) using the number at the bottom of the page.
{% include 'cite_our_work.html' %}
#### Citing Duke MTMC
If you use any data from the Duke MTMC, please follow their [license](http://vision.cs.duke.edu/DukeMTMC/#how-to-cite) and cite their work as:
<pre>
@inproceedings{ristani2016MTMC,
title = {Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking},
author = {Ristani, Ergys and Solera, Francesco and Zou, Roger and Cucchiara, Rita and Tomasi, Carlo},
booktitle = {European Conference on Computer Vision workshop on Benchmarking Multi-Target Tracking},
year = {2016}
}
</pre>
### Footnotes
[^duke_mtmc_orig]: "Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking". 2016. [SemanticScholar](https://www.semanticscholar.org/paper/Performance-Measures-and-a-Data-Set-for-Tracking-Ristani-Solera/27a2fad58dd8727e280f97036e0d2bc55ef5424c)
[^sensetime_qz]: <https://qz.com/1248493/sensetime-the-billion-dollar-alibaba-backed-ai-company-thats-quietly-watching-everyone-in-china/>
[^sensenets_uyghurs]: <https://foreignpolicy.com/2019/03/19/962492-orwell-china-socialcredit-surveillance/>
[^xinjiang_nyt]: Mozur, Paul. "One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority". https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html. April 14, 2019.
|
