Col1

Col2

Col3

Content1

Content2

Content3

NIST 906932. Performance Assessment of Face Recognition Using Super-Resolution. Shuowen Hu, Robert Maschal, S. Susan Young, Tsai Hong Hong, Jonathon P. Phillips↩

TBD

TBD -

Published

2013

diff --git a/site/public/datasets/lfpw/index.html b/site/public/datasets/lfpw/index.html new file mode 100644 index 00000000..f734d332 --- /dev/null +++ b/site/public/datasets/lfpw/index.html @@ -0,0 +1,116 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

LFWP

+ +

+ Datasets + About +

+ +

Labeled Face Parts in The Wild

Who used LFWP?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how LFWP has been used around the world by commercial, military, and academic organizations; existing publicly available research citing Labeled Face Parts in the Wild was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ The dataset citations used in the visualizations were collected from Semantic Scholar, a website which aggregates and indexes research papers. Each citation was geocoded using names of institutions found in the PDF front matter, or as listed on other resources. These papers have been manually verified to show that researchers downloaded and used the dataset to train or test machine learning algorithms. +

+ +

RESEARCH below this line

Release 1 of LFPW consists of 1,432 faces from images downloaded from the web using simple text queries on sites such as google.com, flickr.com, and yahoo.com. Each image was labeled by three MTurk workers, and 29 fiducial points, shown below, are included in dataset. LFPW was originally described in the following publication:
+
Due to copyright issues, we cannot distribute image files in any format to anyone. Instead, we have made available a list of image URLs where you can download the images yourself. We realize that this makes it impossible to exactly compare numbers, as image links will slowly disappear over time, but we have no other option. This seems to be the way other large web-based databases seem to be evolving.
+

https://neerajkumar.org/databases/lfpw/

This research was performed at Kriegman-Belhumeur Vision Technologies and was funded by the CIA through the Office of the Chief Scientist. https://www.cs.cmu.edu/~peiyunh/topdown/ (nk_cvpr2011_faceparts.pdf)
+

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/lfw/index.html b/site/public/datasets/lfw/index.html index 60a6bf0e..2d91b065 100644 --- a/site/public/datasets/lfw/index.html +++ b/site/public/datasets/lfw/index.html @@ -27,7 +27,7 @@

Labeled Faces in The Wild (LFW) is the first facial recognition dataset created entirely from online photos

It includes 13,456 images of 4,432 people's images copied from the Internet during 2002-2004 and is the most frequently used dataset in the world for benchmarking face recognition algorithms. -

Published

2007

diff --git a/site/public/datasets/market_1501/index.html b/site/public/datasets/market_1501/index.html new file mode 100644 index 00000000..72807efc --- /dev/null +++ b/site/public/datasets/market_1501/index.html @@ -0,0 +1,132 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

Market 1501

+ +

+ Datasets + About +

+ +

Market-1501 is a dataset is collection of CCTV footage from Tsinghua University

The Market-1501 dataset includes 1,261 people from 5 HD surveillance cameras located on campus +

Market-1501 Dataset

[ PAGE UNDER DEVELOPMENT]

Who used Market 1501?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how Market 1501 has been used around the world by commercial, military, and academic organizations; existing publicly available research citing Market 1501 Dataset was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ +

(ignore) research Notes

"MARS is an extension of the Market-1501 dataset. During collection, we placed six near synchronized cameras in the campus of Tsinghua university. There were Five 1,0801920 HD cameras and one 640480 SD camera. MARS consists of 1,261 different pedestrians whom are captured by at least 2 cameras. Given a query tracklet, MARS aims to retrieve tracklets that contain the same ID." - main paper
bbox "0065C1T0002F0016.jpg", "0065" is the ID of the pedestrian. "C1" denotes the first +camera (there are totally 6 cameras). "T0002" means the 2th tracklet. "F016" is the 16th frame +within this tracklet. For the tracklets, their names are accumulated for each ID; but for frames, +they start from "F001" in each tracklet.

@proceedings{zheng2016mars, +title={MARS: A Video Benchmark for Large-Scale Person Re-identification}, +author={Zheng, Liang and Bie, Zhi and Sun, Yifan and Wang, Jingdong and Su, Chi and Wang, Shengjin and Tian, Qi}, +booktitle={European Conference on Computer Vision}, +year={2016}, +organization={Springer} +}

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/msceleb/index.html b/site/public/datasets/msceleb/index.html index cf3a654f..be21280c 100644 --- a/site/public/datasets/msceleb/index.html +++ b/site/public/datasets/msceleb/index.html @@ -27,7 +27,7 @@

MS Celeb is a dataset of web images used for training and evaluating face recognition algorithms

The MS Celeb dataset includes over 10,000,000 images and 93,000 identities of semi-public figures collected using the Bing search engine -

Microsoft Celeb Dataset (MS Celeb)

[ PAGE UNDER DEVELOPMENT ]

https://www.hrw.org/news/2019/01/15/letter-microsoft-face-surveillance-technology

https://www.scmp.com/tech/science-research/article/3005733/what-you-need-know-about-sensenets-facial-recognition-firm

Who used Microsoft Celeb?

diff --git a/site/public/datasets/oxford_town_centre/index.html b/site/public/datasets/oxford_town_centre/index.html index 63dc52d4..af020855 100644 --- a/site/public/datasets/oxford_town_centre/index.html +++ b/site/public/datasets/oxford_town_centre/index.html @@ -27,7 +27,7 @@

Oxford Town Centre is a dataset of surveillance camera footage from Cornmarket St Oxford, England

The Oxford Town Centre dataset includes approximately 2,200 identities and is used for research and development of face recognition systems -

Published

2009

@@ -115,16 +115,23 @@

The street location of the camera used for the Oxford Town Centre dataset was confirmed by matching the road, benches, and store signs source. At that location, two public CCTV cameras exist mounted on the side of the Northgate House building at 13-20 Cornmarket St. Because of the lower camera's mounting pole directionality, a view from a private camera in the building across the street can be ruled out because it would have to show more of silhouette of the lower camera's mounting pole. Two options remain: either the public CCTV camera mounted to the side of the building was used or the researchers mounted their own camera to the side of the building in the same location. Because the researchers used many other existing public CCTV cameras for their research projects it is likely that they would also be able to access to this camera.

To discredit the theory that this public CCTV is only seen pointing the other way in Google Street View images, at least one public photo shows the upper CCTV camera pointing in the same direction as the Oxford Town Centre dataset proving the camera can and has been rotated before.

As for the capture date, the text on the storefront display shows a sale happening from December 2nd – 7th indicating the capture date was between or just before those dates. The capture year is either 2008 or 2007 since prior to 2007 the Carphone Warehouse (photo, history) did not exist at this location. Since the sweaters in the GAP window display are more similar to those in a GAP website snapshot from November 2007, our guess is that the footage was obtained during late November or early December 2007. The lack of street vendors and slight waste residue near the bench suggests that is was probably a weekday after rubbish removal.

Footage from this public CCTV camera was used to create the Oxford Town Centre dataset. Image sources: Google Street View (<a href="https://www.google.com/maps/@51.7528162,-1.2581152,3a,50.3y,310.59h,87.23t/data=!3m7!1e1!3m5!1s3FsGN-PqYC-VhQGjWgmBdQ!2e0!5s20120601T000000!7i13312!8i6656">map</a>)

Footage from this public CCTV camera was used to create the Oxford Town Centre dataset. Image sources: Google Street View (map)

Heat map body visualization of the pedestrians detected in the Oxford Town Centre dataset © megapixels.cc

Heat map face visualization of the pedestrians detected in the Oxford Town Centre dataset © megapixels.cc

Demo Videos Using Oxford Town Centre Dataset

Several researchers have posted their demo videos using the Oxford Town Centre dataset on YouTube:

Multi target tracking on Oxford Dataset
[Multi-pedestrian tracking (TownCentre dataset)]https://www.youtube.com/watch?v=nO-3EM9dEd4
Multiple object tracking with kalman tracker and sort
Multi target tracking on Oxford dataset
towncentre
VTD - towncenter.avi

Footage from this public CCTV camera was used to create the Oxford Town Centre dataset. Image sources: Google Street View (map)

Heat map body visualization of the pedestrians detected in the Oxford Town Centre dataset © megapixels.cc

Heat map face visualization of the pedestrians detected in the Oxford Town Centre dataset © megapixels.cc

+ +

Cite Our Work

+ + If you use our data, research, or graphics please cite our work: + +

+@online{megapixels,
+  author = {Harvey, Adam. LaPlace, Jules.},
+  title = {MegaPixels: Origins, Ethics, and Privacy Implications of Publicly Available Face Recognition Image Datasets},
+  year = 2019,
+  url = {https://megapixels.cc/},
+  urldate = {2019-04-20}
+}

+ +

References

a
Benfold, Ben and Reid, Ian. "Stable Multi-Target Tracking in Real-Time Surveillance Video". CVPR 2011. Pages 3457-3464.
a
"Guiding Visual Surveillance by Tracking Human Attention". 2009.

diff --git a/site/public/datasets/pipa/index.html b/site/public/datasets/pipa/index.html new file mode 100644 index 00000000..780b3029 --- /dev/null +++ b/site/public/datasets/pipa/index.html @@ -0,0 +1,120 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

PIPA Dataset

+ +

+ Datasets + About +

+ +

People in Photo Albums (PIPA) is a dataset...

[ add subdescrition ] +

People in Photo Albums

[ PAGE UNDER DEVELOPMENT ]

Who used PIPA Dataset?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how PIPA Dataset has been used around the world by commercial, military, and academic organizations; existing publicly available research citing People in Photo Albums Dataset was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/pubfig/index.html b/site/public/datasets/pubfig/index.html new file mode 100644 index 00000000..2c8bd7b1 --- /dev/null +++ b/site/public/datasets/pubfig/index.html @@ -0,0 +1,117 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

PubFig

+ +

+ Datasets + About +

+ +

PubFig is a dataset...

[ add subdescrition ] +

PubFig

[ PAGE UNDER DEVELOPMENT ]

Who used PubFig?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how PubFig has been used around the world by commercial, military, and academic organizations; existing publicly available research citing Public Figures Face Dataset was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/uccs/index.html b/site/public/datasets/uccs/index.html new file mode 100644 index 00000000..4a0dfb5e --- /dev/null +++ b/site/public/datasets/uccs/index.html @@ -0,0 +1,274 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

UCCS

+ +

+ Datasets + About +

+ +

UnConstrained College Students is a dataset of long-range surveillance photos of students at University of Colorado in Colorado Springs

The UnConstrained College Students dataset includes 16,149 images and 1,732 identities of subjects on University of Colorado Colorado Springs campus and is used for making face recognition and face detection algorithms +

UnConstrained College Students

[ page under development ]

UnConstrained College Students (UCCS) is a dataset of long-range surveillance photos captured at University of Colorado Colorado Springs. According to the authors of two papers associated with the dataset, subjects were "photographed using a long-range high-resolution surveillance camera without their knowledge" 2. To create the dataset, the researchers used a Canon 7D digital camera fitted with a Sigma 800mm telephoto lens and photographed students 150–200m away through their office window. Photos were taken during the morning and afternoon while students were walking to and from classes. The primary uses of this dataset are to train, validate, and build recognition and face detection algorithms for realistic surveillance scenarios.

What makes the UCCS dataset unique is that it includes the highest resolution images of any publicly available face recognition dataset discovered so far (18MP), that it was captured on a campus without consent or awareness using a long-range telephoto lens, and that it was funded by United States defense and intelligence agencies.

Combined funding sources for the creation of the initial and final release of the dataset include ODNI (Office of Director of National Intelligence), IARPA (Intelligence Advance Research Projects Activity), ONR MURI (Office of Naval Research and The Department of Defense Multidisciplinary University Research Initiative), Army SBIR (Small Business Innovation Research), SOCOM SBIR (Special Operations Command and Small Business Innovation Research), and the National Science Foundation. 1 2

In 2017 the UCCS face dataset was used for a defense and intelligence agency funded face recognition challenge at the International Joint Biometrics Conference in Denver, CO. And in 2018 the dataset was used for the 2nd Unconstrained Face Detection and Open Set Recognition Challenge at the European Computer Vision Conference (ECCV) in Munich, Germany. Additional research projects that have used the UCCS dataset are included below in the list of verified citations.

UCCS is part of the IARAP Janus team https://vast.uccs.edu/project/iarpa-janus/

https://arxiv.org/abs/1708.02337

Who used UCCS?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how UCCS has been used around the world by commercial, military, and academic organizations; existing publicly available research citing UnConstrained College Students Dataset was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

UCCS photos taken in 2013

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Date	Photos
Jan 28, 2013	1,056
Jan 29, 2013	1,561
Feb 13, 2013	739
Feb 19, 2013	723
Feb 20, 2013	965
Feb 26, 2013	736

Location

The location of the camera and subjects can confirmed using several visual cues in the dataset images: the unique pattern of the sidewalk that is only used on the UCCS Pedestrian Spine near the West Lawn, the two UCCS sign poles with matching graphics still visible in Google Street View, the no parking sign and directionality of its arrow, the back of street sign next to it, the slight bend in the sidewalk, the presence of cars passing in the background of the image, and the far wall of the parking garage all match images in the dataset. The original papers also provides another clue: a picture of the camera inside the office that was used to create the dataset. The window view in this image provides another match for the brick pattern on the north facade of the Kraember Family Library and the green metal fence along the sidewalk. View the location on Google Maps

Location on campus where students were unknowingly photographed with a telephoto lens to be used for defense and intelligence agency funded research on face recognition. Image: Google Maps

3D view showing the angle of view of the surveillance camera used for UCCS dataset. Image: Google Maps

Funding

The UnConstrained College Students dataset is associated with two main research papers: "Large Scale Unconstrained Open Set Face Database" and "Unconstrained Face Detection and Open-Set Face Recognition Challenge". Collectively, these papers and the creation of the dataset have received funding from the following organizations:

ONR (Office of Naval Research) MURI (The Department of Defense Multidisciplinary University Research Initiative) grant N00014-08-1-0638
Army SBIR (Small Business Innovation Research) grant W15P7T-12-C-A210
SOCOM (Special Operations Command) SBIR (Small Business Innovation Research) grant H92222-07-P-0020
National Science Foundation Grant IIS-1320956
ODNI (Office of Director of National Intelligence)
IARPA (Intelligence Advance Research Projects Activity) R&D contract 2014-14071600012

Opting Out

If you attended University of Colorado Colorado Springs and were captured by the long range surveillance camera used to create this dataset, there is unfortunately currently no way to be removed. The authors do not provide any options for students to opt-out nor were students informed they would be used for training face recognition. According to the authors, the lack of any consent or knowledge of participation is what provides part of the value of Unconstrained College Students Dataset.

Ethics

Please direct any questions about the ethics of the dataset to the University of Colorado Colorado Springs Ethics and Compliance Office
For further technical information about the dataset, visit the UCCS dataset project page.

Downloads

Download EXIF data for UCCS photos: uccs_camera_exif.csv

+ +

Cite Our Work

+ + If you use our data, research, or graphics please cite our work: + +

+@online{megapixels,
+  author = {Harvey, Adam. LaPlace, Jules.},
+  title = {MegaPixels: Origins, Ethics, and Privacy Implications of Publicly Available Face Recognition Image Datasets},
+  year = 2019,
+  url = {https://megapixels.cc/},
+  urldate = {2019-04-20}
+}

+ +

References

a
Sapkota, Archana and Boult, Terrance. "Large Scale Unconstrained Open Set Face Database." 2013.
+
a b
Günther, M. et. al. "Unconstrained Face Detection and Open-Set Face Recognition Challenge," 2018. Arxiv 1708.02337v3.
+

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/vgg_face2/index.html b/site/public/datasets/vgg_face2/index.html new file mode 100644 index 00000000..75d73824 --- /dev/null +++ b/site/public/datasets/vgg_face2/index.html @@ -0,0 +1,142 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

Brainwash Dataset

+ +

+ Datasets + About +

+ +

VGG Face 2

[ page under development ]

Who used Brainwash Dataset?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how Brainwash Dataset has been used around the world by commercial, military, and academic organizations; existing publicly available research citing Brainwash Dataset was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ +

(ignore) research notes

The VGG Face 2 dataset includes approximately 1,331 actresses, 139 presidents, 16 wives, 3 husbands, 2 snooker player, and 1 guru
The original VGGF2 name list has been updated with the results returned from Google Knowledge
Names with a similarity score greater than 0.75 where automatically updated. Scores computed using import difflib; seq = difflib.SequenceMatcher(a=a.lower(), b=b.lower()); score = seq.ratio()
The 97 names with a score of 0.75 or lower were manually reviewed and includes name changes validating using Wikipedia.org results for names such as "Bruce Jenner" to "Caitlyn Jenner", spousal last-name changes, and discretionary changes to improve search results such as combining nicknames with full name when appropriate, for example changing "Aleksandar Petrović" to "Aleksandar 'Aco' Petrović" and minor changes such as "Mohammad Ali" to "Muhammad Ali"
The 'Description' text was automatically added when the Knowledge Graph score was greater than 250

TODO

create name list, and populate with Knowledge graph information like LFW
make list of interesting number stats, by the numbers
make list of interesting important facts
write intro abstract
write analysis of usage
find examples, citations, and screenshots of useage
find list of companies using it for table
create montages of the dataset, like LFW
create right to removal information

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/datasets/viper/index.html b/site/public/datasets/viper/index.html new file mode 100644 index 00000000..5b3ac35b --- /dev/null +++ b/site/public/datasets/viper/index.html @@ -0,0 +1,122 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

VIPeR

+ +

+ Datasets + About +

YouTube Celebrities

+ +

+ Datasets + About +

+ +

YouTube Celebrities

[ page under development ]

Who used YouTube Celebrities?

+ +

+ This bar chart presents a ranking of the top countries where dataset citations originated. Mouse over individual columns to see yearly totals. These charts show at most the top 10 countries. +

+ +

Biometric Trade Routes

+ +

+ To help understand how YouTube Celebrities has been used around the world by commercial, military, and academic organizations; existing publicly available research citing YouTube Celebrities was collected, verified, and geocoded to show the biometric trade routes of people appearing in the images. Click on the markers to reveal research projects at that location. +

+ +

Academic
Commercial
Military / Government

Citation data is collected using SemanticScholar.org then dataset usage verified and geolocated.

+ + +

+ +

Dataset Citations

+ +

Notes...

Selected dataset sequences: (a) MBGC, (b) CMU MoBo, (c) First +Honda/UCSD, and (d) YouTube Celebrities.
This research is supported by the Central Intelligence Agency, the Biometrics +Task Force and the Technical Support Working Group through US Army contract +W91CRB-08-C-0093. The opinions, (cid:12)ndings, and conclusions or recommendations +expressed in this publication are those of the authors and do not necessarily re(cid:13)ect +the views of our sponsors.
in "Face Recognition From Video Draft 17"
International Journal of Pattern Recognition and Artifcial Intelligence WorldScientific Publishing Company

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/index.html b/site/public/index.html new file mode 100644 index 00000000..118814be --- /dev/null +++ b/site/public/index.html @@ -0,0 +1,39 @@ + + + + MegaPixels + + + + + + + + + + +

+ +

MegaPixels

+ +

+ DATASETS + ABOUT +

+ + + + \ No newline at end of file diff --git a/site/public/info/index.html b/site/public/info/index.html new file mode 100644 index 00000000..749c29ba --- /dev/null +++ b/site/public/info/index.html @@ -0,0 +1,50 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

Face Analysis

Results are only stored for the duration of the analysis and are deleted when you leave this page.

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/research/00_introduction/index.html b/site/public/research/00_introduction/index.html new file mode 100644 index 00000000..353e3270 --- /dev/null +++ b/site/public/research/00_introduction/index.html @@ -0,0 +1,101 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

00: Introduction

Posted

2018-12-15

Megapixels

+ +

Posted

Dec. 15

Author

Adam Harvey

Facial recognition is a scam.

During the last 20 years commericial, academic, and governmental agencies have promoted the false dream of a future with face recognition. This essay debunks the popular myth that such a thing ever existed.

There is no such thing as face recognition. For the last 20 years, government agencies, commercial organizations, and academic institutions have played the public as a fool, selling a roadmap of the future that simply does not exist. Facial recognition, as it is currently defined, promoted, and sold to the public, government, and commercial sector is a scam.

Committed to developing robust solutions with superhuman accuracy, the industry has repeatedly undermined itself by never actually developing anything close to "face recognition".

There is only biased feature vector clustering and probabilistic thresholding.

Motivation

Ever since government agencies began developing face recognition in the early 1960's, datasets of face images have always been central to developing and validating face recognition technologies. Today, these datasets no longer originate in labs, but instead from family photo albums posted on photo sharing sites, surveillance camera footage from college campuses, search engine queries for celebrities, cafe livestreams, or videos on YouTube.

During the last year, hundreds of these facial analysis datasets created "in the wild" have been collected to understand how they contribute to a global supply chain of biometric data that is powering the global facial recognition industry.

While many of these datasets include public figures such as politicians, athletes, and actors; they also include many non-public figures: digital activists, students, pedestrians, and semi-private shared photo albums are all considered "in the wild" and fair game for research projects. Some images are used with creative commons licenses, yet others were taken in unconstrained scenarios without awareness or consent. At first glance it appears many of the datasets were created for seemingly harmless academic research, but when examined further it becomes clear that they're also used by foreign defense agencies.

The MegaPixels site is based on an earlier installation (also supported by Mozilla) at the Tactical Tech Glassroom in London in 2017; and a commission from the Elevate arts festival curated by Berit Gilma about pedestrian recognition datasets in 2018, and research during CV Dazzle from 2010-2015. Through the many prototypes, conversations, pitches, PDFs, and false starts this project has endured during the last 5 years, it eventually evolved into something much different than originally imagined. Now, as datasets become increasingly influential in shaping the computational future, it's clear that they must be critically analyzed to understand the biases, shortcomings, funding sources, and contributions to the surveillance industry. However, it's misguided to only criticize these datasets for their flaws without also praising their contribution to society. Without publicly available facial analysis datasets there would be less public discourse, less open-source software, and less peer-reviewed research. Public datasets can indeed become a vital public good for the information economy but as this projects aims to illustrate, many ethical questions arise about consent, intellectual property, surveillance, and privacy.

+ + + + + +

Ever since the first computational facial recognition research project by the CIA in the early 1960s, data has always played a vital role in the development of our biometric future. Without facial recognition datasets there would be no facial recognition. Datasets are an indispensable part of any artificial intelligence system because, as Geoffrey Hinton points out:

Our relationship to computers has changed. Instead of programming them, we now show them and they figure it out. - Geoffrey Hinton
+

Algorithms learn from datasets. And we program algorithms by building datasets. But datasets aren't like code. There's no programming language made of data except for the data itself.

Ignore content below these lines

It was the early 2000s. Face recognition was new and no one seemed sure exactly how well it was going to perform in practice. In theory, face recognition was poised to be a game changer, a force multiplier, a strategic military advantage, a way to make cities safer and to secure borders. This was the future John Ashcroft demanded with the Total Information Awareness act of the 2003 and that spooks had dreamed of for decades. It was a future that academics at Carnegie Mellon Universtiy and Colorado State University would help build. It was also a future that celebrities would play a significant role in building. And to the surprise of ordinary Internet users like myself and perhaps you, it was a future that millions of Internet users would unwittingly play role in creating.

Now the future has arrived and it doesn't make sense. Facial recognition works yet it doesn't actually work. Facial recognition is cheap and accessible but also expensive and out of control. Facial recognition research has achieved headline grabbing superhuman accuracies over 99.9% yet facial recognition is also dangerously inaccurate. During a trial installation at Sudkreuz station in Berlin in 2018, 20% of the matches were wrong, a number so low that it should not have any connection to law enforcement or justice. And in London, the Metropolitan police had been using facial recognition software that mistakenly identified an alarming 98% of people as criminals ¹, which perhaps is a crime itself.

MegaPixels is an online art project that explores the history of facial recognition from the perspective of datasets. To paraphrase the artist Trevor Paglen, whoever controls the dataset controls the meaning. MegaPixels aims to unravel the meanings behind the data and expose the darker corners of the biometric industry that have contributed to its growth. MegaPixels does not start with a conclusion, a moralistic slant, or a

Whether or not to build facial recognition was a question that can no longer be asked. As an outspoken critic of face recognition I've developed, and hopefully furthered, my understanding during the last 10 years I've spent working with computer vision. Though I initially disagreed, I've come to see technocratic perspective as a non-negotiable reality. As Oren (nytimes article) wrote in NYT Op-Ed "the horse is out of the barn" and the only thing we can do collectively or individually is to steer towards the least worse outcome. Computational communication has entered a new era and it's both exciting and frightening to explore the potentials and opportunities. In 1997 getting access to 1 teraFLOPS of computational power would have cost you $55 million and required a strategic partnership with the Department of Defense. At the time of writing, anyone can rent 1 teraFLOPS on a cloud GPU marketplace for less than $1/day. ².

I hope that this project will illuminate the darker areas of strange world of facial recognition that have not yet received attention and encourage discourse in academic, industry, and . By no means do I believe discourse can save the day. Nor do I think creating artwork can. In fact, I'm not exactly sure what the outcome of this project will be. The project is not so much what I publish here but what happens after. This entire project is only a prologue.

As McLuhan wrote, "You can't have a static, fixed position in the electric age". And in our hyper-connected age of mass surveillance, artificial intelligece, and unevenly distributed virtual futures the most irrational thing to be is rational. Increasingly the world is becoming a contradiction where people use surveillance to protest surveillance, use

Like many projects, MegaPixels had spent years meandering between formats, unfeasible budgets, and was generally too niche of a subject. The basic idea for this project, as proposed to the original Glass Room installation in 2016 in NYC, was to build an interactive mirror that showed people if they had been included in the LFW facial recognition dataset. The idea was based on my reaction to all the datasets I'd come across during research for the CV Dazzle project. I'd noticed strange datasets created for training and testing face detection algorithms. Most were created in labratory settings and their interpretation of face data was very strict.

for other post

It was the early 2000s. Face recognition was new and no one seemed sure how well it was going to perform in practice. In theory, face recognition was poised to be a game changer, a force multiplier, a strategic military advantage, a way to make cities safer and to secure the borders. It was the future that John Ashcroft demanded with the Total Information Awareness act of the 2003. It was a future that academics helped build. It was a future that celebrities helped build. And it was a future that

A decade earlier the Department of Homeland Security and the Counterdrug Technology Development Program Office initated a feasibilty study called FERET (FacE REcognition Technology) to "develop automatic face recognition capabilities that could be employed to assist security, intelligence, and law enforcement personnel in the performance of their duties [^feret_website]."

One problem with FERET dataset was that the photos were in controlled settings. For face recognition to work it would have to be used in uncontrolled settings. Even newer datasets such as the Multi-PIE (Pose, Illumination, and Expression) from Carnegie Mellon University included only indoor photos of cooperative subjects. Not only were the photos completely unrealistic, CMU's Multi-Pie included only 18 individuals and cost $500 for academic use [^cmu_multipie_cost], took years to create, and required consent from every participant.

Add progressive gan of FERET

Sharman, Jon. "Metropolitan Police's facial recognition technology 98% inaccurate, figures show". 2018. https://www.independent.co.uk/news/uk/home-news/met-police-facial-recognition-success-south-wales-trial-home-office-false-positive-a8345036.html ↩
Calle, Dan. "Supercomptuers". 1997. http://ei.cs.vt.edu/~history/SUPERCOM.Calle.HTML ↩

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/research/01_from_1_to_100_pixels/index.html b/site/public/research/01_from_1_to_100_pixels/index.html new file mode 100644 index 00000000..9426ef0f --- /dev/null +++ b/site/public/research/01_from_1_to_100_pixels/index.html @@ -0,0 +1,139 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

From 1 to 100 Pixels

Posted

2018-12-04

Adam Harvey

+ +

High resolution insights from low resolution data

This post will be about the meaning of "face". How do people define it? How to biometrics researchers define it? How has it changed during the last decade.

What can you know from a very small amount of information?

1 pixel grayscale
2x2 pixels grayscale, font example, can encode letters
3x3 pixels: can create a font
4x4 pixels: how many variations
8x8 yotta yotta, many more variations
5x7 face recognition
12x16 activity recognition
6/5 (up to 124/106) pixels in height/width, and the average is 24/20 for QMUL SurvFace
(prepare a Progan render of the QMUL dataset and TinyFaces)
20x16 tiny faces paper
20x20 MNIST handwritten images http://yann.lecun.com/exdb/mnist/
24x24 haarcascade detector idealized images
32x32 CIFAR image dataset
40x40 can do emotion detection, face recognition at scale, 3d modeling of the face. include datasets with faces at this resolution including pedestrian.
NIST standards begin to appear from 40x40, distinguish occular pixels
need more material from 60-100
60x60 show how texture emerges and pupils, eye color, higher resolution of features and compare to lower resolution faces
100x100 all you need for medical diagnosis
100x100 0.5% of one Instagram photo

Ideas:

Find specific cases of facial resolution being used in legal cases, forensic investigations, or military footage
resolution of boston bomber face
resolution of the state of the union image

Research

NIST report on sres states several resolutions
"Results show that the tested face recognition systems yielded similar performance for query sets with eye-to-eye distance from 60 pixels to 30 pixels" ¹

"Note that we only keep the images with a minimal side length of 80 pixels." and "a face will be labeled as “Ignore” if it is very difficult to be detected due to blurring, severe deformation and unrecognizable eyes, or the side length of its bounding box is less than 32 pixels." Ge_Detecting_Masked_Faces_CVPR_2017_paper.pdf
IBM DiF: "Faces with region size less than 50x50 or inter-ocular distance of less than 30 pixels were discarded. Faces with non-frontal pose, or anything beyond being slightly tilted to the left or the right, were also discarded."

As the resolution +formatted as rectangular databases of 16 bit RGB-tuples or 8 bit grayscale values

To consider how visual privacy applies to real world surveillance situations, the first

A single 8-bit grayscale pixel with 256 values is enough to represent the entire alphabet a-Z0-9 with room to spare.

A 2x2 pixels contains

Using no more than a 42 pixel (6x7 image) face image researchers [cite] were able to correctly distinguish between a group of 50 people. Yet

The likely outcome of face recognition research is that more data is needed to improve. Indeed, resolution is the determining factor for all biometric systems, both as training data to increase

Pixels, typically considered the buiding blocks of images and vidoes, can also be plotted as a graph of sensor values corresponding to the intensity of RGB-calibrated sensors.

Wi-Fi and cameras presents elevated risks for transmitting videos and image documentation from conflict zones, high-risk situations, or even sharing on social media. How can new developments in computer vision also be used in reverse, as a counter-forensic tool, to minimize an individual's privacy risk?

As the global Internet becomes increasingly effecient at turning the Internet into a giant dataset for machine learning, forensics, and data analysing, it would be prudent to also consider tools for decreasing the resolution. The Visual Defense module is just that. What are new ways to minimize the adverse effects of surveillance by dulling the blade. For example, a researcher paper showed that by decreasing a face size to 12x16 it was possible to do 98% accuracy with 50 people. This is clearly an example of

This research module, tentatively called Visual Defense Tools, aims to explore the

Prior Research

MPI visual privacy advisor
NIST: super resolution
YouTube blur tool
WITNESS: blur tool
Pixellated text
CV Dazzle
Bellingcat guide to geolocation
Peng! magic passport

Notes

In China, out of the approximately 200 million surveillance cameras only about 15% have enough resolution for face recognition.
In Apple's FaceID security guide, the probability of someone else's face unlocking your phone is 1 out of 1,000,000.
In England, the Metropolitan Police reported a false-positive match rate of 98% when attempting to use face recognition to locate wanted criminals.
In a face recognition trial at Berlin's Sudkreuz station, the false-match rate was 20%.

What all 3 examples illustrate is that face recognition is anything but absolute. In a 2017 talk, Jason Matheny the former directory of IARPA, admitted the face recognition is so brittle it can be subverted by using a magic marker and drawing "a few dots on your forehead". In fact face recognition is a misleading term. Face recognition is search engine for faces that can only ever show you the mos likely match. This presents real a real threat to privacy and lends

Globally, iPhone users unwittingly agree to 1/1,000,000 probably +relying on FaceID and TouchID to protect their information agree to a

NIST 906932. Performance Assessment of Face Recognition Using Super-Resolution. Shuowen Hu, Robert Maschal, S. Susan Young, Tsai Hong Hong, Jonathon P. Phillips↩

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/research/02_what_computers_can_see/index.html b/site/public/research/02_what_computers_can_see/index.html new file mode 100644 index 00000000..920f78cc --- /dev/null +++ b/site/public/research/02_what_computers_can_see/index.html @@ -0,0 +1,310 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

What Computers Can See

Posted

2018-12-15

Adam Harvey

+ +

A list of 100 things computer vision can see, eg:

age, race, gender, ancestral origin, body mass index
eye color, hair color, facial hair, glasses
beauty score,
intelligence
what you're looking at
medical conditions
tired, drowsiness in car
affectiva: interest in product, intent to buy

From PubFig Dataset

Male
Asian
White
Black
Baby
Child
Youth
Middle Aged
Senior
Black Hair
Blond Hair
Brown Hair
Bald
No Eyewear
Eyeglasses
Sunglasses
Mustache
Smiling Frowning
Chubby
Blurry
Harsh Lighting
Flash
Soft Lighting
Outdoor Curly Hair
Wavy Hair
Straight Hair
Receding Hairline
Bangs
Sideburns
Fully Visible Forehead
Partially Visible Forehead
Obstructed Forehead
Bushy Eyebrows
Arched Eyebrows
Narrow Eyes
Eyes Open
Big Nose
Pointy Nose
Big Lips
Mouth Closed
Mouth Slightly Open
Mouth Wide Open
Teeth Not Visible
No Beard
Goatee
Round Jaw
Double Chin
Wearing Hat
Oval Face
Square Face
Round Face
Color Photo
Posed Photo
Attractive Man
Attractive Woman
Indian
Gray Hair
Bags Under Eyes
Heavy Makeup
Rosy Cheeks
Shiny Skin
Pale Skin
5 o' Clock Shadow
Strong Nose-Mouth Lines
Wearing Lipstick
Flushed Face
High Cheekbones
Brown Eyes
Wearing Earrings
Wearing Necktie
Wearing Necklace

for i in {1..9};do wget http://visiond1.cs.umbc.edu/webpage/codedata/ADLdataset/ADL_videos/P_0$i.MP4;done;for i in {10..20}; do wget http://visiond1.cs.umbc.edu/webpage/codedata/ADLdataset/ADL_videos/P_$i.MP4;done

From Market 1501

The 27 attributes are:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

attribute	representation in file	label
gender	gender	male(1), female(2)
hair length	hair	short hair(1), long hair(2)
sleeve length	up	long sleeve(1), short sleeve(2)
length of lower-body clothing	down	long lower body clothing(1), short(2)
type of lower-body clothing	clothes	dress(1), pants(2)
wearing hat	hat	no(1), yes(2)
carrying backpack	backpack	no(1), yes(2)
carrying bag	bag	no(1), yes(2)
carrying handbag	handbag	no(1), yes(2)
age	age	young(1), teenager(2), adult(3), old(4)
8 color of upper-body clothing	upblack, upwhite, upred, uppurple, upyellow, upgray, upblue, upgreen	no(1), yes(2)
9 color of lower-body clothing	downblack, downwhite, downpink, downpurple, downyellow, downgray, downblue, downgreen,downbrown	no(1), yes(2)

source: https://github.com/vana77/Market-1501_Attribute/blob/master/README.md

From DukeMTMC

The 23 attributes are:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

attribute	representation in file	label
gender	gender	male(1), female(2)
length of upper-body clothing	top	short upper body clothing(1), long(2)
wearing boots	boots	no(1), yes(2)
wearing hat	hat	no(1), yes(2)
carrying backpack	backpack	no(1), yes(2)
carrying bag	bag	no(1), yes(2)
carrying handbag	handbag	no(1), yes(2)
color of shoes	shoes	dark(1), light(2)
8 color of upper-body clothing	upblack, upwhite, upred, uppurple, upgray, upblue, upgreen, upbrown	no(1), yes(2)
7 color of lower-body clothing	downblack, downwhite, downred, downgray, downblue, downgreen, downbrown	no(1), yes(2)

source: https://github.com/vana77/DukeMTMC-attribute/blob/master/README.md

From H3D Dataset

The joints and other keypoints (eyes, ears, nose, shoulders, elbows, wrists, hips, knees and ankles) +The 3D pose inferred from the keypoints. +Visibility boolean for each keypoint +Region annotations (upper clothes, lower clothes, dress, socks, shoes, hands, gloves, neck, face, hair, hat, sunglasses, bag, occluder) +Body type (male, female or child)

source: https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/shape/h3d/

From Leeds Sports Pose

=INDEX(A2:A9,MATCH(datasets!D1,B2:B9,0)) +=VLOOKUP(A2, datasets!A:J, 7, FALSE)

Right ankle +Right knee +Right hip +Left hip +Left knee +Left ankle +Right wrist +Right elbow +Right shoulder +Left shoulder +Left elbow +Left wrist +Neck +Head top

source: http://web.archive.org/web/20170915023005/sam.johnson.io/research/lsp.html

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/research/index.html b/site/public/research/index.html new file mode 100644 index 00000000..1be8203f --- /dev/null +++ b/site/public/research/index.html @@ -0,0 +1,49 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

← Back to test index

Modal image 1

Modal image 2

Modal image 3

Test table

+ + + + + + + + + + + + + + +

Col1	Col2	Col3
Content1	Content2	Content3

+ +

+ + + + + \ No newline at end of file diff --git a/site/public/test/index.html b/site/public/test/index.html new file mode 100644 index 00000000..e660bb2d --- /dev/null +++ b/site/public/test/index.html @@ -0,0 +1,61 @@ + + + + MegaPixels + + + + + + + + + + + +

+ +

MegaPixels

+ + +

+ Datasets + About +

+ +

← Back to test index

+ +

+ + + + + \ No newline at end of file -- cgit v1.2.3-70-g09d2 From 43c3e3904f80eb56769fba4634729d0e567f9a32 Mon Sep 17 00:00:00 2001 From: adamhrv Date: Wed, 17 Apr 2019 16:51:44 +0200 Subject: update duke --- site/content/pages/datasets/duke_mtmc/index.md | 71 ++++--- site/public/about/assets/LICENSE/index.html | 22 +- site/public/about/attribution/index.html | 22 +- site/public/about/index.html | 30 +-- site/public/about/legal/index.html | 22 +- site/public/about/press/index.html | 22 +- .../datasets/50_people_one_question/index.html | 22 +- site/public/datasets/afad/index.html | 22 +- site/public/datasets/brainwash/index.html | 23 +- site/public/datasets/caltech_10k/index.html | 22 +- site/public/datasets/celeba/index.html | 22 +- site/public/datasets/cofw/index.html | 22 +- site/public/datasets/duke_mtmc/index.html | 231 +++++++++++++++++---- site/public/datasets/feret/index.html | 22 +- site/public/datasets/hrt_transgender/index.html | 22 +- site/public/datasets/index.html | 24 +-- site/public/datasets/lfpw/index.html | 22 +- site/public/datasets/lfw/index.html | 22 +- site/public/datasets/market_1501/index.html | 22 +- site/public/datasets/msceleb/index.html | 22 +- site/public/datasets/oxford_town_centre/index.html | 22 +- site/public/datasets/pipa/index.html | 22 +- site/public/datasets/pubfig/index.html | 22 +- site/public/datasets/uccs/index.html | 33 ++- site/public/datasets/vgg_face2/index.html | 22 +- site/public/datasets/viper/index.html | 22 +- .../public/datasets/youtube_celebrities/index.html | 22 +- site/public/info/index.html | 22 +- site/public/research/00_introduction/index.html | 22 +- .../research/01_from_1_to_100_pixels/index.html | 22 +- .../research/02_what_computers_can_see/index.html | 22 +- site/public/research/index.html | 22 +- site/public/test/chart/index.html | 22 +- site/public/test/citations/index.html | 22 +- site/public/test/csv/index.html | 22 +- site/public/test/datasets/index.html | 22 +- site/public/test/face_search/index.html | 22 +- site/public/test/gallery/index.html | 22 +- site/public/test/index.html | 22 +- site/public/test/map/index.html | 22 +- site/public/test/name_search/index.html | 22 +- site/public/test/pie_chart/index.html | 22 +- 42 files changed, 686 insertions(+), 518 deletions(-) (limited to 'site/public/test/gallery') diff --git a/site/content/pages/datasets/duke_mtmc/index.md b/site/content/pages/datasets/duke_mtmc/index.md index ac0a3f2e..2a8bfe05 100644 --- a/site/content/pages/datasets/duke_mtmc/index.md +++ b/site/content/pages/datasets/duke_mtmc/index.md @@ -18,35 +18,63 @@ authors: Adam Harvey ### sidebar ### end sidebar -[ page under development ] +Duke MTMC (Multi-Target, Multi-Camera) is a dataset of surveillance video footage taken on Duke University's campus in 2014 and is used for research and development of video tracking systems, person re-identification, and low-resolution facial recognition. The dataset contains over 14 hours of synchronized surveillance video from 8 cameras at 1080p and 60FPS with over 2 million frames of 2,000 students walking to and from classes. The 8 surveillance cameras deployed on campus were specifically setup to capture students "during periods between lectures, when pedestrian traffic is heavy"[^duke_mtmc_orig]. -Duke MTMC (Multi-Target, Multi-Camera Tracking) is a dataset of video recorded on Duke University campus for research and development of networked camera surveillance systems. MTMC tracking algorithms are used for citywide dragnet surveillance systems such as those used throughout China by SenseTime[^sensetime_qz] and the oppressive monitoring of 2.5 million Uyghurs in Xinjiang by SenseNets[^sensenets_uyghurs]. In fact researchers from both SenseTime[^sensetime1] [^sensetime2] and SenseNets[^sensenets_sensetime] used the Duke MTMC dataset for their research. +In this investigation into the Duke MTMC dataset we tracked down over 100 publicly available research papers that explicitly acknowledged using Duke MTMC. Our analysis shows that the dataset has spread far beyond its origins and intentions in academic research projects at Duke University. Since its publication in 2016, more than twice as many research citations originated in China as in the United States. Among these citations were papers with explicit and direct links to the Chinese military and several of the companies known to provide Chinese authorities with the oppressive surveillance technology used to monitor millions of Uighur Muslims. -In this investigation into the Duke MTMC dataset, we found that researchers at Duke University in Durham, North Carolina captured over 2,000 students, faculty members, and passersby into one of the most prolific public surveillance research datasets that's used around the world by commercial and defense surveillance organizations. +In one 2018 [paper](http://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_Attention-Aware_Compositional_Network_CVPR_2018_paper.pdf) jointly published by researchers from SenseNets and SenseTime (and funded by SenseTime Group Limited) entitled [Attention-Aware Compositional Network for Person Re-identification](https://www.semanticscholar.org/paper/Attention-Aware-Compositional-Network-for-Person-Xu-Zhao/14ce502bc19b225466126b256511f9c05cadcb6e), the Duke MTMC dataset was used for "extensive experiments" on improving person re-identification across multiple surveillance cameras with important applications in "finding missing elderly and children, and suspect tracking, etc." Both SenseNets and SenseTime have been directly linked to the providing surveillance technology to monitor Uighur Muslims in China. [^sensetime_qz][^sensenets_uyghurs][^xinjiang_nyt] -Since it's publication in 2016, the Duke MTMC dataset has been used in over 100 studies at organizations around the world including SenseTime[^sensetime1] [^sensetime2], SenseNets[^sensenets_sensetime], IARPA and IBM[^iarpa_ibm], Chinese National University of Defense [^cn_defense1][^cn_defense2], US Department of Homeland Security[^us_dhs], Tencent, Microsoft, Microsft Asia, Fraunhofer, Senstar Corp., Alibaba, Naver Labs, Google and Hewlett-Packard Labs to name only a few. +![caption: A collection of 1,600 out of the approximately 2,000 students and pedestrians in the Duke MTMC dataset. These students were also included in the Duke MTMC Re-ID dataset extension used for person re-identification, and eventually the QMUL SurvFace face recognition dataset. Open Data Commons Attribution License.](assets/duke_mtmc_reid_montage.jpg) -The creation and publication of the Duke MTMC dataset in 2014 (published in 2016) was originally funded by the U.S. Army Research Laboratory and the National Science Foundation[^duke_mtmc_orig]. Though our analysis of the geographic locations of the publicly available research shows over twice as many citations by researchers from China (44% China, 20% United States). In 2018 alone, there were 70 research project citations from China. +Despite [repeated](https://www.hrw.org/news/2017/11/19/china-police-big-data-systems-violate-privacy-target-dissent) [warnings](https://www.hrw.org/news/2018/02/26/china-big-data-fuels-crackdown-minority-region) by Human Rights Watch that the authoritarian surveillance used in China represents a violation of human rights, researchers at Duke University continued to provide open access to their dataset for anyone to use for any project. As the surveillance crisis in China grew, so did the number of citations with links to organizations complicit in the crisis. In 2018 alone there were over 70 research projects happening in China that publicly acknowledged benefiting from the Duke MTMC dataset. Amongst these were projects from SenseNets, SenseTime, CloudWalk, Megvii, Beihang University, and the PLA's National University of Defense Technology. -![caption: A collection of 1,600 out of the 2,700 students and passersby captured into the Duke MTMC surveillance research and development dataset on . These students were also included in the Duke MTMC Re-ID dataset extension used for person re-identification. Open Data Commons Attribution License.](assets/duke_mtmc_reid_montage.jpg) +| Organization | Paper | Link | Year | Used Duke MTMC | +|---|---|---|---| +| SenseNets, SenseTime | Attention-Aware Compositional Network for Person Re-identification | [SemanticScholar](https://www.semanticscholar.org/paper/Attention-Aware-Compositional-Network-for-Person-Xu-Zhao/14ce502bc19b225466126b256511f9c05cadcb6e) | 2018 | ✔ | +|SenseTime| End-to-End Deep Kronecker-Product Matching for Person Re-identification | [thcvf.com](http://openaccess.thecvf.com/content_cvpr_2018/papers/Shen_End-to-End_Deep_Kronecker-Product_CVPR_2018_paper.pdf) | 2018| ✔ | +|CloudWalk| Horizontal Pyramid Matching for Person Re-identification | [arxiv.org](https://arxiv.org/pdf/1804.05275.pdf) | 20xx | ✔ | +| Megvii | Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project | [SemanticScholar](https://www.semanticscholar.org/paper/Multi-Target%2C-Multi-Camera-Tracking-by-Hierarchical-Zhang-Wu/10c20cf47d61063032dce4af73a4b8e350bf1128) | 2018 | ✔ | +| Megvii | Person Re-Identification (slides) | [github.io](https://zsc.github.io/megvii-pku-dl-course/slides/Lecture%2011,%20Human%20Understanding_%20ReID%20and%20Pose%20and%20Attributes%20and%20Activity%20.pdf) | 2017 | ✔ | +| Megvii | SCPNet: Spatial-Channel Parallelism Network for Joint Holistic and Partial PersonRe-Identification | [arxiv.org](https://arxiv.org/abs/1810.06996) | 2018 | ✔ | +| CloudWalk | CloudWalk re-identification technology extends facial biometric tracking with improved accuracy | [BiometricUpdate.com](https://www.biometricupdate.com/201903/cloudwalk-re-identification-technology-extends-facial-biometric-tracking-with-improved-accuracy) | 2018 | ✔ | +| CloudWalk | Horizontal Pyramid Matching for Person Re-identification | [arxiv.org](https://arxiv.org/abs/1804.05275)] | 2018 | ✔ | +| National University of Defense Technology | Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers | [SemanticScholar.org](https://www.semanticscholar.org/paper/Tracking-by-Animation%3A-Unsupervised-Learning-of-He-Liu/e90816e1a0e14ea1e7039e0b2782260999aef786) | 2018 | ✔ | +| National University of Defense Technology | Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based Recurrent Attention Networks | [SemanticScholar.org](https://www.semanticscholar.org/paper/Unsupervised-Multi-Object-Detection-for-Video-Using-He-He/59f357015054bab43fb8cbfd3f3dbf17b1d1f881) | 2018 | ✔ | +| Beihang University | Orientation-Guided Similarity Learning for Person Re-identification | [ieee.org](https://ieeexplore.ieee.org/document/8545620) | 2018 | ✔ | +| Beihang University | Online Inter-Camera Trajectory Association Exploiting Person Re-Identification and Camera Topology | [acm.org](https://dl.acm.org/citation.cfm?id=3240663) | 2018 | ✔ | -The 8 cameras deployed on Duke's campus were specifically setup to capture students "during periods between lectures, when pedestrian traffic is heavy".[^duke_mtmc_orig]. Camera 5 was positioned to capture students as entering and exiting the university's main chapel. Each camera's location and approximate field of view. The heat map visualization shows the locations where pedestrians were most frequently annotated in each video from the Duke MTMC dataset. +The reasons that companies in China use the Duke MTMC dataset for research are technically no different than the reasons it is used in the United States and Europe. In fact the original creators of the dataset published a follow up report in 2017 titled [Tracking Social Groups Within and Across Cameras](https://www.semanticscholar.org/paper/Tracking-Social-Groups-Within-and-Across-Cameras-Solera-Calderara/9e644b1e33dd9367be167eb9d832174004840400) with specific applications to "automated analysis of crowds and social gatherings for surveillance and security applications". Their work, as well as the creation of the original dataset in 2014 were both supported in part by the United States Army Research Laboratory. -![caption: Duke MTMC camera locations on Duke University campus. Open Data Commons Attribution License.](assets/duke_mtmc_camera_map.jpg) +Citations from the United States and Europe show a similar trend to that in China, including publicly acknowledged and verified usage of the Duke MTMC dataset supported or carried out by the United States Department of Homeland Security, IARPA, IBM, Microsoft (who provides surveillance to ICE), and Vision Semantics (who works with the UK Ministry of Defence). One [paper](https://pdfs.semanticscholar.org/59f3/57015054bab43fb8cbfd3f3dbf17b1d1f881.pdf) is even jointly published by researchers affiliated with both the University College of London and the National University of Defense Technology in China. -![caption: Duke MTMC camera views for 8 cameras deployed on campus © megapixels.cc](assets/duke_mtmc_cameras.jpg) +| Organization | Paper | Link | Year | Used Duke MTMC | +|---|---|---|---| +| IARPA, IBM, CloudWalk | Horizontal Pyramid Matching for Person Re-identification | [arxiv.org](https://arxiv.org/abs/1804.05275) | 2018 | ✔ | +| Microsoft | ReXCam: Resource-Efficient, Cross-CameraVideo Analytics at Enterprise Scale | [arxiv.org](https://arxiv.org/abs/1811.01268) | 2018 | ✔ | +| Microsoft | Scaling Video Analytics Systems to Large Camera Deployments | [arxiv.org](https://arxiv.org/pdf/1809.02318.pdf) | 2018 | ✔ | +| University College of London, National University of Defense Technology | Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based RecurrentAttention Networks | [PDF](https://pdfs.semanticscholar.org/59f3/57015054bab43fb8cbfd3f3dbf17b1d1f881.pdf) | 2018 | ✔ | +| Vision Semantics Ltd. | Unsupervised Person Re-identification by Deep Learning Tracklet Association | [arxiv.org](https://arxiv.org/abs/1809.02874) | 2018 | ✔ | +| US Dept. of Homeland Security | Re-Identification with Consistent Attentive Siamese Networks | [arxiv.org](https://arxiv.org/abs/1811.07487/) | 2019 | ✔ | + + +By some metrics the dataset is considered a huge success. It is regarded as highly influential research and has contributed to hundreds, if not thousands, of projects to advance artificial intelligence for person tracking and monitoring. All the above citations, regardless of which country is using it, align perfectly with the original [intent](http://vision.cs.duke.edu/DukeMTMC/) of the Duke MTMC dataset: "to accelerate advances in multi-target multi-camera tracking". + +The same logic applies for all the new extensions of the Duke MTMC dataset including [Duke MTMC Re-ID](https://github.com/layumi/DukeMTMC-reID_evaluation), [Duke MTMC Video Re-ID](https://github.com/Yu-Wu/DukeMTMC-VideoReID), Duke MTMC Groups, and [Duke MTMC Attribute](https://github.com/vana77/DukeMTMC-attribute). And it also applies to all the new specialized datasets that will be created from Duke MTMC, such as the low-resolution face recognition dataset called [QMUL-SurvFace](https://qmul-survface.github.io/), which was funded in part by [SeeQuestor](https://seequestor.com), a computer vision provider to law enforcement agencies including Scotland Yards and Queensland Police. From the perspective of academic researchers, companies, and defense agencies using these datasets to advance their organization's work, Duke MTMC contributes value their their bottom line. Regardless of who is using these datasets or how they're used, they are simple provided to make networks of surveillance cameras more powerful. ![caption: Duke MTMC pedestrian detection saliency maps for 8 cameras deployed on campus © megapixels.cc](assets/duke_mtmc_saliencies.jpg) +But from a privacy and human rights perspective the creation and distribution of the Duke MTMC illustrates an egregious prioritization of surveillance technologies over individual rights, where the simple act of going to class could implicate your biometric data in a surveillance training dataset. + +For the approximately 2,000 students in Duke MTMC dataset there is unfortunately no escape. It would be impossible to remove oneself from all copies of the dataset downloaded around the world. Instead, over 2,000 students and visitors who happened to be walking to class on March 13, 2014 will forever remain in all downloaded copies of the Duke MTMC dataset and all its extensions, contributing to a global supply chain of data that powers governmental and commercial expansion of biometric surveillance technologies. + {% include 'dashboard.html' %} {% include 'supplementary_header.html' %} -#### Funding +![caption: Duke MTMC camera locations on Duke University campus. Open Data Commons Attribution License.](assets/duke_mtmc_camera_map.jpg) -Original funding for the Duke MTMC dataset was provided by the Army Research Office under Grant No. W911NF-10-1-0387 and by the National Science Foundation -under Grants IIS-10-17017 and IIS-14-20894. +![caption: Duke MTMC camera views for 8 cameras deployed on campus © megapixels.cc](assets/duke_mtmc_cameras.jpg) #### Video Timestamps @@ -73,16 +101,10 @@ The video timestamps contain the likely, but not yet confirmed, date and times o === end columns -### Opting Out - -If you attended Duke University and were captured by any of the 8 surveillance cameras positioned on campus in 2014, there is unfortunately no way to be removed. The dataset files have been distributed throughout the world and it would not be possible to contact all the owners for removal. Nor do the authors provide any options for students to opt-out, nor did they even inform students they would be used at test subjects for surveillance research and development in a project funded, in part, by the United States Army Research Office. - #### Notes - The Duke MTMC dataset paper mentions 2,700 identities, but their ground truth file only lists annotations for 1,812 -{% include 'cite_our_work.html' %} - If you use any data from the Duke MTMC please follow their [license](http://vision.cs.duke.edu/DukeMTMC/#how-to-cite) and cite their work as:

@@ -94,19 +116,16 @@ If you use any data from the Duke MTMC please follow their [license](http://visi
 }

+{% include 'cite_our_work.html' %} + + #### ToDo - clean up citations, formatting ### Footnotes +[^xinjiang_nyt]: Mozur, Paul. "One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority". https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html. April 14, 2019. [^sensetime_qz]: [^sensenets_uyghurs]: -[^sensenets_sensetime]: "Attention-Aware Compositional Network for Person Re-identification". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/Attention-Aware-Compositional-Network-for-Person-Xu-Zhao/14ce502bc19b225466126b256511f9c05cadcb6e), [PDF](http://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_Attention-Aware_Compositional_Network_CVPR_2018_paper.pdf) -[^sensetime1]: "End-to-End Deep Kronecker-Product Matching for Person Re-identification". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/End-to-End-Deep-Kronecker-Product-Matching-for-Shen-Xiao/947954cafdefd471b75da8c3bb4c21b9e6d57838), [PDF](http://openaccess.thecvf.com/content_cvpr_2018/papers/Shen_End-to-End_Deep_Kronecker-Product_CVPR_2018_paper.pdf) -[^sensetime2]: "Person Re-identification with Deep Similarity-Guided Graph Neural Network". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/Person-Re-identification-with-Deep-Graph-Neural-Shen-Li/08d2a558ea2deb117dd8066e864612bf2899905b) -[^duke_mtmc_orig]: "Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking". 2016. [SemanticScholar](https://www.semanticscholar.org/paper/Performance-Measures-and-a-Data-Set-for-Tracking-Ristani-Solera/27a2fad58dd8727e280f97036e0d2bc55ef5424c) -[^cn_defense1]: "Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/Tracking-by-Animation%3A-Unsupervised-Learning-of-He-Liu/e90816e1a0e14ea1e7039e0b2782260999aef786) -[^cn_defense2]: "Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based Recurrent Attention Networks". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/Unsupervised-Multi-Object-Detection-for-Video-Using-He-He/59f357015054bab43fb8cbfd3f3dbf17b1d1f881) -[^iarpa_ibm]: "Horizontal Pyramid Matching for Person Re-identification". 2019. [SemanticScholar](https://www.semanticscholar.org/paper/Horizontal-Pyramid-Matching-for-Person-Fu-Wei/c2a5f27d97744bc1f96d7e1074395749e3c59bc8) -[^us_dhs]: "Re-Identification with Consistent Attentive Siamese Networks". 2018. [SemanticScholar](https://www.semanticscholar.org/paper/Re-Identification-with-Consistent-Attentive-Siamese-Zheng-Karanam/24d6d3adf2176516ef0de2e943ce2084e27c4f94) \ No newline at end of file +[^duke_mtmc_orig]: "Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking". 2016. [SemanticScholar](https://www.semanticscholar.org/paper/Performance-Measures-and-a-Data-Set-for-Tracking-Ristani-Solera/27a2fad58dd8727e280f97036e0d2bc55ef5424c) \ No newline at end of file diff --git a/site/public/about/assets/LICENSE/index.html b/site/public/about/assets/LICENSE/index.html index 0d3a7878..66d8b3ac 100644 --- a/site/public/about/assets/LICENSE/index.html +++ b/site/public/about/assets/LICENSE/index.html @@ -40,17 +40,17 @@

diff --git a/site/public/about/attribution/index.html b/site/public/about/attribution/index.html index 0a1b8e0f..5fe92b8d 100644 --- a/site/public/about/attribution/index.html +++ b/site/public/about/attribution/index.html @@ -60,17 +60,17 @@ To Adapt: To modify, transform and build upon the database

diff --git a/site/public/about/index.html b/site/public/about/index.html index 4a4ab3c6..b83736d3 100644 --- a/site/public/about/index.html +++ b/site/public/about/index.html @@ -35,7 +35,8 @@

Legal / Privacy

MegaPixels is an independent art and research project by Adam Harvey and Jules LaPlace that investigates the ethics, origins, and individual privacy implications of face recognition image datasets and their role in the expansion of biometric surveillance technologies.

The MegaPixels site is made possible with support from Mozilla

MegaPixels is made possible with support from Mozilla, our primary funding partner.

Additional support for MegaPixels is provided by the European ARTificial Intelligence Network (AI LAB) at the Ars Electronica Center, 1-year research-in-residence grant from Karlsruhe HfG, and sales from the Privacy Gift Shop.

Adam Harvey

@@ -75,6 +76,11 @@ You are free:

PDFMiner.Six and Pandas for research paper data analysis

Please direct questions, comments, or feedback to mastodon.social/@adamhrv

Funding Partners

The MegaPixels website, research, and development is made possible with support form Mozilla, our primary funding partner.

[ add logos ]

Additional support is provided by the European ARTificial Intelligence Network (AI LAB) at the Ars Electronica Center and a 1-year research-in-residence grant from Karlsruhe HfG.

[ add logos ]

Attribution

If you use MegaPixels or any data derived from it for your work, please cite our original work as follows:

@@ -89,17 +95,17 @@ You are free: