- -

What Computers Can See

Posted

2018-12-15

Adam Harvey

- -

Rosalind Picard on Affective Computing Podcast with Lex Fridman

we can read with an ordinary camera on your phone, from a neutral face if
your heart is racing
if your breating is becoming irregular and showing signs of stress
how your heart rate variability power is changing even when your heart is not necessarily accelerating
we can tell things about your stress even if you have a blank face

in emotion studies

when participants use smartphone and multiple data types are collected to understand patterns of life can predict tomorrow's mood
get best results
better than 80% accurate at predicting tomorrow's mood levels

A list of 100 things computer vision can see, eg:

age, race, gender, ancestral origin, body mass index
eye color, hair color, facial hair, glasses
beauty score,
intelligence
what you're looking at
medical conditions
tired, drowsiness in car
affectiva: interest in product, intent to buy

From SenseTime paper

Exploring Disentangled Feature Representation Beyond Face Identification

From https://arxiv.org/pdf/1804.03487.pdf -The attribute IDs from 1 to 40 corre-spond to: ‘5 o Clock Shadow’, ‘Arched Eyebrows’, ‘Attractive’, ‘Bags Under Eyes’, ‘Bald’, ‘Bangs’, ‘Big Lips’, ‘BigNose’, ‘Black Hair’, ‘Blond Hair’, ‘Blurry’, ‘Brown Hair’,‘Bushy Eyebrows’, ‘Chubby’, ‘Double Chin’, ‘Eyeglasses’,‘Goatee’, ‘Gray Hair’, ‘Heavy Makeup’, ‘High Cheek-bones’, ‘Male’, ‘Mouth Slightly Open’, ‘Mustache’, ‘Nar-row Eyes’, ‘No Beard’, ‘Oval Face’, ‘Pale Skin’, ‘PointyNose’, ‘Receding Hairline’, ‘Rosy Cheeks’, ‘Sideburns’,‘Smiling’, ‘Straight Hair’, ‘Wavy Hair’, ‘Wearing Ear-rings’, ‘Wearing Hat’, ‘Wearing Lipstick’, ‘Wearing Neck-lace’, ‘Wearing Necktie’ and ‘Young’. It’

From PubFig Dataset

Male
Asian
White
Black
Baby
Child
Youth
Middle Aged
Senior
Black Hair
Blond Hair
Brown Hair
Bald
No Eyewear
Eyeglasses
Sunglasses
Mustache
Smiling Frowning
Chubby
Blurry
Harsh Lighting
Flash
Soft Lighting
Outdoor Curly Hair
Wavy Hair
Straight Hair
Receding Hairline
Bangs
Sideburns
Fully Visible Forehead
Partially Visible Forehead
Obstructed Forehead
Bushy Eyebrows
Arched Eyebrows
Narrow Eyes
Eyes Open
Big Nose
Pointy Nose
Big Lips
Mouth Closed
Mouth Slightly Open
Mouth Wide Open
Teeth Not Visible
No Beard
Goatee
Round Jaw
Double Chin
Wearing Hat
Oval Face
Square Face
Round Face
Color Photo
Posed Photo
Attractive Man
Attractive Woman
Indian
Gray Hair
Bags Under Eyes
Heavy Makeup
Rosy Cheeks
Shiny Skin
Pale Skin
5 o' Clock Shadow
Strong Nose-Mouth Lines
Wearing Lipstick
Flushed Face
High Cheekbones
Brown Eyes
Wearing Earrings
Wearing Necktie
Wearing Necklace

for i in {1..9};do wget http://visiond1.cs.umbc.edu/webpage/codedata/ADLdataset/ADL_videos/P_0$i.MP4;done;for i in {10..20}; do wget http://visiond1.cs.umbc.edu/webpage/codedata/ADLdataset/ADL_videos/P_$i.MP4;done

From Market 1501

The 27 attributes are:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

attribute	representation in file	label
gender	gender	male(1), female(2)
hair length	hair	short hair(1), long hair(2)
sleeve length	up	long sleeve(1), short sleeve(2)
length of lower-body clothing	down	long lower body clothing(1), short(2)
type of lower-body clothing	clothes	dress(1), pants(2)
wearing hat	hat	no(1), yes(2)
carrying backpack	backpack	no(1), yes(2)
carrying bag	bag	no(1), yes(2)
carrying handbag	handbag	no(1), yes(2)
age	age	young(1), teenager(2), adult(3), old(4)
8 color of upper-body clothing	upblack, upwhite, upred, uppurple, upyellow, upgray, upblue, upgreen	no(1), yes(2)
9 color of lower-body clothing	downblack, downwhite, downpink, downpurple, downyellow, downgray, downblue, downgreen,downbrown	no(1), yes(2)

source: https://github.com/vana77/Market-1501_Attribute/blob/master/README.md

From DukeMTMC

The 23 attributes are:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

attribute	representation in file	label
gender	gender	male(1), female(2)
length of upper-body clothing	top	short upper body clothing(1), long(2)
wearing boots	boots	no(1), yes(2)
wearing hat	hat	no(1), yes(2)
carrying backpack	backpack	no(1), yes(2)
carrying bag	bag	no(1), yes(2)
carrying handbag	handbag	no(1), yes(2)
color of shoes	shoes	dark(1), light(2)
8 color of upper-body clothing	upblack, upwhite, upred, uppurple, upgray, upblue, upgreen, upbrown	no(1), yes(2)
7 color of lower-body clothing	downblack, downwhite, downred, downgray, downblue, downgreen, downbrown	no(1), yes(2)

source: https://github.com/vana77/DukeMTMC-attribute/blob/master/README.md

From H3D Dataset

The joints and other keypoints (eyes, ears, nose, shoulders, elbows, wrists, hips, knees and ankles) -The 3D pose inferred from the keypoints. -Visibility boolean for each keypoint -Region annotations (upper clothes, lower clothes, dress, socks, shoes, hands, gloves, neck, face, hair, hat, sunglasses, bag, occluder) -Body type (male, female or child)

source: https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/shape/h3d/

From Leeds Sports Pose

=INDEX(A2:A9,MATCH(datasets!D1,B2:B9,0)) -=VLOOKUP(A2, datasets!A:J, 7, FALSE)

Right ankle -Right knee -Right hip -Left hip -Left knee -Left ankle -Right wrist -Right elbow -Right shoulder -Left shoulder -Left elbow -Left wrist -Neck -Head top

source: http://web.archive.org/web/20170915023005/sam.johnson.io/research/lsp.html

- -