blob: c95d56f89760127fc2670792d987aac049bb89c8 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
|
# MegaPixels
Origins and ethics of publicy available facial recognition datasets
## Installation
- miniconda / python3.6
- mysql
- nvm, node
- redis
- nginx
```
conda create -n megapixels python=3.6
conda install pytorch torchvision -c pytorch
conda install faiss-cpu -c pytorch
conda install tensorflow
pip install numpy Pillow
pip install dlib
pip install requests simplejson click pdfminer.six
pip install urllib3 flask flask_sqlalchemy mysql-connector
pip install pymediainfo tqdm opencv-python imutils watchdog
pip install scikit-image python-dotenv imagehash scikit-learn colorlog
pip install celery keras tensorflow
pip install python.app # OSX only! needed for matplotlib
sudo apt-get install libmysqlclient-dev
mkdir -p /data_store_hdd/apps/megapixels/faiss/indexes
mkdir -p /data_store_hdd/apps/megapixels/faiss/metadata
```
### MySQL note
You may need to set the database charset to `utf8mb4` in order to import the CSVs:
```
ALTER DATABASE megapixels CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
```
## Development: automatic rebuilds
In development, we can watch a bunch of things and rebuild stuff automatically. These rebuild the HTML and the Javascript:
```
python cli_site.py watch
npm run watch
```
## Running the various servers
Run the web server, which will serve some HTML (you will need to add index.html to URLs... alas!):
```
python cli_flask.py run
```
These servers must be running to use all features of the site (face search, etc.)
```
python `which celery` worker -A app.server.tasks --loglevel=info -E
redis-server /usr/local/etc/redis.conf
```
Note: On OSX, you must run the server with `pythonw` because of matplotlib.
## Building the site for production
```
npm install
npm run build
cd megapixels
python cli_faiss.py sync_metadata
python cli_faiss.py build_faiss
python cli_faiss.py build_db
python cli_site.py build
```
|
