# MegaPixels

Origins and ethics of publicy available facial recognition datasets

## Installation

- miniconda / python3.6
- mysql
- nvm, node
- redis
- nginx

```
conda create -n megapixels python=3.6
conda install pytorch torchvision -c pytorch
conda install faiss-cpu -c pytorch
conda install tensorflow
pip install numpy Pillow
pip install dlib
pip install requests simplejson click pdfminer.six
pip install urllib3 flask flask_sqlalchemy mysql-connector
pip install pymediainfo tqdm opencv-python imutils watchdog
pip install scikit-image python-dotenv imagehash scikit-learn colorlog
pip install celery keras tensorflow
pip install python.app # OSX only! needed for matplotlib

sudo apt-get install libmysqlclient-dev

mkdir -p /data_store_hdd/apps/megapixels/faiss/indexes
mkdir -p /data_store_hdd/apps/megapixels/faiss/metadata
```

### MySQL note

You may need to set the database charset to `utf8mb4` in order to import the CSVs:

```
ALTER DATABASE megapixels CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
```

## Development: automatic rebuilds

In development, we can watch a bunch of things and rebuild stuff automatically.  These rebuild the HTML and the Javascript:

```
python cli_site.py watch
npm run watch
```

## Running the various servers

Run the web server:

```
python cli_flask.py run
```

Two other backend servers must be running to use all features of the site (face search, etc.)

```
python `which celery` worker -A app.server.tasks --loglevel=info -E
redis-server /usr/local/etc/redis.conf
```

Note: On OSX, you must run the server with `pythonw` because of matplotlib.

## Building the site for production

```
npm install
npm run build
cd megapixels
python cli_faiss.py sync_metadata
python cli_faiss.py build_faiss
python cli_faiss.py build_db
python cli_site.py build
```


## Generating 3D Faces

- use <https://github.com/YadiraF/PRNet>