diff options
| author | jules@lens <julescarbon@gmail.com> | 2019-02-13 17:08:55 +0100 |
|---|---|---|
| committer | jules@lens <julescarbon@gmail.com> | 2019-02-13 17:08:55 +0100 |
| commit | 13e6612a7bf891724bc262f5598833169a0af967 (patch) | |
| tree | 259cb448b190a33b331a5a5bf152f2ebbc907ad0 /scraper | |
| parent | 937de8a9180bb3c75f3a739dd02f439356525029 (diff) | |
updating datasets
Diffstat (limited to 'scraper')
| -rw-r--r-- | scraper/s2-scrape.sh | 5 | ||||
| -rw-r--r-- | scraper/s2-tar-papers.sh | 2 |
2 files changed, 6 insertions, 1 deletions
diff --git a/scraper/s2-scrape.sh b/scraper/s2-scrape.sh index 03247441..e1df67c5 100644 --- a/scraper/s2-scrape.sh +++ b/scraper/s2-scrape.sh @@ -13,3 +13,8 @@ python s2-fetch-doi.py python s2-extract-pdf-txt.py python s2-citation-report.py python s2-final-report.py + +rm s2-final-all-papers.tar +tar cf s2-final-all-papers.tar datasets/s2/papers/ datasets/s2/raw_papers/ datasets/s2/pdf/*/*/*.txt +scp s2-final-all-papers.tar lens@neural.garden:www/files/ + diff --git a/scraper/s2-tar-papers.sh b/scraper/s2-tar-papers.sh index 3efdaa02..a3ef1e78 100644 --- a/scraper/s2-tar-papers.sh +++ b/scraper/s2-tar-papers.sh @@ -1,6 +1,6 @@ #!/bin/bash rm s2-final-all-papers.tar -tar cvf s2-final-all-papers.tar datasets/s2/papers/ datasets/s2/raw_papers/ datasets/s2/pdf/*/*/*.txt datasets/s2/doi/* +tar cf s2-final-all-papers.tar datasets/s2/papers/ datasets/s2/raw_papers/ datasets/s2/pdf/*/*/*.txt scp s2-final-all-papers.tar lens@neural.garden:www/files/ |