推薦処理用のデータの準備
パッケージのインストール
(recsys_full) offline$
pip install cryptography
(recsys_full) offline$ pip freeze
...（略）...
cffi==2.1.1
cryptography==50.0.1
pycparser==3.0
...（略）...
データの準備
offline$
psql ml_latest_small -U postgres -c "\copy (SELECT '00000000-0000-0000-0000-' || lpad(cast(user_id AS TEXT), 12, '0') AS user_id, user_name || '@rsl.org' AS email FROM users ORDER BY user_id ASC) to 'data/users_.csv' with delimiter E'\t' csv header encoding 'UTF8'"
psql ml_latest_small -U postgres -c "\copy (SELECT movie_id, title, year, imdb_id, tmdb_id FROM movies NATURAL JOIN links ORDER BY movie_id ASC) to 'data/movies.csv' with delimiter E'\t' csv header encoding 'UTF8'"
psql ml_latest_small -U postgres -c "\copy (SELECT genre_id, genre_name FROM genres ORDER BY genre_id ASC) to 'data/genres.csv' with delimiter E'\t' csv header encoding 'UTF8'"
psql ml_latest_small -U postgres -c "\copy (SELECT lpad(cast(movie_id AS TEXT), 6, '0') || '_' || lpad(cast(genre_id AS TEXT), 2, '0') AS id, movie_id, genre_id FROM movies_genres ORDER BY movie_id ASC, genre_id ASC) to 'data/movies_genres.csv' with delimiter E'\t' csv header encoding 'UTF8'"
psql ml_latest_small -U postgres -c "\copy (SELECT '00000000-0000-0000-0000-' || lpad(cast(user_id AS TEXT), 12, '0') || '_' || lpad(cast(movie_id AS TEXT), 6, '0') AS id, '00000000-0000-0000-0000-' || lpad(cast(user_id AS TEXT), 12, '0') AS user_id, movie_id, rating, rated_at FROM ratings ORDER BY user_id ASC, movie_id ASC) to 'data/ratings.csv' with delimiter E'\t' csv header encoding 'UTF8'"
psql ml_latest_small -U postgres -c "\copy (SELECT '00000000-0000-0000-0000-' || lpad(cast(user_id AS TEXT), 12, '0') || '_' || lpad(cast(movie_id AS TEXT), 6, '0') AS id, '00000000-0000-0000-0000-' || lpad(cast(user_id AS TEXT), 12, '0') AS user_id, movie_id, tag, tagged_at FROM tags ORDER BY user_id ASC, movie_id ASC) to 'data/tags.csv' with delimiter E'\t' csv header encoding 'UTF8'"
offline$ mv data/users_.csv local/
ENCRYPTION_KEY の生成
- 下記ファイルをダウンロードする。
keygen.pyを実行し、キーを生成する。
(recsys_full) offline$ mv ~/Downloads/keygen.py src/
(recsys_full) offline$ python src/keygen.py
<FERNET_KEY>
- 生成したキーをファイルに保存する。
src/offline/local/keys
ENCRYPTION_KEY=<FERNET_KEY>
ユーザの email アドレスの暗号化
- 下記ファイルをダウンロードする。
- recsyslab / recsys-full / src / offline / src /
user2encrypt.py
- recsyslab / recsys-full / src / offline / src /
(recsys_full) offline$ mv ~/Downloads/user2encrypt.py src/
(recsys_full) offline$
export ENCRYPTION_KEY=<FERNET_KEY>
(recsys_full) offline$ python src/user2encrypt.py --in_csv local/users_.csv --out_csv data/users.csv
processing users: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 610/610 [00:00<00:00, 20352.60it/s]
elapsed_time:0.069[sec]
データの確認
offline$ ls data/
genres.csv movies.csv movies_genres.csv ratings.csv tags.csv users.csv
offline$
less data/users.csv
less data/movies.csv
less data/genres.csv
less data/movies_genres.csv
less data/ratings.csv
less data/tags.csv