a/ebooks
1
1
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
Fediverse ebooks bot using neural networks https://social.exozy.me/@ta180m/107833248217767420
72 commits 1 branch 0 tags 164 KiB
Python 100%
Go to file
Anthony Wang 123846ed7d
Fix pip command to install pytorch instead of old torch package
2022-02-26 12:02:44 -06:00
.gitignore Initial commit 2022-02-20 18:58:25 -06:00
bot.py Remove @ mentions 2022-02-23 15:31:02 -06:00
bot_lstm.py Add new transformers bot script and move old one to bot_lstm.py 2022-02-22 16:58:19 -06:00
data.py Modify data generation script for transformers 2022-02-22 16:57:12 -06:00
dataset.py More cleanup 2022-02-21 15:20:00 -06:00
LICENSE Initial commit 2022-02-20 18:58:25 -06:00
model.py More cleanup 2022-02-21 15:20:00 -06:00
predict.py Take top 5 instead of top 3 2022-02-21 16:39:58 -06:00
README.md Fix pip command to install pytorch instead of old torch package 2022-02-26 12:02:44 -06:00
train.py Reformat code with autopep8 2022-02-22 17:55:04 -06:00
train_lstm.py Move train.py to train_lstm.py and add new transformers training code 2022-02-22 16:12:13 -06:00

README.md

ebooks

Fediverse ebooks bot using neural networks

Usage

First, install Python dependencies with pip install psycopg2 pytorch transformers datasets Mastodon.py. Currently only Mastodon with PostgreSQL is supported, but it should be easy to add support for other platforms and databases.

Generate the training data from the Mastodon database using python data.py -d 'dbname=test user=postgres password=secret'. You can skip this step if you have training data from another source.

Next, train the network with python train.py, which may take several hours. It's a lot faster when using a GPU. If you need advanced features when training, you can also train using run_clm.py.

Finally, create a Mastodon application for your bot account and run the bot with python bot.py -i mastodon.instance -t access_token -n 'Initial text'. You can create a cron job to run this periodically. Enjoy!

Resources