Python scripts for building 'Short Jokes' dataset, featured on Kaggle
This repository contains all the python scripts used to build Short Jokes dataset. The dataset contains 231,657 short jokes scraped from various websites.
All the web scraper scripts are present in the
/scripts/scrapers/folder. These scripts are written for specific websites (website link mentioned in the header of each file) and they generate csv files of jokes in
/data/folder with the fixed format:
Jokes from subreddits
/r/cleanjokesare extracted using
scripts/merge_csvs.pyremoves the duplicates from all the csv files and merges the jokes into a single csv to get the final dataset