Char-RNN implemented using TensorFlow.
A TensorFlow implementation of Andrej Karpathy's Char-RNN, a character level language model using multilayer Recurrent Neural Network (RNN, LSTM or GRU). See his article The Unreasonable Effectiveness of Recurrent Neural Network to learn more about this model.
Follow the instructions on TensorFlow official website to install TensorFlow.
If the installation finishes with no error, quickly test your installation by running:
bash python train.py --data_file=data/tiny_shakespeare.txt --num_epochs=10 --test
This will train char-rnn on the first 1000 characters of the tiny shakespeare copus. The final train/valid/test perplexity should all be lower than 30.
train.pyis the script for training.
sample.pyis the script for sampling.
char_rnn_model.pyimplements the Char-RNN model.
To train on tiny shakespeare corpus (included in data/) with default settings (this might take a while):
bash python train.py --data_file=data/tiny_shakespeare.txt
All the output of this experiment will be saved in a folder (default to
output/, you can specify the folder name using
--output_dir=your-output-folder).
The experiment log will be printed to stdout by default. To direct the log to a file instead, use
--log_to_file(then it will be saved in
your-output-folder/experiment_log.txt).
The output folder layout:
your-output-folder ├── result.json # results (best validation and test perplexity) and experiment parameters. ├── vocab.json # vocabulary extracted from the data. ├── experiment_log.txt # Your experiment log if you used --log_to_file in training. ├── tensorboard_log # Folder containing Logs for Tensorboard visualization. ├── best_model # Folder containing saved best model (based on validation set perplexity) ├── saved_model # Folder containing saved latest models (for continuing training).
Note:
train.pyassume the data file is using utf-8 encoding by default, use
--encoding=your-encodingto specify the encoding if your data file cannot be decoded using utf-8.
To sample from the best model of an experiment (with a given starttext and length): ```bash python sample.py --initdir=your-output-folder --start_text="The meaning of life is" --length=100 ```
To use Tensorboard (a visualization tool in TensorFlow) to visualize the learning (the "events" tab) and the computation graph (the "graph" tab).
First run:
bash tensorboard --logdir=your-output-folder/tensorboard_log
Then navigate your browser to http://localhost:6006 to view. You can also specify the port using
--port=your-port-number.
To continue a finished or interrupted experiment, run:
bash python train.py --data_file=your-data-file --init_dir=your-output-folder
train.pyprovides a list of hyperparameters you can tune.
To see the list of all hyperparameters, run:
bash python train.py --help