We make available the WSJ portions of the dataset as the WSJ corpus is freely available from the web. As Gigaword is only available through a license, we're making available the smallest Gigagword subdataset (the "yet" subdataset -- less than 10% of our data). We'll be happy to share the complete Gigaword dataset with a proof of a Gigaword license.

Each zip file contains three directories: train, test, dev. A readme file explains the structure of the data in the pkl files along with file reading instructions.

WSJ dataset -- ACL 2018 version: wsj_acl2018.zip

WSJ dataset -- updated version (based on linguists' annotations): wsj_new_annotations.zip

Gigaword "yet" subdataset: gigaword_yet.zip

Readme file: readme