![Logo of Te Kāhui Roro Reo, the New Zealand Institute of Language, Brain & Behaviour.][1] ![Logo of the UC Arts Digital Lab.][2] # Creating Specialised Corpora from Digitised Historical Newspaper Archives: An Iterative Bootstrapping Approach ### Joshua Wilson Black *UC Arts Digital Lab* *The New Zealand Institute of Language, Brain & Behaviour* <joshua.black@canterbury.ac.nz> <joshua@wilsonblack.nz> This repository provides access to the code required to generate specialised corpora from METS/ALTO digitised historical newspaper archives and to data generated in the reported test case of the method. The code is shared via OSF's integration with GitHub while the data from the test case is stored on OSF's storage service. Details of the GitHub repository can be found on its [page](https://github.com/JoshuaWilsonBlack/newspaper-philosophy-methods). The data stored here consists of: - `corpora`: Candidate corpora at each stage, saved as pickled pandas dataframes. - `candidate_corpus_3_widenet.tar.gz` contains the corpus generated by the classifier after the third iteration with a threshold of 0.4, rather than 0.5. This corpus is not reported in the paper but is provided for the reader to experiment with. - `labels`: Pandas dataframes and csv files of the labels for each iteration of the corpus construction process. - `processed_data`: All 'article' items (i.e. _not_ advertisements) from the Papers Past Open Data Pilot, stored as a series of pandas dataframes. - `meta`: Metadata concerning newspaper codes, items, and their location within the processed data. - `wiki_images`: Logo images for this page. - `corpus-3`: `csv` and plain text versions (zipped) of the third candidate corpus. An interactive dashboard to explore the results over the iterations of the corpus construction process is available [here](https://nz-newspaper-philosophy.herokuapp.com/). The code for this dashboard, which can be run locally, can be found [here](https://github.com/JoshuaWilsonBlack/NPOD_Philosophy_Heroku.) [1]: https://mfr.osf.io/export?url=https://osf.io/download/df89u/?direct=&mode=render&format=2400x2400.jpeg [2]: https://mfr.osf.io/export?url=https://osf.io/download/psjgx/?direct=&mode=render&format=2400x2400.jpeg