Date created: | Last Updated:

Category: Project

Description: UPDATE (November 8, 2023): This archive has been superseded by MLRegTest on Dryad. Please use the Dryad to access MLRegTest. See the WIKI or README for more information and link to Dryad. ===== MLRegTest is a benchmark for machine learning systems on sequence classification, which contains training, development, and test sets from 1,800 regular languages. MLRegTest organizes its languages according to their logical complexity (monadic second order, first order, propositional, or monomial expressions) and the kind of logical literals (string, tier-string, subsequence, or combinations thereof). The logical complexity and choice of literal provides a systematic way to understand different kinds of long-distance dependencies in regular languages, and therefore to understand the capacities of different ML systems to learn such long-distance dependencies. Details are described in the arxiv preprint "MLRegTest: A Benchmark for the Machine Learning of Regular Languages".