Main content

Contributors:

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: UPDATE (November 8, 2023): This archive has been superseded by MLRegTest on Dryad. Please use the Dryad to access MLRegTest. See the WIKI or README for more information and link to Dryad. ===== MLRegTest is a benchmark for machine learning systems on sequence classification, which contains training, development, and test sets from 1,800 regular languages. MLRegTest organizes its languages according to their logical complexity (monadic second order, first order, propositional, or monomial expressions) and the kind of logical literals (string, tier-string, subsequence, or combinations thereof). The logical complexity and choice of literal provides a systematic way to understand different kinds of long-distance dependencies in regular languages, and therefore to understand the capacities of different ML systems to learn such long-distance dependencies. Details are described in the arxiv preprint "MLRegTest: A Benchmark for the Machine Learning of Regular Languages".

License: CC-By Attribution 4.0 International

Wiki

UPDATE (November 8, 2023)

In spring 2023, unintended biases were discovered in the MLRegTest training, dev, and test sets. The sources of error causing these biases have been corrected, and the models have been retrained, and evaluated. The updated paper is being posted to arxiv as version 2. The new datasets and models are available at DRYAD here: https://doi.org/10.5061/dryad.dncjsxm4h

Related S…

Files

Files can now be accessed and managed under the Files tab.

Citation

Recent Activity

Unable to retrieve logs at this time. Please refresh the page or contact support@osf.io if the problem persists.

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.