Papers  /

205.1 Measuring News Similarity Across Ten U.S. News Sites.

  1. Grant Atkins
  2. Alexander Nwala
  3. Michele Weigle

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Communication

Description: News websites make editorial decisions about what stories to in- clude on their website homepages and what stories to emphasize (e.g., large font size for main story). The emphasized stories on a news website are often highly similar to many other news websites (e.g, a terrorist event story). The selective emphasis of a top news story and the similarity of news across different news organizations are well-known phenomena but not well-measured. We provide a method for identifying the top news story for a select set of U.S.- based news websites and then quantify the similarity across them. To achieve this, we first developed a headline and link extractor that parses select websites, and then examined ten United States based news website homepages during a three month period, No- vember 2016 to January 2017. Using archived copies, retrieved from the Internet Archive (IA), we discuss the methods and difficulties for parsing these websites, and how events such as a presidential election can lead news websites to alter their document representa- tion just for these events. We use our parser to extract k = 1, 3, 10 maximum number of stories for each news site. Second, we used the cosine similarity measure to calculate news similarity at 8PM Eastern Time for each day in the three months. The similarity scores show a buildup (0.335) before Election Day, with a declining value (0.328) on Election Day, and an increase (0.354) after Election Day. Our method shows that we can effectively identity top stories and quantify news similarity.

License: CC-By Attribution 4.0 International


Loading files...



  • 205. Web Preservation

    The three papers in Session 205 explore the issues and topics pertaining to Web Preservation with examples of good practice and recent developments. ...

    Recent Activity

    Loading logs...


Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.