# SHARE Maintenance This document wil outline all tasks that need to happen in order for SHARE to continue to function as it stands, with community expectations being met. ## Harvesters ### Harvester Registration On average, we get about 4 or 5 external sources a week registering to be a provider for SHARE, not including tests or incomplete registrations. ##### Actions that need to be taken - Check registration page on a regular basis - Evaluate completeness of the registration - Is it a test? - Have they registered before? - Are they confused about metadata license requirements? - If the registration is complete and all looks good: - Add registrant as a customer in Desk - Send them an email using a template saying the registration is complete. - Check off (and save) that the provider has been contacted on the SHARE Registration admin interface - Continue the conversation as needed and follow up with any questions or comments they might have. - Create a corresponding ticket in JIRA with all relevant information - If they look like they had trouble with the metadata questions (the registration will be complete, but metadata questions not all checked) - Add registratnt as a customer in Desk - Send them an email using the "confused about metadata?" template - Continue the conversation as needed about what they're confused about and what their needs are - If their questions are answered, add their provider as a harvester ticket in JIRA - After the harvester has officially been added, go back into desk and inform the person that it's complete, and close the DESK case. ### Harvester Creation #### OAI-PMH Harvesters ##### Description These are the simplest of the harvesters. They take only a little effort to write, and can be created in under an hour if all goes smoothly. Sometimes, however, a source's OAI-PMH endpoint behaves strangely. Problems we've seen include: - date ranges returning the wrong sets - oai_dc metadata format missing - URLs lead to strange splash pages or are broken - More URLs can be constructed using persistant identifiers - DOIs hidden in descriptions or titles that need to be extracted - Contributors presented in strange formats ##### Actions that need to be taken - Code review for the harvester - Are the short names appropriate? - Tests working correctly? - Correct sets harvested? - Are the URLs returning correctly? - Do we need to modify the URLs in any way to get access to the content we want? - Emailing the registrant with any questions we may have - Merging the harvester into develop ### Releases and hotfixes Before new harvesters are visible to the public, there needs to be a release of scrapi containing all new harvesters ##### Actions that need to be taken - Code reviewed harvesters - Releases - Harvesters merged into develop - Release made via github - Data harvested on the backend - backend updated and restarted - Hotfixes - Harvester fixes made on master - Migrations run to fix old data - backend updated and restarted ### Breakadges and General questions from SHARE at Large Many times, providers who have their sources in production have questions about SHARE - They ask for more information - They want their provider to be updated in some way - They want us to harvest more metadata Providers already in service break - APIs can change - URLs that once worked stopped working These in many cases require email correspondance with the original provider. They also require a new release or hotfix to be made to address the problem ### SHARE Push API As of right now, the SHARE push API is only deployed on staging. We have a group at uPenn who is very eager to see their data on SHARE production. Everything is already in place. ### SHARE VIVO Harvester The VIVO harvester has a pull request, but was left in an unknown state. It's possible we still need more information to run it, but it is NOT currently in operation, either in staging or production. ### Things that require work but could be immensely helpful - The OSF SHARE page is an example, yes, but is very much more incresignly being used as THE place that people go for SHARE content. - Advanced search does not exist. People get confused how to use the API, and go to the search page to search for content. - API documentation is scattered - Docs are pretty thorough, but a little scattered between swagger and notes on many different OSF components. This could use a go over from an outside perspective to make things more unified. - Right now, the desk to person is manual. Desk has an API that could be accessed thru the SHARE registration page that would at the very least create a new case and fill in all of the parameters. - We never formally did the "Metadata Audit" that conformed to our new understanding of the metadata schema. This maybe should happen before a host of new harvesters come into play.