Assessing and Improving the Experience of Underrepresented Populations: A Participatory Design Approach

Introduction Assessment practices provide essential data for informing evidence-based decision-making in libraries. In an effort to better understand library user communities and to assess library services from the perspective of specific groups, the user-centered methodology of participatory design offers a promising path forward. Participatory design is a socially active, values-driven approach to co-creation that seeks to give voice to those who have been traditionally unheard. This paper describes a parallel participatory design process undertaken by Pennsylvania State University (PSU) and Montana State University (MSU). Each project team facilitated a series of 10 workshops with student participants from distinct populations: PSU worked with first-generation students; MSU worked with Native American students. In this paper, we present a brief background of participatory design, case study overviews, and recommendations for practicing participatory assessment.


Introduction
Assessment practices provide essential data for informing evidence-based decision-making in libraries.In an effort to better understand library user communities and to assess library services from the perspective of specific groups, the user-centered methodology of participatory design offers a promising path forward.Participatory design is a socially active, values-driven approach to co-creation that seeks to give voice to those who have been traditionally unheard.This paper describes a parallel participatory design process undertaken by Pennsylvania State University (PSU) and Montana State University (MSU).Each project team facilitated a series of 10 workshops with student participants from distinct populations: PSU worked with first-generation students; MSU worked with Native American students.In this paper, we present a brief background of participatory design, case study overviews, and recommendations for practicing participatory assessment.

Context and Literature Review
Library user communities are multifaceted and diverse.In order to adapt and improve library services for the diversity of library users, the library assessment community has become increasingly attuned to the experiences of traditionally underrepresented populations.In an extensive literature review, however, Neurohr 1 notes that only a few assessment studies address underrepresented students and academic libraries.These relatively few studies include rural and tribal library communities, 2 international students, 3 Latino students, 4 first-generation students, 5 Indigenous peoples, 6 and transfer students. 7Taken together, these studies demonstrate a motivation towards assessing and improving the library experiences of diverse communities.This shift toward critically assessing the needs of multicultural users is significant for our field, 8 as it requires an empathetic appreciation of the individual journey of users. 9The relative scarcity of such studies, however, also demonstrates the significant and continued need to better understand our users' diverse and unique experiences, and to design library services with and for specific populations.The present study is motivated by these needs and adds to a growing body of critical assessment literature by proposing participatory design as a culturally-attuned method for library assessment with traditionally underrepresented populations.
Participatory design is a socially active, values-driven approach for co-creating mutually-desired outcomes across multiple stakeholder groups. 10As a socially active approach, participatory design attends to matters of power, working to identify and rebalance material and social inequality. 11As a values-driven approach, participatory design adheres to a set of design principles that include mutual learning, power sharing, and the equal recognition of expertise among all participants. 12Since it was developed in the mid-1970s, participatory design has been applied across disciplines to co-create new, mutually-beneficial products and services within a diverse range of communities. 13rticipatory design has also been applied in libraries as a methodology for co-designing services with users.Wood and Kompare, for example, applied a range of participatory design tools and techniques for assessing and improving a library website in collaboration with student participants. 14A report by the Council on Library and Information Resources presents a diverse series of case studies showing that participatory design can enable diverse stakeholder groups to communicate and co-create in a library environment. 15s a method for assessing and improving the experience of library users, participatory design can be empowering for participants, responsive to diverse cultural identities, and effective for generating and evaluating new ideas. 16The case studies and discussion below presents the details of participatory design in practice.

Case Study Introduction
To better understand and apply participatory design, PSU and MSU followed a parallel design process.With a parallel design technique, multiple teams follow the same set of processes and requirements, but work independently to complete the project. 17Parallel design allows a single process to produce different results and insights, which can then be shared and combined in refining the final design process.For this project, assessment teams at PSU and MSU applied participatory design as an approach for engaging underrepresented student groups in the assessment lifecycle.Each team followed a similar process and worked with different populations, MSU with Native American students during the spring 2017 semester and PSU with first-generation students during the spring 2018 semester.During the fall 2017 semester, the MSU team led a series of train-the-trainer sessions with the PSU team that provided a practice space for the facilitators that proved valuable for the success of the projects.Details of our process are outlined below through case study descriptions, followed by a discussion of lessons learned and recommendations for practice.

Participatory Design Activities, Sequencing, and Staging
Participatory design seeks to engage participants in a process of creative and critical thinking.Activities created for this purpose serve to help structure dialogue and guide conversation towards productive, community-based insight.For our process, activities were selected from four sources: • Brand Deck 18 • Intuiti Creative Cards 19 • 75 Tools for Creative Thinking 20 • Gamestorming 21 Activities from these resources were sequenced through three primary design stages: 1. Exploration.In this stage, participants get to know each other and begin to explore the problem area.The goal of the exploration stage is to identify key issues that affect participants within the scope of the problem space.In our case, our "problem space" was framed within the students' library experience.
2. Generation.In this stage, participants work together to generate new ideas and strategies for addressing the key problems identified in the exploration stage.The goal of the generation stage is to create multiple options for new services or products that can improve the lives or experiences of the participants.
3. Evaluation.In this stage, participants evaluate the ideas that emerged through the generation phase, and select one or more ideas to move forward towards implementation.
In our in-depth case study discussions below, we include descriptions of key activities that highlight each design stage, including discussion of the selection and sequencing of activities.Activities varied across the groups due to differences in participant feedback and project direction.The full list of activities for MSU and PSU is included in the appendix.

Exploration Phase Penn State
For the group at Penn State, campus partners helped to facilitate the recruitment of six first-generation students who bonded very quickly with one another.These students connected over a variety of things.
Throughout the beginning sessions, conversations discussed anxiety around the cost of attending Penn State, meeting expectations of their parents and family members, pressure they received from their family to perform well and succeed at Penn State, and their desire to not let anybody down during their time as an undergraduate.
In the first session, all participants and facilitators created timelines explaining their experiences that led them to arriving at Penn State (see Figure 1).This activity allowed participants to see commonality with their peers and see that the facilitators were invested in getting to know the participants.In reviewing the timelines, the facilitators were able to see how some participants focused on accomplishments and others had their timelines driven by social events or academic milestones.The foundation created in the first session allowed the group to have a larger breakthrough in the second.
During the second session, facilitators used Intuiti Creative cards.These cards have more abstract drawings on them and required the students to creatively interpret the images.The students were asked to pick three cards to tell the story of where they are, where they would like to be, and how they were going to get there.Several of the students mentioned that the "where they wanted to be" was a place where they felt grounded, and the time it would take to get there was their journey or path.
In working with the Intuiti Creative cards, one of the facilitators asked the students what they felt would be the biggest challenge or barrier to reaching where they wanted to be.After a brief moment of silence, one of the older students, a senior, spoke to the group about being a first-generation student.She talked about what she has learned being a first-generation student at Penn State.This opening up was just what the group needed to really start talking to each other.Looking back, this was a crucial point with the group, because the students felt comfortable sharing their experiences and opening up with one another.During the final, debrief session, the students shared that one thing that surprised them about this experience was how deep they had gone with their peers during the 10 sessions.

Montana State
The group at MSU began with getting-to-know-you exercises.Participants for the project were identified with the help of the director of the university's American Indian Center.With a common identity as Native American students, participants began sharing their similarities and differences through exercises such as the Great Pie.This exercise, drawn from 75 Tools for Creative Thinking, asks participants to draw a pie chart that represents their day-to-day activities (See Figure 2).The Great Pie helps participants begin to share insights about their daily life and to get to know each other.Introductory exercises like this are important for cultivating an open, trustful space of creative thinking.
After participants established a rapport through activities like the Great Pie, we then turned the focus of our conversations to the library experiences of the participants.The activity Mind Map helped illuminate participant conceptualizations of the library as an entity (see Figure 3).The Mind Map allowed each participant to share their idea of the library.The mind map is a useful framework for showing interconnections among different components of the library.In the example mind map above, the library is shown to be understood as primarily comprised of study spaces, coffee, research help, computers, and information resources.Each participant shared their mind map and the group discussed "pain points" present within the maps.Through the discussions around the mind maps and other activities, it emerged that students felt overwhelmed and intimidated by the scope and scale of a large university library.Particularly for Native students, who encounter study spaces occupied mostly by non-Native students, the physical space of the library can appear to be unwelcoming due to long-standing and looming racism.Through the exploration stage of the participatory assessment process, these feelings of being unwelcome and intimidated emerged as a key factor affecting participants' library experience.With this issue in mind, the group moved to the generation stage, and began generating new ideas and strategies for addressing this key problem.

Generation Phase Penn State
It was during the fourth session that the Penn State students and facilitators uncovered their first problem, and these conversations would shape our future sessions.The fourth session was a boundary spanner session; it was both an exploration and a generation phase session.The group did a second Unpeeling the Onion activity to more deeply explore the students' perceptions and relationships to the Penn State Libraries.
In their onions, students identified that, while they enjoyed studying at the library, many felt the library could be crowded and distracting.These elements of the library contributed to the idea that the library was not inclusive to all learning styles.A few of the students in the group talked about how the library was not their favorite place to study; one of the students shared that, before this study, she had only been to the library twice and neither experience was overall positive (see Figure 4).Using this activity allowed us to explore the students' experiences in the library and we began to uncover the problems they saw with the space, services, and accessing resources.By the end of the sixth session, we had identified and articulated our two biggest problems: 1. Due to either no information or misinformation, first-generation students might have issues feeling like they belong in the library or are just intimidated by the library.This might be caused by preconceptions of what a library is, along with no context, limited role models, and lack of instruction.
2. The library as a noisy place, paired with a lack (sometimes) of policy enforcement for those students being rude/disruptive.
After careful reflection, we chose to pursue the first problem; sessions seven and eight focused on activities meant to discover ideas to help with this problem.The facilitators did their best to let the students brainstorm ANY idea, no matter what context the facilitators had on previous solutions that had been employed in the library to solve this problem.By the end of the eighth session, the group had 40-plus ideas that could be used during the evaluation phase.

Montana State
In the exploration phase, the team identified a key problem encountered by Native students-feeling intimidated by, or unwelcome in, the library.To begin the process of generating ideas for responding to this problem, participants completed the activity of Predicting Next Year's Headlines.In this activity, participants are asked to think into the future and to imagine that a publication is writing an article on the results of the project.The prompt for this activity asks the participants to write a headline for that article.This futureoriented activity helps participants share their desires for the project and can generate ideas and consensus for possible outcomes.One participant generated four different possible headlines, including one that read, "Native American Students Help Improve Library Usability."(See Figure 5.) Predict Next Year's Headlines illustrates the generative stage of the design process.The particular example above also illustrates the participatory nature of the process, as the participant recognized that the students themselves were essential contributors to the project.All participants then shared their headlines, and the group discussed shared goals.The evidence from the headlines activity along with the other activities in the generation stage informed these discussions.What emerged from this conversation was a collective desire to improve the ease of use of the library for Native undergraduate students by creating a new library promotion and outreach campaign for the university's Native community.Our next step was to generate further detail around this shared goal.
The activity Clockwise allowed the group to develop ideas for promotion and outreach.In this activity, participants assign different ideas to the numbers around a clock face.Two dice are then rolled so that two numbers can be matched together randomly.The group then discusses the two ideas together, which sparks more new ideas.In our case, the clock face was populated with ideas for promotion and outreach, such as posters, brochures, radio spots, and a social media campaign (Figure 6).The dice roll then combined these ideas together randomly.The group discussed these ideas and continued to mix and match until a feasible and desirable idea emerged (Figure 7).The Clockwise exercise produced the idea for a multi-part promotional poster series and social media campaign.Follow-up conversations revealed that this idea was desirable within the group, feasible in terms of sustainability, and viable insofar as it could be achieved by the group within our time frame.We then turned to the evaluation phase to further refine this idea and drive towards conclusion.

Evaluation Phase Penn State
During the ninth session, the facilitators took the many ideas that had been generated in the previous sessions and discovered four, overarching themes the ideas shared.These four themes were: 3. Tours/gamification of library 4. Library going to the students 5. Library promotion

Increasing student voices
To start the ninth session, the facilitators had the students select the top two themes they wanted to focus on.The students selected "Library going to the students" and "Increasing student voices."With those two themes, the group went through the checklist activity.This activity asked the students to take an idea and think through the necessary knowledge, skills, scale, time, and resources needed to make the idea possible.While this activity did not work out perfectly, it did lend itself well to really talking through some ideas and thinking about the various elements needed to make an idea work.Again, the facilitators made sure to allow any idea to be discussed, regardless if the library had already tried that idea in the past.
From this conversation, a few necessary elements came out through the checklist activity.These elements included: • Creating a strong pitch about the library that an engaging student can deliver to their peers.The group spent a lot of time talking about the necessary personality characteristics someone would need if they were to go into first-year seminar or large lecture classes to discuss the many great things about the library.
• Student leaders needed a holistic picture of the library.Whether it be library student employees or students in a library club, the students agreed that these ambassadors for the library needed a holistic view of the library and the various services and resources it provides.
• Providing incentives to students who participated in library workshops or participated in a library student group.Just like the pitch, there is value in providing an enticing incentive.
• Scale.Throughout the conversation, the facilitators kept asking the students if the ideas they had were scalable and if reaching every student at Penn State was the goal.The students quickly recognized that reaching every student was not possible, and in order to scale most efficiently, the focus should be on foundational classes or large lecture classes that could result in a high impact.
In the end, the Penn State students did not identify one solution, but instead proposed a set of characteristics and traits that should be included in any solutions that would be put forward after their study to help raise students' awareness of the library.

Montana State
The evaluation stage focused on refining and creating the idea produced during the generation phase.In designing the promotional poster series, a leading factor for refinement included the content of the posters.
To help shape discussion on this topic, participants completed a Smiley Voting activity.For this activity, participants brainstormed various library services that could be featured on the posters, including coffee, the writing center, and checking out books and technology.Working on a whiteboard, each participant then placed smiley faces next to the services they thought were the most important to feature (see Figure 8).The mechanics of this activity are important, in that the votes allows for more equal access to the whiteboard.Participants can then express why they placed their votes next to certain services and not others.Consensus emerged from this discussion and the group decided on seven services to feature: finding the library, tutoring, writing center, research help, group study rooms, coffee, and technology checkouts.The number seven also carried specific cultural meaning for members of the group.With the content of the posters collaboratively determined through Smiley Voting and other activities in the evaluation stage, we moved forward in creating mock-up designs of the posters with Paper Prototyping (See Figure 9).The group ultimately created a seven-part promotional poster series and social media campaign, titled #MSULib101 (See Figure 10).

Case Study Outcomes
Participatory design works to produce two primary types of outcomes: practical and political.

Practical Outcomes
Through this process, the first-generation student group at PSU produced new service designs for engaging other first-generation students, while the Native American student group at MSU produced a seven-part poster series and social media campaign designed to welcome Native American students into the library.

Political Outcomes
In addition to co-designing new services, participatory design also aims to generate political outcomes that focus on empowering participants.The foundational values of participatory design include mutual learning, power sharing, and the equal recognition of expertise among all participants.Within this equity-focused, participatory framework, the student participants became expert library users who expressed readiness to advocate for the library to their peers.More than that, the students-members of underrepresented populations who often feel at the margins-developed a stronger sense of place and confidence on campus that will contribute to their success at our institutions.And for the librarian facilitators, the in-depth codesign process enhanced our ability to understand these student populations.We gained new insights into the experience of our student participants that we can apply to better serve these important populations.Ultimately, the participatory design process equipped us with the tools and insights to assess and improve the conditions of their success.

Recommendations for Practice
Based on our experiences at Montana State and Penn State, we have a few recommendations for those interested in implementing a participatory design study at other institutions.The recommendations below are practical and will help ensure that participatory design can be one effective methodology within a library's broader assessment program.

Resources
Time and money can be one of the biggest challenges in supporting this type of project.For the lead facilitator and others involved with the project, there is a lot of time invested in both preparing for the sessions and then the act of leading the sessions with the participants.Because this method relies on the participants to drive the direction of the project, the facilitator cannot plan too far in advance.For both Montana State and Penn State, the facilitators would spend a few hours leading up to and after each session, documenting the work done by the participants and planning ahead for the next session.This time is needed to help shape the project, document the process, and ensure that the results are something the group can work with at the conclusion of the study.
The time commitment for participants is also a challenge to be addressed.Both projects included ten onehour sessions and their success relied on continued student participation.We recommend considering how to address this challenge from recruitment through project completion.In recruiting students for both MSU and PSU, participants were provided monetary compensation for their participation.At PSU, students were promised up to $150 in incentives, such that each participant received $10 at the end of each session and an additional $50 if they participated in all 10 sessions.At MSU, students were paid an hourly wage of $12.
Additionally, the volunteer recruitment form included questions inquiring about commitment to all 10 sessions, interest in scholarship opportunities, and a question asking why they should be selected for the opportunity to participate.While the appeal of a substantial cash incentive was the primary motivation for participating, the questions on the volunteer form communicated the importance of committing to all sessions and the participatory nature of the project.In addition, since all participants are paid for their role in the project, there is a need to have resources that can be drawn from and given to the students.
Participatory design projects of this variety require substantial logistical efforts to ensure smooth session administration and participation.To that end the project requires, arranging for a space to conduct each session, recording equipment as desired, instruments and peripherals for session activities need to be pulled together, coordination is needed for providing food and monetary incentives including necessary forms and signatures, and frequent communication with participants (texts sent the morning of each session) is essential.

Assessment
Assessment for participatory projects often focus on one or more of three key aspects: the design process, the design product, and design sustainability.In terms of process, assessment can focus on the operations of the project and the type and depth of participation that occurred.In terms of product, assessment can focus on the tangible result of the process and its implementation.And in terms of sustainability, assessment can focus on the immediate and longer-term impact of the process and the product on relevant communities.In our cases, the process and the products of our projects were validated with participants throughout the duration of the projects and at its conclusion via informal interviews and reflections.In this way, assessment itself also worked towards participation.

Session Reflections
A habit of regular reflection is key to documenting a participatory design study.The lead and secondary facilitator should anticipate spending approximately an hour after each session documenting what has occurred and begin to connect the dots between sessions.Waiting too long after the session can result in a less robust summary and, perhaps, the loss of valuable data and insight.Another strategy Penn State used was to have someone who did not help facilitate a session watch the video recording.Then, using a facilitator's reflection, the video recording, and reviewing scans of written artifacts, they could write a more general summary of the session.These multiple perspectives can be useful later, after the study has concluded, in writing up reports, scholarship, and moving forward with recommendations made by the participants.During the study, reflections assist in providing direction for the next session and preparing the secondary facilitator for future sessions.

Facilitation
When putting together a group to lead a participatory design study, it is important to create an intentional team.This team should have colleagues from multiple departments and with various roles within the library.
Having team members with different levels of student interactions, various networks, and assorted responsibilities for the oversight of library programs can be instrumental in connecting with the student participants, creating an engaging set of sessions, and implementing student ideas.For both Montana State and Penn State, our assessment departments played a key role in the processes, and they added colleagues who would help in a variety of ways.
From the team, one lead facilitator, who has the capacity to devote considerable time to the project, is crucial.This lead facilitator will serve as the bridge between the sessions, helping to gain trust with the participants, and help their colleagues easily move in and out of sessions as the secondary facilitator.The lead facilitator should have a good understanding of institutional culture and constraints.This insider knowledge will allow them to help guide participants, especially in the evaluation stage.While we want participants to freely share ideas, with no constraints or limitations, a lead facilitator is needed to help shape those ideas and provide any necessary context to the participants.While secondary facilitators are crucial in running the sessions and getting to know the students, the secondary facilitators often noted there was some disconnect when they did not attend the sessions regularly.
Finally, in building this team, it is important to provide an opportunity for facilitators to have some train-thetrainer sessions before actually guiding students through the design process.MSU, for example, offered a series of "design sandboxes" for library staff in the months leading up to the project.These informal sessions allowed the project team to practice and experiment with design activities and sequencing in a low-stakes environment.These training sessions allow the facilitators to try out activities, practice facilitation skills, and learn more about the participatory design process.If all facilitators are trained at the same time, it creates the opportunity for the research team to better get to know one another, which helped contribute to making the process more comfortable for the students.

Participant Recruitment
Your research team should include members who have networks outside of the library.Depending on the student population you hope to explore, you will want to find colleagues who might have connections with these stakeholders, or build in time to build relationships with the communities you wish to work with.These stakeholders might be program coordinators for TRiO or new student orientation, student leaders for relevant student clubs, or other administrative units like the registrar's office who would know how students are classified in the institution's system.When it is time to recruit, you will want to make sure you have the necessary and appropriate community connections in order to build a group of student participants.

Conclusion
The process of introducing an element of participation to assessment allows for more voices to come forward in the assessment lifecycle.As one tool in the assessment toolbox, a participatory design can bring together meaningful user-centeredness with evidence-based decision-making.With its attunement to power dynamics, equal expertise, and mutual learning, participatory design is also well suited for assessment projects that focus on non-traditional user communities.As is evident in the above case studies, participatory assessment put the user in the driver seat, and often these users can co-create new ideas or amplify existing ideas for improving library services.Ultimately, participatory design can be an empowering, culturally responsive, and effective approach for designing and assessing library services and experiences.

Fig 1 .
Fig 1. Timeline activity completed by a PSU participant, showing their journey to Penn State.

Fig 2 .
Fig 2. Great Pie activity completed by an MSU participant, showing various day-to-day tasks.This activity serves an introductory purpose.

Fig 3 .
Fig 3. Mind Map activity completed by an MSU participant, showing a visual representation of the related parts of the library.

Fig 4 .
Fig 4. Unpeeling the Onion activity completed by a PSU participant, showing their relationship and experience with the Penn State Libraries.

Fig 5 .
Fig 5. Predict Next Year's Headlines activity completed by an MSU participant, showing the desired future results of the project.

Fig 6 .
Fig 6.Clockwise activity completed by MSU participants.This activity generated new ideas for outreach services for students.

Fig 7 .
Fig 7. Clockwise activity completed by MSU participants.In combining multiple ideas together, new ideas emerged.Red highlights mark the final idea for a promotional poster series.

Fig 8 .
Fig 8. Smiley Voting activity completed by MSU participants.This activity allowed participants to share votes and produce a ranked priority list relating to different library services that would appear on the promotional poster series.

Fig 9 .
Fig 9. Paper Prototyping activity completed by an MSU participant, showing an early model for the promotional poster series.

Fig 10 .
Fig 10.The final design for the MSU promotional poster series.