Welcome to our 40-minute presentation called "Creating an inclusive, adaptable
model for tracking, assessing, and celebrating student success in research."
My name is Amanda MacDonald and I serve
as the undergraduate research services librarian at Virginia Tech.
While I'll be narrating today's presentation, this presentation was
developed with two VT colleagues, Anne M. Brown, our science
informatics consultant and health analytics coordinator from
University Libraries, and Keri Swaby, the director of the office of
undergraduate research.
I'm sure many of you face a similar situation at your institutions.
Students are finding and engaging in mentored undergraduate research
but there are real issues of access to research where not all students feel
they can participate. Many students are intimidated and don't
even know what research is or how to get started.
Some can't afford to do research at the expense of a campus job, and others don't
even have the space in their schedule. They are coming to it haphazardly.
They are most often not approaching it in a systematic,
intentional, or holistic way. There is a disconnect between students
and information and resources so they often don't know what's available
and where to find it. The students doing research have varying levels of training
and preparedness. Some will jump into projects without any preparation at all,
while others have 101 research type classes
that guide them through the research process. And sometimes students aren't
even recognizing that they are actually engaged in research.
Finally, students often don't recognize the value of the research experiences
they've had and how to share that with future
employers or postgraduate programs or even put it on a resume.
In addition, our administration is working towards requiring every student
to engage in at least one experiential learning experience,
including undergraduate research, which has some real challenges as well.
Many of you struggle with the same issue of tracking and counting students
and faculty engaging in research. Our institution is decentralized,
so often programs overlap and duplicate efforts.
There are challenges related to communication and building partnerships
planning synergistic programming. Graduation check sheets are full and
adding another credited course or two might be prohibitive.
Meaningful assessment of student outcomes is happening
piecemeal and of varying quality across campus.
This is challenging with limited personnel and trained professionals
dedicated to these activities. At a large institution serving a broad
population issues of inclusion and access to
research and related activities are a paramount concern.
Our solution to these problems was to develop the undergraduate research
excellence program, or UREP for short, in a way that
addresses all of these deficiencies. What you're seeing on your screen is our
final badge, UREP scholar, and our micro credentials, those
little badges a student receives along the way as they complete
each stage of the program. Students must earn four of the five
badges to complete the program this is designed to be flexible for
students for example there are some cases in which a student is not able to
disseminate their research as it is part of a larger research
project so the mentoring badge is another way students can engage with
future researchers without having to disseminate their contributions to a
research project to earn that badge students might serve
as an ambassador for the office of undergraduate research
or as an editor for one of the student publications on campus
almost three years ago the stores aligned where the identified needs and
deficiencies across campus matched up with several activities that
were already in place but more importantly several people were
thinking about the same problems from different vantage points and were ready
to take action recognizing that research is happening
all across campus in different forms and at different times in the student's
academic career but with no good way to track it or
assess its impact the university libraries and the office
of undergraduate research came together and identified
key elements that would address the deficiencies and issues
number one a way to track the number of students engaged in research
at vt it's difficult to accurately count the number of students engaged in
research because there are a variety of ways to
do it volunteer for pay for credit but we would like to
know and often get asked to report on it in addition to the student's level of
engagement and progress number two increase the number of
students participating in research in the excellent co-curricular research
prep and supporting programs that offer students great advantages but
no way to incentivize students to do them
and complete them number three recognizing students are doing great
things at vt and do not otherwise get celebrated meaning there are no
certificates for undergraduates and number four incentivizing students
to share and reflect on their experiences
which is part of a holistic research cycle but also a way to help us assess
in addition to what we needed in urep there were things we wanted as well
first and foremost we wanted urep to be student centered
in the developmental stages of the program we made decisions based on past
experience working with student researchers now that the program has
been deployed we survey students as part of our
assessment and evaluation process to ensure that student feedback and
perception leads our decision making and revising
we also wanted to create multiple entry ramps into the program to ensure we
didn't miss engaging with students about this
opportunity and they didn't miss the chance to participate
students could hear about urep during the advanced research skills program
a co-curricular workshop series designed for undergraduate researchers
during a first year experience course that has undergraduate research training
embedded into it during our intro to research course
during an undergraduate research experience like a summer reu
or from our marketing and outreach campaign just to name a few
students are allowed to enter this program at any point during their
undergraduate career at vt which is why the multiple entry points
is so important regardless of their year academic
standing major they must have a path to enter
participate and exceed and succeed in this program we also wanted
to ensure the program was pedagogically sound
while we'll talk about this a little later in the presentation kolp's cycle
of experiential learning was fundamental throughout the designing of the program
lastly we wanted to ensure this program was fully accessible to all students
to do this the program is based in the office of undergraduate research
since this would likely be the office any student or faculty member interested
in undergraduate research would contact
as you can see in this model there are three stages for students
explore engage and scholar explore is the introduction in education
early on in a student's career this is where the exposure to
undergraduate research happens this could be workshops or it could be
intro classes examples can include some of the things
i just mentioned on the last slide the advanced research skills program a
co-curricular undergraduate research workshop series
or the intro to research practices course just to name a few
engage is where the student is actually conducting research
this could be research for credit courses it could be working in a lab
it could be a summer experience like an reu just to name a few
when students do this for course credit it is often at the request of the
student to the faculty member this can sometimes count towards the
student's major as well and other times this will count as an
elective scholar is when the student is all ready
to disseminate the research at this stage they're thinking about
where to present and publish they may place artifacts in an
e-portfolio or even in a university digital repository
they may present at a conference on campus at a national conference like
anchor or even at a discipline specific
conference which might be with their faculty or graduate student mentor
the three-tier design was created because we want every student to have a
well-rounded experience yet we understand that what
undergraduate research looks like across majors can be very different
ideally this program is designed that any undergraduate student
of any major at any point in their undergraduate career could enter this
program and begin tracking for success but the
earlier student starts the more likely they'll
be able to earn the graduation court while the three stages were important to
us to help ensure the student is well-rounded
what students see are five categories in which they can report their
undergraduate research experiences training conducting disseminating
mentoring and then reflection should be done throughout
let's take a look at what the structure looks like from a different view
here you can see that there are three stages again however there is a dotted
line each student's journey will look
different the dotted line is meant to signify that there is not one clear path
especially as we move to scholar in undergraduate research
students might not be able to present or publish on the work they've done
as it might be part of a larger research project so
we need to provide more than one avenue for a student to achieve the highest
level the outer gray circle is also part of
this figure which is where we believe the ref
the reflection falls as reflection is something the program emphasizes
throughout and in multiple ways students can opt to keep research blogs
e-portfolios showing their engagement with research over time
or reflective essays as the student would discuss their research
academic professional and personal experiences goals and future directions
in one space
while urep is a program designed to give recognition to the students work
that they're already doing the program model as i mentioned earlier is still
grounded in theory undergraduate research is experiential
learning so it is not terribly surprising that we see this program
aligning with cold cycle of experiential learning the concrete example could be
taking a class or a workshop series the reflective
essay could be the reflective observation this could lead the student
to abstract conceptualization where the
student may take the mock proposal from the workshop
and maybe find a mentor to work with in which point the student moves to
practical application the conducting of research ideally this
will continually happen for the student far beyond one time in the lab
but all the way through the publication and presentation process
that is also awarded in the program with reflection again
throughout this is where e-portfolios could come into play
so we started to build the model by first surveying what was available and
happening on campus we knew students are engaging in
mentored research in every discipline across campus
undergraduate research training courses for students in any discipline
already exists by way of a highly successful co-curricular program offered
by university libraries called the advanced research skills
program and a credit bearing research preparatory course the office
of undergraduate research is an established
university level service unit providing key research services to students
including ambassadors who are veteran researchers and serve as peer mentors
two campus-wide symposia that provides students with a venue to present their
work and comprehensive programming for summer
researchers that includes professional development activities
the university utilizes the flexible canvas course management system
which students are already familiar with
there are a variety of ways in which we can encourage student participation
the one that seems to excite students the most is a graduation cord
during our first semester launching the program a student actually emailed
carrie our director of the office of
undergraduate research and said rumor on the drill field is that there are
graduation cords we were so thrilled to be a rumor
this spring we are also adding badging or digital credentialing into the mix
i think the badges help incentivize students to keep progressing through the
program as they can see with the badge and the
pathway where they are and where they need to continue going to
complete the program one of the key elements of badging is
that it allows students to complete just one part of the program
so a student might do training and earn the tier 1 badge
explore or do training and conducting and earn the tier 1 and tier 2 badges
explore and engage this means that a student might only get one or two badges
and that's totally okay if that aligns with their goals
we want this program to serve students and align with their goals and in doing
so we must recognize that not every student will want to go
the entire way through the program there should still be a celebration for
their hard work this summer we are offering a
certificate for summer researchers who complete the training so that
certificate will also count towards urep in regards to the training section of
the program
to enter the program students self-enroll via a google form
then students are enrolled in canvas via a course that consists of
five modules there are no assessments here
students upload proof of undergraduate research participation
via assignments for each section as administrators we verify
their uploads micro credentials are awarded as students complete each
section when the final europe scholar badge is
awarded then the student earns a graduation cord for completing the
program europe itself has no training or
experiences for students rather it is designed as a place for
students to track and report their experiences via assignments
once the experiences are verified students will receive a badge for each
tier explore engage in scholar and micro
badges for each category training conducting
disseminating mentoring and reflecting to earn their
graduation cord they must fully complete all three tiers of the program as you
can see our programmatic goals align directly
with the needs that we've established prior to creating urep up provides
students a chance to track their undergraduate research
journey including trainings validate their experience through
recognition of their research efforts and achievements
showcase their experiences for potential employers and professional graduate
schools
now that we've discussed program development and deployment
we should also discuss what we know right now about the students who are
enrolling and completing urep in terms of enrollment we have students
enrolled from all of vt's major colleges this enrollment
data tells us that the majority of students are from stem fields
which is not surprising it also shows us lowest enrollment is from architecture
and urban studies and business knowing this information
can allow us to consider if there are programmatic adjustments
that need to be made for the program to appear more appealing to students
in those colleges or if there was not enough outreach to students in those
colleges
as we move towards completion rates for the program the non-stem
majors decrease drastically this visualization shows us that
students in the colleges of science and engineering are most likely to complete
the program again these results may tell us that we
need to consider reasons for this are there aspects of the program that
could be revised or redesigned to make students and other colleges
equally as successful in terms of program completion
when we take a closer look at those who have received graduation cords in terms
of majors it becomes really clear that students
majoring in the life sciences earned the most cords
from this visualization in the previous one we know that we need to make an
effort to engage students in other domains
this engagement will hopefully be in the form of outreach and participation
but also in gathering student feedback from other domains in order to help
improve and grow this program
the last visualization shows student participation over time
what we can see here is that the enrollment remains consistent over time
in that students are progressing through the program who aren't set to graduate
until 2022 which means freshmen and sophomores are
enrolling in this program so they must have some level of exposure
to undergraduate research and or an understanding of the value of
undergraduate research
this semester we compiled historic participant data
and surveyed past participants which included any student who enrolled in
but did not necessarily complete the program between
inception through this spring the focus of the survey was to get feedback on the
program and its value to students we observed some clear trends that will
inform us moving forward students are signing up for the program
as upperclassmen this makes sense since students tend to
engage in research beyond their freshman year
academic advisors are important to getting the word out and encouraging
students to join your rep students felt that the nuts and bolts of
the program were well designed and easy to navigate and follow
from registration to submitting artifacts
students indicated that they would recommend the program to others
suggesting their perceived satisfaction and value of the program
students identified several specific benefits from participation
which coincided with the key reason students chose to participate in the
first place number one the graduation cord this is
the real carrot we were so excited after that first
semester of the program you know when we found out we were a rumor across the
drill field number two the cv resume booster aspect
number three recognition for time and energy invested in research
at vt students don't get recognized in any other way for all of the time and
energy they invest in undergraduate research
and lastly the majority of the survey respondents indicated their
plan to pursue graduate school or a research related job after graduation
throughout the survey students strongly indicated the high impact they felt
research had on their learning overall college experience and helped to
inform their post-graduation plans but they did not perceive europe as
necessarily contributing to these instead europe was recognized as a way
for students to be celebrated for all of their hard work
and time investment in research help students to plan
and eventually organize and document their research related experiences
and reflect on their experience which they were not otherwise required to do
student reflections universally explore their positive personal and professional
growth through undergraduate research and the
relationship with their research mentor
overall response to the program has been extremely positive
students are excited about the program and outcomes
this student shown here was one of the inaugural winners of the research
excellence award this year based on her completion of urep and
strong presentation of her research project at our annual spring symposium
she is proudly wearing her graduation cord her kind words have encouraged us
to continue improving ureth she said quote not many students
get to have a unique experience surrounded by supportive people
and administration i'm appreciative for all that this program has done for me
and i can't wait to take all these skills to my next life experience
end quote as i mentioned earlier student feedback
is extremely important to us as such survey respondents provided
insightful feedback and identified areas for improvement
several students indicated that they wished they had learned about the
program sooner since several only discovered it in
senior year and participated because they had satisfied all of the
requirements rather than using it as a way to
intentionally plan their research journey
moving forward we will promote the program at every opportunity we can to
engage with students in any discipline and at any stage in our academic career
we will invest more time focusing on sophomores which seems to be the sweet
spot when students are ready to think about research and planning and
tracking their experience we will be more explicit in describing
the purpose and value of the program really calling out the fact that the
program is a holistic approach to undergraduate
research students wanted more flexibility
in what would count towards the program as some felt they could not easily
fulfill the requirements of certain categories
starting the program earlier should help with this since students can plan ahead
rather than try to fit their experiences into the program categories
but we can also provide provide students with an option to submit
additional artifacts not explicitly listed in each category
and defend their inclusion adding this feature will add a greater
administrative burden
some students outside of stem felt there weren't enough options for them to
complete urep the program was designed to be
inclusive so this suggests that our messaging and instructions need
to be modified to help in this process we will reach
out to academic advisors to learn about courses and practices in their areas
which may count towards urep as well as terminology
that can be very different across disciplines
this feedback coupled with our visualizations confirms we have work to
do to make this appealing and more
accomplishable for non-stem students in terms of adapting the model
the first thing that you can do is identify a program at your institution
that you think this model could support an existing program helps
but it's not necessary identifying what exists at your school
this could be certificate certifications medals
plaques transcript notation graduation cords badging
look to see what carrots already exist whatever program you're doing what does
the well-rounded student look like what other units are involved who can
you partner with where does it fit in in terms of the
budget will anything be split what do you already have in place in
terms of the budget who or what can you leverage
do you need any technology what will your managing platform look like
how will you get your data how will you store it safely
what are your goals and outcomes what is your assessment criteria
how will you score all of these questions will help you think about how
to take this model and adapt it to your institution
we are thinking about several things as we move into the next academic year
one is that we may streamline the program this could include potentially
removing our point system to relying fully on badging
and milestones with students as this would make it much more clear for them
in relation to the student feedback we are going to consider ways in which we
can make this program more appealing for students
in non-stem majors we are planning to do a programmatic
assessment of student learning outcomes including a longitudinal study
additionally we plan to meet with academic advisors and underserved
colleges to examine courses that may count
towards components of urep and how we can tailor the language to be
more inclusive
thanks so much for listening to our presentation if you have questions
or are interested in offering a program like this at your institution
please reach out we would love to collaborate
special thanks to undergraduate student victoria french for assistance in data
cleaning and analysis and i would also like to note that the
undergraduate student and this research was funded by an acrl
academic library impact research grant from 2019
our grant proposal was titled how libraries show
impact creating an adaptable model for tracking assessing
and celebrating student success thanks so much