thank you so much for coming for the open access
edition of the digital discussions speaker series and I'm super
excited to introduce two of my friends and colleagues from
the University of Michigan libraries. Um Emily Puckett Rogers
is a special collection of special projects librarian for the
operations research and learning and teaching units. And at
some point Emily I'm gonna ask you to decode that
, because is there a lot of units at the
University of Michigan library? She received a. B
. A. In anthropology from the University of Arkansas
and an M. S. I. From the
Masters of Science of Information from the University of Michigan
Schools School of Information. In 2010, she assists
the University of Machine Library decision makers with major initiatives
and projects including assessment and evaluation of services and collections
. Prior to this position, Emily served as the
Open educational coordinator for the University of Michigan's Medical schools
, open michigan initiative. And in this position she
focused on building a community of scholarly and academic resource
creation and sharing. Um She facilitated training, outreach
and workshop activities to inform the academic community about open
educational practices, tools, resources and legal considerations.
Emily is currently serving on the open access committee for
the University of Michigan library. Emily. Is it
a larger committee? Is it for the University of
Michigan larger is it the michigan libraries? Um Right
now it's just librarian, but we want to possibly
next year and open it up to faculty and students
and staff members as well. Oh cool, thank
you. And Sarah Barbara is also a special projects
library in for. Um but she's for the library
administration at the University of Michigan Library. She has
a B. A. From Princeton University and ecology
and evolutionary biology and a master's of Science and ecology
and evolutionary biology from the University of Michigan as well
as an M. S. I. From the
School of Information at the University of Michigan Sarah support
sardine and executive associate director of the library and the
research project management and process improvement needs. And I'm
really excited to introduce them both as a student at
um we went to school together and both Sarah and
Emily are extremely influential in my thinking about open access
and scholarly practice. So thank you both so much
for doing this for open educational, open access week
. Sorry, I'm like merging my acronyms and so
I have your slides and I'm gonna sit and turn
it over to you guys. I think I have
yourself a great thanks so much. Yeah, this
is so exciting to sort of be here with you
in this awesome room by the way. I am
so jealous industry has faced like this, it's really
, really neat. Uh Yeah so this is a
bill that I think is a round table event,
so feel free to interrupt us at anytime with questions
. Um If I don't hear you for whatever reason
, like wait while we have the camera and then
you know I can say what? So we also
have some question questions for you people throughout our presentation
. Um So so yeah, but if we hear
crickets when we post something we'll just move on.
But if we start a discussion that's ready to um
so next blood. Um I wanted to start off
with a great introduction to some open access initiatives and
programs company at the University of michigan library where Emily
and I are and uh we have a wing of
our library called Michigan publishing and they have a really
long and wonderful history of publishing, open access journals
and developing open access models. I really encourage you
to check that out if you, I have heard
heard them before. They have a couple of books
series that are open access. They have a scholarly
monograph series that's open access. And it's really quite
cool some of the initiatives they're doing. Another thing
is we sort of have a blanket creative commons license
for all of our sort of web content on our
website. Um Deep Blue is our institutional repository and
that's fairly well used actually, we have a very
long history of involvement with high trust. Um How
do you trust is a large scale collaborative repository for
digital content. Um And University of Michigan was a
family member. Um and in collaboration with indiana University
, we sort of administered the repository. So we
have some of our library staff are heavily involved with
high trust. Um Also the our administration, we're
very lucky to have very supportive administrator. So our
our previous team who just stepped down in august,
he was uh, well pardon me, he was
a provost at the University of michigan who was very
instrumental in the google book scanning project. And also
how do you trust and getting that started? Um
Well, and as he was dean of our library
, he was on the board of this digital public
library of America, and also how do you trust
our current dean James Hansen? Um We got on
the board of Euro Space Internet too, and the
Digital preservation Network. So okay, we've got really
supportive administrators there, they've been very interested in opening
up intellectual property and things like that. Um That's
been very exciting explaining place to work in that ground
. Um Also we wanted to mention chi this is
sort of a university of michigan as a whole effort
were major contributor. District high, which is an
open source learning management. It's kind of like blackboard
by open source um and it's used across the country
, so like if you sort of the lay of
the land at you and then with in terms of
our open access initiatives and projects, um so if
you go to the next blood, um now we
get into sort of uh we are starting to meet
at this time or veggie burger vegetarian. Um the
sort of motivation, we wrote this white paper with
this summer um and our boss, associate university librarian
for research at our library, Elaine Westbrook's uh email
. That's one day I was like, hey,
I'm really interested in all metrics and thinking about how
all that you're expecting with libraries. We have started
to hear a little snippets of, I don't know
if it's about all measures and libraries and she wanted
us to sort of look into it for her to
think strategically about how it fits in with our sort
of whatever is going on with our research unit at
the moment. Um So we looked into the landscape
um and and wrote this paper and the link there
as a link to the paper in our digital in
our repository institutional repository. We're still looking into our
sort of next steps of the library. We'll get
into that a little bit later on presentation. Um
But yeah, so this is so we wrote this
white paper and now we're talking to you a little
bit about it. Um So if you go to
the next slide, um I just wanted to start
off talking a little bit about what metrics he didn't
mean. I think a lot of times people say
it's all about how many times with my paper tweeted
, right? Like who's heard that? Like I
heard that looks like that's like the first thing that
comes to mind. Um, but we wanted to
really emphasize the definitions that are emerging around having an
emphasis on the open, often social web, but
really the open web um, sort of the aspects
about measurement of scholarly impact and influence and measurement at
the article or really artifact level. Right? Um
, so those are some of the critical aspects to
this definition that makes it more than just how many
times was my paper tweeted? And I would mention
that a lot of times you read uh, oh
all countries, it was first defined in the tweet
. And this is that tweet on the slide.
This is that tweet that they're talking about. I
don't actually know if that's true. Jason korean is
a PhD student at the University of north Carolina Chapel
Hill and he tweeted this and he's very involved in
all metrics and researcher on all metrics. Um But
one thing I like about this tweet is it really
gets at that article level or artifact level and that's
a big aspect of all. Net. Um So
even next slide. So I guess a little guy
in the next slide, I just wanted to step
back once more before getting back in the all metrics
. Looking at the um our University of michigan libraries
investment in aspects that are sort of related to or
tangential to or supportive of all metrics. But are
we're not doing anything tools or service lives right now
on all the others. But here are some sort
of other things that are sort of related. Um
Within the recent there's a wing called research Data services
uh unit. Um, and they are very involved
in public researchers, say developed data management plan or
promote tools for faculty to use. Um, We
have started to implement use of both orchid and data
site, which are tools to create the identifiers for
researchers and also data set. Um, we've had
vendors come by people from big share plum analytics,
which is an all metrics company, which we'll talk
about more later um, to have conversations with them
about this. We have a research guide that sort
of mentioned all the tricks. So, so we're
not, we don't have any specific electric services or
tools that we're developing at the moment, but this
was sort of the landscape as we were writing this
. And uh, okay. So if you go
to the next slide, um, I wanted to
talk a little bit about the whole landscape and really
I think a couple of things really enable all metrics
to take off. The first is around scholarly communication
in general and the changing ways of scholarly creation.
So researchers are producing artifacts in their scholarly process,
um, that are not just journal articles and books
. They may be doing that too, but they're
also producing, um, patents and uh, blog
post, the defect source code, you know,
the also figures grants and it's a range of artifacts
are coming out of scholarly research and an electric stove
. You can sort of measure impact on each of
those. Right? And we can't, we couldn't
, you know, you can't really do that before
. I suppose you can, but it's sort of
all ventures really can take into account any of those
another thing that's happening feed. Right? So scholarship
can be pretty quickly put on the open web,
right? You can researchers, not at all,
but a fair shot people are starting to, you
know, discuss ideas on their blogs or share,
you know, have conversations in these open arenas where
you can track that and you can track the impact
that's having also scale. Right? So one of
these Jason Priem who was cited in the two sides
ago about the tweet, um, he, he
really argues journal impact factor is not the scale we
should be looking at when we're talking about assessing impact
of someone's research, right? You want to get
down to that artifacts level. Um, and so
all metrics really capitalizes on these three aspects of the
open web and also the shipping stands of how scholarly
communication happening these days. Um, I want to
plant the seed of a discussion question or a question
for you all and then I'm not going to have
you answer this yet then a phenomenal talking about concerns
and then we'll get the answers. But I wanted
to ask, um, for you to think about
is what have I missed from this List of concerns
and almost two. Um, and so what kind
of discussions and conversations using libraries need to be called
in? Um, what kind of questions do we
need to be racing in these conversations? So getting
back to what the concerns that we have listed.
We see to sort of big issues that we think
batteries need to be in the room when those conversations
are having first of all, um, there is
an issue of standardization. So in what cases might
a download from an institutional repository mean the same thing
or more or less impact wise than a view of
the presentation on slide share. Right. So you
have a presentation you put in your eye, are
you put it on the slide share? Like how
are those equating or does it tweet ever mean the
same or or more than say a a traditional playstation
. And you know, the answer is it's probably
different by discipline. It's probably different by researcher by
maybe research topic open maybe over time. Um this
summer ability was a very exciting separations of all metrics
. Um, There was an organization that national um
information standards organization. Nice. So um put out
, announced a study we're gonna study proposed to develop
community based standards or recommended practices in the field of
alternative metrics. So they're really paying attention to the
energy issue. Um So that's one concern that we
think libraries that's going to be in the room in
those conversations. Another thing is something that's always been
true when you think about Beverly metrics or anything around
citation um metric values. So all measured values.
Traditional metric values, they don't necessarily impact doesn't necessarily
equate quality. Right? You think about a retract
paper that make it a lot of maybe a lot
of conversation about that, right? And so you
might get a lot of high numbers in your metric
, whatever you're measuring. But that doesn't necessarily need
that that artifact is of high quality research lies or
scholarship wise. So that's just something to be aware
of. So now I want to return to that
question and I just wonder, you know, if
you have any other sorts of ideas about conversations in
this arena where we as libraries need to be in
the room or we as universities need to be in
the room and informing that discussion. Yes, I
have two questions that came up for me. Um
11 is first is do you think there is a
way to measure quality? Because before we even think
about doing so you have to think media about possible
that's in the first place because everything has been called
about quantity. But do you really think there is
a measure quality? Take that into? Yeah.
Um So I heard your question is it summarizing it
isn't really any way to measure quality wasn't right?
Yeah. I'm not trying to see that there isn't
but just getting clear on that question for you.
Sure. Yeah. And I think that's a really
good question. I mean libraries historically are sort of
you go there for trusted information, right? Or
you go there because there's been some level of selection
right? Of the resources that are in the library
. But I don't know. I I think that
you know, Oh wait, oh no, I'm
hearing my own so I hope that was saying.
But I think aspects of peer review in an ideal
world have that measures have that element of assessment of
quality. Um but I don't know if we have
any one, a perfect way to do that right
now. What are people in the room thing?
Yeah, I'm sorry, my computer went to sleep
and so the slides will be back up in just
a minute. Um Actually, Sarah, I think
that's a really great point. And you made a
great point to about measuring quality. And it's sort
of prompted me to remember that one of the concerns
that I tend to have about libraries and all metrics
is is there? And this is a question I'm
not sure how to answer. Is there an impetus
or incentive for the library to help scholars better articulate
the value and influence of their own work through both
traditional metrics as well as alternative metrics. And I
think that the library is in a position that can
help lend a bit more value to the question of
all metrics? Uh Yeah. All right. So
one thing today, so we actually had a blank
bushman from plum analytics come talk to us as our
closing keynote speech for open access speak today. And
one of two of the things that he mentioned um
in terms of new models of uh scholarly communication is
not really dissemination. Are the concept of is the
concept of portable peer review. So you mentioned rubric
and periods for science. So those are new constructs
where um pure view is happening in a space where
you can you can take that experience with you so
that um if you have an article that you want
to publish, it doesn't get pure reviewed individually each
time from scratch every time you want. It has
to go to another publisher, another journal, for
example. And he made the point. And I
think we've also had this discussion on campus that um
, also metrics or article level metrics are not necessarily
replacing the standard values and the standard metrics, but
there's something that's going to live alongside and augment the
landscape of scholarly communication research, communication and dissemination.
But one of the things that I'm going to talk
about a little bit more. When we were looking
into this um, work over the summer, we
had some lightweight interviews with researchers and they're really thinking
about Chunking out what 11 researcher called chunking out his
research and packaging the research in different ways to different
audiences. So those audiences are going to find a
different value um, in the research. So a
scholarly audience might find a different value in the research
findings and say the public or say high school students
or something like that. And the conversation around all
metrics and article electrics are prompting some researchers to think
about that aspect of their work and that specific interaction
with different communities, some care, some of them
. Um that's that's another point. And then I
would just been um actually, if you want to
move to the next slide, this this kind of
gets at your uh question for them. Um I
think I think a lot of people would say libraries
are in a place where this can be the people
for raising the profile of these kinds of measures um
amongst the faculty. Um so Lewinsky people are in
Korea, have a paper in college and research libraries
where they say that, you know, librarians can
provide support by informing these kind of emerging conversations right
getting, getting in these spaces where we're talking about
the issues that are coming up and um but also
supporting experimentation with emerging tools. And particularly I'm going
to give you an example of that in a slider
too, but particularly around engaging with faculty and researchers
on campus um to educate them about metrics whether we're
talking traditional or all metrics um and providing that outreach
to raise that profile. So so actually and I
just have one, you know, just another special
questions. Just it's actually just for my own interest
, have any faculty come to you all asking you
about all the tricks. So so we have a
mix in the room of both faculty students and librarians
. And I know that Yeah, I raised my
question, librarians, Faculty member faculty. Have you
thought about it all and students, have you thought
about this for your own research or for following other
people's research or assessing other people's research when you're writing
a paper that they come across. Okay. But
so one of the fascinating things for me, because
part of my job titles for the digital humanities where
a lot of this discussion is sort of um taken
for granted that people are thinking about all metrics.
And I have realized that that's only true in the
library. At least my experience in talking with faculty
right now, that the idea of alternative metrics or
even not even at a metric level, but articulating
the value of digital work, that conversation is being
had, but it's not being paired with all metrics
. So I find a really interesting disconnect. It
could be useful in tenure promotion considerations, but you
have to establish the value of before you can actually
apply it, right. And that's what the concern
is that I've been hearing from junior faculty saying I'm
not sure I can make this case yet with this
, but with tenured faculty it's a little different.
Well, initially humanity has always been a promotion.
Mhm. I'll speak as a faculty member. I
think that it depends a lot on the field of
scholarship also. So I'm in history where the whole
idea of impact factor just does not, I'm not
a big issue. The last time I checked the
journal in history that had the highest impact character was
literally, but I think there there's a slightly different
issue and I'd be curious how the all metrics fits
in, which is a lot of scholarly journals still
are published by professional associations. And access to those
journals requires either a library subscription or in many cases
a personal membership. And so, you know,
an article can't be tweeted, you can tweet the
citation, but unless you have access to the library
of our personal membership, you may only be able
to get to the title or an abstract. And
so I think that we had a good discussion yesterday
, you know, about posting your own research and
so on. But you know, if I were
to go back for five years for the american american
historical review, you know, I wouldn't have access
to those articles except to the library. And so
I couldn't repost them or retweet them. So I
think that really needs to be to be factored into
consideration in terms of what's the source of the of
the scholarly work and there how does that affect the
possibility of um being reposted or otherwise kind of circulated
in ways that the all the metrics are captured.
I've had experience as a library with badly um contacted
me. He was in the International affairs and he
was actually trying to get his battle in his college
, his school, um enthusiastic about all metrics because
he's an urban planning and urban affairs and he feels
like so much of the work they do has an
impact on the public, that it's important to hear
what they have to say to recognize their comments.
And he was he brought together people from the library
from the graduate school and taking lane, who is
the one that's working on electronic faculty activity reports here
so we can think about how to incorporate those theirselves
. Yeah, I really like that example um,
of involving the public in the discussion and that's one
of the nice aspects it is tied into the previous
uh, point. You know, when you can
, when you can have these conversations out in the
open, you can really involve the public who don't
have individual memberships to these society publications or may not
have access actually to an academic library. Right.
A public library may not have access, it may
not subscribe to those kind of resources. So I
really like um that example that you just brought up
and yeah, I was you know, the serious
on this point, I would I would say that
is difficult when things are closed, right? It
really limits the discussion that you can have. And
then that limits then with yeah, the measurement of
any discussion. I think. I think that's an
important point where I'm just going to talk a little
bit later too about this is an example of not
if open access is happening by how open access is
happening in scholarship. Um and what we're seeing with
all metrics are the ability for us to start engaging
with open access and open access journals and open access
dollar scholarship and principles. Um So the materials that
are accessible now, we're seeing the use of those
materials and I think that's gonna be interesting to see
how um closed publish publishing companies um that are sort
of experimenting with open access levels but not totally shifting
over to completely over. That's not all like wiling
um like l severe and things like that. How
they start thinking about all metrics when they start seeing
what you can do with open API. S.
And open systems where you actually can get at that
article level information. Some of them I know are
opening up some of their our articles to be able
to be pulled into larger systems where you can start
to display this but not everything. So that's going
to be an interesting trends to watch in the future
I think. And there was a question or a
comment. I can see you you're wearing blue accordingly
with I'm a graduate student and I must admit that
this is this uh access week is the first of
my remote anything including digital rights and then great about
open access and so I also want to see them
in a discipline that is just as close. I'm
in philosophy where there doesn't seem to be so much
uh huh Outside because society so sort of that's open
, more open exorcists and yeah, this is obviously
a tentative assumption but my impression is that my department
doesn't really no much think about all pressures and we
have improvement attention to go. So it can also
be the nature culture and then this right here aware
or willing to much too guards do you need do
you need me to repeat that a little bit?
Um So yeah, I think that actually your point
brings up sort of similar to what tom was saying
that some disciplines have really oh been on the cutting
edge of all metrics in part because of government initiatives
. So like the National Science Foundation requires that some
data is made available. So by the nature of
having the funding you have to you have to you
know whether you want to or not but then other
disciplines and many of the humanity is not all,
but many of the humanities and social sciences tend to
be a little slower um to consider how their research
maps to one in online environment, how it maps
to an open environment and then how you begin to
assess it because it's a different kind of assessment in
the humanities, So and your background, if I
remember is computer science and philosophy, Right. Parents
. Mhm. But my masters, Yeah. Mhm
. That's very different. Oh, okay. So
uh immediate in the interest of time, we move
on to the next slide. Right? So when
when we were thinking about this, you know,
a few of them were not doing anything right now
with saying all metric company or are rolling or not
rolling out any services running right now, we're just
studying the lay of the land. So we wanted
to get um some stories from some someone who was
doing something so truly this there, so I happen
to be in Pittsburgh today, but because of the
same family, this does not have to do with
the fact that this is a micro case study about
the University of Pittsburgh, so they're totally separate.
Um But in any case um we heard through our
A. U. L. For research that the
University of Pittsburgh was experimenting with plum analytics and all
metrics company. So we spoke with Tim de Laurentiis
, who is the director of the Office of scholarly
Communication and Publishing and the head of information technology at
their university library. Um They partnered with plum to
um and by the way, I just love this
and I feel like they may have missed a marketing
opportunity because it's plum analytics and Emily sure work and
like University of Pittsburgh pictures left a pit of the
plum, I love it, I don't know that
they have used that to their advantage as much as
I think they could. Um So plump and they
did that for several reasons, it was right for
them they were interested in, they were really looking
for opportunities, put themselves in faculty spaces. They
were revamping their library liaison program. Um And they
really wanted to get a deeper understanding of scholarly communication
issues facing their particular faculty. So the time was
really right. They also um the creators of plum
out on there were people who had built a product
that they also work that they also used. That
they have um a different context and so they had
to build up trust in them. So it all
seemed to work out great for them and they haven't
lessons learned. Um But the way this worked was
um They identified some charging faculty um 32 researchers across
nine schools 18 departments. Um and they asked for
current TV um which you know if they didn't get
current C. D. S. You know if
they're missing some recent stuff that wasn't going to end
up getting counted. So they asked for currency.
They had students enter data from the seat into their
um institutional repository and then the plum tool wouldn't mind
that. And then got and search the web for
metrics about each of those artifacts. If you go
to the next slide uh you can kind of see
um some of the a little bit of the indicates
that the researchers so you'll notice the top bubble is
an artifact summary. So uh first of all the
plumbing tool you know allows a whole array of different
kinds of artifacts. These include papers, book chapters
, books but also a blog post, grants,
figures, datasets, clinical trials, source code.
A dissertation Work here More More Than one Reservation.
Levitt letters, patent these sorts of things. So
um you can have any number of different kinds of
artifacts and be mind across the web. Um Then
they have a whole lot of metrics on usage.
And they group these metrics different countries. So they
have metrics that usage which are things like um views
, clicks, downloads. So anything anything viewed or
downloaded from their institutional repository. Um There any any
figure views on fixed share or text you got plus
or holdings World Cat. Um even the kind of
thing. Um You have something called captures which are
things like um favorites. And how many times you
your artifacts has been put into a mental a group
or a site you like group the number of readers
on Mandalay for a particular artifacts. How many actors
you have get hub mentioned are things like how many
links does your artifacts have from Wikipedia? Or how
many sort of uh blog accounts do you have on
research flogging or? Um I don't know those sort
of things. Uh Social media is again that think
that everybody thinks of, how many times was your
paper tweeted or shared on facebook or even a plus
one on the cool plus um or or what score
doesn't have I read it. And then citation.
They still they have you know some traditional metrics as
well. So they look at cross raft Microsoft Academic
Research, Pub Med scopus and then also the U
. S. Patent Office. And let's get citations
. So they're capturing that information as well. Um
Now for all of this data they have they display
it in sort of a table form and you can
tweak what whether you're looking at a particular artifact or
a set of artifacts or a set of types of
metrics and take any good export all of that Cst
files what you can manipulate however you need to um
and assessed however you need to. But they also
display this information visually and something that's a little bit
overwhelming If you have not seen this kind of interface
before. Um, something called the numbers and it's
on the next slide and I'm just going to describe
it before me go to the next slide. Um
you'll see these okay or are you okay? So
concentric circles and in the very middle is the name
and I write it out today and this is actually
a similar centers because the next concentric circle you see
a whole uh, like a doughnut. Um and
it says you probably can't see a teeny tiny this
is presentation now in a more comprehensive circle. This
is like a personal researchers output. So it will
have that first concentric circles, presentations and articles and
book chapters and it will be different places of that
. And I just clicked on presentation so you can
see the impact of just the presentation. The next
most concentric circle is are the actual presentation. So
I clicked on the thing that I have boxed in
green and a pop up window comes up and it's
that pop up window that's up to the upper left
and that's sort of a summary of that actual presentation
. Plus the big metrics today that they found on
it. Like the numbers fill you with that presentation
. Um I don't really know how useful this centers
interface is but one thing I'm trying fascinating about this
and this is what I wanted to even raise this
with plums shows you two kinds of numbers. This
is the default numbers book, they have another one
you can collect default or you can click Impact and
it's an impact numbers. And what that does is
it wakes the slice of the circle based on its
impact. Which gets back to our issue about standardization
of quality, right? Because it's saying uh this
this presentation was viewed 294 times on flight share,
download it twice. Favorited once and was tweeted ones
. Well, okay, so it's downloaded twice and
tweeted one. The weight of that circle of that
particular paper or presentation or whatever would change if it
was tweeted choice and downloaded one based on some algorithms
, they've decided for how to wait eight different metrics
. So it's really fascinating. And that's something that
both 10 million eighties that pick and Emily and I
are agree there's like an issue there, right?
Like what is that algorithm? And it darn well
, might very uh the weight of each of those
measures might vary across disciplines, across individual researchers or
whatever. So that's that's fascinating. Yeah. You're
gonna jump in, you speak up. I can't
quite hear you on my end. Yeah, I
wanted to know if there's a way to yeah,
find out of what the different disciplines Do because my
work actually could spend three yeah distant points. And
I wondering if there's a way to get that information
about, About what each Yeah. Are you asking
about that impact algorithm? So I think so.
I'm not really sure. I think plunges uses uh
huh. The same algorithm across all of the research
is and what we're saying is there like people want
to look at that and how they're waiting different metrics
by some people might feel like, well for me
, twitter is where all of my readers are or
for me my readers are going to go right away
with my share or or mentally all the people that
are important to me or meddling. So we're different
things are done in different, different disciplines. What's
what they use into To pressure impact 20s? They
use different tools more or less than others. And
I would like to maybe get access to that information
because as I said, I am, my work
spends multiple disciplines and I I sort of want to
know better what you can expect To go to a
different one. That what Right, So we're not
there yet. I don't know that we're there yet
at all. And that's actually what one thing I
think that institutions and libraries can do if they're interested
in engaging in this a field in all metrics in
general, is to do more research about that very
same thing. So that because that's a that's a
big gap right now. And even when I was
looking at a lot of them tools that are coming
out are treating treating are generally the same. You
know, they're kind of doing a baseline assumption of
where people share and how they share. And yet
we know from, from disciplines to individual researchers that
their staff strategies are very context dependent. So,
so yeah, I think that's something that planalytics knows
that they don't know that either. Um and that's
that's the big next step I think for the analysts
to progress as a field. Yeah. And so
now I'm just gonna close my section, say with
that, I'm gonna pass out to Emily he's gonna
start to talk a little bit more about researchers on
on um university of michigan campus. All right,
past the doctor. But of course I'll jump in
again in the way the discussion if there, you
know, when there is a so differently. And
so should I, should I in the meantime,
as a graduate student kind of approach my library to
see what they can do for the for me about
if they are if they are interested or fighting out
. Yeah, I'm actually your librarian. So I'm
pleased on core philosophy. And so you're absolutely welcome
to come talk with me when it comes to waiting
all metrics and sort of assigning a value. It's
very much what Emily and Sarah are saying it's a
very new field and um, it's hard to wait
like the discipline of philosophy. I mean, even
philosophy because there's everything from like almost computational logic to
something. You know, it's a diverse field within
itself. It would be hard even within one discipline
to wait. The value of a particular all metric
. And so that's what sort of, well,
let me let me talk but then we can talk
for sure. Absolutely. Yeah. Yeah. I'm
gonna Yeah, yeah, so, um, I'm
going to spend a little bit of time and please
do feel free to continue to interrupt or if you
have questions. But talking about when we um started
a research over the summer, we did a little
bit of lightweight uh, interviews with some researchers in
these two units and we recognize that this is only
a very small start and this is not conclusive or
general Izabal, but I just wanted to share kind
of what what we found from our conversations with these
researchers and then what we've decided to do is next
steps to address this gap specifically. So, um
, in general, when we talked to researchers within
the physics department and within the, what we call
a country, the University of Michigan Transportation Research Institute
, we found that um, the faculty who are
engaging with all metrics or what we would consider early
adopters. They are faculty who generally have some sort
of web presence beyond maybe their standard um, profile
in the university directory. They might have their own
web page, their own profile on google scholar.
Um there possibly using google analytics, things like that
. Um, so they have a sense of the
um for them internally, um, the value of
having a web presence and maintaining my web presence.
So that's something to consider. Um is that not
all faculty um use the web like this? Um
, but there they are again targeting, they they're
specifically wanting to target their research on a package the
research to target specific audiences for specific intentions not coming
to that in a little bit. Um, but
they're also not using the same words that were using
it at least as librarians or in um situations where
we're talking about scholarly communication. So they're not saying
the words open access, they're not saying the words
all metrics. Um they're thinking in terms of citation
statistics or that journal that gives me a lot of
hits even though that might be an open access journal
. Um So that's something to really consider is that
we're using myself as a library and I'm using a
little bit of a different language to discuss this stuff
and we need to um I understand that and potentially
reframe the conversation um in terms that are relevant to
researchers and scholars that we're engaging with. So next
five. So one of the conversations we had the
researcher, um he was interesting. He was he
said that the research should speak for itself specifically.
He was against blogging, but yet he had um
his own mailing list for distribution of his research.
So he was disseminated little bits and pieces updates of
his research to people who had asked to be kept
in the loop over time. And then he also
um would distribute his research in terms of small white
papers and things like that to media outlets. So
for example, our university newspaper, the record picks
up a lot of stuff and turn other media outlets
pick up a lot of stuff. So he's being
very specific about his dissemination. So he started using
uh metric dot com as a way to get a
holistic sense of how people are engaging with this research
because of these strategies. Um but he doesn't necessarily
see the value. Yet he's experimenting. So he's
not saying that this is necessarily a good or a
bad thing. Um And again, this was one
of the people that we talked to that didn't know
the term open access, didn't know the term institutional
repository, but yet he publishes it open access journals
and prefers them. Um When we got speaking about
this. And he also publishes his work in our
institutional repository that we call Deep Blue because he sees
the value in making that accessible to people for a
stable U. R. L. It's easier for
him um to put something into an institutional repository that
puts things into a Kia format and he can just
send out the D. O. S. And
things like that. Um But he doesn't doesn't really
know, he doesn't know right? So that was
kind of a another conversations like this is kind of
typical of some of the conversations we're having um informally
with other people. And the next slide is another
person's perspective. Um This person was a younger um
faculty member who was starting to see this change in
the way that his own students on his own researchers
were starting to look for information. So he is
noticing that people are going to google first even though
they know how to search. Um the database is
they know how to search their own scholarly publications in
their own journals, but their first steps still is
considerable and finding what they can find that's completely open
access and free to download and use without a log
in even. Um so so there's that ease of
access, but starting to impact people's research habits and
search habits in general. Um again this is somebody
who is using the web very actively to promote their
own work um or to engage in different communities for
this, but they're still seeing um the they're trusting
the value of traditional metrics for impact. So um
the H index and the general impact factor, Those
are the things that they still look forward to illustrate
for them the value. But they are starting to
experiment and supplement with all metrics. And again they
this person was specifically interested in um what when trying
to phrase is matching the perception of their work to
this in the nation venues. So that's the person
who wanted to chunk the research into different formats for
different people. So how can you package it in
different ways? Um And again that's, he mentioned
that some scientists that he has worked as in the
past, I feel like when you disseminate to the
public you're dumbing down your research. But he felt
very strongly that there was a value to disseminate his
research to um to the public or parts of his
research to the public. And this person was actively
um putting the research out to end their data out
as well too, provide opportunities for people to look
at it and say, oh, that's a cool
project, I might want to work with you.
So to showcase the cool stuff that this person is
doing, um so that potentially there are graduate students
who might come to the program to partner with this
researcher to do work or um in this case,
this is a group that has to attract a lot
of outside funding sources. So this was a way
to potentially engage um grant, um, foundations or
federal agencies in um looking at the type of research
they're doing so that they can get more money for
future work. So, so that that is we
we discussed this, but that is a question um
that I would be interested in learning from you,
how, how y'all are engaging in this kind of
stuff yourself, or if you're engaging in at all
if these kinds of tools are useful, do you
think they're useful to you and your own discipline?
I think that conversation is still beginning here attack.
I think there's definitely interest, but we're still beginning
to kind of map out the the entire research environment
. That that's something that we haven't yet. We've
started discussions about it. We have this really interesting
Center for Digital Research and Scholarship that's working very hard
at kind of mapping, who's working where and in
what ways with digital research. And it's still,
I think we are beginning of our second year,
is that correct? I think so. Yeah.
Well, no, just cedars work in general sort
of thinking through what services and tools digital researchers need
. Um but cedars is a year just a year
old, so we're still at that initial stage.
Um Mhm. Yeah, I mean I think that's
that's what we're finding is like even the people that
we would consider early adopters are not considering themselves that
and they're not thinking of best in terms of shifting
the way that they think about their research and their
dissemination fundamentally. It's just a new set of web
tools that they're experimenting with right now. So um
I think you can go to the next slide if
you want. Okay. Um so one thing I
just wanted to point out is that I see this
, but I think other people see this as the
way um of illustrating what we can actually start doing
with open access. I mentioned this before. Um
but it's not a matter of if open access is
going to happen anymore. I think it's a matter
of how it is already happening and this is important
to consider in terms of publication models as well as
the tools that people are starting to build on top
of these open systems to engage in scholarship or make
visible the work and scholarship in different ways. And
that's where all metrics is coming in. So it's
kind of the things that you can do that we
can start doing now that open access is becoming slightly
more um stable in certain places. I mean if
you look at the world of scholarship, this is
definitely not. We haven't shifted to the full adoption
rate yet. But um, I think we have
enough open access opportunities within, with institutional repositories with
journals, certain generals and certain disciplines that we can
start seeing these interesting things happening with the tools like
all metrics. Um, so I think that that's
an interesting next phase of open access, even though
that we're not still is not as we haven't reached
that saturation point just yet, but I would like
to stress that all of this is built on ideas
of transparency, participation and collaboration. Um And this
is kind of a fundamental perspective with a lot of
open movements in general, with open software, with
open education with open access. Um and now with
, you know, thinking about pulling in um the
different ways that people are engaging with your work or
examining your work or sharing your work. Um and
so that's that's just something that I think will be
really interesting in the next couple of years to see
this next step of what we can start really doing
the interesting things and the fun things you can start
doing now that we have open access, um some
some resources to play with. So I just wanted
to point that out and then I'm thinking, move
to the next slide. Um This is what we're
actually looking to do at the university of michigan,
so kind of what we've been discussing um throughout this
whole conversations that we need to learn more about our
own community here at the University of michigan. Um
And I know y'all are, I think you're all
about of time, but we're gonna be having conversations
um and we're gonna be developing case studies with several
different groups um with both within the library and without
the library. So potentially looking at um what does
that metric seems to help finance is what does it
mean to faculty development, What does it mean to
our collection? Um And these are some of the
questions that we might be asking about that. So
that's something that we intend to do in the next
several months to set those up based on interests based
on people want to um in the library want to
take up those research questions that you against the next
guy. We're also looking at potentially building some infrastructure
to support these things. Um, we don't necessarily
, we have, we have a long history of
building things ourselves here at michigan. So this is
kind of falls in line with that tradition. Um
we might engage and we probably will engage in some
vendor solutions, but I think one thing we want
to really look at um for ourselves is how we
can start capturing some of those nontraditional assets that are
being developed by our own faculty members and our own
students and our own staff members even, that are
part of the solidly process that have not traditionally been
captured and thought about in um the scholarly world,
so not just the outcome of the product of research
, but the process of research itself to. Um
so that's where our research data services group is working
on the data side to to really think about how
we're capturing and storing and making available to re use
different types of data sets. Um for example,
and then these are two questions that we might ask
when we think about potentially building onto the infrastructure we
have are augmenting the infrastructure we have, like our
institutional repositories for us, what do we want to
focus on that would be our multi metric. Um
and then it's also a chance for us to take
a pause and kind of imagine what we think the
future status scholarly communication and research should be and how
we at the university michigan library should engage with that
specifically. So kind of look at that big picture
kind of guy and um start building something toward that
vision that we have. So um I think the
next slide is the last slide and these are just
some challenges we know we've got coming up. I
think we'll just open it up to compensation for anybody
who wants to stick around. I see that you
guys are raising challenges and concerns and I guess I'm
one of those people who have used like google metrics
for a scholar and a U. Analytics but not
really thinking about them in terms of all metrics.
And one thing that I'm kind of observing for your
presentation is that the focus, the ways in which
these kinds of things are determined seem to be oftentimes
based on how things are accessed and how often versus
how things are actually used. And so that is
kind of a concern for me as a faculty member
, especially if these kinds of tools are being used
to uh determine the value of persons work um for
things like tenure promotion. And so I wonder are
there any steps being taken toward measuring the use of
the particular artifacts as opposed to how much glasses?
Because you know, I mean if we download them
, we could just immediately delete it because we find
it not useful. So I think I missed the
last little bit of that, but I think that's
a really, really excellent question. And again,
another reason why librarians and faculty members need to be
in the room of these discussions. Um I will
say I do think that some of the um metrics
that you know, all metrics cover um do point
to you. Um you know, I would say
a tweet is not necessarily using it. Like I
could retweet or just tweeted out about people, but
I haven't really read, but I kind of know
about the top. It is a topic that's not
used, that sort of like maybe shouting out about
it. Um, I think that when you look
at downloads that gets a little bit more to it
, but you don't know if I'm a PhD student
might download like 100 papers one day and then step
through 50 of them or all 100. But um
, it's, you know, I might not actually
reading that I download. Um I wonder if more
in the citation reference management for some of those metrics
might get a little bit at it, but I
guess it's a sort of an unsatisfying answer. But
I think ultimately you're right like this is something that
is definitely a challenge that something we need to address
. Because it's not necessarily clear. We're not,
we don't really have something that's like this is a
metric that says how much of your artifact was,
you know, truly consumed by a reader or a
like another scholar or a member of the public,
whatever, you don't really have that magic metric.
Yeah, I I agree with you. I think
this is a big sticking point right now in the
world about metrics um in the world, I think
actually a metrics in general. Um and that's something
in with my head on a former Open Education coordinator
that we worked really hard to address and deal with
this factor of the use of materials and how you
can make visible the chain of innovation that can happen
or the chain of, you know, um adap
adaptations that can happen um by making your content available
to be reused. And a lot of this has
to do with packaging material and then setting up systems
that allow you to to kind of track that chain
of um that the downstream use the chain and chain
abuse. Um Further one of the things that and
so that's that's something we're thinking about with with our
own institutional repositories, right? Is like how can
we um put put stuff in there so that is
easily reusable and then how can we figure out ways
to encourage the the um backwards kind of citation of
that, say, or the attribution of that,
I should say. Um uh when you re using
works for example, example, you know, on
this slide, I reuse this image that would license
so that I was able to legally reuse it and
I give attribution to the person and I told you
worry about it, but I didn't necessarily feel that
person like I can leave a comment on their site
and that's what we sometimes you um but that that
is a sticking point. Um and I think that's
something that a lot of, I think that might
be another one of those kind of next down the
road of trying to figure that out. And I
think that's also part of a culture of shifting the
way we think about production consumption in general in an
academic environment. So that's a habit issue, as
much as an infrastructure issue to. Um So we
need to figure out some tools to make it easy
for people to do that, but we also um
need to have conversations with people out why it's important
to think about that too. And one of the
things that I think that for example plumbing Alexis thinking
of is trying to this gets a little bit I
used is looking at still Alive and if your book
or your article is put into a syllabus, so
what does that mean? What does that look like
? Um So that's another, it's not kind of
another circulation statistic, but it's something interesting. I
think that's interesting part of that conversation. Part of
the thing that I mean what you just said,
I think brings up for me is the issue of
how much technological literacy is going to affect people's ability
to have their work behind impact. You think that
an issue or not so much? Can you rephrase
that a little bit? Sure. Well, I
guess I was just wondering if we are teaching or
you guys are teaching people how to kind of package
their work so that it can be transportable. That's
what you were saying, right? You're standing.
So to my understanding, can the question is sort
of like this techno technological skill bias, a particular
scholar in favor of having a higher measure of influence
through different all metrics? Yeah, so one of
the things that when you head to their long pilot
, um, they surveyed their faculty afterwards and they
noticed the faculty of mixed reviews around the value of
the pilot project, but it tends to fall um
, sort of along generational lines, which you might
suspect correlates with the number of different aspects. Right
? So, um, more early career scholars were
sort of more into the all metrics pilot than later
career scholars and there's a number of reasons for that
surely will generate that. But maybe one also has
to do with like the early career scholars just are
out there on the over web and are more kind
of uh excited about these discussions in these more open
areas. I don't know if that at all yet
, So there's that factor, but I I kind
of disagree with that. I don't think that we
necessarily need to make everybody super coefficient. Um and
people have different reasons for wanting to have a lot
of presents and it is becoming more of a normal
, I know that um we're at the University of
michigan teaching students a lot more about the importance of
having, maintaining a presence, are being cautious about
her thoughtful about it. Um but I do think
that's one of the things that libraries potentially can do
and institutions can do is figure out ways to make
it easier for people to engage in this stuff without
having to um pay a lot of attention to this
. So because we're seeing people that are playing around
with these tools are the ones that already have the
web page that they maintain actively or at least in
profile, that they maintain actively and they're very thoughtful
and they put a lot of effort and attention into
that already. So I think um that's something that
that this can potentially bring, we know that there's
less Matthew factory, like the rich get richer kind
of thing, We know there's this kind of closed
environment right now, people that are using all metrics
and that are engaging in this space and we need
to figure out how to level that playing field a
little bit more without totally requiring that everybody become,
you know, then everybody gets a twitter profile or
whatever, whatever it is. Does that answer your
question? Really? But we're thinking about that too
. It's good to know that you're working on it
. Yeah. Has there been a conversation about opening
access to the pieces that make an artifact and what
I consider an artifact on your screen and also access
to the process of making purified. So right,
in other words, if you have a paper for
instance and we talked about open access papers, a
papers is a combination of many different points, as
you said, datasets or conversations or specific research access
. But very often one thing I'm curious about is
what is the process, What's the assembly line that
happened to get to that point where you have a
paper we spent disseminates into these different ways of talking
to twitter or facebook, whatever it might be,
but there's a process that happens before that. And
I'm curious to know if there's a way to sort
of understand this causes any that's a place I think
some sort of cross disciplinary where it starts to happen
because you want to sort of take a part of
that and say, I want to get into this
part of the assembly chain and collaborating on this part
of the process to address my issue in a completely
different field. That question makes sense. Yeah.
And I think that that's what we want to start
thinking about. If we're going to build any anything
into our existing infrastructures of our institutional repositories or um
what we can capture, because we we want to
start looking at some of those nontraditional assets that are
part of the process of research and scholarship. Um
, we there's the question of the level of granularity
um, around this and and the practices around that
, but I think that's something that this whole our
university is starting to shift toward in general, even
less than the education is. Um starting to share
some of the process of learning in the classroom processes
, starting to share um those like drafts of student
work or drafts of certain pieces of research, um
sharing the data sets that you built at a certain
phase in your research or sharing, you know,
if you're collecting um images or if you're collecting sound
bites or things like that and storing them. Um
But one of the things that we I think we
need to figure out is how to how to mark
those as pieces of the process um and how to
link those then up with the final private products so
that you see that. Um So again on the
flip side of the re use is the pro pro
to use, I don't know. Um And like
the re use of those kind of proto materials or
this process materials um That's that's something actually um I'm
not trying to be a pitch for a vendor,
but some of the stuff that is on being thrown
up on a fake share and get hub. Those
are two places where people are starting to put drafts
of things and um, wanting to figure out how
to version of them or fork them. That's a
term that comes from open source software. And those
are some really interesting, um, emerging practices of
people who are starting to deal with that right now
to put their scholarship or the process of the scholarship
up on the hope of lab, um, and
make it available for comments. And most of the
time it's people within their discipline were commenting or engaging
with that, but it is, but it's done
in a completely open setting where like I can see
it even if I'm not a physicist or whatever.
Um, so, so some of those conversations that
are used to be dark and dark social webs,
like on listeners and things like that are starting to
the house and open spaces where you see the conversation
attached to the artifact too. So those are two
places that are playing around with that and they're probably
more um, that we kind of should just start
uncovering and looking at that are pretty interesting. Yeah
, Emily I'd add to that the Git hub and
picture I think are the two most prominent. But
then there's some really interesting work in weird spaces like
tumbler where I'm losing the historians name. He is
working on, he's an art historian, so he
is posting on tumbler an artifact and his analysis.
So it's like a piece every couple of days and
that's a really neat space. And some of the
dissertation blogs that are happening. Um, and in
the sciences there's like open science notebook, um,
sort of movement, but in the humanities are several
different like, Wordpress type blogs that are documenting the
process of writing a dissertation or something, which are
harder I think, than fiction or get hub because
there's limited track ability for those kinds of things.
But they're really interesting, odd examples, you know
, kind of growing up in addition to the more
sort of stable versions um vendor. Yeah, I
would, yeah, those are really great examples.
I would also say that reminds me that sort of
connects that concept of say we're working through your dissertation
on a blog or putting up art artifacts on tumbler
and sort of doing some analysis out there on tumbler
reminds me that there's sort of an effort going on
a little bit of a movement and say, just
in the undergrad world of helping undergrads get to get
pulled together sort of e corp folios, right?
And starting to document all of the work that they
have this time in the underground education as well,
where they're starting to get in the mindset, or
at least I don't know if this is more universal
that I know. At the University of Michigan Library
, we have an initiative to help train our undergrad
uh huh, to build their build successful e portfolios
that as they're going through their underground education and they're
building up skills and doing projects their their aggregated all
in one place and getting this sort of literacy um
around that. And and it strikes me that there's
a connection there that could be made uh the undergrad
level, right? You think about like a faculty
member of the public, however, a distributor going
through that, but it's been it could start even
earlier, right? Yeah. It already has started
, I think. And this is where there's a
lot of pockets of, um, different little communities
of practice at all stages that are formal and informal
, researchers are formal and informal scholars that are doing
really interesting things right now on the web. So
it's also just kind of uh, finding those and
looking at those models. And I'm not looking a
little bit outside of maybe your discipline to you to
find some of that stuff. So yeah, I
totally agree. I think it may it may be
important if you're looking at maybe the workflow, the
process of how some of these articles or research gets
published, saying, for instance, and institutional repository
, that the library has taken a very active role
in that entire workflow. And especially this department are
the Center for Digital Resource Research and Scholarship has a
long history of taking taking an active role in getting
someone's research and making it available online ads, for
instance, and for for graduate students. And then
for professors, E K G B itself was of
course part of DR Fox Research in Computer science.
So I myself worked with lots of computer science students
over the years into mm and helping them to understand
because they're building citation, for instance, maybe a
tool that will take citations and find all metrics in
itself to show other people how often something has been
sighted. Well, they're very interested in how to
get that original products so that they can mind and
link those citations together. And now we've all seen
these cyber like various tools that are available. So
that process, the library has been very active.
So I think you can just contact my director,
Bill Macmillan. Yeah, it's Gilly has been very
actively involved in that for years. That workflow is
making things available on making research about. I think
a big part of the challenge of this is that
we want to wait until this stuff is stable and
it's not gonna be stable for quite a while,
I think. And so that's not the attitude I'm
trying to take and I think I'm trying to think
we're both there and I'm trying to foster and our
library is just to start playing around with this a
little bit. If there are some low risk things
that we can do um to build some pilots out
at the same time that were um trying to better
understand and engage with our community. Um Specifically then
I think that that will be valuable and figuring out
how to share that with other libraries or other institutions
, I think that that will be valuable to the
whole process. Um So so that's a it's a
big its desire to kind of hesitate, hesitate.
Um Wait a little bit. But I think that
actually now is a good time for all of us
to start just playing around with us a little bit
more if we have um if we can do something
that's lightweight enough, that's not a huge burden on
all of us um and we can do some really
interesting things to push this along and to really figure
out what's working and what's not working and ask some
of those hard questions and maybe get a couple of
answers to those hard questions. Mhm. Well thank
you so much, Emily and Sarah, I really
appreciate you taking time to chat with us. And
it's interesting because it's at least for me there's all
of these um overlaps and metrics and open access and
sort of data curation, data management. So I'm
really fascinated to see like what those next steps will
be. You know, thank you so much for
joining us. Thank you.