Bill Hart Davidson is an associate professor of rhetoric and
writing at Michigan State University, and he's the co
director of the Writing and Digital Environments, or Wide
Research Center at Matrix, a digital humanities and social
sciences research center. He teaches courses in technical communication
, interaction, design and research methods. He recently
can pleaded a term as president of the Association of
Teachers of Technical Writing, and in January he will
begin an appointment as associate dean of graduate studies in
the College of Arts and Sciences at Michigan State.
Bill is added at a book along with Jim Rodolfo
of the University of Kentucky called Rhetoric in the Digital
Humanities Rhetoric and Or In Rhetoric and the Digital Humanities
, forthcoming from the University of Chicago Press and available
early next year. Bills writing has also appeared in
journals such as Technical Communication, Technical Communication Quarterly,
The Journal of Business and Technical Communication, Computers and
Composition, Cairo's and the Journal of Community Informatics,
and in lots and lots of edited collections as well
through his work at wide. He also produces writing
software and is a co founder of a spinoff venture
, Drawbridge. Drawbridges, a public private partnership created
to make Eli Review a software service that supports writing
instruction. And on a more personal note, his
Twitter bio describes him as father, academic, juggler
, cyclist cook, and he describes his philosophy of
life in a single word. Give. And we
here at Virginia Tech over the past two days have
taken him up on that philosophy of life with our
philosophy, which is paid. We have put Bill
through his paces with lots of meetings and lots of
meals, and he has performed with good spirits and
admirably all along the way. Yesterday I emailed my
old faculty mentor from my PhD days and told him
that Bill Hart Davidson was on campus and he responded
with one line prince of a man, one of
my favorite people in the field. And that certainly
has been my experience as well. Please join me
in welcoming Bill Heart. Dave, I want to
say thank you to Quinn. That was embarrassingly nice
. Uh, introduction and thank you all for giving
me part of your afternoon sharing with with me and
, um for coming to talk with me. Um
, I'm going to speak a little bit about a
research project in wide that we have been doing now
for about a year, but which is really still
quite experimental. And so it's exciting to have moments
like this one where we have assembled groups from all
over the campus. I understand we have some folks
from computer science and folks from the business school and
folks from all over the humanities, and we're thrilled
to have an opportunity to talk. And I'm hoping
that it is a chance for you to give me
some feedback and push back a little bit on some
of the things that we're saying. Um, my
topic. I want to start with the title because
we use where I use a word in that title
. That is a rhetorical term of art. The
word topic. Um, it has a special meaning
in, um in rhetorical inquiry that comes from the
Greek word to poi from which it, uh,
inherits. And it isn't quite the same as we
think about when we talk about using the word topic
in everyday speech. And so I'll start there with
some things that you might have heard before I heard
this on the plane. Ladies and gentlemen, the
captain has turned on the fasten seat belt sign.
If you haven't already done so, please start your
carry on luggage underneath the seat in front of you
and an overhead bin. Um, so toe poi
show up not exactly as topics but ways of framing
that can act as a constraint on the on the
speaker but also, um, portend the expectations of
the listener or reader. And so they form a
kind of mini version of a social contract, Um
, and as a result they tend to show up
a lot. And this This is why when you
hear the Greek translation the most common Greek translation of
the word toe boy, it's commonplace. And so
keep that those two words in mind. And that's
a really helpful shorthand. It's a commonplace that we
return to when a situation seems to call for it
. And so if you are sitting on an airplane
and you're about to push back from the jetway,
it not only is commonplace, but there's this commonplace
that occurs in a string of them occur. The
interesting thing about that is that the exact syntax can
vary. Um, the words could be a little
bit different I was on a Southwest flight once.
Um, I think I was going to m l
a. Uh, So it was a writer on
Christmas time, and the crew did the whole thing
in rhyme, Um, uh, to the to
the same rhythm as twas the night before Christmas.
Right? And it was quite entertaining, and it
worked out perfectly fine, because we had we had
the common places in our head. If we were
going to be a little more specific, we could
start describing these. And you can see I have
a shot here from the delta in flight safety video
. Um, that shows that we can kind of
see these commonplaces change. Um, format or change
in this case, uh, mode. We're seeing
the visual correspondence. Um, in technical communication,
we might call these rhetorical moves. And that's the
word I'm going to start. I'm going to use
to stand in for this more fancy pants. Greek
word. Um, And just to step back,
I'll tell you that our computational project is to try
to teach the computer how to find rhetorical moves and
turns out that humans are pretty good at it.
Amazingly good at it. and computers. It's a
hard, hard problem. We had a chance to
talk with group from the business school that Quinn is
part of, and and we had a great talk
about this this morning, and it really gives you
an immense respect for the Hermeneutic ability of a human
being to tease out meaning from a text or even
from a visual artifact. But if I'm talking about
again that toe boy or the topics associated with,
um uh, the the message that you here at
the beginning of a flight, we might call them
part of a common set of user assistance. So
now I'm looking at Carlos, um, uh,
moves that we teach students in a technical communication class
. So one when we're guiding action, we really
make three basic moves that are really important, and
they cut across lots of different situations and so they
can vary. But you want to stage the action
to be performed. You want to tell people about
the prerequisites you want to foreshadow the success conditions.
Then you want to coach them through the steps that
they have to take, and you offer clarifying detail
. Here's a Here's a little one that always impresses
people. UH, we tech com People recognize two
kinds of clarifying detail, and they often show up
his images. One shows you how to do the
step, like it shows you the knob and a
little arrow to turn it, and the other one
shows you what the knob should look like. If
you've done it correctly, that's the process versus outcome
Visual. Often, when you're cursing those Ikea things
, it's because you have one and not the other
, right. Um, the one that you need
. And then we have alerting behaviors, right?
Warning. If you go too far, you'll punch
a hole or be careful you don't want to hurt
yourself. Um, alerting moves now as varied as
writing to guide action is, and you can think
about this manifesting and lots of different specific genres.
Say, baking a cake on the back of a
box of brownies. You'll see these moves recur,
staging, coaching and alerting. And after even with
just this amount of education that I've given you today
, if I pass around a bunch of Duncan Hines
boxes in a highlighter, you could all find them
. You could say, Here's the staging moves.
And here's the coaching moves. And they would say
, Well, how did you know that this picture
of two eggs and a picture of a glass of
water is a staging move right? How do we
get the computer to recognize that? Well, that's
a list of prerequisites, right? It's a visual
, and so these moves can show up in lots
of different ways. Here's the seat back card that
none of you ever look at even though they tell
you to. So if you look at the language
that I told you before about stowing your things in
the overhead bin, visually we see the process and
outcome. Visuals are indicated by arrows, and we
see the right way and the wrong way. Your
tray table must be in the upright and locked position
, and your seat back must be right so we
can reproduce the text and we can see the moves
happening. And we can even recognize them once we
are sensitized to them. So this is really what
we're trying to to think about ways to see if
we could teach the computer to do that, and
I'll talk about why I'll come back and talk about
why that's a useful thing. So this is a
research group in our writing and Digital Environments Research Center
at Michigan State that consists of not just me,
but a close colleague of mine has worked on just
about everything you'll see here today is our post doc
researcher named Ryan O Miso, who earned his PhD
from Ohio State. And it was with us.
We're fortunate for two years, and also my colleagues
, Dean Ray Berger, who is the director of
Matrix. Jeff Grable, Um, who is the
chair of my department and Liza Pots. Um,
so all of the work that I'm going to talk
about here, they get some credit for. I
get all the blame they get all they get to
share the credit. Broadly speaking, the piece that
I'm going to talk about today metaphorically involves our attempts
to create some testing methods, and here the metaphor
I'll invoke is a little more specific. We're thinking
of them of them as essays, Um, which
are certain kind of chemical test to determine what compounds
are in a A particular thing. In our case
we're using the compound is a complex rhetorical text or
a group of texts and the rules of grammar and
syntax provide the markers that we're seeing, what we
need to see. And they also provide these little
hooks upon which we can build subsequent parts of the
analysis. If we're sticking with that s a language
. They are the proteins or other sites that we
can bind to in order to amplify some of the
signals and see if what we're seeing is in there
, and we could dampen down the noise. Um
, and that's a really I know I got all
sciences there, Um, but that's part of what
we're trying to reach for our ways to describe a
kind of analytic procedure that we really haven't experimented with
very much So please at the end. And and
, uh, when as these things occur to you
, jot them down and push back. And I'm
always looking for better ways to describe that. But
as I mentioned, humans are really good at this
end. Computers? Not so. It's not so
much. We had a few specific questions that we've
been working on all year, and two in,
uh, boldface. Here are are ones that I'll
try to address and show you a little bit of
the work that we've done. Um, we started
with a very pragmatic thing. That is the top
question. And that is we had a big data
set that was human coded, um, as part
of a qualitative study that Jeff had led, Um
, that I'll talk about in just a minute.
And it was arduous and painful, and so they
were hoping that we could find some ways to make
it a little easier on them. Um, along
the way, we evolved some other questions that messed
with our group, which isn't, which was slight
words were slightly different, but which this dataset gave
us an occasion to explore. And the one in
the first boldface one is one that we address this
morning a little bit with the research group that you
have here on campus that is looking at postings in
the Motley Fool financial Services website Forum. And they're
doing some analytic procedures on that. And it is
what is the best way to represent a text as
a computational object. If we're going to turn a
text into something you can do math with or on
, or do you use mathematical techniques to transform what
are the best ways to do it? Um,
and of course it depends. But our answer for
today is a graph. And I'll, um,
if you walk away with one strange idea about what
this guy talked about today, it's you talk kind
of about how we can see text as graphs.
Um, we then needed to find some more specific
ways to break down that graph, even if we
could get the transformation to happen and understand what was
going on. But ultimately, the boldface question at
the bottom is what we were after. Can we
identify rhetorical moves? That character characterized the way discussions
carried out in different genres develop over time. And
what we're doing has a lot of, um,
similarity to techniques in digital humanities in the literature area
called Distant Reading. And I don't know if any
of you have read there's a big chronicle piece about
a year ago on distance reading. Distance reading is
a word is kind of a made up term,
um, to deliberately address uh itself as a kind
of opposite to close reading. So if we think
about the ways we generally make meaning with texts we
do so through close reading of the text. A
distance reading is supposed to be a complimentary procedure to
close reading. It doesn't negate the effects of a
close read, but it gives you another tool and
another way to do that. And the idea is
, if we stand back and instead of trying to
derive deep meaning from a single instance or a few
instances, what happens if we step back and look
at millions of instances? What can we learn from
those moments? And, of course, that's what
the digital availability of a lot of these textual materials
now give us the ability to do so, whether
it's the Google Books Project and the Ingraham um uh
, widget that got released and that we all like
to enjoy playing with or you can see patterns and
word adjacent seas across, uh, you know,
500 years or something like that in the Google Books
, uh, happy trust repository. So there's some
really exciting things that are going on in that area
. So our initial dataset Jeff and his team had
been working with some, um, science museums to
two in particular. One is the museum of Life
and Science in Raleigh Durham, North Carolina. It's
actually in Durham and the the Science Museum of Minnesota
in Saint Paul. And they had some money from
the Smells, the Institute for Museum and Library Services
, to do an analysis of both museums. Digital
facilitation strategies to see if they were contributing in a
measurable way to stem learning facilitation is in the two
informal learning, the informal learning world of museums and
science centres. What pedagogy is to folks like us
. These are the things that the museum staff due
to try to encourage, learning to happen in specific
ways. But because it's informal learning, they don't
have all of the tools that we have at our
disposal. They don't have learning objectives and lessons,
and they certainly don't have tests, right? No
one going to Science Museum is going to sit still
for that. So they were really interested in trying
to figure out if the things that they had been
doing an online discussion forums in blogs and message formats
like that on lately on Facebook and Twitter. Um
, if any of those things were helping them to
meet their institutional goals of encouraging more scientific deliberation in
the community, etcetera. And in order to do
that, they had gathered up a bunch of this
material, Um, really years and years worth of
it. And I'll show you some of the actual
data here in a minute. That's kind of fun
to see. Um, And they took a subset
of that that was humanly possible with well fed,
pizza driven graduate students. And they coded it by
hand, um, using some specific categories. And
they were looking for these moves that would indicate that
something was happening that looked like scientific reasoning. And
because these are posts initial posts and then follow up
comments, they happen in a timeline. They their
interactions that start and then proceed. Some of them
are still going on, But they have they develop
over time, and so that that time dimension was
something that they wanted to pay attention to. And
it resulted in these coding sheets that we had in
the on the walls of wide. And I think
when Quinn visited, actually, they're still hanging there
. They look kind of like these PCR tests like
we were doing DNA or something like that, but
we color coded the moves and looked at instances of
them and then try to stand back and see.
All right, is something happening in this in this
thread that trends towards more reasoning, like science scientists
might reason or etcetera, etcetera. And so I
just took a lot of time. And of course
, they could only look at a few posts in
much detail. And so their group came to our
research group and said, Maybe you can help us
with this, and we we saw in that some
interesting opportunities. That's, um Ah, take a
little bit of a look there. Um, here's
what we were obsessed with at the time. So
that's their research problem. Our research problem. It
was slightly different, and that is we were interested
in the possible use of graph theory. Techniques are
sometimes called network analytics to evaluate what was going on
in Texas and in large groups of text and corpus
is text corpus is. But what we were not
interested in is in the ways that, um,
a lot of people were using social network analysis,
um, to do that. And by that I
mean we weren't thinking about the nodes in the graph
the dots being people and the lines are the edges
of the graph being messages of one sort or another
. That's a common way to do it. What
we were thinking of is that the nodes are something
about the text, like pieces like maybe rhetorical moves
eventually or things that are indicators for those. And
the edges represent relationships among those things. So we
had the idea. I'll show you why we had
this idea. But we had not seen very many
, uh, precedents for this because our social science
colleagues were more interested in who are using social network
analysis, were more interested in analyzing what happens among
human agents. And we were really trying to map
these sort of trends in larger scale discourse to find
out what the whole group was doing. And that
was really the key difference for us, so you
can see where our interests started to align. These
were two research groups working independently up until this point
, until we talk to each other and they were
like, Oh, we can help each other.
So here's the chicken thread. I'll talk. I'll
show you what these things look like. So this
is a very typical exchange, only showed you one
part of it, Um, but it's very typical
of the whole thread. It consists of about 857
as of the other day, when I checked and
made the slide posts to a single blog post by
a museum person about backyard chicken raising, which is
a very big deal all over the country. Um
, and in this one, we see anonymous user
Patty asks a question. And then Dr Jackie Jacobs
, University of Minnesota Sota poultry specialist, answers the
question. And I have enormous respect after doing this
project for Dr Jackie Jacobs poultry specialists, because not
only does she know a lot about poultry, but
she is extremely patient, and she answers all of
these questions even when she sees them and variations of
them over and over and over again. Um,
and so she's a wonderful person. She's actually one
of the one of theirs, the science scientist affiliates
at the um at the Science Museum of Minnesota.
So she doesn't work at the museum, but she
has signed up and, uh, monitors the threads
that she's part of. So 857 of these posts
is a lot a ton of text there. This
is actually a rather short exchange, and it's a
short response by Jackie. It's also tends to be
on the shorter side of the kinds of questions or
stories that people tell about their chickens. So you'd
be surprised what people are very passionate about their chickens
. It started in August 3rd 2006, and it's
, um, as of May 5th, 2000 and
12, which is the last time we took a
big snapshot for the For the data set. It
had 857 posts. And the question of what we're
trying to ask across that whole thread is. Is
something happening here that that resolves to people learning to
do science? Or is this helping, uh,
the the Greater Minnesota Metro Minneapolis Metro area, Twin
Cities Metro area? Um, with scientific literacy?
That's the kind of question that that the museum directors
desperately wanted an answer to. And of course,
we couldn't give that broad of an answer. But
maybe we could scale that back and say, Can
we see science happening in this thread? Yeah.
So in order to kind of show you how we
went about what I'm going to talk about is how
we went about answering this. Um, the answer
is our human coders did see some evidence of some
interesting learning going on scientific learning going on in the
chicken threat. And they were pretty confident that it
was It was among all the threads that they looked
at. It was one that was they would hold
up as an example. And so we started with
what the human coders were already recognizing in that way
. But in order to get to a place where
we could evaluate that in a less supervised way using
the computer algorithms, we had do some theorizing about
what was going on and how we might do that
. So again, not networks of humans like this
. So if I were to to take here,
I picked a same same list or same blog.
But I picked a very short thread that has only
about 17 posts. I'm going to show you how
it grows over time. Um, the diagram here
gets built according to the way we think of most
socio grams or social social network analysis. So each
bubble is going to be a response. J.
Gordon made the original post, and we see some
arrows. It's a directed graph in that way responding
, indicating who's responding to who? All right,
So we C. J. Gordon, the museum
expert person and then all these other people start weighing
in pretty soon as soon as these first three posts
after the initial blog, Um, something starts to
happen that are human coders could recognize. That is
what we'll say is a matter of concern arose.
It wasn't a serious matter of concern. This is
not the chicken thread. This is a threat about
some scientists who found a giant squid and, among
other things, in the Giants in the story,
they couldn't resist tasting it. Oh, and the
article specialised, uh, speculates about why that was
. And this led to all manner of stories about
what it might mean to take some of this massive
creature into oneself, and and that they started telling
jokes, reincarnation jokes. So So the thing in
the gray circle is what we're interested in. There
is what's going on in this thread. So people
are just telling jokes about squid, right? Oh
, so I promise you some theories, So bear
with me. This is the piece of the computational
rhetoric side of things that, um, the rhetoric
shins are at the table, I think, in
most cases to provide. And that is So what
do we What would we expect to go on in
a situation like this? Why is it that we
think that we can train a computer or teach a
computer to recognize these things at all? So there's
this guy, Russian, um, theorist named Bactine
. Some of you all have read him, and
some of you will, if you haven't already.
Others won't know who this person is. Um,
but among other things, he's in this particular passage
. It's a famous essay he has called the problem
of speech genres and he says some things in this
passage that, um resolved to what we think of
in writing studies as threshold ideas about genre. What
I mean by that is the way writing studies people
use the word genre and the way normal people who
aren't corrupted like us use the word genre completely different
. And once you cross the threshold, it's hard
to see the other way. Um, our view
of genre in writing studies is something like this.
So I have all these writing states. People here
will throw something at me if I get it wrong
. Um, we see the regularities that arise in
textual forms that we call genres, um, to
be the result of repeated or habituated action. And
these habituated actions are usually responses to recurring social situations
. Where genre conventions these textual forms are most stable
. They are generally reinforced by power. Institutional power
law. Right? Um, sometimes that power is
implicit. Sometimes it's quite explicit. It doesn't get
any more explicit than a form that you fill out
their imposing the structure so that they can normalize the
response right. It's also quite mundane and boring in
on one level. So nobody is imposing genres,
too do much other than maintain social order. And
sometimes that social order is, you know, itself
quite trivial. Sometimes it's not. Um, so
Bactine noticing this is, uh is a kind of
a theory theoretical precursor to this understanding and writing studies
. That is the folks who develop this understanding of
genre and writing. Studies have been reading Bactine among
other people, especially Carolyn Miller and this is the
idea from which that flow. So if we go
back to that idea that Martinez talking about, I'll
just point to the orange text here. Here's what
back Tina saying If I could paraphrase a little bit
, he has an idea that in order to be
understood in order to create mutual understanding among speakers,
we have to repeat ourselves, and we not only
have to repeat what we're saying in the immediate moment
, we have to say what has been said before
in situations like what we've like the one that we're
in. That's how you get recognized to be doing
whatever it is you're up to. And that means
that every word you say has been said before,
except for the very few that aren't and it's surrounded
by lots of things that have been said before.
That's how you are able to disambiguate. What this
means is, um, it's a little dismaying,
I ought to say for especially, we have creative
writers in the room because what it means is that
originality is the kind of exception, whereas re utterance
or reuse is the rule and on a on a
very mundane level, especially at the at the,
uh at the level of everyday interaction everyday discourse.
It's pretty un controversial to think about, but it
does do some interesting things to unsettle how we make
meaning and what it means to write something down.
We have to be recognized first, so we have
to re utter something. Um, the orange text
says this means that any utterance proves to be something
that is best understood, not considered in isolation with
respect to its author. That is not attributed only
to Quinn saying this was a really broody chicken.
Now what am I supposed to do? Jackie Jacobs
, but in response to other kinds of utterances of
that type as a link in a chain of speech
communication with respect to other related utterances so that all
those things share some things in common. And again
, um, this isn't terribly, um, controversial
if we step back and think about scientific discourse as
a whole. So if we think about the genre
and the relative stability of the article of the scientific
article in Science, um, this makes a ton
of sense. We want We want scientific articles,
particularly within disciplines, to be very stable. So
that we can use them in particular predictable ways.
We want to be able to We expect something of
the abstract. We expect something of the methods we
expect, and those things have relationships to one another
. And those get reiterated every time you publish one
. There's in that case institutional authority, keeping that
genre together, right, and you can take little
little deviations from it. But if you go too
far, you get smacked. If you go way
too far, you just it'll be okay. You're
just not writing that genre. You're doing something else
, all right, so that's the idea, right
? And that's an abstraction that I've sort of lived
with and came come to grips with. And since
I was a graduate student, since I was dismayed
that not everything coming out of my mouth was my
idea. But that was part of this sort of
social fabric that I was in. But it was
that link in the chain of speech communication with respect
to other related utterances that really jumped out when we
started rethinking what we were doing were like That means
that every text is a graph. It's a network
, but not each text within itself. But the
text pieces in one utterance and in another utterance and
in another utterance and another utterance. That's what he's
telling us. That's the best way to understand them
. And I'm like, Well, maybe, But
let's maybe let's try it. So let's go back
to our thread and I'll we'll get to where we
eventually got to. We're now a little bit further
down the line in the giant squid thread, and
we could see another concern arises, which is.
People keep telling reincarnation jokes on the right hand side
there, and people like what? That's the next
one, but it predominates. More reincarnation jokes.
A few people try to break in, but by
that time there is momentum. One guy, the
guy, The thing at the top. Mhm Masonic
co toothless. It's like the next highest size squid
. Not quite giant but super big. I don't
know what the size squid it is, but so
this was This is one of the only on topic
posts we might say that comes in and it says
, Hey, I work at this other science museum
. We have a video of some people catching a
squid like this I think it's the Humboldt Bay squid
. If you've ever seen one of those guys,
um, they ignore him, they just keep on
with themselves. Yes. So what we what we
wanted to try to do is could we again,
this isn't hard for a human to map. We
can kind of you could read it down and kind
of see what's going on. And what we wanted
to see is could we teach the computer to see
something, uh, to teach the computer to find
the grey circles? That is what What? What
is it about those bubbles in the gray circle on
the right and the grace work on the left?
That makes us put them in those circles. Um
, and And how can we How can we see
that? So this is where we turn to the
idea of some network metrics. This is a very
simple picture of a network, and I'm gonna probably
offend the math people in the room by kind of
going through and talking a little bit about what we
mean when we evaluate a graph. So a graph
has a computational object because it's a well understood structure
that we can turn into a set of numbers,
and we can perform a set of mathematical operations to
derive and also to analyse. And it's actually not
terribly complex, and it's really interesting once you know
how it works. So if every circle in the
graph is what we call a node and every line
is an edge, one of the ways that we
can start to evaluate what the graph structure is is
the amount of connections that anyone node has, and
that's called its degree. So if you look at
the note in the middle of the graph, it
has a degree of six, right, And this
is a non directed graph, meaning that we're not
caring about the the lines having arrows going in or
out. But in some cases, we might be
like if we're setting up a communication network, we
might be interested in whether the messages traveling from me
to you or from you to be, and in
that case, we would measure two kinds of degree
in degree and out degree. But in each case
, we reflect that value as a property of the
node. The node in the middle has degree of
six and on the edges degree of three, right
? Nothing has a has a degree of less than
two, so that's one way to think about it
, the other the other now. So let me
point out something that's interesting about that. That isn't
irrespective of the shape of the network. So we
could move. We could turn things around, and
we could. I could mash it up and draw
it slightly differently and still maintain the degree values here
. But there's something else unique about the node that
says D six in the middle, and that is
its position in the graph relative to the other.
Other notes. It occupies a position that we call
the brokerage position, and that's because it sits between
one group of nodes on the left and another group
of nodes on the right. It's the middle man
, right? It's a powerful note. Because of
that, in some kinds of networks, right,
it can become the switchboard by which messages traveling from
one side must pass to the other. So if
you own that node, you could charge for traffic
across it. It's like a little bridge. Um
, there are whole countries that have built their economy
on being that node. Uh, and, uh
, by the same token, it has a little
bit of a privilege if we think about traffic over
that network. So everything that happens over the network
, if it crosses from one side to the other
, has to go through No d right. So
that means that we have a property for that too
. That means that no d has something called high
between the centrality. So those are just two of
the network metrics that we have, and they are
not invented by me. They're part of a section
of mathematics called graph theory. And hundreds of years
ago, people started working on all of this,
and they not only, um, taught us ways
to give us some metrics for understanding how network structure
looks in a static picture like this. But they
also have worked on networks that grow and change over
time so that you can model, for example,
what the impact on a network will be if you
add X more nodes and where do you need to
add edges now? We're all grateful for this because
otherwise our cable TV would work a lot less well
if people weren't able to analyze the structure of the
network and find out where you need to put repeaters
and where they need to run more lines and all
that kind of stuff. Um, it also has
all manner of other kinds of things. Like if
you imagine a big matrix like this with a lot
less regularized space between nodes, et cetera, You
get algorithms, for example, for filtration. Like
, how long does it take for a particular soil
density for water to filter through it? So that
tells you how much water you need to put on
certain kinds of soils in order to achieve irrigation.
So they're all manner of practical reasons to think about
networks and lattices and matrices and to apply these analytics
to them. And we saw some potential value because
of the back Tinian theory conjecture of doing that with
text. So here's the squid thread. This is
something. This is a picture of something called topic
mapping, using a really simple, um, set
of natural language processing techniques without working too hard,
we strip away, um, all but the stems
of words. And then we look at, um
how often they appear and their adjacent seas to other
words. And what we can do is we can
start to see what topics characterize what's being talked about
. So without any human reader looking at that thread
I just showed you, we can say with pretty
pretty good confidence that this is a threat about squid
. And if we wanted to search for that,
this is a way that we could do it.
Some of you might recognize between the centrality and Eigen
vector centrality associated with another popular algorithm that most of
us use all the time, which is the Google
page rank algorithm. That's the way we return.
Valid search results, um, is that they evaluate
the structure of networks um, caused by hyper linking
, and that helps them return to us a little
bit more valid Search results when we search for specific
terms connected to or on a particular page. So
they're doing something like this. Um, but if
you look just below squid, something else was happening
that we were really excited about. Now, as
reparations, we were were promiscuous and in our use
of these methods will use all of them and we
don't know what we're really doing. Um, and
one of the things we were doing is leaving words
in that typical stop word lists, uh, would
throw out like words like if squid if and think
were interesting to us because those were indicators that something
interesting was going on. And in this case,
um, when you go back and you see people
framing conditional statements with if and saying think a lot
like I think, um, they are indicators of
some hedging that was going on. But it was
interpersonal hedging, right? They were sort of making
jokes around this idea of the thing of the eating
squid and what what the Squid's power could do to
the scientists. But this gave us an idea.
It was completely like, Oh, I wonder if
we could find other examples of this because hedging behavior
is something that we're actually interested in. So we
ran the same thing on the chicken thread, and
we zeroed in on words that we thought had a
high correspondence to these hedge moves. Now, when
we say heads, we mean things that scientists do
to to deliberately, uh, limit the the audacity
of their claims when the evidence is less than convincing
, right? It's, uh it's what you're you're
being careful about what we say. I'm doing a
lot of it right now. Um, so in
this case, we we were looking at some interesting
patterns that we saw in the degree distribution. I'm
sorry. Only so these are just the links going
in in our network diagram turning every word in the
text into a node and using, um And where'd
adjacency is to see which words were most connected to
other words using Just agree. We saw one thing
that we expected to see from based on other research
. Is that all the way at the top and
something that looks a lot like a power law distribution
for those math people in the room is what we
would expect to see is the topic word. And
that way. Up the top 7. 50.
All right, I'm sorry. Down here is the
word egg. So that wasn't too surprising in the
in this thread about chickens and eggs, that egg
is the outlier word. But it was relatively surprising
to see clustered here with, um, all of
these modal verbs would should may, Will, Might
, can and not too far away from all of
those were other conditions which I think was there again
. Like I think if you do this, you
might And we're like, Oh, this is good
. This is good. So when we come back
to seeing rhetorical patterns, we were like, Well
, this this this degree thing is just not giving
us enough sensitivity. So we we wanted to see
if there was a way to more precisely evaluate what
role these topic words and models or indicators of hedging
were playing. Because we had a hunch now that
there were little networks inside here that corresponded to some
behavior that we that we perhaps we're hoping, uh
, to see. And and in fact, we
recall that in the science, um, are in
the hand coated edition that are human Raiders we're seeing
. They were indicating that some science learning was going
on. So what we did next is we took
each instance of the word egg, right, And
we looked at, and I told this story earlier
today. This is another kind of discovery story we're
looking at. How often it Rikers? So it
was. There were in across all of those posts
That giant data set that I showed you it showed
up more than 400 times the word egg, and
it would sometimes show up, closer together and sometimes
further apart. But when we look at the distribution
of how long it would it would, how many
tokens would be in between each recurrence? We saw
that there was an interesting stabilisation point that is,
there was a big chunk of of these intervals that
were about the same size about 17. And we
were like, What is going on there? So
we didn't actually understand that very well. We could
always toggle back and forth between the clear text and
see the actual natural language text thread and what we
were seeing in the, uh in the analysis.
And what we were pretty confident of is that when
the cycles were long, when eggs showed up and
there were, like 300 tokens before another iteration of
eggs showed up. That was when Jackie Jacobs was
talking our poultry expert because we could see that happening
quite a lot. But in the other times,
um, it looked like it was more genuine,
genuinely dia logic, which is a good sign to
us. And it looked like there was more of
this kind of questioning and proffering a potential, uh
, idea. And then there was, uh,
answering or modifying in other words, more deliberative activity
of the sort that we thought that our human Raiders
were looking at. So, on a whim,
we said Yes, but what does that? How
does that actually show up? And so we started
pulling those out as individual pieces of the graph and
throwing them into a software package called Jeffy, which
helps us visualize some of these networks. And that's
what you're seeing here. So here's one instance of
egg and we walk around here until we get to
the next one and then another one starts. And
so what we were seeing is when we zoomed in
from the big picture of the network, our graph
was a series of cycles. Now I'm gonna come
back to my title Common places. So this is
where we were like, Oh, interesting. Okay
, these are places that group is returning to.
In other words, what holds this deliberative discourse together
is a mention of the word egg and some pattern
, and it turns out in here. You'll see
some of these. I believe there's something this one
. If there's one of hedge value words, so
here's what it kind of looks like. This is
a typical cycle. So here's our high value,
high centrality value token egg. You can see the
number is quite a lot bigger than all the others
. And then there's an if then then there say
, not Can will all of our models right?
And all of these are, um, roughly commonly
represented in these kinds of cycles that we're talking about
. So these are all the way across. This
is what what we started thinking of is our egg
cycles, or are rhetorical cycles mhm? Let me
try to help you understand what we think that means
. So I'm gonna walk back. I promise I
would do this for the research team today. Here's
here's how we prepared the sample and here's here's the
I won't go into all the technical parts, but
how we got to a picture like that. So
first we take all the post and we stem them
so that we're seeing just the stems of the words
Take off the endings and we calculate the frequency distributions
for the tokens. Then we process the tokens as
diagrams to start making drawing edges. And then,
according to when they were posted, we build the
graph. That is, the more tokens that are
added. We get more links, right? As
people post to the comments to the threat, this
effectively converts the whole text into a big graph.
At this point we're in Bactine land. We no
longer have any way to trace back. Specifically,
um, in the in this view, who said
what to who. So we're not thinking about individual
speakers or individual agents at all. We're just looking
at the structure of the whole. Then we calculate
the between the centrality and we order the tokens high
to low. And that gives us our candidates for
these walks or cycles. Then the high between these
terms are used as the anchors and we plot those
. And then we code the walks for the presence
of these high between this tokens that indicate hedging and
that's what we call our hedge finder. If we
see enough of those, we think that we can
do something interesting. Which is to say in this
thread science is happening or it's more science c maybe
right, that's a hedge. Let's see what these
look like so that you can give a sense of
what I'm talking about. So I pulled this one
off of our test set. Um, because as
we couldn't just go by, we had to evaluate
our little hedge finding analysis here. Um, so
this is the heads that comes from a scientific article
. It's actually a Cochrane review of the use of
electronic devices to monitor, um and help type two
diabetes patients. Um, Cochrane Reviews are really reliable
sources of us, uh, for us, for
hedges, because they try to translate the results of
research for practitioners. And they have this passage in
them. That is a what's called a plain language
summary of the science. And they say, Okay
, primary care doctors. Here's what the research says
. And now here's what you should do. And
so it's super hedged all the time, it says
, appear to have right based on the strength of
the evidence. So these are the hedges that were
kind of dealing with. What we did is we
built a test set that included about 900 sentences that
were hedged hedges and 900 that weren't. And all
of these came from scientific articles so that we could
have a at least a prima facie argument that this
is science, science, people doing science. And
then we set to work trying to see if our
heads finder could detect in a random string of these
sentences what the humans could do. Now we're a
lot more narrow. We're just trying to see if
you can see a hedge and the computer can see
a hedge. The rules more or less go like
this. So when we're building the algorithm, this
is kind of our, uh our parameters, if
you will, were trying to tune walks of in
token link. That repeat seemed to act as indicators
for types of genres. And so in. In
this case, as I mentioned before, Jackie Jacobs
, long explanations were kind of out of genre for
what we were looking for, which were these more
deliberative moments. So they were more like an essay
on chickens rather than reasoning about chickens. So within
that narrow range that was relatively stable, Um,
we were seeing that head cycles are structures similar to
those greatly weakened utterance boundaries, right. The permeable
to the author's expression ideas. Um, so we
mark those statements as hedges or non hedges in the
same way we think that humans are seeing them in
that way. That is that they're picking up the
same signal, if you will. So if we
showed these to a human, we would just say
, Is that a hedges in a non hedge?
Right? And, uh, yeah, I'll just
skip this one part. Let me let me show
you how we did that. Now we've tried to
be careful about how we're training, um, our
little algorithm to do this because again, we do
not want, um, we do not want to
have the machine returning, a result that a person
would also see. So in this case, sentences
that admit to incomplete our limitations and knowledge. That's
probably our clearest example. And then there are some
other kinds of hedges here that are a little bit
more subtle. So if you look at the third
one, they're the ones that predict outcomes which are
less than definitive or speculative. So you see may
or might in those kinds of things, note that
In every case, the hedge is more than just
one word or the co presence of two stable tokens
. It is a relationship among one or more tokens
that might vary but which are predictable from a particular
list. And so it is probably a stable co
variant relationship at some point, that's what That's the
assumption that we were looking at. So we use
these 900 heads 909 hedge sentences tested it to train
the classifier. We tested it against 155 hand coated
155 non hedge sentences. And then we did that
. We repeated that, um, randomized, the
order in which they happen in each pass relied on
that randomized training and about 80% of the time,
um, the computer marks it, right. So
, um, the way it looks is decidedly non
exciting. It spits out an excel CSP Excel spreadsheet
, and and it has the sentence and it says
hedge or non hedge, just like you would in
the in the data set that I showed you before
. And it still gets it wrong about 20% of
the time. But it has some possibility. Has
some has some some thoughts. So we're starting to
think of what? What is this thing that we're
that we're doing here? Like, we have to
give it some names? Um, and we realized
that as we've gone along, and as I've presented
it for you here today, we're getting really much
more narrow and more narrow and more narrow. And
that's what I was saying before about how, really
, uh, excited. You get about humans ability
to interpret text because the signals that we're picking up
when we read all of these things are really ridiculously
difficult to train a non human to do. And
I think we're not in any trouble at any time
soon in that regard. Um, but we also
think that there are some reliable indicators for lots of
different kinds of genres that could be incredibly useful.
So what is the science museum do with something like
this? Well, here's what we hope to have
them doing, maybe as soon as the end of
this year. And that is in an unsupervised way
using the hedge finder to see which threads are doing
the kind of work to facilitate science, learning that
they hope that they would do and which threads are
not. So that then that we can start thinking
about well, which facilitation strategies might have some impact
on those things? Is it, Um, And
so that's That's the key question, right? The
what happens in order to create that that pattern?
Because we have a lot of cases threads that go
on for a long time. We see them move
in and out of the desired head zone, for
example, for this particular test, and one of
the things that we're able to see is questions have
a remarkable ability to bring people back into that 17
or 18 token cycle. That seems to be the
sweet spot for people reasoning in interesting ways. And
that's that was a great that was grateful to me
because that's what teachers do all the time. And
that's what scientists do all the time as they ask
questions. And it makes sense on its face to
say, um, if somebody's going on and on
a question might bring the group back. Uh,
like I'm going on and on right now, Uh
, mhm. It leaves open, though a fair
amount of questions, and I want to just kind
of throw out a few. And as I mentioned
, this is all very experimental work. We think
that the Heads Finder is one very narrow test linked
to one very specific kind of discourse that runs on
a fairly specific kind of medium or mode of text
. Textual interaction, which is a list serve discussions
and online discussions. Does it work? If you
if the utterances are scientific articles of scientists talking back
and forth for each other, does it work?
If there, um, newspaper articles, we couldn't
tell you, but I suspect it would be a
lot noisier, and Messier does more Heggie equal more
science C We think so, and it can be
a useful tool. But in finding uh, sorting
through millions of threads to find out which ones are
worth taking a closer look at, however, there
are probably some real limits there. For example,
we were talking about some today that, um,
there's a lot of interpersonal hedging going on in these
threads because they're humans dealing with other humans. So
some of these hedges are not about the subject matter
there about Well, I don't want to offend this
person, so they're treading lightly instead, so we
aren't necessarily seeing a 1 to 1. Correspondence with
more heads, behavior and more scientific reasoning, um
, are are special topics common. That's a slightly
different version of my question that I asked special topics
our topics that are key to a particular kind of
discourse or genre. Here we will feel pretty confident
in saying they appear to be, and that is
, hedging is common. If you're if it's scientist
talking to other scientists, or if it's a scientist
talking to a layperson about a scientific topic, hedging
is there. And so that's what we would expect
to see right. Our common topics common so common
topics are topics that are a little bit more broad
things like cause and effect. Do we see that
all the time when people are doing deliberative? A
reasoning of any sort? Well, Aristotle encourages us
to teach all of our students that that's how it
always happens. But we don't really know that they're
that common are common sense approach, and our close
reading will say that in many instances, cause and
effect is an effective way to reason. But we
don't know, but now we might have a shot
at starting to find out if those common places that
we think about when we associate good communication or effective
communication in one area can actually be validated against large
examples of multiple situations of those things. So I'm
gonna stop there in case I, uh, have
any or in case you would like to ask any
questions. I know it can be a little bewildering
, but I appreciate so much you're listening to me
today and welcome any questions you might have. I've
got a microphone since we're recording this session, If
you wouldn't mind raising hand, I'll bring the mic
to you so you can ask the questions. Building
a question that, yeah, we talked to worry
about the medical research that we're doing in this school
And, um, for us, we're looking at
how experts speak versus well known experts speak. So
for us, hedging actually in debt indicates non expertise
. I'm not sure the stock's gonna go up,
but I think it might versus somebody who's an expert
says Apple's going out by now something like that.
So I think it's interesting that in your testing data
, when you're looking at those science articles and grabbing
hedging words as evidence of science that that last section
is looking at how they're kind of translating it for
the practitioner and later. And so I think that
that may be something that carries on in the future
of your research. As far as this relationship of
hedging indicating science, which seems contradictory to what we're
doing because I don't think scientists as expert but hedging
for us as a non expert, Um, if
a scientist is talking to a scientist versus a scientist
taught me to lay audience, how does their vocabulary
change and or how does their hedges? Yeah,
that that's the interesting thing about this and the thing
that we would have to be careful about. We
talked a little bit about what is the likely outcome
in the near term of this line of research is
that we'll have a little piece of analytic code that
we could actually give your research group, and you
could run to find hedges in your corpus. But
what those hedges mean in your corpus, our technique
could be silent about it would be like buying a
chemical test from, you know, one of those
scientific companies where you can find out the salinity of
something. Unless you know what that is An indicator
for it won't do a lot of good. So
this is as close as we get in the humanities
. Two basic science. And that's part of why
it's a little dry. I apologize for that.
But, um, I think what we would definitely
need to do in, um in response to a
problem like the one that you're pointing out is one
is have more kinds of tests, right? So
if we can do if we can develop a hedge
finder, odds are we could find other word proteins
and genre characteristics to develop other tests. Then we
could use those, perhaps in combination and with some
nuance, um, to do some more interesting work
and maybe those. That is where we start to
cross some boundaries. Perhaps. Uh huh. I
practiced My question was maybe not fully understanding everything,
but I just have 111 question. The populations that
you look at the exchanges where the groups basically homogeneous
and because if you're looking for, um accepting certain
kinds of proteins from things, it's going to depend
on the culture and the people within that group.
So where if you were to do this in another
group or another strand of, you know, say
posts, it may be very different. And so
we may find other proteins. Uh, yeah,
I guess it would depend on. That's a great
, great question and a great point. I would
say it depend on the criteria for homogeneity, but
I suspect that they are very homogeneous. They're the
kind of people who go to science blogs, which
means there are a lot of kids. Um,
and there were a lot of folks who are engaged
, I guess, in there in in the museum
itself, in many cases, but also in issues
related to whatever the science is talking about. Now
, that wasn't always the case. We did see
a pretty wide range of literacy skills in the posts
. That could have been a function of age.
It also could have been a function of, um
where the, uh you know where the folks are
coming from. They weren't always. You could kind
of tell that they weren't always, um, first
language as English speakers. Um, but we also
didn't have a lot of demographic information because many of
especially about the, uh, the folks who are
coming in as guests, they are posting with anonymous
things. I think I had another one in here
. Yeah, here's another one. Here's a Here's
an example. So we're guessing this is someone in
the UK five pence coin all English speakers to That's
pretty. That's another limiting factor here. Uh huh
. In this methodology, you are stemming the words
before you analyze them, which means not looking at
them in terms of parts of speech, Whether this
is a noun or this is a verb, what
sort of losses and gains would be involved with running
this sort of graphing through part of speech analysis,
perhaps the ratio of announced the verbs in a in
an utterance give you a clue as well as to
whether this is science or not, it might mhm
we we had early on, we we we.
So the easy way to say that is we were
aware of part of speech tagging as a possible preparation
step. But in working with the with the questions
that we are working with, we never got to
a place where it made a whole lot of sense
to us to do to run that particular procedure,
but we certainly wouldn't rule it out. And I
don't know. I can't think off hand. I'm
pretty good at thinking on my feet. I can't
think of a way. It would change what we
have. What I talked about today, Um,
in part because what were often seeing related to the
high centrality content words is either it's either a concrete
or an abstract noun as the topic, Um,
even if it's Jared. So even if it's fishing
as opposed to have fish there, they've identified it
in order to talk about it. And so that's
a good question. I don't know. Maybe it's
time for one more question. This is This is
a big picture question. Your associate graduate school folks
in humanities, uh, on the writing studies and
one of the community when the Slams against writing studies
is that it keeps me part of the towards social
science and away from the humanities background. And now
we're talking about computation models of basic science and English
studies. I wonder if you talk about the pluses
and minuses in the big picture. Ideologically, politically
, institutionally, about this sort of move where English
studies textual studies, one person at a book come
close reading, moving towards what most would consider science
for universities. Ideologically, politically, economically. Yeah
, and we thought I have thought a lot about
those things. So I'll talk about one thing that
I see as an unqualified benefit. And that is
this project has been able to bring to the table
more diverse, Um, disciplinary perspectives and almost any
other project that I've that I've been engaged in.
We can get computer science people. We can get
who do natural language processing. We can get linguists
. We can get literature, Folks, we have
someone at Michigan State who does is interested in,
uh, what's it called? The style? Um
, a tree. So author attribution, which is
kind of the reverse of what we're after, Uh
, and we there's a there's because it, um
because it centers around the text in in an interesting
way. What is satisfying about the way those discussions
come about is those disciplinary interests are still all sort
of returning to Well, how would a human makes
sense of this? Um, the other thing that
we've tried really hard to do is well, we're
not necessarily trying to replace the human in all of
this equation. What we're trying to do is supplement
what is really impossible for a person to do.
Uh, but yet which these science museum folks and
others are feeling increasingly, uh, responsible for doing
, which is what's going on on all your list
. Serve. We paid all this money to blog
. What is the blog doing? And to know
what the blog is doing is ridiculous on the scale
of all of these posts, right? So I
think we talked a little bit about this today.
Is that the rhetorical question is driving the rest of
it, and that isn't That hasn't always been the
case when I've gone to interdisciplinary meetings, but it's
very exciting for us because, um, the computer
science folks are able to say, Well, what
would you What would the theory expect to happen?
And if I said that the curve looks like this
, I'll explain what that means. But is that
the curve you would expect to see? And I
go, I don't wait. All right, so
I have to kind of figure that out, but
we're really trying to draw a shape that is conforming
to an interpretation of a text. We're just trying
to do it with lots and lots and lots of
examples. So I think in that way, um
, I don't see any threat to any of those
seats at the table In terms of where to their
disciplinary expertise. It does occasion a very different way
of working. Um, we couldn't do this project
outside of the research center. I don't think,
um, not because it requires a lot of resources
to do. It was relatively cheap to do,
except that we had to. Somebody had to pay
all those graduate students to find 900 head sentences.
You didn't think I did that, right? Um
, okay. But apart from that, these are
free tools out on the internet, and we're stringing
together a little python script here and a little.
Our script here, I forgot, are we did
some more, um, and so that it wasn't
the technical infrastructure of the center, but it was
the social infrastructure of the research center that allows us
to have these have these projects that go forward.
I think Bill has about 15 more minutes before we
need to drag him off to his next meeting.
I know that some of you need to go,
but if you would like to stick around and talk
with him a little bit more, we'd be happy
to have you do that. We've got additional refreshments
. We hope to see many of you back here
on October 2nd. Wednesday at one PM for next
digital discussions, conversation will be on the big data
project on the Spanish 1918 Spanish influenza project That's happening
right here at Virginia Tech. So once again,
thank you to Bill Hart Davidson.