[ Introduction] All right, I would like to introduce professor
Kathleen Moore of James Madison University.
She is an expert in technology, intelligence
analysis, and moreover how to train students
and new analysts in these disciplines. So this panel is called the dark web and
issues teaching the new analyst.
[Kathleen Moore] I will clarify I will by no means
claim to be an expert, a lot of people a lot smarter than me
involved in this area as well. I'm very excited to talk to you about
this today. Something happened about a month and a
half ago that sparked an area of research for me
and it's three things that had occurred.
One was, we had our spring career fair at James Madison University and I was
having conversations with government agencies and
various corporations. Two, an interesting conversation I had to have
with some parents of a student, and then three, an experience I had taking
students to IBM Cambridge where we participated in
an incident response simulation. So it was the culmination of these three
things happening all within two weeks of each other
that you know really got me started on
this particular research I'm going to talk to you about today. Now I'm right in
the middle of it, so these are preliminary findings.
But I think it can foster a really interesting discussion here.
So to start, what determines proficiency in dark web analysis?
This is something I want you to think about just for now because we're going
to come back to it at the end. And basically I'm going to jump around a
lot but I promise i will bring it all back to center by
the time we wrap up. So keep this in mind and then hopefully
we can have a good conversation about this when we're done.
Okay, so we talk about training students, and I say that, training,
we're teaching them how to use you know tools and how to analyze a
very specific environment we talk about in higher education
and is extremely complicated this is a very dynamic subject
and we have what 45 minutes here today yeah i could probably go on for this for
a couple hours so to limit it down when you talk about
this we have to talk about some systemic issues uh surrounding deep and dark web
and understanding you know kind of what plays into things that we're seeing
today so first of all glut of jobs in the job
market second obviously education talking about
cyber security versus cyber intelligence and then lastly uh educational
infrastructure so let's begin
i started looking i went back and i was um
had captured some information i don't even know why i did it
from indeed.com about available cyber security jobs and i did this back in
december and in looking at this it was really
kind of interesting um you could i had grabbed the data it
was about 68 000 jobs and you could look at it according to education
and you could see everything that was available
from high school uh to a doctoral degree the number of jobs
to address something here pause
do we know where it's coming from ah okay
all right i'll continue then so we can look
taking the jobs i you know basically had categorized them according to
education that was required but then i also could go through and
um analyze them of according to experience
and i'm like okay well you know this is interesting and
i kind of sat on this data for a while and then of course i told you a month
and a half ago i had all of these things happening all at
once that made me revisit this so first and foremost when we talk about
cyber security unemployment rate it's zero percent
it's absolutely zero percent now let's compare that national unemployment rate
3.9 and dropping so obviously what does this
mean we have a predicted 2 million jobs that
will not be filled by 2019 which is going to equal additional
pressure on the job market that's an issue yes how many people have
possibly a cyber security job open in their
area yeah how many can you say no of course not i'm retired but
three years in it yeah particularly and i retired three years ago and at that
time not taking anything there was a shortage
already or looking at me have about 200 000
cyber security okay yes uh gareth i believe you mentioned something about
that yesterday as well in the united kingdom
yeah i mean our students do cyber security program and i tell them when
they start they will get a job at the end as long as they're
marginally competent yeah basically yeah um even if they're not yeah exactly
i i can open up my email right now and i will probably have about 10 inquiries
saying do you have any bodies for me and this is a daily basis anywhere from
like probably about 10 to 15 inquiries do you have someone you can send to us
i don't know if anyone else is experiencing this but this is a yeah
regular part of my experience so obviously we have an issue all right
so think about where we are right now
within 200 miles of where we are we have 75 different institutions that teach
some form of cyber security now i don't include online programs but
again institutions teaching this now if we think about the different
educational paths that you can take and this is brick and mortar meaning you
have in-person attendance or you're taking it online or you're
doing a hybrid program meaning you take it online and then you occasionally
have to meet in a classroom we have a lot of things that we can do
here so uh if we look at low end skills mostly
kind of basic technicians um you can there's plenty of high school
programs that now are providing vocational training
there's basic certificates that you can achieve either at a community college or
online or you can get an associate's degree
through a community college for low to mid-level obviously we have
the traditional bachelor's degrees we do have professional certificates
once we start getting into higher skills we have graduate certificates and then
advanced degrees and of course with advanced degrees we have that also
geared towards research and education so we have all of
these different career paths or educational paths and
then again just within this area 75 different
institutions not including online so think about that
together we have this question why is higher
education failing to produce enough cybersecurity
professionals it's a good question right absolutely
well we have some standard reasons you know one is
interest you can't force students into it as much as we would like to
uh we have basic stem issues i have no problem once
i have young women who are coming into my classroom getting them to
look at maybe even computer science as a minor or as a double major i have
absolutely no problem sending them to coding camps but getting them to
actually take you know a dedicated course work
alongside men is very intimidating for them um james
madison not the most diverse university so we
also have issues with representation of minority students you
know everybody experiences this to some level of
another different variables involved we also have attrition people sign up
not for them they drop out i've had plenty of students midway through a
program get a job and leave it happens
in looking at some of these programs i have to ask myself they're
they're they're uh saying that they're giving students a cyber you know
they're educating them in cyber security but when i really look at their
curriculum there's maybe one class that's offered to them is that really
cyber security probably not so we have to wonder
you know you know separating the wheat from the chef who's really teaching this
topic matter but then you can also ask i'm sorry we
had a question
um is that do you think because on the computer science field is mostly com
like men in that field do you think that's partially the reason
again i can't speak for all universities but it's
over 90 men uh james madison
okay so other reasons that this could be happening
well sometimes we can ask is degree structure
actually a problem i have a computer science major who has taken two
intelligence uh courses with me one in cyber
intelligence one in social media and i ended up basically tutoring him
through a history class and also a literature class
very bright young man he has a very bright future ahead of him
but you know he really struggled with the jane
austen that was assigned to him in his literature course and he needs his
literature course in order to get his degree and i'm a big proponent
of the liberal arts education but it's not necessarily for everyone
do we necessarily need you know is there maybe a tipping point where maybe
certain courses are a little too much for certain types
of student this is a person who uh is very
dedicated to his studies he's you know a perfect day
in his computer science courses but when he's gone into the liberal arts courses
he has really really struggled so i'm wondering
is this necessarily a good thing do we necessarily have to have liberal arts
education for certain people who are on a certain
type of career trajectory and that's a conversation worth having
and then also um i still see that there's a failure to
kind of understand the space when we're talking about cyber security
there's definitely the technical side but then there's also
the analytical side particularly in this room today we have people who are
you know doing dark web analysis and political science and sociology i do it
in intelligence analysis but who here is actually in a computer
science program teaching you are so we've got
two so again we you know we have this type of practice occurring outside
the traditional discipline where some of that training is normally
occurring
so briefly to kind of talk about a difference between cyber security and
cyber intelligence everybody pretty much know
the difference yes
yep i got it i got the slide all right cyber security we're talking about cyber
security it's very inward looking we have people
who are really constantly examining internal systems
for threats um it's very technical and proactive and
of course they also are reactive when we talk about things
like incident response now there's varying degrees of this i
know that we could probably nitpick and add you know dozens of more things to
this but broadly this is what we're talking about
when we get into cyber intelligence though
this is really outward looking these are the analysts who look outside and they
are looking at the greater macro environment what is going on in the
world and are we prepared for an attack like
that um it's risk assessment it's using ttps
typical you know structured analytical
methodologies and it's also talking about communications these are people
who are really tasked with communicating with
the technical part of their organizations but then also the
decision-making part of their organizations
and as we all well know not all of our decision makers are the most
technically oriented persons so we do have a difference between the two
so keep this in mind i promise i'm going to pull this all together
all right so let's go back to talking about those job listings
in looking at all of this i went back and i took a sample about 10
so i was looking at 6 800 jobs and out of that 52 percent did not
specify a skill level that was required
there was very vague it was very generalized and really didn't talk about
the skill 74 percent interchanged cyber security
and cyber intelligence skills so what they were describing was really
a cyber intelligence person or they were what they were describing
was saying cyber intelligence but they were really looking for a cyber security
person and that can be really confusing we're
finding that there's a little bit of a communication disconnect
uh i'm in the middle of this research so i don't have hard numbers but then i
also found that there were many that required a degree but when they
really got specific about the skills that were required this is something you
could find in any high school educational or
vocational program so it was very low tech but also
whoa what did i do
here we go we also had required no degree but then had very advanced skills
and then of course lastly when we talked about dark web analysis
not a single one of those 6800 jobs specified anyone looking at any having
any type of skill set of analyzing dark web uh i'm in the middle of
actually uh querying a lot of these companies so
again i'm in the middle of my research but right now i've probably
contacted over 200 of these companies and
i asked is this something that you would require of this job and every single one
of them said yes like okay why is it not in your
job description good point so let's bring this back to higher
education when we talk about dark web analysis in
higher education it operates in extremely unique space
right now and i'm not sure why this is maybe
because it's gaining traction maybe because it's still so
new but i also like to refer to it as a black hole and i'm sure there's plenty
of academics here who would agree with this
there's many universities who are offering cyber security but they're not
offering cyber intelligence we have university administration or
even department administration that really has no
knowledge about dark web i mean they really
honestly um they kind of recognize the space but
they really don't know what is exactly being taught within their curriculum
they couldn't name anything here's this professor you might want to
talk to admin has a little bit of knowledge but
they have a tremendous amount of fear um eric i'm going to be bringing into
in the discussion afterwards but i have taught at an r1 university i have taught
at a teaching college i teach now at a comprehensive university
and all three of them would prefer that my students not
be on deep web let alone dark web on university networks tremendous amount
of fear i've had multiple conversations with the
lawyers administration insurance companies at
some point and we'll talk a little bit more about
the difficulties many teach forensics you know
basically how do we kind of deconstruct a cyber incident
which is a valuable skill in cyber security but not really teaching
investigation so when we look at all of these students who are going into law
enforcement or are going into business and
you know looking at intellectual property protection or
we have uh people who are still you know doing media studies and about to go out
into the world we have a lot of them that are not able
to go have these skills going out into the
jobs they're we're waiting until we're teaching them
on the job they're leaving university and getting those skills
after the fact and the problem with this is when we talk about investigation
investigation usually requires a certain amount of subject matter expertise
we're better really to teach that in a university
it's a good question we need to develop more of these interdisciplinary skills
so uh we have represented political science sociology
peace and justice studies um we've got you know kind of a broad swath of people
who are you know doing dark web analysis in
different areas and everybody's bringing to it
their own unique skill sets from you know their areas of personal research
then we have to look at the fact that uh given university concerns the majority
of them do not have proper infrastructure by
this they do not have a dedicated lab they do not have partitioned networks
or you know commonly what we might refer to as an esoc
so many of them don't have it that takes money
also uh don't have instructor instructors who are knowledgeable
i was reached i was looking at those 75 different institutions i was looking at
their web pages couldn't find anything about dark web i
couldn't find anything about deep web i was going through the course catalogs
i immediately found a few that would talk about
cryptocurrencies in bitcoin um many will talk about ethical hacking
but when we talk about structured courses
about this unique area of the internet not a lot of formalized courses so of
course i call them and there are some professors who are
teaching it but it's very piecemeal and a lot of them say to me it's i'm
basically risking a forgiveness rather than a permission factor here
uh they don't necessarily publicize it because they could get in trouble with
their universities or their colleges
you have you make a yes distinctions that i don't
really understand so what's the what's the distinction that you're drawing
between the dark web and the deep web and between computer security and
computer intelligence okay well deep web
basically um we have a when we talk about the
we're more concerned with dark web analysis here for today but the problem
that we find at a higher education level they don't even want us
looking at you know the real benign areas of deep web
they find that to be very problematic mostly because they don't understand
they think that dark web and deep web are
interchangeable they're not knowledgeable so there's a lot of fear
which means they end up putting a lot of restrictions
we talk about cyber security versus cyber intelligence
um cyber intelligence people um you know slightly technical but
again mostly end up being uh like typical analysts they end up you know
scanning environments they end up being uh more social science oriented
socio-technical oriented rather than being the people who are
dealing with the internal systems they end up
um serving as a bridge between your computer secure or cyber security
analysts and then those who are in charge of a
given organization thanks okay all right where was i okay so
knowledgeable of structure instructors per domain in talking with
some of these uh different institutions i was directed
to people who are actually doing dark web research
in other areas in english department peace and justice
religious studies which i found really interesting
very few of them were actually teaching this within computer sciences
and again just to kind of reinforce we have students who are going out into the
workforce everybody's telling me that they got
these skills you know let's bring it back to that you know that spring career
fair i like to go up to you know potential
employers and say what did you think of the students can
you give me any feedback what are you looking for
and every single one of them we need students who are familiar with
navigating dark web and analyzing dark web
so obviously we are now behind the curve when we talk about it from an
educational perspective and then of course there's also access
to courses for the few that actually offered them a lot of them end
up being um you know buried in computer sciences and
then it ends up being an upper-level course
and then students who might be interested in it say that you are
studying you are in religious studies you don't have access to computer
science because you're not a computer science
major yes quick question in regards to what the
employers are looking for are they it's a very different thing if
an employer has a bunch of dark web experts or deep
web experts and they need more of them or the fact that they don't
have one at all and they need a newer young employee to come in and be
that expert for them is it more the first case
the second case or some combination of both i imagine it's a combination of
both because about 50 of my students end up
going into national security although i think some of us in the room
could say that national security doesn't necessarily have the requisite
skill sets all the time as well um it depends but but it depends but you
know the communicated need is that we need
these people who have these skills and again if we're setting up a barrier
to entry because you're not a computer science major
again it creates additional problems so we can talk about recommendations to
this and these are just things that i have been thinking about but what i'd
like to do is when we conclude this let's kind of go back to that
you know um original question you know what really determines proficiency in
this area so for recommendations i think we need
to update and redefine the cyber security or cyber intelligence
skills model um if you were to hire um
anybody here ever have to hire someone for specifically cyber security
anyone yep okay what do they need to know
what would you expect of them
it depends on the role that we were we would be hiring them in for
so you have a different skill set level correct okay so we have skill sets from
across the board so initially we look for a cyber security analyst
that's usually our level one or a tier one role
and the requisite skill set there would be knowledge of information technology
systems how to network how to troubleshoot how
to spin up and turn down systems administer
systems from there we can teach them what to
look for from the analytical side um level two is your pen
tests um so your your traditional hacking
where they can go forward and they can now
with the understanding that they've received from level one they can go
through and kind of hack systems and understand the
consequences of what those actions are and then level
three would be incident response so now they are forensically imaging systems
and they can detect what exploits someone used and
build out a timeline of how how an attack occurred okay um sir
okay so roughly the same thing so we have the idea of what a cyber security
expert should know do you specifically reference dark web
do you find that people coming in actually have that experience
so i have not referenced that specifically in a job description
that i've been trying to fill that i think
there have been places where i've had candidates who did have that experience
but it wasn't something that we were specifically asking for
okay and yes and no yes i know okay it's just it's i would
say it's 50 50 depends on the candidate okay so you know if we talk about cyber
security and typically with cyber security now
that we have kind of also the growing function of threat hunting
then it makes sense that we have people who know how to navigate this space
right so within this you know you didn't
mention dark web to me you didn't mention dark web to me
but should we redefine the skill set to say that this should be something that
you are familiar with coming in the door or getting a job
something to think about and if we're going to say that then does it require
what kind of skills what kind of technical skills but then also what kind
of analytical skills and this is where we can look at domain
specification you know whether you're a sociologist or
peace and justice or something other we can look at all education
alternatives um there are plenty of high school students who have
gone through these vocational trainings that
you know they've got something that they can build upon and joe you were saying
that it seems like you guys are willing to kind of train on the job
right so maybe thinking about for the low end skill set
developing apprenticeships um i've gone through and i've
started kind of gathering data and there's a few apprenticeships that i've
found out there for cyber security or at least
cyber security technicians uh but not a lot and maybe this is a jumping off
point because some students as we well know
for the academics in the room there are some students who simply aren't going to
thrive in a collegiate environment maybe thinking about something about the
coding camp business model coding camps have been extremely
successful people who are doing career change or trying to encourage women to
get involved in computer programming and coding you
can go to these camps and they can be funded by
someone else or you pay for the camp and then you get reimbursed by an employer
once you successfully graduate is this something that you know could be
built upon something to consider i think we need to
promote dark web analysis as an interdisciplinary tool
i can't imagine anybody teaching media studies but not
teaching dark web analysis for journalism
or peace and justice for activists or i'm an intelligence analyst so obviously
looking at it for terrorism or if you are in business
intellectual property protection i can't imagine
i have a hard time actually finding certain
um domains where this isn't relevant except you know maybe that english
course for art majors so thinking of it as an
interdisciplinary tool not solely the domain of computer
sciences maybe bringing it out of computer sciences
and making it available more widely
definitely encouraging double majors and structured minors and computer science
students so i have quite a few computer science
students who take my classes and mine are mixed
methods and they succeed on the qualitative side they really
struggle on the or they succeed on the quantitative
side they struggle on the qualitative side and more
importantly i get computer science students who i make
them do you know a case study on a cyber incident and
being able to communicate what happened to provide a cohesive narrative
that would make sense to someone who wasn't necessarily technically oriented
is a real struggle for them they are not asked to do public speaking
in the majority of their classes i had two students who didn't even know where
to begin i had to put them kind of through a boot camp
so bringing that again into the second thing that had occurred
and that was um i had taken students mostly female
to ibm cambridge and we participated in our x-force labs uh
cyber incident response i wasn't really quite sure
how the day was going to go we were there with carnegie mellon and rit
uh cyber security students all male and they are you know very good cyber
security programs well in lack of understanding it was
interesting in that you know ibm really uses it to kind of
showcase their uh computer programs and they're they're very good they are very
very good but what they were trying to do
was show where the tech ends and analysis needs to begin
and that was interesting i wasn't expecting that so all of a sudden
you know maybe their program isolates someone who is of particular interest or
they could they could take this data and they could
show that the hack originated with this one particular person
under these specific circumstances does that necessarily mean
that this person is complicit and from the computer science side that
was it there was you know silence it was crickets you know couldn't really
answer that but what was great is that my students were able to jump in
and say well no we can have some alternative hypotheses here
let's look at her social structure let's pull up data that connects her to other
people here let's look at this let's look at that
you know um you think that she she's being
tagged as being suspect because she came in at 6 00 a.m
uh multiple times throughout this week well let's go through her emails
and see if there's anything that would account for that
and ibm definitely was not expecting that
you know they were trying to you know basically let the crickets go
and then you know say you have to be able to do this also they were putting
students into um they're breaking the students up into
teams where they're having them communicate
what exactly happened what's considered a response that you would want to give
to the media versus a response you'd want to give to
your board of directors how do we communicate this story and
again they set it up because they they know that the cyber security
students are going to really struggle with this but then
it was really great to see that i could you know put my students in here and
they they know how to do this we teach them how to talk about technology
so it really drives home a point that you know when we have these
collaborative teams when we have people who are more
analytically oriented versus those who are more technically oriented
we have maybe a fuller maybe not the best solution but a fuller solution to a
larger problem and i have a lot of students who come in
thinking oh i'm a computer science major i'm going to
take cyber intel and it's going to be really easy
but it's a really different way of thinking and we've found that the
computer science students have really struggled
but then we've also found that the intelligence students have really
benefited because now they're being you know exposed to
more technical oriented information they're getting working
in these technical teams which is something they don't necessarily get to
do within the intel program so because we
don't necessarily require that you are a double major for computer sciences or
even a minor
if we want change at an industry level or at a university level
now i know virginia tech is working on this
i know that obviously arizona state university
is accomplishing this as well you have to have
stronger industry partnerships because typically the one thing that
universities respond to is businesses coming to saying i'd like
to hire your students but dot dot dot they need these skills and then
sometimes it's not enough for me to say i need to be able to teach these skills
if it's someone saying i have 10 positions i need to fill i can't hire
your students because they don't have these skills
that's a different type of pressure and they're very responsive to that
so definitely you know looking at internships
looking at helping structure content um you know looking at that open
communication where we can constantly revise
you know what kind of curriculum we might need to build is really putting us
dependent on part industry partnerships and it's not
it doesn't always have to be necessarily about money i know universities always
want to push the money aspect of it but for a lot of us um you know who are
trying to apply this make it into an applied program
you know what we really need is the feedback of what's going on out there
what are you seeing and what do we need to pred
you know prepare these students for next
and then of course this is where we come back to are there others
so are there other things that we're not thinking is there maybe ways of re
thinking needs i mean i was just kind of spitballing these one night so
are there things that we haven't considered so basically
you know let's open up this forum and let's have a discussion about it