My name is Aaron Brantly, I'm a professor in the Department of
Political Science at Virginia Tech and part of the
Integrated Security destination area. iId like to introduce Dr. Nicolas Christin
from Carnegie Mellon University. He is jointly
appointed in the school of computer science and engineering and public
policy. His research focuses on computer and
information security, and he has spent the better part of the
past decade studying online crime. So without further ado, I'll turn it over to Nicolas.
[Nicolas Christin] Thank you Aaron, thank you for the
introduction and thank you very much for having me here.
I'm going to talk today about online anonymous marketplaces
and our efforts in trying to quantify the economic activity on those
marketplaces. So this is going to be a subset of the
type of activities that you were introduced to
in the previous presentation and let me first tell you why you should
care. You've probably heard about things like
silk road or alpha bay and you might wonder, you know, why
should we even care about trying to analyze the economic volumes?
After all, the police will get the real data when they actually do
seize the marketplace and seize the servers.
Well, it's probably useful to get some intelligence
prior to seizure, that's the first thing. And you don't have access to this
data until you actually, you know, seize the server, so when you're just
observing it from the outside you can't rely by definition on having
the server available. In addition to that, a seizure is not
you know a panacea in the sense that a number of marketplaces that were seized
the police was able to get access to some of the servers but not necessarily
to the entire back end. They may end up you know seizing encrypted containers
and and things of that nature that don't necessarily yield
actionable clues. The other reason why you should care is that you
may think that, well, we can do blockchain forensics. Bitcoins are traceable, we can
very roughly estimate the amounts of money flowing in
and out of certain marketplaces and essentially that should tell
us everything we need to know. And indeed there's been lots of progress,
spearheaded by academic research, in 2013 also
2012 2013 that eventually yielded a number of companies china this is
being uh the most famous one which do give good estimates indeed but
it's not foolproof it's made significantly more complicated
by new cryptocurrencies newer
cryptocurrencies such as monero or zcash
and it also gives you very good or decent
metrics for aggregate marketplaces but it doesn't give you a lot of information
about what individual vendors are doing and i'll make the case that
this is actually very important intelligence. My favorite one
was something that i got in fact when I submitted once a grant proposal and I
got this response from a reviewer who obviously was not in the field
saying oh well it's easy you just scrape a bunch of websites
and um it's easy indeed uh as it is easy to take a photo finish
of a sprint race, and if you are a very good photographer like
I am, this is typically what you end up with
when you take a picture. You're measuring a very fast moving object
with you know equipment that may not necessarily be
the most um the best the best equipment possible
and that doesn't tell you anything right? If I'm asking you who won that race
you can't tell from this, you can't even tell what the order is yeah probably
there's those two blurry figures that are ahead of the pack but i really would
be hard-pressed to give you in your ranking in fact what you really would
want is this and to get this
as i'm going to argue in a few minutes that
it is indeed just scraping a bunch of websites but it is not as easy as just
you know simply using off-the-shelf software
so today's talk is based on publications that are publicly
available you will have a link on the last slide to my
website the first two are academic publications published in in
security and information science conferences
the last one is a report that was published by two european union
agencies with whom we worked very closely the
european monitoring center for drugs and drug addiction
and europol so let's take a step back and if i'm talking about illicit
internet commerce probably the image that it conjures
is this kind of website canadian pharmacy
get some free viagra online and that was indeed
what you would see you know from the early 90s
until now there are still a lot of such sites
but um since 2010 2011 there's been an increasing number of
online anonymous marketplaces they've existed for a while in fact
there were precursors to silk road there were things like farmers market for
instance from those of you who are familiar with
with that but what silk road did was uh in 2011 to become really the first i
would say mainstream online any news marketplace
and to plug it back into the previous
presentation you can notice probably i don't have a mouse
pointer but you can probably see that
this address is a dot onion so this is not a website that you could access
with your regular chrome or firefox browser
you wouldn't you would need to use the tool browser bundle to go to that
website it's what's called a hidden service and
i'm not going to go over the details of those because they were
discussed in the previous presentation but suffice to say
it is very challenging to figure out where the physical location
of that server actually is in that specific case it was in iceland
but it took a lot of investigative effort to figure out
where the server was it was run by a um an american citizen based in
san francisco with servers in finland france and estonia i want to say
the web server itself was apparently in iceland
so what are those online animus marketplace
yes um so is is that still running this website still running silk road has
been taken down it's gone but there are others that
have essentially taken its place and run on the exact same business model
so those online anonymous marketplaces as you can see if for a second you just
ignore the type of goods that are being sold
they're very much like your typical electronic commerce marketplace
now if you look at what is being sold as you can see
pretty much everything is contraband or narcotics or counterfeits
so they are essentially kind of the amazon marketplace
of illegal goods in the sense that silk road itself
and it's uh closely related other marketplaces
do not sell goods themselves they are a marketplace
they introduce vendors to sellers but they don't sell products
themselves so saying it's the amazon.com of illegal goods
is actually a little bit misleading it's really more like the amazon marketplace
right we give you a space where vendors and buyers can meet
on those marketplaces you will mostly find drugs you will find quite a lot of
digital goods credit card numbers fake identities uh
on some of them you will find weapons although that is not a very profitable
line of business um for reasons that will become clear
when i when i go over the data you will never
find child pornography on those types of marketplaces
the child pornography community so to speak is completely separate
these people just like to sell drugs they like to
engage in you know illicit business but it's a very very different community
right and they actually uh completely ban
things like child abuse and things like that this is these are completely
different services uh which for us as researchers is good
because we can actually spend our time studying those
marketplaces without risking to run into this type of really horrible content
they try to provide customers with safety with
convenience with variety accountability and with competition so
essentially if you look at those last five bullet
points this is exactly what any online marketplace
would do regardless of whether they're selling
drugs or t-shirts right uh they are really trying
to bring those uh those illicit products to the masses
the technology they rely on as mentioned before
uh they rely on uh anonymizing services such as the tour network that was
previously introduced a couple of those rely on a competitor
to tour called i2p it's really marginal at this point i
think that there was one that was registering a little bit of
business but by and large uh they're on tour for
customers there is uh absolutely no cost to browse
those marketplaces in fact all you need is little browser and you can create an
account it's free in fact it's so much free that they're
probably not tracking your information as much as
facebook or google would there's really no information needed for vendors
if you're interested in selling things on those marketplaces
you're most likely on a vast majority of those you're gonna have to first
pay a bond which will range between 50 and 500 give or take depending on the
marketplace that bond may be refunded after a number
of successful transactions they are very often by invitation only
for vendors particularly when a marketplace that is
fairly small grows very quickly administrators tend to be really worried
about undercover agents operating and they try to
somewhat police uh who is coming in and those vendors
um frequently um are evaluated based on public feedback
so think about airbnb or ebay it's exactly the same
model of you know you decide whether a vendor is trustworthy
based on the reputation that has been garnered
from previous sales payments until i would say last year um
payments were primarily and almost exclusively
in bitcoin uh marketplaces often act as an escrow agent
you have people that are and that's in fact the um sorry that's in fact the
major line of business of those marketplaces is to act as an escrow
agent much like ebay doesn't directly profit
from the sales of the goods on ebay but from a commission that is taken
to make sure that the sale completes to the satisfaction of both
parties those marketplaces derive their profit
from acting as a trusted escrow agent the escrow may sometimes act as a mixing
service sort of money laundering tool so to
speak uh although uh this tends to be a little
bit overrated they are not very good
laundering uh facilities some of them rely on
encrypted messages for this they use pgp um but that's basically the two build
the two major building blocks are tour and cryptocurrencies
uh nowadays they've mostly switched to uh
well they haven't switched there's still a lot of transactions in bitcoin
but we've seen the emergence of monero uh as
a more uh private type of cryptocurrency we've seen also
marketplaces diversifying using bitcoin cash using
ethereum for reasons that don't really have
anything to do with security but more with economics
so some of the questions that we wanted to answer when we started to look into
those marketplaces were very simple we wanted to figure out
how much was being sold what was being sold whether there was a
large number of vendors that were relevant
whether we saw bulk sales whether we saw on the other hand
retail level sales only and also what could be potentially successful
interventions from a policy and enforcement perspective
we started collecting this data in november of 2011.
so um this is uh research that's been going on now for
many years uh in november of 2011 silk road was pretty much the only game
in town there was another marketplace called black market reloaded
that had almost no tangible business at the time they
became a little bit bigger subsequently i will have a slide about that but
initially it was pretty much a monopoly the last time i
actually ran statistics on what we had collected was in august of 2015 we had
collected about 35 marketplaces corresponding to about three terabytes
of data and there were depending on the scrape
depending on the copy of the website that we were looking at
between 27 pages uh which was a website that was hosted out of a high school
apparently and that got shut down within like three days
because the owner just really didn't know what they were
getting themselves into to over three hundred thousand pages per
scrape and that was uh the somewhat infamous alpha bay marketplace
which later uh in 2017
had about half a million pages we're still collecting
uh i did a quick check a couple of days ago we have more than five terabytes of
data uh there's more marketplaces that have
been act that have been added to our monitoring regimen um and so that's an
ongoing collection effort so when we want to compute those numbers
we need several properties and those properties are very much
at odds with each other the first uh the first property is that we don't
want the operators of a marketplace to know
that we are actually monitoring them they know that they are being monitored
they are not stupid they know that there is a number of law enforcement agencies
a number of academics like myself that are very interested in figuring out
uh what their activities are but uh
we really should devise our scraping procedures so that we don't have you
know a telltale sign of
us trying to get as much data as we want
those crepes need to be complete and it's very
hard to do when you're going over tour which is a reasonably slow network
uh you're measuring your marketplace which is very large
and a marketplace that tends to go down fairly often
as we will see in a few minutes some marketplaces have very
very low uptime 65 70 percent now imagine amazon.com with an uptime of
70 percent no one would go shop there right um and
this is sometimes what we have to deal with
we need also the scraping to be as instantaneous as possible
remember this picture that i showed earlier of this
racing women well you want very short exposure right when you're
shooting a picture like that so you want the the picture to be as
quick as possible and that's completely at odds with
trying to be stealth right because if you are like blasting
as much as you can you will get noticed uh
so it's pretty hard to do uh so what we've done is we've built a measurement
platform that i will just very briefly uh uh mention uh
we start with a browser we log in manually we create some accounts
manually we have to solve captures that we can actually uh
outsource to um to capture breaking services
and then when we have all of the basic uh plumbing in place so to speak then we
are running our scrapes and those are using a bunch of different
tool circuits so i was smiling when i heard the
previous speaker because i'm for instance using tor probably for like
gigabytes of data per day and um so that's one of the uses is for
people who collect intelligence data uh so that gets into our database and
then we pass it and we distill those website copies so
to speak into a past database on which we can
run operations and then that gets into another framework uh
just for kicks some of the anti-scraping measures that we've encountered
some operators who detected us started to impose rate limits kicking us out
because uh our accounts were they were very
strange customers they looked at a bunch of items and they never bought anything
and they looked fairly quickly right well you know we are
we are prudent customers um we would get thrown out after a while
uh this is a somewhat basic anti-scraping measure where they
invalidate your account after like a couple of hours
uh they tend not to do that that often because it really annoys
regular regular and legitimate customers as well
we got a bunch of user accounts suspended
and my favorite one was this one marketplace
that didn't stay up for very long but where they when they detected us they
didn't do anything apparently except that they started
showing us pictures of items that were for sale that were all benign
all of a sudden they were only selling t-shirts and paper clips
right and then we logged into it and we were like no that's not right i mean
when i'm logging in as you know a regular user i see all of
the marijuana and the cocaine that is being sold
when my scraper is logging into it it's all you know kitties and unicorns
so we had to adjust a little bit how complete is the data well like i
said those marketplaces are very dynamic they're also
not very reliable in terms of infrastructure reliability
they take days to scrape for the largest ones and so we don't really know
how much we are getting every time we are getting it
fortunately statistics can come to our help to uh to figure out how much we are
we're getting and this is a problem that biologists
have known how to address for years this is a technique called mark and
recapture so let me just take a slight detour and assume that
so i live in pittsburgh there's a bunch of old houses those old houses sometimes
have mice mine doesn't it's very clean but
some of the others do and let's assume that you want to figure out the number
of mice that you have in a traditional pittsburgh basement
how do you count them well on the first day
you put a nice trap with a nice piece of cheese in it
and you see at the end of the day the number of mice that
ended up in that trap you count them you see that there are 10 mice
what do you do next you take a magic marker
and you put a big x on each of those mice and you open the cage and you let
them go go half fourth and multiply well don't
multiply because that actually would screw up our statistical analysis uh but
you release them and then you set up another piece of
cheese same trap and on the second day you see how many
mice you capture secondly you captured 13 mice what does
that tell you on his face nothing tells you that
there's at least 13 mice in the basement but what you do then is that out of
those 13 miles you count the number of mice that have
an x on them those are the mice that you captured on both days
let's say that it was five and then you can use
a fairly simple mathematical estimator that tells you that
well you capture 10 on the first day 13 on the second day there was an overlap
of five you can project that there's probably 26
miles total uh in that specific example the real
number is 24 but that would give you a population
estimate of 26. this is pretty much what we're doing
right we are not capturing nice we are capturing web pages uh we are not
capturing 10 pages we are capturing hundreds of
thousands of those the difference is that some of those
pages may appear overnight some may disappear so
statistically the argument is a little bit
harder to make but we also have the benefit of repetition
we're not doing this twice we're doing it 10 15 20 times in a row
and so using a statistical estimator we can figure out
how complete we are and unsurprisingly it really depends
on how reliable the marketplace is so here what i'm doing is that i'm
plotting the number of scrapes i'm sorry the coverage that we have so
the fraction that we think we've captured as a function of
the number of scrapes and the main point that i want to make here
is that if you look at the leftmost point
at one scrape you can see that it's pretty bad in some cases with one scrape
you may not capture more than half of a marketplace
that's this point for agora here right even for the more reliable ones
like silicon one was you only capture three
quarters of the website so you need repeated measurements to be able to get
a better idea of what's going on and that's going to be important because
a few studies that we've seen rely on what we call one-shot scrapes
and they may be really underestimating what's going on
because of those properties then what do you do when you have your
scribe well you analyze them and so let's look at what a typical
listing page looks like this is a listing from silk road it's
very old it's from a few years ago um but it's very common i mean it's
newer marketplaces change the interface a little bit but it it always looks the
same so you've got an item title you've got a seller name
an associated price a description of the item and most
importantly a set of feedback now that feedback is really cool because
it's very often mandatory much like on ebay when you purchase
something you have to leave a feedback a piece of feedback for the uh for the
seller it's pretty much the same on those
marketplaces and that becomes an acceptable proxy for sales volume all
you need to do is to see when the feedback was issued
and that information is helpfully provided
you don't really care about the message you don't really care about the rating
but you care about the fact that there was one piece of feedback deposited
three days ago one four days ago one seven days ago and so forth
and that tells you roughly when sales occurred
and you know what price the item was at that time
so you can project how much revenue this item uh produced
there's an assumption which is that each feedback corresponds exactly to
one transaction and that's a somewhat rough assumption we'll we'll
have occasions to uh to discuss it a little bit further
on many marketplaces you can purchase several quantities of one item and just
leave one piece of feedback so that's going to give you a
conservative estimate on some marketplaces you don't have to
leave a piece of feedback but there are relatively strong
incentives for you to do so if you don't have a piece of feedback it
is likely that the vendor is not going to leave a piece of feedback for you as
a buyer as well or they are going to leave a bad piece
of feedback for you which may preclude prevent you from engaging in additional
transactions in the future think about your transactions on uber or
airbnb if you leave a one-star rating to a driver
and they can figure out that you left a one-star rating
they're probably going to leave a one-star rating for you and next time
it's boring like today you're gonna wait for a while to
get your next uber so there are some fairly strong incentives for the system
to work reasonably well however as i will mention toward the the
end of the stroke it's it's not foolproof but it's a
reasonable proxy now you may have seen some studies where
people are simply counting the number of item listings
it's much easier to do right you don't have to go through that whole business
of scraping the entire web page in fact you can just rely on what they
tell you we're selling 10 568 drug listings but
there are several problems with that number one some
marketplaces just lie about those numbers
they do why because they want to look bigger
so alphabet was particularly notorious they had a hundred thousand something
drugs listings that's a lot of drugs listings they
weren't technically lying what they didn't tell
you is that this number was cumulative since the
beginning of alphabet that's all of the listings that
had been ever put on alphabet the number of active listings
that you could actually find at any given point in time ten times less
was about ten thousand uh they still were making a lot of money don't get me
wrong right but uh so number one marketplace
is my life the second thing is that this is
this assumes that there exists some stable ratio between transaction volume
and number of listings and that's a very questionable
proposition think about your local car dealer shop
do you measure their business by counting the number of cars that are on
their parking lots not really right i mean you see that
somebody is selling 500 cars that doesn't tell you how much
they sold yesterday or how much how many they will sell tomorrow um
so we actually ran a quick uh measurement and we we saw that
if you're using item listings you may be off by a factor of four
on the evolution marketplace so you could be overestimating it by four times
or you can be underestimating it uh by 25 there are some other
interesting analysis challenges one of them is holding prices
uh so this is what happens when vendors want to signify that an item is
out of stock but don't want to delete the item
why do they not want to delete the item because if they do release the item they
may lose their associated feedback and remember feedback and reputation is
really the real currency of those marketplaces uh so what you see
is somebody selling a paper clip that would go for two cents
all of a sudden lists it for a thousand dollars
no one's gonna buy that right so effectively that's
i'm out of stock don't buy this the problem is when you're doing
automated analysis you need to be able to figure out that this is happening
otherwise you're going to completely poison
your estimates right those outliers are going to completely ruin
any hopes of a meaningful estimate so we need to look at
historical prices i'm going to skip over this but
basically it's very simple statistical heuristics
and they tend to perform very well at filtering those those outliers
product categories well surprise surprise people lie about those as well
so this is something where vendors can
self-select a category of items that they're selling
but we've seen somebody for instance selling ak-47s
that was somehow listing them under potted plants
i don't really see the connection but if we had
classified this as gardening tools and
supplies it would have been a little bit surprising they didn't sell a lot of
those ak-47s for that matter but um
so what we do is that we don't really first the product categories we look at
the item description and from there based on text analysis we extract what
we think is the right category what this plot shows is that
this works reasonably well basically we the diagonal shows
uh essentially that most items are properly classified right so
uh those ak-47s would be properly classified as weapons
okay so once we have all of that what do we do
well we can get interesting estimates so this is what this plot shows
what this plot shows is as a function of time the revenue
of most of the major online analyst marketplaces between 2013 and 2015.
the first thing that you will note is that there are some gaps on this
plot that those are times for which we don't have data because we turned up the
measurement infrastructure or we simply didn't collect data
the second thing that you realize is that there seemed to be some sort of
monopoly here yes that is right it was silk road one
uh the other two marketplace is active at
the time black market freeloaded and um atlantis
were really not registering many sales and then what happened well silk road
was taken down so this is you know what i was alluding
to earlier the operator of that marketplace got arrested
sent to jail forever and uh what what did people do well as you
can see black market forwarded which was very
small when we started collecting data here very quickly got a boost
right i mean it went from
very very low sales per day to essentially
almost
the uh the sales that that silk road was uh was involved in
uh there was another marketplace called the sheep marketplace that was operating
at the same time and that was also uh quite uh quite that got also
quite large quite quickly both of those shut down fairly quickly i
mean the operators realized that it was nice to have a lot of business
but uh it was also making them the targets of law enforcement
and uh they basically left the the ecosystem very quickly
then there were a period of a couple of weeks where there wasn't much going on
a number of small marketplaces appeared most of them were
scams but then what we saw is the emergence of
something called silk road 2 which was run and operated by people who
were involved in the original silk road they were forums administrators and
leading vendors and very quickly they managed to
generate revenue similar or even exceeding those of the original silk
road but what was more interesting here is
that you started seeing competition with other marketplaces such as agora
or pandora or a little bit later evolution
also emerging at the same time so you started to see more diversity in the
ecosystem as opposed to having a simple monopoly
uh removed to some sort of oligopoly where
the top four or five marketplaces really matter
there were probably about 10 or 15 marketplaces active at that time maybe a
little bit more maybe 25 most of the other ones were very small
and didn't really matter then
in november of 2014 a joint operation between europol uh the fbi and the hs i
believe led to a number of those marketplaces to
be taken down in particular the silk road 2 operator was arrested
again a san francisco person
hydra was also taken down hydra was based out of
well the operator was in hungary some of the other marketplaces got also
taken down probably the largest one not represented on this graph was called
cloud9 so a bunch of marketplaces got taken
down but what you see is that the two that were fairly large and that
didn't get taken down namely agora and evolution just got a
big boost from that because well people participating in those
marketplaces just moved to the marketplaces that were still
alive so here you see a gray area and if you
remember what i was talking about earlier about the coverage
we don't have enough coverage to make meaningful projections so i just removed
that part of the data because statistically it doesn't make sense
i mean it's we just didn't have enough data to be able to uh to project how
much was there so the main takeaway from this 2015
study was that the ecosystem as a whole was very
resilient to both law enforcement takedowns and to scams
a number of those marketplaces also absconded with
a lot of their customers money and people recover within a couple of weeks
they just decide to move to another marketplace
this is the situation as of last year in 2017.
there's a couple of other marketplaces that i'm not plotting here there were
things like nucleus nucleus was actually fairly large uh
black bank and a couple of others but i want to focus
on one in particular which was very very large called
bay alpha bay started in december of 2014
and very quickly became the leading marketplace and i had
numbers it's on the same scale as the other graph that basically exceeded the
entire ecosystem before so as you can see despite all of those takedowns in
2014 20 2013 2014 uh the ecosystem as a whole
grew right uh and this is actually an underestimate
we see sales in excess of 600 dollars a day
it probably topped a million dollars a day
fairly frequently uh in 2017. now uh what is being sold on those
marketplaces so what this plot shows is a breakdown by category
and i want to draw your attention to the bottom three categories this is between
2013 and 2015 and what you see is that if you go up to
that line that's about three quarters of all sales
cannabis and dna also known as ecstasy and other stimulants so cocaine
essentially using that classification so that's
between two thirds and three quarters of the sales
and they are all what we would call party drugs right
you do see a little bit of psychedelics so that's
also stuff that people use to get high uh
you see what else you see benzodiazepines
you see a little bit of opioids but those are mostly heroin
at the time we didn't see that much fentanyl and those other substitutes it
was mostly just you know straight heroine uh we see a
slight increase in prescription drugs um and
so essentially it's mostly drugs we do see a little bit of
other and miscellaneous items which mostly
are fraudulent ids banking credentials stolen credit card numbers and things of
that nature but those items spell in comparison to
narcotics alphabet is very interesting so we have
here a slightly different breakdown of categories uh we've got
cannabis we've got stimulants that now include ecstasy
so basically ecstasy the yellow one cocaine is a separate category opioids
they did tend to increase a little bit i mean we don't see you know
the same increase as we did see on the street
but we did see a renewed interest in opioids
but what is more interesting is this part
so these are the early days of alphabay back in early 2015.
and what was happening at the time is that the marketplace didn't really have
much traction they were trying to make a name for
themselves and what did they do they sold stolen credit card numbers and
things like that that's how they bootstrapped their business
as people realized that this was a functioning marketplace
the drug trade started to pick up but this is something that is quite
interesting there is somewhat of a uh porosity between the two lines of
business of selling strong digital credentials and evolving into a
drug marketplace one of the last points i want to make is
that uh vendors actually are very interesting as
well what this plot shows is that the top one percent of vendors
make more than a hundred thousand dollars everybody else doesn't make a
lot and in fact your median vendor is going to make
about three hundred dollars most people are small fish you don't
even need to worry about them uh that i'm going to skip in the
interest of time what we also see is that uh most of the
sales are of the retail uh nature we don't see
that many box sales and that makes sense you're not gonna ship you know pounds of
cocaine through the u.s post office right you're gonna send a bunch of mdma
but you're not going to send like kilos and kilos and kilos of drugs
we were able to validate a lot of this data when people did get arrested and
criminal complaints actually revealed how much they were making
but feedback is not a complete panacea uh
in particular uh what we know uh from uh bitcoin transaction analysis is
that our alphabet coverage is about two thirds to three quarters of
what really took place uh private communication suggests that
on hanza which i didn't plot here but uh there
was actually really few feedback that were deposited we don't really know why
uh we're also missing about 30 of the funds for silk road in 2013
and part of the reason is that people actually didn't spend the money
they were using sell code as a bank so you can imagine how happy they were when
the server was seized the bottom line is that with the move
toward more privacy-centric coins like monero
feedback remains a very decent proxy and i'd like to invite you to go take a look
at this website all of the data that we have is publicly
available we do delayed releases and we partially
partially anonymize it so you're going to get data until 2015
but that's going to be able to you know take some of that story and you can play
with those graphs and like you know zoom in on a specific category if you want to
know how much mdma was sold and we are currently in the process of
actually moving toward a commercialization of
of some of this data collection uh the research remains at carnegie mellon i
mean the analysis of the data is uh is pure
research that's sponsored by uh by dhs but the uh the data collection part is
really a somewhat tedious software engineering
effort uh and so we are currently uh incorporating with an eye
toward uh providing intelligence not just to
law enforcement but also to uh to the commercial sector uh pharmaceutical
industries very interested as well as people selling luxury goods
uh clothing and things like that because they they start seeing counterfeits
uh on those websites so what do you take away from this uh
it's useful hopefully and not straight forward to
to measure marketplaces uh they do represent a significant
business uh they're primarily primarily used for recreational drugs
they have historically recovered from takedown efforts and scams
uh and a few set a few small sets of highly influential vendors
are really like the anchors of those marketplaces uh
and so with that i'm gonna conclude and uh
i don't know if we have time for questions but i'm gonna be around
until the middle of the afternoon thank you
the url at the bottom is my webpage you can download those papers from there
i think uh since we're bringing a little behind if we want to ask questions
during lunch