Okay, thank you. So I'm going to talk
about the ecosystem for the new HPC. Uh HPC
here is really referring to heterogeneous parallel computing rather than
traditional, high performance computing that you'll typically hear about
on campus. Um And the reason for that generally
is when people talk about high performance computing, they're
thinking about large scale supercomputing and aspects of that nature
and a lot of what I've done, although it
tends to get uh tends to get pigeonholed in the
supercomputing, high performance computing realm. Uh It actually
spans much larger uh breadth of activities. We're doing
quite a bit of work with embedded devices uh this
particular maybe not this one, but these particular devices
right now they have integrated CPU and GPU cores inside
them. So there's a tremendous amount of parallel computing
capability here that needs to be untapped in order to
get the appropriate uh end user experience. So with
that, let me give you a high level overview
of some of the things that we're doing. So
I said that we're doing all work that encompasses parallel
computing from system software to middleware to applications tools and
libraries and um you'll see a number of the projects,
although it's probably not coming up in the greatest
uh it's a little bit faded out but see systems
networking, green computing and renaissance renaissance, kind of
the rebirth of computing and how it supports all the
different myriad of aspects of our life from typical scientific
computing to what we do from day to day with
our smartphones uh and the like, um and we
do it all the way from something as small as
a mobile phone or or in this case particularly ipad
two things that you've probably heard a little bit more
about uh in terms of what we've been doing and
that is uh with our hokey speed supercomputer which debuted
as the most energy efficient supercomputer, most energy efficient
commodity supercomputer in the US Last year. Well a
little over a year ago November 2011. Um
to give you a little bit of a perspective on
on some of the work that we're doing, I'm
gonna focus primarily today on a broader scope project in
the area of heterogeneous parallel computing with a bit of
a focus on graphics processing units. Um That's not
where all of our work resides. Uh You can
see this synergistic uh diagram here. We're do we've
been doing work with um uh cloud computing in particular
making use of the Microsoft Azure cloud is part of
the NSF Microsoft grant on how you can leverage not
just the cloud but the combination of the client and
the cloud to affect uh the answers that you get
at speed at which you get them. Um We've
also been doing uh work in extensive work in in
the green computing space. A lot of work on
energy proportional uh data centers. Um This has been
a particular interest to google of late um I don't
see my PhD student here but we've had us several
overtures from google in terms of adapting what we've been
doing in the green computing space for their data centers
in terms of energy proportionality. And what that typically
means is being able to only expend as much energy
as you impart work load on the processor. So
if you actually did a graph of of the amount
of energy you consume with very low amount of workload
, you'll find that the idle power power consumption just
leave the computer on is pretty high. Um They
like your desktop on and running. It's it's on
the order of about 150 watts. That's just burning
uh burning power even though the workload on it is
zero. So the workload on it is zero,
you like the power consumption or energy consumption to be
zero, you want to be energy proportional. Um
And so those are some of the things that that
are of interest to google were also, let's see
other things. We we've recently done some work with
Professor Yao and cybersecurity in terms of leveraging, heterogeneous
computing for uh for for cyber security aspects and I
won't be able to get into that very much.
Um And in some sense, what started it all
, some of the work with Professor Alexa knew bree
f where we are taking his molecular dynamics code and
mapping it onto these emerging heterogeneous computing environments in order
to accelerate the discovery of binding sites um for molecules
. Um So the short of it is is that
we're encompassing all aspects of parallel computing from the embedded
space to the high performance computing space. We want
to provide scientists and engineers with scalable and efficient computational
tools and system software that enable them to concentrate on
their science and engineering rather than on the computer science
and engineering. And a lot of the work that
I'll be presenting a day is is has been contributed
by many, many graduate students and postdocs as well
as myself. And so I just want to recognize
these people here rather than at the end of the
talk when I probably just throw the slide up there
really fast and not get a chance to really properly
acknowledge the folks that are uh responsible and and contributing
to this larger endeavor that we have in terms of
heterogeneous parallel computing. So let me just start out
with some quick background here. Um uh This is
something that occurred about a decade ago. Um It
was dubbed compute nick because the japanese had debuted a
supercomputer that was five times faster than anything else in
the world. And then if you added the next
20 supercomputers in the world, they would equal this
single supercomputer. And so this this event got dubbed
as compute nick. It's a play on words with
respect to the Sputnik event and it led to a
the study that was put out by the U.
S. Council of competitiveness that was funded by NSF
and the Department of Energy. And what is interesting
to see here. I mean this really stunned me
when it first came out is that This is a
sampling of over 200 companies in the us and not
just computer companies but A whole swath of companies including
Fortune 500 companies like Procter and gamble. Where do
you find Procter and Gamble? You're like what's Proctor
in jail? PNG what's that? Their consumer products
, you walk into a grocery store? Probably about
half the products in there are either made by or
distributed by Procter and Gamble Cincinnati. Ohio okay.
And they're one of the many that said that they
could not exist or compete unless they had high performance
computing. So if you look at this chart There's
only 3% that said they could exist and compete without
high performance computing. And the other 97% said that
they needed it to compete. And the one example
I don't have it, I don't have a picture
of it here. But one of the examples that
is always amusing to talk about is a they do
computational fluid dynamics on the way that Pringles potato chips
fly off the conveyor belt and how they stack them
inside the potato chip. Can I wonder how did
they get it in? So perfectly do they have
a human stacking them. No of course not they
want to automate this process and they do this through
through high performance computing in this case. Um so
more recently this came out, so we had the
computer pick one, which was the japanese coming out
and and smashing us supremacy and supercomputing. And similarly
something occurred recently was called tien ho won. A
people viewed this as the second coming of compute nick
. Um but I didn't really view it that way
. It was only, it was only 43% faster
Than the previous number one supercomputer. But the reason
why it really garnered a lot of attention was it
was$20 million 42% less power. And so what
really is this, this is really what I call
the second coming of the Bay Wolf cluster, which
is a further commoditization of high performance computing. It's
commoditize ng it by using commodity parts that you can
largely by off the shelf, you go up to
your best buy and you take those parts, you
integrate it with other parts and then you put glue
software that integrates this into a singular um a singular
cohesive integrated computing environment rather than what looked like would
otherwise look like a gaggle of separate pcs or servers
. Okay, so what do I mean by a
Bay Wolf cluster? Well, the first coming of
the Bay Wolf cluster, the idea is that back
in the nineties, the only people that could run
on supercomputers were those few privileged people that had access
to the so called big iron supercomputing environments. And
so what folks at Nasa Goddard did is they said
, well we can build our own supercomputer, we
can With the confluence of Lennox being available, which
is an open source operating system with PCS that were
getting powerful enough to do computing power, people would
just buy PC towers. This is a picture from
1994 by the way. Um they would gaggle them
together and then they would create their own supercomputer and
so now what you're seeing is they're leveraging this.
They back then leverage commodity components from the operating system
to typical hardware that you can buy from the store
. Now we're doing the second coming of it,
is that in addition to leveraging these pcs, we're
going to leverage something inside of this called the graphics
processing unit to accelerate computation And the graphics processing unit
is what's driving everyone of the displays that is open
right now on your laptops and typically your displays 1920
by 1080 pixels, that's a million pixels that have
to get updated every 30 milliseconds. That's a tremendous
amount of computing capability in terms of doing a very
specific task. And that's updating the pixels on your
display. So then a question as well, can
you not then leverage that computing capability, that simple
computing capability and scale it to do millions of computations
on something else, rather than just doing the pixels
on your display and that's where the notion of graphics
processing for computing comes about. So you can see
that this is just a visual display that uh borrowed
out of google and the ideas that you can,
instead of just updating the pixels on the display for
that picture, you can start using it for for
real computation. And so um we have projects that
are related to all of these areas right now ongoing
in support of the uh the sciences and engineering that
are running on top of Gps. Um so this
is a an example of some of the speed ups
that we're getting and this is generally with respect to
a serial CPU. Okay, so if you happen
to run it on a quad core CPU that is
four CPU processors and of course you have to divide
all of those speed up by a factor of four
. Um so you can see that the speed up
varies tremendously. And this is where you have this
confluence of underlying architecture married with the system software and
application software and how that maps onto the underlying hardware
. So from an algorithmic perspective, you have things
that are very regular in computation, like the display
of pixels on a laptop. That's very regular,
all of the, all of the pixels are doing
the same thing. We're actually all of the processing
cores that are generating the pixel color are doing the
same thing, they're calculating the color. Okay,
So if you have computations that are the same way
let's say you have this huge a rate and every
array element is doing the same thing. Okay,
so it's a single instruction multiple data. You have
a single instruction that says okay, calculate this on
every single one of the array elements That you have
. That's a very regular computation and that's how you
generally get much higher speed ups is noted like in
the upper left we have 146-fold speed up versus something
that you have this more irregular. So if you
think about things like a graph traversal or graph search
that your facebook has come out with. That's a
very irregular computation. You go along different nodes of
your graph and some partial portions of the graph will
be much more computational intensity in terms of the way
it's connected. Other parts are less computation intensive.
And so you start to have an unbalanced workload.
It's very irregular. That doesn't allow you to exploit
the capabilities of graphics processing unit to its fullest.
And so you'll get lesser speed up slow down in
the 17 fold or 18 fold for example there.
So way to view this is that this is my
my way of a uh I don't want to call
it dumbing it down but just bringing it down to
a level that everyone can understand my own parents is
that you kind of you the CPU as your sport
utility vehicle. It does everything well but nothing Expertly
well. Um so you can turn left and right
, it can go pretty fast on the highway.
Um but like if you wanted to corner around a
90° turn at 150 miles an hour, it's probably
not a good idea because you're probably gonna roll the
sport utility vehicle over. So it does things well
. Generally um the GPU is very specialized, it
does things very specifically well uh and it goes in
this case the drag cars, it goes straight.
If you if you reach an intersection, a decision
point and you say well if there's traffic in front
of me then I want to turn left. It
doesn't do that well. If you think of a
drag car racer goes 400 miles an hour, however
fast it goes, it doesn't just be able to
turn on a dime, it also doesn't have brakes
. So it doesn't have preemption. Can't preempt kernels
or execution kernels, that's the same thing. And
what a Gpu does, you have to execute the
code until completion. You can't just preempt it in
the middle and start up another job. So all
of this is uh so you can kind of view
this as a metaphor. CPI you might be your
left brain. Gpu might be your right brain ideas
that you have a, you no longer have a
single type of brain or single type of multiple types
, uh multiple instances of a single type of brain
, but in fact we have two types of different
brains that you need to be able to figure out
which tool or process or tool is appropriate for the
task that you have at hand. The idea of
what we're trying to do in our work is that
we want to alleviate the burden of you as an
end user to have to manually do that mapping and
that optimization. So this is what we're calling.
Heterogeneous parallel computing. And again, I'll just just
just very briefly show you this is uh this is
something that we have in terms of hokey speed that
uh the Gpu accelerated super computer for the masses um
and it's 209 nodes where each node consists of a
traditional CPU motherboard. So this is uh where the
CPU resides. The two yellow blobs is where the
cpus get dropped in. Um there were 26 core
Cpus uh so there's 12 cores of computing capability on
that. And that is then supplemented by Gpu s
two and video. Tesla fermi Gpu cards. Um
so I'm going to break the flow here just for
a little bit uh for a moment. So it's
like I'm going to talk very briefly about the the
opportunity for folks to be able to uh take a
quick tour of this facility. Uh March one or
2nd we have graduate preview weekend that's coming up.
Uh and I need uh participants in terms of either
providing posters or uh serving on panels to answer questions
to incoming prospective graduate students uh to tell them what
life is like here, who's doing research on what
and what have you uh as well as just dealing
with logistics, we have we have a need for
drivers to get back and forth between on campus and
off campus for uh these prospective students. Um We
also will be taking to the dinner and things of
that nature. Yes. Barbara. Mhm. If
you're Mhm. Here. Mhm. Right. Sure
. Yeah, So what I'm going to do is
I'm going to send out a sign up sheet,
it just has name, email, and then what
your contribution might be for the graduate preview weekend on
March one and second, which I should put here
. Um This is voluntary, I'm not gonna force
you all, but you know, this is uh
this is giving back to our community, so the
contribution will either be a poster panel or logistics or
any combination of those or all of them. Okay
, And this will be on, March 1st two
weeks from today. Okay, so back to our
original programming. So uh this uh what we did
is we're trying to look at this notion of trying
to commoditize supercomputing uh for the masses, and really
that's actually a little bit of a red hearing.
What I just said is that we're not necessarily doing
. Supercomputing a lot of the work that we're doing
in the embedded space is really just trying to extract
all the performance that we can get out of a
parallel computing environment, whether it be a traditional supercomputer
all the way down to a desktop and even smartphone
or or uh mobile device. And so the whole
uh this is, I'm going to skip over that
. So the whole idea is we want to do
a software ecosystem that supports heterogeneous parallel computing exploits intra
node parallelism and then commoditize is this for everyone to
use. So it's not so difficult that you have
to become a PhD in power beating and heterogeneous computing
in order to enable uh in order to enable the
capability of extracting that kind of performance to support your
work and it can span a number of places.
I mean like I said, we've worked with Professor
YaO and cybersecurity. There's some serious scaling problems there
in terms of tackling intrusion detection and and leaks and
things like that. There's things in the in the
life sciences like next generation sequencing, the amount of
data that's being generated is doubling at a much faster
rate than their computing capability. So depending on what
you look at the amount of data and next generation
sequencing a DNA sequencing is doubling every nine months And
our computing capabilities only doubling every 18 months. So
we can't just throw hardware at the problem and expect
it to to to be able to bridge that.
We have to take a more fresher, more innovative
look at how we can uh how we can leverage
the hardware combined with system software, algorithms and applications
in a unique way in order to bridge that gap
. And we're gonna do this through parallel computing and
it's everywhere now. I mean it's gonna be it's
almost impossible now to go out and buy a computing
device with a single core inside of it. Pretty
much every phone now is multi core. All smartphones
are multi core laptops are all multi core and so
on. And so what this amounts due ubiquitous parallelism
. And the key is how to extract that performance
and scale out. Um so one way of doing
this as well before I get to that. So
what this amounts to is in short is that our
free lunches over, what we used to do is
we just go out the best buy and we get
the next computer that has the next higher gigahertz clock
rating and we would get faster performance, but we
can't do that anymore. And now the burden falls
on you all to figure out how to exploit the
parallel hardware to get the appropriate performance or the necessary
performance. Um And so the question here is what
can we do as researchers in parallel computing to lower
this cost of concurrency. So one way of going
about doing this, this is the Berkeley view,
it started out at Lawrence Berkeley National Lab and it's
now part of uh the UC. Berkeley landscape.
The idea is that applications target existing hardware and programming
models. But what we should do is we should
design hardware that keeps in mind future applications. So
the idea is rather than build the hardware and then
evaluate the hardware with various benchmarks and applications, we
instead think about what future applications there will be and
have that influence hardware design in the future. And
how do you go about doing that? So what
is this this magic uh software? Well, what
Berkeley posited was? Well, If you look at
least signed at scientific codes, there's 13 computational dwarfs
as they call it, where they view a computational
dwarf as a pattern of communication and computation. That
is common across a set of applications. So those
of you that are uh taking or hopefully have taken
an algorithms class the theory class. You think about
the various types of algorithms that you leverage in order
to achieve, uh what your end goal is.
So you can think about and body problems, uh
you can calculate the interactions between every pair of n
bodies within the universe and cosmology. The gravitational pull
between every pairs of n bodies. You can then
take that same execution signature, shrink it down to
the molecule level and look at understanding the electrostatic surface
potential interactions between different pairs of atoms within a molecule
. To understand the proclivity of another molecule to bind
to a particular site. That's that's an example of
one computational dwarf in end body one that it spans
from the large in outer space down to the very
tiny that you can't even see in atomic and molecular
space. So these are these are the ones that
came out in fact. So why did they get
started calling by being called dwarf? Turns out there
were seven original ones. And that's the reason why
uh got dubbed as dwarfs and Snow White and the
seven dwarfs for those that aren't familiar with the literature
. And 1/6 additional ones got added since then.
The purpose of what I'm going to talk about is
not to to go through each, each one of
these in great detail, but just to give you
an idea that what we're looking to do is in
creating our software ecosystem. We're going to use these
dwarfs as a guide with which to help us uh
Personalized super computing for the masses through this notion of
um het originate of hardware and software that automatically tunes
for uh tunes for performance with respect to performance,
power and program ability. And we're going to do
it. The this benchmark of sweets. Uh the
dwarfs that I just talked about except our own version
of it and I'll make allusion to it Now,
I also talk about later is called open dwarfs.
So if you do a google search on open dwarfs
um with the F. S. At the end
, instead of the es you'll find an open source
project they're available for people to use. Um So
we're gonna look to have a multidimensional understanding of how
to optimize uh these different aspects for each one of
those aspects or some combination thereof. And we're doing
it from embedded space all the way to the data
center space. Um In particular, what we're seeking
to do is uh create an ecosystem, such as
the one that's noted in the in the lower box
here, the software ecosystem to support the myriad of
applications that we've been collaborating with. Um So most
recently we we landed a very large, multi million
dollar grant to support this work, we're doing an
avionics composites and they're used in these micro drones.
So the thing is that it seems kind of silly
for us to stovepipe a solution for each and every
one of these applications and so we should create uh
abstractions of you oil. And this is what computer
science is all about, create abstractions and and powerful
enough tools that will be able to support all of
these um uh different types of applications. And so
when I'm going to give you a very, very
fast um breeze through of this ecosystem is shown here
in this yellow line. But what I want to
point out is that we're gonna talk about the dwarfs
and we're gonna use that as a way to not
just guide hardware design, which we have been doing
uh, uh, indirectly through an NVIDIA AMD and
intel they have shown interest in specific dwarfs and,
and how that's going to then uh, affect their
architecture will have to see, um, uh,
we have a source to source translation and optimization framework
. So all of those applications up there, they
write their parallel codes in different languages. So,
and there are times that we need to bridge the
gap between the languages that their programming in and the
languages that are underlying that underlying those languages or support
those languages for heterogeneous parallel computing. So it's like
, yeah, it's like if if you have somebody
that speaks english and someone that speaks Swahili in the
same room or you have to find that bridge,
you need a translator to bridge those two. And
that's what this translator is seeking to do, seeking
to bridge what the applications are doing with what worse
doing software wise with heterogeneous computing or in this particular
case, uh, graphics processing units. So we
do the source, the source translation where the universal
were not a universal translator. We love to be
a universal translator, but that's it's a non trivial
task. But we, we carve out the piece
that we're able to try and do and we have
this translator and then what we try to do is
we're trying to optimize this for the user automatically.
Okay. Um, we, we haven't been able
to do it automatically yet. This is more of
a cartoon or a vision of what we see going
forward we're doing right now. Is these architectural optimizations
? Um I had a student uh last year um
who finished his Master's degree and I basically made him
a human compiler. So there's a list of optimizations
that he had to go through. And I said
, all right, I don't really, I'm not
a computer expert, so I don't know how to
formalize this and automate this right now. But we're
going to pretend to do is you're going to be
a human compiler. You're gonna go through all the
different permutations of optimizations that will allow us to improve
the performance of the code. Okay. So um
so he went and did that. And so we
we ended up first initial parallelization. Just source the
source translation would get us 88 fold, speed up
on a GPU. And then once he applied all
his architectural where optimizations, which took It took many
, many weeks. Uh he got another 4.2 fold
. So the aggregate The performance improvement ended up being
about 371 372 fold over cereal. A serial vectorized
execution on an intel processor. Um A lot of
this these optimizations were trying to formalize into a model
so we can decide based on the model if we're
optimizing for performance or power or energy efficiency or both
uh performance and power at the same time, we
have a model that then guides where we map the
the tasks onto the processors. That is is the
CPU better for this task or the gpu in your
own brain? It automatically does it it figures out
your left brain or right brain what you're using but
we don't have that luxury in this kind of system
. We actually have to figure out which one is
better suited for a particular task at hand involves a
lot of modeling involves some profiling to understand how different
things mapped to the underlying architecture demands an understanding of
how data structures are represented and how the algorithms map
underneath in order to get the good performance um in
the last box is this task scheduling system, the
idea here, is that a lot of the things
that you think that you would like to paralyze that
design and compile time cannot be fully on. All
the parallelism cannot be fully uncovered because there's all sorts
of data dependencies and control pads that you're not sure
about whether or not you're going down or not and
you won't be able to expose that parallelism until runtime
. And so we have a task scheduling system here
that will on the fly uh in real time decide
what mapping is to make between the tasks and the
Cpus and Gpus and other heterogeneous uh computing uh devices
in the environment. I'm not going to talk about
this last bullet here but this gives you a high
level view. The questions on this high level view
. I'm gonna You keep talking here a little bit
to stall and give you a chance to think about
if you have any questions. But this is this
is a high level of you and what I'm gonna
do is just gonna give you like 2-3 brief slides
on each the box is to give you an understanding
of what it is that we're doing in each space
. And what I will say is that a lot
of this work is it's they're all related right now
but they're not connected the way they're connected here.
Right? This is this is what we would love
to get to in the next 3 to 5 years
. Okay. And that's that's part of the recent
latest large grant that we that we've gotten but we
have pieces of this ecosystem put together. Okay,
so I'll talk very briefly about the dwarfs. So
this is I already gave you this example um an
example of a computational dwarfism, body and body problems
are studied in cosmology, particle physics biology engineering.
So we did one with the cosmology. Uh This
is outer space and then one with molecular modeling,
this is work with Professor alexei, you know every
f who's a professor in computer science here. Um
They all have similar structures and this benchmark can provide
meaningful insight to people in across these fields and the
optimizations that we apply in this particular realm also will
apply in this realm, even though they seem like
they're very different fundamentally from an algorithmic standpoint. They
share a number of similarities that allow us to apply
the same optimizations to both spaces. So the first
instance creation of the are dwarfs is uh it was
originally called open cl and 13 doors, since change
is called open dwarfs. Um So we provide common
algorithmic methods that is dwarfs in a language that's write
once run anywhere. Okay. And I should say
write once run anywhere in the city notion of the
word? Not in the job on the word.
Okay. So the idea is that if you have
a c environment uh tool environment on your target host
, you'll be able to uh recompile the the code
and run it on that new target platform. And
this is part of a larger umbrella project um for
the NSF Center for high performance, reconfigurable computing.
This is a joint uh effort by Computer Science and
Electrical and Computer Engineering, um and we are being
renewed for another five years by NSF. Um This
is where we're at right now. Um These are
the ones that are done, there are a few
that are still in progress that we're trying to complete
population of uh so that people have a full set
of dwarfs to look at what I think has been
interesting. I'm gonna throw I'm gonna open up a
Pandora's box because these applications are just there written by
hand, there's a non optimized um there's not a
target architecture in mind. Uh they're just generic algorithms
and we decided, well let's just take these and
see if we could use um use this to run
on other platforms. And the reason oh, let
me back up one minute. So the reason why
open Cl is the open computing language, is because
if you write your code and open C l you'll
be able to run at on any supporting infrastructure that
supports open Ceo, just like any infrastructure to support
C. But you can run it right now on
Cpus Gpus, F P G A S ap us
, which are accelerated processing units, you don't have
to rewrite your code across all of these as little
as two years ago, you would be writing p
thread codes for your see your CPU and then you
would have to report what parts you want to accelerate
and write it in something called cuda and you run
that on your Gps. Oh, and then if
you want to run on f P G A s
, you have to translate that and running right,
uh in V H D L or very log.
So the portability aspect of being able to run any
parallel computing devices just gets to be a pain.
And so it turns out that Apple was the one
that came up with this open cl language because they
were leveraging the graphics processing unit to run a system
. So I come up with open cl well using
the GPU to some operating system tasks. And so
when they had a video GPU, they were programming
in cuDA. When they decided that after the three
year contract was over, they said, oh wait
, you know, we want to use an AMG
GPU because it's cheaper because and video, it feels
like they have us over a barrel. Um and
so we're gonna switch to an AMG well then they
have to switch to brook GPU programming language. So
rather than have to deal with this overhead of having
to continually translate back and forth, they created open
seal as a common language that would just run on
Cpus Gpus and what have you. So you can
imagine the Macbook airs, there's not an extra GPU
to run uh the operating system on it will just
back off and run onto the CPU, it will
run on any parallel computing device that you have underneath
it. So we did this and we just tried
to do this and this is just a really small
digest version. We tried it on uh we use
the Open Sea el software development kit from A.
M. D. And from intel and we ran
out on intel processors. This is just a straightforward
benchmarking exercise. And you see it's what's pretty interesting
is the amount of time that it takes to do
each one of the different dwarfs. Um So,
you know, C F D is, is a
structured grid dwarf, and you see that the AMG
has much better performance, it completes in two million
less than two milliseconds, whereas the intel one largely
completes in four milliseconds. That's a that's a that's
a quartile box between the 1st and 3rd quartile,
But there's some significant outliers all the way out at
12:00. Yeah, what we ought to do it
, but we haven't had time to do is really
trying to understand why this is going on, but
we just really put this out there for people to
, to to catalyze further research in this, we
don't have time for say to do it ourselves,
although, and if you are interested, uh we
can get you hooked up and get going. Um
So, other interesting things is that it depends on
the dwarf, you see over here, until platform
for the Fft fast way to transform, which is
a spectral method does quite well, it's 0.7 milliseconds
and that the AMG one does quite poorly and this
is on the CPU we're running open cl code on
the CPU in this case and we thought that it
would be, we thought for sure that the intel
platform would win across the board because even though we
were using vendor specific sDK s A M D S
, D. K. And intel sDK we were
running it on intel hardware, so it turns out
this case, that isn't always the case that the
intel will run better even though we have it rigged
in some sense to run better on intel. All
right. So that's just a flavor. Uh I'll
give you another perspective on the power, I just
talked about performance. This is a power graph.
So this is an F P. This is a
CPU you can see it F P G A which
is a programmable processor. You can program the devices
on the F P G A to configure how you
want them. These are two high powered Gpu s
and you can see here are too low powered Gpus
and Cpus. So this Mac mini has a CPU
and Gpu. It's not on the same die,
but it's on the same package. You can see
how energy efficient that is. So, when I
talk about heterogeneous parallel computing, I really am talking
about it from the perspective of small mobile embedded spaces
, desktop spaces all the way up to high performance
computing spaces such as these. And this one is
hard to see you there and if you want to
find out more, um we have a number of
publications on the different dwarfs, uh you just go
to synergy dot CS dot VT dot e d U
. And just click on publications and you can do
a search for that. Um the we have a
summary paper that that was submitted and published this past
year. It's just a brief uh research note.
It's only four pages that gets you uh orientation in
terms of what we're doing and then the publications that
are underneath it are are deeper dives for a particular
dwarf. Well, okay, so I'm gonna move
on to the next box. Um but before I
do, I want to point out that this is
what we have of the dwarfs. And one of
the problems was that in order to generate all these
dwarfs, it took two years of students uh contributing
to this sweet to develop. Now it wasn't two
years of full time student. I mean you all
have taken classes, you go on internships and things
like that, but this is just getting an idea
. It took us a while to do this and
on top of that, um when I noticed that
we were able to uh once you Take the notion
of the code and try to optimize it with respect
to the underlying target architecture, we were able to
get an additional 4.2-fold speed up. So we went
from 88 fold to 3 71 fold. So what
can we do with respect to leveraging these dwarfs uh
to affect better performance? Well, we'd like to
get the time that it takes to develop these things
down and we'd also like to eliminate the need for
the end user to have to optimize and that's where
we get into this source of source translation and optimization
framework. Okay, so we're not going to try
and be the universal translator. That's not something that's
probably ever going to be done in my lifetime or
likely your lifetime as well. But um we're looking
in particular at codes that have been cuda accelerated uh
on video gps and we want them to be translated
to a language that will run anywhere and that would
be open cl and so if you take a look
at the prevalence of code, you see a cuda
code, um, I should update these numbers but
you can see that the cuda source code has had
over a million results. This was back a few
years ago actually and then open cl source code there
was were significantly less. And so if you went
to the video website, you just see all these
applications that are being accelerated. Um and many of
them are captured by some aspects of the dwarfs that
I've been talking about. Um, so everything from
more scientific types of things like life sciences, um
, uh two government and defense to financial markets to
oil and gas. So what we did was we
created something called Cuticle, it's a coded open cl
sources source translator. It's implemented as a clang plug
in. Uh Klang is a, is a production
quality compiler framework and and we leverage is heavily because
it's like there's no sense in reinventing the wheel,
These people are compiler experts, we know uh we
know that what they've done is been tested and uh
so we're relying on that framework and then we make
Collins to this clang framework to support the primary cuda
constructs found in cutesy and the cuda runtime Api.
And what we found is that the codes that were
manually ported that you saw on the previous page that
were manually ported or written uh that we could we
could take the ones that they reported. We could
do is automatically and get the same performance. So
the work that took two years for students to do
, we can do in seconds now. So the
students have been obsolescence. I'm just kidding. We
of course the students that in fact there was a
very bright uh integrate uh integrated B S M s
student who worked on this project uh to realize it
as an automated translator, There have been many requests
for other translators, but there's only so many,
so much time in a day. Um we we
we bit off the largest chunk and that was recruited
open cl but there's been movements to getting open cl
to Kuta and open MPI open C. L as
well. Um I'm not expecting everyone to understand this
picture. I'm just, this is just a high
level overview. Um we're really making heavy uh um
reliance on this clang framework. And so we're making
use of their traverse, identify and rewriting libraries in
order to take this cuda source code over here and
generate appropriate open cl host file code and open C
L kernel file code. So the host code runs
on the CPU and the kernel file codes run on
the GPU. Um City. Oops, so just
to give an idea of how this is done so
far, um it's done pretty well. So the
number of lines of code is not the total lines
of code, this is the number of co two
lines of code. Um so for example, uh
professor of movies embody molecular modeling code is about 7500
lines of code, of which 2500 some are for
the GPU. And we were able to translate all
but five of those automatically. So you can imagine
having to do this manually by hand versus oh wait
, It's all done. Other than five lines,
I have to go in and twiddle five lines and
make it run, We've since done this, we
were up to 100 applications now. Um it's not
perfect, but it helps you get a good part
of the way. We don't do any transformations to
the code. We're doing source the source. So
what you have in cuda code still looks the same
when you have open cl code. And so you
can develop then from the open cl code at that
point. For more information about that. Uh you
can go to uh this publication um on cuda to
open cl sources source translation. Now That takes care
of the 2009 to 2011 time that it took to
develop the dwarfs and compressed it to seconds. Okay
, um what about the performance aspect? We saw
that 88 fold, 371 fold improvement. Well,
that's where the architecture where optimizations came in and and
this part we have not automated. This is something
that we're looking to automate. But I wanted to
give you just a brief flavor of what it is
that we're doing on architecture optimizations. And so,
uh, this is probably hopefully self evident, but
the performance of different CPUS are not equivalent. Neither
are different GPS. Um, if you take a
look at the peak performance of an NVIDIA Gpu versus
an AMG Gpu for single precision floating performance. You
see that the AMG GPU is supposed to be theoretically
twice as fast as the NVIDIA Gpu. Yeah.
When you come over here and you look at the
speed up with respect to hand tune SSC, which
is uh, you're using essentially the vector rising extensions
in a serial processor. You see that we got
328-fold speed up on the GTX to 80 when we
moved it to a newer, allegedly faster GPU card
, we took a performance hit. And the reason
is the optimizations that were valid for this older and
video architecture don't necessarily apply anymore to the newer architecture
. They applied well enough that you still get about
the same performance, but they don't necessarily apply enough
that you have better performance and it was even worse
with the AMG Gpu. Okay. Um it turns
out that some of the, some of the optimizations
did apply, you got some speed up but you
didn't get as much speed up as you would expect
because the peak performance powers is supposed to be twice
as good as the NVIDIA. So if we were
to extrapolate here, you would expect 328 times two
. You would want it to be whatever is 656
times faster, but it's only 224. Okay.
Um so uh I'm just gonna skip that one.
Let's go here. So, so what we ended
up doing is we've identified a number of uh manually
my human compiler uh and I we worked and found
a bunch of different manual optimizations with respect to optimize
AMG Gps. And I should say that this is
for the old generation because the new generation, they
completely yank the rug out from under us and have
changed the underlying architecture completely, but you don't have
to understand all of the different uh optimizations here.
Um all you're going to point out is um that
we applied a lot of these in isolation as well
as in combination. And what you see is that
we see different speed ups entailed by the different compiler
optimizations, I guess. Maybe I'll just take one
point is like, here's an example, loop unrolling
two way or four way. So what that does
is you have an iterative loop And you executed 100
times. Well, one way you can do it
is to avoid having the overhead of having to do
with the branching back to the top, you unroll
it a second time, so you take that loop
and you replicate it now, instead of doing that
single loop 100 times, you're doing two copies of
loop 50 times and you've eliminated a branch point.
Now the trade off there of course, is that
now your code is larger, it's going to take
up more space. It may then push out data
or other code that you need to execute and push
it out to disk, which then impacts your performance
with respect to the code because you're swapping from memory
to disk. This is all trade offs. Okay
. So we then combine them in different ways and
ultimately we ended up with uh combining these particular ones
, we got a 4.2 fold speed up when my
student did the first part, he was really excited
like, oh, now we can combine all these
, we'll get this huge multiplication speed up. Yeah
, I wish, I wish it was that easy
. Um It turns out that some of these optimizations
are are conflicting with one another and so you have
to find out which ones figure out which ones are
more amenable to combination. And so ultimately, here's
the summary slide for the architecture optimizations. So we
went with a basic implementation, that's the speed up
over a hand tune S. S. C.
Code architecture underwear means we'll just use the optimizations for
the other platform on the current platform. You swap
them, just see you see how well they do
. And so some of the optimizations do transcend the
architecture and that they apply to both architecture. But
then when when we actually tune it with respect to
the individual architecture, um so you remember that original
2 24 now is up to 3 71 fold speed
up Barbara? Yes, like where? Yes.
Excellent question. So the question was uh with the
Gpu is changing so quickly, how much, how
much do we get out of these optimizations? We
get it for about a year and then yeah,
and then we have to go on to the other
ones. Uh So the the answer is, I
mean yeah, like 18-24 months. However, because
of the way that the heterogeneous computing environments, particularly
Gpus have been evolving. They've been converging more to
similar are similar like architecture so much like the way
you see intel and am D Cpus are reasonably the
same. Um They may be implemented under the covers
a little bit differently but they they still have some
of the same artifacts the notion or the hope is
is that many of the architecture independent optimizations that are
across all GPS will then apply and that the library
or set of architecture dependent optimizations will be hopefully smaller
. Um The tricky your part is going to be
is a lot of the performance modeling in terms of
movement of data because right now the Gpu, a
discrete Gpu like on hoagie speed sits out on the
Pc I express for us. So you have this
overhead of moving data back and forth to compute.
And there are other architectures now that put the CPU
and GPU on the same guy, it's like,
oh wait, then I'll have to move the data
. That's true. But you have fewer Gpu cores
and the memory that you're using is slower on the
discrete one that memory space. So you start to
think about, oh wait, you've got to model
all this so you can figure out All right.
If I know this is what the application signatures like
, I know that it's gonna bang on the CPU
a certain amount or the computing part, a certain
amount of memory. Certain amount. I have a
better idea of the costs involved in moving data.
Computing data in terms of trying to decide if I
want to use the GPU cores that are on the
die or that our remote on the discrete Gpu roger
in the last Yeah. Mhm problem in the course
years because of power constraints that so that your run
the course slower or blank, that's it also have
to select operating of course you're That's right. So
the comment question by Professor eric was it was uh
there's there's forecast that the doubling of the number of
courses at some point going to reach an end.
Maybe it's this coming decade, maybe it's the next
. Not sure yet. And then the question ends
up being, well that may end up, you
have to slow down the process of speeds and and
and try and save energy because you're reaching a power
limit. There may be other technologies in the area
of quantum computing that might come to bear. We
don't know. Um but part of part of what
we're trying to do, in fact, with respect
to uh let me show this with respect to this
box here which is in purple, the performance power
models is what we're trying to model. The processors
, the CPU and Gpu in a way that we
can manipulate the level of parallelism. The frequency and
voltage is with which they operate on so that we
can optimize for power as best as possible. That's
one step toward it. I'm not saying that we're
completely answered it but it's a very good point.
Well, it's it's exciting times in another 10 years
. We'll have to see what uh what what the
future may bear. So this is where uh this
particular paper is uh submitted and future work is ideally
as we want to combine these aspects. Um I've
been uh in discussions with a colleague uh compiler colleague
who is really interested in the notion of instead of
just looking at blocks, uh control blocks uh to
optimize between control points. If we can then go
beyond the control blocks and start to look at these
notions of dwarfs and try and get a medal level
understanding of how we can approximate the better performance by
doing certain mapping of data and computation in places.
But that's a really, really difficult problem um that
I am ill equipped to answer but that's why I
have my colleagues and the compiler area to help me
out um performance and power modeling. All right.
I mean there's a breeze through these last couple of
30 time. So this just gives you an idea
of some of the challenges we face on the y
axis you have execution time on and energy consumed uh
and on the X axis the first number is the
number of Cpus and the second number is the number
of threads per CPU So the idea is you think
, okay, we have parallelism. So if you
throw more processors and more threads per processor at the
problem, you should get better performance. That's not
the case in this case is the energy sort when
I used all four cores and two threads per core
, I got the worst performance on top of that
, I consume the most amount of energy. So
there's a lot of space here to try and figure
out from a modeling perspective, how do you automate
this process so that you can optimize either performance Power
or some hybrid of the two co optimization. Right
. Of these two things. All right. So
we're looking at a framework that has higher accuracy prediction
, identify portable predictors that will go across different architectures
um and then do a multi dimensional characterization of it
. Uh sequential code, intra node parallel inter node
parallel as well as looking at different power aspects,
which is this is the part that google has been
particularly interested in of late. We're able to through
software monitor power consumption of subsystems within a computer without
having to open it up and attach a silla scopes
and digital digital meters. Um This is some algorithms
. Uh so we actually cast this as a linear
programming problem for for the work that we've done um
eliminating the notion of multi core for the time being
. We're just looking at multi processor. Um and
I'm not going to get into the details. I'm
just gonna say that we applied that and we got
results such as the following where um we're running on
a bunch of different parallel codes And we affected,
we put in a bound of less than 5% performance
uh impact. We violated it in this one case
, in the cognitive gradient case it was more like
an 8% impact on on performance. But what you
see is on average we saved uh 19% uh of
of power consumption, I'm sorry, energy consumption.
In one case we actually got energy improvement, which
is yet something else that we didn't we haven't looked
in and in great detail. Um so the last
thing I'm going to wrap up with is this notion
of the task scheduler, which is heterogeneous task scheduling
system. And so what we wanna do is automatically
spread tasks across different heterogeneous computing units from Cpus gpus
JPs we want this automatic so that uh application scientists
and engineers don't have to worry about the mundane details
of doing this mapping and so you want this runtime
system to intelligently use what's available resource wise and optimize
for performance portability. Um we're taking a languages approach
to this. Um right now, initial point is
we're focusing on open open mp. Um it's a
directive based model. I'm not expecting you to take
understand all of this, but the point is,
is that um what is being what was done traditionally
is an opera left. What's been proposed by the
committees is an upper right. And what we're proposing
to add is on the bottom and the idea is
that this just identity helps the compiler identify what regions
are parallel and this is one step towards doing this
fully automated instead of having uh user directives or compiler
directives that are inserted. Um so this is the
way programs typically run you have an open uh,
mp parallel region, you run in parallel on the
CPU, then you join and then you fork again
you said, oh, here's the accelerator region running
on the GPU. So you're running either on the
CPU or the GPU. But the semantics of the
language are such that you can't do both at the
same time, which is what we would like to
do. We want to be able to it will
run both on the Cpus and the GPS at the
same time. And so there's there's actually a linear
linear optimization scheduling model behind this runtime scheduling system as
well that I'm I'm not going to get into here
. I'm just gonna show you some results. So
, these are codes that have already been paralyzed at
design and compile time. All we're going to do
is see if we can extract any additional performance out
of it at runtime automatically. And so it's all
, it's all um normalized to running on the CPU
. So Cpus speed one and what you see is
that on the Gpu alone, uh, we do
worse than most of the case in two of the
cases and we do better in two of the cases
, Gem and K means. And that's because these
are more amenable to parallelization on the GPU, the
four other ones, which I won't get into the
details of their all four of these are dynamically trying
to decide through either a static decision making process or
dynamic one and then some some uh variations of it
. They're trying to decide how much work to put
on the Cpus and Gpus concurrently, simultaneously running at
the same time. So what you see here is
that That parallel code that we got 371-fold speed up
on now at runtime we get an additional eight fold
speed up automatically through our runtime system. On the
other hand, you see here that the CPU was
best for the code. All the other ones did
worse and that's it. But our auto adapting one
did pretty well. The reason why I didn't do
quite as well is because there's time involved in solving
the linear uh optimization problem to decide to move all
of the work onto the CPU. All right.
So that that's a recent publication, uh you can
look at a heterogeneous task scheduling for accelerated open mp
. Um We're actually generalizing it to open cl and
in general parallel computing environments. So there's a ton
of work that's still to be done. Um And
this is just the part that we're doing with respect
to heterogeneous computing. Uh Like I said, we
have we have two projects and cloud computing um we
have a lot of things going on in the green
computing space as well. But this is where we
are at. And uh we're looking to go next
. And uh that's a sampling of publications, which
I'm going to. You're not gonna be able to
grok all this, but you just go to synergy
dot CS dot VT dot e d u. And
you'll see a number of these publications. Um be
remiss I would be remiss if I didn't acknowledge a
number of the funding agencies that have supported this generously
supported this work. Um Especially some of my really
aggressive and maybe uh idealistic views of where you'd like
to be. But this this these funds have helped
support us trying some things that we might not otherwise
would have tried. And if you want more information
, you can go to any of those websites and
my contact information is above. Okay. Mhm.
We have time for one more one question or should
we let him go? Yeah, go ahead my
Yes, yes, yes, that's an excellent question
. So I'll just summarize it. The question was
really about uh these uh these optimizations are well and
good uh but their system level more or less than
and there are things like data dependencies that occur that
will uh not necessarily allow those optimizations to apply.
And so that is definitely the case. But what
I would say is that um the data dependencies,
we're trying to address it at two levels. One
is back here, the architecture where optimizations, those
are system levels. It's not really paying attention to
uh run time data dependencies because it can't, it's
doing it at compile time but we're hoping to do
is try to get some notion of what the computational
and communication pattern is. So at a higher level
we may be able to apply some crude optimizations that
would say, well, you know what based on
this type of application signature, it doesn't pay for
you to move the computation to the discrete Gpu use
the undie Gpu because the cost of moving that data
for this particular application signature is too high. So
that kind of optimization, we would hope that it
could be done at compile time. I think that
one is going to be the harder, much harder
one to do. Definitely harder for me because I'm
not a compiler person. Um what we're doing here
though is in the runtime system based on those data
dependencies, we're figuring out where the time is being
spent and we're automatically reallocating where the tasks are on
Cpus and Gpus and Cpus at runtime based on the
execution control flow through live, live execution of a
program. Yeah, we're doing Yeah, yeah.
So we could do it. These are excellent questions
. Those questions like we're trying to manually do this
and in some sense, initially we did this manually
because we wanted to understand the effectiveness or the efficacy
of potentially making use of all these devices and if
we were omnipotent, how we would go about orchestrating
who gets what now we've gotten to the point that
we learn this through the runtime system. We see
how it's executed the past and we learn and try
to predict how to divvy up the work on the
different Cpus and Gpus and how much of it to
divvy up on the Cpus and Gpus. Okay,
so that's all, that's that. That part is
automatic right now. Okay. Yeah. Yeah.
You want right. Oh I right. Uh huh
. Mhm um, so, so the question is
, how would companies be leveraging cell phones and mobile
platforms and what have you? I mean It's not
necessarily the case that those 97% of the companies are
making you specifically of mobile phones and and desktop systems
. Many of them are just making use of servers
or the cloud or what have you. Um but
in the case for if you're trying to think about
ways that companies could make use of it. I
mean you could think of uh, if you had
some special iphone app or android app that does real
time financial data analytics that allow you to figure out
whether or not you want to trade a stock or
not. That's one example or real. I'm uh
weather forecasting and we've got to go to the weather
, I'm really coarse grains slow being forecast but maybe
sometime in the future you'll be able to use the
capability that you have on your phone link up into
a also as these heterogeneous resources to be able to
do your own localized weather forecasting within the new River
Valley area, for example, a little bit pie
in the sky. But let me just give you
an idea. All right, well, uh,
thanks. I'll stick around for a little bit.
I, uh, I've got to run to actually
, I'm going to go to another meeting in two
minutes, technically, but I'll stick around for five
minutes or so if you have any questions. All 1190 01:08:18.350 --> A:middle L:90% right.