Today we have a little bit of an experiment. Our department has been selecting research awards for
best dissertation or best thesis for a number of years. This year we decided to ask the two
recipients to give a talk today about their work so you get a sense of the top research in the
department both at the PhD and the master's level. The two recipients are Alex Endert now at PNL for
the PhD and Paul Sathre who's here in the front row for the master thesis. Alex is on his way
to France to the KAI conference right now so he said he couldn't stop in Blacksburg unfortunately
but he recorded a video at the TED conference that was took place here I think a year ago or so
so we have that video so I'm gonna play Alex's presentation at the TED Virginia Tech conference.
We won't be able to take any questions because Alex's advisor is on sabbatical so there's nobody
here that knows how his work but then we'll follow that up with Paul's presentation on
his master's thesis and he's here you can ask him lots of questions okay so we're gonna get
started I'm gonna put the mics down and start the video let me know if you can't hear it.
We as humans have a great ability to understand information at a level that really makes sense
to us and other humans we can look at a stack of paper start to read through it figure out
relationships between documents between text and really understand these implicit relationships
not relationships that are directly correlated or based on statistics and probability
but relationships that are based on our own experiences our own mental models and the way
that we perceive the world and the information around us a lot of us have been in a situation
like this whether it's your digital inbox full of emails from returning from a trip on a vacation
um getting back to the office and seeing a big pile of paper and then organizing it somehow
laying it out in a way where you begin to understand and project your mental model
onto some sort of canvas or onto some sort of space around you there's an inherent flexibility
in this because while you can organize and lay out your information in the space you don't have
to tell anything or anyone or any system why you simply place it near each other because to you
that is where it makes sense to you that's where that information relative to the other information
really occupies meaning in its absolute physical location you can access that information directly
by glancing over by grabbing it moving it near other information even by using things like a
pencil or a highlighter and actually teasing out more abstract pieces of information from that
from that actual text document and narrowing down on what it is that you're focusing on
it is essentially your ability to offload and externalize what your mental model is based on the
information that you're presented into some sort of space the space makes a lot of meaning to you
because essentially you're the one who created it therefore it is an extension of your memory
or an externalization thereof into some sort of surrounding area where that relationship
between those pieces of information hold true given your domain expertise or your experiences
well how about when we start to increase the amount of information
uh whether it's a week-long vacation and you come back and your inbox has tens hundreds
or maybe even thousands of new messages how are you what do you do when you are overwhelmed with
this amount of information now in the physical sense this might seem a little ridiculous
but in the digital age where there's information all around us whether it's twitter emails
news groups news websites we're essentially bombarded with information and lacking knowledge
there's a field called visual analytics that tries to attack this exact challenge essentially by
saying we as humans have intuition that systems do not we have complex computational advancements and
fields such as data mining statistics probability information retrieval and we have this wonderful
ability to visualize information in that same type of space where users have created it so
similar to how you might lay out documents on a big conference table we can computationally
attempt to generate a similar type of view one where every dot here is actually a text file from
a big news feed and then the relative distances between them indicate some type of similarity
you can abstract that out one level and then see these themes so you see that down here you
might have a little more of a government political political theme and then up here in the middle you
might have more about animals and wildlife and the nasa and space and golf and other types of
things all in order to try to mimic that same type of space that we as humans can manually create
except the fact that this is done in seconds where several thousand
news feeds might take a human a very long time to do depending on how fast you read
now where it gets more complicated is that while leveraging and exploiting all this
complex mathematics what has happened is that we create tools similar to this one
where users can then begin to explore the space and what i mean by that is you essentially have
this beautiful representation that we can create where users can gain information from it based
on whatever computational structure these these algorithms have extracted out of the information
and then what we've done is we've placed toolbars and rather all sorts of types of
user interfaces off to the side where users can begin to bias or control the space a little bit
so by saying that in this example you know i'm really not interested in all of the political
documents i'm more so interested in a lot of the things going on with with wildlife and animals
well then i can go over here and i can start to steer or adjust these models and ask for another
representation that more closely reflects my focus or what it is that i'm interested in
however that essentially requires you to put aside your expertise in the data or of the
data and leverage expertise you may not have and myself included in that of these complex models
so you're essentially being required to go outside of your area of expertise
which is whatever data it is that you're looking at and into an area of expertise
where only a few are truly experts dimension reduction models information retrieval data mining
uh connecting the dots graph theory all these types of things that are essentially able to
crunch all these numbers and give you these views but may not be the most intuitive for a user
who's an expert in history or just a casual reader of news groups to try to understand whatever
happened to the space that is directly manipulable manipulable um you can essentially go in and touch
it and directly interact with it in the canvas in which it is created meaning in the space on
the floor on a conference table or whatever this visual spatial paradigm is that we can create
so with semantic interaction let's take a look at how we've changed that you can see
first of all there's no user interface off to the side there's no direct control of the mathematical
underpinnings that create the space instead we are asking the user to just interact and begin
to organize and we have placed the statistics and data mining in a role where it simply listens
to the user and tries to understand what those informal relationships are so by placing two
documents closer together there's some sort of implied similarity and let's try to leverage all
that computational power as a way to understand or try to mine for that understanding and then
amplify it across the entire data set to whereas the user can focus on intuitive
relationships between a subset of the actual data and essentially amplify that out into the greater
data set what this does is it essentially creates this idea that you are co-learning from each other
the system is listening to the user and trying very hard to understand why certain relationships
and certain responsibilities between information is created amplify that onto out onto the rest of
the data set and then learn again from the user based on what has happened so some of these areas
and clusters and groups were created based on what the user has done and others are created based on
what the system has interpreted from the user and then amplified to the rest of the data set
so when we think about how that fundamentally changes the way that we approach this side
this complex idea of knowing uh it essentially says traditionally with these tools with these
direct manipulation direct controls over uh complex parameters we ask questions like this
should i use the first or third eigenvalue of my pca decomposition raise your hand if you know
what that mean all right so there's maybe two or three um now how about something like this
this email from my mom is similar to that one from my dad who here can probably figure that one out
okay so essentially what we're saying here is we're changing the emphasis back onto the user
and their domain and what they actually know and what they understand we're placing the user back
in that critical role within the science of visual analytics that is essentially to leverage their
intuition and their ability to create informal relationships as opposed to specify explicit
and formal relationships which would not really leverage the user's mental model
or their understanding of the world or things that cannot be computationally generated
if we take a look at what this means in terms of just the flow of knowing or gaining understanding
it really changes from one where the user has to think through math so i can't say these emails are
similar these are all the ones from amazon so i want you to infer that this is about shopping and
maybe bring over the other ones from ebay as well instead i have to think through the math i have
to say you know what i need to weight these six dimensions a little bit higher and place a certain
spatial constraint onto my decomposition algorithm that places them in this particular region
and then hit the regenerate button and simply hope that the system has understood
my implicit relationship that i'm trying to mine for i'm trying to understand
instead with semantic interaction we shift it around and we say this is actually a medium for
interaction and communication between a user and a system a spatial metaphor is one where both can
understand and both can communicate through to a user it is simple to understand that two documents
close to each other are similar therefore if i disagree with how that has been mined i simply
move them apart i can highlight directly within the text i can search i can do all of the things
that i'm currently doing as part of my analytic workflow within this one environment and then the
math is placed in the background and saying that let me help the user with the quantity of data
given a small set of data five ten documents i'm sure we shouldn't i'm sure we won't need this
you can simply lay it out on our desk and go i've got a pretty good understanding
of what's going on here but when you say ten thousand ten million a hundred million
it doesn't become very feasible for users to understand that by themselves
so with that i really urge you all whether you are on the user side understanding that second
question that i said or on the researcher side or you're working on those complex
statistical parameters from the beginning to think about how we can advance this
field onto one where the power is placed back onto the user where we begin to shift
that that understanding back into the minds of the user where we can go from having
this overwhelming amount of data yet a lack of knowledge to one where we can take our knowledge
and try to amplify it in order to understand all of the data that is around us thank you
our next presenter
hdmi i did okay
nope that did it all right everyone can hear me all right so uh good morning everyone uh
thank you all for coming i know you're pretty much required to be here but for
the guests in the audience that aren't um thank you for being for coming here to see my talk um
so today i'm gonna briefly talk take about 30 minutes of time to talk about some of
the work i've done over the past year and a half that's gone into my master's thesis
and as the title would suggest we're going to talk a lot about source-to-source translation
specifically related to these two languages uh cuda and opencl and we're going to talk
about some of the issues that make that process quite a bit more complex than it might appear
at first glance so i'd like to start with a bit of ultra high level motivation um parallel
heterogeneous computing is everywhere it used to be that we could just sit around and let the guys
at the national labs with the big super computers deal with parallel processing but that's simply
not the case anymore you actually have to really struggle to find even well even just a cell phone
that has a single core uh most of them these days have multiple cores and even their own gpus
but that doesn't mean necessarily that we're doing heterogeneous computing in that situation just
means that they've got multiple different types of processor cores what we mean by heterogeneous
computing is that you've got multiple different types of processing hardware in a machine
and you're using at least two ver two different pieces of hardware in the same computation to
reach a result and the general paradigm for that we're going to be talking a lot about
graphics processing units today the general paradigm for that is that you start some
computation on the cpu you offload some data to a gpu or some other accelerator device
you let it run for a while you copy the results back rinse and repeat um and this isn't the still
is not just an application that's applicable just to supercomputing and national labs and things
like that it's uh it's gotten so prevalent that it makes its way into software you use every day like
since uh mac osx i think it's 10.6 snow leopard portions of the kernel have been gpu accelerated
as well as with adobe photoshop cs6 so why do we use gpus to accelerate things well this is a slide
i borrowed from nvidia and it's just here to show that for the right kind of problems without adding
any new extra hardware to your machine you can get several orders magnitude speed up sometimes
so gpu computing is one of the major player major players in heterogeneous computing but it requires
special purpose languages so that we can program general uh purpose computations on those devices
and one of the first ones to really popularize that was nvidia's cuda because it allowed you to
program your device kernels in a familiar c-like language a couple years after that apple decided
they wanted to write accelerated computations but not be tied down to a specific hardware
vendor like they would be if they use cuda so they started work on something called the open
compute language or opencl and that's a vendor agnostic standard it's run by the kronos group
the same people that do opengl and several other standards um and that's uh the target language
for my talk today and there are there have also been some other historic languages like brook
plus as well as some newer ones such as open ac which allows you to specify parallel computations
in a director-based way similar to openmp uh but the big paradigm here again is just that
you specify some code to run on the host and then you copy that code down to the device but
before we move on one of the big things i'd like to mention here though is when you write cuda
code you're pretty much stuck running on a video device but if you write opencl code you can run
on that same nvidia device as well as all sorts of other stuff not just gpus but potentially cpus
cell broadband engines fpgas and all sorts of cool stuff like that so what my thesis is about
is specifically about functional portability it's about taking those cuda codes that are currently
bound only running on nvidia hardware and by translating them to opencl effectively
unchaining them so that so that they can run across all of these unique hardware devices
so our overarching vision for the this project is that as an anecdote a couple a couple years ago
myself and several other students in the synergy lab i started work on what's what we now call the
open doors project and it took i think it was about six or seven of us and it took a year and
change to port just 13 cuda applications to opencl so that's ultra tedious it takes a lot of time and
you spend most of that time just debugging a new opencl code you've written so it's really
just not cost effective due to that translation from kuda dope and cl by hands what we want to do
is get rid of that manual process or at least reduce it as much as possible by providing a tool
that can automatically perform that translation for you in several orders of magnitude less time
so to digress just a little bit i'm a big fan of open standards and open source software and
that really played a huge role in my thesis work we've already talked a little bit about opencl
it's an open standard like i said but the big advantage of opencl is that it finally
gave us the write once run anywhere paradigm for parallel computing and one and one of the
the key motivations for our work is that source code generally outlives hardware so if you're
writing your source code in hardware specific language if that hardware goes bust then you're
stuck either running on old hardware or having to manually port your code to a new language and
as an example of that i spent one summer at los alamos and pretty much
the whole summer i was tasked with just porting this old cosmology code that had
had been built for operon cpus and the cell broadband engine the same thing that powers
the playstation 3. i was tasked to port that to opencl so that took about three months
but then once it was ported to opencl with just a few minor tweaks we were able to run it
simultaneously on the video gpus amd gpus and the ibm power 7. so that's the big advantage of using
opencl is you can write the same code and run it on all sorts of cool devices assuming the vendors
provided an implementation but another open source project that's really really important to my work
is that the clang and llvm open source compiler project um it's really important
sorry it's really important because this work essentially wouldn't have been possible without
them we rely on clang to do all our cuda parsing pre-processing lexing and and other associated
compiler features and that's allowed us to build a robust cuda dopencl translator in just
about 3 400 lines of code which is staggering when you think about the size of the problem
uh so for those of you without the biggest background in compilers your source code
actually goes through several distinct stages from when it's high-level source code before you can
actually actually execute it on the device and for coot and opencl and other c like languages
the first thing it has to do is go through a pre-processing step so that you can bring in all
your header files that you've included and perform all your macro expansions and that that code then
goes through alexa where it gets chopped up into tokens and those then get passed to a parser where
a parse tree is generated and then it goes through several other stages uh some semantic analysis to
generate an intermediate representation hopefully some optimization and then code generation before
binary comes out the other end so uh we built this tool a previous master student student started it
and i've been working on it since it's called cuticle for cuda to open cl and where it fits
into this claim compilation process is right after the parse tree is generated it's implemented as
its own ast walker so it looks through this parse tree specifically for cuda structures of interest
that need to be translated and pretty much ignores everything else so to give you a high level view
of where cuticle's been where it is now and where we would and where we would like it to go
the first alpha came out with the previous masters student gabriel martinez i believe it was the
summer of 2011. and it was well designed um very well designed actually but it was pretty much
just a proof of concept it wasn't production ready and it couldn't handle all the code we
cuda code we tried to throw at it so two years later uh with all my contributions i've i'm going
to talk about some of my efforts to look at a very large population of cuda applications specifically
searching for uh source code structures and programming practices that were difficult
to represent in opencl so i took that knowledge and then reintegrated it into the translator to
dramatically improve its coverage uh reliability as well as robustness and we and that work's
been published at uh uh p2s2 workshop at the international conference on parallel processing
and my work as well as gabriel's work is all wrapped up in into a journal submission for the
journal parallel computation that should appear sometime this year and we've gotten to the point
where the tool's gotten so robust that our funding agencies have had access to a beta version of the
binary for about five months now and we've got some big big plans for the future but i'm gonna
get come back to that a little bit later so in the right situations translation is easy
but that relies on it being the right situations essentially meaning that there's no ambiguity
in the translation between those language for example um say this water if you're translating
spanish there's a clear one-to-one correspondence between water and agua so that's pretty easy
that's one example of when translation is easy another example is when there's a well-defined
rules for how you map a high-level structure onto multiple low-level structures like when
you're taking this c statement and translating it to the llvm statements below that and then
finally one other situation where translation is pretty easy is when a single high level statement
maps down to multiple low-level statements that each have a one-to-one correspondence in whatever
target language you're working on so you can uh you can translate those low-level statements
to build one one larger statement in the uh output language so i've got an example of a cuda opencl
translation at the bottom there and i'll explain that in a little bit more depth so the way cuticle
works is it walks along this ast looking for structures of interest that need to be translated
so it looks at this cuda function call it's just a simple uh hardware intrinsic power function takes
two arguments for the base and exponent so it looks at it and sees ooh we've got a we've got
a cuda kernel function power so all right we're gonna need to translate that but first we need to
see if any of the children need to be translated it sees that it's got two arguments they're both
array references but each of those is going to have an index so we have to look farther still
we see that we've got this thread index structure which is cuda specific so and then we have to look
one step farther and we see that we're using the x and y fields of those structures so to perform
the high localized string based rewrite we have to work backwards from the bottom up so first we
translate the x and y fields to zero and one which are the appropriate arguments for the get local id
call which is an opencl call that directly maps to the thread index structure in cuda
we back out one more level we pass through the array references without any changes because
they're just standard c and then we back out one more level and change the name of the of
the power function to match uh the appropriate version in opencl to give you the final version
so that's a a loose overview of how cuticle actually does what it does
but unfortunately there's some situations where translation isn't easy and
as kind of a counterpoint to the previous slide that's when there is no clear
when there is an ambiguity between the languages or there is no one-to-one mapping of a structure
onto a structure in another language um so for example uh one real-world example of that is when
a word is heavily overloaded like love i mean i i love i love my work i love my dog and i love to
eat good food but in those situations it all means slightly different things so it's more important
uh to take the the connotation the context and the meaning that
you're actually implying by that word than the word itself at face value
so um just like with natural languages programming languages can have idiomatic expressions like
for example if you're trying to translate the american colloquialism of putting all your eggs
in one basket we don't literally mean you've got this gigantic basket and you're putting a whole
bunch of eggs in it what we mean is something more associated with you should diversify your risk
so similarly programming languages also have idiomatic expressions that are quite frankly just
difficult if not impossible to map onto another
language for example cuda's thread fence is one of those situations
and then third situation when translation isn't easy is when dealing with dialects for example um
there's all sorts of different versions of spanish you can totally correct me on this if i get
anything wrong here but say you built a translator assuming the latin american dialects of spanish
in that situation most likely you could safely ignore the second person plural informal form the
vosotros form which is essentially equivalent to the southwestern virginia and y'all
so if you built a translator assuming you didn't have to use that form to translate to english
and then suddenly you got castilian spanish that used that form it wouldn't be able to handle
that situation so similarly similarly there's several different portions of
the cuda specification that different people make use of so if you don't build a translator
that's built to handle all of those you're going to have some things that fall through the cracks
so from here i'd like to transit transition into a discussion of two
um solid examples of where we found situations that translation is difficult
and these these examples both both fall under the general category of complex semantic conversions
which means essentially that code and opencl provide the same functionality but there's a
significant enough difference in how you access that functionality that the translation's
non-trivial it's going to take something more than say a perl script or a regular expression
so the first example of this is how could an opencl specify their parameters to kernels if
you remember your languages class some language is used pass by value semantics and others use
pass by references so when you're specifying uh parameters to a cuda kernel you use pass by value
semantics so you can send it anything you can send it to variable a mathematical
expression numeric constant macro expansions function return values all sorts of cool stuff
but when you're converting to pass by reference if you're assuming that you've only got variables
if you remember your c that's fairly easy to do you simply prepend the ampersand operator
and suddenly you get a reference to the variable but if you naively apply that same approach to
the other two variables in this example you end up with something that's not necessarily valid
it might not compile and if it does chances are you're going to get a segmentation fault because
you're taking the address of a variable multi multiplying it by a floating point number and
then implicitly casting it back to a pointer so that so this is one of those situations where yes
they provide the same functionality but you have to be aware of certain nuances in the language
so to correct that translation what we've done is built a module into cuticle that can actively
recognize when you've got a parameter that is not a unique variable name it then looks at the
parameter expression infers the appropriate type of that expression generates a variable of that
type and then passes a reference to that instead so that's fixed this one issue by providing some
emulation of the original semantics now the next example of a complex semantic conversion
is related to how cuda and opencl refer to devices in cuda devices are generally referred to by
simple integers saying which of the encoder devices and machine you want to use however
opencl uses the opaque cl device type and this uh this issue generally comes into play when you're
trying to pick a specific device to execute on in cuda it's pretty simple you use this one function
called cuda set device you send it an integer which of the end devices you want to use and
boom your entire runtime setup to use that device now in opencl there's no one-to-one mapping for
the cuda set device call it actually takes quite a few lines most of which aren't even in here
to provide an equivalent operation you'd have to scan through all the devices in your system
so you had an array of them that you could use that integer to to reference
then you'd have to save all the data for your old platform every old device everything associated
with it your platform all your kernels and the cue you used to send instructions down to that device
then you'd have to manually reset the device and everything else associated with that
so the way we've handled providing translation for this call is by implementing our own shim handler
that encapsulates all this extra opencl work that has to be done
into a single function call that you can then send the exact same integer you sent to cuda
so um these are the these are the only two uh com language complexities i'm going to talk about
today but to discover these we had to do a large deal of work to look through cuda source code
searching for structures like this so we looked at a big sample population
of cuda applications and we chose 79 applications from the cuda sdk and these are provided by nvidia
to demonstrate the breadth of what cuda is capable of so by including those we got a
smattering of just about everything that's in the cuda specification then we chose an additional 17
samples from the rodini benchmark suite at the university of virginia and these were chosen
because quite frankly the samples from the cuda sdk are written in highly standardized format
and they use some non-canonical cuda utility functions that you're really not supposed to use
in regular production cuda code additionally though these samples were written by
much like our open dwarfs project a group of several students and their individual programming
styles are all evidenced in the source code so that adds a nice variance to our sample population
then finally since these applications were all fairly small on the order of a couple hundred
lines of source code to the low thousands we added three more what we call large applications
um which are quite a bit more production ready and these are gem a molecular modeling suite
built by our own alexia neufriev or at least worked on by them
a molecular dynamics simulation from the university of delaware
and finally izps which is a neural network simulation from the rochester institute of
technology and all combined i looked at a little bit over 100 000 source lines of code in total
so uh what are some of the challenges we identified well we've already talked about
two of them and i'm not going to keep you here all day it's a great day outside so we want to
get out of here fairly quickly so i'm not going to talk about the rest of these but if you're
interested in learning more ask me questions or check out our two paper the two papers
about these particularly particularly the journal paper once it becomes available
but as you can see a larger population of applications from both the code sdk and rodinia
samples made use of these these features so after adding all these upgrades to the
translator we wanted to see uh what percentage of cuda structures we were actually managing
to translate and what percent was going to be left for the programmer to do by hand
so we looked at 15 of the applications seven from the cuda sdk five from the rodina benchmark suite
and then our three large applications and for 13 of these all of all the sdk samples all the
rodania samples and gem we we went through this output opencl source code manually to repair it
to the point that it would give you the exact same result as the original cuda if you ran it on the
same nvidia hardware and then for the other two since they were so large we simply went through
them uh essentially commenting and counting lines that would need to be translated by hand
so uh one a few of the important things to notice are we've got one application
translated completely so you take your cuda code run it through our translator and without
any changes you can compile it and get an opencl code that produces the correct result
but for three of the applications i briefly wanted to draw attention to
because before i got my hands on this translator it couldn't translate any of these zero percent
translation and now we're getting upwards of 90 on each of those which essentially means you've
you you can loosely think of that as meaning that you've reduced the time it would take to
translate that code by hand by about 10 fold so after since we had 13 of these applications that
were translated to where they gave you the exact same result between opencl and cuda we figured
what the heck why not see how they perform see if you're going to pay any performance penalty
for translating your code to opencl and what we found out is that with modern driver stacks
there's really no difference sometimes opencl is a little faster sometimes kudos a little faster
but the difference is so small to be almost insignificant
so not only do you have access to new platforms by translating to opencl
you now you don't actually pay any performance penalty for doing that translation
and then finally after looking at the performance of these translated applications we wanted to kind
of quantify how much uh the reliability of the translator had grown by all the upgrades i added
to it so the way to read this graph is the top two lines represent the translator before i got
my hands on it and the bottom two lines represent uh the translator now in its current state
and the first and third lines represent uh its perform its reliability translating the cuda sdk
samples and the second and fourth lines represent its reliability when translating the samples
from uva and what uh the way to read the boxes is essentially that green bars represent applications
for which for every single cuda source file it produced the appropriate two opencl output files
yellow yellow boxes represent situations where it produced at least one opencl output file but
not all the ones that were required and red bars represent applications that absolutely no opencl
was output at all and then on the bottom graph you can see i've preserved the green bars from
the original prototype but all the various other shades of green represent the contributions of
specific individual upgrades to the overall reliability of the tool so we've gotten to
the point where we can practically practically translate all these applications um at least in
part and the remaining applications that haven't been translated are all due to
inconsistencies in the original source code so to come back to our high level roadmap of where
cuticle's been and where it's going we've already talked about its past and present a little bit
so to look towards the future the next major upgrade it really needs is quite a bit of
engineering work to finish providing what we call functional functional portability that once
you move your code to opencl it's going to give you the exact same result as the original cuda
and what it really needs is some upgrades to provide access to the high speed memory regions
of the gpu as well as interacting with opengl and the low-level driver api but there's also
some structures that we realized through my work we weren't ever going to be able to translate so
as part of the future we're going to have to figure out what we want to do about that
and then looking a little bit farther down the road assuming we've got a translator
that can give you the exact same result no matter what architecture you run on
we eventually want to be able to optimize your code for that device because it's not true that
the same optimizations that work on cuda hardware are going to be the best optimizations for amd
gpus or cpus or the cell broadband engine so eventually we want to be able to automatically
take optimizations out of cuda code reapply some device agnostic optimizations then if you know
the architecture you're trying to target add some optimizations for that specific device
so to wrap up with uh i guess three high-level takeaways from this talk
um let's see here there we go robust automatic source resource translation from cuda opencl
is essentially achievable now it's not nearly as straightforward as it might
look from first glance if you just look at these languages they're so similar that
you think you could simply provide a perl script to do it and there are some current important
current important limitations but hopefully with some maturation of the opencl ecosystem several
of those will just go away on their own the next major conclusion is that once again once
you've translated your code your application performance is retained but not only do you
keep the same performance when you're running on the same device now you have access to all sorts
of new devices with only minor tweaks required as well as a whole a whole slew of new users
and then finally arguably one of the biggest conclusions is that what used to take uh
several developers hours uh hours days weeks even years now can be done in just seconds by our tool
so to wrap up briefly need to give some acknowledgements uh many thanks to my advisor
dr wufang who's who supported me through all this as well as uh dr mark gardner in the office of i.t
um he's been my primary contact on this project and has been just fantastic as well as uh my work
pays a great debt it was a great debt to gabriel martinez the guy who started the cuticle tool
and my work's been funded by the nsf center for high performance reconfigurable computing
so with that that essentially brings us to the end of the talk
but one important thing to notice is if you're interested in this tool stay tuned sometime in
may we're going to be releasing the same binary version that went to our funding agencies
online so you can download it and try translating translating your cuda code yourself
and with that uh i guess we're done and we can take questions
questions
sure um
ah one of the uh so the question you asked was whether we do anything uh explicit to
move comment original comments from the cuda code to the opencl code and actually it's
not that we have to explicitly move them it's that our highly localized string based approach
means they never disappear um we're literally only changing the cuda strings
that need to be edited to become opencl the rest of the code every single space
line terminator and comment you've put in there will still be retained in the output opencl
so the future stuff about um
so it seemed like what you were thinking about doing and i know this is probably not your problem
is after cuticle then you were going to look at optimizing but it seems like
you could also have different flavors of cute like you know cuticle mic cuticle amd whatever that at
translation time if i knew the target i would do some things a little bit good right um does
that make sense yeah uh either approach would be valid to have uh customized optimizers that
work on the opencl like completely separate from cuticle you could do that or you could
integrate it with the cuticle translator as it's currently built it would be a little difficult to
wrap up device specific optimizations in that but yes you could use simple if defs to have
different flavors like the amd flavor of cuticle versus the say cell broadband engine flavor
yeah other questions sure um is your is any of your code available open source
or is it just the binary let's see while clang and llvm are obviously open source i eventually
want cuticle to become open source but until our funding agency has access to the source code we
can't give it out to the rest of you but you will be you will have access to the binary very soon
sure did you have documentation because i remember you said that you know a few lines don't get
changed within some of the translation between the samples right so if you were to you know give
people the binary to use and then they use it on the code and then they translate and then they're
like whoa what's happening it's not working as i expected do you have like set things that you know
always go wrong um well all uh i believe all the issues we identified during my profiling have uh
they all go through a unified uh error reporting mechanism so that when a translator when the
translator hits a structure it can't translate or just isn't currently handling it not only outputs
a compiler-esque error to the standard error stream which tells you the exact source file line
and column number as well as providing a severity of the flag and a brief description of what it
can't translate so not only does it output that on the standard error stream since we're rewriting
code and we figure they're going to have to go through the opencl by hand afterwards we answer
that exact same string as a comment directly at that location in the code so when they're going
through they can search for those flags and be reminded of exactly what they need to change
so those the percentages you know in terms of success rates
that was with respect to um opencl code that was going to run on nvidia but right i think
that those percentages would change much if you were targeting another platform um
right but if i was targeting another platform i think it would the percentage would stay about
the same or might there be some other changes that i need to make in my mind
they should stay about the same um but i i know of at least one situation where they won't um
there's this feature on nvidia gpus well you could call it a feature or just a hardware intrinsic
for all current nvidia gpus they've got this uh behavior where 32 sequentially indexed threads
are all bundled together into a warp so they run at the same time and people have figured
out that since they are executing lockstep and there's an implicit synchronization between every
operation they can actually relax their explicit synchronization operations like reduction and they
get a significant performance benefit out of that but there's no source code associated with that
so it's really difficult for the translator to notice so if you run that on a platform
that doesn't have that same behavior yes your reduction is really likely to break so you'll
have to fix that by hand but unfortunately we had a speaker i guess last fall from william
and mary who's done some work to try to figure try to automatically detect when
that behavior might be happening so we might be able to work with them and integrate something
any other questions
that's thankful