Morning everybody. Mhm. Morning, thanks. Thanks
for coming. We uh a little bit difficulty with
some of the new microphones, but I think we
got them working. I'm gonna try
hard not to trip over this cord. Our table
apparently needs to be rebooted. Um but and no
tables get rebooted, but thanks for coming. Um
That's data management a nutshell. And we wanted to
talk a little bit this morning before we go straight
into data management, wanted to talk a little bit
about data management. Look for computers and what.
Yeah, this is old fashioned data management. Not
done very well. It was a Supreme Court Records
Room and the Gambia. These are uh Supreme Court
records that date all the way back to the 17th
century and there was some civil unrest and this is
the result. Um So before we start talking about
digital data and data management, I want to emphasize
that we know how to do analog and we think
we know how to do digital, but we don't
always get it right and it's when we don't get
it right, that the management really comes into play
and can help us out. Um That's just another
example. Yeah. This is here in our own
country. Where was this again, Nathan University of
Maryland Libraries after the earthquake? Yeah, two years
ago you can google this and it gets worse.
The more you look at it, the more your
heart breaks. Um then where do you think?
Oh man, this is this is not what we
hope to ever have to deal with. But if
we keep going along the disaster lane, I'll read
you the quote here. Jackson Mississippi Estimated they would
spend at least$2 million Hurricane Katrina. A systematic
program of producing microfilm or some other form of backup
would have eliminated the need for such recovery and probably
would have proved far less expensive. Which county was
that? Jackson County? Yeah. Where was Katrina
again? Katrina was helped me out new Orleans and
a whole bunch of place. That's right person type
. Yeah. All the way over in Mississippi.
This is one county, not even, not even
close to the center. The damage, It's gonna
spend$2 million dollars backing up. It's just their
court records, Other places got it worse, two
lane and so forth. And it's not just county
court records. I just print, you know,
a lot of people keep their service in the basement
. Yeah. And when we were looking for examples
of this chilling was one of the ways we looked
at and it was, I think we just chose
not to use those pictures that was so heartbreaking.
It was people in the Hazmat suits down in mold
and uh so when people start saying well you know
if you if you go digital and you've got the
data management problem. Yeah which is true. You
do. Um I always like to go to example
the library of alexandria burned. Mhm. And then
when the second one around going digital we at least
have the advantage that um Yeah well least have the
advantage. The backups are a little easier but that's
just one part of the sort of matrix that has
to come together to do a good data management program
. As an example of sort of hitting across all
of the different squares that have to be checked and
the Matrix um I thought we'd play a little short
examples about four or five minutes. Right. I
mean Hello my name is dr judy Benign. I'm
an oncologist at N. Y. U. School
of Medicine. Hello dr judy Benign. I read
your article on B cell function. I think that
I could use the data for my work on pancreatic
cancer. I am not an oncologist. I know
but I think I could use the daily for my
work on pancreatic cancer. Do you have the data
? Everything you need to know is in the article
No. What I need is the data. Will
you share your data? I am not sure that
will be possible but your work is in pub met
central and was funded by NIH. That is true
and it was published in science which requires that you
share your data. I did publish in science then
I am requesting your data. Can I have a
copy of your data? I am not sure where
my data is but surely you saved your data.
I did. I saved it on a USB drive
. Where is the U. S. B drive
? It is in the box, it is in
a box at home. I just moved but can
I use your data? There are many boxes,
so many boxes I forgot to label the boxes.
Hello again. Thank you for sending me a copy
of your data on a USB drive. I received
the envelope yesterday. You were welcome but I will
need that back when you were finished. That is
my only copy. I did have a question.
What is your question? You might find the answer
in my article. No I received the data but
when I opened it up it was in hexi decimal
. Yes, that is right. I cannot read
exodus Imo you asked for my data and I gave
it to you. I have done what you asked
but is there a way to read the hex a
decimal? You will need the program that created the
exoticism of file. Yes, I will. What
is the name of the program site? Oh,
synth. I do not know this program. It
was a very good program. The company that made
the program went bankrupt in 2007. Do you have
a copy of the program? I do not use
this program anymore because the company that made it went
bankrupt, Maybe you can buy a copy on Ebay
. I have good news. You again? I
talked to my colleague. She knew a person with
a copy of the software. Then why do you
need me? Everything you need to know about?
The data is in the article I opened the data
and I could not understand it. If you have
the program you will find it is clear. Well
I noticed that you called your data fields SAM is
that an abbreviation? Yes, it is an abbreviation
of my co authors name. His name is Samuel
. Lee. We call them some. I see
. And what is the content of the field called
? SaM one? Ah yes. SaM One is
the level of C x c R for expression.
And what is the content of the field called?
SaM to? That is logical. If you think
about it, what is the content of the field
called? SaM too? I don't remember what about
Sam 3? Is there a guide to the data
anywhere? Yes, of course, it is the
article that was published in science. The article does
not tell me what the field names mean. Is
there any record of what these field names mean?
Yes. My co author knows what the content of
SAM two is. And SAM three And Sam four
. Can I talk to your co author? I'm
not sure I would very much like to talk to
your co author. Well, he was a graduate
student. He went back to china two years ago
. Can I have his contact information? He is
in china. His name is SAM lee. I
think I cannot use your data. You could check
the article to see if what you need is there
. Please stop talking now. Mhm. Yeah,
it sort of makes you want to face palm,
but I mean, how often you run into that
? This is uh it's not terribly uncommon, sadly
successfully transitioned back without Texan work. That's right.
Got a link up. No, there's a link
on the bottom of this slide where everything above that
link comes from. Um you're probably familiar with the
recent White House directive that says, Well, read
the key part here. Federal agencies with more than
a$100 million dollars and research and development expenditures must
develop plans to make the published results of federally funded
research freely available to the public within one year of
publication, researchers must better account for and manage the
digital data resulting from federally funded scientific research. So
it's not just for publishing science or admit it's it's
a Anyone with a federal grant from an agency that
has more than$100 million dollars in research and development
expenditures went from just NIH possible. And then to
a few journals. NSF some sub agencies or sub
directorates of any H and now it's all federal agencies
with with large research and development expenditures. How many
is that? Well, let's see at Virginia tech
alone, we've got we get funding from NIH NSF
uh Department of Defense, um Nasa, uh Department
of Agriculture, um this Department of Energy. Um
Let's see there's Department of Interior Funding, there's a
department department health is part of an age. So
the list of alphabet soup. Can you name any
more federal agencies? I mean pretty much all of
them. So All of them have$100 million.
I don't know if they all do, but those
are the ones that apply that those I mean when
you look at the list of funding agencies in terms
of research dollars that that we get at Virginia Tech
from from various funding agencies. you know, it
applies to all the yeah like 2/3 of the money
that we get from from federal programs interesting because the
White House specifically scientific research. So what does that
mean? There's actually a lot of that directive was
pretty open to interpretation. Okay. Mhm. I
like the first part citizens deserve easy access to the
results, you know that they paid for. So
I would have to go. So this is um
an answer to a question I posed to environmental studies
researcher and he said I'd have to go in and
make the data little more. I know the best
practices that I should keep it all better than I
do and my students keep it better because I teach
them to but I do as I do rather than
what I teach. So this is an environmental studies
researcher here and he told me this when I was
interviewing him for my dissertation research and and this this
is um as we're putting this together recall this one
but there's I recall this one in particular but there's
a whole bunch of other people said pretty much the
same thing. Um It's a lot of work to
make your data manageable too, You know it's easy
to write sample and Sam to Sam three Sam 4
and especially when you're working with several other people.
Um and you know, it's easy to have a
spreadsheet where even if you label it, do you
see what that label means somewhere? Um Think about
the spreadsheets that you have right now in your office
that you work on. If you got hit by
a bus today, what's one of your colleagues be
able to take over what you're doing and roll with
it. What do they have, what they need
to be able to continue your work and is that
important? Um So when you think about the uh
the little cartoon, the two bears, um so
they were doing this practice, This person knows that
he should, that the bear didn't know that he
should be doing that, but this person knew that
he should. But let's think about what his incentives
are average in tech for, for for doing this
or not doing it. His his performance, he's
an associate professor by the way his performance is measured
by, um, by the articles he publishes by
his teaching and by a service. Um, there's
, there's no line in the faculty annual report.
There's no line in the promotion and tenure requirements that
say manager data. Well, um, it's since
it's brand new that, you know, this directive
is brand new that applies to research grants. You
don't have to have a research grant to get tenure
, although it depends on your, probably you do
in some departments on campus. But um, so
this person, it has very little incentive to actually
follow through and manage his data in a way that
it could be used by other people. He knows
what his fields mean. He's not as bad as
the bear who doesn't know what it feels. Me
, his grad student that he can't contact anymore,
know what it feels meaning. This guy knows what
his fields mean. But in order to make it
so that other people you can read those can understand
his data. Um He uh would it would take
a lot of extra work and what benefit would there
be for him to do that? Pretty much,
there'd be no benefit. Um He might maybe somebody
else should be able to use his research and how
the Virginia tech recognize that they don't there's if you
get if somebody else uses your data there is you
get zero credit for that. As far as having
a financial professional incentive you get maybe um warm fuzzies
and and brownie points in your in your community.
But that's it's not the same as a as a
the same incentive as his promotion Virginia. Right?
Remember that? Yeah. Yeah. So what I'm
talking about data management, how to do it beyond
incentivizing it. Um There are some some ways we
can talk with faculty about maintaining their data and making
it available that can standardize and make it easier for
others to use. And that that starts with open
standards and what those open standards are. Changes from
data format, the data format. Um So why
we didn't list everything here just wouldn't fit on the
slide. There's nobody done. Um Which is sort
of the next point there. Part of the next
point is that there's a lot of open standards based
on the data and part of it is keeping data
in multiple formats in multiple ways is a big part
of making sure that is available to other people to
use um as a geographically distributed preservation. This gets
difficult with data that can't really be shared to the
public. Um It has to be an itemized after
its after its uh they're finished with the research.
Um those faculty generally know that and they're often exempt
anyway from, they can say that they're exempt because
with a lot of the Department of Defense grants some
of them, you know, the Department of Defense
does oceanographic Research. You know, if you're if
you're plotting the floor of the ocean or find a
better way to make more accurate maps of the floor
of the ocean. This is really important for naval
research. That's not top secret stuff though, as
opposed to design propulsion for weapons systems. So,
so some data is not to be shared. Um
but with the with the open standards that's uh multiple
formats, so you don't write something in a program
that that's no longer supported when it goes bankrupt when
the company goes bankrupt in two years. Or let's
say if you write your data in a standard in
, let's say you save it in Excel. Everyone
uses Excel. Right. Well, can you open
an Excel file? Um And can you open itself
from from 1998 in your current software environment? Not
without some little bit of monkey wrenching around. Uh
And often times it was for me, yes,
he was formatting. So we talk about open standards
and we talk about migration, which is up their
migration and obsolescence. Um I have a colleague who
does a lot of data preservation work with the government
. And I asked him once at a conference,
I said, you know, how do you measure
the sort of market and know what you how do
you measure what you've got stored and look at the
format you have and whether or not some of your
data should be migrated forward based on the software that
created it. And I thought he was gonna have
this big long fancy answer documentation research to do and
he said they do a lot of the research.
They said the key factor is whether or not to
go down the best buy and buy a program that
will open it. And I thought that's actually pretty
simple way of keeping track of it. If the
market is still supporting an application that will open that
data, you don't have the obsolescence problem yet.
But he said it is something that they have to
keep on top of all the time and privacy and
security. I think we talked a little bit about
that and on the library services for data management,
what is we here in the library can provide.
This is the part where you pay attention in case
somebody at what we hope your pay attention alone.
But if somebody asks you later and say, what
is the library do for us about this data management
stuff that I've been hearing about this? This is
the answer right here for today. Right. So
we do data management plan consulting. DNP is data
management planning. Um we do that through FBI sessions
, we do one on one consultations and we support
the DNP tool which is a um something you can
log into with your, with your kid and you
can choose your choose the the agency that you're applying
for funding with and you can see the requirements for
data management planning for that agency. And it will
help you um answer all those questions by just giving
you one question time and then it puts out your
answers in a text file. Then you can format
it for your proposal. Right? We have a
storage and access through V tech works. Um Yes
, you don't talk about me. Sure, I
heard of it. Um So with storage we can't
just store anything that somebody throws at us without we
we have to budget for things in advance or to
plan. Um So if somebody says Nathan, I've
got some data that like to archive and say,
what is it? And how much is my first
question I asked and I don't even care what's it
about, but basically if they if they say they
have tabular data spreadsheets and I say ok, and
that those can be small. There typically are,
they can be really large, but typically I don't
see them getting into the gigabytes. Um So I
say okay, um if somebody says they have,
you know, land set files um and that they've
got, you know, for gigs or four terabytes
of uh imagery of uh of uh laser output,
you know, which I can't even imagine what that
looks like or how it works, but um I
know that I can't upload four terabytes through a web
interface. Um and that they can't either. Um
And I know that if we put that anywhere on
a library system, um Curtis or paul or somebody
will call with a question, what's this about?
So we like to inform our our partners here in
the library about what's happening well in advance so that
they can prepare for it. Um and access is
through the tech works so we can index that data
and see what it means, provided that somebody at
the researcher actually knows um and can can and can
describe that. And um once it's in V tech
works, even if you can't just download the set
, you can at least contact the researcher. Maybe
you've got that. We might have put the data
in there. Maybe we're storing it somewhere else in
the library. Maybe it's in a cloud storage that
jews got a contract with. So we can we
can handle larger data sets but it just takes a
bit of planning in advance. Which is sort of
the next point As one part of the automatic consultant
library like to work with faculty early on. And
is there preferences are developing their data Um and help
to try and avoid Sam one Sam to Sam three
field construction. Um as well as where it's going
to end up when they're done with their projects.
These projects usually have termination dates and then they're responsible
a lot of times to the granting agency or to
the thunder for maintaining the status for a while.
Um So all that falls under data management consultant and
uh you want talk about version Sure. I'll say
something about dimension consulting though. I have yet to
meet a faculty member who can in advance Stay estimate
within 100 GB of or within five gigabytes of how
much data they're going to produce with their research.
Um And I also haven't met one yet who will
say what the fields will be before they start collecting
the data and I can relate. So it's it's
not it's not an easy diametrical insulting isn't easy.
Even if you know the field, it's not it's
not a sit down one time either. It's an
ongoing thing as the project develops. So for versioning
and basically if you create your data in a proprietary
format, because that's all that exists for your field
, or maybe not all that exists, but maybe
all of your colleagues use the same thing. Um
We start versioning. Um Well that's one reason to
do versioning. Uh So we don't want to lose
that. We don't want to lose anything vital when
we transfer it to an open standard to it to
a good standard. Um I was using finger quotes
for in case the video didn't catch that. Um
So if if we have a um we create multiple
versions because when you transfer information from one file format
to another, you can always lose bits of information
. A really simple example of that is uh say
this PowerPoint presentation that we're going through. I don't
think we used any notes at the bottom. We
didn't add any any lecture notes. But if you
do that in the PowerPoint file and then you say
that as a pdf, you lose all the lecture
notes. So you want to really save both versions
because PowerPoint is not going to be supported this version
of PowerPoint, There's no There's a very little likelihood
that you'll be able to open this in 20 years
without um without downloading something else. So we'll create
a pdf which is more likely to be opened to
be able to be open in 20 years. But
we lose information. So that's but that's not,
that's just a very simple example of power points.
It could happen with any file format or let's say
you have your your cells, your columns in in
your spreadsheet format of one way, we've got these
equations that apply to everything in this row. Well
, if you say, if you reformat that to
another file, if you save that another file format
, maybe that file, maybe that format doesn't support
the equations that you that you wrote. So that's
one kind of versioning, the other kind is um
okay, I've written an article about this research that
I did, and I'm going to not just uh
I'm going to archive the data for the research,
and I'm going to archive the software that I used
to process the data. I designed this software myself
. By the way, I didn't actually do this
, I'm speaking, you know, as if I
was someone who did. Um So I've got this
piece, I've got a python program that I wrote
, and the article that I wrote is based on
the python program. Now, uh three years later
, let's say, you know, I've I've come
under based on reviews, have gotten from the article
and based on some of the things I've learned,
I'm going to uh adjust the python program, I'm
gonna tweak it a little bit. And and and
also I've been continuing to gather data in the last
three years, so my data has changed. I
still have that old data, but now I've got
this new data, I'm not going to replace the
old file because there's an article written about that,
I'm going to create a new file um and and
write a new article about that. So if you
have one data set that changes and somebody asks for
your data set and you give it to them,
they're going to get different results from your article three
years ago, then then then you did, and
then you don't have the same kind of transparency for
your research that you would have. Otherwise, if
you if you employ versioning so, and then when
you change the software, you're changing how the data
is processed. And if you give that to somebody
else, they should be able to get the same
answers as you. So if you every time you
change the software, you need to uh track that
change. Um so we archive both versions of the
software and both versions of the data so that other
people can look at read your article and say,
I wonder if they fudge these if they fudge the
results at all? Well, if you show them
the data and you show them the software, then
you designed to process the data, then they can
see that no, you're not full of it.
You actually did do your homework and that you're actually
uh the things that you wrote in the article are
actually true because they can do the same. They
can use the same tools to get the same results
. So that's what we do diversity and that's how
we diversity. Is that clear or do that take
too long? Mhm. I thought we'd open it
up to questions. Yeah. Um I like got
imagine as well. Mhm. Mhm. Yeah.
You have to log in, right. This is
an open source or something that we've created here.
Um This is it's not purchased. It's a it's
developed by California Digital Library and johns Hopkins and a
few other institutions. You via is a partner,
right? Excuse me. And we are now a
contributing member. And I don't know what that means
to. I don't think we actually contributed code um
or money at this point. I think we we
got on early enough that they weren't um that they
weren't saying that they weren't asking for money and I
don't know if they are yet, but I think
that's that might be part of their future, but
you log in with an institution, it goes into
your authentication system at your institution. So for us
it's the period and all those institutions have their own
authentication tools and then it gives you a drop down
list where you can choose what kind of what agency
you're applying for. You make it even bigger scott
. Yeah, so we've got both public and private
funding represented here in this list. Um there's about
12 or 16 different NSF, you see all those
NSF ones in there for different directorates, they each
have their own requirements. There's one for any any
age, this one for no uh NIH mls um
Gordon anymore, so go down from there. Okay
, so almost all of them are for NSF,
I thought the list is longer than that. Actually
, there's a that's all that they have enabled I
guess for the DNP tool. Um There's obviously other
agencies now that as of february require data management planning
or that announced in february, I can't remember.
That takes effect. So and this um plan um
actually could you go back to one of the NSF
ones and I think they it's a good way to
shoot shows up differently as well. Alright. Just
put in a test under that required field there.
So you put the formal solicitation number in here.
This um that's not required. Um But if you
had one researcher who's who's applying for multiple programs or
if you had one librarian who's doing all the grants
for institution or for all the grants that come to
them, they might organize, they might help organize
their their uh their data management planning uh services by
so they can go back and say, what did
I do last year? Okay. Here's all the
solicitation numbers from last year. Uh I'm gonna go
back and see how many of these got funded,
How many didn't so that be a reason to use
a solicitation number, comments just know it's why don't
you go on to the next page, you know
before you do you can see on the left there's
this number list that is kind of telling you where
you are in in the in the data management plan
. So we're not even to the first one yet
, so I can go down. So can you
expand the help? Their hateful. So this is
the basically the first question they're asking you to to
answer when you uh for answering the types of data
. And you see there's formatting tools in there which
I've always kind of ignored because I figure I'm just
going to format it when I I put it into
my text into my word document. Uh huh.
But it will take some time to get some of
this down in it to get something look at at
least. All right. And we got the next
one. So, you know, we've got answer
about types of data then data standards. Um um
So this is asking about how you're going to describe
the data. Are you going to call it San
Juan San to San three or are you going to
call it um you know mets dot title or whatever
it is. So um are you going um so
that's asking about metadata standards and then the next one
after that is what's your policy for access and sharing
? Is this a it's the graphic europe. Is
the is the research that you're proposing going to have
commercial value to to yourself. Are you planning on
patenting something from your research? If so, then
you can say that I'm exempt from sharing this specific
data. Um or you can say I'm using lots
of human subjects in my research and therefore I should
not show share my data because that would violate their
there are their privacy. You can see how this
is a really good way to help folks think through
this stuff up front though and see how it's related
, how the data management plans related to the rest
of the proposal. Um because a lot of people
can skimp on certain elements of the proposal and that
they want to get funded, but they will think
um they'll put so much effort into the research question
and methods, but not into what they'll do with
the results and how they disseminate them. NSF requires
a broader impact statement. So how are you gonna
not just make this be this ivory tower project,
but how is it going to get out to the
people? Well, maybe they write in something about
I'm gonna work with an area high school teacher and
and create lessons will teach us well. That should
be reflected in your access policies for access and sharing
well. And a lot of people do that without
even having to talk to him. High school teacher
beforehand. They'll just make it up as they go
. But the policies for access and sharing this ties
in directly to how um what their broader impact statement
is or how they're going to make this relevant for
society. Yeah. As well as the policies for
reuse and redistribution. If you're going to create lesson
plans out of this, um how will other people
be able to access those lesson plans? Will people
be able to create their own lesson plans from,
from the, from your research and archiving preservation.
And this is we have kind of a blanket statement
we use for if people want to use the tech
works, but it might be, it might it
might make more sense for certain disciplines to use a
disciplinary repositories because really you want to get it to
the audience where it's going to make, have the
biggest impact. People aren't interested in the company to
V tech works to find like the ultimate data set
on astronomy. They might be, there might be
a specific um uh repository that's specifically for astronomy data
and people. Australian researchers go there to look for
it. Yeah, there it is. And you
can say this is plain text rich text given the
formatting, I would have gone with plain. Mhm
. I haven't tried it with me for a minute
. I want to see what it does never from
anything in there, download it right on my desktop
. Yes, for many dozen off a little bit
. So that's how the DNP tool works. Um
Were there any other, it looks like a lot
of, it's not exactly self service for the P
. I. S. Um knowing say the some
of the formatting questions particularly they move more on to
the archiving very side of things. Um Where in
working with researchers, should we point them to the
templates for the individual funders that are available for the
funding requirements part on there and get them thinking about
those those questions. And has that been part of
the consultation process? Um But where if there's still
up against the deadline to submit things that as they
tried to translate the questions of the problems that are
asked in the tool library jargon? Yeah. Or
jargon from any number of sources. The ones that
are not their jargon. What what do you recommend
in terms of pointing people who don't work with us
in a timely fashion to to you mean like find
those things? Do we have or is there a
way, for example, that work as a complement
to the tool? We have some kind of utility
that will allow them to see the questions in advance
before they're they're actually filling in the boxes and that
could say and here are some of the standards of
Dublin core does this and you can uh these are
are sufficient for whomever. So there's a number of
online tutorials for for what that deconstruct the different parts
of the data management plan that you can go through
. Uh We don't have anything up like that for
procedures yet with the research hub. But um there's
, we can customize our DNP tool presence so that
shows up, we can also create an html web
page on the library side that would have that and
other libraries have done that. Um University Minnesota is
kind of famous for, there's um and although after
I was recommended to visit that the first time I
noticed there were lots of dead links, but I
hear lots of people talking about the university Minnesota say
. Um but Eva has good resources, California digital
library resources. So when people ask questions like that
, sometimes I'll refer them to the the website of
another institution for for the for those kind of tutorials
and that will be a part of makes them right
. Yes. So about the dress consumers. I
wonder question about Vicky works. Where is that actually
over the computing center or the partially here or how
Were the back end? Back end is on the
is in the server room on the 4th floor?
It is okay. At least it was. And
September 2011 when they showed it to me. I
still haven't moved it since. I wouldn't know and
I don't particularly care where it lives. I'm just
not in the basement. Yeah. What about the
sixth floor either? Because some library, I've worked
in libraries where there's leaks on the top floor,
usually throughput for don't put it by the window either
. Well, I'm curious what kind of capacity do
not want to fill it up. I think I
think they start us with two terabytes and they didn't
say that that's your limit. They said that they
were given you for this year Um if you need
more in six months, let us know. So
we I think we have and that was back in
to fall of 2011, so we might be a
we probably we probably have more than that now because
they don't think we filled that. So in the
project, hope of the historical aerial photos at this
point. Uh You gave me Right, right.
Some are, but I don't think there's been a
amend that upload uh for that, so I don't
think those are those are positive. Yeah, yeah
, terabytes question. And you said, well,
if you've got four terabytes you're hello? Mhm.
I just wondered how we're going. Things are going
so fast. It used to be that Just a
small amount of data would fill a computer now.
Of course we've got 100 give advice or whatever in
the interest. Little computers. Um I saw a
new thing about how some MIT students had constructed some
weather satellites, like a shoebox and I was just
wondering is there anything out there where people thinking about
how data storage, how pretty soon terabytes are going
to be the new megabytes. Well the terrible when
I said the terabytes can't be upload through us is
basically you can't upload terabytes to any web interface you
can use FTp or or other or other other internet
tools that can transfer larger amounts of data over longer
periods of time. But you can't do it through
like an html page. It's not a question of
storage of the question of bandwidth, other ways of
getting it in there. So storage is becoming a
problem. It's storage some dating like for instance,
space exploration, all those pictures, terabytes of terabytes
or not? It is. And Mhm Yeah,
they started talking about petabytes um which I think 1000
terabytes to petabytes. Um But you actually surprised that
the scientific data like the Sky Survey, the Sloan
Digital Sky Survey uses one of them. Um Takes
up an awful lot of room. It's it's it's
a giggle. Hundreds of gigabytes of data. Um
but it's a it's a video and audio that actually
eats up a lot of space. Um We're talking
about versioning and migration when you're dealing in uh and
photos, video and audio. They're different compression rates
what we listen to on our phones versus what you
might want to have recorded for for editing in the
studio letter. And so generally best practices to hang
onto an uncompressed version. Let me start doing that
. You're talking a lot of data. And uh
one of my favorite examples is that the steven Spielberg's
project. To show a foundation. Thank you.
Show a foundation of doing oral history and their server
room Could easily hold 10 of the digital side service
. And they're already full of their capacity. So
they had to expand some. Um So something outrageous
later. They had to invent, they had to
invent robots, storage for them for them, you
know? Well another thing is sad storage is yeah
is less of a problem than than uh bandwidth.
So access. Yeah. So um say like amazon
Glacier is a cloud service. It costs a penny
per gigabyte per month. So I don't care.
Goodbye. It's that's a dollar a month for storage
. You get charged not for storing it really when
you get a charge of dollar month restoring it,
you get charged for downloading it. That's when we
get you. So um and same with a flicker
if zero photographer I believe so um No, well
so maybe you have nice photos in your office.
Well um flicker for the pro accounts. You're not
charged for storage. Treasure uploading new things. You're
not charged for the things you uploaded last month.
You're charged with things. You're uploading this month.
It's I mean you're not charged, you're basically you
pay a an annual rate and you are allowed so
many gigabytes per month to upload. It doesn't matter
how long you been doing that for. But it's
part of data management. We're talking about cloud services
. We're talking about that. Sometimes we have to
point out the faculty that depends on where their funding
comes from. They can't use these clouds sources because
amazon is not gonna tell you where they, where
the computers are. Might not be in this country
where your dad is sitting and depending on your depending
on faculties funding research um they can't put it there
, you know, even if amazon says well secured
and it's password protected. Yeah, they won't tell
you where where it's sitting, where that physical server
is and for some funding that can't be outside of
the country, so Exactly. You know, it
might be on one of amazon servers in china and
I don't think d o d will be very happy
about that. So it's actually the transport of data
that's more expensive items that it's more of a technical
, it's more expensive, is more of a technical
challenge. So when somebody wants to upload data to
be tech works, if they want to do something
very large, they have to basically upload it to
a different server through um through a file transfer protocol
. And they're secure ways of doing that. But
basically they get it twisted, they give them a
a server address and they upload it. They log
into that space and they uploaded there, uh,
just through a direct uh ftp address or essay or
ssh or something. But um, they don't,
they don't upload directly to be tech works if it's
very large because basically there, their internet service provider
isn't going to handle that level of capacity for through
the web. I'm kind of surprised how text is
this whole? Yeah. Hmm. I mean,
you're basically people, people just friday. Okay,
well, here's my plan. Here's this, here's
that I'm kind of surprised. This is a more
structure types of data. I don't know in terms
radio buttons, checkboxes, things like that. They're
a little bit easier, for instance, or whoever
to figure out a table. Well, this doesn't
this, the mp tool generates text that goes into
a supplementary section of your grant proposal, which they
received as text. The Mp told you when you
submit this, it doesn't go to NSF. It
goes, it goes to your downloads folder And any
attached to the grand you trash it as a supplementary
page two grand. Well, still the agency is
having to basically, what's that information? What?
Yeah, it seems like, I mean, it's
a huge burden on asset in my age or whatever
. Yes, as a huge burden producers as well
. Uh anyone who's on the student's side around here
who worked with folks trying to make sense of end
note and actually make those records conformed all the different
style managers. It's not Endnotes fault, it's not
the data's fault. It's just that there's a lot
of specifications that have to be made, A lot
of people don't do. And my sense is,
and this is really getting back to part of my
earlier question is there's so many different questions out there
that what do you do with the things that are
outliers from the check boxes that the template doesn't doesn't
account, but it's very hard to You have five
radio buttons and then the other box I think I
would assume, but they feel easier to have to
let researchers put it in their own words, limiting
them to a certain set. Mhm. I guess
the disadvantage is a lot of people don't express what
you're doing very well or vaguely or whatever. I
still don't understand what your day is coming from that
. Yeah, way, you know, producer and
the reviewer, it ensures that money is kind of
people what? Mm He said, I think our
our is just about up questions. I am,
yeah, we started really early. I'm only three
weeks early. Thanks for coming. Thank you.
Mhm. Mhm.