[POOR AUDIO QUALITY]
All the difference that we know today, and
make sure that you come out with
a clear understanding of the difference between green
versus gold open access
as your [NOISE] are on top of that series.
I want to also make a call for
depositing data packages on open data,
archival quality website,
and [inaudible] reproducibility.
Didn't offer a new publication model.
There's some new publication models
are involved in a couple,
and I'm going to you about these,
and how they work,
what they're trying to achieve.
Finally, check your biases at the door,
and I want to tell you what that means,
to check your environment so that you're
trying to be aware
of the biases in
review process and in academia in general.
Why am I talking about this? Why am I here?
A couple of things about me to share with you is I had
the audacity of giving a talk like this
but very much improvised in 2012
in a workshop about disability science into this.
Many people are true and
it became hard and I put it aside for a while,
it was very much found in social media for awhile.
It has helped people
have short recipes for what we mean by reproducibility.
Last year I wrote a little piece
in science about the hardship
of attempting to research
how from my PhD days until today this time,
which was a challenge it means,
and accepting that challenge.
Again, that piece which was received,
I wanted to share with you.
Then the idea of reproducibility packages that
we've been refining over the years with my students,
[NOISE] I want to show you a little bit about that.
It appeared in Natural [inaudible] earlier this year.
I've been very consistently
being concerned with open access
of the the outputs from my group,
and in fact the impact story,
profile that I have with my work,
I now have a 98 percent open access record,
which this is one paper somewhere that I
reminded that I didn't want to post.
It took some time and years to develop
a consistent policy in our group and now my students
sitting in the room know that this is really
an integral part of
how we do things.
Just a snippet of what the reproducibility
PI manifesto was about.
It was a commitment that I made publicly in front
of an audience of
people discussing about reproducibility.
I said, "Well, the first thing I commit
here publicly to do,
is I'm going to teach all my graduate students
about reproducibility.
It's just want to be part of something that
I consider my role as a mentor."
All of our research code, and also our writing,
we write our papers under version control,
which means we are keeping
a history of all the changes in
the file and now we're doing that publicly as well.
Writing our papers on GitHub publicly.
The paper is being developed it's
live and in fact it's the most recent year we have
started posting other peer reviews that we
get and our discussion on
the revision of the peer review on
the issue tracker of GitHub publicly.
We did comments about this.
You can ask me later about it.
I'm sure that you're already thinking
about questions in regards to that,
but it all started with the idea of publishing
our writing or keeping our writing under version control.
This is specific to our computational work.
We always carry out verification and validation,
and make those reports public,
and that's same for reports.
For the main results,
we also share the data,
plotting script and figures under CC-BY.
I want to tell you more about that in a little while.
This is something integral
to the way it [NOISE] works right now.
When we submit a paper,
the main result in that paper,
we want those main results,
they're presented in a figure in a plot.
We want those results to have the data and
the oldest program that
generated that figure available to anywhere,
so there's no mystery of
what number is behind that door on
the slot and put those under
CC-BY and talk a little bit more about what that implies.
We always upload our preprints before submission
to a journal, I'm a big preprints fan.
Today I'm going to talk about that as well.
We release the code,
any code for a computational and research group.
So we generate code.
At the time of submission
of a paper at least or earlier.
Right now we don't even have to worry about this
because we're developing all the codes already opened.
Our code is on GitHub before we even start
getting results or writing the paper.
But this was 2012.
I also made a pledge to add
a representative paragraph inside the paper at
some point that we describe what
it is that our effort for reproducibility was.
Where is the code?
Where are the figures?
What types of the codes we use?
Just make it plain to the reader where
they can find anything and that we
are attempting to share the physical paper.
Finally, to develop a consistent open-science policy
within the group and keep an up-to-date web presence,
because what good is it
to do open-science if
there's nobody else finds your stuff.
You have to have to, I believe,
communicate and make discoverability
of your work easier for people.
This is a screenshot of this paper.
Not a technical paper,
it's open style,
back of the magazine type
[inaudible] on a reproducibility,
and you're going to my early PhD days
to what we do today.
Describing the emotional aspect
of all the pain and suffering caused.
I invite you to read it.
I think that also have to inspire from other people,
and [inaudible] the computations that I have
engaged with other people afterwards.
This is the TechBlog. It's an interview by Jeffrey Perkel
for the term or the theory of digital toolbox.
Several others have been interviewed there to discuss,
GitHub, to discuss how they do some theories.
In particularly my
My Q&A was focused on this idea of
different packages and what we do in
our group to provide that.
Let me tell you about these.
This we call them for sure Repropacks now.
The idea is that as I said before,
for main result in the paper,
we're going to share the data,
the plotting script, and the figure under
CC-BY under attribution constitution license.
We need to file a bundle
with the data, the running script,
that is the code that generate it,
the figure, and the figure itself as a PDF file.
Then we put that on a repository.
[BACKGROUND]Going to be a word
for it and include it before submission of the paper.
I'm always creating feedback [BACKGROUND] I'm loud.
It was working fine just give me a sec.
I guess I was a bit too loud but [LAUGHTER] all right.
Good. Thank you for help.
[LAUGHTER] Where's the speaker?
Do we have speaker that is
feeding back into the mic somewhere?
[BACKGROUND] Small audience.
I don't think I need the other one.
I'm saying that we need to file a part
under archival quality repository
for data that wasn't reused in big shares.
We get a DOI for
that file before the submission of the paper.
When we create the final version of
this manuscript to submit.
The fraction of the figure,
we cite this file bundle.
Given citation within the ally to
the deposit is filed by the long fixture.
Occasionally this was originally
a measure to increase the reproducibility of our result.
In fact, this initially was included
in our reproducibility statement in the paper
explaining that the figures with logistics were
available with the purpose of the reader be able to,
well, if they want to regenerate the figure they can.
They don't have to actually take a screenshot and
use a digitizer to get the flawed if they
want to compare with the result, isn't that typical?
Now we have the actual numbers
together with the figure available on the repository.
But in conference doctrine, convert conversation,
I often started to stress
the fact that this strategy also
mean that I am claiming the copyright of my sixth year.
I had a publication releasing them under SEC by
adopters and then reusing them in my own paper,
under the terms of this license.
That leaves the figures open for future use
by others without having to
request permission from the publisher,
even after I have filed my copyright transfer agreement.
For the first time.
Let's take five and negative two first one,
I was really nervous.
I thought the journals were
going to complain. It happens.
By sixth time I started talking about this in
conferences, not officially yet.
Then after a few years,
I said, Okay, let's find out.
I wrote to the University General Council
with the whole explanation of the things that I said.
I want to find out if this really does
have this side effect that I think it does.
Which is I'm retaining the compromises,
this figures and releasing
the membership to guide and you've got to use them,
and the problem of transfer of
doing it just not nullify this.
It took about six months to get an email until I finally
have definitive answer from
the General Council of the Assistant General Council
where is due up to this day.
Yes, after thinking about this,
I sent chiastic thing.
They sent me a few copyright agreements.
I sent them the copyrighted books
that I filed most papers.
She [NOISE] said she
had to do homework and came back to me.
I have come to the conclusion
that you have releases
under Xiv and the subsequent tracks,
copyright transfer agreement cannot undo,
then license that has already
been issued on this insurance.
Good, Now I was thinking about it.
[LAUGHTER] Now I can talk about it.
I wrote, and I did
some tweetstorm about it and everything at that point and
I am happy that it's
sealed by the General Counsel of Universities.
No journalists ever complained about it.
I think they have really woken up too late.
[LAUGHTER] I have read enough.
Once a lot of people doing it in the mind by the way.
I don't know. Why males want the figures on
their own without the rest of the paper I'm
signing away to cover the paper anyway.
But that's another matter.
This is how a referral path we call it looks like,
you might have some figures as the repository uses.
This is one particular result in
a paper that we include three figures
together and we have data and we
have some ReadMe on how the problem that you now have.
Some type of generating
those figures and so on. That's how it looks like.
It can be viewed and downloaded of course by anyone
under the Xiv by class.Preprints,
based upon your preprint and in physics and math
and applied math and some areas of
computer by an archive is a way of life.
You said you have to have a paper and
registered the first question
is about the archive preprint.
Because if you don't have an archive
preprint, it doesn't exist.
You cannot put up submitted your CV
and not have an archive preprint
because that's vaporware.
[NOISE] Preprints are growing by all metrics right now.
We have seen in the last four years,
five years an explosion of different 'Xiv site.
I'll show you a couple of documents.
But this is the case in our 516,
a group rate growth
in all of these subfields that they publish.
This is data up to 2016.
I didn't find the original graph used by Paul Ginsberg,
the founder of artifact.
But I found a tweet [LAUGHTER] from Carl T.
Bergstrom of Heam talking
to the slide but just even cooler,
I think with the data actually in 2017.
I cannot quite read from that graph.
But it gives you just an idea of the growth
in preprint or the archives in particular.
But it was I saying this then all of this explosion of
different now sites,
engineering archive via Wi-Fi, the so-called.
Even ten archive now.
Germans have been real
hard-nosed about preprints for the longest time.
The American Chemical Society
now I'm accepting preprint and they
have launched their own preprint service, go figure.
This is a big misconception that a lot of
people have that really
the majority of journals now accept
manuscripts that have been
previously submitted to preprint servers.
You're not actually losing out on any possibilities that
all publishing in your preferred journal
like putting your work on preprint archives.
All of submitted journals
of Nature of Publishing Group, Springer,
Taylor & Francis,
as I mentioned, the American Chemical Society.
I think I was in a panel
of the American Chemical Society meeting
just this year talking
about pre-prints for the Chem archive journal.
They said that 85% of
their journals are accepting pre-prints.
I think chemistry was
the last holdout field for preprint.
I was able to find that this article devising type of
preprint servers figures for 2016,
113,000 submissions to the RFIs
and you use the Bio Archive going quite a bit there.
Peer J Pre-prints back on and so on,
started to grow at either end of this year,
I suppose that the numbers will be a little higher.
In general, an open access,
I want to make sure
that I'm talking about pre-prints would make
this distinction between green and gold open access,
which really distinguish every venue of the article,
nothing more, green open access
refers to this idea of self archiving,
either in a domain repository like archive or on so,
or an institutional repository or if one
of those data repositories and so on.
Preprint that appear in a actual open access journal
or a traditional journal that has
an open-access option always by paying a fee,
of course, in that case.
There are two differences. The first is
green open access is free for us as readers.
While ago, the open access imposed
a fee called the processing fee or APC.
People interested in this field always talk about APCs.
Green Open Access.
Our second involves gold peer review,
whereas it's the preprint deposited ahead of peer review.
Whereas gold Open Access includes peer review,
at least in the reputable journals and a whole bunch of
so-called predatory open access journals
that are to do the review but do not.
That's quite a concern,
of course, for the field.
But they still charge a fee, obviously.
[LAUGHTER] You may conclude
that peer review different pain to feed
the APCs worthwhile because you're
open access article is more
reputable or more special in some way.
That's what expensive journals want you to think before.
But it's not necessarily to pay
big money to make it work accessible.
You're savvy about this.
I've never paid APC ever.
Do just remember that 98% of open access records,
I've never paid APC.
What we do suppose your mask
is going to be print, server,
submit to a journal,
go into peer review,
revise the paper and then update the pre-print.
I've put a link and a
DOI to where the table body appears.
When you get to the final revision,
we revise twice, rejected,
then we submit again.
We had another group, you might see
three revisions until eventually
it may appear public
and you'll see the DOI of the journal there.
If you look five meters can act as
the latest on the author
corrected version of the paper
on archive after peer review.
I don't remember the exact number and
I should draw this but there was somebody
like maybe 65 or
75% of all archive papers and in a journal.
Sorry, I didn't look up that number,
but it's definitely more than 50%.
The other portion, there's some rubbish,
but there's also some good quality papers
from office and onto.
We don't need anymore to go through traditional journals
like Gregory Gentleman who publish their approval.
The bumper is conjecture or something like that
and was offered a Fields Metal work.
Does not go enough, official journal,
you produce the fields middle as well.
The produced journal that you can
use pre-print service in
traditional repositories and other auditors
in tandem with traditional journals to
achieve open access without paying the big bucks.
That's what you do,
especially if you're in a developing country.
There are a bunch of them.
They have various different uses.
Dataverse coming out of Harvard
is used more in the social sciences.
The Open Science Framework is coming up quite a
bit in various fields. I do dry.
That comes out and
then autocompletion and we use picture quite a lot.
Picture is a commercial bank by bank middle is
free as long as you the model
they like is free when your material is open.
If you want a private version,
then you have to do to both private data.
Then we worked on with denoted code,
add some other objects
because it has a nice integration
with GitHub and do a bit more about that.
Our workflow for these retro pack.
I want to describe it to make sure that gets us down.
We write a paper.
The paper is due,
you're satisfied with the paper and you have
some results there that I renamed shelf of the paper.
At that point we're going to grab
this figures with all of
the materials that aren't in the search,
create a figure make this file upload to fixture.
Cite in the paper and then that this
posted available for anyone to
read because we're sharing it
under a CC BY license. I think this is a workflow
that I am going to keep spreading the word
that we could be making a work,
sharing our work more by the simple trick.
For now it works and how the journalists are
going at some point objects, but why not?
Have you ever tried to get
use a figure from another paper through
the frog rights or whatever it is called? It's horrible.
It's very painful as a waste of time.
[LAUGHTER] Let me tell you now about some new journal.
I would say that savvy
open scholars are interested in
lashing the hurdles for researchers to get
credit for all of their output
including software and data.
There are not many,
but a few new way in journals led
by scholars that wants to provide.
Please join us live,
do things like carry out
double open peer review in public and things like that.
One of them that I'm involved with
is joint a little bit sore software.
The Journal of Open source software is
described as a developer friendly journal
or research software packages.
It isn't academic journals or
proper academic journals in formal peer review process.
It has part of its mission is to increase the quantities,
improve the quality of the software that is submitted.
A measure of JOSS is to
enable citations credit to authors of research software.
The idea is that you've developed
a package for a research software package,
we've a series of quality measures
that are giving you the author
[NOISE] reviewer checklist.
Then you would write
a short paper describing the software research.
You don't have to write a whole research paper
with new results,
and having an actual domain paper that uses the software,
but it is a paper that describes the software itself,
describes the research application.
If maybe it was used in some research paper,
you might cite that paper
but the paper is about the software.
You might think that writing a paper about
software is a weird thing to do,
especially if the software itself is
available in a public repository with
documentation on how to use it and
have a given website where users
and maybe even examples of how to use the software.
But the thing is, writing
a paper about software is currently
the only sure way for authors to gain career credit.
Because it creates a style of
entity that can go into the ancient days,
because citing software,
even though a lot of people have been talking
about citation for software,
it's not just not part of the
academic and workflow and nobody cites software,
probably nobody knows how to cite software,
and there's no infrastructure
to count citations on software.
But there's infrastructure to
count citation to the paper.
We write the paper, yes, at CrossRef DOI.
Software itself gives in a software repository and
gets another DOI,
the normal DOI for the package.
But then at the CrossRef,
DOI for the paper is through metadata
linked to that software repository,
so that becomes protonated data.
But this is a journal with
a CrossRef membership and therefore it
can be indexed and can be cited.
You have also a rigorous peer review process
and first class editorial board if I say to myself,
[LAUGHTER] I'm part of the editorial board.
Several other that people there I admire and they're very
experienced in building and
reviewing timelines in software more than myself.
JOSS is our experience of the Open Source Initiative,
it's fiscally sponsored by a project called Numbfocus,
and is non profit in the US.
Also the home of open-source projects such as
numPy jupiter [inaudible],
octopi, high table,
standard, and I forget,
but so there's a bunch of open source projects,
and the idea is open source projects,
you might have 5,6 people
working on in different institutions.
Its very hard for an open source projects to get
funding and not run a file of IMS code.
But Numfocus thrives on
an umbrella organization by
fiscal sponsorship to those projects,
funds can go to an Numfocus
and be protected by our 501c3 status.
Numfocus now has also accepted to fiscally sponsor JOSS,
which means the JOSS [NOISE] receive funding from
foundations or any energies that and
want to provide funding
and [inaudible] does not does not require
[inaudible] which are allowing JOSS to pay
the technical debt of having these developed
[inaudible] by developers originally.
What's the JOSS to infrastructure?
GitHub first of all,
so GitHub the papers are written,
Oh I'll you more about that later,
but they exist in the software repository but GitHub
provides one of the infrastructure to do the peer review.
GitHub for those who don't know is
an open source software hosting and
collaboration platform that has business users,
and I don't know how many billions
of projects living on it.
Zenodo of the data repository by CERN,
to provide the DOI for the software package itself.
ORCID for author identification.
You log in to submit a paper with your ORCID ID.
CrossRef for DOI minting,
and citations, tracking and so on.
The only thing that was written originally from
scratch with a custom web app and
a Ruby bot just to streamline some of the workflows,
so you have a couple of cool
things about that in a moment.
All of those things, the custom web app and the bot,
are open source so that they could
be also adopted by others.
The JOSS sustainability model.
CrossRef membership costs $275 per year.
CrossRef DOIs costs one dollar per articles.
Then Heroku web hosting costs $19 per month.
In the first year with 111 articles,
but I just divide it by 100 to make it easy,
that comes to about six dollars per article.
The next time that you up to $3,000 or $2500,
APCs for your open access articles,think about this.
I think if it's really worth it,
just saying that you
need to pay $2,000 to get peer review.
But review is still volunteer effort by your colleagues,
an so we're just
using existing infrastructure to do all of
the peer review without the need for
these expensive platforms that publishers wants to.
Which by the way are pretty awkward to use.
Yeah. Another question?
Not just products. [BACKGROUND]
I have a question.
[LAUGHTER] [inaudible]
Which do what?
Editing, I don't know [OVERLAPPING]
Editing. How is that I editing?
No you have to [OVERLAPPING] [inaudible]
You may have that.
Well, everything is done voluntarily,
and it can be done voluntarily.
Eventually, maybe you need some admin help maybe.
We publish a 100 papers in
the first year but ended with this model.
Copy editing is not that expensive.
You could hire a copy editor by page or something.
I don't think it's going to come out
at $2,000 per article.
Maybe. I don't know. I haven't experienced a copy editor
adding a lot of added to papers to be honest.
I've had experienced reviewers adding value,
making comments than the copy editors.
Sometimes they even bend things up.
On average, the value added,
not significant in my opinion,
but I grant you that it varies by field.
Surely some of the society journals
will have more serious support,
but recently, I do actually
believe reduced their budgets are stretched.
The first thing that went out the door of the
copy editing for this journal. Yes.
There are others [inaudible] very well.
[inaudible].
I don't know if somehow those editors out there
making money out of their roles,
maybe they are, I don't know.
But it's not normally going to solve all the problems,
but for some applications,
we could talk potentially move to something like
that with a reasonable expense,
a [inaudible] expense, it's not the case.
I'm just showing this as a new wave journal, I would say.
I call a new wave, but it doesn't mean that this is
going to be generalizable to every field.
But for some,
let me tell you a little bit about
the peer review process.
The author would write
the article which is up to a 1,000 words,
it's not a lengthy article describing the software.
Unstated as a markdown file,
maybe not in D in
the cost divisional software repository.
Then there would be maybe a routine check
by somebody for the moment,
it's been mostly the editor in chief,
but we're starting to find ways to share this work too.
Routine check to move
that to what's called the pre-review issue.
When I use the word issue,
I'm using the vernacular of
GitHub workflows where issue is actually
the beginning of a dialogue on
a particular web page that has
some structure to how people interact.
That's the issue tracker.
Then the other would submit via a web app.
I missed that, by using web submission tool.
I have a screenshot [NOISE] a couple of slides ahead.
The submission process consists on
providing the URL of the code repository.
Of course the authors identified by ORCID ID,
but you may have to add coauthors and submit the data.
A short descriptive is
actually just the blurb that appears in the name of
the title of the paper on
the rolling ticker of new submissions.
But there's no files to upload because the files already
exists on the code repository and
the paper is actually there.
The reviewer will go to
the code repository and find the paper.
A pre-review issue is created in that place.
The editor is assigned,
and this is where the bot comes into play,
the bot's called freedom,
and we just type @freedom
for example assign,
@lbarba, that's my GitHub handle at editor.
On assign an editor because the thing at
uses the GitHub API
in the background and outside of [inaudible].
Then I'm going to
get to work finding an appropriate reviewer
, and at some point,
the reviewer will agree,
email us up on our personal emails
or maybe just a little
message right there on the issue tracker,
using the GitHub handle,
somebody to bring their attention
to the fact that I'm mentioning them.
At some point somebody will agree to be
reviewer and I will just write there comment
@freedom assigned Olivier as reviewers and boom,
you get assigned as the reviewer.
From that point, the reviewer will go to the repository,
start analyzing the [inaudible].
We have a checklist for them to
[NOISE] see that it downloads,
installs, the dependencies are clear.
They test, they test run,
the results are correct.
Is the format claims match.
It should be for [inaudible] some of
the cultural expectations of the open source.
Developers are there,
and then they'll run into issues or
anything has happened about
the documentation or about the [inaudible],
they'll start posting their comments right there on
the open and public issue tracker.
Where the author then
responding right there and the editor,
as well sometimes coming in the discussion and
the whole process goes on in public, in real time.
That's the peer review process of JOSS.
First year, 111 articles published, 41 under review.
I don't have the updated stats since May.
The median time for a paper to
remain under is 32 days and
93 unique reviewers participated
in the publication of those hundred plus articles.
One of the things that people were concerned
about with the launch of this journal
was while in whole review process
is non anonymous and open and public,
then you're not going to get reviewers.
They're just going to reduce and that doesn't happen.
I just read people so far are happy to
review in public and their conversation is
all civil and constructed
with the purpose of
increasing the quality of the software package.
Software that is used
seriously for research should be peer-reviewed
and the quality should be improved through
a collaborative process and that is the philosophy.
If we shut off the [NOISE] submission page,
those are all the things that you need to do.
A title, the address of where the code resides,
the software version that you are asking to be reviewed,
suggest an editor and then confirm that you
are the primary author
or authorized by the [inaudible] submit,
and confirm that you have read,
and here's the job code of conduct.
Description is just a short description that gives
the website as I said right here.
Here is a screenshot of papers under review.
I brought this from yesterday.
I think case number 2 of the open reviews and
[NOISE] they are tabs appearing on some of the reviews.
I don't have the cursor here,
but you see the first item has pending major enhancement.
At that point, the reviewer may
have listed something that indicate
a major enhancement is needed to do work
and that calls for a flag.
For that reason, the review for that paper is paused,
meaning that the author said, wait a minute,
this is going to take too much time for me to do,
and so I'll get back to you guys and ping you when
ready because I cannot do this work right now.
Potentially, that's what happens,
but you see there on the right a little chat bubble with
the number 9 indicating that they [NOISE] have
nine comments made on that review issue.
You see going down the line, 7,
11, 8, 9 and so on,
those are active review issues
with people adding comment,
reviewer and also mostly but also the editor.
[NOISE] Since I don't
have [inaudible] describe it for the first item there,
right under the big red type
that says pending major enhancement,
you see a small thing that says 10 of
18 that indicates that on the reviewers checklist,
so the things that they need to indicate satisfied,
10 have been checked and it
remains to be checked once you get into the 18.
[NOISE] That means that the paper
has satisfied the reviewers.
[inaudible] off the bottom for example,
has satisfied the reviewer.
The reviewer has marked
all the 18 items probably at that point.
You got one, when you see that it has
27 comments on the right,
27 postings from the author and reviewer.
But generally that one right now,
that one was submitted August 2, I see.
At this point, it's
ready to be accepted but then there is
the author speeding find a repository with
a revised TOI when it came to be issued in public.
Now, I'm sending you that.
This is using an existing framework,
GitHub for all of the peer review
and some other bits of infrastructure,
also freely available,
and just a few tools were written,
developed from scratch to
facilitate the administration of the peer review process.
Those tools are available
also in the open source model for anyone else to adopt.
At this first year,
year and a half or whatever it is, has gone by,
we've realized five in
discussions with people and interactions
with other people that well,
when you read the statement that appears in
an article about first-year of JOSS,
JOSS was born to answer the needs
of research software developers, in particular,
as I explained earlier, to
thrive in the current merit traditions of science.
Citation, could. But we may have come up
on what we think is perhaps a generalizable formula
for digital science because
it would not be hard for
someone to simply fork, as we say in open source.
The open source tools utilized
the same models for
another journal, possibly another field.
As long as it's a field that is somewhat close to
the open-source mechanics that they'll be
comfortable with this tool that we're using.
[NOISE] Yes.
[inaudible].
Double-open, fully open public peer review.
But pre-publication.
Not post-publication review which is another model.
Now, I feel like I have to discuss
a little bit about free open-source software.
It's a big influence in our work
and our way of looking at academia,
and I want to make sure some of
the key points are also clear to you.
I really believe that free open-source software is
a human invention of a huge impact.
The reason is that it poses an alternative to
intellectual property instruments that really want to
control how we use a creative work,
and the outshoot of this is
people can coordinate their work freely,
freedom, within the confines of the law
and they make access and distribution a priority,
and this with the purpose of creating value together.
Freedom is power, and in this case,
the power of open source comes
from not just being able to read the source code,
but from being able to contribute and to build from it.
That is where most of the power lies,
and for this power to be realized,
and this is for something that I think is very important
for all researchers to understand,
is that it's not sufficient to make
the source code available to read.
We must attach a license
that allows others to modify and to distribute the code.
I see a lot of people with this understanding about this,
about the importance of attaching
a license to any software
that you wish to share.
This is a message that I want to stress,
and I'm saying every research
should know about open-source license.
I'm serious when I say
every research because here is some results
from the UK research Software survey in [NOISE] 2014,
I don't know 3000 people or
something like that were surveyed,
and the results indicate that 92 percent
of academics is the word
they use for researchers in the UK,
used research software, 92% of them.
69 percent [inaudible] say
that their researcher would not be practical without it.
Fifty-six percent develop their own,
but 21 percent of those have no training but
that's another matter altogether.
[NOISE] That's something we do need to improve.
But my point here is that
more than half are developing their own software,
and 92 percent by using
research software on a day-to-day basis.
So software is more important to
the research enterprise than
anyone really cares to admit.
That's why I'm adamant that we
should all know about open-source licenses.
Everyone developing software in academia should
have some working knowledge about licenses
and I wanted to give you a quick overview.
First thing to understand is that it's not enough to
put the source code online.
you have to attach a license to it.
Understand the software is a creative work and
compromise is attached to it the moment that you created.
So without a license,
your software is in a legal limbo
if you put it out there.
Anyone that knows this scenario,
if they run into your software and
you put it out there and doesn't have a license,
they might find it super useful and they want to use it.
But if you don't have a license on it,
they won't touch it because
it's not legal to do so [NOISE]
unless you have permission.
Always add a license to
any software that you plan to make public.
I recommend this paper
a quick guide to software
licensing for the scientist's program.
I think it should be required reading of
every single graduate student in any field.
It's not so hard to read,
it's only a few pages and it gives you
the basic understanding of software licenses
that you need to have. It appears
under open access in PLOS computational biology.
Let me give you the quick [NOISE] nuggets of this paper.
The first question, the first issue was some issue to
understand the difference between
permissive and copyleft license.
This is the most important decision I would say,
these terms are often obfuscated,
often confused that's why it is important to clarify.
When we talk about permissive licenses,
a permissive license gives you more freedom.
I have fewer restrictions.
In fact the only restriction that I've used
could be that the original author receive credit,
receive a citation, receive a mention,
in any distribution of
the software or any derivative works
and most basic restriction on it.
They allow full use,
distribution, and modification.
Even commercial uses are allowed or
incorporated into software into other
proprietary pieces of sofware, that's also allowed.
They only require getting credits.
This is really the best choice for academic use.
Examples of these permissive licenses
out of the Berkeley Software Distribution also
known as BSD is the MIT license and the Apache license.
In fact, permissive license originated,
as you can see from the academic institutions.
By default, this, I believe,
should be your choice.
This is a screenshot from
choosealicense.com, so you can read it.
This is it actually might do that,
but they don't think they can read it from here,
of course, is a very small font,
but you can go to choosealicense.com
to get very clear what you
should pay attention to is that do redox in
the left top left and choose the license
that gives you a very clear color-coded idea
of what the conditions in the case
of some permissions may be.
Let me make immediate contrast with
copy-left licenses.A copy-left license
restricts the use of the software by requiring that
any derivative works also
be under the license of the original.
Another word for this model is
the Share-alike model and
many developers want to ensure they've
open access to use on their work or posterity.
But we need to understand that this is more restricted.
It does guarantee perpetual access to
the source code and all derivatives.
We need to be ready to recognize
that it is an additional restriction on you.
Copy-left licenses, GPL licenses are
considered restrictive in comparison to BSD or MIT.
That's the nugget of it. Here's a screenshot of Choosealicense.com,
and you see that the conditions of
blue dots and limitations, more dots.
The text of this goes on and on and
beyond the screen screenshot where the MIT license,
that was this screenshot that you saw again.
[NOISE] One final thing to understand about
all the lines of business is the idea of compatibility.
Incompatibility Licenses and incompatibility can arise
from very subtle wording in the licenses,
so the open source and [NOISE] strongly
recommend you using an OSI-approved license
instead of attempting to craft your own.
Because if you try and craft your own and
some universities and institutions think
that they're smarter than a whole knowledge base
that has to find a time and try to make your own license.
But there's a lot of knowledge over
the years that has gone into this already.
This problem is not that your license
might be better or worse than another,
the problem is the idea of compatibility of
different licenses for collaboration in the future.
In the paper that I cited a few slides ago,
there is this is very good figure
that explains this idea of compatibility,
the idea of directional compatibility.
If you are basing your work in
something that is BSD or MIT,
something upstream has certain conditions.
The compatibility behave differently whether
a piece of code is built into or from another.
Staff at university, what is it called,
technology transfer offices are
often ignorant about these things.
Any researcher that uses software should inform
the staff because when they go to talk to
the office of technology transfer
they may find somebody who is
supposed to be the expert on the subject,
but they normally don't know about open source.
It's important for researchers
themselves to be very involved,
I've said that already, but I repeat it.
I won't say more details about
this in the interest of time.
How to choose is simple in my opinion.
For academic work,
choose the simplest license,
that's always the best unless you
have a very good reason not to.
Most of the research code in our group has been
released under the MIT license.
Just recently,
we became more aware of a subtle difference
between the MIT license and
BSD 3 license I wanted to share this with you.
This illustrates how subtle language variation matter.
But we recently decided to change
the BSD 3-clause license.
It's very similar to MIT,
its very short, but there
is a small difference in the wording here.
Does anybody in the room notice it?
[BACKGROUND]
Substantial portions. What is a substantial portion?
That is completely ambiguous and undefined.
They are both copyright notice in
this permission notice shopping including profits are
substantial portions of the software.
By saying you're both copyright notice
shall be included is saying,
you need to attribute me because
the copyright notice has my name.
You need to include that in all substantial portion.
Somebody could say, you
can run into somebody using your code,
not fighting you and say wait a minute,
why are not citing me well?
I changed 75% of this story,
so it's not a substantial portion
and so I decided not to cite you.
Well, the problem is in academia we need
all of this citation and all the credit we can get.
Even if you paid 2% of
your work in mind please give me a citation.
BSD 3-clause assures that [NOISE]
has more inspiration from the word that
you guys should include or
the word that you based word on should
include the mention of the original copyright holder.
Word of advice, always
writing to your brand proposal
that your research circle,
will is be released some day and they're
annoyed by a group software.
For life, because as I said,
the office of the [NOISE]
doesn't really know much about this.
You may then be in the middle of your research,
then decide to release your software and may run into
somebody from the Office of Technology tried to say, no,
you can't really use this because I don't know,
maybe we can signup make them money
out of it than you might run into
six months of trying to explain to them
why there's still money to
be made in this piece of research,
software should be released under open source.
But if you put it on the brand that it's funding
the software and research
then the moment that somebody says,
why releasing this software.
That you have opened GitHub page with
this software and the brand,
you're ready to receive the grant money,
you approve this conditions,
that part of the contract with funding agency and
[LAUGHTER] it can save you
a lot of beef and a lot of typing and email
back and forth to explain
to the staff at the tech office.
Why software needs to be free.
A couple of final things about this topic,
if you're participating in
open source software and open access,
a lot of it has to do with
coordinating with other people.
The idea of sharing,
the purpose of sharing, [NOISE] its
important to work with other people.
We use tools like
github to help us coordinate our work with others,
as technology plays a vital role,
but the idea is to coordinate people,
that their actions through sometimes platform and have
communications to make it more effective
working together then of course,
rituals and these communications
and so all of this is social,
a lot of social dynamic going on and with this,
the scores come despite that
we bring to every social interaction.
After scholars reading conference
events and publications,
then you should be aware of this counting
to a shabby role in open science venues.
Because I told you about the model of your jobs,
where we do double open peer review.
Computer science conferences outdone the opposite and
tried to double blind peer review and
tried to excuse the identity of
the author to counteract big possibly implicit biases.
This unknown reason for peer review
and their research statements about it that
give a very complete evidence of
the existence of this assemblage.
But women and minorities and
researchers from institutions that are less prestigious.
Jason Smith from Harvard University
versus Lorena Barba from George Washington University
There's a prejudgment that comes from
the fact that I'm in
a 50 University and Harvard is Number 1.
I'm also a woman and I'm Hispanic.
There's an immediate bias that
people bring to that difference
and we cannot deny.
The best way to deal with it is to become informed,
to share the information with others,
to understand that there is research behind this,
that it's not just feminists crying of these,
but there's research and we should [NOISE]
and bring this to
our communities in conferences and publications.
We participated in an intuition review.
Research confirms the pervasive consistent associations
that we make with the male.
In particular, this study found that
70% out of 300,000 men and women participants link works.
Science which more quickly with male than female names.
These were all control situations.
Gender stereotypes because some adjustments
of our appreciation of the work,
of our expectations of
performance and so on in the sciences.
The only reasonable way that I see a few with
this maintaining our value of
openness is to become
educated and then to develop instruments
for sharing this knowledge within our community,
for example recently,
I was involved with psi by conference.
I don't expect you to read this,
I will explain it to you.
We work with some people that have
particular training in diversity issues
that were interested in this community.
To develop care and review
our guidance where we made particular emphasis on
the idea of explaining
the double open peer review system.
We described the fact that
peer review [inaudible] bias just got hiring process.
Then we developed some tips for interrupting that bias.
The tips are for example,
this is also important in hiring committees to
acknowledge the existence of
implicit bias in the beginning
before you start on your people.
You bring some awareness
immediately that this does exist.
Because, if you do it after you already
pulled from some opinions about
the work and about the people.
Before even starting the review is
just stop and think about this and
bring it to the floor a moment of clarity.
Refreshing yourself every so
often while doing this review,
for example by going back to any rubric or
checklist that described the metrics
by which you should be reviewing or evaluating.
I'm [NOISE] going back to
that guidance so that you're focusing
on the performance or the quality.
Having some niceties of
being polite and being
grateful for the opportunity to participate in
that community so that the whole way, the whole process.
We have several tips that were moved from
[inaudible] remembering right now, they're on the web.
They just pushed aside by [NOISE]
and we're revising them.
But the idea is that there are techniques to
interrupt bias and make people
more aware of how these come
into play in our assessment of the work of other people.
This is included in this talk because it's
particularly important that when we
move to more open processes.
There have been some
younger early career researchers from
minority groups that have expressed
that open science of this problem.
If we are going to do
double open peer review or things like that.
Then we need to address
the challenges of these Greek
for early career researchers,
for example if they're
going to do a critical review of some of the
more senior with how we're going to do,
how are we going to protect them.
These types of issues, that
if you are involved in open science,
[NOISE] should be part of your toolbox of workmanship.
With that, I want to
leave it open for questions, and any comments.
I don't have time to prepare,
so I hope that didn't go over very much.
[APPLAUSE] Yes, you had your hand up.
It's about the open data question,
going back to when you're talking about repositories.
There are plenty views
in social science, behavioral science, medicine,
where there's a disclosure risk in research subject,
they get even crazier when you get into qualitative data.
Yes.
How within the framework
of open data can that be dealt with?
Well, I acknowledged that I've got in that field,
so I know for example
that extremely hard to anonymize data set.
That is, itself every such question.
But within my ignorance,
the answer I'll give you is
that the default should be open,
but in the circumstances where open is a problem,
then that gets special dispensation.
But our general position,
in my opinion should be unless
you're in a case where is you have no data,
have data or something on privacy concerns are involved,
then the data should be open.
Because [NOISE] anonymizing is
nearly possible as far as I know right now.
Because one part of that could be restricting access.
Yes, at that point yes,
there's a good reason to
restrict the access of data that is sensitive.
But everyone else, you have no excuse.
Do you think that there is bias in coding?
Gender, or racial, or ethnic?
But what do you mean in coding?
Just programs are written,
procedures have been written differently by
various groups of people,
or I don't know if that's true.
In the open-source software world,
we know that there is bias in things
like if somebody contributes
something to an existing project
through what's called a pull request,
so I make a change that I saw that
change out there and I used
the tools that form to
ask the owner of
the project to accept a change to their project.
There are differences in
the acceptance rate of
pull request between women and men.
There's a paper about that [inaudible] But I think
the difference was the other way around not like
you expected for all pull requests,
and I'm trying to remember,
so the paper that I saw found a positive,
the more pull requests
from women were being accepted than for men,
so the contrary that you would expect.
But further it has led to,
where the fact that there are
very few women in the first place that are contributing,
so there's so few,
like 10 percent or whatever that they're
already the well-known people in
the field or something like that.
There's selection bias in the whole process.
The problem of bias in
coding starts much earlier than at that stage.
It starts in the computer science undergraduates,
the incorporation of women
and minorities to computer science at
earlier stages rather than at
that point contributions with the two projects.
It's a very interesting problem
that needs for all the people who
found that community to be interested and aware.
Yes, in jobs,
how would you remain
in public security risk
involved so that [inaudible]
Only as the admin check.
The check is essentially that co-exist almost.
The desk rejection that you will have
another pre-review issue there also,
the public in the sense that there may be like
an email to the editor.
Yes, we think that it does not reach
ourselves but it's just an API wrapper to some
of the website. Yes, I agree.
Then we have an internal honesty.
We've agreed upon that.
If TAs agreed that something is not suitable,
and thus so then that is enough for desk rejection.
And then it happens in open
that there's a discussion with your friend in open issue.
But we couldn't they go through something very
short done in check where
we would find feed into slippery.
Having been in the side but in somewhere,
so you write some codes,
analyze data, and then [inaudible] data.
[inaudible]
Oh, 50 percent of the submissions to JOSS are Python.
They happened, the most inscripting languages.
Sure. Where do you get the problem?
Is it the [inaudible]
The requirements and the offered guidance says that
it should be a full-featured solution to a problem,
so no half-baked solution.
It doesn't count.
Well, but you're analyzing in particular,
it should go from a to b very
clearly to do an analysis of some process,
not least, something that is viral to it.
Of course, that is for the August and ages to the side.
But the idea is that it
should be some finished piece of code.
We're using code and software to run.
There's no difference between code and software,
it's the magic here.
It's just code, the piece of code that could be
just one long code of [inaudible] I remember we said
different file that work
together to got a research from them.
Or to help to aid
the research enterprise in some particular way.
Okay. The way that [inaudible]
Yes,
you can still do that.
But would you rather
in your other data that you've just mentioned,
include a citation to
an extra data that it's just about the software.
That's great. I want to know why.
I guess what I'm saying is that
the paper that know [inaudible] data
towards that paperwork are thousands and thousands
of papers
[OVERLAPPING]
[inaudible]
Custom scripts that only serve that purpose, and then,
are disposable after the paper is
published, then that would be.
If I ever find the scope [inaudible]
to why read the paper I know?
Oh, I want to do exactly that.
I don't want to spend to be [inaudible]
Somebody introduced it. That is in software.
The [inaudible] we apply there is
would somebody else cite this?
Would anyone else use it
and feel they want to give you credit by suggestion?
That's different care we use.
If yourself where it's useful to
just what other researchers that is not in your group,
and when that researcher uses it,
they'll find that helpful enough that they want.
[inaudible]
Because of the people are trying to give
[NOISE] a reward for writing many different [inaudible].
Yeah, of course there's
some new submissions of reward system in that sense,
but your reputation is on the line.
If we're serious and we wanted to
talk about this and
you think that somebody else was going to use it.
The reviewers are going
to check just the quality of the software.
It doesn't run, doesn't install with
the text that you provide works;
like you said they were supposed to work.
Do you have a read me that
allows somebody else to use that software,
and do you describe in the paper
a justification of that code? That's it.
I think also that it's necessary
all those checks that you've mentioned.
I have code written [inaudible].
[LAUGHTER] But someone could not just
grab that code from my computer and use it,
report on their working site.
[OVERLAPPING] I don't have to do a lot of work,
but I think that you work
where you're supposed to from the beginning,
[LAUGHTER] then you wouldn't have to do a lot of work.
In that sense, Job is
trying to improve the quality of his software.
For all those people out there who've just written
a little bit to use the code to their specific work.
They don't find it worthwhile to go the extra mile.
[OVERLAPPING] If you were
trying to pick the software and you are
thinking can't because actually is quite useful,
so I would like somebody else to actually be able
to take advantage of all these three months of work,
unless if we repeat the code in six months, who knows.
You take the trouble to actually write the code in a way
that is well-documented,
as test, as examples how to run it.
If you're close to that,
but not quite there because you need to
add a Read Me and add some tests,
and other contribution guides.
But you have be accounted of that.
If you do that, it'll get you to
a quality then needed to submit to
Jobs and get a paper from
your software, then that's great.
You will go that extra mile
and improve the quality of this software.
Essentially what the people do this and let
the other people who actually might do code
the [inaudible] The way
that I could really understand that.
That I'm not going to publish my code [inaudible]
Yes, one of the requirements, of course,
when you write the paper is to cite
any other work that doesn't fit
any other code that does [OVERLAPPING].
You've all heard. At what point does it become enough?
Well, the same rules
apply even when you rely on other people's research.
You always rely on previous research,
but all doesn't reproduce it.
For a code when even the same authors
could submit a new paper for the same software,
if they've added new features and made
some major improvements that runs
a new meaning of the software.
We just did that, the fact that submitted
the second paper on a particular piece of code
where the whole code rule from Python 2.7, 5.3.
In addition, there's a whole new functionality
added where all of the calculations were
previously done for real numbers
and other done from complex numbers
because there's a new application
there that this software the would solve.
With all of these new addition,
we've done a new release of
the software and we submitted a new page.
As such we [inaudible] the
previous one and describe all the differences.
There's a way for these things to
just organize through community discussions.
All the discussions are happening publicly.
Whenever new neighbor is submitted,
there's a discussion on the review issue where
people who are grappling
with the same questions that you have,
and then there's next paper that fits in that mode,
we've already had that discussion,
it's publicly linked to that.
This is our decision for that time.
This is similar to data submission 25 or I don't know,
that have developed. Yes.
You can [inaudible]?
No, we try not to.
Imagine, you can [inaudible].
We're try not to judge it in terms of novelty,
but in terms of that just solved a new problem that
the other one is solved.
It's up for the authors to
decide whether they want
to issue a new release on [NOISE].
I think an interesting part of
this and also getting datasets and we don't have,
we haven't had mechanisms to do this.
Now we can start doing this.
What happens when those steps available?
Is it going to be picked up and used?
You can set up a time to make
your little script that makes
figure available in five years later but use it?
We don't really know the answer.
But after I would imagine
as this stuff is made available jobs,
there will be some cultural and norms
and form around this.
You made
available the script that made this first figure.
I can do that in
two days or how long it takes, whatever.
But where it's holding up a new algorithm, yeah,
but maybe this stuff will be weeded out in time,
then it's not necessarily up to others to do this, but-
[inaudible].
For new students who's never done any scripting before,
you made a script that takes the data.
Somebody you've never done this before
and that's awesome.
If you've documented,
you can figure out what the code does.
It's in the item of user software.
I would trust that the authors value
their reputation enough to spread some in their products.
That's how it works in Open source.
People know that this stuff is out there and
people are proud of their work and they show it off,
and there's credibility and friends people say.
Friends in vernacular or
when somebody has a lot of quality he has friends
because he's contributed
several projects that people don't know [inaudible].
He contributed to
their big Scientific Python
library and he has applications.
Time is up for this term,
but I want to thank you all for coming and [OVERLAPPING]