WEBVTT
1
00:00:03.140 --> 00:00:08.039 A:middle L:90%
We are doubly honored today to have Dr. Leon Osterweil
2
00:00:08.039 --> 00:00:11.279 A:middle L:90%
and Dr. Lori Clarke both from University of Massachusetts
3
00:00:11.839 --> 00:00:15.480 A:middle L:90%
and both distinguished lecturers in our program
4
00:00:15.490 --> 00:00:18.710 A:middle L:90%
eminent folks in the software engineering community. So let
5
00:00:18.710 --> 00:00:21.750 A:middle L:90%
me embarrass them by saying a few things about their
6
00:00:21.750 --> 00:00:24.870 A:middle L:90%
achievements and I'm just cherry picking here. Their bios are
7
00:00:24.870 --> 00:00:28.370 A:middle L:90%
much more impressive and much longer. So, Dr. Clarke is a
8
00:00:28.379 --> 00:00:32.250 A:middle L:90%
current chair of the Department of Computer Science at the
9
00:00:32.250 --> 00:00:34.810 A:middle L:90%
University of Massachusetts. She is a fellow of the
10
00:00:34.810 --> 00:00:38.570 A:middle L:90%
ACM and of the IEEE. Her most recent award
11
00:00:38.570 --> 00:00:43.009 A:middle L:90%
already mentioned one, is uh, the University of Massachusetts
12
00:00:43.020 --> 00:00:48.090 A:middle L:90%
Outstanding Accomplishments in Research and Creativity award given in 2011
13
00:00:48.090 --> 00:00:50.829 A:middle L:90%
And she's won other awards for her work.
14
00:00:50.840 --> 00:00:54.659 A:middle L:90%
Uh she is a board member and former co-chair
15
00:00:54.670 --> 00:00:58.200 A:middle L:90%
of the Computing Research Association's Committee on the Status of
16
00:00:58.200 --> 00:00:59.820 A:middle L:90%
Women and has done a lot of good things for
17
00:00:59.820 --> 00:01:06.620 A:middle L:90%
diversity in the community. And um we definitely honored because actually both
18
00:01:06.620 --> 00:01:19.030 A:middle L:90%
of these individuals are awardees in of a very high award in the SIGSOFT, ACM SIGSOFT community which is the Special Interest Group on Software Engineering for ACM.
19
00:01:19.040 --> 00:01:33.930 A:middle L:90%
They have both won outstanding research awards for their research in software engineering. Leon won it a few years earlier than Lori did. He's older, because he's older.
20
00:01:33.930 --> 00:01:37.819 A:middle L:90%
Let's talk a little bit about Dr. Osterweil, Dr. Osterweil is a professor in
21
00:01:37.819 --> 00:01:40.709 A:middle L:90%
the Department of Computer Science at University of Massachusetts,
22
00:01:40.719 --> 00:01:42.579 A:middle L:90%
and he has also served as dean of the College
23
00:01:42.579 --> 00:01:47.400 A:middle L:90%
of Natural Sciences and Mathematics for four years at Massachusetts
24
00:01:47.409 --> 00:01:49.000 A:middle L:90%
In addition to that, chaired the computer science
25
00:01:49.000 --> 00:01:53.280 A:middle L:90%
departments at UC Irvine and the University of Colorado
26
00:01:53.280 --> 00:01:57.060 A:middle L:90%
in Boulder. Uh and I mentioned, um,
27
00:01:57.439 --> 00:02:00.359 A:middle L:90%
he's uh has has received the outstanding research award from SIGSOFT
28
00:02:00.359 --> 00:02:02.469 A:middle L:90%
often, many other rewards. Both of them are
29
00:02:02.469 --> 00:02:07.049 A:middle L:90%
ACM fellows. Uh, and Dr. Osterweil also
30
00:02:07.060 --> 00:02:12.460 A:middle L:90%
has recently got received SIGSOFT's The Most Influential Educator award
31
00:02:12.840 --> 00:02:15.990 A:middle L:90%
So again, I'm cherry picking among the rewards
32
00:02:15.000 --> 00:02:17.090 A:middle L:90%
I'm very happy to have them visiting with us
33
00:02:17.099 --> 00:02:20.860 A:middle L:90%
today and I think we're going to be interested in
34
00:02:20.870 --> 00:02:23.490 A:middle L:90%
the applications that they're making out of some of their software
35
00:02:23.490 --> 00:02:29.800 A:middle L:90%
engineering research. Thank you, Barbara. I
36
00:02:29.800 --> 00:02:30.159 A:middle L:90%
think one of the things that Lori and I are
37
00:02:30.159 --> 00:02:32.159 A:middle L:90%
most proud about is that yes, we've both won
38
00:02:32.159 --> 00:02:36.550 A:middle L:90%
the outstanding research award, but it's not for work
39
00:02:36.550 --> 00:02:38.719 A:middle L:90%
we've done together. It's for work we've done separately
40
00:02:38.729 --> 00:02:40.120 A:middle L:90%
So each of us has done a body of
41
00:02:40.120 --> 00:02:45.520 A:middle L:90%
work that was recognized separately and individually. But what
42
00:02:45.520 --> 00:02:46.539 A:middle L:90%
we're going to talk about today is something that we
43
00:02:46.539 --> 00:02:52.360 A:middle L:90%
are doing together. And as you can see it
44
00:02:52.360 --> 00:02:54.629 A:middle L:90%
is all about using process modeling and analysis to reduce
45
00:02:55.039 --> 00:02:59.020 A:middle L:90%
errors in healthcare. So many of you may be
46
00:02:59.020 --> 00:03:00.569 A:middle L:90%
aware of some of these statistics, but they're jarring
47
00:03:00.569 --> 00:03:04.419 A:middle L:90%
statistics. Uh, there was a study by the
48
00:03:04.430 --> 00:03:08.810 A:middle L:90%
Institute of Medicine in 1999 that estimated that 100,000 people
49
00:03:08.810 --> 00:03:16.949 A:middle L:90%
die in us hospitals from preventable errors and notice the
50
00:03:16.960 --> 00:03:21.490 A:middle L:90%
word avoidable, preventable is emphasized. So this is
51
00:03:21.490 --> 00:03:24.780 A:middle L:90%
not incorrect diagnoses or a surgeon whose hand slipped or
52
00:03:24.780 --> 00:03:28.280 A:middle L:90%
something like that. This is people getting the wrong
53
00:03:28.280 --> 00:03:30.719 A:middle L:90%
type of blood, People having one cancerous lung
54
00:03:30.729 --> 00:03:35.150 A:middle L:90%
left in while the healthy lung is removed and things
55
00:03:35.150 --> 00:03:38.219 A:middle L:90%
of that sort 100,000 per year in the US.
56
00:03:38.539 --> 00:03:42.310 A:middle L:90%
It is widely believed that this is an underestimate by
57
00:03:42.310 --> 00:03:45.030 A:middle L:90%
about a factor of two. That would make this
58
00:03:45.030 --> 00:03:46.050 A:middle L:90%
one of the leading causes of death in the United
59
00:03:46.050 --> 00:03:49.560 A:middle L:90%
States far more than the number of people who die
60
00:03:49.560 --> 00:03:53.199 A:middle L:90%
from auto accidents and job related injuries and so on
61
00:03:53.199 --> 00:03:55.860 A:middle L:90%
and so forth. This does not count near misses
62
00:03:57.240 --> 00:04:00.719 A:middle L:90%
where people have been maimed perhaps for life or subjected
63
00:04:00.729 --> 00:04:05.060 A:middle L:90%
to a great deal of pain and aggravation and agony
64
00:04:05.439 --> 00:04:09.349 A:middle L:90%
and hundreds of billions of dollars wasted. So this
65
00:04:09.349 --> 00:04:13.930 A:middle L:90%
is an enormous problem. That number basically is told
66
00:04:13.939 --> 00:04:16.000 A:middle L:90%
people and people don't quite exactly internalize it the way
67
00:04:16.000 --> 00:04:19.790 A:middle L:90%
we would like Um, consider one fully loaded Boeing
68
00:04:19.790 --> 00:04:26.060 A:middle L:90%
747 crashing every day of the year, 365 days
69
00:04:26.439 --> 00:04:29.329 A:middle L:90%
. And that's the number of people. And clearly
70
00:04:29.339 --> 00:04:30.800 A:middle L:90%
if there was a 747 that went down every day
71
00:04:30.800 --> 00:04:33.319 A:middle L:90%
killing everybody on board, there would be a tremendous
72
00:04:33.319 --> 00:04:36.029 A:middle L:90%
human cry about the safety issues and what are we
73
00:04:36.029 --> 00:04:38.959 A:middle L:90%
gonna do about it and so on and so forth
74
00:04:39.540 --> 00:04:42.139 A:middle L:90%
. And uh, but that sort of hew and
75
00:04:42.139 --> 00:04:44.449 A:middle L:90%
cry has, has not yet arisen, although the
76
00:04:44.449 --> 00:04:46.610 A:middle L:90%
health care community is now very focused on trying to
77
00:04:46.610 --> 00:04:48.740 A:middle L:90%
do something about this. There was a follow up
78
00:04:48.740 --> 00:04:51.089 A:middle L:90%
study 10 years later in 2000 and nine to see
79
00:04:51.089 --> 00:04:54.079 A:middle L:90%
how they, how things seem to be going.
80
00:04:54.139 --> 00:04:56.839 A:middle L:90%
And there was no evidence that things had improved,
81
00:04:56.839 --> 00:04:58.829 A:middle L:90%
even though there had been 10 years of focus on
82
00:04:58.829 --> 00:05:00.560 A:middle L:90%
this problem and we're now a few years further down
83
00:05:02.379 --> 00:05:06.160 A:middle L:90%
This is a quote from a 2009 National Research Council
84
00:05:06.160 --> 00:05:13.240 A:middle L:90%
report and the thing that is emphasized here inherent intellectual
85
00:05:13.240 --> 00:05:15.759 A:middle L:90%
complexity really starts to get into what it is we
86
00:05:15.759 --> 00:05:18.120 A:middle L:90%
focus on and what it is we think is compelling
87
00:05:18.129 --> 00:05:21.269 A:middle L:90%
for computer scientists. The plain fact is that health
88
00:05:21.269 --> 00:05:25.949 A:middle L:90%
care has become an extremely complicated enterprise these days.
89
00:05:26.339 --> 00:05:30.000 A:middle L:90%
And it involves the interplay of many different people,
90
00:05:30.000 --> 00:05:32.250 A:middle L:90%
different kinds of people. Many devices. There has
91
00:05:32.250 --> 00:05:38.040 A:middle L:90%
been the advent of all kinds of software innovations like
92
00:05:38.040 --> 00:05:41.029 A:middle L:90%
electronic health records and so on. And each of
93
00:05:41.029 --> 00:05:44.480 A:middle L:90%
these is an intricate player in its own right.
94
00:05:44.529 --> 00:05:46.910 A:middle L:90%
The interplay of them has created systems that are far
95
00:05:46.910 --> 00:05:50.120 A:middle L:90%
beyond the capabilities of the people who are participants to
96
00:05:50.120 --> 00:05:54.240 A:middle L:90%
understand. And as a consequence, we now have
97
00:05:54.240 --> 00:05:56.639 A:middle L:90%
doctors who are very well trained in the medical arts
98
00:05:56.649 --> 00:05:59.740 A:middle L:90%
but are not systems people, but they are now
99
00:05:59.750 --> 00:06:01.360 A:middle L:90%
in meshed in these systems and trying to wield them
100
00:06:01.939 --> 00:06:04.509 A:middle L:90%
. So, um, that takes us to the
101
00:06:04.509 --> 00:06:08.209 A:middle L:90%
larger issue which is of interest to us. And
102
00:06:08.209 --> 00:06:11.819 A:middle L:90%
that is the fact that such systems are really what
103
00:06:11.829 --> 00:06:14.360 A:middle L:90%
runs our society these days. So, health care
104
00:06:14.360 --> 00:06:18.079 A:middle L:90%
is a glaring example. And this estimate about 100,000
105
00:06:18.079 --> 00:06:23.259 A:middle L:90%
deaths is extremely compelling. But pretty much every other
106
00:06:23.269 --> 00:06:26.350 A:middle L:90%
large scale system is the same kind of a thing
107
00:06:26.740 --> 00:06:30.160 A:middle L:90%
. A system that involves the participation and interaction of
108
00:06:30.160 --> 00:06:33.860 A:middle L:90%
lots of different kinds of people with increasingly complicated devices
109
00:06:33.870 --> 00:06:36.579 A:middle L:90%
. So one of our other studies is on elections
110
00:06:36.589 --> 00:06:40.579 A:middle L:90%
where we're trying to look into why elections don't work
111
00:06:40.579 --> 00:06:42.670 A:middle L:90%
as well as they should. And we're discovering the
112
00:06:42.670 --> 00:06:46.089 A:middle L:90%
same thing. Too many people doing too many uh
113
00:06:46.100 --> 00:06:49.620 A:middle L:90%
intricately interconnected kinds of things supported by devices which think
114
00:06:49.620 --> 00:06:51.459 A:middle L:90%
they're going to make all the problems go away,
115
00:06:51.459 --> 00:06:56.509 A:middle L:90%
but which somehow seem to just be complicating the issues
116
00:06:57.240 --> 00:07:00.459 A:middle L:90%
. So we call these human intensive systems. They
117
00:07:00.459 --> 00:07:03.310 A:middle L:90%
involve collaborations among people software, hardware devices and there
118
00:07:03.310 --> 00:07:08.269 A:middle L:90%
are some examples flight control, which is well studied
119
00:07:08.329 --> 00:07:14.170 A:middle L:90%
emergency response, how an emergency room works voting devices
120
00:07:14.180 --> 00:07:15.449 A:middle L:90%
, the entire electric power grids. All of these
121
00:07:15.449 --> 00:07:19.240 A:middle L:90%
things are examples of human intensive systems and they all
122
00:07:19.240 --> 00:07:23.040 A:middle L:90%
seem to be at the point where they are taxing
123
00:07:23.040 --> 00:07:26.319 A:middle L:90%
the ability of the human participants to deal with them
124
00:07:26.329 --> 00:07:30.759 A:middle L:90%
. So our approach which we call continuous process improvement
125
00:07:30.139 --> 00:07:34.569 A:middle L:90%
is basically a riff on continuous process improvement as practiced
126
00:07:34.579 --> 00:07:39.850 A:middle L:90%
by industrial engineers and and often in business management so
127
00:07:39.850 --> 00:07:42.269 A:middle L:90%
on. And it says basically if you want to
128
00:07:42.269 --> 00:07:46.649 A:middle L:90%
understand what's actually going on for the purpose of improving
129
00:07:46.649 --> 00:07:47.480 A:middle L:90%
it, making it better, making it safer and
130
00:07:47.480 --> 00:07:49.610 A:middle L:90%
so and so forth. One way you can do
131
00:07:49.610 --> 00:07:53.910 A:middle L:90%
that is to take the processes these systems we think
132
00:07:53.910 --> 00:07:56.610 A:middle L:90%
of as being composites of lots of different processes.
133
00:07:56.639 --> 00:08:00.889 A:middle L:90%
Take these processes, model them take the models and
134
00:08:00.889 --> 00:08:03.810 A:middle L:90%
evaluate the models and here what we're going to be
135
00:08:03.810 --> 00:08:07.800 A:middle L:90%
talking about today is the arsenal of analytic techniques that
136
00:08:07.800 --> 00:08:11.720 A:middle L:90%
we're throwing at these process definitions and after you discover
137
00:08:11.720 --> 00:08:16.769 A:middle L:90%
some defect or some shortcoming, fix the defect and
138
00:08:16.779 --> 00:08:20.709 A:middle L:90%
go back. So this is basically for those of
139
00:08:20.709 --> 00:08:22.800 A:middle L:90%
use our study industrial engineering, the shoe heart cycle
140
00:08:22.800 --> 00:08:26.959 A:middle L:90%
or the deming cycle of continuous process improvement. Notice
141
00:08:26.959 --> 00:08:31.300 A:middle L:90%
however, that we now have a couple of things
142
00:08:31.300 --> 00:08:33.149 A:middle L:90%
we put on the bottom because what's happening here is
143
00:08:33.149 --> 00:08:37.940 A:middle L:90%
a mostly intellectual exercise and taking a model improving the
144
00:08:37.940 --> 00:08:39.509 A:middle L:90%
model, making the model better and so on and
145
00:08:39.509 --> 00:08:41.129 A:middle L:90%
so forth. But the model has not yet engaged
146
00:08:41.139 --> 00:08:43.889 A:middle L:90%
the world. And the goal of course is to
147
00:08:43.889 --> 00:08:46.740 A:middle L:90%
save 100,000 lives a year and that involves engaging the
148
00:08:46.740 --> 00:08:50.000 A:middle L:90%
world. So the next loop is after you get
149
00:08:50.000 --> 00:08:52.740 A:middle L:90%
to the point where the model has been improved sufficiently
150
00:08:52.750 --> 00:08:58.840 A:middle L:90%
you believe then actually deploy these models in the real
151
00:08:58.850 --> 00:09:03.110 A:middle L:90%
context and evaluate them in the real context in this
152
00:09:03.110 --> 00:09:05.620 A:middle L:90%
case, the clinical setting and from the experience in
153
00:09:05.620 --> 00:09:09.059 A:middle L:90%
the clinical setting, this is outside loop. So
154
00:09:09.159 --> 00:09:13.009 A:middle L:90%
we have been mostly engaged in the inner loop and
155
00:09:13.009 --> 00:09:15.789 A:middle L:90%
now we are with our latest research grant moving into
156
00:09:15.789 --> 00:09:18.019 A:middle L:90%
this outer loop where we're intending to actually deploy these
157
00:09:18.019 --> 00:09:24.000 A:middle L:90%
things in the world. Okay, so the focus
158
00:09:24.000 --> 00:09:26.750 A:middle L:90%
of the talk is on the technologies so we are
159
00:09:26.750 --> 00:09:28.929 A:middle L:90%
not doctors, although we're learning a lot about health
160
00:09:28.929 --> 00:09:31.259 A:middle L:90%
care. One of the things we're learning is do
161
00:09:31.259 --> 00:09:35.600 A:middle L:90%
not ever go to an emergency room and of course
162
00:09:35.600 --> 00:09:37.309 A:middle L:90%
if you need an emergency room you sort of have
163
00:09:37.309 --> 00:09:39.129 A:middle L:90%
to go to an emergency room. But the idea
164
00:09:39.129 --> 00:09:41.519 A:middle L:90%
when we ask emergency docs, what do you fear
165
00:09:41.519 --> 00:09:43.000 A:middle L:90%
most? And they say mostly we fear about ever
166
00:09:43.000 --> 00:09:46.299 A:middle L:90%
having to be a patient in our own emergency room
167
00:09:46.309 --> 00:09:48.480 A:middle L:90%
. Uh but mostly we're going to give you the
168
00:09:48.480 --> 00:09:52.950 A:middle L:90%
systems approach and the system approach basically is summarized here
169
00:09:52.960 --> 00:09:56.759 A:middle L:90%
, starting out by uh using a language which was
170
00:09:56.759 --> 00:10:01.289 A:middle L:90%
a home grown language which will describe uh just a
171
00:10:01.289 --> 00:10:03.389 A:middle L:90%
modest amount of detail called little Jill, which is
172
00:10:03.389 --> 00:10:07.590 A:middle L:90%
a process language and is designed to model processes of
173
00:10:07.590 --> 00:10:09.809 A:middle L:90%
these exact kinds with the diversity of different kinds of
174
00:10:09.809 --> 00:10:13.289 A:middle L:90%
performers that are interacting with each other in all kinds
175
00:10:13.289 --> 00:10:16.120 A:middle L:90%
of complicated ways, throwing exceptions and working in peril
176
00:10:16.120 --> 00:10:18.399 A:middle L:90%
and so on so forth. Uh that is to
177
00:10:18.399 --> 00:10:22.070 A:middle L:90%
capture the process. Then we have a tool called
178
00:10:22.070 --> 00:10:24.379 A:middle L:90%
propel which Laurie will be talking about, which is
179
00:10:24.389 --> 00:10:30.370 A:middle L:90%
an exquisitely uh precise tool for capturing exactly what kinds
180
00:10:30.370 --> 00:10:33.669 A:middle L:90%
of behaviors you would like to see. And Laurie
181
00:10:33.669 --> 00:10:35.409 A:middle L:90%
will have some examples of that. Then having captured
182
00:10:35.409 --> 00:10:39.210 A:middle L:90%
the process with little Jill and the decide arata with
183
00:10:39.210 --> 00:10:41.240 A:middle L:90%
propel. Then there is a model checking tool which
184
00:10:41.240 --> 00:10:45.700 A:middle L:90%
is also something that's been built at u mass um
185
00:10:45.710 --> 00:10:48.980 A:middle L:90%
called flavors and we use flavors to determine. Is
186
00:10:48.980 --> 00:10:50.250 A:middle L:90%
it possible to execute this process in such a way
187
00:10:50.250 --> 00:10:56.629 A:middle L:90%
that that the constraints are violated. Um there's another
188
00:10:56.629 --> 00:11:01.480 A:middle L:90%
kind of analysis. We do fault tree analysis which
189
00:11:01.480 --> 00:11:03.950 A:middle L:90%
is different from the model checking. The model checking
190
00:11:03.950 --> 00:11:05.360 A:middle L:90%
basically says here are a set of tasks if they're
191
00:11:05.360 --> 00:11:09.289 A:middle L:90%
done right, will there be incorrect and worrisome sequences
192
00:11:09.389 --> 00:11:13.230 A:middle L:90%
fault tree analysis says supposing some of these tasks were
193
00:11:13.230 --> 00:11:16.379 A:middle L:90%
done wrong. What would be the ramifications? And
194
00:11:16.379 --> 00:11:18.649 A:middle L:90%
in doing this we're hoping to identify for example single
195
00:11:18.649 --> 00:11:22.039 A:middle L:90%
points of failure tests that are done which need to
196
00:11:22.039 --> 00:11:24.659 A:middle L:90%
be right which have some strong possibility being done well
197
00:11:26.440 --> 00:11:28.750 A:middle L:90%
. And then as time permits we may also talk
198
00:11:28.750 --> 00:11:31.350 A:middle L:90%
about a discrete event simulator which we built whose job
199
00:11:31.350 --> 00:11:35.899 A:middle L:90%
is to study the optimal allocation of all of these
200
00:11:35.899 --> 00:11:39.259 A:middle L:90%
diverse resources in order to make these processes run better
201
00:11:39.639 --> 00:11:41.909 A:middle L:90%
. So there is a battery of things that we're
202
00:11:41.909 --> 00:11:46.240 A:middle L:90%
doing taking different looks at these processes. So this
203
00:11:46.240 --> 00:11:50.059 A:middle L:90%
does involve working with medical professionals which is interesting.
204
00:11:50.440 --> 00:11:52.539 A:middle L:90%
Um There's a lot we we will be saying about
205
00:11:52.539 --> 00:11:56.409 A:middle L:90%
this as we go along but these are the three
206
00:11:56.419 --> 00:11:58.500 A:middle L:90%
areas that we have actually applied these technologies to.
207
00:11:58.509 --> 00:12:03.419 A:middle L:90%
First is chemotherapy example, breast cancer, chemotherapy.
208
00:12:03.419 --> 00:12:07.080 A:middle L:90%
We're working with mostly with colleagues at the Bay State
209
00:12:07.080 --> 00:12:09.549 A:middle L:90%
Medical Center which is a very large tertiary care facility
210
00:12:09.809 --> 00:12:13.120 A:middle L:90%
that operates one of the largest emergency rooms in the
211
00:12:13.120 --> 00:12:16.309 A:middle L:90%
U. S. Northeast. Um We've also working
212
00:12:16.309 --> 00:12:20.539 A:middle L:90%
with a professor of nursing Beth Henman on the in
213
00:12:20.539 --> 00:12:22.210 A:middle L:90%
patient blood transfusion process and we have a very well
214
00:12:22.210 --> 00:12:26.139 A:middle L:90%
articulated specification of the process and properties that should adhere
215
00:12:26.139 --> 00:12:31.080 A:middle L:90%
to and then we are doing emergency department patient flow
216
00:12:31.090 --> 00:12:33.539 A:middle L:90%
trying to determine How can you move patients through the
217
00:12:33.539 --> 00:12:37.129 A:middle L:90%
emergency room faster. Where we've discovered is the average
218
00:12:37.129 --> 00:12:39.679 A:middle L:90%
waiting time in an emergency room is anywhere from 6-8
219
00:12:39.690 --> 00:12:41.899 A:middle L:90%
hours. And at first it seems as a very
220
00:12:41.899 --> 00:12:46.059 A:middle L:90%
simple uh problem to fix you just throw more resources
221
00:12:46.059 --> 00:12:46.820 A:middle L:90%
at it and then the emergency room doctor said no
222
00:12:46.820 --> 00:12:48.639 A:middle L:90%
, no, no, you can't continue to throw
223
00:12:48.639 --> 00:12:52.230 A:middle L:90%
resources at it because we will lose money. So
224
00:12:52.230 --> 00:12:54.789 A:middle L:90%
it actually is a complicated optimization problems. So this
225
00:12:54.789 --> 00:12:58.120 A:middle L:90%
is something of a sort of a roadmap of what
226
00:12:58.120 --> 00:13:00.159 A:middle L:90%
we're trying to do. We're trying to apply these
227
00:13:00.159 --> 00:13:03.940 A:middle L:90%
technologies to make big improvements in these areas with the
228
00:13:03.940 --> 00:13:07.259 A:middle L:90%
expectations that what we have learned in the approaches that
229
00:13:07.259 --> 00:13:11.360 A:middle L:90%
we're taking will apply to other processing and other domains
230
00:13:11.399 --> 00:13:16.220 A:middle L:90%
. Indeed. So what we've discovered about these processes
231
00:13:16.230 --> 00:13:20.269 A:middle L:90%
is that they are incredibly complex. So very often
232
00:13:20.269 --> 00:13:22.809 A:middle L:90%
people ask this language which you're about to see little
233
00:13:22.809 --> 00:13:26.019 A:middle L:90%
Jill, you invented this took a lot of effort
234
00:13:26.029 --> 00:13:26.850 A:middle L:90%
, why didn't you use something off the shelf?
235
00:13:28.340 --> 00:13:30.860 A:middle L:90%
And what we discovered is that the languages that exist
236
00:13:30.870 --> 00:13:33.649 A:middle L:90%
on the shelf to define processes really lack the semantic
237
00:13:33.659 --> 00:13:37.059 A:middle L:90%
richness that's needed to really capture what real processes are
238
00:13:37.059 --> 00:13:41.899 A:middle L:90%
like. So here are just some of the descriptors
239
00:13:41.909 --> 00:13:45.070 A:middle L:90%
. The languages the processes themselves are complex. They
240
00:13:45.080 --> 00:13:46.600 A:middle L:90%
tend to be concurrent. They tend to be exception
241
00:13:46.600 --> 00:13:48.789 A:middle L:90%
rich. So when we told our nurse we have
242
00:13:48.789 --> 00:13:52.129 A:middle L:90%
a language that deals with exceptions. Is that something
243
00:13:52.129 --> 00:13:54.159 A:middle L:90%
that's important? She said exceptions are my whole life
244
00:13:54.639 --> 00:13:58.159 A:middle L:90%
. So most processed languages do a poor job of
245
00:13:58.169 --> 00:14:01.570 A:middle L:90%
dealing with exceptions. They need to support human choice
246
00:14:01.580 --> 00:14:05.779 A:middle L:90%
and flexibility tasks may involve multi processing and multitasking.
247
00:14:05.789 --> 00:14:09.289 A:middle L:90%
And the list of the separator for this language goes
248
00:14:09.299 --> 00:14:11.289 A:middle L:90%
on and on and on. At the end you
249
00:14:11.289 --> 00:14:13.929 A:middle L:90%
wind up with a very, very complicated language which
250
00:14:13.929 --> 00:14:16.610 A:middle L:90%
really needs to borrow a lot from programming languages.
251
00:14:16.139 --> 00:14:18.809 A:middle L:90%
And that's why we invented our own. And in
252
00:14:18.809 --> 00:14:22.169 A:middle L:90%
fact the work we've done and applying it has indicated
253
00:14:22.200 --> 00:14:24.669 A:middle L:90%
that the language itself still doesn't have all the stuff
254
00:14:24.669 --> 00:14:31.000 A:middle L:90%
it needs and has led to improvements, complexity.
255
00:14:31.039 --> 00:14:37.090 A:middle L:90%
The languages has a strong semantic basis. It's defined
256
00:14:37.090 --> 00:14:39.620 A:middle L:90%
in terms of finite state machines. And the main
257
00:14:39.620 --> 00:14:41.570 A:middle L:90%
reason for that is if we're gonna do definitive analyses
258
00:14:41.570 --> 00:14:46.720 A:middle L:90%
, we need a precisely defined language Has timing,
259
00:14:46.730 --> 00:14:50.090 A:middle L:90%
resource utilization specifications and so on and so forth.
260
00:14:50.100 --> 00:14:52.820 A:middle L:90%
And then at the end the last one is probably
261
00:14:52.820 --> 00:14:54.210 A:middle L:90%
the hardest. You know, some of you may
262
00:14:54.210 --> 00:14:56.700 A:middle L:90%
be thinking about 10th order logics that could do all
263
00:14:56.700 --> 00:15:00.509 A:middle L:90%
of these things. It needs to be understandable by
264
00:15:00.509 --> 00:15:03.330 A:middle L:90%
a medical professional because at the end we have this
265
00:15:03.330 --> 00:15:05.610 A:middle L:90%
process and we're about to study it and maybe deployed
266
00:15:05.769 --> 00:15:07.899 A:middle L:90%
. Is this your process? Is this what you
267
00:15:07.899 --> 00:15:11.179 A:middle L:90%
do? And medical professionals need to see something visual
268
00:15:11.639 --> 00:15:15.059 A:middle L:90%
. So the language is visual in a moment,
269
00:15:15.059 --> 00:15:16.399 A:middle L:90%
you'll see an example of it. It's centered on
270
00:15:16.399 --> 00:15:20.539 A:middle L:90%
this idea of a step and instead of thinking of
271
00:15:20.539 --> 00:15:22.460 A:middle L:90%
this as being a step think of this as being
272
00:15:22.460 --> 00:15:24.029 A:middle L:90%
a procedure. So the language is in fact a
273
00:15:24.029 --> 00:15:28.000 A:middle L:90%
hierarchical decomposition language with some things added pre and post
274
00:15:28.000 --> 00:15:33.870 A:middle L:90%
conditions interface badge that describes the flow of artifacts through
275
00:15:33.870 --> 00:15:37.350 A:middle L:90%
it. Uh it is scoped language so it has
276
00:15:37.350 --> 00:15:39.679 A:middle L:90%
scoped exception handlers and then it finally has a flow
277
00:15:39.679 --> 00:15:43.480 A:middle L:90%
of control specify here that says the order in which
278
00:15:43.480 --> 00:15:45.700 A:middle L:90%
the sub steps can be executed. Instead of talking
279
00:15:45.700 --> 00:15:50.870 A:middle L:90%
about the picture here is the impatient blood transfusion process
280
00:15:50.870 --> 00:15:54.399 A:middle L:90%
. It's the beginning of a hierarchical decomposition and it
281
00:15:54.399 --> 00:15:58.580 A:middle L:90%
says basically patient blood transfusion process. Uh inpatient is
282
00:15:58.580 --> 00:16:03.929 A:middle L:90%
hidden under this triangle is a sequential execution of a
283
00:16:03.929 --> 00:16:07.190 A:middle L:90%
cleaning plus number of instances of carry out physician order
284
00:16:07.190 --> 00:16:11.529 A:middle L:90%
for transfusion which consists of the sequential execution of check
285
00:16:11.529 --> 00:16:14.000 A:middle L:90%
for the type, prepare the documentation for blood.
286
00:16:14.000 --> 00:16:15.149 A:middle L:90%
Pick up, pick up the blood, do single
287
00:16:15.149 --> 00:16:18.529 A:middle L:90%
unit transfusion cleaning plus number of times, do a
288
00:16:18.529 --> 00:16:22.190 A:middle L:90%
follow through check. And then up here we have
289
00:16:22.330 --> 00:16:25.610 A:middle L:90%
an exception handler and as they said exceptions are very
290
00:16:25.610 --> 00:16:27.149 A:middle L:90%
important. So you notice that two of these steps
291
00:16:27.149 --> 00:16:30.759 A:middle L:90%
have these green triangles and those are preconditions checks the
292
00:16:30.759 --> 00:16:33.799 A:middle L:90%
first one. Check to be sure that this has
293
00:16:33.799 --> 00:16:37.340 A:middle L:90%
been prescribed by a physician. So when you actually
294
00:16:37.340 --> 00:16:38.519 A:middle L:90%
get into this and you listen to people telling you
295
00:16:38.519 --> 00:16:41.019 A:middle L:90%
what it is the process entails. Very often they
296
00:16:41.019 --> 00:16:44.460 A:middle L:90%
leave out these important points until you pull them out
297
00:16:44.460 --> 00:16:47.250 A:middle L:90%
of them. So for example, we always say
298
00:16:47.250 --> 00:16:48.840 A:middle L:90%
since the language as pre and post conditions, is
299
00:16:48.840 --> 00:16:52.909 A:middle L:90%
there a precondition? Well of course the physician needs
300
00:16:52.909 --> 00:16:56.590 A:middle L:90%
to prescribe this blood transfusion. And if not then
301
00:16:56.590 --> 00:17:00.659 A:middle L:90%
this precondition guard kicks you out of this entire process
302
00:17:00.330 --> 00:17:04.380 A:middle L:90%
. There's another one before you carry out the transfusion
303
00:17:04.420 --> 00:17:07.269 A:middle L:90%
confirmed that the patient has filled out a consent.
304
00:17:07.740 --> 00:17:11.069 A:middle L:90%
So this is legally important and it's also pragmatically important
305
00:17:11.069 --> 00:17:14.220 A:middle L:90%
. But in this case and we can see that
306
00:17:14.220 --> 00:17:15.859 A:middle L:90%
this exception gets thrown up to apparent where there is
307
00:17:15.859 --> 00:17:21.309 A:middle L:90%
a matching exception handler and this icon says continue.
308
00:17:21.319 --> 00:17:22.839 A:middle L:90%
Don't do any of this stuff. Don't do the
309
00:17:22.839 --> 00:17:29.970 A:middle L:90%
transfusion without permission. Okay, so I said it's
310
00:17:29.970 --> 00:17:33.359 A:middle L:90%
a hierarchical decomposition language. So just briefly we will
311
00:17:33.369 --> 00:17:37.880 A:middle L:90%
decompose the single unit transfusion process into this. And
312
00:17:37.880 --> 00:17:40.180 A:middle L:90%
you can see how things start to get quite complicated
313
00:17:40.190 --> 00:17:42.160 A:middle L:90%
. So the entire transfusion process that we have is
314
00:17:42.230 --> 00:17:47.710 A:middle L:90%
100 and 50 or 202 150 steps and it gets
315
00:17:47.710 --> 00:17:51.869 A:middle L:90%
down to quite a complex, quite a detailed level
316
00:17:51.880 --> 00:17:53.039 A:middle L:90%
and we stop here. We don't really go too
317
00:17:53.039 --> 00:17:55.759 A:middle L:90%
far. I just wanted to give you a sense
318
00:17:56.140 --> 00:17:59.470 A:middle L:90%
um uh there's a parallel step over here which says
319
00:17:59.480 --> 00:18:03.380 A:middle L:90%
discard transfusion materials and record infusion info. Both have
320
00:18:03.380 --> 00:18:04.589 A:middle L:90%
to happen but the order which they happen is not
321
00:18:04.599 --> 00:18:10.970 A:middle L:90%
important. Uh there's a suspected transfusion reaction process and
322
00:18:10.970 --> 00:18:14.230 A:middle L:90%
this is an example of while the blood transfusion is
323
00:18:14.230 --> 00:18:15.069 A:middle L:90%
going on. The nurse is supposed to be checking
324
00:18:15.069 --> 00:18:17.960 A:middle L:90%
to be sure that everything is going okay. If
325
00:18:17.970 --> 00:18:21.630 A:middle L:90%
not an exception is thrown. And this black bar
326
00:18:21.630 --> 00:18:23.200 A:middle L:90%
means that there's a whole long decomposition of what the
327
00:18:23.200 --> 00:18:27.059 A:middle L:90%
nurse does and artifacts get passed along with this.
328
00:18:27.069 --> 00:18:30.230 A:middle L:90%
So the nurse has information what actually went wrong so
329
00:18:30.230 --> 00:18:34.099 A:middle L:90%
that hopefully the nurse does the right thing. Okay
330
00:18:34.109 --> 00:18:37.619 A:middle L:90%
, so we're actually rather proud of the language.
331
00:18:37.619 --> 00:18:40.220 A:middle L:90%
Some of us are very, very proud of the
332
00:18:40.220 --> 00:18:42.920 A:middle L:90%
language, but it has been pointed out that the
333
00:18:42.930 --> 00:18:45.920 A:middle L:90%
pictures are a little hard to draw, a little
334
00:18:45.920 --> 00:18:48.460 A:middle L:90%
hard to grasp. So Lori and some of the
335
00:18:48.460 --> 00:18:49.779 A:middle L:90%
students a while back said what we really need is
336
00:18:49.779 --> 00:18:55.250 A:middle L:90%
some way of projecting this in english or in some
337
00:18:55.250 --> 00:18:56.859 A:middle L:90%
structured form of english. So there is now a
338
00:18:56.859 --> 00:19:02.170 A:middle L:90%
narrative view of this and this is the narrative rendering
339
00:19:02.539 --> 00:19:03.849 A:middle L:90%
of that process. It's a narrative rendering of a
340
00:19:03.849 --> 00:19:07.359 A:middle L:90%
small part of that process. And on the left
341
00:19:07.480 --> 00:19:07.970 A:middle L:90%
, what you see is a sort of a table
342
00:19:07.970 --> 00:19:11.119 A:middle L:90%
of contents with these live links and you can click
343
00:19:11.119 --> 00:19:14.509 A:middle L:90%
on these links and that will take you to different
344
00:19:14.509 --> 00:19:17.519 A:middle L:90%
parts of the process and what we've discovered is that
345
00:19:17.529 --> 00:19:21.099 A:middle L:90%
some medical professionals really like this view, some medical
346
00:19:21.099 --> 00:19:23.529 A:middle L:90%
professionals really like that view. But all medical professionals
347
00:19:23.529 --> 00:19:26.160 A:middle L:90%
like the fact that there are both and of course
348
00:19:26.160 --> 00:19:29.839 A:middle L:90%
since this is automatically generated, we know that there
349
00:19:29.839 --> 00:19:33.670 A:middle L:90%
is coherence between this textual view and that pictorial view
350
00:19:34.740 --> 00:19:37.220 A:middle L:90%
. Okay, so I'm gonna finish up talking about
351
00:19:37.230 --> 00:19:40.720 A:middle L:90%
the process modeling quickly and turned it over. Glory
352
00:19:40.720 --> 00:19:44.670 A:middle L:90%
to talk about analysis. We've discovered quite a lot
353
00:19:44.670 --> 00:19:49.210 A:middle L:90%
in doing this mostly. Um, one of a
354
00:19:49.220 --> 00:19:52.950 A:middle L:90%
couple of the main points are indicated there first,
355
00:19:52.950 --> 00:19:56.029 A:middle L:90%
the processes are not terribly well understood by the performers
356
00:19:56.150 --> 00:20:00.039 A:middle L:90%
. Since there's so many performers, each performer understands
357
00:20:00.039 --> 00:20:02.720 A:middle L:90%
what it is they do. The doctor knows what
358
00:20:02.730 --> 00:20:04.509 A:middle L:90%
the doctor does and the doctor has expectations about what
359
00:20:04.509 --> 00:20:07.430 A:middle L:90%
a nurse does. And then when you ask the
360
00:20:07.430 --> 00:20:08.940 A:middle L:90%
nurse, the nurse very often doesn't do that.
361
00:20:08.980 --> 00:20:11.880 A:middle L:90%
We had a number of psychodramas between our doctor and
362
00:20:11.880 --> 00:20:15.099 A:middle L:90%
nurse domain experts where the nurse said, okay,
363
00:20:15.099 --> 00:20:17.960 A:middle L:90%
now the doctor does this and the doctor said,
364
00:20:17.960 --> 00:20:18.069 A:middle L:90%
no, no, no, I don't do that
365
00:20:18.069 --> 00:20:19.619 A:middle L:90%
, I do this, this and this, and
366
00:20:19.619 --> 00:20:22.150 A:middle L:90%
then the nurse does this. And then the nurse
367
00:20:22.150 --> 00:20:22.279 A:middle L:90%
popped up and said, no, no, I
368
00:20:22.279 --> 00:20:25.839 A:middle L:90%
don't do that. And then there's the pharmacist and
369
00:20:25.839 --> 00:20:27.059 A:middle L:90%
then there are the people who push the journeys and
370
00:20:27.059 --> 00:20:30.460 A:middle L:90%
each one has a view of what they do and
371
00:20:30.940 --> 00:20:33.259 A:middle L:90%
Presumptions about what other people do. And sometimes those
372
00:20:33.259 --> 00:20:37.490 A:middle L:90%
Presumptions are wrong. So in order to tie these
373
00:20:37.490 --> 00:20:40.460 A:middle L:90%
processes down, there is a tremendous amount of time
374
00:20:40.460 --> 00:20:42.339 A:middle L:90%
spent interviewing and going over and over and over again
375
00:20:42.720 --> 00:20:47.410 A:middle L:90%
eliciting details in order to do that. It's really
376
00:20:47.410 --> 00:20:51.309 A:middle L:90%
important to tie down the terminology. Um there is
377
00:20:51.309 --> 00:20:55.579 A:middle L:90%
the word transfused which is used in connection with blood
378
00:20:55.589 --> 00:20:56.569 A:middle L:90%
and you would think it would be well understood what
379
00:20:56.569 --> 00:21:00.880 A:middle L:90%
transfused is. But sometimes the word infused is used
380
00:21:00.890 --> 00:21:03.559 A:middle L:90%
. Sometimes a distant difference matters. Sometimes it doesn't
381
00:21:04.140 --> 00:21:07.319 A:middle L:90%
, there is a word check, There's a word
382
00:21:07.319 --> 00:21:11.150 A:middle L:90%
verify, there is a word compare and these words
383
00:21:11.150 --> 00:21:14.670 A:middle L:90%
until we started interacting with these folks were used in
384
00:21:14.670 --> 00:21:15.779 A:middle L:90%
a very casual sort of away and it was never
385
00:21:15.779 --> 00:21:18.329 A:middle L:90%
quite clear whether they were different or not. They
386
00:21:18.329 --> 00:21:22.319 A:middle L:90%
are much more precise and crisp and the discourse now
387
00:21:22.329 --> 00:21:23.720 A:middle L:90%
and in fact our professor of nursing says that the
388
00:21:23.730 --> 00:21:29.710 A:middle L:90%
process orientation has caused her to teach nursing differently to
389
00:21:29.710 --> 00:21:30.900 A:middle L:90%
think about it differently and to teach it differently,
390
00:21:30.910 --> 00:21:34.839 A:middle L:90%
which is one of the more gratifying things. Takes
391
00:21:34.839 --> 00:21:38.619 A:middle L:90%
many iterations to define a process. And uh,
392
00:21:38.630 --> 00:21:44.039 A:middle L:90%
we probably actually are involved in an illicit ation of
393
00:21:44.039 --> 00:21:48.549 A:middle L:90%
a new process right now at the University of Massachusetts
394
00:21:48.559 --> 00:21:52.109 A:middle L:90%
Medical Center and we're actually measuring how long it takes
395
00:21:52.109 --> 00:21:53.150 A:middle L:90%
, how many iterations and how many hours it takes
396
00:21:53.150 --> 00:21:56.140 A:middle L:90%
to do this. But it's a lot. So
397
00:21:56.150 --> 00:22:00.579 A:middle L:90%
a bunch of observations. Medical professionals tend to think
398
00:22:00.579 --> 00:22:03.170 A:middle L:90%
in terms of war stories and this actually sort of
399
00:22:03.180 --> 00:22:07.269 A:middle L:90%
causes the whole thing to go longer but more interestingly
400
00:22:07.720 --> 00:22:08.440 A:middle L:90%
. Well, we try to get details of the
401
00:22:08.440 --> 00:22:11.680 A:middle L:90%
process is, they tell us about grotesque things that
402
00:22:11.680 --> 00:22:15.119 A:middle L:90%
have happened. It's a scary to do this.
403
00:22:15.150 --> 00:22:18.920 A:middle L:90%
But these are typically negative scenarios when what we would
404
00:22:18.920 --> 00:22:21.849 A:middle L:90%
like is this sort of the positive, what is
405
00:22:21.849 --> 00:22:26.039 A:middle L:90%
it you are trying to do? Um They're initially
406
00:22:26.039 --> 00:22:29.650 A:middle L:90%
scared by the terminology by the fact that they're dealing
407
00:22:29.650 --> 00:22:32.079 A:middle L:90%
with the systems people but after a while we do
408
00:22:32.089 --> 00:22:33.000 A:middle L:90%
get comfortable with each other but it does take a
409
00:22:33.000 --> 00:22:37.200 A:middle L:90%
while. Um We have not yet found a simple
410
00:22:37.200 --> 00:22:41.450 A:middle L:90%
process. So we are working on a process called
411
00:22:41.450 --> 00:22:44.400 A:middle L:90%
verify I. D. And have been working on
412
00:22:44.400 --> 00:22:47.000 A:middle L:90%
this for a couple of years, verify ID happens
413
00:22:47.000 --> 00:22:48.680 A:middle L:90%
all over the medical domain. And verify ideas.
414
00:22:48.680 --> 00:22:52.779 A:middle L:90%
Just simply do we have the right person here?
415
00:22:52.789 --> 00:22:56.410 A:middle L:90%
Do we have the right blood bag here? And
416
00:22:56.420 --> 00:22:57.609 A:middle L:90%
while you seem like they must be very very simple
417
00:22:57.609 --> 00:23:00.160 A:middle L:90%
things when you think about all the complexities that are
418
00:23:00.160 --> 00:23:04.900 A:middle L:90%
possible, they're emergencies situations that are different situations where
419
00:23:04.900 --> 00:23:07.430 A:middle L:90%
the patient can talk, where the patient can't talk
420
00:23:07.440 --> 00:23:11.279 A:middle L:90%
where the patient can talk but not in english and
421
00:23:11.279 --> 00:23:12.819 A:middle L:90%
on and on and on. So every one of
422
00:23:12.819 --> 00:23:17.990 A:middle L:90%
these processes is amazingly complex. And when you extrapolate
423
00:23:17.990 --> 00:23:18.900 A:middle L:90%
this to the way our world works it's almost a
424
00:23:18.900 --> 00:23:22.710 A:middle L:90%
wonder that anything works because every simple process turns out
425
00:23:22.720 --> 00:23:26.349 A:middle L:90%
to be not simple at all. And that's where
426
00:23:26.349 --> 00:23:29.339 A:middle L:90%
I think one of the features of the language,
427
00:23:29.339 --> 00:23:32.390 A:middle L:90%
the fact that the language has these tags connected to
428
00:23:32.390 --> 00:23:37.289 A:middle L:90%
it. Basically our students are now they know when
429
00:23:37.289 --> 00:23:40.309 A:middle L:90%
they come and ask somebody for information about the process
430
00:23:40.319 --> 00:23:41.589 A:middle L:90%
. They want to know what is the sequence in
431
00:23:41.589 --> 00:23:44.529 A:middle L:90%
which things happen. What are the exceptions, what
432
00:23:44.529 --> 00:23:47.640 A:middle L:90%
are the preconditions where the post conditions and just the
433
00:23:47.640 --> 00:23:51.589 A:middle L:90%
asking of these questions actually causes the solicitation to go
434
00:23:51.589 --> 00:23:56.750 A:middle L:90%
better because the questions are suggested. So there is
435
00:23:56.759 --> 00:24:00.559 A:middle L:90%
no best process. There are only processes that are
436
00:24:00.569 --> 00:24:03.670 A:middle L:90%
good at meeting their requirements or meeting the needs of
437
00:24:03.670 --> 00:24:07.819 A:middle L:90%
the domain experts. And when you interview the domain
438
00:24:07.819 --> 00:24:10.640 A:middle L:90%
experts, we've discovered there are two kinds of questions
439
00:24:10.640 --> 00:24:12.029 A:middle L:90%
you ask, what are you doing and why are
440
00:24:12.029 --> 00:24:15.490 A:middle L:90%
you doing that or what is the goal or what
441
00:24:15.500 --> 00:24:17.859 A:middle L:90%
are you trying to achieve by doing that? And
442
00:24:17.859 --> 00:24:21.539 A:middle L:90%
both of these questions are unfamiliar and difficult, but
443
00:24:21.539 --> 00:24:22.619 A:middle L:90%
on the other hand, what we discover is you've
444
00:24:22.619 --> 00:24:25.930 A:middle L:90%
asked, what do you do? They will leave
445
00:24:25.930 --> 00:24:27.740 A:middle L:90%
out things that come out when you say what are
446
00:24:27.740 --> 00:24:32.980 A:middle L:90%
you trying to do and vice versa. So this
447
00:24:32.980 --> 00:24:34.029 A:middle L:90%
has been an exciting thing. We now have some
448
00:24:34.029 --> 00:24:37.970 A:middle L:90%
processes that I think are really quite remarkable. But
449
00:24:37.970 --> 00:24:40.849 A:middle L:90%
the purpose of doing this was to create a baseline
450
00:24:40.859 --> 00:24:44.130 A:middle L:90%
so that we can then take those processes and study
451
00:24:44.130 --> 00:24:47.960 A:middle L:90%
them and infer things about them, hopefully finding defects
452
00:24:47.960 --> 00:24:49.259 A:middle L:90%
that will cause us to be able to improve them
453
00:24:49.839 --> 00:24:52.710 A:middle L:90%
. And at at this point I'd like to turn
454
00:24:52.710 --> 00:24:56.250 A:middle L:90%
it over the lorry because she's the one that's going
455
00:24:56.250 --> 00:25:00.670 A:middle L:90%
to tell you the story about verification. Mhm.
456
00:25:02.240 --> 00:25:14.269 A:middle L:90%
Uh huh. Mhm. Mhm. Oh. Mhm
457
00:25:17.339 --> 00:25:19.700 A:middle L:90%
. Okay. So after spending a lot of time
458
00:25:19.700 --> 00:25:22.960 A:middle L:90%
creating these models, then there's a couple of questions
459
00:25:22.960 --> 00:25:26.539 A:middle L:90%
that you might ask about these models? So one
460
00:25:26.539 --> 00:25:30.319 A:middle L:90%
is is the model consistent with the real process and
461
00:25:30.319 --> 00:25:33.769 A:middle L:90%
in fact there's a lot of work that we do
462
00:25:33.869 --> 00:25:37.150 A:middle L:90%
in terms of having different individuals look at the look
463
00:25:37.150 --> 00:25:41.240 A:middle L:90%
at the process models asking, giving them scenarios and
464
00:25:41.240 --> 00:25:44.269 A:middle L:90%
then getting to a certain point in saying and then
465
00:25:44.269 --> 00:25:47.799 A:middle L:90%
what would happen or doing shadowing if people wear eye
466
00:25:47.799 --> 00:25:52.970 A:middle L:90%
tracking devices and that's a whole part of this validating
467
00:25:52.970 --> 00:25:56.410 A:middle L:90%
the model that I'm not going to talk about because
468
00:25:56.410 --> 00:25:57.799 A:middle L:90%
I'm going to talk about the other question. And
469
00:25:57.799 --> 00:26:02.019 A:middle L:90%
that is let's assume that the model is a pretty
470
00:26:02.019 --> 00:26:07.180 A:middle L:90%
realistic representation of the process is the process, right
471
00:26:07.190 --> 00:26:07.920 A:middle L:90%
? Does it do what it's supposed to do?
472
00:26:07.930 --> 00:26:11.329 A:middle L:90%
And are there different kinds of errors that could be
473
00:26:11.329 --> 00:26:14.859 A:middle L:90%
in that process? And so that's what I'm going
474
00:26:14.859 --> 00:26:17.660 A:middle L:90%
to talk about is what we can do to try
475
00:26:17.670 --> 00:26:22.069 A:middle L:90%
to find errors in the process model. So we
476
00:26:22.069 --> 00:26:25.470 A:middle L:90%
do a number of different things and we talked about
477
00:26:25.470 --> 00:26:29.369 A:middle L:90%
some of them. The model checking looks to see
478
00:26:29.559 --> 00:26:33.579 A:middle L:90%
does this process model satisfy the requirements as it do
479
00:26:33.579 --> 00:26:36.240 A:middle L:90%
what it's supposed to do, then we do some
480
00:26:36.250 --> 00:26:38.910 A:middle L:90%
what we call safety analysis and that is um if
481
00:26:38.910 --> 00:26:44.369 A:middle L:90%
things aren't done right, where is the process vulnerable
482
00:26:44.380 --> 00:26:48.420 A:middle L:90%
for catastrophes? We also can use the models to
483
00:26:48.420 --> 00:26:52.559 A:middle L:90%
drive simulation and we've also done some work on requirements
484
00:26:52.559 --> 00:26:56.230 A:middle L:90%
generation. So if you're going to have devices within
485
00:26:56.230 --> 00:27:00.140 A:middle L:90%
the process coordinating with the people? What are the
486
00:27:00.140 --> 00:27:03.579 A:middle L:90%
requirements that you might have on that device if it's
487
00:27:03.579 --> 00:27:04.269 A:middle L:90%
going to work within the process or what are the
488
00:27:04.269 --> 00:27:07.180 A:middle L:90%
requirements on the process of? It's going to work
489
00:27:07.190 --> 00:27:10.049 A:middle L:90%
within that device. And these are examples of the
490
00:27:10.049 --> 00:27:12.799 A:middle L:90%
analysis now to support this analysis we basically have an
491
00:27:12.799 --> 00:27:18.900 A:middle L:90%
environment of analysis capabilities and I I hope you can
492
00:27:18.900 --> 00:27:21.849 A:middle L:90%
see the arrow here. So here's this process model
493
00:27:21.900 --> 00:27:26.259 A:middle L:90%
that lee was describing and um so we have the
494
00:27:26.269 --> 00:27:29.019 A:middle L:90%
editor to help you create that model but then we
495
00:27:29.019 --> 00:27:32.140 A:middle L:90%
have a range of analysis tools that's what's in the
496
00:27:32.150 --> 00:27:36.400 A:middle L:90%
turquoise down this this column here that analyzed the process
497
00:27:36.400 --> 00:27:38.130 A:middle L:90%
model and usually with the analysis there might be some
498
00:27:38.130 --> 00:27:41.380 A:middle L:90%
secondary information that we have to provide and then we
499
00:27:41.380 --> 00:27:45.200 A:middle L:90%
get this feedback reports and in terms of a process
500
00:27:45.200 --> 00:27:48.279 A:middle L:90%
improvement environment, the view is that we would get
501
00:27:48.289 --> 00:27:52.180 A:middle L:90%
this feedback, we take that back to the medical
502
00:27:52.180 --> 00:27:55.569 A:middle L:90%
professionals and then there would be an iteration where we
503
00:27:55.569 --> 00:27:59.220 A:middle L:90%
would improve the process and repeat this. So I'm
504
00:27:59.220 --> 00:28:00.950 A:middle L:90%
gonna start talking about some of these analysis techniques in
505
00:28:00.950 --> 00:28:07.690 A:middle L:90%
a little more detail and um leary described one and
506
00:28:07.690 --> 00:28:10.829 A:middle L:90%
that is just creating alternative representations. And he showed
507
00:28:10.829 --> 00:28:14.700 A:middle L:90%
you the narrative view and we found that very often
508
00:28:14.710 --> 00:28:18.259 A:middle L:90%
just looking at the two different views helps discover errors
509
00:28:18.259 --> 00:28:26.549 A:middle L:90%
in in the process. Mhm Whoops here it is
510
00:28:26.559 --> 00:28:27.829 A:middle L:90%
. Okay. But what I want to talk about
511
00:28:27.839 --> 00:28:30.880 A:middle L:90%
now is the model checking. So how many people
512
00:28:30.880 --> 00:28:37.309 A:middle L:90%
here are familiar with model checking? Oh, not
513
00:28:37.319 --> 00:28:40.680 A:middle L:90%
too many. So um let me just give you
514
00:28:40.680 --> 00:28:45.039 A:middle L:90%
a high level overview. And basically what model checking
515
00:28:45.039 --> 00:28:51.519 A:middle L:90%
does um is it takes the representation and creates a
516
00:28:51.519 --> 00:28:56.750 A:middle L:90%
relatively concise model that allows you to reason about all
517
00:28:56.750 --> 00:29:00.440 A:middle L:90%
the possible ways that you could actually execute that model
518
00:29:00.450 --> 00:29:04.470 A:middle L:90%
. So it was originally developed for hardware systems and
519
00:29:04.470 --> 00:29:07.390 A:middle L:90%
then moved to software systems. And in fact both
520
00:29:07.390 --> 00:29:11.150 A:middle L:90%
lee and I and Barb at times have worked on
521
00:29:11.160 --> 00:29:15.470 A:middle L:90%
um developing these techniques where you can look at all
522
00:29:15.470 --> 00:29:18.279 A:middle L:90%
the possible, execute double paths through the program and
523
00:29:18.279 --> 00:29:22.460 A:middle L:90%
determine if they meet the requirements. And so there's
524
00:29:22.460 --> 00:29:23.589 A:middle L:90%
a whole lot of work that's done in terms of
525
00:29:23.589 --> 00:29:26.549 A:middle L:90%
how you can recognize equivalence classes and how you can
526
00:29:26.740 --> 00:29:33.039 A:middle L:90%
keep this so it can be optimized and unlike Regular
527
00:29:33.039 --> 00:29:37.779 A:middle L:90%
, the're improving where you can make very strong statements
528
00:29:37.779 --> 00:29:38.150 A:middle L:90%
about what you want to prove, you're a little
529
00:29:38.150 --> 00:29:41.049 A:middle L:90%
more restricted in what you can prove. But if
530
00:29:41.049 --> 00:29:44.559 A:middle L:90%
you can show that it's satisfied on all paths,
531
00:29:44.559 --> 00:29:45.630 A:middle L:90%
you've actually proved it. So it's a very strong
532
00:29:45.630 --> 00:29:51.069 A:middle L:90%
, very nice technique. Um But one of the
533
00:29:51.069 --> 00:29:53.730 A:middle L:90%
things that people sometimes forget to mention is that coming
534
00:29:53.730 --> 00:29:57.259 A:middle L:90%
up with what you're going to actually prove is hard
535
00:29:57.640 --> 00:30:00.900 A:middle L:90%
and people get it wrong because since this is we're
536
00:30:00.900 --> 00:30:03.769 A:middle L:90%
doing this in an automated way, you have to
537
00:30:03.769 --> 00:30:07.150 A:middle L:90%
say it very, very precisely and we're dealing with
538
00:30:07.160 --> 00:30:08.369 A:middle L:90%
medical professionals. So it's important to be able to
539
00:30:08.369 --> 00:30:12.259 A:middle L:90%
capture that precisely. So one of the projects that
540
00:30:12.259 --> 00:30:17.890 A:middle L:90%
we've worked on is represented here, the model checker
541
00:30:17.900 --> 00:30:18.660 A:middle L:90%
. And I'm not gonna go into the details of
542
00:30:18.660 --> 00:30:21.819 A:middle L:90%
the model checker but I am going to talk about
543
00:30:21.819 --> 00:30:23.259 A:middle L:90%
how do we represent what we want to prove.
544
00:30:23.940 --> 00:30:26.960 A:middle L:90%
And this is a system that we developed called propel
545
00:30:27.539 --> 00:30:36.720 A:middle L:90%
and propel um basically tries to um give people hints
546
00:30:36.730 --> 00:30:38.869 A:middle L:90%
. So there was some work that was done that
547
00:30:38.880 --> 00:30:42.380 A:middle L:90%
by Dwyer of Runnin and Corbett on property patterns.
548
00:30:42.390 --> 00:30:47.349 A:middle L:90%
And this was for verifying software systems and they discovered
549
00:30:47.349 --> 00:30:52.289 A:middle L:90%
that a lot of these requirements of which out when
550
00:30:52.289 --> 00:30:56.460 A:middle L:90%
you think about them in terms of smaller statements,
551
00:30:56.470 --> 00:31:00.799 A:middle L:90%
a collection of properties would be would be the requirements
552
00:31:00.799 --> 00:31:03.309 A:middle L:90%
for a system. If you look at these small
553
00:31:03.319 --> 00:31:07.569 A:middle L:90%
concise statements, a lot of them fall into a
554
00:31:07.579 --> 00:31:11.279 A:middle L:90%
restricted number of patterns. But it turns out that
555
00:31:11.289 --> 00:31:12.890 A:middle L:90%
each one of those patterns is very hard to really
556
00:31:12.890 --> 00:31:15.569 A:middle L:90%
get it right. There's a lot of details.
557
00:31:15.569 --> 00:31:18.160 A:middle L:90%
And what propelled does is it goes through and it
558
00:31:18.160 --> 00:31:22.099 A:middle L:90%
tries to help you figure out what are your options
559
00:31:22.109 --> 00:31:23.369 A:middle L:90%
, what has to be determined And it does this
560
00:31:23.369 --> 00:31:26.630 A:middle L:90%
with three different representations. A question tree, a
561
00:31:26.630 --> 00:31:32.750 A:middle L:90%
natural language, pull down menus or with an extended
562
00:31:32.750 --> 00:31:34.140 A:middle L:90%
view of a finite state automata. I'll give you
563
00:31:34.140 --> 00:31:37.619 A:middle L:90%
an example of what this looks like. So here's
564
00:31:37.619 --> 00:31:40.960 A:middle L:90%
an example of a property. The patient's identification must
565
00:31:40.970 --> 00:31:45.319 A:middle L:90%
be verified prior to transfusing each unit of blood product
566
00:31:45.329 --> 00:31:48.759 A:middle L:90%
. Okay so what we're really talking about are two
567
00:31:48.759 --> 00:31:52.420 A:middle L:90%
events. That's verifying the patient ID. And transfusing
568
00:31:52.420 --> 00:31:55.319 A:middle L:90%
blood and how they're related to each other. So
569
00:31:55.319 --> 00:31:59.369 A:middle L:90%
if we look at the question tree view um it
570
00:31:59.369 --> 00:32:01.829 A:middle L:90%
turns out most properties have primary only one or two
571
00:32:01.829 --> 00:32:06.069 A:middle L:90%
primary events. So the first question after you've sort
572
00:32:06.069 --> 00:32:07.650 A:middle L:90%
of filled in what the parameter names are is are
573
00:32:07.650 --> 00:32:10.589 A:middle L:90%
you dealing with one event or two events? And
574
00:32:10.589 --> 00:32:14.160 A:middle L:90%
in this case we have two events. And if
575
00:32:14.160 --> 00:32:15.460 A:middle L:90%
you're dealing with two events then the next question is
576
00:32:15.460 --> 00:32:17.839 A:middle L:90%
well how do they relate to each other? If
577
00:32:17.839 --> 00:32:22.079 A:middle L:90%
you verify patient ID must you transfused blood and you
578
00:32:22.089 --> 00:32:24.640 A:middle L:90%
know you might decide not to do the transfusion for
579
00:32:24.640 --> 00:32:28.849 A:middle L:90%
some reason. So you would pick the second choice
580
00:32:28.849 --> 00:32:31.289 A:middle L:90%
here that just says you can't transfuse blood until you've
581
00:32:31.299 --> 00:32:34.619 A:middle L:90%
done to verify the patient I. D. And
582
00:32:34.619 --> 00:32:36.650 A:middle L:90%
this is the question tree. And it goes on
583
00:32:36.660 --> 00:32:40.180 A:middle L:90%
asking these questions well what's happening at the same time
584
00:32:40.180 --> 00:32:45.009 A:middle L:90%
? And what corresponds to this? Is having a
585
00:32:45.019 --> 00:32:51.140 A:middle L:90%
finite state automata representation? And these questions really correspond
586
00:32:51.140 --> 00:32:52.440 A:middle L:90%
to where we sort of have these dashed lines.
587
00:32:52.450 --> 00:32:57.039 A:middle L:90%
Those are options we have to decide can you repeat
588
00:32:57.039 --> 00:32:59.069 A:middle L:90%
this. Is there a loop? Is there something
589
00:32:59.069 --> 00:33:01.670 A:middle L:90%
that can happen between verify patient ID and transfused blood
590
00:33:02.140 --> 00:33:05.819 A:middle L:90%
? Um and so on and so forth. So
591
00:33:05.819 --> 00:33:09.440 A:middle L:90%
those questions just correspond to filling out this information here
592
00:33:09.450 --> 00:33:14.490 A:middle L:90%
. This is an example of of having different labels
593
00:33:14.500 --> 00:33:15.990 A:middle L:90%
on a transition. So it's not only is there
594
00:33:15.990 --> 00:33:19.309 A:middle L:90%
a transition if there's a transition, what is the
595
00:33:19.309 --> 00:33:22.769 A:middle L:90%
label? So for example, we can ask do
596
00:33:22.769 --> 00:33:25.769 A:middle L:90%
you have to verify patient ID? And the answer
597
00:33:25.769 --> 00:33:29.559 A:middle L:90%
is no. And that means that's an accepting state
598
00:33:30.740 --> 00:33:32.750 A:middle L:90%
. In terms of the finite state automata and the
599
00:33:32.750 --> 00:33:36.230 A:middle L:90%
finite state automata is going to use the the basis
600
00:33:36.230 --> 00:33:39.019 A:middle L:90%
for doing the model checking the verification. So I'm
601
00:33:39.019 --> 00:33:42.630 A:middle L:90%
showing you the question tree. You could do it
602
00:33:42.640 --> 00:33:45.390 A:middle L:90%
in terms of filling out the templates on the finite
603
00:33:45.390 --> 00:33:51.440 A:middle L:90%
state automata. Or you can use these english language
604
00:33:51.450 --> 00:33:55.549 A:middle L:90%
expressions and here you have a a template and then
605
00:33:55.549 --> 00:33:58.579 A:middle L:90%
you can choose which one you want, you can
606
00:33:58.579 --> 00:34:00.960 A:middle L:90%
select it and it fills it out. And the
607
00:34:01.539 --> 00:34:05.400 A:middle L:90%
thing is when you're all done, you have a
608
00:34:05.410 --> 00:34:08.530 A:middle L:90%
finite state of thomas, a representation verification. And
609
00:34:08.530 --> 00:34:13.079 A:middle L:90%
you have an english description. Okay. So we
610
00:34:13.079 --> 00:34:15.829 A:middle L:90%
were talking this morning about english descriptions and if you
611
00:34:15.829 --> 00:34:20.269 A:middle L:90%
look at this, this is really very detailed and
612
00:34:20.340 --> 00:34:24.539 A:middle L:90%
you wouldn't expect that any uh english description that you
613
00:34:24.539 --> 00:34:28.889 A:middle L:90%
picked up would really provide all this information, you
614
00:34:28.889 --> 00:34:30.670 A:middle L:90%
really have to be driven by knowing that you need
615
00:34:30.670 --> 00:34:37.119 A:middle L:90%
these details in order to do the verification. So
616
00:34:37.130 --> 00:34:40.309 A:middle L:90%
, especially when you're working with domain experts that aren't
617
00:34:42.539 --> 00:34:45.340 A:middle L:90%
in computer science, you really need something to help
618
00:34:45.340 --> 00:34:49.300 A:middle L:90%
them think about all these details. So in terms
619
00:34:49.300 --> 00:34:52.400 A:middle L:90%
of our process and modeling and analysis, we talked
620
00:34:52.400 --> 00:34:55.300 A:middle L:90%
about developing the process model, we also have to
621
00:34:55.300 --> 00:35:00.760 A:middle L:90%
develop these requirements to do the verification. And so
622
00:35:00.760 --> 00:35:05.059 A:middle L:90%
like for the blood transfusion we have about 60 some
623
00:35:05.059 --> 00:35:07.619 A:middle L:90%
properties that we then want to verify and see if
624
00:35:07.630 --> 00:35:12.920 A:middle L:90%
they're true for the process model. So using the
625
00:35:12.920 --> 00:35:15.940 A:middle L:90%
system called flavors, which I'm, I'm not going
626
00:35:15.940 --> 00:35:20.250 A:middle L:90%
to describe, we do that verification. So some
627
00:35:20.260 --> 00:35:24.309 A:middle L:90%
observations about doing this in terms of specifying the properties
628
00:35:24.920 --> 00:35:32.360 A:middle L:90%
, sometimes just specifying the properties, we realized that
629
00:35:32.360 --> 00:35:36.150 A:middle L:90%
there was problems in the process model. They would
630
00:35:36.150 --> 00:35:37.719 A:middle L:90%
say, oh this is very, very important and
631
00:35:37.719 --> 00:35:42.070 A:middle L:90%
then we'd realize that the process didn't have that information
632
00:35:42.070 --> 00:35:44.530 A:middle L:90%
in it and very often it didn't go down to
633
00:35:44.530 --> 00:35:46.090 A:middle L:90%
the level of detail that we needed. So doing
634
00:35:46.090 --> 00:35:50.579 A:middle L:90%
the properties, finding out what are these requirements helped
635
00:35:50.579 --> 00:35:53.760 A:middle L:90%
us determine what should be the granularity of the process
636
00:35:53.760 --> 00:35:55.840 A:middle L:90%
model. What should be the scope, what should
637
00:35:55.840 --> 00:36:00.079 A:middle L:90%
be covered because there's always these questions about what do
638
00:36:00.079 --> 00:36:01.150 A:middle L:90%
you want to talk about? So for example,
639
00:36:01.150 --> 00:36:05.610 A:middle L:90%
you're going to fill out a document. Well,
640
00:36:05.610 --> 00:36:07.730 A:middle L:90%
do you really have to say answer question one,
641
00:36:07.730 --> 00:36:09.760 A:middle L:90%
answer question two. Not unless you're going to reason
642
00:36:09.760 --> 00:36:12.199 A:middle L:90%
about it. You don't need to go down to
643
00:36:12.199 --> 00:36:15.070 A:middle L:90%
that level of detail. But if there was something
644
00:36:15.070 --> 00:36:16.440 A:middle L:90%
that was really important, then you would want to
645
00:36:16.440 --> 00:36:20.639 A:middle L:90%
make sure that was in the process model. Um
646
00:36:20.650 --> 00:36:23.889 A:middle L:90%
The other thing is lee talked about how important exceptions
647
00:36:23.889 --> 00:36:28.619 A:middle L:90%
are. Well that means that the properties also have
648
00:36:28.619 --> 00:36:30.889 A:middle L:90%
to deal with that. If something is supposed to
649
00:36:30.900 --> 00:36:35.920 A:middle L:90%
always be true unless the patient dies or the patient
650
00:36:35.929 --> 00:36:37.739 A:middle L:90%
is unconscious or whatever your exceptions are, then you
651
00:36:37.739 --> 00:36:40.800 A:middle L:90%
have to capture that because if you're going to really
652
00:36:40.800 --> 00:36:45.300 A:middle L:90%
do this verification, it has to be an accurate
653
00:36:45.300 --> 00:36:50.190 A:middle L:90%
statement of what you're doing. The other point here
654
00:36:50.190 --> 00:36:52.420 A:middle L:90%
is that for the medical professionals it was pretty hard
655
00:36:52.429 --> 00:36:59.039 A:middle L:90%
for them to understand the difference between um having a
656
00:36:59.039 --> 00:37:01.929 A:middle L:90%
particular scenario and he was talking about negative scenarios,
657
00:37:01.929 --> 00:37:04.480 A:middle L:90%
these war stories. But even, you know,
658
00:37:04.480 --> 00:37:07.349 A:middle L:90%
having a particular scenario that talked about what happened versus
659
00:37:07.349 --> 00:37:12.960 A:middle L:90%
having a process model that described everything that was supposed
660
00:37:12.960 --> 00:37:15.340 A:middle L:90%
to happen. That's like a program where you talk
661
00:37:15.340 --> 00:37:21.170 A:middle L:90%
about all the cases versus a requirement or a property
662
00:37:21.179 --> 00:37:23.489 A:middle L:90%
that's really talking about what's supposed to be true no
663
00:37:23.489 --> 00:37:28.550 A:middle L:90%
matter how you actually implement or represent the process.
664
00:37:29.130 --> 00:37:31.260 A:middle L:90%
And this is something as computer scientists that we deal
665
00:37:31.260 --> 00:37:35.019 A:middle L:90%
with all the time. But for people who are
666
00:37:35.019 --> 00:37:37.880 A:middle L:90%
outside of computer science, these are hard concepts and
667
00:37:37.880 --> 00:37:40.940 A:middle L:90%
I don't know if people have been following the kinds
668
00:37:40.940 --> 00:37:45.710 A:middle L:90%
of what people again this afternoon Jeanette wing have been
669
00:37:45.710 --> 00:37:47.710 A:middle L:90%
talking about computational thinking and how that has so much
670
00:37:47.719 --> 00:37:51.199 A:middle L:90%
influence in other areas. But this is just really
671
00:37:51.199 --> 00:37:53.030 A:middle L:90%
a perfect example of where we're taking things that we
672
00:37:53.030 --> 00:37:55.889 A:middle L:90%
do in computer science that we learn when we learn
673
00:37:55.889 --> 00:38:00.039 A:middle L:90%
programming and software development and how it applies to other
674
00:38:00.039 --> 00:38:05.500 A:middle L:90%
domains. Um I won't go into the to the
675
00:38:05.500 --> 00:38:07.469 A:middle L:90%
model checking except to just say here's this high level
676
00:38:07.469 --> 00:38:12.630 A:middle L:90%
view of what it looks like. Um So we
677
00:38:12.630 --> 00:38:15.469 A:middle L:90%
have now we have the property that we want to
678
00:38:15.480 --> 00:38:20.530 A:middle L:90%
verify that we created using propel we have the process
679
00:38:20.530 --> 00:38:23.329 A:middle L:90%
definition that was from the process editor. Then we
680
00:38:23.329 --> 00:38:27.800 A:middle L:90%
do that. Um the model checking, finance state
681
00:38:27.800 --> 00:38:30.739 A:middle L:90%
verification which really basically says is there any way I
682
00:38:30.739 --> 00:38:37.170 A:middle L:90%
could execute this process that didn't satisfy the property And
683
00:38:37.170 --> 00:38:39.969 A:middle L:90%
if so then you see the counter example and in
684
00:38:39.969 --> 00:38:44.650 A:middle L:90%
fact in doing this in creating the models we found
685
00:38:44.650 --> 00:38:47.309 A:middle L:90%
a lot of errors. Um Now first what happens
686
00:38:47.309 --> 00:38:50.949 A:middle L:90%
is we tend to find errors in the model or
687
00:38:50.949 --> 00:38:52.769 A:middle L:90%
in the properties you know we've made mistakes we didn't
688
00:38:52.769 --> 00:38:57.130 A:middle L:90%
think of you know we didn't think about particular issues
689
00:38:57.139 --> 00:39:00.730 A:middle L:90%
but after we clean that up and we actually found
690
00:39:00.739 --> 00:39:05.139 A:middle L:90%
a really important errors in the in the processes themselves
691
00:39:05.320 --> 00:39:08.309 A:middle L:90%
. For example we found deadlock. Now we never
692
00:39:08.309 --> 00:39:14.599 A:middle L:90%
expected to find a deadlock example in processes with people
693
00:39:14.739 --> 00:39:16.880 A:middle L:90%
where people would really get stuck and when we found
694
00:39:16.880 --> 00:39:19.440 A:middle L:90%
it we went to them and we thought of this
695
00:39:19.449 --> 00:39:22.039 A:middle L:90%
we must have made a mistake somewhere but they when
696
00:39:22.039 --> 00:39:25.099 A:middle L:90%
they showed them the counter example and in fact they
697
00:39:25.099 --> 00:39:30.519 A:middle L:90%
said yeah that in fact does happen. So the
698
00:39:30.519 --> 00:39:34.630 A:middle L:90%
situation was a patient would come in and depending how
699
00:39:34.630 --> 00:39:36.920 A:middle L:90%
they entered the hospital but if they entered in a
700
00:39:36.920 --> 00:39:40.849 A:middle L:90%
particular way there'd be a request for their blood and
701
00:39:40.940 --> 00:39:45.139 A:middle L:90%
the blood bank would be waiting for some additional information
702
00:39:45.519 --> 00:39:50.380 A:middle L:90%
and the nurse would be waiting for the blood and
703
00:39:50.389 --> 00:39:52.030 A:middle L:90%
after a while someone would notice that this poor patient
704
00:39:52.030 --> 00:39:54.150 A:middle L:90%
has been lying there for a long time and someone
705
00:39:54.159 --> 00:39:58.800 A:middle L:90%
pick up the phone and break the deadlock. Now
706
00:39:58.800 --> 00:40:01.199 A:middle L:90%
the processes were complicated enough that even though they knew
707
00:40:01.199 --> 00:40:04.610 A:middle L:90%
this happened, they didn't quite know how to fix
708
00:40:04.610 --> 00:40:07.380 A:middle L:90%
it. And in fact the first fix that they
709
00:40:07.380 --> 00:40:09.079 A:middle L:90%
proposed when we could show them, the counter example
710
00:40:09.090 --> 00:40:15.139 A:middle L:90%
ended up breaking other properties. And that's because when
711
00:40:15.139 --> 00:40:17.750 A:middle L:90%
you're dealing with concurrency and exceptions, things get really
712
00:40:17.750 --> 00:40:21.380 A:middle L:90%
complicated and it's hard for someone to really see all
713
00:40:21.380 --> 00:40:25.510 A:middle L:90%
the implications just like with the program. Um We
714
00:40:25.510 --> 00:40:32.369 A:middle L:90%
had other examples. Another example was mm a missed
715
00:40:32.369 --> 00:40:36.769 A:middle L:90%
event where it was possible for someone to go through
716
00:40:36.769 --> 00:40:39.309 A:middle L:90%
and not really have there and wait updated. And
717
00:40:39.309 --> 00:40:44.099 A:middle L:90%
when you're doing chemotherapy, the drug is really based
718
00:40:44.099 --> 00:40:45.590 A:middle L:90%
on height and weight and so if you don't have
719
00:40:45.590 --> 00:40:49.320 A:middle L:90%
that up to date, that's a really big problem
720
00:40:49.329 --> 00:40:52.050 A:middle L:90%
. And other other examples. Okay, so in
721
00:40:52.050 --> 00:40:55.940 A:middle L:90%
terms of overall observations, um what we did find
722
00:40:55.940 --> 00:40:59.260 A:middle L:90%
a lot of areas just doing the modeling even before
723
00:40:59.260 --> 00:41:02.329 A:middle L:90%
we had to apply the model checking. Um And
724
00:41:02.329 --> 00:41:06.559 A:middle L:90%
as I said initially we found errors were our errors
725
00:41:06.570 --> 00:41:07.980 A:middle L:90%
in the process modeling or in the properties. But
726
00:41:07.980 --> 00:41:10.739 A:middle L:90%
after that we found real errors in the system.
727
00:41:15.510 --> 00:41:21.530 A:middle L:90%
Um since there weren't experts here about model checking other
728
00:41:21.530 --> 00:41:23.849 A:middle L:90%
than barb maybe? Um uh some of the things
729
00:41:23.849 --> 00:41:29.300 A:middle L:90%
that make model checking hard didn't occur. Um In
730
00:41:29.300 --> 00:41:31.519 A:middle L:90%
fact we had few false positives and we did have
731
00:41:31.519 --> 00:41:35.159 A:middle L:90%
to worry about optimizations, but things seem to work
732
00:41:35.159 --> 00:41:38.210 A:middle L:90%
out pretty well. Um But the other thing is
733
00:41:38.210 --> 00:41:43.809 A:middle L:90%
that the process is, so one of my takeaways
734
00:41:43.809 --> 00:41:45.420 A:middle L:90%
from all this is if you really have a process
735
00:41:45.420 --> 00:41:50.960 A:middle L:90%
that's complicated enough that you need a modeling language to
736
00:41:50.960 --> 00:41:52.889 A:middle L:90%
represent it to reason about it, then you really
737
00:41:52.889 --> 00:41:59.409 A:middle L:90%
need this automated analysis because the errors are really very
738
00:41:59.409 --> 00:42:01.599 A:middle L:90%
, very hard to find. And I think there's
739
00:42:01.599 --> 00:42:06.380 A:middle L:90%
a lot of people who are advocating uml um which
740
00:42:06.380 --> 00:42:07.780 A:middle L:90%
is not a very well defined language and doesn't have
741
00:42:07.780 --> 00:42:13.599 A:middle L:90%
very strong analysis capabilities. And I think without that
742
00:42:13.610 --> 00:42:15.309 A:middle L:90%
it's really gonna be very hard to get correct models
743
00:42:16.699 --> 00:42:20.230 A:middle L:90%
. Okay, so let me go back to this
744
00:42:20.230 --> 00:42:22.980 A:middle L:90%
picture. I was talking about the model checking.
745
00:42:22.989 --> 00:42:29.969 A:middle L:90%
Let me talk about these two analysis techniques which are
746
00:42:29.969 --> 00:42:37.630 A:middle L:90%
safety analysis and um lee mentioned this uh what the
747
00:42:37.630 --> 00:42:39.820 A:middle L:90%
safety analysis is versus the model checking. So the
748
00:42:39.820 --> 00:42:45.510 A:middle L:90%
model checking if you say transfused blood or or um
749
00:42:45.099 --> 00:42:47.679 A:middle L:90%
or check their idea. It assumes that if that's
750
00:42:47.679 --> 00:42:50.469 A:middle L:90%
what the task was. That's what you did.
751
00:42:50.639 --> 00:42:52.730 A:middle L:90%
But when you're dealing with humans, sometimes they don't
752
00:42:52.739 --> 00:42:55.250 A:middle L:90%
do it right, when you're dealing with hardware,
753
00:42:55.349 --> 00:42:58.960 A:middle L:90%
sometimes it doesn't do it. Right. So the
754
00:42:58.960 --> 00:43:01.650 A:middle L:90%
safety analysis says what happens if something isn't done?
755
00:43:01.650 --> 00:43:05.039 A:middle L:90%
Quite right. Are you going to detect that?
756
00:43:05.050 --> 00:43:07.889 A:middle L:90%
Is there enough checking that goes on? And is
757
00:43:07.889 --> 00:43:08.949 A:middle L:90%
it done in the right place that you would detect
758
00:43:08.949 --> 00:43:14.300 A:middle L:90%
those problems so that you didn't really have errors reaching
759
00:43:14.300 --> 00:43:16.349 A:middle L:90%
the patients? So one of these the failure modes
760
00:43:16.349 --> 00:43:21.480 A:middle L:90%
and effects analysis basically says if one thing goes wrong
761
00:43:21.480 --> 00:43:22.849 A:middle L:90%
here in the process how would that propagate through the
762
00:43:22.860 --> 00:43:28.659 A:middle L:90%
process? And what they do with failure mode effect
763
00:43:28.659 --> 00:43:34.800 A:middle L:90%
analysis is they basically create take a a failure and
764
00:43:34.800 --> 00:43:37.809 A:middle L:90%
they track it through the system showing what would go
765
00:43:37.820 --> 00:43:39.900 A:middle L:90%
wrong. So if your input to a particular task
766
00:43:39.900 --> 00:43:44.119 A:middle L:90%
was wrong you'd assume that your output would be wrong
767
00:43:44.119 --> 00:43:45.179 A:middle L:90%
and in that outfit gets used and so on and
768
00:43:45.179 --> 00:43:47.500 A:middle L:90%
so forth. And here this is done. I
769
00:43:47.510 --> 00:43:50.260 A:middle L:90%
don't know if you can read it but here this
770
00:43:50.260 --> 00:43:53.659 A:middle L:90%
is showing a particular trace through the system. And
771
00:43:53.659 --> 00:43:57.000 A:middle L:90%
one thing is if you let's say you put in
772
00:43:57.000 --> 00:44:00.150 A:middle L:90%
the wrong bed number for patient then what could happen
773
00:44:00.150 --> 00:44:02.070 A:middle L:90%
and you trace that through and I won't go through
774
00:44:02.070 --> 00:44:06.099 A:middle L:90%
the details but basically you end up with the wrong
775
00:44:06.099 --> 00:44:09.250 A:middle L:90%
patient receiving the blood. So that's an example of
776
00:44:09.250 --> 00:44:12.670 A:middle L:90%
how you could trace that through the system. The
777
00:44:12.670 --> 00:44:14.940 A:middle L:90%
other let's say you put in the wrong blood type
778
00:44:14.949 --> 00:44:16.190 A:middle L:90%
and you might then have the right patient but the
779
00:44:16.190 --> 00:44:20.840 A:middle L:90%
right patient is receiving the wrong blood and that's traced
780
00:44:20.840 --> 00:44:22.750 A:middle L:90%
through that's what this trace through here through the process
781
00:44:22.750 --> 00:44:25.469 A:middle L:90%
shows. So by looking at this you can see
782
00:44:25.469 --> 00:44:31.400 A:middle L:90%
what would be the hazards that could arise. So
783
00:44:31.409 --> 00:44:36.260 A:middle L:90%
um failure modes and effects analysis has been used in
784
00:44:36.269 --> 00:44:38.599 A:middle L:90%
by industrial engineers. And what we can do is
785
00:44:38.599 --> 00:44:43.110 A:middle L:90%
we automatically create these traces to show what could happen
786
00:44:44.489 --> 00:44:47.329 A:middle L:90%
. An alternative is to think about the hazard.
787
00:44:47.340 --> 00:44:51.179 A:middle L:90%
This is the condition that might cause loss of life
788
00:44:51.179 --> 00:44:54.190 A:middle L:90%
or some kind of serious injury and say how could
789
00:44:54.190 --> 00:44:59.750 A:middle L:90%
this happen? And basically again, this is an
790
00:44:59.750 --> 00:45:04.110 A:middle L:90%
industrial engineering approach. Usually you bring together experts about
791
00:45:04.119 --> 00:45:06.389 A:middle L:90%
the process and they talk about the hazard and then
792
00:45:06.389 --> 00:45:08.449 A:middle L:90%
they create what's called a fault tree that says,
793
00:45:08.449 --> 00:45:13.199 A:middle L:90%
well this hazard could only occur if these things happened
794
00:45:13.210 --> 00:45:14.989 A:middle L:90%
and then you look at each one of those things
795
00:45:14.989 --> 00:45:17.480 A:middle L:90%
, how could they occur and you go continue on
796
00:45:17.480 --> 00:45:20.579 A:middle L:90%
down until you get to the level where you feel
797
00:45:20.579 --> 00:45:22.429 A:middle L:90%
. You have enough detail and then you can use
798
00:45:22.429 --> 00:45:28.099 A:middle L:90%
that paltry to do some reasoning about how likely are
799
00:45:28.099 --> 00:45:30.889 A:middle L:90%
collections of things, how likely are certain collections of
800
00:45:30.889 --> 00:45:35.909 A:middle L:90%
things to occur that could cause the hazard. So
801
00:45:36.289 --> 00:45:39.389 A:middle L:90%
what we've done is automate that based on the process
802
00:45:39.389 --> 00:45:44.920 A:middle L:90%
model and use the process model to drive that well
803
00:45:45.289 --> 00:45:47.860 A:middle L:90%
what could go wrong and to create the fault tree
804
00:45:47.869 --> 00:45:52.150 A:middle L:90%
automatically. So to to see an example of this
805
00:45:53.280 --> 00:45:57.840 A:middle L:90%
, I need a little more information in that the
806
00:45:57.849 --> 00:46:00.519 A:middle L:90%
blood transfusion process that you were looking at before and
807
00:46:00.530 --> 00:46:05.969 A:middle L:90%
here just it's just annotated with the flow of artifacts
808
00:46:05.980 --> 00:46:10.360 A:middle L:90%
. And here for example we have the blood unit
809
00:46:10.360 --> 00:46:15.590 A:middle L:90%
that's going in to perform transfusion and that was uh
810
00:46:15.679 --> 00:46:19.460 A:middle L:90%
output from pick up blood from the blood bank that
811
00:46:19.460 --> 00:46:22.309 A:middle L:90%
got the blood type information and there's more information there
812
00:46:22.309 --> 00:46:24.590 A:middle L:90%
. But you can just see a little bit of
813
00:46:24.599 --> 00:46:29.489 A:middle L:90%
that from that. You can sort of can you
814
00:46:29.500 --> 00:46:32.369 A:middle L:90%
read this or not? So it's sort of it
815
00:46:32.369 --> 00:46:40.800 A:middle L:90%
starts off with okay um with the hazard did it
816
00:46:43.480 --> 00:46:45.929 A:middle L:90%
with the help. I'll use this with the hazard
817
00:46:45.940 --> 00:46:49.949 A:middle L:90%
at the top here. Um basically saying the blood
818
00:46:49.949 --> 00:46:53.300 A:middle L:90%
unit performed is wrong. That goes to perform transfusion
819
00:46:53.780 --> 00:46:58.099 A:middle L:90%
. I'll take that. Okay. And and have
820
00:46:59.780 --> 00:47:04.900 A:middle L:90%
it doesn't like me. Uh And how could that
821
00:47:04.900 --> 00:47:07.360 A:middle L:90%
happen? And this is the blood unit from pick
822
00:47:07.360 --> 00:47:08.400 A:middle L:90%
up blood from blood banks one. Well how could
823
00:47:08.400 --> 00:47:12.170 A:middle L:90%
that happen in here here we have an or gate
824
00:47:12.179 --> 00:47:14.780 A:middle L:90%
and that could happen if this happens or if that
825
00:47:14.780 --> 00:47:16.389 A:middle L:90%
happens and so on and so forth. Okay.
826
00:47:16.400 --> 00:47:20.599 A:middle L:90%
And you create this tree. Now this has created
827
00:47:20.599 --> 00:47:22.980 A:middle L:90%
automatically from the process model. Knowing about what the
828
00:47:22.980 --> 00:47:29.329 A:middle L:90%
artifact flow is and The the events here are labeled
829
00:47:29.340 --> 00:47:30.940 A:middle L:90%
E one. E 2. And what we're first
830
00:47:30.940 --> 00:47:37.389 A:middle L:90%
going to do is create just network flow equations perhaps
831
00:47:38.980 --> 00:47:42.360 A:middle L:90%
. Yeah, these network flow equations. And if
832
00:47:42.360 --> 00:47:44.860 A:middle L:90%
you look at this, you know E one,
833
00:47:44.869 --> 00:47:46.130 A:middle L:90%
there was just one thing. So E one equals
834
00:47:46.130 --> 00:47:49.550 A:middle L:90%
E. To you look at E two. Well
835
00:47:49.550 --> 00:47:52.260 A:middle L:90%
that's an ore of E. Three or on E
836
00:47:52.260 --> 00:47:55.719 A:middle L:90%
four. So that's that representation here and so on
837
00:47:55.719 --> 00:47:59.590 A:middle L:90%
and so forth. So you have all these equations
838
00:47:59.599 --> 00:48:00.750 A:middle L:90%
and we want to we want to everyone as our
839
00:48:00.750 --> 00:48:05.119 A:middle L:90%
top events. So E one equals E. To
840
00:48:05.119 --> 00:48:07.710 A:middle L:90%
and we do the substitution and we put it in
841
00:48:07.710 --> 00:48:12.820 A:middle L:90%
disjunctive normal form and that gives us what are called
842
00:48:12.829 --> 00:48:17.949 A:middle L:90%
um um minimal cut sets. And basically if any
843
00:48:17.949 --> 00:48:22.449 A:middle L:90%
one of these are true, then the hazard could
844
00:48:22.449 --> 00:48:25.889 A:middle L:90%
occur. And so this says E for all by
845
00:48:25.889 --> 00:48:32.030 A:middle L:90%
itself, which is the input blood type is correct
846
00:48:32.039 --> 00:48:35.090 A:middle L:90%
. But you picked up the wrong blood unit.
847
00:48:35.469 --> 00:48:37.170 A:middle L:90%
Okay. That all by itself gets the patient the
848
00:48:37.170 --> 00:48:40.559 A:middle L:90%
wrong type of blood. Okay. And this and
849
00:48:40.559 --> 00:48:43.820 A:middle L:90%
this would be a single point of failure. If
850
00:48:43.820 --> 00:48:45.250 A:middle L:90%
this went wrong all by itself, you would have
851
00:48:45.250 --> 00:48:47.929 A:middle L:90%
this hazard. Here's another single point of failure.
852
00:48:47.940 --> 00:48:52.440 A:middle L:90%
Here's a combination. Well, if these you can
853
00:48:52.440 --> 00:48:53.980 A:middle L:90%
have a collection of things but if there were likely
854
00:48:53.980 --> 00:48:59.500 A:middle L:90%
to happen, um the collection was likely to happen
855
00:48:59.500 --> 00:49:01.949 A:middle L:90%
or had a relatively high probability of happening. That
856
00:49:01.949 --> 00:49:06.679 A:middle L:90%
means the hazard could occur. And this kind of
857
00:49:06.679 --> 00:49:12.050 A:middle L:90%
analysis shows you where the process is vulnerable and we
858
00:49:12.050 --> 00:49:15.429 A:middle L:90%
did that kind of process analysis and in fact we
859
00:49:15.429 --> 00:49:20.179 A:middle L:90%
found places where the processes we were modeling were vulnerable
860
00:49:20.179 --> 00:49:22.449 A:middle L:90%
to these kinds of errors and the medical professionals were
861
00:49:22.449 --> 00:49:28.659 A:middle L:90%
quite horrified and then change their processes to reduce the
862
00:49:28.659 --> 00:49:32.150 A:middle L:90%
probability that these errors would occur. Um, so
863
00:49:32.150 --> 00:49:35.949 A:middle L:90%
let me just quickly because I'm running out of time
864
00:49:35.949 --> 00:49:39.050 A:middle L:90%
, say something about this. So these techniques,
865
00:49:39.059 --> 00:49:44.489 A:middle L:90%
the medical professionals were in fact familiar especially with the
866
00:49:44.500 --> 00:49:50.099 A:middle L:90%
failure modes and effects analysis, the analysis being able
867
00:49:50.099 --> 00:49:53.480 A:middle L:90%
to create these trees, these fault trees automatically.
868
00:49:53.510 --> 00:49:57.960 A:middle L:90%
It was really a big savings because in fact industrial
869
00:49:57.960 --> 00:50:00.579 A:middle L:90%
engineers usually do this manually takes a lot of time
870
00:50:00.590 --> 00:50:02.630 A:middle L:90%
. And you really have to have a deep understanding
871
00:50:02.630 --> 00:50:06.519 A:middle L:90%
of the process, You can say well for our
872
00:50:06.519 --> 00:50:07.619 A:middle L:90%
process model and you have to have a deep understanding
873
00:50:07.619 --> 00:50:10.710 A:middle L:90%
of the process and that's true. But for each
874
00:50:10.710 --> 00:50:15.059 A:middle L:90%
process model, we can create a number of the
875
00:50:15.059 --> 00:50:19.389 A:middle L:90%
fall trees for a number of different hazards and we
876
00:50:19.389 --> 00:50:22.380 A:middle L:90%
can change the process easily and then recreate the fault
877
00:50:22.380 --> 00:50:25.210 A:middle L:90%
trees and to give you a sense of how hard
878
00:50:25.210 --> 00:50:30.070 A:middle L:90%
this is. These fall trees are in fact not
879
00:50:30.070 --> 00:50:32.840 A:middle L:90%
small. Okay, here's one for the blood transfusion
880
00:50:32.929 --> 00:50:37.000 A:middle L:90%
and in talking to industrial engineers, they say yes
881
00:50:37.010 --> 00:50:39.699 A:middle L:90%
, they look like this. Can you imagine creating
882
00:50:39.699 --> 00:50:43.280 A:middle L:90%
something like this manually and having any faith in it
883
00:50:44.659 --> 00:50:46.090 A:middle L:90%
. Okay, so, um, you know,
884
00:50:46.099 --> 00:50:50.760 A:middle L:90%
really being able to do this automatically is really very
885
00:50:50.769 --> 00:50:54.579 A:middle L:90%
, very important. Okay. I'm I think we're
886
00:50:54.579 --> 00:50:59.110 A:middle L:90%
not going to talk about simulations just say we do
887
00:50:59.110 --> 00:51:00.789 A:middle L:90%
it. Okay. Just to say that, you
888
00:51:00.789 --> 00:51:02.059 A:middle L:90%
know, one of the things that we're trying to
889
00:51:02.059 --> 00:51:05.789 A:middle L:90%
do is if you spend the time to create that
890
00:51:05.800 --> 00:51:07.400 A:middle L:90%
process model, which we realized, you know,
891
00:51:07.400 --> 00:51:09.250 A:middle L:90%
takes a lot of time and effort. We want
892
00:51:09.250 --> 00:51:13.710 A:middle L:90%
to leverage that. And so I've talked about finding
893
00:51:13.719 --> 00:51:16.630 A:middle L:90%
errors and properties, talked about doing safety analysis.
894
00:51:16.769 --> 00:51:21.389 A:middle L:90%
The other thing we can do is um simulations and
895
00:51:21.389 --> 00:51:23.969 A:middle L:90%
we can use those models to dr simulations so you
896
00:51:23.969 --> 00:51:28.150 A:middle L:90%
can determine what happens when you change the resources,
897
00:51:28.159 --> 00:51:30.989 A:middle L:90%
when you're executing a process model, for example.
898
00:51:30.260 --> 00:51:34.380 A:middle L:90%
So I'm going to skip over all these wonderful slides
899
00:51:34.380 --> 00:51:37.780 A:middle L:90%
about what we learned about simulations. Um There's a
900
00:51:37.789 --> 00:51:40.239 A:middle L:90%
lot of work in simulations and there's a lot of
901
00:51:40.239 --> 00:51:45.829 A:middle L:90%
commercial simulation systems that the medical community is used to
902
00:51:45.829 --> 00:51:47.639 A:middle L:90%
using. The thing is that the models are usually
903
00:51:47.639 --> 00:51:51.329 A:middle L:90%
based on queuing theory. And so they can't do
904
00:51:51.329 --> 00:51:53.739 A:middle L:90%
the fine grained representations that we can do in our
905
00:51:53.739 --> 00:51:57.840 A:middle L:90%
process modeling. And that's why we believe and we're
906
00:51:57.840 --> 00:52:00.949 A:middle L:90%
seeing, we seem to be getting initial results that
907
00:52:00.949 --> 00:52:06.679 A:middle L:90%
show that we get we get better predictive information.
908
00:52:07.750 --> 00:52:12.070 A:middle L:90%
Okay, so, overall observations, we found really
909
00:52:12.070 --> 00:52:16.260 A:middle L:90%
several important errors in the work that in the work
910
00:52:16.260 --> 00:52:21.829 A:middle L:90%
that we did. Um Initially we found errors in
911
00:52:21.829 --> 00:52:23.110 A:middle L:90%
our models and our properties. But correcting these is
912
00:52:23.110 --> 00:52:27.440 A:middle L:90%
really important before you actually use this to tell you
913
00:52:27.440 --> 00:52:30.639 A:middle L:90%
anything about the real processes. We found really important
914
00:52:30.639 --> 00:52:35.820 A:middle L:90%
errors in the processes that we did model and in
915
00:52:35.820 --> 00:52:40.059 A:middle L:90%
the chemotherapy process They saw a 70% reduction in the
916
00:52:40.070 --> 00:52:44.429 A:middle L:90%
errors that reach the patients. So they had been
917
00:52:44.429 --> 00:52:47.590 A:middle L:90%
gathering information on errors were they worked with us and
918
00:52:47.590 --> 00:52:51.949 A:middle L:90%
then the the errors that reach patients went down drastically
919
00:52:51.949 --> 00:52:54.570 A:middle L:90%
after they improve their processes. So that's really very
920
00:52:54.579 --> 00:53:00.250 A:middle L:90%
very strong results. Just let me just quickly say
921
00:53:00.250 --> 00:53:04.050 A:middle L:90%
something about the future. Um So what what we
922
00:53:04.050 --> 00:53:07.440 A:middle L:90%
have been really describing is this view of creating a
923
00:53:07.440 --> 00:53:10.980 A:middle L:90%
process model, doing static analysis and this feedback.
924
00:53:13.050 --> 00:53:17.750 A:middle L:90%
But we're really where we're going is taking those models
925
00:53:17.750 --> 00:53:22.190 A:middle L:90%
now after they've been validated using them to provide guidance
926
00:53:22.190 --> 00:53:28.079 A:middle L:90%
to medical professionals while they're executing those processes and then
927
00:53:28.079 --> 00:53:30.099 A:middle L:90%
we're going to be able to gather information about what
928
00:53:30.099 --> 00:53:36.010 A:middle L:90%
actually happened. Uh We talked about using probabilities and
929
00:53:36.010 --> 00:53:37.380 A:middle L:90%
the fault tree analysis. They don't really know what
930
00:53:37.380 --> 00:53:40.260 A:middle L:90%
those probabilities are, they're just estimating them. But
931
00:53:40.260 --> 00:53:45.050 A:middle L:90%
if we were actually following them while they were actually
932
00:53:45.059 --> 00:53:47.539 A:middle L:90%
executing the processes we start to get that information.
933
00:53:47.550 --> 00:53:52.269 A:middle L:90%
Then we could do some probabilistic analysis and again improve
934
00:53:52.269 --> 00:53:55.469 A:middle L:90%
their process models and then expand the loop. So
935
00:53:55.469 --> 00:53:59.460 A:middle L:90%
we have a static loop as well as a dynamic
936
00:53:59.460 --> 00:54:01.530 A:middle L:90%
loop. And that's actually what we're what we're working
937
00:54:01.530 --> 00:54:05.820 A:middle L:90%
on now. So I think based on the time
938
00:54:05.829 --> 00:54:17.070 A:middle L:90%
I'll end and open it up for questions. Yes
939
00:54:17.190 --> 00:54:21.719 A:middle L:90%
. So one of the examples that you you found
940
00:54:21.730 --> 00:54:24.489 A:middle L:90%
focused on a deadlock scenario but it wasn't quite clear
941
00:54:24.500 --> 00:54:29.219 A:middle L:90%
how your body can currency in the form that you're
942
00:54:29.230 --> 00:54:30.500 A:middle L:90%
using. I saw in the example language you have
943
00:54:30.510 --> 00:54:37.360 A:middle L:90%
this parallel bars. Yes. But you're also using
944
00:54:37.360 --> 00:54:39.739 A:middle L:90%
the finance state machine, which isn't concurrent but you
945
00:54:39.739 --> 00:54:43.079 A:middle L:90%
have an extended version of that so that they have
946
00:54:43.090 --> 00:54:46.409 A:middle L:90%
more Petrie net like Okay, so the in the
947
00:54:46.420 --> 00:54:52.789 A:middle L:90%
process language itself, the equal sign that says you
948
00:54:52.789 --> 00:54:53.719 A:middle L:90%
can do this in parallel is like a fork and
949
00:54:53.719 --> 00:54:58.239 A:middle L:90%
join. But there's also a general message passing.
950
00:54:58.250 --> 00:55:00.630 A:middle L:90%
So you can talk about synchronized and unseen. Quran
951
00:55:00.630 --> 00:55:07.280 A:middle L:90%
is a NSYNC rin ized um control and communication.
952
00:55:07.289 --> 00:55:10.559 A:middle L:90%
Okay. The properties themselves that was the finite set
953
00:55:10.559 --> 00:55:15.480 A:middle L:90%
of tomato and so that's basically the requirements and that's
954
00:55:15.480 --> 00:55:20.250 A:middle L:90%
really talking about the ordering of events and when we
955
00:55:20.250 --> 00:55:24.469 A:middle L:90%
do the analysis. Um, basically we then create
956
00:55:24.469 --> 00:55:29.440 A:middle L:90%
an interleague model of the concurrency to see if the
957
00:55:29.440 --> 00:55:31.320 A:middle L:90%
property is going to be violated. Which is basically
958
00:55:31.320 --> 00:55:34.550 A:middle L:90%
equivalent if you were going to use a Petri net
959
00:55:34.559 --> 00:55:35.989 A:middle L:90%
. I mean you could do this with a Petri
960
00:55:35.989 --> 00:55:38.699 A:middle L:90%
net model and do the concurrency analysis that you would
961
00:55:38.699 --> 00:55:49.599 A:middle L:90%
do over a Petri net explosion. No, I
962
00:55:49.599 --> 00:55:52.500 A:middle L:90%
didn't show you what that model looks like. Black
963
00:55:52.510 --> 00:55:55.170 A:middle L:90%
. Yeah, but again, you know, we
964
00:55:55.170 --> 00:55:59.510 A:middle L:90%
do lots and lots of optimizations. So a property
965
00:55:59.510 --> 00:56:02.329 A:middle L:90%
is only talking about so many events and we take
966
00:56:02.329 --> 00:56:05.550 A:middle L:90%
the model and we get rid of anything that doesn't
967
00:56:05.829 --> 00:56:07.409 A:middle L:90%
impact those events. But yes, the model gets
968
00:56:07.409 --> 00:56:12.829 A:middle L:90%
very large fault tree. One is is large.
969
00:56:12.840 --> 00:56:15.280 A:middle L:90%
Just again because the real model is much larger and
970
00:56:15.280 --> 00:56:20.769 A:middle L:90%
there's so many different things that could happen. I
971
00:56:20.780 --> 00:56:23.570 A:middle L:90%
wondered if manual car sex are unique that they may
972
00:56:23.570 --> 00:56:27.760 A:middle L:90%
not be and they they wouldn't be explanation on the
973
00:56:27.760 --> 00:56:36.670 A:middle L:90%
side of the street common affection so theoretically perhaps.
974
00:56:36.670 --> 00:56:39.420 A:middle L:90%
But these are real processes and the processes we looked
975
00:56:39.420 --> 00:56:44.210 A:middle L:90%
at really basically the cut sets don't tend to be
976
00:56:44.210 --> 00:56:45.480 A:middle L:90%
one or two. So the medical processes that we
977
00:56:45.480 --> 00:56:47.469 A:middle L:90%
looked at tend to be robust, but they may
978
00:56:47.469 --> 00:56:50.630 A:middle L:90%
have four or five and there may be a couple
979
00:56:50.630 --> 00:56:52.369 A:middle L:90%
of there, maybe a dozen or two of them
980
00:56:52.400 --> 00:56:54.170 A:middle L:90%
, but they're not thousands. And that's because these
981
00:56:54.170 --> 00:56:57.909 A:middle L:90%
are real processes. They're not random graphs. And
982
00:56:57.909 --> 00:57:00.650 A:middle L:90%
also you're really concerned that when the terms have a
983
00:57:00.650 --> 00:57:02.550 A:middle L:90%
few events in them because those are the ones that
984
00:57:02.550 --> 00:57:05.250 A:middle L:90%
means that only a couple of things need to go
985
00:57:05.260 --> 00:57:07.480 A:middle L:90%
wrong. If you have a term that has lots
986
00:57:07.480 --> 00:57:08.949 A:middle L:90%
of things in it and they'd all have to be
987
00:57:08.949 --> 00:57:16.750 A:middle L:90%
done wrong, then it's a safer situation. If
988
00:57:16.750 --> 00:57:20.219 A:middle L:90%
I would like to use a little Jill to model
989
00:57:20.219 --> 00:57:23.340 A:middle L:90%
different processes. What's the best way to start?
990
00:57:23.349 --> 00:57:29.590 A:middle L:90%
I'm thinking a particular objective Is uh, for cybersecurity
991
00:57:29.590 --> 00:57:34.130 A:middle L:90%
in 30 minutes. But you know, the system
992
00:57:34.130 --> 00:57:37.090 A:middle L:90%
admins network admins, they could open new accounts for
993
00:57:37.099 --> 00:57:42.360 A:middle L:90%
users purchase new devices, putting in other things they
994
00:57:42.360 --> 00:57:45.480 A:middle L:90%
may affect the security. So little Jill is downloadable
995
00:57:45.489 --> 00:57:47.340 A:middle L:90%
. So you can go to our site and fill
996
00:57:47.340 --> 00:57:50.599 A:middle L:90%
out the form and you can download. There's a
997
00:57:50.610 --> 00:57:52.409 A:middle L:90%
visual editor and there's a tool set and those are
998
00:57:52.409 --> 00:57:55.110 A:middle L:90%
all downloadable and we in fact have, as I
999
00:57:55.110 --> 00:57:58.659 A:middle L:90%
said earlier, we have an election project which is
1000
00:57:58.659 --> 00:58:00.320 A:middle L:90%
looking at security, the security of elections. And
1001
00:58:00.320 --> 00:58:04.989 A:middle L:90%
we actually have a student who's we have a student
1002
00:58:04.989 --> 00:58:07.679 A:middle L:90%
who's working on automatically synthesizing attacks. So it does
1003
00:58:07.679 --> 00:58:12.739 A:middle L:90%
appear that this is possible to take the analysis technologies
1004
00:58:12.739 --> 00:58:16.340 A:middle L:90%
. We have an automatically synthesized attacks on a system
1005
00:58:16.340 --> 00:58:21.530 A:middle L:90%
that's defined in little joke. And um what was
1006
00:58:21.530 --> 00:58:23.110 A:middle L:90%
the other application remember the other application asked about but
1007
00:58:23.110 --> 00:58:25.349 A:middle L:90%
it's been applied to a lot of different app.
1008
00:58:25.400 --> 00:58:29.239 A:middle L:90%
We have working with some people in Australia who are
1009
00:58:29.239 --> 00:58:31.550 A:middle L:90%
operating a cloud computing facility and they're very concerned about
1010
00:58:31.659 --> 00:58:35.329 A:middle L:90%
where eras originate, how they propagate, how to
1011
00:58:35.340 --> 00:58:37.139 A:middle L:90%
find the steps that need to be done better and
1012
00:58:37.139 --> 00:58:38.630 A:middle L:90%
so on. It really seems to be a fairly
1013
00:58:38.630 --> 00:58:43.369 A:middle L:90%
robust toolset that is applicable to a lot of different
1014
00:58:43.369 --> 00:58:45.179 A:middle L:90%
problem domains and a lot of different kinds of issues
1015
00:58:45.179 --> 00:58:49.880 A:middle L:90%
in those domains. So our website will guide you
1016
00:58:49.880 --> 00:58:52.889 A:middle L:90%
through downloading the little Jill editor for start and then
1017
00:58:52.889 --> 00:58:55.889 A:middle L:90%
after that probably we should interact. Right? So
1018
00:58:55.889 --> 00:59:00.170 A:middle L:90%
your work is an excellent example of country take nothing
1019
00:59:00.619 --> 00:59:05.489 A:middle L:90%
. Computer science constant salt office And so if we're
1020
00:59:05.489 --> 00:59:07.130 A:middle L:90%
taking this concept a little bit the privacy of our
1021
00:59:07.139 --> 00:59:13.110 A:middle L:90%
lives and self process. So traditional software engineering,
1022
00:59:13.110 --> 00:59:17.119 A:middle L:90%
it seems that the traditional processes right as exemplified by
1023
00:59:17.119 --> 00:59:21.920 A:middle L:90%
the waterfall model. The problem to be insufficiently flexible
1024
00:59:21.929 --> 00:59:24.650 A:middle L:90%
model collection, a real world software development as a
1025
00:59:24.650 --> 00:59:29.570 A:middle L:90%
result, uh actual physical being embraced. Right?
1026
00:59:29.579 --> 00:59:35.030 A:middle L:90%
So where the process is more intuitive and incremental hasn't
1027
00:59:35.039 --> 00:59:37.460 A:middle L:90%
been more observations that they want. A job process
1028
00:59:37.460 --> 00:59:42.750 A:middle L:90%
model would be more appropriate for the government training song
1029
00:59:43.320 --> 00:59:45.650 A:middle L:90%
. No, we have we have not found that
1030
00:59:45.019 --> 00:59:50.639 A:middle L:90%
basically what we have is a mechanism for determining whether
1031
00:59:50.639 --> 00:59:54.489 A:middle L:90%
processes if they're suitably precisely defined can be analyzed in
1032
00:59:54.489 --> 00:59:59.730 A:middle L:90%
a variety of ways. So there is in this
1033
00:59:59.730 --> 01:00:02.840 A:middle L:90%
scenario you're talking about, there are different development environments
1034
01:00:04.239 --> 01:00:07.099 A:middle L:90%
and different development environments basically levy different requirements on the
1035
01:00:07.110 --> 01:00:10.400 A:middle L:90%
processes in order to determine whether you have a process
1036
01:00:10.400 --> 01:00:13.650 A:middle L:90%
that is suitable for your environment. You might take
1037
01:00:13.650 --> 01:00:15.659 A:middle L:90%
these tools and do this kind of analysis. So
1038
01:00:15.659 --> 01:00:17.550 A:middle L:90%
we actually have some papers where we start to do
1039
01:00:17.550 --> 01:00:22.869 A:middle L:90%
some analysis of hypothetical scrum process for example. Well
1040
01:00:22.869 --> 01:00:23.889 A:middle L:90%
there is no definition of a scrum process. So
1041
01:00:23.889 --> 01:00:28.829 A:middle L:90%
we hypothesized different variations on the scrum process. Try
1042
01:00:28.829 --> 01:00:31.710 A:middle L:90%
to infer the kinds of characteristics those processes had and
1043
01:00:31.710 --> 01:00:34.389 A:middle L:90%
then it's up to you as a sort of a
1044
01:00:34.389 --> 01:00:36.889 A:middle L:90%
development manager. Do you want to process that?
1045
01:00:36.889 --> 01:00:38.949 A:middle L:90%
Has these characteristics or not? We've also taken the
1046
01:00:38.949 --> 01:00:43.440 A:middle L:90%
basic waterfall chart and we've elaborated it so that instead
1047
01:00:43.440 --> 01:00:45.110 A:middle L:90%
of seeing lots of back groups with no semantics on
1048
01:00:45.110 --> 01:00:49.809 A:middle L:90%
them, we now have iteration generally in the form
1049
01:00:49.809 --> 01:00:53.570 A:middle L:90%
of Rikers asian that shows how context gets carried from
1050
01:00:53.570 --> 01:00:55.849 A:middle L:90%
step to step and when you go back to a
1051
01:00:55.849 --> 01:00:59.590 A:middle L:90%
previous step, how that context gets carried with it
1052
01:00:59.599 --> 01:01:01.800 A:middle L:90%
and how that if done right can support the rework
1053
01:01:01.800 --> 01:01:05.110 A:middle L:90%
that causes a water foolish kind of a world to
1054
01:01:05.119 --> 01:01:07.820 A:middle L:90%
work better. And in reasoning about all these different
1055
01:01:07.820 --> 01:01:09.920 A:middle L:90%
kinds of processes, we hope to create enough knowledge
1056
01:01:09.920 --> 01:01:14.719 A:middle L:90%
so that people can intelligently decide which processes they should
1057
01:01:14.730 --> 01:01:16.429 A:middle L:90%
be using as opposed to just simply saying, I
1058
01:01:16.429 --> 01:01:19.900 A:middle L:90%
hear scrum is good and then making up what that
1059
01:01:19.900 --> 01:01:27.000 A:middle L:90%
means to them. So in the context of other
1060
01:01:27.000 --> 01:01:30.309 A:middle L:90%
checking, Tyler or software, typically it's very easy
1061
01:01:30.320 --> 01:01:32.300 A:middle L:90%
, it's not very easy to build a model from
1062
01:01:32.309 --> 01:01:35.670 A:middle L:90%
from the program. For example, you have to
1063
01:01:35.679 --> 01:01:37.619 A:middle L:90%
know all the programs have in order to do a
1064
01:01:37.630 --> 01:01:42.570 A:middle L:90%
very accurate. You have you we all start learning
1065
01:01:42.570 --> 01:01:45.039 A:middle L:90%
and all that in this context. We're designing a
1066
01:01:45.050 --> 01:01:50.019 A:middle L:90%
question here uh to the medical professionals. And so
1067
01:01:50.510 --> 01:01:52.420 A:middle L:90%
so my question is when do you, when do
1068
01:01:52.420 --> 01:01:53.409 A:middle L:90%
you stop and say, you know, the model
1069
01:01:53.409 --> 01:02:00.269 A:middle L:90%
is never Yes. I mean, you know,
1070
01:02:00.269 --> 01:02:01.690 A:middle L:90%
it's like a program. In fact it's probably never
1071
01:02:01.690 --> 01:02:06.789 A:middle L:90%
totally accurate and you probably forgot some things. I
1072
01:02:06.789 --> 01:02:08.010 A:middle L:90%
mean in terms if you don't want to reason about
1073
01:02:08.010 --> 01:02:12.679 A:middle L:90%
it, we wouldn't know that. Um so yeah
1074
01:02:12.690 --> 01:02:15.050 A:middle L:90%
, it's an ongoing. So we see this.
1075
01:02:15.050 --> 01:02:17.500 A:middle L:90%
These are living documents, not something you create and
1076
01:02:17.500 --> 01:02:21.989 A:middle L:90%
throw away. And I think that's moving to online
1077
01:02:21.989 --> 01:02:23.789 A:middle L:90%
guidance keeps them alive. I mean as you're going
1078
01:02:23.789 --> 01:02:27.760 A:middle L:90%
to continue to use them. So I think it's
1079
01:02:27.769 --> 01:02:30.650 A:middle L:90%
it's something that you will continue to evolve the process
1080
01:02:30.650 --> 01:02:34.340 A:middle L:90%
and medical processes always change. There's new, they
1081
01:02:34.340 --> 01:02:36.510 A:middle L:90%
get new findings and you have to put that into
1082
01:02:36.510 --> 01:02:37.730 A:middle L:90%
the processes so you want to update them. So
1083
01:02:37.730 --> 01:02:42.719 A:middle L:90%
it's an ongoing living effort and the properties continue to
1084
01:02:42.719 --> 01:02:45.059 A:middle L:90%
live to because usually the properties don't change too much
1085
01:02:45.059 --> 01:02:50.150 A:middle L:90%
when the process models change, we think of processes
1086
01:02:50.150 --> 01:02:52.789 A:middle L:90%
as software. So there was a paper that get
1087
01:02:52.789 --> 01:02:54.000 A:middle L:90%
written like that. They have requirements, they have
1088
01:02:54.000 --> 01:02:58.380 A:middle L:90%
architectures, they have designs, they have code which
1089
01:02:58.380 --> 01:03:00.940 A:middle L:90%
you've just seen and they evolve. So that suggests
1090
01:03:00.940 --> 01:03:04.190 A:middle L:90%
that the things that software engineers have learned about how
1091
01:03:04.190 --> 01:03:06.659 A:middle L:90%
to do that should apply to processes as well.
1092
01:03:06.679 --> 01:03:08.880 A:middle L:90%
But the problems also get carried over to this domain
1093
01:03:08.880 --> 01:03:13.150 A:middle L:90%
as well. And what you've just asked about is
1094
01:03:13.159 --> 01:03:15.349 A:middle L:90%
among the problems that software engineers have grappled with for
1095
01:03:15.349 --> 01:03:17.989 A:middle L:90%
a long time. So applying them to the process
1096
01:03:17.989 --> 01:03:22.420 A:middle L:90%
domain seems to have created some benefits. And then
1097
01:03:22.420 --> 01:03:24.739 A:middle L:90%
one thing I wanted to be sure to mention is
1098
01:03:24.739 --> 01:03:27.929 A:middle L:90%
that we really regard this as a two way street
1099
01:03:28.300 --> 01:03:30.190 A:middle L:90%
. So, as software engineers were always ask,
1100
01:03:30.190 --> 01:03:30.650 A:middle L:90%
you, can you can you write a tool for
1101
01:03:30.650 --> 01:03:32.590 A:middle L:90%
this, can you do that? And yeah,
1102
01:03:32.599 --> 01:03:35.590 A:middle L:90%
software engineers can always take what it is. We
1103
01:03:35.590 --> 01:03:37.610 A:middle L:90%
know, go to other domains and help people with
1104
01:03:37.610 --> 01:03:39.809 A:middle L:90%
their problems by writing code. We are doing that
1105
01:03:39.820 --> 01:03:43.579 A:middle L:90%
to some extent. We're writing an analyzing process code
1106
01:03:43.590 --> 01:03:46.030 A:middle L:90%
. But in doing so, we're discovering things about
1107
01:03:46.030 --> 01:03:50.429 A:middle L:90%
software and software engineering, which software engineers don't know
1108
01:03:51.000 --> 01:03:53.260 A:middle L:90%
and which enriches the study of software engineering. So
1109
01:03:53.260 --> 01:03:54.860 A:middle L:90%
one of the things we didn't spend much time on
1110
01:03:54.860 --> 01:03:59.949 A:middle L:90%
here is that a process definition is very rich and
1111
01:03:59.949 --> 01:04:03.420 A:middle L:90%
specification about resources. Most processes are very heavily focused
1112
01:04:03.420 --> 01:04:09.599 A:middle L:90%
on resource specification utilization. And we have technologies to
1113
01:04:09.599 --> 01:04:11.610 A:middle L:90%
quite a good job of that. And then we
1114
01:04:11.610 --> 01:04:13.340 A:middle L:90%
go back to software engineering and we say, how
1115
01:04:13.340 --> 01:04:15.190 A:middle L:90%
come there isn't a focus on resources and software engineering
1116
01:04:15.250 --> 01:04:19.210 A:middle L:90%
and you discover that there is a certain impoverishment of
1117
01:04:19.210 --> 01:04:23.309 A:middle L:90%
certain kinds of things that software engineers try to do
1118
01:04:23.320 --> 01:04:25.190 A:middle L:90%
, which could be fixed if there was more of
1119
01:04:25.190 --> 01:04:27.559 A:middle L:90%
a focus on resources. So this really is a
1120
01:04:27.559 --> 01:04:30.019 A:middle L:90%
two way street where we're learning about software engineering by
1121
01:04:30.019 --> 01:04:33.860 A:middle L:90%
applying it to these application domains while also enriching those
1122
01:04:33.860 --> 01:04:41.300 A:middle L:90%
donates. Maybe have time for one more question in
1123
01:04:41.300 --> 01:04:45.480 A:middle L:90%
here. Um so uh your inner nature of the
1124
01:04:45.480 --> 01:04:47.050 A:middle L:90%
mission that want to work on uh huh. I
1125
01:04:47.050 --> 01:04:50.210 A:middle L:90%
want to know this place needs to add up if
1126
01:04:50.210 --> 01:04:55.349 A:middle L:90%
you have any thoughts. Well, so again,
1127
01:04:55.349 --> 01:04:58.789 A:middle L:90%
we are actually spending time working on scrum. Yeah
1128
01:04:59.289 --> 01:05:01.289 A:middle L:90%
, so that's 11 of one of the agile methods
1129
01:05:01.429 --> 01:05:03.380 A:middle L:90%
, there are others that we could look into.
1130
01:05:03.380 --> 01:05:05.949 A:middle L:90%
But we're taking scrum as an example. So for
1131
01:05:05.949 --> 01:05:09.340 A:middle L:90%
example, you know the scrum has this burned down
1132
01:05:09.340 --> 01:05:11.349 A:middle L:90%
list and people come into the morning, take things
1133
01:05:11.349 --> 01:05:13.969 A:middle L:90%
off the burned down list. If you had a
1134
01:05:13.980 --> 01:05:15.630 A:middle L:90%
discipline on how people did that, would it actually
1135
01:05:15.630 --> 01:05:18.550 A:middle L:90%
cause that to work better? So we have created
1136
01:05:18.550 --> 01:05:23.000 A:middle L:90%
the scrum process definition, we've loaded it up with
1137
01:05:23.010 --> 01:05:27.380 A:middle L:90%
resource specifications, specifications of job mixes, and we're
1138
01:05:27.380 --> 01:05:30.110 A:middle L:90%
running discrete events simulations and we're discovering, you know
1139
01:05:30.250 --> 01:05:32.610 A:middle L:90%
, surprisingly to me that it seems to make a
1140
01:05:32.610 --> 01:05:36.869 A:middle L:90%
huge difference how these tasks actually get assigned. And
1141
01:05:36.869 --> 01:05:40.599 A:middle L:90%
there are some scrum environments where people worry about this
1142
01:05:40.610 --> 01:05:43.030 A:middle L:90%
somewhere, they don't, it seems to matter.
1143
01:05:43.039 --> 01:05:45.619 A:middle L:90%
So this is something which we believe can help people
1144
01:05:45.619 --> 01:05:47.619 A:middle L:90%
to learn about software processes. And in fact,
1145
01:05:47.619 --> 01:05:50.869 A:middle L:90%
that was the original application domain. And the main
1146
01:05:50.869 --> 01:05:54.619 A:middle L:90%
reason we're working with medical people and election officials is
1147
01:05:54.619 --> 01:05:57.480 A:middle L:90%
because software engineers really didn't seem to like this idea
1148
01:05:57.480 --> 01:06:00.420 A:middle L:90%
at first and but they're now warming up to it
1149
01:06:01.090 --> 01:06:04.099 A:middle L:90%
. So I just want to remind everybody that was
1150
01:06:04.789 --> 01:06:09.809 A:middle L:90%
recession battered and fried in 106 from 4 to 5
1151
01:06:09.820 --> 01:06:13.480 A:middle L:90%
. And I know there's some graduate students meeting with
1152
01:06:13.489 --> 01:06:15.510 A:middle L:90%
dr Esther Rolle and Dr Clarke this afternoon at K
1153
01:06:15.510 --> 01:06:17.599 A:middle L:90%
. W. Two. So let's thank our son
1154
01:06:18.190 --> A:middle L:90%
. Yeah.