Program design: emergency response

0:00- [Instructor] Let's design a program that uses nested data. Here we have a data set of emergency response incidents in New York City. We stored our data set in a list of dictionaries where each dictionary record represents a single incident. The city wants to analyze this data set to better understand any patterns in the timings, types, or locations of incidents, and then apply those insights to make sure enough of the right kind of first responders are available.

0:28Let's start by grouping the incidents by type. This will tell us which incidents are the most common, and that should tell us which first responders are most in demand, the fire department, law enforcement, or EMTs. Now we could write a function that takes in the dataset and a type, which counts how many incidents of that type we have.

0:49But there are so many different types of incidents. For one, how do I know what all the types are, so that I know what to pass in? And two, if I call this function once for 30 different types, then I'll be iterating over my data set 30 separate times. And that sounds pretty inefficient, especially if my data set gets larger.

1:12Instead, this feels like a good use case for a data transformation. That is I iterate over my dataset just once and I convert it to a more useful data structure. In this case, I might want a dictionary that maps the type of incident to account of the number of incidents.

1:31So let's get our accumulator pattern going. We start by initializing an empty dictionary, and then we wanna iterate over every incident. Since our dataset is a list of dictionaries, our loop variable will hold the next incident record on each iteration. And then inside the loop body, we just wanna increment the count for the current incidents type.

1:57And then finally, we return our new dictionary at the end. Alright, let's jump over to main.py and test that out. We'll import our incidents module and our OEM module, which contains our dataset, and then we'll call our count by type function. Mm. And that gives us a key error, why?

2:21Well, back in our loop body, we increment the count for the current type, but what if that type doesn't exist in the dictionary yet. So we need to first check if that type exists and if it doesn't, we need to add a new item instead of incrementing it. We can do this with the in operator, but we can also use the get method here, on the right hand side of the assignment, we access the current count for that type, defaulting it to zero, and then we just add one to increment it.

2:56Now let's try this again, no more key error. Great, but there are actually quite a lot of types in here and it looks like all the incident types use a common prefixing scheme. There's the main type like fire a dash, and then a more descriptive subtype. I'm just trying to understand the main category of incident, like if it's a fire, then the fire department is needed.

3:22So I don't really care about this subtype. It would be more helpful to know how many total fires there were. So let's normalize those types of it. We'll use the split method to split on that dash, and then we'll take the first element of that list, which should be the broad category.

3:40Much better, and to clean up that output a bit, maybe you'll iterate over the dictionary and print out a nicely formatted sentence. Onto our next analysis. We wanna know when the busiest hour of the day is, where busiest just means the most number of incidents, but how do we calculate that result from our dataset?

4:03To get that final result it might be easiest to break down this problem into two steps. Where each step follows a very familiar pattern that we already know how to solve. First, we can do a data transformation that maps hour of the day to number of incidents, and then we can use that data structure to get the busiest hour.

4:27We just iterate over the dictionary and find the key or hour with the highest value. Let's start with the data transformation. If this sounds familiar, it's because we already solved this problem before, except this time instead of type, our keys are hour of the day, so let's copy the structure of the accumulator pattern we used before.

4:51But how do we get the hour of the day? Well, each incident record has a time field, and if we just take the first two characters of that string, that should tell us the hour. So we can just use a slice here and then cast that string to an integer and we'll quickly print out that data structure to spot check that we did it correctly.

5:14Great, now we just need to write the function that takes in that dictionary of hour to incident count. This effectively boils down to the problem, find the key with the maximum value. So we'll wanna iterate over the dictionary and keep track of the busiest hour we've seen so far so that by the end of the loop we'll have the absolute maximum.

5:38Since we need both the hour and the count, we'll iterate by item. And then on each iteration we'll check if the current hours count is greater than the maximum. If it is, that hour becomes our new maximum and we update our variables accordingly. Putting all of our analysis together then we see that the busiest hour of the day is 11:00 am and by far the most common incident type is a fire, that tells us that we need the most staffing in the fire department and in the middle of the day.

6:14To really confirm this, we can next do a breakdown by both type and hour, or cross reference by location so we know which borough needs to happening the most.