Program design: flight delays

0:00- [Instructor] Let's design a program with lists, the accumulator pattern and string methods. We've collected flight data from several different airports. We wanna do some analysis here to understand if there's any patterns in a particular airport's delays, and maybe to see which airline is the biggest offender.

0:17Before we try processing it, let's understand our data format. Here we have a table of past flights. For each flight, we have the date, the airline, the flight number and a bunch of delay fields. The delays are all represented in minutes. For now, we're mostly interested in departure delays.

0:33That is, how long after the scheduled time the flight took off. When we download this data, the computer puts it in CSV format, where the columns of each row are separated by commas. I did some quick work to get that data into my Python program, and now I have a list of CSV strings.

0:50Each element represents a row of the table. To start, let's create a new module to hold all of my analysis functions. I think I wanna calculate the average delay time per airline, so I probably want a function that takes in the name of an airline, our flight data, and returns out the average delay for that airline in minutes.

1:13To make this easy, I can just accumulate a list of all that airline's delays. The average then is just the sum of that list divided by the length of that list. Remember that each element of flights is a CSV string, so I need to parse that string to get the airline and delay info.

1:31To get a list of all the fields, I can split my string based on the comma separator. And if I check my data format real quick, I can see that the airline is in the second column, so that'll be Index 1. If the airline doesn't match, we skip it. Otherwise, we need to know the flight delay.

1:51The departure delay field is gonna be at Index 0, 1 2, 3, 4. So I extract that field and append it to my list. Let's go back to the main file and test it. We import both our modules, and then pass the flight data into our get_average function. For now, we'll just pick a test airline.

2:10Let's say Delta. Whoop, that's a type error. The stack trace says it's coming from the return statement in my get_average function. Weird, let's print the state of my list right before that line. Okay, these are strings, not integers. And also weird that there are some negative numbers in here, but one thing at a time, the type error's definitely coming from the string thing.

2:34So if we want all these values to be integers, we should cast them before we append them to the list. Great, now, where are those negative numbers coming from? It looks like our data does have some negative numbers in the departure delay column. My guess is that maybe this means that the flight left early.

2:53But if I'm talking about delays, I don't care if the flight left early. I don't want negative numbers to be skewing my delay average, so I think I wanna treat those as zeros. If the flight left early, there was no delay. All right, we have this working for one airline, but I wanna compare the delays for all airlines.

3:11How do I know which airlines to check? Well, I'm gonna run this analysis on data from a bunch of different airports. And the airlines that fly out of each airport are different. So the best source of truth is the data itself. Let's write a function that iterates over the flight data and collects a list of unique airline names.

3:32So we start with an empty list, we accumulate inside the loop, and we return the final list at the end. Okay, and on each iteration, we extract the airline name, which we know is at Index 1, and then we append it to the list. But this is just a list of each row's airline, we wanna list of the unique airlines that fly out of a given airport.

3:58So if an airline already exists in the list, we don't wanna append it again. Better, but not best. It looks like maybe our data has some different capitalizations of airline names, so they're getting counted as separate airlines. Now, we could just convert all the airlines to the same case inside the find_airlines function, but the get_average function is also using the airline name, so it must be affected by the same issue.

4:27Instead of forcing every analysis function to recognize and handle this data inconsistency, we can use a step called pre-processing. This normalizes our data once upfront, so the rest of our code doesn't have to think about it. We'll create a new module for this, since this is a different class of functionality.

4:47We iterate through each flight, normalize it, and append it back. To normalize, I'm gonna convert every airline name to all uppercase. Now, one other thing, we also had all those negative delays that we converted to zero inside the get_average function. Now, I think for all calculations I'm gonna do, I don't care about early departures, so I'm gonna move that zero conversion out to the pre-processing step too.

5:15That way, no one else has to worry about it. We call this pattern separation of concerns because each piece of code can focus on a single task. Our analysis shows that JetBlue has the worst average delay at this airport. To continue on, we can analyze the delays at other airports or maybe add a function to learn more about delay causes.