Hello, my name is Nizar Dijani, and in these lessons, you will learn about Pandas, SQLite, BeautifulSoup, and Scrapey. And in this lesson, you will learn about using the Pandas library to source data. Python Pandas is a powerful library for data manipulation and data analysis. It is commonly used for data sourcing, data cleaning, and data transformation. To use pandas for data sourcing, we first have to install it. And for that, I'll open a terminal window, then I'll type pip install pandas. Then I'll hit enter, and the results indicate that I already have it installed. And so now I can go to my code examples and work with the pandas library. In my first code example, I first have to import the pandas library, and I also alias it using the as keyword, followed by the letters pd. Then I will call my first data sourcing method in pandas, and that is to read my data from a CSV file, which is a text file with comma separated values. Now the method is called read underscore CSV. And I believe this is descriptive enough to mean that we are reading from a CSV file. And as an argument to this method, I just pass it the name of the file that I want to source the data from. And in this particular case, the file is called file dot CSV. So let me show you this file first. As you can see, it's a very small sample file with a header of first name, last name, and H. Then I have four lines of data, and all are separated by commas. Now I'll return to my code page again. And so what this line of code does is that it reads the data from this text file, and then creates a data frame with that files data. And to see my newly created data frame, which I call df, I just printed out and see my data. So I'll run this code now. And there you go. I get the same data displayed here as a data frame. And now I just use the pandas library in Python to source data from a text file. Now, the goal of sourcing data into a pandas data frame is that we first can display it here and use the data, but also we can manipulate the data as we wish. Then we can dump it back out again into its new format. For example, the data frame that was printed out here includes an index column on the far left. I can manipulate the data frame to hide that column by adding the attribute index underscore COL and set it equal to zero. And when I run this code again, I now just get my raw data with no index column. Again, this is just one small example of manipulating the source data to something else. By the way, since I'm using a Jupyter Notebook, I don't need to use the print function to display my output. For example, instead of typing print df, I can simply remove the print function altogether, and just keep the df part of the command. Then when I run this code cell, I get what looks like a cleaner display or a nicer rendering of the data. Now, that's a Jupyter Notebook feature. But since we can be using non Jupyter type tools to write our code, I prefer to use the full print code. All right, now just like I sourced data from my CSV file, I can also use other data sources like an Excel file or even JSON. My next code cell uses the read underscore Excel method, and I pass it an argument of the Excel file's name and path. By the way, I'm not specifying a path in my file argument, because the data file is in the same directory as my Python file, which in this case is a Jupyter Notebook file. But if my data file is elsewhere, then I can provide the correct path leading to the data files location in this argument. And if I run this code, I get my expected data frame with the same data. Nowadays, data can also come in a JSON format, which stands for JavaScript Object Notation. This is very similar to the dictionary style data format, where we have key value pairs. Actually, let me pull up the JSON file example that I will use next. So as you can see, I have squiggly braces, and each contain a group of key value pairs. I have the first name key with the value of mark. My second key is last name with a value of Smith. And my last key is age with a value of 34. Then I have three more records. And they're all separated by comma. So that's the basic JSON format. Now I'll go back to my code. And in my next code example, just like I had the pandas methods, read underscore CSV, and read underscore Excel, we also have read underscore JSON. And I still pass it my data files name that I want to source data from. Then I assign it to a variable called df, which is short for data frame, because that's what pandas doing. Then it takes the data from the source and creates a data frame for us to use. Then I print out my data frame to see what my output is. So let's run this code cell. And again, I get the same data as before. So up until now, I was able to source data from a CSV file, an Excel file, and a JSON file. Now I can go the other way as well. Just like I can read data from a source, I can also send data to a destination. Again, the overall goal is to source the data, clean it up, then dump it to a destination in its cleaned up format. This is a very common practice in data analysis. And my next three code cells do the last part of this journey, which is dumping the cleaned up data to some destination. The next code cell uses the two underscore CSV method, which means I will send data in my pandas data frame to a CSV file. Then I also have another example with a method to underscore Excel. And the last example is using the method to underscore JSON. So again, I can read data using the read methods, and I can dump data using the two methods. For example, my first of these three examples is the two underscore CSV method. But please do note that this is a data frame method, not a pandas method. Up here, the three different read methods were part of the pandas class. See how I have PD dot read underscore CSV, and PD dot read underscore Excel, and PD dot read underscore JSON. These read methods are the ones that create my data frame. And once I have my data frame, then I call the two methods, which are part of the data frame object, as you can see on the last three examples, we have df dot two underscore CSV. So it's not PD dot two underscore CSV. All right, so the two underscore CSV method also takes the files name as an argument. But this will be the file that will be created with the data from the data frame. Again, presumably after the data has been cleaned up, using the data frame options. So let's run this code cell. And there is no output because it just creates a file for us. And let me see if the file has been created successfully, and what this file contains. And there we go, the file has been indeed created for us. And when I open it, I see the correct data as expected. Now, what happens if I don't want the index column, which appears on the far left, which is a default data frame behavior? Well, I can go back to my code. And I will add a new attribute called index, and set it equal to false with a capital F. Then I'll run this code again. And I'll go to the tab that had my file opened. And when I refresh it, I no longer see my index call. How about if I don't want to include my header row? Well, that's also customizable. So I'll go back to my code. I'll add another attribute called header. And I'll set it equal to false. Again, with the capital F, then I'll run this code. Then I'll switch to my file tab. And I'll hit refresh. And now the header row has been removed. And if you don't want the file to use commas, which is the default separator here, I can change that as well. So I'll go back to my code. And I'll add the sep attribute and set it equal to a semicolon. Then when I run this code and go back to my file tab, then refresh it. I now see that the separator character has been changed to a semicolon. Now, please remember that these are just the basic steps to get started with data sourcing using pandas. Depending on the complexity of the data and your analysis, you may need to apply more advanced techniques. So in this lesson, you learn to source data using the pandas library. And in the next lesson, we will work with another library called SQLite. Thanks for watching.
