Pandas Library: Data Manipulation in Python

• Pandas is a Python library used for data manipulation and analysis

Adding Pandas to Spyder

• Use import pandas as pd

Reading Excel Files

• Python supports .xls and .xlsx formats via Pandas
• Use import pandas as pd
• To read .xls files: xls = pd.read_excel('file_name.xls')
• To read .xlsx files: xlsx = pd.read_excel('file_name.xlsx')

Data Type of a Single Column

• To return the data type of a single column: df1['column_name'].dtypes

Iris Data Sample

• Iris_data_sample is not a default built-in dataset in Pandas

Pandas Installation Directory

• Use import pandas followed by pandas.__path__ to find the directory of the pandas library

Unique Data Types in a DataFrame

• Use dataframe.get_dtype_counts() to get the number of unique data types
• The function returns a Pandas series object that contains the counts of each data type in the Pandas object

Reducing Memory Usage

• Converting data types from object to category reduces memory usage

Handling .txt Files with Special Characters

• To read .txt files, use these codes:
• data_txt1=pd.read_table('Iris_data.txt',delimiter=" ")
• data_txt2=pd.read_csv('Iris_data.txt',delimiter=" ")

Finding Duplicate Values in a DataFrame

• To find duplicate values, use pandas.DataFrame.duplicated() function

Built-in Python Functions

• Python has many useful built-in functions, refer to the documentation for each library

Importing .csv Files

• Use pandas.read_csv("path") to import .csv files

Statistical Summary

• describe() function from the Pandas library returns the statistical five-number summary

Pandas DataFrame Drop Functions

• Key difference between pandas.DataFrame.drop() and pandas.DataFrame.dropna()
• pandas.DataFrame.drop() removes rows or columns by specifying label names/index/column names
• pandas.DataFrame.dropna() drops rows with missing values

Dealing with Special Characters

• Converting special/junk characters to NaN simplifies data cleaning because Pandas offers functions to handle NaN values
• pandas.DataFrame.dropna() drops rows with NaN values
• pandas.DataFrame.fillna() fills NaN values

Importing 'os' Library

• Importing the os library is not required for using the cd command

Categorical Data

• Refer to the documentation to understand categorical data

Spearman's Correlation Coefficient

• Refer to the documentation for information on Spearman correlation in Python

Entire Column as Object with One 'NaN' Value

• If there's only one NaN value in an entire column, the column becomes an object if its data type was previously category

DataFrame Select Data Types Function

• dataframe.select_dtypes(include=None, exclude=None) - Include and exclude arguments specify which data types to select

Lambda Function with Apply Function

• Lambda returns a value based on specified conditions and apply functions applies it across rows/columns

Accessing Values in a Specific Index Range

• If df is the name of the DataFrame:
• df.at() accesses single values
• df.loc() or df.iloc() accesses a range of values

DataFrame Dimensions

• DataFrame consists of only 2 dimensions: rows and columns
• Each column stores data for a specific dimension

Selecting Multiple Columns

• There are methods to select multiple columns from a DataFrame, refer to the documentation for guidance