Pandas Operations Reference

pandas is an open-source Python library for data manipulation and analysis. It provides two primary data structures: DataFrame (2D labeled table, like a spreadsheet or SQL table) and Series (1D labeled array). pandas makes it easy to load, clean, transform, aggregate, and analyze structured data. It's the foundation of Python data science and works seamlessly with NumPy, Matplotlib, and scikit-learn.

loc selects data by label (row index names and column names): df.loc['row_name', 'col_name']. iloc selects by integer position (zero-based): df.iloc[0, 1] selects row 0, column 1. When your index is integers, loc and iloc may look similar but behave differently — loc includes the end of a slice, iloc does not. Use loc for label-based selection and iloc for positional selection.

pandas uses NaN (Not a Number) to represent missing values. Key operations: df.isna() or df.isnull() detects NaN values; df.dropna() removes rows (or columns) with NaN; df.fillna(value) replaces NaN with a specified value; df.fillna(method='ffill') forward-fills from the previous valid value. For numeric data, df.fillna(df.mean()) fills with column mean.

groupby() splits data into groups based on one or more column values, applies a function to each group, and combines the results. It follows the split-apply-combine pattern. Example: df.groupby('category')['sales'].sum() groups by 'category' and sums 'sales' per group. You can chain multiple aggregations with .agg({'col1': 'sum', 'col2': 'mean'}).

merge() is a SQL-style join that combines DataFrames on a common key column (like JOIN in SQL). It supports inner, left, right, and outer joins. concat() stacks DataFrames along an axis — axis=0 stacks rows (like UNION ALL in SQL), axis=1 stacks columns side by side. Use merge() when combining tables with a shared key; use concat() when adding more rows or columns.

Use .apply() for element-wise or row/column operations. For a single column: df['col'].apply(my_func) applies my_func to each element. For rows: df.apply(my_func, axis=1) applies to each row. For multiple columns at once, use .applymap() (element-wise across DataFrame). For vectorized operations on numeric columns, prefer NumPy functions directly (faster than apply): df['col'] * 2, np.sqrt(df['col']).