Data Handling using Pandas - II
Changing the index and reshaping
Use reset_index, set_index, pivot and pivot_table.
Index. When you take a slice of rows, the new DataFrame keeps the original index labels, so they may not be continuous (for example 3, 7, 11). reset_index() creates a new continuous index 0, 1, 2...
set_index('Name') makes a column the index; reset_index() turns it back into an ordinary column.
Reshaping. Data can be arranged in different shapes for analysis.
- pivot(index=, columns=, values=) reshapes a DataFrame: one column's values become the row labels, another's become the column labels, and a third fills the table. Combinations that do not exist become NaN.
- pivot() fails with the error 'Index contains duplicate entries' if an index-column pair occurs twice, since it cannot decide which value to use.
- pivot_table() solves this: it aggregates the duplicates. By default it takes the mean; with aggfunc= you can use sum, max, min, len and others, even several at once.
Example: if a shop has sales by store and year, with two entries for one store-year, pivot() fails, but pivot_table(..., aggfunc='sum') adds them. pivot reshapes; pivot_table also aggregates.
Practise this lesson
Questions that get harder as you improve, a Tutor that helps without giving the answer away, and progress you can see. Free for every learner.
No account, no email address. Opens the real product.