Working with Pandas Dataframes - manipulate data

Question

i was trying to find out the solution by myself but unfortunately couldn't succeed.

-> i have a scientific dataset with 12 columns representing 4 different monitoring stations and each station measures 3 different kind of pollutants. the data contains around 70000 chronological timestamps (rows). since 2 out of the 3 pollutants are given in a different unconvenient unit i have to multiply those columns by a certain value (i.e. have to manipulate 8 columns)

how can i do that so i get the dataframe arranged in the same way but with the manipulated columns integrated?

thank you in advance for any help provided!

Answer 1

Let's start with a simplified version of your problem.

Imagine I have a dataframe of temperatures for various cities over time. However, for some of those cities, I have the temperatures in Celsius, and I need to convert those columns to Fahrenheit. Other cities are already in Fahrenheit.

Here's the example dataset:

And here it is in code form:

import pandas as pd

df = pd.DataFrame({'Denver': {0: 21.5, 1: 17.1, 2: 24.6, 3: 20.7, 4: 17.6, 5: 25.0, 6: 21.2, 7: 23.0, 8: 18.6, 9: 20.1}, 'Colorado Springs': {0: 70.2, 1: 79.7, 2: 74.8, 3: 64.6, 4: 68.9, 5: 65.3, 6: 61.5, 7: 69.5, 8: 63.7, 9: 62.4}, 'Aurora': {0: 18.9, 1: 20.7, 2: 21.8, 3: 21.8, 4: 24.6, 5: 17.3, 6: 17.1, 7: 20.7, 8: 16.3, 9: 22.4}, 'Fort Collins': {0: 63.1, 1: 63.3, 2: 69.0, 3: 75.6, 4: 73.4, 5: 78.9, 6: 71.1, 7: 70.4, 8: 71.0, 9: 75.1}, 'Lakewood': {0: 68.9, 1: 64.4, 2: 72.7, 3: 61.1, 4: 66.8, 5: 77.6, 6: 68.9, 7: 77.5, 8: 71.3, 9: 76.0}, 'Thornton': {0: 80.0, 1: 77.9, 2: 75.4, 3: 62.8, 4: 77.3, 5: 69.8, 6: 76.3, 7: 78.0, 8: 69.1, 9: 63.3}, 'Arvada': {0: 23.9, 1: 21.5, 2: 22.6, 3: 15.8, 4: 22.3, 5: 16.2, 6: 25.6, 7: 25.2, 8: 25.7, 9: 25.3}, 'Westminster': {0: 69.2, 1: 68.5, 2: 66.4, 3: 68.6, 4: 80.0, 5: 68.4, 6: 64.6, 7: 66.2, 8: 73.0, 9: 76.6}, 'Pueblo': {0: 62.4, 1: 76.3, 2: 62.3, 3: 75.0, 4: 80.0, 5: 62.9, 6: 67.5, 7: 64.3, 8: 78.7, 9: 71.5}, 'Greeley': {0: 79.0, 1: 64.4, 2: 65.5, 3: 68.7, 4: 73.6, 5: 73.6, 6: 60.6, 7: 74.0, 8: 75.6, 9: 77.6}})

In this example, I want to convert the values for Denver, Arvada, and Aurora from Celsius into Fahrenheit, then save that back into the same dataframe. For every temperature value in those columns, I need to multiply by 1.8, and add 32.

Fortunately, Pandas makes it easy to manipulate an entire column. Converting Denver to Fahrenheit looks like this:

df["Denver"] = df["Denver"] * 1.8 + 32

In Pandas, multiplying a column by a scalar (i.e. single value) means that you multiply each element by that scalar.

Then, I can do the same thing for Aurora and Arvada. Normally, I would use a loop, to avoid writing the same piece of code three times.

celsius_cities = ["Denver", "Arvada", "Aurora"]
for col in celsius_cities:
    df[col] = df[col] * 1.8 + 32

However, since you only have eight columns, and since you're multiplying by a different value each time, a loop might not be helpful for you.