I have a large CSV
file that it is opened with Dask.
import numpy as np
import pandas as pd
import hvplot.pandas
import hvplot.dask
import intake
data = '../file.csv'
ddf = intake.open_csv(data).to_dask()
ddf.head()
Datetime latitude longitude Temp_2m(C)
1 1980-01-02 03:00:00 30.605 50.217 5.31
2 1980-01-02 04:00:00 30.605 50.217 5.36
3 1980-01-02 05:00:00 30.605 50.217 7.04
4 1980-01-02 06:00:00 30.605 50.217 10.24
I want to plot Temp_2m(C)
monthly with hvplot. Plot with hourly data of Datetime
is done correctly, but when I want to group Datetime
as follow, It return an error.
# Convert 'Datetime' column to 'datetime64'
ddf["Datetime"] = ddf["Datetime"].astype("M8[us]")
# set index column
ddf = ddf.set_index('Datetime')
g = pd.Grouper(freq='M', key='Datetime')
month_ddf = dff.groupby(g).mean()
# plot
month_ddf.hvplot('Temp_2m(C)')
ERROR:
ValueError: all keys need to be the same shape
what is my mistake?
for reply @frankr6591:
month_ddf.describe()
Dask DataFrame Structure:
latitude longitude Temp_2m(C)
npartitions=1
float64 float64 float64
... ... ...
Dask Name: describe-numeric, 89 tasks
I used to_datetime() and got correct plot with .plot()... ran into problems installing hvplot.