python - operations in pandas DataFrame -


i have large (~5000 rows) dataframe, number of variables, 2 ['max', 'min'], sorted 4 parameters, ['hs', 'tp', 'wd', 'seed']. looks this:

>>> data.head()    hs  tp   wd  seed  max  min 0   1   9  165    22  225   18 1   1   9  195    16  190   18 2   2   5  165    43  193   12 3   2  10  180    15  141   22 4   1   6  180    17  219   18 >>> len(data) 4500 

i want keep first 2 parameters , maximum standard deviation 'seed's calculated individually each 'wd'.

in end, i'm left unique (hs, tp) pairs maximum standard deviations each variable. like:

>>> stdev.head()   hs tp       max       min 0  1  5  43.31321  4.597629 1  1  6  43.20004  4.640795 2  1  7  47.31507  4.569408 3  1  8  41.75081  4.651762 4  1  9  41.35818  4.285991 >>> len(stdev) 30 

the following code want, since have little understanding dataframes, i'm wondering if these nested loops can done in different , more dataframy way =)

import pandas pd import numpy np  # #data = pd.read_table('data.txt') # # don't worry ugly generator, # emulates format of data... total = 4500 data = pd.dataframe() data['hs'] = np.random.randint(1,4,size=total) data['tp'] = np.random.randint(5,15,size=total) data['wd'] = [[165, 180, 195][np.random.randint(0,3)] _ in xrange(total)] data['seed'] = np.random.randint(1,51,size=total) data['max'] = np.random.randint(100,250,size=total) data['min'] = np.random.randint(10,25,size=total)  # , here starts. creators of pandas pull hair out if see this? # can made better? stdev = pd.dataframe(columns = ['hs', 'tp', 'max', 'min']) i=0 hs in set(data['hs']):     data_hs = data[data['hs'] == hs]     tp in set(data_hs['tp']):         data_tp = data_hs[data_hs['tp'] == tp]         stdev.loc[i] = [                hs,                 tp,                 max([np.std(data_tp[data_tp['wd']==wd]['max']) wd in set(data_tp['wd'])]),                 max([np.std(data_tp[data_tp['wd']==wd]['min']) wd in set(data_tp['wd'])])]         i+=1 

thanks!

ps: if curious, statistics on variables depending on sea waves. hs wave height, tp wave period, wd wave direction, seeds represent different realizations of irregular wave train, , min , max peaks or variable during exposition time. after this, means of standard deviation , average, can fit distribution data, gumbel.

this one-liner, if understood correctly:

data.groupby(['hs', 'tp', 'wd'])[['max', 'min']].std(ddof=0).max(level=[0, 1]) 

(include reset_index() on end if want)


Comments

Popular posts from this blog

android - Pass an Serializable object in AIDL -

How to provide Authorization & Authentication using Asp.net, C#? -

How to use Authorization & Authentication in Asp.net, C#? -