python - operations in pandas DataFrame -
i have large (~5000 rows) dataframe, number of variables, 2 ['max', 'min'], sorted 4 parameters, ['hs', 'tp', 'wd', 'seed']. looks this:
>>> data.head() hs tp wd seed max min 0 1 9 165 22 225 18 1 1 9 195 16 190 18 2 2 5 165 43 193 12 3 2 10 180 15 141 22 4 1 6 180 17 219 18 >>> len(data) 4500 i want keep first 2 parameters , maximum standard deviation 'seed's calculated individually each 'wd'.
in end, i'm left unique (hs, tp) pairs maximum standard deviations each variable. like:
>>> stdev.head() hs tp max min 0 1 5 43.31321 4.597629 1 1 6 43.20004 4.640795 2 1 7 47.31507 4.569408 3 1 8 41.75081 4.651762 4 1 9 41.35818 4.285991 >>> len(stdev) 30 the following code want, since have little understanding dataframes, i'm wondering if these nested loops can done in different , more dataframy way =)
import pandas pd import numpy np # #data = pd.read_table('data.txt') # # don't worry ugly generator, # emulates format of data... total = 4500 data = pd.dataframe() data['hs'] = np.random.randint(1,4,size=total) data['tp'] = np.random.randint(5,15,size=total) data['wd'] = [[165, 180, 195][np.random.randint(0,3)] _ in xrange(total)] data['seed'] = np.random.randint(1,51,size=total) data['max'] = np.random.randint(100,250,size=total) data['min'] = np.random.randint(10,25,size=total) # , here starts. creators of pandas pull hair out if see this? # can made better? stdev = pd.dataframe(columns = ['hs', 'tp', 'max', 'min']) i=0 hs in set(data['hs']): data_hs = data[data['hs'] == hs] tp in set(data_hs['tp']): data_tp = data_hs[data_hs['tp'] == tp] stdev.loc[i] = [ hs, tp, max([np.std(data_tp[data_tp['wd']==wd]['max']) wd in set(data_tp['wd'])]), max([np.std(data_tp[data_tp['wd']==wd]['min']) wd in set(data_tp['wd'])])] i+=1 thanks!
ps: if curious, statistics on variables depending on sea waves. hs wave height, tp wave period, wd wave direction, seeds represent different realizations of irregular wave train, , min , max peaks or variable during exposition time. after this, means of standard deviation , average, can fit distribution data, gumbel.
this one-liner, if understood correctly:
data.groupby(['hs', 'tp', 'wd'])[['max', 'min']].std(ddof=0).max(level=[0, 1]) (include reset_index() on end if want)
Comments
Post a Comment