Python: Generate new dfs from a list of dataframes using for loop

Jack — Mon, 02 May 2022 13:43:59 GMT

I have a list of dataframes (for this example 2) and want to apply a for-loop to the list of frames to generate 2 new dataframes.

To start, here is my starting dataframe called df_final:

First, I create 2 dataframes: df2_b2c_fast, df2_b2b_fast:

for x in df_final['b2b_b2c_prod'].unique():
    locals()['df2_' + x ] = df_final[(df_final['b2b_b2c_prod'] == x ) ]

Second I calculate the correlation between ADV and pkg_yld:

df_corrs_fast_b2c=df2_b2c_fast.groupby(['bus_nm','id']).corr(method='spearman').unstack().iloc[:,1]
df_corrs_fast_b2b=df2_b2b_fast.groupby(['bus_nm','id']).corr(method='spearman').unstack().iloc[:,1]

Third I convert each to dataframes:

corrs_b2b_fast = (df2_b2b_fast[['adv', 'id']]
        .groupby('id')
        .corrwith(df1['pkg_yld'])
        .rename(columns={'adv' : 'correl'})
        .reset_index())     
 
corrs_b2c_fast = (df2_b2c_fast[['adv', 'id']]
        .groupby('id')
        .corrwith(df1['pkg_yld'])
        .rename(columns={'adv' : 'correl'})
        .reset_index())

Here is one of the 2 dataframes corrs_b2c_fast:

Question: I want to consolidate the steps where (1) I extract my 2 dataframes, (2) estimate the correlations and (3) convert to 2 dataframes using a for loop (corrs_b2b_fast, corrs_b2c_fast); I started below but got stuck:

df_list=[df2_b2b_fast, df2_b2c_fast]      # Subset of dfs
 
for x in df_list['b2b_b2c_prod']:  
 
locals()['corrs_' + x ] = df_list[(df_list['b2b_b2c_prod'] == x ) ]     # Create new 2 dfs from main df 'b2b_b2c_prod'
x= x.groupby(['bus_nm','id']).corr(method='spearman').unstack().iloc[:,1]  # calculate corr between pkg_yld and ADV 
 
  # stuck here...lines needed to create dataframes corrs_b2b_fast, corrs_b2c_fast??

What's wrong?

Re: Python: Generate new dfs from a list of dataframes using for loop

Aviral-Bhardwaj — Fri, 02 Dec 2022 09:44:45 GMT

thanks

topic Re: Python: Generate new dfs from a list of dataframes using for loop in Data Engineering

Python: Generate new dfs from a list of dataframes using for loop

Re: Python: Generate new dfs from a list of dataframes using for loop