Search
 
SCRIPT & CODE EXAMPLE
 

PYTHON

using pandas stack and subset to return a dataframe object of highly correated pairs

# save absolute value of correlation matrix as a data frame
# converts all values to absolute value
# stacks the row:column pairs into a multindex
# reset the index to set the multindex to seperate columns
# sort values. 0 is the column automatically generated by the stacking

df= old_df.corr().abs().stack().reset_index().sort_values(0, ascending=False)

# zip the variable name columns (Which were only named level_0 and level_1 by default) 
# in a new column named "pairs".
df['pairs'] = list(zip(df.level_0, df.level_1))

# set index to pairs
df.set_index(['pairs'], inplace = True)

#d rop level columns
df.drop(columns=['level_1', 'level_0'], inplace = True)

# rename correlation column as cc rather than 0
df.columns = ['cc']

# drop duplicates. This could be dangerous if you have variables perfectly correlated with variables other than themselves.
# for the sake of exercise, kept it in.
df.drop_duplicates(inplace=True)

# subset to include desired range
df[(df.cc>.75) & (df.cc <1)]
Comment

PREVIOUS NEXT
Code Example
Python :: a string varible in python 
Python :: codegrepper is cool 
Python :: custom header footer in odoo 
Python :: Add one to a column pands 
Python :: how to plot quantity of each value of a feature in python 
Python :: python script to execute shell azure cli commands in python 
Python :: arcpy line density 
Python :: airflow set ui color of operator ui_color 
Python :: with suppress(exception) python 
Python :: pubmed database python 
Python :: Factory reset the filesystem micropython 
Python :: python check if division has remainder 
Python :: #finding the differences between setA and SetB: 
Python :: python generic class inheritance 
Python :: python source script custom functions 
Python :: how to solve spacy no model en 
Python :: sns nan matrix 
Python :: flask run function every minute 
Python :: clipping path image using python 
Python :: &lt;function chr(i, /)&gt; error in python 
Python :: how to change continuous colour in plotply 
Python :: Qt convert image to base64 
Python :: fast comand exit python windows 
Python :: how to change the type of a values in list from str to object python 
Python :: combining sparse class 
Python :: how to write string in python 
Python :: how to convert small letters to capital letters in python 
Python :: fibonacci program in python 
Python :: indentation error in python atom editor 
Python :: with open("[Followed][{}]".format(self.username), "a+") as flist: 
ADD CONTENT
Topic
Content
Source link
Name
8+5 =