corr(self, other: 'DataFrame | Series | None' = None, pairwise: 'bool | None' = None, ddof: 'int' = 1, **kwargs)
This function uses Pearson's definition of correlation (https://en.wikipedia.org/wiki/Pearson_correlation_coefficient).
When other
is not specified, the output will be self correlation (e.g. all 1's), except for ~pandas.DataFrame
inputs with :None:None:`pairwise` set to :None:None:`True`.
Function will return NaN
for correlations of equal valued sequences; this is the result of a 0/0 division error.
When :None:None:`pairwise` is set to :None:None:`False`, only matching columns between :None:None:`self` and other
will be used.
When :None:None:`pairwise` is set to :None:None:`True`, the output will be a MultiIndex DataFrame with the original index on the first level, and the other
DataFrame columns on the second level.
In the case of missing elements, only complete pairwise observations will be used.
If not supplied then will default to self and produce pairwise output.
If False then only matching columns between self and other will be used and the output will be a DataFrame. If True then all pairwise combinations will be calculated and the output will be a MultiIndexed DataFrame in the case of DataFrame inputs. In the case of missing elements, only complete pairwise observations will be used.
Delta Degrees of Freedom. The divisor used in calculations is N - ddof
, where N
represents the number of elements.
For NumPy compatibility and will not have an effect on the result.
Return type is the same as the original object with np.float64
dtype.
Calculate the rolling correlation.
cov
Similar method to calculate covariance.
numpy.corrcoef
NumPy Pearson's correlation calculation.
pandas.DataFrame.corr
Aggregating corr for DataFrame.
pandas.DataFrame.rolling
Calling rolling with DataFrames.
pandas.Series.corr
Aggregating corr for Series.
pandas.Series.rolling
Calling rolling with Series data.
The below example shows a rolling calculation with a window size of four matching the equivalent function call using numpy.corrcoef
.
>>> v1 = [3, 3, 3, 5, 8]This example is valid syntax, but we were not able to check execution
... v2 = [3, 4, 4, 4, 8]
... # numpy returns a 2X2 array, the correlation coefficient
... # is the number at entry [0][1]
... print(f"{np.corrcoef(v1[:-1], v2[:-1])[0][1]:.6f}") 0.333333
>>> print(f"{np.corrcoef(v1[1:], v2[1:])[0][1]:.6f}") 0.916949This example is valid syntax, but we were not able to check execution
>>> s1 = pd.Series(v1)
... s2 = pd.Series(v2)
... s1.rolling(4).corr(s2) 0 NaN 1 NaN 2 NaN 3 0.333333 4 0.916949 dtype: float64
The below example shows a similar rolling calculation on a DataFrame using the pairwise option.
This example is valid syntax, but we were not able to check execution>>> matrix = np.array([[51., 35.], [49., 30.], [47., 32.], [46., 31.], [50., 36.]])This example is valid syntax, but we were not able to check execution
... print(np.corrcoef(matrix[:-1,0], matrix[:-1,1]).round(7)) [[1. 0.6263001] [0.6263001 1. ]]
>>> print(np.corrcoef(matrix[1:,0], matrix[1:,1]).round(7)) [[1. 0.5553681] [0.5553681 1. ]]This example is valid syntax, but we were not able to check execution
>>> df = pd.DataFrame(matrix, columns=['X','Y'])This example is valid syntax, but we were not able to check execution
... df X Y 0 51.0 35.0 1 49.0 30.0 2 47.0 32.0 3 46.0 31.0 4 50.0 36.0
>>> df.rolling(4).corr(pairwise=True) X Y 0 X NaN NaN Y NaN NaN 1 X NaN NaN Y NaN NaN 2 X NaN NaN Y NaN NaN 3 X 1.000000 0.626300 Y 0.626300 1.000000 4 X 1.000000 0.555368 Y 0.555368 1.000000See :
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them