3,656 questions
0
votes
1
answer
35
views
How to group by a column and calculate correlation coefficients between multiple columns?
I'm encountering some issues when trying to perform grouped correlation calculations in DolphinDB. Here's my scenario:
I'm using DolphinDB to calculate correlations between multiple columns in a table....
- 1
0
votes
1
answer
90
views
Correlate2d using pearson zscore normalization
Given two 2d arrays I want to calculate the cross-correlation similar as done in scipy.signal.correlate2d but instead of using a fill_value=0 I want to calculate the Pearson R from the data overlap of ...
- 170
0
votes
1
answer
63
views
Different correlation functions showing different values? [closed]
I created a correlation table of my data in R to prepare for an exploratory factor analysis:
corrmatrix <- cor(data, use="pairwise.complete.obs")
Then, I used two different functions to ...
0
votes
1
answer
64
views
Remove redirected requests in Vugen
I had recorded a web application in dev tools and save it as har file and later recorded the file using Vugen, in this application at login level we are getting 5 https request and all of them are ...
- 23
0
votes
0
answers
27
views
Logistic regression variables correlation but low GVIF
I'm making a logistic regression model to predict female presence on boards in tech SMEs. I was going to take out companies with only 1 employee, as they don't have boards, but my supervisor told me ...
- 1
1
vote
1
answer
98
views
Using scipy.ndimage.correlate only calculate elements with full overlap
I am trying to use scipy.ndimage.correlate to replicate the output of IDL convol() function. The IDL function only calculates elements where there is full overlap between the input and the kernel.
So, ...
- 13
0
votes
1
answer
92
views
R correlation: I'm getting inconsistent correlation results with cor() function
I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:
data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary ...
- 359
0
votes
3
answers
150
views
How do I make a correlation matrix for 27 variables which doesn't look crowded? [closed]
I need to make a correlation matrix of 27 variables for a research project I'm working on. I also ideally need to be able to put it on Word. I've tried
round(cor(correlation_matrix_data, use = "...
- 11
3
votes
8
answers
153
views
convert source target value dataframe into a correlation matrix
I have a data.frame containing different combinations of a group and count values where both of them exist. I need to plot or create a matrix similar to correlation matrix. I have come up with a ...
- 5,655
0
votes
0
answers
55
views
Multiple correlation testing in correlation vs rstatix yield different p-values?
I want to create a table that just simply tests the correlation between the two variables of the dataframe. The dataframe consists of 50 rows, with approximately 50 variables. Below is an example ...
- 3
0
votes
2
answers
74
views
Plot correlation matrix using heatmap
I have the following DataFrame (sample image).
I estimated the correlation matrix between Sales, Income, and Com for each county with the intend on plotting it into a heatmap.
import numpy as np
...
- 15
2
votes
0
answers
92
views
change color of text in correlation matrix made in ggplot [closed]
I would like to add colors to the text labels to a correlation matrix in ggplot. Other than the colors, my plot looks satisfactory to me. I will paste in the script for the correlation matrix first, ...
- 31
0
votes
1
answer
31
views
How to replace values in a matrix based on a conditional variable
I have this data:
data(mtcars)
correlation_matrix <- cor(mtcars)
matcor <- cor(correlation_matrix, method = "spearman") #Faz a correlations
print(matcor, digits = 2) #Exibe as ...
1
vote
0
answers
107
views
Non-Symmetric Calculation for Correlation Matrix in Polars Plugin
I have written a Polars plugin to calculate the symmetric correlation matrix inspired by Polars DS project.
On the Rust side I have:
#![allow(unused_imports)]
use polars::prelude::*;
use pyo3_polars::{...
- 649
1
vote
0
answers
92
views
Calculate correlation on dict type variables
I have a dataframe named hyperparam_df which looks like the following:
repo_name file_name \
0 DeepCoMP deepcomp/util/simulation.py ...
0
votes
2
answers
700
views
Spearman Rank Correlation in SciPy vs. Manual Calculation - Mismatch in Results
I am trying to compute the Spearman rank correlation coefficient between two ranked lists using scipy.stats.spearmanr and manually using the formula:
However, I am getting significantly different ...
0
votes
0
answers
92
views
Correct value for fillna for Pandas dataframe correlation
I have a huge array of products and their prices timeseries data.
I want to find the correlation between those products prices. Most of the data has null values when the product is not yet into the ...
0
votes
2
answers
115
views
How to use column names in a list of values in a loop?
The code below is what I am currently using. It will take a tibble of population data and join a separate tibble with three different variables. I then am only running the process of taking three ...
- 13
0
votes
0
answers
63
views
Numpy.correlate returns incorrect value
Sorry, first timer.
I am trying to verify cross correlation between time series S1, S2, especially the correlation coefficient.
S1. S2
0.029 2.470
0.030 1.750
0.030 2.200
0.030 2.670
0....
0
votes
0
answers
39
views
Correlating paired time series data
I am looking to analyze a dataset with many paired time series, but it’s the first time I deal with such data type and was wondering what’s the best way to analyze them. I am looking for the effect of ...
- 1
0
votes
2
answers
425
views
Phase of two signals by correlation slightly off, scipy signal.correlate
I want to get the phase shift of two signals via the correlation function. In general I set up a sine with frequency f0 and a 2nd sine (with noise) and a given phase phase_deg. I correlate both ...
- 321
1
vote
2
answers
72
views
Correlate columns in two pandas dataframes with varying data types
I have two Excel worksheets, one of which ("edit") is a slightly modified version of the other ("base"). I want to figure out if any columns have been added, deleted, or moved. I ...
- 304k
2
votes
1
answer
145
views
How to run a correlation test using dummy variables
I am quite new to using r and am struggling to find a few to actually find a pearson correlation coeffcient from a set data. I am attempting to analyze whether there is a correlation between scores ...
- 23
1
vote
2
answers
108
views
How to generate a sample with a given spearman coefficient
In order to create a dataset to test a statistical calculation package, I want to be able to generate a sample that is correlated to a reference sample with a given searman coefficient.
I managed to ...
- 647
0
votes
1
answer
87
views
How to improve visibility of correlation plot in R? [closed]
I am trying to create a correlation plot in R, but the output looks blurry or unclear, and I can't view the details properly. Below are the steps I followed:
library(corrplot)
library(Hmisc)
col = ...
- 589
0
votes
1
answer
81
views
Change correlation matrix plot range to be from 0 to 1 - R
How can I change the plot range to be from 0 to 1 as the correlation matrix is always positive.
Script:
require(RColorBrewer)
library(corrplot)
set.seed(123)
data <- matrix(runif(100, min = 0, max ...
- 161
-1
votes
1
answer
101
views
Polychoric Correlation and CI
I am trying to calculate some polychoric correlations (reproducible code here)
library(polycor)
Var1 <- (c(1,2,3,1,2,2,3,2,2,1,2))
Var2 <- (c(2,2,3,1,2,1,3,2,2,1,2))
df <- as.data....
- 145
0
votes
2
answers
79
views
How to make a ggplot combined of main scatterplot with two histograms?
I have a dataset df I want to analyze and visualize with a scatterplot for correlation with histograms. Moreover, color data points in blue for one and yellow for the other, but pairs of non-zeros for ...
- 129
1
vote
1
answer
165
views
Handling Correlation ID Changes in Event Sourcing When an Entity Switches Context
I'm working on an event-sourced application that crawls sports betting games from different bookmakers. I have two primary aggregates in my system:
Game: Represents a sports betting event for a ...
- 12.8k
2
votes
1
answer
107
views
Psych library: factor analysis not convergence with mixed data
I want to compute factor analysis with mixed data (i.e., continous, categorical and binary) but I have a lot of warnings and it does not converge resulting in Nans ( objective num NaN,
criteria NaN NA ...
- 33
0
votes
1
answer
237
views
spearman's rho plot [closed]
I have done a Spearman correlation and need a scatterplot with a line matching the Spearman's rho.
How can I do this?
> db
id Var_1 Var_2 Var_3 Var_4
1 5 8.17 83.08 10.28 19.81
2 6 4....
- 483
0
votes
0
answers
56
views
Correlation and Multicolinearity Analysis for BigData
Pandas is very well optimized to handle the most common scenarios that we need when we preprocess or analyze data.
What about BigData, specifically BigData (1k+ columns, 100k rows) in BigQuery?
The ...
- 513
0
votes
2
answers
52
views
Why geom_raster() is not displaying the good columns orders in my heatmap?
I'm using a double triangular matrix (10x10) representing 2 types of correlations between these variables.
The dataset is like this one (4x4).
ff <- data.frame(Var1=as.factor(c("V1", &...
- 45
1
vote
1
answer
206
views
Why is this correlation coefficient given by pandas unrealistically low?
I am coding in python and I am correlating a row in pandas (index 2500) with a sinusoidal function that I defined (sine_modulation). When I print the value I obtain by using
row_correlation(...
0
votes
0
answers
32
views
Correlational analysis focusing on one variable and indicating significant values
I am able to make a correlation matrix that is formatted with the significant values and stars indicating how significant but dont know how to make it so it focuses only on VOC.
Output I have:
...
0
votes
0
answers
59
views
Automating Correlation Analysis Using Excel/related software
Let's say we have a large dataset with several samples over time. I'm looking for an automated way to find correlations over a specific time period. For example, in cycles, we might observe that when ...
- 373
1
vote
0
answers
64
views
How compute correlation between images
I try to compute correlation between classes. The dataset involves several folders which are named by class. Each folder includes several images, I have used the below code to input images and ...
- 23
0
votes
1
answer
137
views
Adding p-value and r-value in separate lines in correlogram using ggpairs in R
I am trying to plot a correlation matrix using the ggpairs function in the GGally R package. By default it displays the correlation coefficient in digits and adds stars next to it to represent the p-...
- 559
-1
votes
1
answer
54
views
How to use id_token (value) from html response in jmeter correlation
i am trying to replace id_token value in the request but its getting error.
I have used regular expression ext to correlated the value and used this function: id_token" value="(.*?)"/&...
- 1
-1
votes
1
answer
45
views
Time-series correlating suspiciously high
I have two time series from 1986-01-01 to 2024-04-01 following CPI prices of meat and fish and I want to find the correlation between the both of them.
They are huge dataframes so below is a truncated ...
- 1
0
votes
1
answer
39
views
JMeter dynamic correlation assistance
I am new to Jmeter. I have been trying to perform recorded scripts on ServiceNow catalog item that I have created. I think I was able to correlate the x-User-Token from the login Sample to the later ...
0
votes
1
answer
190
views
Is there a way to calculate a Spearman's correlation row by row in a data frame?
I have a large data frame, the first four rows of my data frame look like this:
X1 <- list(c(1, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5), c(1.5, 1.5, 5, 5, 5, 5, 5), c(1, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5), c(1, 4....
- 3
0
votes
1
answer
105
views
Get formatted correlation coefficients with significance levels in stata
I have a correlation matrix I estimate in Stata:
estpost corr var1 var2 var3 var4, matrix listwise
I want to get the output to LaTeX, simultaneously displaying only 3 decimal places and ...
- 3
0
votes
1
answer
79
views
How can I efficiently generate every possible cluster of items such that everything within the cluster is correlated with each other?
The following is an example dataframe called matrix that I have in my program:
I have a variable called "plimit" which is currently 0.7.
I want an algorithm such that clusters are made from ...
0
votes
1
answer
83
views
Spearman correalation in R between different categories on likert scale data
i have a data set with 2 columns.One column is the answers in a likert ordinal Scale and the second column is the category of each question.
df
# A tibble: 50 × 2
answers Cat
<...
- 1,358
0
votes
1
answer
113
views
correlation table with modelsummary
I am trying to generate a correlation table using the datasummary_correlation() function from the modelsummary package. I am using a custom function corstars to display significance levels with ...
- 401
0
votes
1
answer
157
views
Getting ACF using statsmodels for a list of lags
I am able to use statsmodels.graphics.tsaplots.plot_acf to get a plot of the acf's for a specific list of lags, as the lags input accepts a list of values. I would like to have the raw data of the acf'...
- 235
2
votes
2
answers
429
views
Issue in aligning stat_cor r-value and p-value displayed on two lines in R
I am using stat_cor with ggplot to add r-value and p-value to a scatter plot. I have used label.sep = "\n" to display the r-value and p-value on two lines. However the labels are not aligned ...
- 559
0
votes
1
answer
52
views
Approach to compute pairwise correlations in R for 2 different matrices with partially matched features
I would like to ask for a more "complex" case scenario I'm phasing for analysing high-throughput biological data, aiming to estimate some pairwise correlation coefficients. The input data ...
- 203
1
vote
1
answer
99
views
Create covariance matrix on columns that are not full
I'm looking to create a variance/covariance matrix or a correlation matrix (I don't care which) on stock data.
The issue is that different symbols trade at different times, so my dataframe is not full....
- 93