441 questions
2
votes
0
answers
55
views
GLMNET gives different solutions for lasso-based algorithms when I expect the same solution
There are many ways to implement an adaptive lasso model. One of them would be to:
Solve an unpenalized regression model and extract the betas
Define w=1/abs(beta)
Use these weights in an adaptive ...
1
vote
0
answers
77
views
My computation time doubled when I use parallel option in glmnet
I am trying to simulate a LASSO estimator for approximately 10,000 repetitions and naturally thought that parallel computing would significantly speed up the process. However, when I tested a simple ...
- 11
3
votes
0
answers
58
views
How to fix exploding Coefficients for Lasso Regression [closed]
I trid to implement the coordinate descent algorithm for the Lasso Regression (in R) following this post. The only difference is the scaling of the cost function by 1/n according to my text book here ...
- 31
-1
votes
1
answer
85
views
Scratch implementation of ridge regression
I am trying to implement Ridge regression but I feel like I am missing something with the Python operators. Here is my code:
import numpy as np
x = np.random.rand(10, 2)
y = np.random.rand(10, 1)
...
1
vote
1
answer
147
views
Access to Lasso coefficients using Tidymodels
I'm trying to follow along to one of Julia Silge's blog posts about lasso regression using tidymodels however I am getting stuck when trying to get access to the coefficients.
I wish to plot something ...
- 2,917
0
votes
0
answers
238
views
Unexpected coefficient behaviour of sklearn Lasso for small alpha values
Here is some basic code generating a regression problem.
import numpy as np
import pandas as pd
from sklearn.linear_model import Lasso, LinearRegression
n = 100
np.random.seed(42)
X = np.random....
- 37
1
vote
0
answers
92
views
After using ga.lasso from the miselect package how do I pool results?
ive been running multiple imputation on a dataset using the mice package creating 5 mids objects. Using those objects ive performed variable selection using the cv.galasso function from the miselect ...
- 19
0
votes
1
answer
117
views
How can I display predictor importance + feature name in multivariate regression?
I'm exploring a dataset with the goal to find any interesting relationships (there are a bunch of variables of interest and I want to see which features or feature combinations predict them).
As a ...
- 5
0
votes
1
answer
100
views
Applying standardisation to LASSO and KNNREG in R
To my Understanding standardisation is used for robustness. However, I'm having difficulty in understanding how to apply standardisation to my data in a LASSO and KNN Regression in the cv.glmnet and ...
- 13
0
votes
1
answer
174
views
Implementing custom lasso regression with fixed features in sklearn pipeline for variable selection
There are two posts related to this topic in R language including fixed regressor in a Lasso regression model and fixed effect Lasso logit model
I am writing a feature selection model using Lasso ...
- 399
1
vote
1
answer
124
views
How to redraw a LASSO regression plot using ggplot?
After the LASSO regression is constructed, plot(fit) can be employed to plot the LASSO graph. But I want to redraw this graph using ggplot and beautify it. I use the following code to redraw it, but ...
- 317
0
votes
1
answer
306
views
Penalized regression with cph() function of rms package in R
I am little bit crazy trying to run a penalized cox ph regression model with functions of rms package. I’ve done it with glmnet package but I would like to analize the outputs with rms and Hmisc ...
0
votes
1
answer
100
views
Logistic Lasso on large gene dataset specifically through the Knockoff package in R
This question is perhaps in an uncanny valley between CrossValidated and StackOverflow, as I'm trying to understand the methodology of functions in an R package, in the context of executing them ...
1
vote
0
answers
109
views
Double selection lasso in and NA's handling [closed]
I work in a team where everyone uses Stata, and I work in R.
I have created an efficient workflow that allows me to export the results quickly.
The problem I ran into was when implementing the double ...
- 701
1
vote
1
answer
424
views
sklearn ridgeCV versus ElasticNetCV
When setting l1_ratio = 0, the elastic net reduces to Ridge regression.
However, I am unable to match the results obtained from sklearn's ridgeCV versus ElasticNetCV. They appear to produce very ...
- 378
1
vote
0
answers
411
views
Python logistic regression in statsmodels using l1 penalty with class weights
I would like to run logistic regression in statsmodels using an l1 penalty (lasso) and class weights due to a class imbalance. There are several posts that explain how to either implement logistic ...
- 73
2
votes
1
answer
218
views
Extend Coordinate Descent for LASSO to Adaptive LASSO
I currently have a simple coordinate descent algorithm to solve LASSO in python:
def lasso_nb(X, y, alpha, tol=0.001, maxiter=10000):
n, p = X.shape
beta = np.zeros(p)
R = y.copy()
...
- 173
0
votes
0
answers
148
views
Lasso regression does not work properly in tidyverse
I was trying to replicate Julia Silge's great lesson on Lasso regression using tidymodel but even if the data cleaning gives exactly the same result, I have found that the same coding for the ...
- 197
1
vote
1
answer
108
views
Overlap Lasso Plot with R ggplot2
I'm trying to reproduce the plot of the experiment in the paper "Group Lasso with Overlap and Graph Lasso - Jacob, Obozinksi and Vert". I was able to produce the data but I don't know where ...
- 11
-1
votes
1
answer
134
views
Error in the developing Linear regression model/ R is not responding
I was in the process of analysis of a dataset related to property sales using r-studio and what I tried to do was develop a model. all was smooth but this code
lm_model <- lm(`SALE PRICE` ~ ., ...
- 1
0
votes
1
answer
442
views
Penalty factors in glmnet with multimomial logit
I'm trying to fit an adaptive lasso for a multinomial logit regression with glmnet. My problem is the following: when I try to use the penalty matrix (a 2x3 matrix) penalty.factor in cv.glmnet I get ...
- 23
1
vote
1
answer
408
views
Advice on implementing L1 regularisation in logistic model for feature selection
I am working on an academic project where the aim is to understand which features predict a binary outcome. I have therefore used a logistic model. However, there is a problem with colinearity as ...
- 37
1
vote
1
answer
85
views
How to use CAST package for a shapefile (polygons) in R?
any help with the following is really appreciated!!
My goal: I need to run a lasso model for variable selection for my data (which is in sf polygon format).
My data: As said above, is a sf object. ...
1
vote
1
answer
166
views
Is there a way to get the beta coefficients from a lasso model using spatial (polygons) data?
I'm trying to run a lasso model for spatial (polygons) data in R. It has been very very difficult to find a package that allows me to do run the spatial lasso model and at the same time allows me to ...
-1
votes
1
answer
65
views
The difference between random states in sklearn
I am writing this to understand more about regressions in machine learning. When I set 42 for random_state, lasso seems to predict badly. But it is vice versa when I set 2. Is there any way to choose ...
1
vote
0
answers
129
views
the goodness of fit to the elasticnet model looks bad
import os
import pandas as pd
from sklearn.linear_model import ElasticNet
from sklearn.model_selection import train_test_split, GridSearchCV, cross_validate
from sklearn.metrics import ...
- 11
0
votes
1
answer
83
views
How to re-write my iterative LASSO via lars() code run on N datasets to using cv.lars instead so I don't have to hardcode the lambda value
Here is a link to the GitHub Repository for this project.
I wrote R scripts that iteratively estimate N LASSO on each of N randomly generated synthetic datasets within a file folder because LASSO is ...
- 173
3
votes
0
answers
144
views
How to calculate the marginal effect of lasso logistic regression?
I have seen the method of calculating marginal effect for glm, gam and polr packages, but how to calculate marginal effect for lasso regression, such as glmnet packages?
Is it extracting the ...
- 31
1
vote
0
answers
114
views
LASSO with glmnet, year dummies and penalty.factor, different results than with demeaned data
Not sure if this is a question for Cross Validated or Stackoverflow, will try both.
I am running a LASSO using glmnet. I have panel data. I first run cv.glmnet() to select the best lambda, then I run ...
- 383
1
vote
1
answer
637
views
Is regularization (Ridge/Lasso) implemented in statsmodels' GLM model?
I have the following model:
from statsmodels.formula.api import glm
modelSpecification = glm(
formula="wage ~ workhours + gender",
data=train,
...
- 296
0
votes
1
answer
107
views
How do I perform this cross-validation for ridge/lasso regression using randomly generated data?
Here is my code:
library(leaps)
library(glmnet)
set.seed(7)
x <- runif(100,0,1)
y <- 1 + 2*x^2 + 4*x^3 + x^4 + rnorm(100,0,1)
data_1<- data.frame(x,y)
xridge<- model.matrix(y~x+I(x^2)+I(x^...
0
votes
0
answers
215
views
Difficulty matching scikit-learn LASSO results with L1 Regularization or AdamW
I’m trying to replicate some basic linear regression results from sci-kit learn’s LASSO implementation into various pyTorch linear and non-linear models (dense & LSTM) and finding that the ...
- 21
0
votes
1
answer
331
views
Predicting on a glmmLasso model
I'm trying to use the glmmLasso pacakage and am able to run a model and get a summary output and extract the fitted values, but I'm wondering if/how I can make predictions with it?
Am I missing a step?...
- 57
0
votes
1
answer
111
views
Implementing HMLasso in Python with Cvxpy leads to problem with DCP rules
In an attempt to implement the HMLasso (Lasso with High missing rate, see https://www.ijcai.org/proceedings/2019/0491.pdf) in Python, I wanted to use Cvxpy. However, I've been facing an exception that ...
- 133
2
votes
1
answer
327
views
How can I pass an extra variable to a tidymodels fit function?
I am writing a tidymodels engine to fit the joint LASSO described here and implemented in the fuser package. This is a LASSO model for regression that allows partial sharing of information between ...
- 3,068
1
vote
0
answers
193
views
Lasso learner classif.cv.glmnet in mlr3: Access the final model with lamda and β coefficients that is used for performance evaluation and prediction?
I am trying to do a lasso regression for a binary classification task in mlr3 using the learner lrn("classif.cv_glmnet").
My goal is to train this learner on the final task and access the ...
- 39
0
votes
0
answers
64
views
l1 regularized logistic regression with the response -1 or 1, not 0, 1
I want to fit the model using l1 regularized logistic regression.
The response is -1, or 1, instead of 0 or 1.
I am trying to reproduce the simulation of High-dimensional Ising model selection using ...
- 1
0
votes
1
answer
58
views
How to iteratively remove just the intercept terms from the variables selected by n glmnet functions run on n datasets in R
I have run N individual LASSO Regressions on N different data sets using the glmnet() function from the package of the same name in RStudio using the following lines of code:
# This function fits all ...
- 173
0
votes
1
answer
70
views
How to systematically replicate the results of running n LASSOs on n data sets in R using enet() with lars()
My code used to fit k LASSO Regressions on k csv file-formatted data sets via the enet() function from the following:
set.seed(150)
system.time(LASSO <- lapply(datasets, function(J)
...
- 173
1
vote
1
answer
405
views
Inconsistent results trying to reproduce the variables selected by LASSO ran using enet(lambda = 0) with lars(type = "lasso")
The scripts which are referenced and from which the lines of code included came from can all be found at my GitHub Repository for this project, along with mini practice datasets (called ten, top 50, ...
- 173
1
vote
1
answer
269
views
Can glmmLasso be used with the Tweedie distribution?
I have a linear mixed effects model and I am trying to do variable selection. The model is testing the level of forest degradation in 1000 sampled points. Most points have no degradation, and so the ...
- 13
1
vote
1
answer
297
views
Validation but not cross validation in Python sklearn Lasso
I need to train a LASSO model using sklearn. I am given a pair of specifically designed training and validation datasets.
The goal is to let the algorithm autogenerate a sequence of alphas (the L1 ...
- 1,820
0
votes
1
answer
283
views
How to Holiday day Variable in Regression model?
I want to build a regression model with a dataset that contains Holiday days such as , ,"Christmas","New year","17th of May - National day","Easter","Kr. ...
- 309
0
votes
1
answer
91
views
Correcting the output (strictly in terms the features/variables selected) of n LASSO Regressions on n datasets using glmnet
Note: This is a follow up to a previous question I have asked here on SO which received an answer that runs, but generates incorrect output. So, this question will incorporate the proposed answer to ...
- 173
0
votes
1
answer
41
views
LassoCV getting axis -1 is out of bounds for array of dimension 0 and other questions
Good evening to all,
I am trying to implement for the first time LassoCV with sklearn.
My code is as follows:
numeric_features = ['AGE_2019', 'Inhabitants'] categorical_features = ['familty_type','...
1
vote
1
answer
217
views
How to replicate my results from running n LASSOs iteratively using elastic net but now with glmnet
I have successfully run LASSO Regressions on each of n data sets within the 'datasets' list in my R Environment/Workspace using the enet function from the elastic net library in R, but just to ensure ...
- 173
0
votes
1
answer
221
views
How to get just the variables selected by LASSO Regressions ran using glmnet on each of k datasets
I am running 40 LASSO Regressions sequentially on each of the 40 datasets in a list object called 'datasets' in R:
datasets <- lapply(filepaths_list, fread)
# change column names of all the columns ...
- 173
-1
votes
1
answer
117
views
What is a simple to use library besides elastic net which can fit LASSO Regressions for output verification
I have run a series of LASSO Regressions in RStudio sequentially, one on each csv-formatted dataset within a file folder using the enet() function from the elastic net package in R with its lambda ...
- 173
0
votes
1
answer
54
views
How to convert a row of binary indicators into a row of distinct (nominal) labelled Variables selected by a LASSO
I have a set of candidate regressors (aka Independent Variables or 'IVs') which has been selected by a LASSO Regression I just ran in the following form:
True_IVs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 ...
- 173
-1
votes
1
answer
398
views
Lasso Regression
For the lasso (linear regression with L1 regularization) with a fixed value of λ, it is necessary to
use cross–validation to select the best optimization algorithm.
I know for a fact that we can use ...
- 25