49 questions
4
votes
3
answers
100
views
How to define the search space for a simple equation optimization
I'm trying to learn skopt, but I'm struggling to get even a simple multivariate minimization to run.
import skopt
def black_box_function(some_x, some_y):
return -some_x + 2 - (some_y - 1) ** 2 + ...
- 226
2
votes
0
answers
254
views
Discrete Real dimension spacing in scikit-optimize
Let's say I am searching over a dimension:
from skopt import space
search_space = [
space.Real(1, 10, name="my_scale")
]
How can I make this Real number to be searched with discrete ...
- 85.1k
0
votes
1
answer
48
views
How does gbrt_minimize from scikit decide how many parameter splits to try
From my understanding of https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html and gradient boosting decision trees in general, I assume that for N ...
- 97
0
votes
1
answer
72
views
skopt.gbrt_minimize (gradient boosted regression trees) - how does it calculate the gradient of the loss function if it is not differentiable?
I am optimising 11 parameters for a Jansen & Rit whole-brain model as follows:
search_space = [
Integer(1, 10, name="A"), Integer(10, 40, name="B"), Integer(90, 200, name="...
- 97
0
votes
0
answers
147
views
when to use feature selection when doing hyperparameter optimization?
I am using skopt(scikit-optimize) to find best hyperparameters for random forest model. I have a lot of features. To avoid overfitting, I'd like to add feature selection like using RFE. But I am not ...
- 4,338
8
votes
4
answers
9k
views
How to fix the 'numpy.int' attribute error when using skopt.BayesSearchCV in scikit-learn?
When I run the following code on the official documentation, it has an error.
Minimal example
from skopt import BayesSearchCV
from sklearn.datasets import load_digits
from sklearn.svm import SVC
from ...
- 81
2
votes
0
answers
633
views
Skorch NeuralNetRegressor and GridSearchCV - Custom Parameters
I have the following model defined, that I would like to apply Hyperparameter tuning to. I want to use GridSearchCV and change the number of layers etc.
class Regressor(nn.Module):
def __init__(...
- 430
2
votes
1
answer
105
views
Train and test data setup for sklearn
I'm creating a classification model to predict the outcome of sports event(win/loss) and am running into a data setup conundrum.
Currently the data is setup as follows:
example_data = [team_a_feat_1, ...
1
vote
1
answer
214
views
skopt's gp_minimize() raises a ValueError: Provided transformers should be a Transformer instance
I am new to bayesian optimization module (skopt). I was playing around with their toy example but encountered the following error while executing gp_minimize() function. Below is my code:
res_gp = ...
- 1,894
0
votes
3
answers
538
views
TypeError: predict() got an unexpected keyword argument 'return_mean_grad'
Using scikit-optimize and scikit-learn, I get the following error on calling the gp_minimize function:
TypeError: predict() got an unexpected keyword argument
'return_mean_grad'
I think it may be ...
1
vote
0
answers
296
views
Subclass EarlyStopper in scikit-optimize
I can't figure out how to subclass EarlyStopper to use it as callback in scikit-optimize (gp_minimize). Based on the documentation. How should I think when subclassinging?
Documentation: https://...
- 1,255
1
vote
1
answer
366
views
Output from noisy optimization in skopt.gp_minimize
When using skopt.gp_minimize on a noisy dataset with unknown variance - is the returned minimum the x-values found for one specific sample of data or the minimum of the surrogate function? And either ...
- 31
2
votes
1
answer
1k
views
How to run Scikit's gp_minimize in parallel?
I am unable to make skopt.gp_minimize run at multiple cores. According to the documentation, the parameter n_jobs should set the number of cores. However, setting n_cores>1 seems to have no effect. ...
- 2,842
0
votes
2
answers
787
views
Fitting a model with some known parameters to an experimental dataset in python, in order to optimise other parameters
I have an experimental dataset 1 which plots intensity as a function of energy. These are arrays of 1800 datapoints.
I have been trying to fit a model to this data, given by the equation below:
...
- 1
3
votes
1
answer
535
views
How to restart BayesSearchCV from a checkpoint
I am performing hyperparameter search over a large space using skopt.BayesSearchCV. I am running it on a machine that restarts at 7pm every day. I want to be able to save the state of the ...
- 271
0
votes
1
answer
476
views
BayesSearchCV of LGBMregressor: how to weight samples in both training and CV scoring?
While optimizing LightGBM hyperparameters, I'd like to individually weight samples during both training and CV scoring. From the BayesSearchCV docs, it seems that a way to do that could be to insert a ...
- 168
0
votes
4
answers
2k
views
Fitting a curve to some datapoints
the fitted curve doesn't fit the datapoints (xH_data, nH_data) as expected. Does someone know what might be the issue here?
from scipy.optimize import curve_fit
import numpy as np
import matplotlib
...
0
votes
1
answer
2k
views
Scoring parameter in BayesSearchCV class confusion
I'm using BayesSearchCV from scikit-optimize to train a model on a fairly imbalanced dataset. From what I'm reading precision or ROC AUC would be the best metrics for imbalanced dataset. In my code:
...
- 135
1
vote
0
answers
401
views
Can the ask function in scikit-optimize be parallelized?
I'm using the skopt (scikit-optimize) package, using the ask-tell syntax. I'm using python 3.7, on a windows machine The ask function call takes a long time (first call ~1 minute, then increases 1 ...
- 65
0
votes
0
answers
780
views
How to use scikit-optimize to change the hidden layers in MLPRegressor?
I am attempting to use scikit-optimize to tune the hyperparameters of a scikit-learn multi-layer perception regressor (MLPRegressor). However, the problem is that scikit-optimize seems to convert the ...
- 327
-1
votes
1
answer
6k
views
TypeError: __init__() got an unexpected keyword argument 'iid' while running GridSearchCV for time series data
I have a data Gemini_ETHUSD_d.csv which you can download from this link
I try to re-run the code below from this link:
from sklearn.model_selection import TimeSeriesSplit
from sklearn.model_selection ...
- 10.1k
0
votes
1
answer
117
views
what does model_queue_size do?
General question:
Using scikit-optimize for a black box optimization. Can't find in the doc what model_queue_size does. I'm doing the ask-tell because I can parallelize the calculation of y as ...
- 65
0
votes
1
answer
789
views
Getting a negative R-squared value with curve_fit()
I've read a related post on manually calculating R-squared values after using scipy.optimize.curve_fit(). However, they calculate an R-squared value when their function follows the power-law (f(x) = a*...
- 65
0
votes
1
answer
755
views
How do I extract the best features when using BayesSearchCV?
When using BayesSearchCV from Skopt for Feature Selection, how can I extract the feature names of the optimal set of features? I only know how to get the number of features together with other ...
- 11
0
votes
1
answer
1k
views
Black box optimization with Scikit Optimize
I have to optimize a black-box problem that depends on external software (no function definition neither derivatives) that is quite expensive to evaluate. It depends on several variables, some of them ...
- 107
1
vote
0
answers
513
views
Pipeline that scales, then tunes hyperparameters in a nested RFECV. What am I doing wrong?
I'm trying to build a basic ML pipeline that will select features while tuning hyper parameters at the same time.
The code is below.
#pipeline for full feature selection - hyperparametertuning
...
- 19
5
votes
1
answer
2k
views
XGBoost and scikit-optimize: BayesSearchCV and XGBRegressor are incompatible - why?
I have a very large dataset (7 million rows, 54 features) that I would like to fit a regression model to using XGBoost. To train the best possible model, I want to use BayesSearchCV from scikit-...
0
votes
1
answer
767
views
Scikit-learn - how to use single static validation set for CV object? [duplicate]
In Scikit-learn RandomSearchCV and GridSearchCV require the cross validation object for the cv argument, e.g. GroupKFold or any other CV splitter from sklearn.model_selection.
However, how can I use ...
- 1,599
0
votes
1
answer
1k
views
skopt: How to dynamically change bounds during optimization?
I have just started using skopt so please feel free to redirect me to any basic tutorial that I might have missed. Anyway, here it goes:
I have an optimization problem where I calculate a 2-peaks ...
- 89
2
votes
1
answer
2k
views
How are the test scores in cv_results_ and best_score_ calculated in scikit-optimize?
I'm using BayesSearchCV from scikit-optimize to optimise an XGBoost model to fit some data I have. While the model fits fine, I am puzzled by the scores provided in the diagnostic information and am ...
4
votes
2
answers
2k
views
what is the kappa variable (BayesianOptimization)
I read some posts and tutorials about BayesianOptimization and I never saw explanation about kappa variable.
What is the kappa variable ?
How can it help us ?
How this values can influence the ...
- 1,355
0
votes
1
answer
2k
views
BayesSearchCV parameters
I just read about Bayesian optimization and I want to try it.
I installed scikit-optimize and checked the API, and I'm confused:
I read that Bayesian optimization starts with some initialize samples.
...
- 1,355
1
vote
0
answers
1k
views
RuntimeError: can't start new thread
My objective was to use Scikit-Optimize library in python to minimize the function value in order to find the optimized parameters for xgboost model. The process involve running the model with ...
- 31
4
votes
2
answers
5k
views
lightgbm.basic.LightGBMError: Check failed: (best_split_info.left_count) > (0)
There's a couple of other questions similar to this, but I couldn't find a solution which seems to fit. I am using LightGBM with Scikit-Optimize BayesSearchCV.
full_pipeline = skl.Pipeline(steps=[('...
- 231
0
votes
1
answer
877
views
Using @use_named_args from Scikit Optimize
I'm having a problem on using @use_named_args from Scikit Optimize. The problem is that my objective function accepts the arguments NamedTuple and I can't change this because this is the requirement ...
- 1,035
0
votes
1
answer
1k
views
How to use scikit-learn optimize in a class (especially the use_named_args decorator)?
I am using the scikit-learn optimize package to tune the hyperparameters of my model. For performance and readability reasons (I am training several models with the same process), I want to structure ...
1
vote
1
answer
1k
views
BayesSearchCV ValueError: All integer values shouldbe greater than 0.000000
I'm trying to tune an xgboost model with BayesSearchCV for multiclass classification. Here's my code.
n_iterations = 50
estimator = xgb.XGBClassifier(
n_jobs=-1,
objective="multi:softmax&...
- 612
0
votes
0
answers
268
views
Can changing constraints be used with the scikit-optimize API?
The canonical use case for scikit-optimize is an optimization objective given a fixed set of hyperparameters, where skopt is given full control to explore the space. However, one may wish to ...
- 763
1
vote
2
answers
3k
views
Error when running any BayesSearchCV Function for randomforest classifier
I am trying to use an RF classifier but every time i try to run the bayessearchCV function, i get an error returned. Attached is my specific example and an example that you can run and reproduce.
I ...
- 153
3
votes
4
answers
1k
views
TypeError inside the `scikit-optimize` package
When I use scikit-optimize version 0.7.4 to optimize a scikit-learn 0.23 model:
rf = BayesSearchCV(
RandomForestClassifier(
min_samples_leaf=0.01, oob_score=True
), {
...
- 58
1
vote
1
answer
476
views
Adding 0 to log-uniform parameter distribution in BayesSearchCV
I'm using scikit-optimize to do a BayesSearchCV within my RandomForestClassifier hyperparameter space. One hyperparameter is supposed to also be 0 (zero) while having a log-uniform distribution:
...
- 53
0
votes
1
answer
5k
views
No module named 'sklearn.externals.joblib' ERROR
I am trying to implement a Bayesian optimization but already starting having an error while importing 'gp_minimize' from scikit-optimize package
from skopt import gp_minimize
res = gp_minimize(f, ...
1
vote
1
answer
1k
views
Resume a gp_minimize process from a checkpoint in Skopt
I want to resume a gp process but I am getting strange messages. I start my gp process providing only x0 and y0=None. My initial points are 30 and n_evals = 50. I stop it at evaluation no 20. Then I ...
- 11
2
votes
0
answers
417
views
Running multiprocessing Pool.map multiple times in one program ends up blocking
I am trying to optimize a function that is relatively expensive to evaluate. The function operates across a series of data points, and can be evaluated in parallel. Each data point evaluation requires ...
- 21
1
vote
1
answer
1k
views
Example of using a KerasRegressor in scikit-optimize
I am using the awesome scikit-optimize toolbox for hyperparameter optimization.
My goal is to compare keras and scikit-learn models.
According to the example https://scikit-optimize.github.io/stable/...
- 331
1
vote
1
answer
480
views
ValueError when using Bayesian Optimization over a scikit GP model
I am utilizing a Gaussian process regression with 4 inputs and one output. The goal is to find the optimal X by performing a bayesian optimization over the fitted model.
I fit the model with the ...
- 151
0
votes
1
answer
362
views
How can you merge several search result files (pkl) to plot them all together?
I have used a couple of computers to run the same script and now have several result_files.pkl.
results1 = gp_minimize(func=fitness,
dimensions=dimensions,
...
- 65
1
vote
1
answer
445
views
Resume gaussian process from checkpoint in skopt
I would like to be able to resume a gaussian process from a checkpoint with the library skopt. After some research I couldn't find a way to do so.
Here is a simple code to show what I want to do:
...
- 8,654
0
votes
0
answers
420
views
var_data = kwargs[var] error - lmfit Model
I am trying to dertmain a coeffecient by fitting experimental data using lmfit.Model.
When I run my code I get the following error:
Traceback (most recent call last):
File "C:\***.py", line 82, ...
- 13