This repository was archived by the owner on Feb 28, 2024. It is now read-only.
Repository navigation
This repository was archived by the owner on Feb 28, 2024. It is now read-only.
gp_minimize() produces ValueError while tuning xgboost classifier #1155
Description
I am trying to tune the hyperparameters for a xgboost classifier. The target is binary (0/1) and the training set is all numerical. When I am trying to run gp_minimize(), I am getting the following error: Provided transformers should be a Transformer instance. Got <skopt.space.transformers.Identity object. Below is my code (basically a verbatim copy of the example provided in the skopt documentation except I am trying out a xgboost classifier ) that I am trying to use:
scale_pos_weight = len(df_feats[df_feats.isPS==0])/len(df_feats[df_feats.isPS==1])
xgbc = XGBClassifier(scale_pos_weight=scale_pos_weight,
objective='binary:hinge')
space = [Integer(1, 20, name='max_depth'),
Integer(100, 1000,name='n_estimator'),
Real(10**-5, 10**0, "log-uniform", name='learning_rate'),
Real(0.5, 1,"uniform", name='subsample'),
Real(10**-5, 10**1, "uniform", name='gamma'),
Real(10**-5, 10**0, "uniform", name='alpha')]
# The decorator below enables the objective function
# to receive the parameters as keyword arguments.
@use_named_args(space)
def objective(**params):
'''
Scitkit Learn Optimize requires an objective function to minimize.
We use the average of cross-validation mean absolute errors as
the objective function (also called cost function in optimization)
'''
xgbc.set_params(**params)
return np.mean(cross_val_score(xgbc, xTrain_t.values, yTrain.values, cv=5, n_jobs=-1,
scoring="f1")) #"f1"
res_gp = gp_minimize(objective, space, n_calls=20, random_state=256)
Below is the traceback to help with the understanding of the issue I am facing:
Traceback (most recent call last):
Cell In[92], line 1
res_gp = gp_minimize(objective, space, n_calls=20, random_state=256)
File ~/opt/anaconda3/envs/datamonitor/lib/python3.10/site-packages/skopt/optimizer/gp.py:252 in gp_minimize
space = normalize_dimensions(dimensions)
File ~/opt/anaconda3/envs/datamonitor/lib/python3.10/site-packages/skopt/utils.py:599 in normalize_dimensions
dimension.set_transformer("normalize")
File ~/opt/anaconda3/envs/datamonitor/lib/python3.10/site-packages/skopt/space/space.py:493 in set_transformer
self.transformer = Pipeline(
File ~/opt/anaconda3/envs/datamonitor/lib/python3.10/site-packages/skopt/space/transformers.py:292 in __init__
raise ValueError(
ValueError: Provided transformers should be a Transformer instance. Got <skopt.space.transformers.Identity object at 0x7fe10044d090>
I couldn't figure out what is wrong and there isn't any stackoverflow topic on this particular issue.
Activity
jecjecj commentedon Jul 27, 2023
I am having the same issue!
AdamCoxson commentedon Dec 19, 2023
I had similar issues:
I had depreciated use of np.int and then the same AttributeError as you
In the file skopt/space/tranformers.py go to
line 275. Change np.int to int or int64 in
return np.round(X_orig).astype(int)line 290. To remove the check manually, I commented out this entire block:
Hopefully gp_minimize keeps ticking along with some development. All available high-level Bayesian Optimisers do the exact same thing so I don't really want to switch all my code.