Convert pandas data frame to categorical for keras

Question

I am trying to preprocess data in python for use in deep learning keras functions.

I use categorical crossentropy as loss function in model fit. It requires categorical variable as target.

My target data sample:

    y_train = y_train.astype('category')
    y_train.head()

When I tried to convert data frame column to categorical:

    num_classes=2
    y_train = keras.utils.to_categorical(y_train, num_classes)

It produced an error: IndexError: index 1 is out of bounds for axis 1 with size 1.

How do I convert the data properly?

By the way, which keras models are better for binary classification (yes, no) if I have sample of 3800 observations with 2300 numeric (float32) features each? The features describe mostly graphical objects.

Ioannis Nasios · Accepted Answer · 2019-04-03 08:48:50Z

1

Unfortunately i didn't manage to reproduce your error. Running:

a=pd.DataFrame(np.concatenate([np.zeros(3),np.ones(3)]) ).astype('int').astype('category')
from keras.utils import to_categorical
to_categorical(a, 2)

I get an output:

array([[1., 0.],
       [1., 0.],
       [1., 0.],
       [0., 1.],
       [0., 1.],
       [0., 1.]], dtype=float32)

Maybe it's a versioning issue!

The good news are that you don' t have to use categorical_crossentropy for a binary classification problem. You can can use binary_crossentropy loss and feed you model with your y_train as a target as it is.

Regarding you last request about which keras model is better for binary classification, Keras pre-trained models are referring to images. You seem to have tabular data, though you wan't be able to use a pre-trained model but you will have to run a custom model on your own.

answered Apr 3, 2019 at 8:48

Ioannis Nasios

8,5474 gold badges41 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alexander Borochkin Over a year ago

My mistake was that there were 3 category in data not two. I converted this column appropriately.

Collectives™ on Stack Overflow

Convert pandas data frame to categorical for keras

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related