Slow pytorch prediction when running inside slackclient #492
Description
Description
Trying to get NLP predictions from a pytorch model is really slow when called within my slack.RTMClient.run_on function wraper. I still get the expected result, but in 10x the time. This is not a GPU issue since everything is running on CPU.
What type of issue is this? (place an x in one of the [ ])
- bug
- enhancement (feature request)
- question
- documentation related
- testing related
- discussion
Requirements (place an x in each of the [ ])
- I've read and understood the Contributing guidelines and have done my best effort to follow them.
- I've read and agree to the Code of Conduct.
- I've searched for any related issues and avoided creating a duplicate issue.
Bug Report
Filling out the following details about bugs will help us solve your issue sooner.
Reproducible in:
slackclient version: 2.1.0
python version: 3.6.8
OS version(s): Windows 7
Steps to reproduce:
- Create a slack bot and Load a pytorch model
- Call .predict outside of the slackclient (runs normally)
- Call .predict inside of the slackclient (runs much more slowly)
Expected result:
The .predict to take the same time outside and inside of the slackclient
Actual result:
Takes 10x the time to get a prediction
Attachments:
Here is the code I am using :
import os
import json
import time
import slack
from fastai import *
from fastai.text import *
import numpy as np
class FrenchClassifier:
def __init__(self, num_of_words=80, temperature=0.1):
self.mdl_path = Path('models')
self.language_model = load_learner(path=self.mdl_path, file='prediction.pkl')
self.classifier = load_learner(path=self.mdl_path, file='classification.pkl')
self.num_of_words = num_of_words
self.temperature = temperature
def classify(self, text):
response_tensor = self.classifier.predict(text)
response_dict = {
'category': str(response_tensor[0]),
'conf': np.amax(response_tensor[2].numpy())
}
return response_dict
def complete(self, text ):
response = self.language_model.predict(text, self.num_of_words, temperature=self.temperature)
return response
@slack.RTMClient.run_on(event='message')
def read_text(**payload):
data = payload['data']
if 'client_msg_id' in data.keys():
if '!polarity ' in data['text']:
channel_id = data['channel']
text_input = data['text'].replace('!polarity ', '')
response_dict = fc.classify(text_input)
response = f'Catégorie {response_dict["category"]} avec {round(response_dict["conf"]*100, 2)}% de confiance'
webclient = payload['web_client']
webclient.chat_postMessage(
channel=channel_id,
text=response
)
elif '!complete ' in data['text']:
channel_id = data['channel']
text_input = data['text'].replace('!complete ', '')
response = fc.complete(text_input)
webclient = payload['web_client']
webclient.chat_postMessage(
channel=channel_id,
text=response
)
if __name__ == '__main__':
with open('slackToken.json', "r") as slack_token:
creds = json.load(slack_token)
fc = FrenchClassifier()
slack_token = creds["botAccessToken"]
rtmclient = slack.RTMClient(token=slack_token)
rtmclient.start()