Today we are going to create a slack integration to recognize toxic messages. We will use a public machine learning data model. The end result will be something like this:

Slack user posts a message. We analyze the message using machine learning algorithm and post back a message like this:

To accomplish this we will need to create a slack app. Configure to allow observing user actions. Then we will create our own "api" which will listen to slack events, process the messages using toxicity machine learning algorithm, and send a message to our channel. Now let's get started.

Create a slack app

Go to https://api.slack.com/apps and create an app. You wont need anything special except an active development slack workspace where to test out and install our app. As usual finding an appropriate name for your app might be the first challenge. We will call our app Toxicity Meter.

Create an api

Before we can use any of the web hooks or observe any of the features coming from Slack we need to set up something that is capable of listening. Calling this an api is a bit of a stretch but technically it qualifies. We will be creating a restful service using node js and express.

You can read about how to set it up here - https://techflarestudio.com/setup-a-simple-api-using-node-js-and-express/.

Configure web hooks and observers

To get this running we will need 2 things

Write our machine learning processing logic

First install the toxicity module and include it in your api handler. You can find the pretrained toxicity module here - https://github.com/tensorflow/tfjs-models/tree/master/toxicity .

Install it it your node project by running

npm install @tensorflow/tfjs @tensorflow-models/toxicity

Then include it in your api handler (in our example index.js)

var toxicity = require('@tensorflow-models/toxicity');

Next we need to create the logic that will take the content from the slack message and analyize the toxicity level. We do this by loading the module and passing the Slack request 'text' parameter into it. 

// The minimum prediction confidence. For testing we want to set this low
const threshold = 0.1;

toxicity.load(threshold).then(model => {
  const sentences = req.body.event.text;

  model.classify(sentences).then(predictions => {
	result = 'User: ' + req.body.event.user + '\n'; 
	for (var key in predictions) {
		if(predictions[key].results[0].match) {
			result += 'Detected ' + predictions[key].label.toUpperCase() + 
			' with a probability of ' + predictions[key].results[0].probabilities[1] + '\n';
		}
	}
  }
}

Run classify function and it will return the result in form of predictions. Each prediction returns the probability for the type analyzed, for example, 99% probability for the message containing insult. In our example we have simply concatenated the results in a single variable. What you do with the predictions is up to you.

In a real usage case you would also set the threshold higher and filter out the low probability results. Perhaps trigger some type of action only if we are fairly certain that something contains the type of rating.

Send a message

Once we have the results we want to do something with them. In this example we are going to send the response back to Slack channel. We can do this by sending a post request to incoming webhook. You can get the webhook in the Slack app configuration. 

request.post({
    url: 'https://hooks.slack.com/services/TTxxxxU8Z/BTxxxxB9/G29xxxxxxxxxxxPp7',
    json: { 
    	text: result
        },
    headers: {
    	'Content-Type': 'application/json'
        }},
    function (error, response, body) {
    	res.send(response);
    }
);

Define the webhooks url as our post target. Make sure to include the result as "text" parameter in the post body.

Summary

That is it for today. We have created a simple api that is capable of listening to Slack messages in any channel. Analyze the content using a pretrained machine learning model. Once the results are in we send back a message to Slack containing the result.

This is a fun example so let us know if you would like to explore it further. In future articles we will be exploring the machine learning algorithm in detail and see the possibilities of training our own model to recognize other patterns in text. Let us know if you have any ideas.