Detecting Action Tendencies in Emails. A simple Neural Network machine learning model for text classifiction – specifically, training an email action item detector on a portion of the Enron Email Dataset.
The Enron email dataset contains approximately 500,000 emails generated by employees of the Enron Corporation. It was obtained by the Federal Energy Regulatory Commission during its investigation of Enron's collapse.Each email is labeled with several categories (e.g. legal advice, humor, internal company policy, etc.).
The task of action item detection is identifying sentences in an email that may require action on the part of the recipient. In practice, a system that can effectively identify action items could alert you to high-priority emails, or even automatically extract a todo list from your inbox.
- Tokenization
- Bag-Of-Words
- Put our model in training mode (this allows the weights to be updated)
For each example in our training set (batch by batch):
- Pass our input tensors through the model to get predicted labels
- Calculate the batch's loss (a measure of how poor the predictions were compared to our true labels)
- Calculate the batch accuracy (the percent of examples the model correctly labeled)
- Update the model's weights
- Keep track of the loss, training accuracy, and runtime for later
- Simple Neural Network
- The final test accuracy of the model is 93.07%
- Recall actually gives an estimate of how many of the Actual Positives our model capture through labeling it as Positive (True Positive). Applying the same understanding, we know that the model has a recall of 0.6177.
- Precision talks about how precise/accurate the model by determining how many of the predicted positives are actual positive, we know that the model has a precision of 0.8719.
- The model has a higher precision(87%) in classifying action items ("1",12.7%).
- The 61.8% recall suggests that it does better than average in classifying the action items in the whole test sample.