The ADL Anti Semitism Expert
Last Updated On: 2025-09-01 04:31:53 -0400
NOTE: As with much of the OHI documentation, read this only if you expect foul, disturbing language.
This document talks about current problems with the existing expert_anti_semitism pipeline component.
There are the following problems with Grant’s anti semitism model:
- Incorrect training data leading to false positives
- Technically correct training data leading to false positives
- Performance and Scalability
Each of these is talked about below.
Overview: Adding Algorithmic Intelligence Alongside a Machine Learning Model
Right now our expert_anti_semitism model is fairly simple:
- It looks at inputs based on their relationship to the training data and renders a judgement
My overall suggestion is that we start to enhance its operation by adding algorithmic intelligence to it as a front end to the machine learning model itself. I would personally implement this as a pre-processor that runs before we dispatch content to the machine learning model and, when the pre-processor succeeds then I would output UNCLASSIFIABLE as the judgement and I would then batch these results into a special S3 bucket where they can be reviewed.
It is important to note that this can all be easily implemented in expert_anti_semitism. It does not require any changes to Grant’s work.
Incorrect training data leading to false positives
I recently looked at the output of the expert_anti_semitism pipeline component and I found this:
{ “bucket”:1591822800020, “sequence”:1591822817738900093, “sequence_range”:70420, “hashcode”:”00C3WdIc6onMd2RUyhctdyfcG5Q”, “resource”:”https:\/\/twitter.com\/lalisamanv\/status\/1270685484600049665”, “date_found”:”2020-06-10T21:00:17Z”, “index_method”:”TWITTER_TASK”, “version”:”6.5.338”, “last_updated”:”2020-06-10T21:00:17Z”, “source_hashcode”:”hW1b064pS3GJirjXupdZp3zOlvE”, “source_resource”:”https:\/\/twitter.com\/lalisamanv”, “source_link”:”https:\/\/twitter.com\/lalisamanv”, “source_publisher_type”:”MICROBLOG”, “source_publisher_subtype”:”twitter”, “source_date_found”:”2020-05-21T00:58:50Z”, “source_update_interval”:0, “source_http_status”:0, “source_spam_probability”:0.0, “source_content_length”:0, “source_setting_update_strategy”:”INDIRECT”, “source_setting_index_strategy”:”DEFAULT”, “source_setting_author_policy”:”DEFAULT”, “source_user_interactions”:0, “source_setting_minimum_content_metadata_score”:0, “source_title”:”#WeWantRos\u00e9Solo”, “source_description”:”#WeWantRos\u00e9Solo”, “source_favorites”:0, “source_followers”:0, “source_following”:0, “source_verified”:false, “source_image_width”:0, “source_image_height”:0, “source_favicon_width”:0, “source_favicon_height”:0, “source_likes”:0, “source_parsed_posts”:0, “source_parsed_posts_max”:0, “permalink”:”https:\/\/www.twitter.com\/lalisamanv\/status\/1270685484600049665”, “identifier”:”1270685484600049665”, “canonical”:”https:\/\/www.twitter.com\/lalisamanv\/status\/1270685484600049665”, “domain”:”twitter.com”, “site”:”twitter.com”, “main”:”exactly”, “main_length”:7, “main_checksum”:”-jYdxplMtPRBMNQqVhYn5yG8Svg”, “main_authoritative”:true, “main_format”:”TEXT”, “published”:”2020-06-10T11:53:22Z”, “author_name”:”lalisa”, “author_link”:”https:\/\/twitter.com\/lalisamanv”, “author_handle”:”lalisamanv”, “author_avatar_img”:”https:\/\/pbs.twimg.com\/profile_images\/1269489155739795457\/rCz4BtH2_bigger.jpg”, “author_user_id”:”1187686328655155201”, “author_gender”:”UNKNOWN”, “shared_type”:”NONE”, “replied”:false, “type”:”POST”, “sentiment”:”NEUTRAL”, “lang”:”en”, “likes”:0, “metadata_score”:304, “shares”:0, “normalized”:{ “author”:{ “fid”:”1187686328655155201”, “username”:”lalisamanv”, “email”:null, “submitter”:”#WeWantRos\u00e9Solo”, “display_name”:”lalisa”, “avatar”:”https:\/\/pbs.twimg.com\/profile_images\/1269489155739795457\/rCz4BtH2_bigger.jpg”, “url”:”https:\/\/twitter.com\/lalisamanv”, “description”:”#WeWantRos\u00e9Solo”, “verified”:false, “gender”:”UNKNOWN”, “followers_count”:0, “following_count”:0, “statuses_count”:null, “likes_count”:0, “favorites_count”:0, “shares_count”:0 }, “post”:{ “title”:”exactly”, “uuid”:””, “language”:””, “fid”:”1270685484600049665”, “parent_fid”:””, “views”:””, “permalink”:”https:\/\/www.twitter.com\/lalisamanv\/status\/1270685484600049665”, “body_length”:””, “body”:”exactly”, “body_html”:””, “created_utc”:”2020-06-10T21:00:17Z”, “score”:””, “reblogs_count”:””, “replies_count”:””, “images”:[
],
"spam_probability":null,
"retrieved_at":"2020-06-24T18:35:27.033505",
"lang":"en",
"s3_fid":"00427f26b66b11ea9915acde48001122"
},
"thread":{
"name":"",
"fid":"",
"type":"",
"in_reply_to_author_fid":"",
"in_reply_to_fid":""
},
"links":[
],
"expanded_links":[
],
"medias":[
],
"mentions":[
],
"experts":{
"expert_invective_anti_semitism":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_anti_semitism"
},
"expert_invective_anti_black":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_anti_black"
},
"expert_invective_anti_islam":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_anti_islam"
},
"expert_invective_anti_woman":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_anti_woman"
},
"expert_invective_anti_lgbtq":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_anti_lgbtq"
},
"expert_invective_trump":{
"matches":[
],
"score":0.0,
"expert_type":"invective",
"expert_name":"expert_invective_trump"
},
"sentiment_expert":{
"matches":null,
"score":{
"Positive":0.2850418985,
"Negative":0.0888516158,
"Neutral":0.6226921082,
"Mixed":0.003414392
},
"judgement":"NEUTRAL",
"expert_type":"sentiment",
"expert_name":"sentiment_expert"
},
"anti_semitism_expert":{
"matches":null,
"score":[
-0.0418760702,
0.1496950686
],
"judgement":1,
"expert_type":"anti_semitism_adl",
"expert_name":"anti_semitism_expert"
}
} } } And, yes, that's a single tweet composed of nothing but the word "exactly" and, yet, the anti_semitism_expert blithely pronounced it as:
"anti_semitism_expert":{
"matches":null,
"score":[
-0.0418760702,
0.1496950686
],
"judgement":1,
"expert_type":"anti_semitism_adl",
"expert_name":"anti_semitism_expert"
}
I brought this up to Grant and he attributed it to incorrect training data where, perhaps, the tweet in question was a response to something that was anti_semitic. That may be the reason but we still have the issue of our anti_semitism classifier labeling a single, innocuous word as anti_semitic.
Proposed Solution
I would add to the anti_semitism expert a preprocessor which calculated an information complexity score for the content (perhaps Flesch-Kincaid; in the context of say more than 3 words) and then only bother to run the expert_anti_semitism pipeline component if the score is high enough.
I would also argue that if the word count of a tweet is:
- 1 -or-
- A tweet is just emojis
that we NOT run the expert_anti_semitism routine entirely and instead add a judgement for expert_anti_semitism of UNCLASSIFIABLE
Technically correct training data leading to false positives
When we first started on development, back when we had Grant’s twitter tool, I noticed that:
I hate jews
was properly recognized as anti_semitic but so too was:
I hate niggers
The reason for this is that the string “I hate niggers” has only 3 tokens and there is such a preponderance of things labeled as “I hate xxx” related to anti_semitism that the noun value “niggers” doesn’t correspond to anything about anti_semitism is simply lost.
Proposed Solution
This problem has been known about for ages but with the expert_anti_semitism pipeline component now actually being used, a possible solution would be to write a specific algorithm for the “I hate” case which looked at the overall content and assessed its “jewishness”. We could do this by simply feeding the content through the invective_anti_semitism expert and looking for any terms that indicate content of a jewish nature (or feed it through all the invective experts except anti_semitism and look for hits). And then if we don’t find content that is jewish (or only find content that isn’t) then we label this content as UNCLASSIFIABLE.
Performance and Scalability
We now understand our performance numbers pretty well:
- 500 million tweets per day
- 40% of those are english so 200 million tweets are what we have to be able to process per day
- Right now we taking 100 ms or so to run a single tweet through the expert_anti_semitism code
Example timing info:
free_bytes_in_sqs
256054
aws_common.add_to_sqs -- free_bytes_in_sqs + length_of_njd
start size of njd: 72
classification_time 0:00:00.101995
start_size_of_njd: 8084
. end_size_of_njd: 8314
unicode error; oops
8314
free_bytes_in_sqs
100 ms, according to Grant, is right on the money:
100 ms sounds like it’s in the right range for all the work, between moving things between cpu/gpu and running the classification. I expect that the actual call to result = model(inputs…) should take between 10-20 ms on average, proportional to the size of the text.
If we have 200 million tweets and we know that it takes .1 seconds per tweet then we need 20 million seconds (200,000,000 * .1) to process all the tweets for a given day. At 86,400 seconds per day that means we have a compute time of 200,000,000 / 86400 or 2,314.8 days to classify all tweets for a given day.
Obviously we can apply multiple boxes and / or parallelism to scale down 2,314 days to a single day but we may be facing a simply intractable problem of using Bert at this scale.
Back in February Grant and I explored (and it was almost all Grant) applying an SVM style model instead of Bert. SVM has the advantage of being both faster and smaller. Perhaps we need to look at SVM again.