Honestly, whoever has an idea for a spam detection measure for Mastodon, and by that I do mean an implementation, get in touch with me, I'll pay for it.
I've been thinking about solutions for the past few days but the more I think about them the more they appear pointless.
E-mail deals with spam using Bayesian filters or machine learning. The more training data there is, the more accurate the results, a monolith like GMail benefits from this greatly. Mastodon's decentralization means everyone has separate training data, and starts from scratch, which means high inaccuracy. It also means someone spamming a username could potentially lead to any mention of that username be considered spam due to the low overall volume of data, unless you strip usernames
However, if you strip usernames from the checked text, the spammer could write messages using usernames...
@Gargron do what WTDWTF does
there's no secret magic to it
users require a published post to edit their profile
users with zero or negative upvotes require mod approval to post
registering an account from an IP that is already associated with an account requires admin approval
about a month into this policy the spammers completely gave up
@ben We don't have a true emergency with spammers signing-up on a given instance. Approval-only registrations mode is a good tool for weeding those out. The problem we are experiencing is the spammer signing up on random open instances and sending spam remotely. Therefore, solutions based on IPs or captchas are not appropriate. Even if we release the perfect protection against local spammers, servers that haven't upgraded will continue to make this a problem.
@Gargron @ben We need to stop thinking about handling spam going out and start thinking about spam coming in, then. My instinct here is to read individual posts on their way in and handle spam detection at that level (likely on a separate lower-priority thread/task/whatever to prevent lagging out incoming posts).
@bclindner @Gargron @ben That imposes the cost on the victim of spam, which leads to an arms race. Better to try to impose the cost on the spammer.
Perhaps allow an instance to enable a setting that says if sending instance is n versions behind, reject messages?
Zombie instances would get gradually de-federated.
@daedalus That might help as an intermediate step but currently our problem exists with no real spam filtering existing on the Mastodon system whatsoever save for some rate limiting.
I'm honestly glad nobody's set up an auto-spammer script. We might be well and truly fucked if that happens before we can implement proper spam detection systems.
@Gargron I think one important aspect that has been pointed out by @ben is that users can be asked to classify the people they follow, and this can be used to compute some kind of credibility score for the new profiles, in order to limit their activity on a timescale at which modération is effective. I don't see any automated solution, tbh.
admins are responsible for the servers they run, and if those servers are the source of a disproportionate amount of spam, it doesn't matter whether the root cause is malice or simply inactivity from the admins. the end result is the same.
@gargron Surely a message containing tons of usernames and nothing else would be spam 99% of the time, though, so that doesn't sound like a problem for a Bayesian model.
@Gargron Honestly I'd pay to see someone do that, and then promptly ban them for it 😂
The more I think about email-like detection systems, the more I think as long as implementation is sound, it will help a lot with curbing common spam as the network grows and older instances and instances lots of users amass bigger datasets and higher confidence levels on spam detection.
Imperfect? Yeah. An arms race? Yeah. But it's a start.
@Gargron I can only assume this must be how the early adopters of email must have felt when the system started getting big.
@gargron would it be possible to provide some kind of built in trainable spam detector for Mastodon, and have an opt-in option to share data with a global pool of training data? that way instances could collaborate to fight spam
@Gargron I'm going to hazard a guess that > 90% of spammers aren't going to try to be clever.
An idea for spam containing links
@Gargron when I was working on online advertising, some of the ads we had in our inventory came from other big platforms, like Tubemogul. To know which brand to invoice for the ad display, we used the clickthrough (the url where the user is sent on click on the ad) to determine the domain of the url, after all the redirections.
You can use a similar system to list which domains are often shared in toot and blacklist them if the number of toot containing it is increasing at a specific time. Then, instance admins would receive a notification and could whitelist them if they want / if it's not spam.
@Gargron We don't need to start from scratch on each instance. Tools like rspamd and spamassassin come around with pre-trained sets.
That means we can make community efforts to build a repository of spam messages in order to pre-train filters.
And of course it's not super effective, but if we really want spam protection, we have to start somewhere.
@sheogorath @gargron I think Mastodon/ActivityPub also benefits from being able to build a reputation on instances, rather than individual users. Older instances, or at least ones that don't remove spambots when reported, can accrue a poor reputation and their incoming messages can be more highly scrutinized. And instance admins will be incentivized to decrease spam (the same way email sending servers are) so that their legitimate outgoing messages won't be ignored.
@gargron and in a way Mastodon has already been practicing for this and has already established mechanisms. Servers that don't moderate for harassment are silenced or eventually defederated by other instances, based on each instance's tolerance for dealing with an instance that doesn't sufficiently handle reports.
@Gargron Dealing with spam is hard. I worked at an email anti-spam company for 10 years. Bayesian was one method, but was troublesome and needed some hand-holding; we often cleared the bayesian data because it started to false-positive a lot. Most of our effective spam blocking came from: greylisting, DNS blacklists, a spam rules database that we added to based on the spam we saw, and users reporting messages (which we had tooling to analyse to then create more rules).
@Gargron what about distributed model training which are trained on each instance and then shared between them. Kinda like Google does with assistant on phones ...
not an implementation
@Gargron Maybe an idea to allow spam detector bots, and instances can indicate which ones it accepts.(including ones just run by the same org/person running the instance)
The bots could send toots giving probabilities, which might be shown in the UI, where users can respond, or cause spam to be removed if the certainty is high enough.(or get opinion of other spam detectors)
PMs/follower-only.. uh maybe it'll need a way to invite the spam bots.
@gargron you could federate the Maschine learning data too. But instances have to be at least x days seen by another instance/weighted by number/age of users to be accepted to prevent abuse of this.
Affinity groups can train reliable classifiers with a smaller corpus than groups without affinities, so message quantity isn't an obstacle. I could build a milter in about a week. Markov-based classifiers are a bit dodgy on short messages, so I'd have to test some other algorithms
Feature files can be transferable and non-reversible. An admin could use this data structure from one classifier to presort a training set for another, tuning for local requirements
@Gargron Why not just federate the training data?
@Gargron there's a protocol for federated learning. It's hard though.
Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!