Follow

Started work on a PR that will add full-text search for a certain subset of toots (e.g. your own) github.com/tootsuite/mastodon/

Still WIP because need to figure out how to introduce new database (elasticsearch) in a non-distruptive manner for poor sysadmins, and the syntax for the actual search might need some adjustment.

@Gargron Did you look at postgresql's capabilities for full-text search?

@val You are using postgres full-text search every time you get a username autocomplete on Mastodon. However the massive corpus of 500 character messages will seriously affect postgres both in performance and disk usage. By outsourcing this functionality to a secondary system, we're making sure this functionality will not negatively impact critical, core functionality of Mastodon.

@Gargron Good point.
Would it work to move the corpus in another postgresql database?

@val @gargron postgres doesn't do "fuzzy" n-gram matching though, which feels very different from what people have come to expect from full-text search

@Gargron I've run an elasticsearch cluster in the past and I would be really hesitant to introduce it. You need at least 3 instances for sharding/recovery to work and probably not less than 24GB of RAM.

@Gargron Literally was just wishing for this one hour ago, thank you! Especially for favorites, perfect.

@Gargron make elasticsearch an optional extra, it's huge and hard to manage :-/ for queries, i'd suggest just using the "simple query string". Worked on this a bit at GL, shout if you want eyes!

Sign in to participate in the conversation
Mastodon

Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. This server is run by the main developers of the Mastodon project. Everyone is welcome as long as you follow our code of conduct!