and a little interface for it. this is trying to spell the words using phonetic information (using a sequence-to-sequence neural network), the temperature parameter basically controls how the probabilities are distributed (at low temperatures, only the most likely characters are generated according to the information in the model; at higher temperatures, any character might be generated)
I need to stop playing with this, I have other stuff to do geez
still at work on this english nonsense word vae. here are some nonsense words sampled from the latent space of the latest trained model...
these are generated by feeding the decoder with normally-distributed random numbers. pretty happy with how they all seem like jabberwockian-yet-plausible english words
by contrast, results of feeding normally-distributed random numbers into the decoder on the RNN without the VAE:
not as good! which is encouraging, since it shows that the VAE model does actually have a "smoother" space than the non-VAE model.
(I have to admit that when I started this project I was like, "why do you even need a variational autoencoder, if just plugging random vectors into the decoder was good enough for jesus it's good enough for me," but there really is something magical and satisfying about being able to get more-or-less plausible generated results for basically any randomly sampled point in the distribution)
(posted this mainly because "butrick → jorserich" seems like something mastodon people would like, e.g. "my name is Butrick Jorserich, follow me at jeans.butrick.horse/@rich")
in which I accidentally leave off the "end" token when predicting spelling from sound, and it just keeps on spelling until it's ready to stop
remarkable → remarymarkamal
wysiwig → irzerwizkian
bemuse → bismebishews
unenforceable → unofironsfinars
shutters → shurtsithaters
capstick → capstickapsitk
vittoria → viltovitria
beilenson → billabinsancin
peers → pieespianes
paste → past-pasest
excitable → exexaitabile
phibro → fib-to-birbo
croney → crainkrine-y
tangle → tangitangle
"how doth the little crocodile improve his shining tail and pour the waters of the nile on every golden scale" → neural network spelling by sound but with probabilities of [aeiou] zeroed out → "hv d'th thy lyttl crch-dykly mpr h's shynnyng thyl hnyd ph thy whytrs f thy nyl hwhn avry ghqlynd schqly"
apparently the trick to training a VAE w/annealing is to *never* let the KL loss go below the reconstruction loss. otherwise you get beautifully distributed, wonderfully plausible reconstructions that have almost nothing to do with your training data, i.e., "allison" becomes
@aparrish Sounds like a distant branch of the Joestar family tree from JoJo's Bizarre Adventure
@aparrish As the Most Interesting Man in the World, I don't always wear jorts. But when I do, I wear Butrick Jorserich.
@aparrish evollove it!
Tangitangle was always my favourite pokemon
@aparrish reverb (and renoun, etc)
Ooh now I want a bot that runs posts through this
@aparrish this reads like how a super-low bitrate mp3 sounds
@flussence 100% exactly what I'm going for 😁
@aparrish I'd buy that as a lost snippet from Beowulf on first examination
@aparrish it snuck a single ‘a’ in there! Th mshjn rhblyhn bhgnz
@aparrish A lyttl Chaucerian English?
@aparrish what’s my name what’s my name
@courtney according to this, it's Che-cotort-an-ratily Statstaintangent
@aparrish thank you this is perfect
@aparrish weeeeelllll if somebody ain't FANCY
@aparrish oh, that's how I was already pronouncing it
@aparrish mama say mama sa mammasavinstawn
@aparrish Prof. Dodgson would have laughed at that. Try the program on Jabberwocky.
"I know how to spell banana I just don't know when to stop"
what hath the halting problem wrought?
@aparrish that jargon is Star-Trek-level!
I think you should adopt "houghtrodhan" as your secret identity when in Britain.
Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. This server is run by the main developers of the Mastodon project. Everyone is welcome as long as you follow our code of conduct!