Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
The original server operated by the Mastodon gGmbH non-profit

Administered by:

Server stats:

329K
active users

mastodon.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.0-nightly.2025-04-01

#multi

3 posts2 participants0 posts today
Public
N-gated Hacker News

Oh joy, yet another paper with a title that sounds like a new 🤦‍♂️. Thank goodness for the Foundation's donations, because without them, we'd clearly be lost in a sea of gobbledygook 🤷‍♀️. Can't wait to "multi-token" my way to - or was it ennui? 📚🔍
arxiv.org/abs/2504.00927 -token

arXiv logo
arXiv.orgMulti-Token AttentionSoft attention is a critical mechanism powering LLMs to locate relevant parts within a given context. However, individual attention weights are determined by the similarity of only a single query and key token vector. This "single token attention" bottlenecks the amount of information used in distinguishing a relevant part from the rest of the context. To address this issue, we propose a new attention method, Multi-Token Attention (MTA), which allows LLMs to condition their attention weights on multiple query and key vectors simultaneously. This is achieved by applying convolution operations over queries, keys and heads, allowing nearby queries and keys to affect each other's attention weights for more precise attention. As a result, our method can locate relevant context using richer, more nuanced information that can exceed a single vector's capacity. Through extensive evaluations, we demonstrate that MTA achieves enhanced performance on a range of popular benchmarks. Notably, it outperforms Transformer baseline models on standard language modeling tasks, and on tasks that require searching for information within long contexts, where our method's ability to leverage richer information proves particularly beneficial.
Public
Hacker News

Multi-Token Attention

arxiv.org/abs/2504.00927

-Token

arXiv logo
arXiv.orgMulti-Token AttentionSoft attention is a critical mechanism powering LLMs to locate relevant parts within a given context. However, individual attention weights are determined by the similarity of only a single query and key token vector. This "single token attention" bottlenecks the amount of information used in distinguishing a relevant part from the rest of the context. To address this issue, we propose a new attention method, Multi-Token Attention (MTA), which allows LLMs to condition their attention weights on multiple query and key vectors simultaneously. This is achieved by applying convolution operations over queries, keys and heads, allowing nearby queries and keys to affect each other's attention weights for more precise attention. As a result, our method can locate relevant context using richer, more nuanced information that can exceed a single vector's capacity. Through extensive evaluations, we demonstrate that MTA achieves enhanced performance on a range of popular benchmarks. Notably, it outperforms Transformer baseline models on standard language modeling tasks, and on tasks that require searching for information within long contexts, where our method's ability to leverage richer information proves particularly beneficial.
Public
gee Ⓐ⚑

La dernière mise à jour de « Sales Gosses ! » vous permet de sauvegarder des skins multiples, de choisir différents modes de jeu… et vous offre une jolie atmosphère de printemps, avec ses papillons et ses décos de Pâques !

studios.ptilouk.net/sales-goss

#SalesGosses #JeuxVidéo #JeuVidéo #Miseàjour #Casual #Fun #PVP #jeu #multi
studios.ptilouk.net/sales-goss

Sales Gosses !Sales Gosses !
More from Ptilouk.net Studios
Public
Journal of Plant Ecology

is essential for early seedling establishment, and varies under different environmental conditions. Hao Wu et al. explored of -elements in seeds of and its association with , , and . Results showed that seed elements were highly variable across different populations, with macroelements exhibiting lower variability than most of the microelements.
Details: doi.org/10.1093/jpe/rtae090

Relationship and comparison of seed and leaf elements (N, P and N:P) for the 18 natural populations of Euptelea pleiospermum in China.
ExploreLive feeds
About