•1 min read•from Machine Learning
[R] Joint Embedding Variational Bayes (TMLR ’26)
Our take
I’m excited to share my recent paper, "Joint Embedding Variational Bayes," published in TMLR '26. This work introduces operational variational semantics into joint-embedding architectures for non-contrastive representation learning. We present three key innovations: factorizing embedding likelihood into directional and radial components, anchoring posterior uncertainty to likelihood scale, and employing a heavy-tailed Student-t likelihood. These advancements enable the model to effectively learn anisotropic uncertainty, which we evaluate through OOD detection experiments, including comparisons with VI-SimSiam. I invite you to explore the full details in
Disclosure: first author.
The paper was just published in TMLR, and I figured it might be of interest to some people here. It is fairly dense mathematically, but straightforward conceptually: to add operational variational semantics to joint-embedding architectures for non-contrastive representation learning, we make three coupled choices:
- Factorize embedding likelihood: the likelihood is split into directional and radial terms, so angular alignment and representation norm are modelled separately. The radial/norm term does not drive accuracy on its own, but the factorization avoids the norm-direction coupling that otherwise produces pathological solutions.
- Anchor posterior/likelihood uncertainty: the posterior variance is tied to the likelihood scale, so uncertainty directly governs both inference and the embedding likelihood.
- Use heavy-tailed likelihood: the likelihood uses a Student-t form rather than Gaussian. This matters empirically, since as the likelihood approaches the Gaussian limit, training becomes unstable and the model fails catastrophically.
These allow the model to learn anisotropic / feature-wise uncertainty, which is evaluated in a downstream OOD detection experiments, including against VI-SimSiam.
arXiv | OpenReview | Code
[link] [comments]
Read on the original site
Open the publisher's page for the full experience
Tagged with
#no-code spreadsheet solutions#rows.com#natural language processing for spreadsheets#machine learning in spreadsheet applications#generative AI for data analysis#enterprise-level spreadsheet solutions#automated anomaly detection#AI-driven spreadsheet solutions#Excel alternatives for data analysis#data cleaning solutions#Joint Embedding#Variational Bayes#non-contrastive representation learning#heavy-tailed likelihood#embedding likelihood#posterior variance#Student-t distribution#anisotropic uncertainty#representation norm#likelihood scale