Commission B3 Astroinformatics and Astrostatistics



  • François Lanusse (CEA/CNRS, France)

    A New Era of Multi-Modal Self-Supervised Learning for Astrophysics

    Abstract: Deep Learning has seen a recent shift in paradigm, from training specialized models on dedicated datasets, to so-called Foundation Models, trained in a self-supervised manner on vast amounts of data and then adapted to solve specific tasks with state-of-the-art performance. This new paradigm has been exceptionally successful not only for large language models (LLMs) but in other domains such as vision models. However applications of this new approach in astrophysics are still very scarce, for reasons ranging from new architectures to the (surprising) lack of availability of suitable large scale datasets. In this talk, I will discuss our recent work on deploying such a Foundation Model approach in the context of representation learning for astronomical photometric and spectroscopic observations of galaxies. Our aim is to embed these inhomogeneous observations (e.g. different types of measurements, different instruments, etc...) into a shared latent space, in a completely self-supervised manner. These embeddings can then be used for a variety of downstream applications (e.g. redshift estimation,  morphology classification) with very simple machine learning methods and reach near optimal performance. More specifically, I will present our AstroCLIP method which allows us to align embeddings between data modalities, and our more recent results on building highly effective image embeddings based on a vision transformer architecture. I will also comment on these results from an information theory point of view, and conclude on future prospects for this approach in the context of upcoming large scale galaxy surveys.




Astrostatistics Newsletter

If you have been wanting to stay up to date on the happenings in the astrostatistics community, or to learn more about astrostatistics, please subscribe to Astrostatistics News.  Astrostatistics News (AN) is a newsletter designed to inform, promote, cultivate, and inspire the astrostatistics community.  AN serves the astrostatistics community by highlighting and describing recent research developments in astrostatistics at an accessible level to the diverse backgrounds of its members, sharing interesting new algorithms, software, or data sets, promoting relevant events, and striving to inspire new researchers to join in the fun.  We anticipate 2 - 3 issues per year, with the potential for more.

To subscribe to Astrostatistics News, go to and select the “Join group” button.  You will need to be logged into your Google account to join the group.

Back to Commission Homepage


Donate to the IAU

Donate to the IAU

General Assembly 2024

IAU General Assembly 2024

IAU Strategic Plan 2020–2030

Strategic Plan

IAU Code of Conduct

Code of Conduct

Symposia and Meetings



How to Become a Member

Deceased Members

Deceased Members

Centre for the Protection of the Dark and Quiet Sky from Satellite Constellation Interference


IAU Catalyst

Latest Catalyst

IAU e-Newsletter
Volume 2024 n° 1

Latest e-Newsletter

Subscribe to the e-Newsletter


IAU Office of Astronomy for Development

Office for Astronomy Development

IAU Office for Young Astronomers

Office for Young Astronomers

IAU Office for Astronomy Outreach

Office for Astronomy Outreach

IAU Office of Astronomy for Education

Office of Astronomy for Education

International School for Young Astronomers

International School for Young Astronomers

WG Small Bodies Nomenclature Bulletins

WG Small Bodies Nomenclature Bulletins

IAU WG Women in Astronomy Newsletters and Ensemble Magazine

WG Women in Astronomy Newsletters