VGL-stGen is a multimodal large language model built by adapting a pretrained medical vision-language model. It is designed to jointly model lung histology and spatial transcriptomics, aiming to learn a shared representation across these two data modalities.
The model is trained using paired lung hematoxylin and eosin (H&E) images and spatial transcriptomics data, with single-cell references incorporated into the training process. The abstract indicates that training integrates image-expression and additional information, though the full details are not specified in the available text.
This approach could help researchers connect tissue morphology with gene expression patterns at the spatial level, potentially improving analysis of lung tissue samples. The work is described in a preprint on arXiv.