Genomic AI models!

14 Oct, 2025

What are genomic AI models:

• These models are like LLMs (large language models), meaning that they are trained using vast amounts of data, which in this case are long stretches of DNA containing up to billions of bases.
• Since they can process much longer sequences of DNA, they can also predict long-range interactions (Trans regulation).
• Think about chatGPT, but instead of it being trained on massive human text, it is trained on massive amounts of DNA sequences from various individuals and species.

Two types of genomic AI models:

• Supervised and sequence-to-function models: trained on functional genomic data (gene expression, methylation patterns. These models aim to predict the function of a certain DNA sequence based on this data.
• Unsupervised or self-supervised ‘genomic language models’ (gLMs). These models are trained on genomic sequence data and used to predict the next base or fill in gaps in known DNA sequences.
• The main difference is that one tries to predict the function of a given sequence and the other tries to predict the composition of a sequence.
• Both use neural networks but are trained in slightly different ways.

Alpha fold and transformer-based architecture

• Alpha fold is an AI model built by DeepMind. It predicts the 3D structure of proteins from their amino acid sequence.
• It works by using a transformer-based architecture and learns statistical relationships between amino acid residues.
• A transformer based architecture uses attention: crossing every word in a sequence with another word and determining their relationship and essentially produces a “map of relevance”.
• This allows the model to predict how amino acids that are far apart in a long chain of amino acids will interact to fold in a certain way.
• Alpha fold was extremely revolutionary because to find the shape of a protein one had to:
  • Before: use experimental methods like X-ray crystallography, NMR spectroscopy, Cryo-EM. This could take months or years and was a very costly way of doing it.
  • Now: with alpha fold, one can predict the structure of a protein from its amino acid sequence in days or even hours. Over 200 million proteins predicted, basically all of the proteins we know exist.

Evo

• Evo is another super cool genomic AI model that is able to predict microbial genes that are vital for survival, the effect of mutations and the craziest one, it can generate new DNA sequences with over 1 million base pairs!
  • This is very useful for CRISPR treatments.
• This model was trained with massive amounts of data - over 9.3 trillion nucleotides and 2.7 billion prokaryotic genomes.
• Evo uses a different type of neural network architecture, known as StripedHyena, which uses attention but replaces some of the terms crossed with more efficient sequence operators to make it better for ultra-long DNA sequences.

RegLM:

• regLM is a genomic AI model designed to generate synthetic cis-regulatory DNA elements like promoters and enhancers.
• It uses the HyenaDNA architecture using prompt tokens that encode functional labels (high activity, low activity, or cell type specific behavior). Biology-focused and uses purely hyena operators, no attention, unlike the StripedHyena architecture.
• This model is trained to predict the next DNA base in a sequence. It is trained on yeast and human cis-regulatory elements (CREs) to learn general patterns and motifs common to regulatory DNA.
• regLM is able to product synthetic CREs with realistic biological properties. This helps in synthetic biology, where scientists need tailored regulatory parts for engineered organisms

Limitations of genomic AI models:

• Trans regulation is challenging to predict: some regulatory elements act distant from a gene, and it is hard to infer a causal relationship between those elements.
• Because there are only about 20-25K genes in the human genome, and only a small number of those are active in a cell, it limits the number of regulation scenarios a model can learn from.