Han Yuan

Thanks!

8/
- Decima (www.biorxiv.org/content/10.1...)
- grelu (www.biorxiv.org/content/10.1...)
- scooby (www.biorxiv.org/content/10.1...)
- flashzoi (www.biorxiv.org/content/10.1...)
We hope this community continues to grow and drive innovation in scalable, data-efficient regulatory modeling!

7/ Finally, our PEFT-based Borzoi transfer approach aim to contribute to a growing ecosystem to make large input DNA models like Borzoi more accessible for personalized regulatory genomics research, alongside tools like:

6/ Additionally, we characterize the tradeoffs of adapter insertion strategies and have some interesting observations. E.g., as we try to adapt conv layers, intermediate activations, rather than trainable parameters, become the practical bottleneck for efficient memory usage.

5/ The transferred models:
- Run on a single GPU with less than 20GB
- Accurately predict gene expression and specificity
- Identify key regulatory factors driving differential expression
- Predict cell-type specific genetic variant effects

4/ Parameter-efficient fine-tuning (PEFT) achieves efficient transfer by updating a small subset of parameters and maintains strong performance. We implemented multiple PEFT modules for both attention and conv layers in the TensorFlow Borzoi codebase and applied to both bulk and scRNA-seq datasets.

3/ Borzoi is a large pre-trained model that learns the general rules of gene regulation. But transferring the model to a custom dataset remains challenging.
- Training from scratch is slow.
- Updating only the final layer underperforms.
- Updating all parameters (full fine-tuning) is expensive.

2/ Code: github.com/calico/baske...
Tutorial: github.com/calico/baske...

Parameter-Efficient Fine-Tuning of a Supervised Regulatory Sequence Model DNA sequence deep learning models accurately predict epigenetic and transcriptional profiles, enabling analysis of gene regulation and genetic variant effects. While large-scale training models like E...

1/ DNA sequence models like Borzoi predict gene expression and variant effects across tissues — but how can someone adapt the model to a custom experiment? @drkbio.bsky.social, Johannes Linder and I propose a solution via parameter-efficient fine-tuning (PEFT).
www.biorxiv.org/content/10.1...

Working with a great team to organize
@keystonesymposia.bsky.social AI in MolecularBiology, this September! We aimed for a wide range of biological topics, and emphasized speakers who blend sophisticated machine learning with compelling biological questions and analysis.

Shift augmentation improves DNA convolutional neural network indel effect predictions Determining genetic variant effects on molecular phenotypes like gene expression is a task of paramount importance to medical genetics. DNA convolutional neural networks (CNNs) attain state-of-the-art...

⚡️ Our latest preprint is on bioRxiv!

Shift augmentation improves DNA convolutional neural network indel effect predictions

www.biorxiv.org/content/10.1...

yep, nice to see more contents here! thanks! the credit goes to Dave and Johannes. hope you find the model useful!

likewise! sorry i'm bouncing back and forth b/w twitter and bluesky nowadays and sometimes lose track of threads!