fairseq2

In line with FAIR mission, we open source Seamless Interaction Dataset to the community in order to accelerate the field. Download on github github.com/facebookrese... and @hf.co huggingface.co/datasets/fac...

[4/4] Collaborate efficiently with reproducible experiment setups using FAIRSeq2. Identify root causes swiftly and share lessons learned with the community. Create your benchmarks and contribute!

[3/4] Beyond TensorBoard and WanDB, FAIRSeq2 supports torch profilers (set trainer.profile and common.profilers.torch.enabled=True) to inspect potential infra issues. Dive deep into your training processes with various profilers and metric recorders.

[2/4] e.g., the tokens/s metric allows easy computation of Model FLOP Utilization – a great metric to check resource utilization efficiency. We achieve up to 48% MFU on 8 GPUs and maintain 37.6% across 4 nodes (32 GPUs). Experience effective and efficient distributed training!

[1/4] 🛠️ FAIRSeq2 – your go-to tool for reliable benchmarking and diagnosing infra issues! With native logging of metrics, monitor training performance in real-time and ensure great visibility. #AI #MachineLearning #fairseq2

🚀 Transform your LLM post-training with fairseq2! We make complex post-training into a breeze, so that you can make fairseq2 your paper machine!

Feel free to check our tutorials out:
- SFT: facebookresearch.github.io/fairseq2/sta...
- DPO: facebookresearch.github.io/fairseq2/sta...

This project was made feasible by the excellent open-source LLM training library @fairseq2.bsky.social; I highly recommend giving it a look! It made both SFT and DPO a piece of cake 🍰

Nothing explains better than a vivid example:

🚀 Big news for LLM researchers! #fairseq2 now has native support in #vLLM. Deploy your fine-tuned language models with vLLM in just one command for lightning-fast performance. Ready to accelerate your research like in FAIR? Check this out: facebookresearch.github.io/fairseq2/sta...

Large Concept Models extends fairseq2 and demonstrates what we can imagine beyond traditional sequence models -- check it out here: ai.meta.com/research/pub...

Seamless Communication - AI at Meta A significant step towards removing language barriers through expressive, fast and high-quality AI translation

Seamless Communication as the best innovation recognized by TIMES and Nature -- check it out here: ai.meta.com/research/sea...

🖼️ A gallery of open-source projects and papers powered by #fairseq2! 🚀

Seamless Communication and Large Concept Models are 2 vivid examples that showcase the potential of what we are building.

More exciting FAIR research built on fairseq2 is on the way!

Check out our doc for more details: facebookresearch.github.io/fairseq2/stabl…

👋 Hello world! We’re thrilled to announce the v0.4 release of fairseq2 — an open-source library from FAIR powering many projects at Meta. pip install fairseq2 and explore our trainer API, instruction & preference finetuning (up to 70B), and native vLLM integration.