Build your LLM training/finetuning/instruction-tuning/eval pipeline in any configuration, in <30 lines of code.

Including

1. Build you custom architecture/dataset, or download easily from Huggingface. 

2. Train in single/multi GPU settings in DDP/FSDP settings. 

3. Finetune (Q)LoRA adapters for low-resource settings.

4. Automatic setup of best hyperparameters (including automatic selection of largest possible batch size, optimal learning rate, etc.)

5. Inbuilt model utilities. Including hyperparameter schedulers, regular model checkpointing based on best performance, W&B logging. 

6. Evaluate model on any benchmarks with just one line of code.