AGI Math Model 🧠
PARAMANU-GANITA 🧠 is our first step towards AGI.
A language model with extremely strong mathematical reasoning abilities.
We present PARAMANU-GANITA, a 208 million parameter novel Auto Regressive (AR) decoder based language model on mathematics. The model is pretrained from scratch at context size of 4096 on our curated mixed mathematical corpus.
PARAMANU-GANITA despite being 35 times smaller than 7B LLMs, outperformed generalist LLMs such as LLaMa-1 7B by 28.4% points, LLaMa-2 7B by 27.6% points, Falcon 7B by 32.6% points, PaLM 8B by 35.3% points, and math specialised LLMs such as Minerva 8B by 23.2% points, and LLEMMA-7B by 3.0% points in GSM8k test accuracy metric respectively.
PARAMANU-GANITA also outperformed giant LLMs like PaLM 62B by 6.4% points, Falcon 40B by 19.8% points, LLaMa-1 33B by 3.8% points, and Vicuna 13B by 11.8% points respectively.
| Model | Parameters | GSM8k Pass@1 |
|---|---|---|
| LLaMa-1 | 33B | 35.6 |
| LLaMa-1 | 7B | 11.0 |
| LLaMa-2 | 13B | 28.7 |
| LLaMa-2 | 7B | 11.8 |
| Code LLaMa | 7B | 10.5 |
| Code LLaMa | 34B | 29.6 |
| Falcon | 40B | 19.6 |
| Falcon | 7B | 6.8 |
| MPT | 30B | 15.2 |
| MPT | 7B | 6.8 |
| GPT-J | 6B | 34.9 |
| Vicuna | 13B | 27.6 |
| PaLM | 8B | 4.1 |
| PaLM | 62B | 33.0 |
| Minerva | 8B | 16.2 |
| LLEMMA | 7B | 36.4 |
| PARAMANU-GANITA | 208M | 39.4 |
https://goatstack.ai/topics/paramanu-ganita-a-mathematical-marvel-nqvgdy
https://arxiv.org/abs/2404.14395
German Engineering with Indian heart ❤️
- Interested in learning how we can help you to leverage GenAI technology to grow your business? Just write to us!
- We also offer novel smaller, efficient, powerful, faster domain adaptive sector-agnostic foundation generative language models & multimodal AI models as SaaS.
Our models don’t even need GPU to run inference.
Contact us today to schedule a consultation.
- Email: info@bharatgpts.com
