Ashish Patel 🇮🇳’s Post

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone by Microsoft We're thrilled to introduce phi-3-mini, a groundbreaking language model that fits right in your pocket! With 3.8 billion parameters trained on 3.3 trillion tokens, it rivals giants like GPT-3.5 in performance—all from your phone. 🔍 Our unique approach utilizes a mix of refined web data and synthetic data, enabling top-tier performance without the bulk. The phi-3-mini not only promises powerful AI capabilities on mobile devices but does so with an emphasis on safety and reliability, aligning with Microsoft’s responsible AI principles. 📊 Showcased results from benchmarks such as MMLU and HellaSwag confirm that phi-3-mini isn't just small, it's mighty, offering insights and interactions previously only possible on high-end servers. 🌐 Looking ahead, they're committed to enhancing multilingual support and further refining our models to handle a wider array of tasks more effectively. Stay tuned as we continue to push the boundaries of what's possible in AI technology, making it more accessible than ever. #AI #Innovation #Microsoft #LanguageModels

Introducing LLaVA-Llama-3-8B is released! XTuner team releases the new multi-modal models (LLaVA-Llama-3-8B and In the realm of multi-modal models, the XTuner team has taken a monumental leap forward with the launch of LLaVA-Llama-3-8B and its variant, LLaVA-Llama-3-8B-v1.1, powered by the groundbreaking Llama-3 LLM. These models aren't just a step up from their predecessors—they're a giant leap, smashing previous benchmarks and setting new standards of excellence. Why is this a game-changer? 🌟 The performance of these models doesn't just surpass the previous iteration; it obliterates it, redefining what we thought was possible in the process. And guess what? There's even more on the horizon with LLaVA-Llama-3-70B. For those looking to dive deeper, explore the models and their capabilities here: Model: https://lnkd.in/gC2YUrmB) https://lnkd.in/gGXZjyV8 Code: https://lnkd.in/gPYs7qgi ↓ Check out https://lnkd.in/gredE7Nu to get a weekly summary of the top models, repos and papers in AI. Read by 180,000+ engineers and researchers.

LLaVA-Llama-3-8B is released! XTuner team releases the new multi-modal models (LLaVA-Llama-3-8B and LLaVA-Llama-3-8B-v1.1) with Llama-3 LLM, achieving much better performance on various benchmarks. The performance evaluation substantially surpasses Llama-2. (LLaVA-Llama-3-70B is coming soon!) Model: https://lnkd.in/gC2YUrmB) https://lnkd.in/gGXZjyV8 Code: https://lnkd.in/gPYs7qgi

Exciting news in the LLM community! 🚀 The latest rankings are in, and it’s a close competition at the top. With GPT-4 variants showing strong performance, we see a notable entry with Llama-3-70b-Instruct securing a spot within the top ranks. This is more than a scoreboard—it’s a testament to the rapid advancements in LLMs language models. Organizations are pushing boundaries, as shown by the significant movement in positions. A key takeaway for anyone in tech or AI: stay adaptive, innovative, and ready to embrace the evolving landscape.

Zamba: A 7B Mamba-like SSM hybrid model trained for 1T tokens ✦ Zyphra, a leading AI research company, has unveiled Zamba - a novel 7 billion parameter foundation model. ✦ Zamba represents a significant advancement in the field of large language models, showcasing innovative architectural choices that enable impressive performance with a relatively small parameter count. 🎯 Zamba's Key Highlights: 1. Compute-Efficient Architecture: → Zamba's novel hybrid design combines Mamba blocks with a global shared attention layer. → This architecture is more compute-efficient during both training and inference compared to vanilla transformer models. → It demonstrates the scalability and performance capabilities of Sparse Transformer (SSM) models. 2. Impressive Performance: → Zamba approaches the performance of state-of-the-art models like Mistral and Gemma, despite being trained on significantly fewer tokens. → It notably outperforms larger models like LLaMA-2 7B and OLMo-7B on a wide range of benchmarks, using less than half the training data. 3. Two-Phase Training Approach: → Zamba was trained in two phases: first on lower-quality web data, followed by an annealing phase on high-quality datasets. → This two-phase approach appears to significantly improve the model's quality. 4. Open-Source Commitment: → Zyphra is releasing all Zamba checkpoints open-source under the Apache 2.0 license. → This level of transparency is crucial for advancing the understanding of large-scale language models and enabling further innovations. 5. Efficient Development: → Zamba was developed by a small team of 7 researchers in just 30 days, using 128 NVIDIA H100 GPUs. → This demonstrates that highly capable and efficient models can be created without the need for massive teams and computational resources. 🧰 Implications and Impact: → Zamba represents a major step towards developing compact, parameter and inference-efficient models that can outperform larger, more resource-intensive models. → The open-source availability of Zamba's checkpoints and architectural details will enable researchers and developers to dive deep into the model, explore its unique characteristics, and contribute to further advancements in the field. - Zamba's efficiency and accessibility have the potential to democratize advanced AI capabilities, bringing cutting-edge language technologies within reach of a wider audience. Zamba is a groundbreaking development in the world of large language models, showcasing how innovative architectural choices and a commitment to open science can lead to impressive performance and efficiency. As the AI community explores and builds upon Zamba's unique capabilities, we can expect to see further advancements in the quest for compact, high-performing, and accessible AI systems. P.S. What do you think is this game changing ?