Do you know which LLMs are best in Armenian? We are releasing ArmBench-LLM 1.0, which compares popular LLMs on our Armenian benchmark. It checks both knowledge (e.g. grammar, history, literature etc.) as well as generation capabilities (e.g. translating text, summarizing emails etc.). It also gives insight on cost vs accuracy in the spend report. Quick insights: • Gemini 3 Flash is the overall leader • Qwen 3.5 27B is the only OSS model in top 10 • Grok scores 18.75 on math exam in Armenian Links in the comments. #opensource #ArmenianAI #Metric #ArmBench
Finally. Been waiting for this. Can't wait to try it on our models.
Interesting to see Qwen holding its place as OSS. Shows open models are catching up in niche domains.
This is great! Thanks for sharing! I wonder if you did it for Gemma 4 or other open models as well?
Metric•6K followers
3dArticle: https://huggingface.co/blog/Metric-AI/armbench-llm Leaderboard: https://metric-ai-armbench-llm.hf.space/