Showing posts with label lightning attention. Show all posts
Showing posts with label lightning attention. Show all posts

18.6.25

MiniMax-M1: A Breakthrough Open-Source LLM with a 1 Million Token Context & Cost-Efficient Reinforcement Learning

 MiniMax, a Chinese AI startup renowned for its Hailuo video model, has unveiled MiniMax-M1, a landmark open-source language model released under the Apache 2.0 license. Designed for long-context reasoning and agentic tool use, M1 supports a 1 million token input and 80,000 token output window—vastly exceeding most commercial LLMs and enabling it to process large documents, contracts, or codebases in one go.

Built on a hybrid Mixture-of-Experts (MoE) architecture with lightning attention, MiniMax-M1 optimizes performance and cost. The model spans 456 billion parameters, with 45.9 billion activated per token. Its training employed a custom CISPO reinforcement learning algorithm, resulting in substantial efficiency gains. Remarkably, M1 was trained for just $534,700, compared to over $5–6 million spent by DeepSeek‑R1 or over $100 million for GPT‑4.


⚙️ Key Architectural Innovations

  • 1M Token Context Window: Enables comprehensive reasoning across lengthy documents or multi-step workflows.

  • Hybrid MoE + Lightning Attention: Delivers high performance without excessive computational overhead.

  • CISPO RL Algorithm: Efficiently trains the model with clipped importance sampling, lowering cost and training time.

  • Dual Variants: M1-40k and M1-80k versions support variable output lengths (40K and 80K “thinking budget”).


📊 Benchmark-Topping Performance

MiniMax-M1 excels in diverse reasoning and coding benchmarks:

AIME 2024 (Math): 86.0% accuracy
LiveCodeBench (Coding): 65.0%
SWE‑bench Verified: 56.0%
TAU‑bench: 62.8%
OpenAI MRCR (4-needle): 73.4% 

These results surpass leading open-weight models like DeepSeek‑R1 and Qwen3‑235B‑A22B, narrowing the gap with top-tier commercial LLMs such as OpenAI’s o3 and Google’s Gemini due to its unique architectural optimizations.


🚀 Developer-Friendly & Agent-Ready

MiniMax-M1 supports structured function calling and is packaged with an agent-capable API that includes search, multimedia generation, speech synthesis, and voice cloning. Recommended for deployment via vLLM, optimized for efficient serving and batch handling, it also offers standard Transformers compatibility.

For enterprises, technical leads, and AI orchestration engineers—MiniMax-M1 provides:

  • Lower operational costs and compute footprint

  • Simplified integration into existing AI pipelines

  • Support for in-depth, long-document tasks

  • A self-hosted, secure alternative to cloud-bound models

  • Business-grade performance with full community access


🧩 Final Takeaway

MiniMax-M1 marks a milestone in open-source AI—combining extreme context length, reinforcement-learning efficiency, and high benchmark performance within a cost-effective, accessible framework. It opens new possibilities for developers, researchers, and enterprises tackling tasks requiring deep reasoning over extensive content—without the limitations or expense of closed-weight models.

  Anthropic Enhances Claude Code with Support for Remote MCP Servers Anthropic has announced a significant upgrade to Claude Code , enablin...