Back to home

Tag

MoE

MoE, or Mixture of Experts, is an architecture that activates only a subset of experts per token or task, balancing total parameter count, inference cost, and quality. It shows up in open coding models, long-context agents, and other systems built for efficient scaling.

2 articles

Mistral Models Guide Turns Picking Easier
Tools & Apps/Jun 22

Mistral Models Guide Turns Picking Easier

A practical breakdown of Mistral’s model lineup, pricing, and when I’d pick each one over OpenAI.

UniPool shares MoE experts across layers
Research/May 8

UniPool shares MoE experts across layers

UniPool replaces per-layer MoE experts with one shared pool, cutting redundancy and improving validation loss in five LLaMA-scale models.