OraCore · Topic ·research

UniPool shares MoE experts across layers

UniPool replaces per-layer MoE experts with one shared pool, cutting redundancy and improving validation loss in five LLaMA-scale models.

1 articles in this thread ·Last updated 19h ago·First seen Jul 5, 2026

Timeline

  1. UniPool shares MoE experts across layers seed

    UniPool replaces per-layer MoE experts with one shared pool, cutting redundancy and improving validation loss in five LLaMA-scale models.