Model Selection Algorithms

Model Selection Algorithms

Model selection algorithms are dynamic inference optimizations where a choice is made between two or more models for execution. The hottest area of such research is Mixture-of-Experts, because of the GPT-4's rumored architecture. Another example is “big-little” architectures, where a heuristic attempts to send “easy” queries to a faster “little” model. Various other ensemble architectures are possible with multiple models.

Another practical example of a different type of model selection is the deployment architecture, which may be deciding which server to send the request to, where each server may have different models or multiple instances of the same model. Other areas of research with similar aims include cascades and collaborative inference.