We built a model-routing layer for switching between hosted and self-deployed models

We’re the team building SwiftScale, and we’ve been working on a routing layer for

applications that need to use more than one AI model or provider.

Instead of exposing a different integration for every provider, the application

uses one OpenAI-compatible API. The routing layer can then select an eligible

model based on:

- Task fit

- Regional availability

- Provider health

- Latency

- Context requirements

- Cost

- Project policy

Direct Model IDs are still available when an application needs deterministic

model selection. We’re also working on treating self-hosted inference clusters

and serverless providers as candidates in the same model pool.

Architecture overview:

We’d be interested in how others decide which routing decisions should be

automatic and which should remain under application control.

Disclosure: We’re the team building SwiftScale.

1 view