Language router list

Response

200 - application/json

routers

object[]

Show child attributes

routers.enabled

boolean

required

Is router enabled?

routers.models

object[]

required

the list of models that could handle requests

Show child attributes

routers.models.enabled

boolean

required

Is the model enabled?

routers.models.id

string

required

Model instance ID (unique in scope of the router)

routers.models.anthropic

object

Show child attributes

routers.models.anthropic.baseUrl

string

required

routers.models.anthropic.chatEndpoint

string

required

routers.models.anthropic.model

string

required

routers.models.anthropic.defaultParams

object

Show child attributes

routers.models.anthropic.defaultParams.max_tokens

integer

routers.models.anthropic.defaultParams.metadata

string

routers.models.anthropic.defaultParams.stop

string[]

routers.models.anthropic.defaultParams.system

string

routers.models.anthropic.defaultParams.temperature

number

routers.models.anthropic.defaultParams.top_k

integer

routers.models.anthropic.defaultParams.top_p

number

routers.models.azureopenai

object

Show child attributes

routers.models.azureopenai.apiVersion

string

required

The API version to use for this operation. This follows the YYYY-MM-DD format (e.g 2023-05-15)

routers.models.azureopenai.baseUrl

string

required

The name of your Azure OpenAI Resource (e.g https://glide-test.openai.azure.com/)

routers.models.azureopenai.model

string

required

This is your deployment name. You're required to first deploy a model before you can make calls (e.g. glide-gpt-35)

routers.models.azureopenai.chatEndpoint

string

routers.models.azureopenai.defaultParams

object

Show child attributes

routers.models.azureopenai.defaultParams.frequency_penalty

integer

routers.models.azureopenai.defaultParams.logit_bias

object

Show child attributes

routers.models.azureopenai.defaultParams.max_tokens

integer

routers.models.azureopenai.defaultParams.n

integer

routers.models.azureopenai.defaultParams.presence_penalty

integer

routers.models.azureopenai.defaultParams.response_format

object

routers.models.azureopenai.defaultParams.seed

integer

routers.models.azureopenai.defaultParams.stop

string[]

routers.models.azureopenai.defaultParams.temperature

number

routers.models.azureopenai.defaultParams.tool_choice

object

routers.models.azureopenai.defaultParams.tools

string[]

routers.models.azureopenai.defaultParams.top_p

number

routers.models.azureopenai.defaultParams.user

string

routers.models.client

object

Show child attributes

routers.models.client.timeout

string

routers.models.cohere

object

Show child attributes

routers.models.cohere.baseUrl

string

required

routers.models.cohere.chatEndpoint

string

required

routers.models.cohere.model

string

required

routers.models.cohere.defaultParams

object

Show child attributes

routers.models.cohere.defaultParams.chat_history

object[]

Show child attributes

routers.models.cohere.defaultParams.citiation_quality

string

routers.models.cohere.defaultParams.connectors

string[]

routers.models.cohere.defaultParams.conversation_id

string

routers.models.cohere.defaultParams.preamble_override

string

routers.models.cohere.defaultParams.prompt_truncation

string

routers.models.cohere.defaultParams.search_queries_only

boolean

routers.models.cohere.defaultParams.stream

boolean

unsupported right now

routers.models.cohere.defaultParams.temperature

number

routers.models.error_budget

string

routers.models.latency

object

Show child attributes

routers.models.latency.decay

number

Weight of new latency measurements

routers.models.latency.update_interval

string

How often gateway should probe models with not the lowest response latency

routers.models.latency.warmup_samples

integer

The number of latency probes required to init moving average

routers.models.octoml

object

Show child attributes

routers.models.octoml.baseUrl

string

required

routers.models.octoml.chatEndpoint

string

required

routers.models.octoml.model

string

required

routers.models.octoml.defaultParams

object

Show child attributes

routers.models.octoml.defaultParams.frequency_penalty

integer

routers.models.octoml.defaultParams.max_tokens

integer

routers.models.octoml.defaultParams.presence_penalty

integer

routers.models.octoml.defaultParams.stop

string[]

routers.models.octoml.defaultParams.temperature

number

routers.models.octoml.defaultParams.top_p

number

routers.models.openai

object

Add other providers like

Show child attributes

routers.models.openai.baseUrl

string

required

routers.models.openai.chatEndpoint

string

required

routers.models.openai.model

string

required

routers.models.openai.defaultParams

object

Show child attributes

routers.models.openai.defaultParams.frequency_penalty

integer

routers.models.openai.defaultParams.logit_bias

object

Show child attributes

routers.models.openai.defaultParams.max_tokens

integer

routers.models.openai.defaultParams.n

integer

routers.models.openai.defaultParams.presence_penalty

integer

routers.models.openai.defaultParams.response_format

object

routers.models.openai.defaultParams.seed

integer

routers.models.openai.defaultParams.stop

string[]

routers.models.openai.defaultParams.temperature

number

routers.models.openai.defaultParams.tool_choice

object

routers.models.openai.defaultParams.tools

string[]

routers.models.openai.defaultParams.top_p

number

routers.models.openai.defaultParams.user

string

routers.models.weight

integer

routers.retry

object

required

retry when no healthy model is available to router

Show child attributes

routers.retry.base_multiplier

integer

routers.retry.max_delay

integer

routers.retry.max_retries

integer

routers.retry.min_delay

integer

routers.routers

string

required

Unique router ID

routers.strategy

string

required

strategy on picking the next model to serve the request