Skip to content

Configuration

Note

Also checkout customizing HuggingFace behaviour

Bases: BaseSettings

Configuration for the application. Values can be set via environment variables.

Pydantic will automatically handle mapping uppercased environment variables to the corresponding fields. To populate nested, the environment should be prefixed with the nested field name and an underscore. For example, the environment variable LOG_LEVEL will be mapped to log_level, WHISPER__INFERENCE_DEVICE(note the double underscore) to whisper.inference_device, to set quantization to int8, use WHISPER__COMPUTE_TYPE=int8, etc.

Source code in src/speaches/config.py 
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
class Config(BaseSettings):
    """Configuration for the application. Values can be set via environment variables.

    Pydantic will automatically handle mapping uppercased environment variables to the corresponding fields.
    To populate nested, the environment should be prefixed with the nested field name and an underscore. For example,
    the environment variable `LOG_LEVEL` will be mapped to `log_level`, `WHISPER__INFERENCE_DEVICE`(note the double underscore) to `whisper.inference_device`, to set quantization to int8, use `WHISPER__COMPUTE_TYPE=int8`, etc.
    """

    model_config = SettingsConfigDict(env_nested_delimiter="__")

    stt_model_ttl: int = Field(default=300, ge=-1)
    """
    Time in seconds until a speech to text (stt) model is unloaded after last usage.
    -1: Never unload the model.
    0: Unload the model immediately after usage.
    """

    tts_model_ttl: int = Field(default=300, ge=-1)
    """
    Time in seconds until a text to speech (tts) model is unloaded after last usage.
    -1: Never unload the model.
    0: Unload the model immediately after usage.
    """

    vad_model_ttl: int = Field(default=-1, ge=-1)
    """
    Time in seconds until a voice activation detection (VAD) model is unloaded after last usage.
    -1: Never unload the model.
    0: Unload the model immediately after usage.
    """

    api_key: SecretStr | None = None
    """
    If set, the API key will be required for all requests.
    """
    log_level: str = "debug"
    """
    Logging level. One of: 'debug', 'info', 'warning', 'error', 'critical'.
    """
    host: str = Field(alias="UVICORN_HOST", default="0.0.0.0")
    port: int = Field(alias="UVICORN_PORT", default=8000)
    allow_origins: list[str] | None = None
    """
    https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
    Usage:
        `export ALLOW_ORIGINS='["http://localhost:3000", "http://localhost:3001"]'`
        `export ALLOW_ORIGINS='["*"]'`
    """

    enable_ui: bool = True
    """
    Whether to enable the Gradio UI. You may want to disable this if you want to minimize the dependencies and slightly improve the startup time.
    """

    whisper: WhisperConfig = WhisperConfig()

    # TODO: remove the underscore prefix from the field name
    _unstable_vad_filter: bool = True
    """
    Default value for VAD (Voice Activity Detection) filter in speech recognition endpoints.
    When enabled, the model will filter out non-speech segments. Useful for removing hallucinations in speech recognition caused by background silences.


    NOTE: having `_unstable_vad_filter: True` technically deviates from the OpenAI API specification, so you may want to set it to `False`.

    NOTE: This is an unstable feature and may change in the future.
    """

    loopback_host_url: str | None = None
    """
    If set this is the URL that the gradio app will use to connect to the API server hosting speaches.
    If not set the gradio app will use the url that the user connects to the gradio app on.
    """

    # TODO: document the below configuration options
    chat_completion_base_url: str = "http://localhost:11434/v1"
    chat_completion_api_key: SecretStr = SecretStr("cant-be-empty")

    unstable_ort_opts: OrtOptions = OrtOptions()

    otel_exporter_otlp_endpoint: str | None = None
    """
    OpenTelemetry OTLP exporter endpoint. If set, telemetry will be enabled.
    Example: 'http://localhost:4317'
    Shadows OTEL_EXPORTER_OTLP_ENDPOINT environment variable.
    """

    otel_service_name: str = "speaches"
    """
    OpenTelemetry service name for identifying this application in traces.
    Shadows OTEL_SERVICE_NAME environment variable.
    """

    preload_models: list[str] = []
    """
    List of model IDs to download during application startup.
    Models will be downloaded sequentially if they do not already exist locally.
    Application will exit if any model fails to download or is not found in the registry.
    Example: ["Systran/faster-whisper-tiny", "rhasspy/piper-voices"]
    """

stt_model_ttl 

stt_model_ttl: int = Field(default=300, ge=-1)

Time in seconds until a speech to text (stt) model is unloaded after last usage. -1: Never unload the model. 0: Unload the model immediately after usage.

tts_model_ttl 

tts_model_ttl: int = Field(default=300, ge=-1)

Time in seconds until a text to speech (tts) model is unloaded after last usage. -1: Never unload the model. 0: Unload the model immediately after usage.

vad_model_ttl 

vad_model_ttl: int = Field(default=-1, ge=-1)

Time in seconds until a voice activation detection (VAD) model is unloaded after last usage. -1: Never unload the model. 0: Unload the model immediately after usage.

api_key 

api_key: SecretStr | None = None

If set, the API key will be required for all requests.

log_level 

log_level: str = 'debug'

Logging level. One of: 'debug', 'info', 'warning', 'error', 'critical'.

host 

host: str = Field(alias='UVICORN_HOST', default='0.0.0.0')

port 

port: int = Field(alias='UVICORN_PORT', default=8000)

allow_origins 

allow_origins: list[str] | None = None

https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values Usage: export ALLOW_ORIGINS='["http://localhost:3000", "http://localhost:3001"]' export ALLOW_ORIGINS='["*"]'

enable_ui 

enable_ui: bool = True

Whether to enable the Gradio UI. You may want to disable this if you want to minimize the dependencies and slightly improve the startup time.

whisper 

whisper: WhisperConfig = WhisperConfig()

_unstable_vad_filter 

_unstable_vad_filter: bool = True

Default value for VAD (Voice Activity Detection) filter in speech recognition endpoints. When enabled, the model will filter out non-speech segments. Useful for removing hallucinations in speech recognition caused by background silences.

NOTE: having _unstable_vad_filter: True technically deviates from the OpenAI API specification, so you may want to set it to False.

NOTE: This is an unstable feature and may change in the future.

loopback_host_url 

loopback_host_url: str | None = None

If set this is the URL that the gradio app will use to connect to the API server hosting speaches. If not set the gradio app will use the url that the user connects to the gradio app on.

chat_completion_base_url 

chat_completion_base_url: str = 'http://localhost:11434/v1'

chat_completion_api_key 

chat_completion_api_key: SecretStr = SecretStr(
    "cant-be-empty"
)

unstable_ort_opts 

unstable_ort_opts: OrtOptions = OrtOptions()

otel_exporter_otlp_endpoint 

otel_exporter_otlp_endpoint: str | None = None

OpenTelemetry OTLP exporter endpoint. If set, telemetry will be enabled. Example: 'http://localhost:4317' Shadows OTEL_EXPORTER_OTLP_ENDPOINT environment variable.

otel_service_name 

otel_service_name: str = 'speaches'

OpenTelemetry service name for identifying this application in traces. Shadows OTEL_SERVICE_NAME environment variable.

preload_models 

preload_models: list[str] = []

List of model IDs to download during application startup. Models will be downloaded sequentially if they do not already exist locally. Application will exit if any model fails to download or is not found in the registry. Example: ["Systran/faster-whisper-tiny", "rhasspy/piper-voices"]

Bases: BaseModel

See https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py#L599.

Source code in src/speaches/config.py 
14
15
16
17
18
19
20
21
class WhisperConfig(BaseModel):
    """See https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py#L599."""

    inference_device: Device = "auto"
    device_index: int | list[int] = 0
    compute_type: Quantization = "default"  # TODO: should this even be a configuration option?
    cpu_threads: int = 0
    num_workers: int = 1