Skip to content

Configuration

Note

Also checkout customizing HuggingFace behaviour

Bases: BaseSettings

Configuration for the application. Values can be set via environment variables.

Pydantic will automatically handle mapping uppercased environment variables to the corresponding fields. To populate nested, the environment should be prefixed with the nested field name and an underscore. For example, the environment variable LOG_LEVEL will be mapped to log_level, WHISPER__INFERENCE_DEVICE(note the double underscore) to whisper.inference_device, to set quantization to int8, use WHISPER__COMPUTE_TYPE=int8, etc.

Source code in src/speaches/config.py 
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
class Config(BaseSettings):
    """Configuration for the application. Values can be set via environment variables.

    Pydantic will automatically handle mapping uppercased environment variables to the corresponding fields.
    To populate nested, the environment should be prefixed with the nested field name and an underscore. For example,
    the environment variable `LOG_LEVEL` will be mapped to `log_level`, `WHISPER__INFERENCE_DEVICE`(note the double underscore) to `whisper.inference_device`, to set quantization to int8, use `WHISPER__COMPUTE_TYPE=int8`, etc.
    """

    model_config = SettingsConfigDict(env_nested_delimiter="__")

    api_key: SecretStr | None = None
    """
    If set, the API key will be required for all requests.
    """
    log_level: str = "debug"
    """
    Logging level. One of: 'debug', 'info', 'warning', 'error', 'critical'.
    """
    host: str = Field(alias="UVICORN_HOST", default="0.0.0.0")
    port: int = Field(alias="UVICORN_PORT", default=8000)
    allow_origins: list[str] | None = None
    """
    https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
    Usage:
        `export ALLOW_ORIGINS='["http://localhost:3000", "http://localhost:3001"]'`
        `export ALLOW_ORIGINS='["*"]'`
    """

    enable_ui: bool = True
    """
    Whether to enable the Gradio UI. You may want to disable this if you want to minimize the dependencies and slightly improve the startup time.
    """

    whisper: WhisperConfig = WhisperConfig()

    # TODO: remove the underscore prefix from the field name
    _unstable_vad_filter: bool = True
    """
    Default value for VAD (Voice Activity Detection) filter in speech recognition endpoints.
    When enabled, the model will filter out non-speech segments. Useful for removing hallucinations in speech recognition caused by background silences.


    NOTE: having `_unstable_vad_filter: True` technically deviates from the OpenAI API specification, so you may want to set it to `False`.

    NOTE: This is an unstable feature and may change in the future.
    """

    loopback_host_url: str | None = None
    """
    If set this is the URL that the gradio app will use to connect to the API server hosting speaches.
    If not set the gradio app will use the url that the user connects to the gradio app on.
    """

    # TODO: document the below configuration options
    chat_completion_base_url: str = "http://localhost:11434/v1"
    chat_completion_api_key: SecretStr = SecretStr("cant-be-empty")

    unstable_ort_opts: OrtOptions = OrtOptions()

api_key 

api_key: SecretStr | None = None

If set, the API key will be required for all requests.

log_level 

log_level: str = 'debug'

Logging level. One of: 'debug', 'info', 'warning', 'error', 'critical'.

host 

host: str = Field(alias='UVICORN_HOST', default='0.0.0.0')

port 

port: int = Field(alias='UVICORN_PORT', default=8000)

allow_origins 

allow_origins: list[str] | None = None

https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values Usage: export ALLOW_ORIGINS='["http://localhost:3000", "http://localhost:3001"]' export ALLOW_ORIGINS='["*"]'

enable_ui 

enable_ui: bool = True

Whether to enable the Gradio UI. You may want to disable this if you want to minimize the dependencies and slightly improve the startup time.

whisper 

whisper: WhisperConfig = WhisperConfig()

_unstable_vad_filter 

_unstable_vad_filter: bool = True

Default value for VAD (Voice Activity Detection) filter in speech recognition endpoints. When enabled, the model will filter out non-speech segments. Useful for removing hallucinations in speech recognition caused by background silences.

NOTE: having _unstable_vad_filter: True technically deviates from the OpenAI API specification, so you may want to set it to False.

NOTE: This is an unstable feature and may change in the future.

loopback_host_url 

loopback_host_url: str | None = None

If set this is the URL that the gradio app will use to connect to the API server hosting speaches. If not set the gradio app will use the url that the user connects to the gradio app on.

chat_completion_base_url 

chat_completion_base_url: str = 'http://localhost:11434/v1'

chat_completion_api_key 

chat_completion_api_key: SecretStr = SecretStr(
    "cant-be-empty"
)

unstable_ort_opts 

unstable_ort_opts: OrtOptions = OrtOptions()

Bases: BaseModel

See https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py#L599.

Source code in src/speaches/config.py 
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class WhisperConfig(BaseModel):
    """See https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py#L599."""

    inference_device: Device = "auto"
    device_index: int | list[int] = 0
    compute_type: Quantization = "default"  # TODO: should this even be a configuration option?
    cpu_threads: int = 0
    num_workers: int = 1
    ttl: int = Field(default=300, ge=-1)
    """
    Time in seconds until the model is unloaded if it is not being used.
    -1: Never unload the model.
    0: Unload the model immediately after usage.
    """
    use_batched_mode: bool = False
    """
    Whether to use batch mode(introduced in 1.1.0 `faster-whisper` release) for inference. This will likely become the default in the future and the configuration option will be removed.
    """

ttl = Field(default=300, ge=-1) class-attribute instance-attribute

Time in seconds until the model is unloaded if it is not being used. -1: Never unload the model. 0: Unload the model immediately after usage.

use_batched_mode = False class-attribute instance-attribute

Whether to use batch mode(introduced in 1.1.0 faster-whisper release) for inference. This will likely become the default in the future and the configuration option will be removed.