pydantic / pydantic-settings

Settings management using pydantic

Home Page:https://docs.pydantic.dev/latest/usage/pydantic_settings/

Repository from Github https://github.compydantic/pydantic-settingsRepository from Github https://github.compydantic/pydantic-settings

πŸ› YAML-related values in config dict are never used

HomerusJa opened this issue Β· comments

The SettingsConfigDict provides three keys related to the YamlConfigSettingsSource:

class SettingsConfigDict(ConfigDict, total=False):
    # ...
    yaml_file: PathType | None
    yaml_file_encoding: str | None
    yaml_config_section: str | None
    # ...

These are lines 65-67 from pydantic_settings.main in v2.10.1. Based on the presence of these values, you would expect them to be used in the configuration of the sources (lines 349-432 in main.py for me), but they aren't. In fact, they are never used in the entire file.

The initialised settings sources are:

  • DefaultSettingsSource
  • InitSettingsSource
  • EnvSettingsSource
  • DotEnvSettingsSource
  • SecretsSettingsSource

πŸ” Reproduction

You can replicate this behaviour using the following two files in the same directory. Run main.py with the command uv run --script main.py.

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "pydantic-settings[yaml]",
# ]
# ///


# main.py
from pydantic_settings import (
    BaseSettings,
    SettingsConfigDict,
)


class Config(BaseSettings):
    model_config = SettingsConfigDict(yaml_file="config.yaml")

    some_value: int
    another_one: str


settings = Config()
# config.yaml
some_value: 2
another_one: "A random string"

When running this, the default behaviour in this case (where no setting sources are specified except the uninitialized YamlConfigSettingsSource) is that an empty dictionary of values is returned from BaseSettings._settings_build_values, as seen in lines 430 to 432.

            # no one should mean to do this, but I think returning an empty dict is marginally preferable
            # to an informative error and much better than a confusing error
            return {}

This leads to the following error in our case:

Traceback (most recent call last):
  File "/home/jakob/code/pydantic_settings_bug/main.py", line 23, in <module>
    settings = Config()
  File "/home/jakob/.cache/uv/environments-v2/main-e030d240e66b6dd2/lib/python3.13/site-packages/pydantic_settings/main.py", line 188, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        **__pydantic_self__._settings_build_values(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<27 lines>...
        )
        ^
    )
    ^
  File "/home/jakob/.cache/uv/environments-v2/main-e030d240e66b6dd2/lib/python3.13/site-packages/pydantic/main.py", line 253, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for Config
some_value
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing
another_one
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing

As you can see, input_value={} shows the default behavior mentioned above.

βœ… Workaround

My current solution is to use the settings_customise_sources hook. Following is a working example:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "pydantic-settings[yaml]",
# ]
# ///


# main.py
from pydantic_settings import (
    BaseSettings,
    PydanticBaseSettingsSource,
    YamlConfigSettingsSource,
)


class Config(BaseSettings):
    some_value: int
    another_one: str

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (
            YamlConfigSettingsSource(
                settings_cls=settings_cls,
                yaml_file="config.yaml",
            ),
        )


settings = Config()
print(settings)

πŸ’‘ Proposed Fixes

A couple of options for resolving this:

  • Remove the YAML-related keys (yaml_file, etc.) from SettingsConfigDict. These are currently misleading and unused.
  • Support them properly by integrating YamlConfigSettingsSource into the default source chain, respecting these values.
    • This would introduce a breaking change to the settings_customise_sources hook, since the number of parameters would need to grow.
    • An opt-in mechanism (e.g., via use_yaml=True) could soften the impact.

Removing the unused config keys would also technically be a breaking change, but given they currently do nothing, it seems the cleaner path.

Thanks @HomerusJa for reporting this.

YamlConfigSettingsSource is not one of the default sources. They are sources that are not enabled by default and considered as other sources.

As they are not default sources, they have to be enabled by settings_customise_sources.

Agree, this is not a good design and we need to fix it in the next major version.

I am busy these days and don't have time to work on refactoring for a major version. Also, pydantic-settings V2 is doing great and there is no urgent need for refactoring.

BTW, I will consider these points whenever I want to refactor and work on V3

Thanks!

Okay, thanks for considering this. It might be a good idea to warn the user when they set any of the YAML-related parameters, right? Also, I wonder why the default behaviour for BaseSettings._settings_build_values is to fail silently. The comment mentions that he thinks that this is slightly superior to throwing a descriptive error. Frankly, I cannot see a reason why that might be the case. Do you have an idea?

Also, I tried to find the person with that opinion through the blame, but failed as that information got lost in the refactoring from v1 to v2 (or my git skills are not good enough. That is also very possible (-:)

Okay, thanks for considering this. It might be a good idea to warn the user when they set any of the YAML-related parameters, right? Also, I wonder why the default behaviour for BaseSettings._settings_build_values is to fail silently. The comment mentions that he thinks that this is slightly superior to throwing a descriptive error. Frankly, I cannot see a reason why that might be the case. Do you have an idea?

It might happen for the default sources. Like you might have env_file in your SettingsConfigDict but disable the DotEnvSettingsSource. So we don't check it.
Would be great if you can make a PR and improve it if you are interested.