add_logger_name not working with many logger factories

[Drachenfels]

I guess it's not a bug in itself, more like a clarification needed polite request.

In some places of documentation, there is mention that there is add_logger_name, like here: https://www.structlog.org/en/stable/standard-library.html#processors

But upon adding it to my configuration I suddenly had an error like this:

def add_logger_name(
    logger: logging.Logger, method_name: str, event_dict: EventDict
) -> EventDict:
    """
    Add the logger name to the event dict.
    """
    record = event_dict.get("_record")
    if record is None:
>           event_dict["logger"] = logger.name
E           AttributeError: 'PrintLogger' object has no attribute 'name'

Upon testing a lot of stuff I figured out it's because logger factories I use, depends on context it is:

• None -> defaults to PrintLoggerFactory() [during development]
• structlog.BytesLoggerFactory() [once deployed]

Both of them fail because they do not have name, the only logger factory that works seems to be: structlog.stdlib.LoggerFactory()

It seems this gotcha crops up every now and then (after checking closed issues here on github). Maybe it makes sense to add Note or Warning next to https://www.structlog.org/en/stable/api.html#structlog.stdlib.add_logger_name and maybe even https://www.structlog.org/en/stable/standard-library.html#processors that lists which factories will work with this processor or that most won't work.

That brings me to a question.

I have module in my project:

app/tools/logging.py

import logging
import sys
from typing import Union

import orjson
import structlog


def _configure_logger(log_level: Union[str, int] = "INFO"):
    shared_processors = [
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S", utc=True),
        structlog.dev.set_exc_info,
        structlog.stdlib.add_logger_name,
    ]

    if sys.stderr.isatty():
        processors = shared_processors + [
            structlog.processors.StackInfoRenderer(),
            structlog.dev.ConsoleRenderer(),
        ]
        logger_factory = None
    else:  # pragma: no cover
        processors = shared_processors + [
            structlog.processors.dict_tracebacks,
            structlog.processors.JSONRenderer(serializer=orjson.dumps),  # pylint: disable=no-member
        ]
        logger_factory = structlog.stdlib.BytestLoggerFactory()

    log_level = getattr(logging, log_level.upper()) if isinstance(log_level, str) else log_level

    structlog.configure(
        cache_logger_on_first_use=True,
        wrapper_class=structlog.make_filtering_bound_logger(log_level),
        processors=processors,
        logger_factory=logger_factory,
    )


def get_logger(name):
    if not structlog.is_configured():
        _configure_logger()

    logger = structlog.get_logger(name)

    return logger

Else in the code:

app/main.py

from app.tools.logger import get_logger

logger = get_logger(__name__)

def run():
    logger.info("Hello world!")

What I find unclear is that we provide the name explicitly and as such BytesLoggerFactory could (should?) instantiate logger using it. It seems it does not, value is just swallowed and used just to load configuration (I guess). At this stage, I am confused. We have name = __name__ but no processor can access it unless we use one specific factory: structlog.strdlib.LoggerFactory.

I encountered this issue when I tried to add a processor that logs only every 10th message for 'app.main' only, other loggers run with standard configuration, in order to avoid merciless spamming logs but see some activity.

P.S.
I understand that another way of dealing with every 10th message is perhaps adding a filter to the standard python logging config and allow structlog to consume it this way, would that work?

[Hynek Schlawack]

So firstly: structlog.stdlib.add_logger_name() (as the module name suggests 🤓) only works if you configure structlog to use stdlib logging. Which given you seem to care about performance, you probably won't want to.

IIRC, structlog's own loggers have no concept of logger names, but you can always do: log = structlog.get_logger(module=__name__) to get that name in.

BytesLogger's job is only writing out bytes – nothing else:

structlog/src/structlog/_output.py

Lines 243 to 322 in 752b702

 class BytesLogger: 
     r""" 
     Writes bytes into a file. 
  
     :param file: File to print to. (default: `sys.stdout`\ ``.buffer``) 
  
     Useful if you follow 
     `current logging best practices <logging-best-practices>` together with 
     a formatter that returns bytes (e.g. `orjson 
     <https://github.com/ijl/orjson>`_). 
  
     .. versionadded:: 20.2.0 
     """ 
     __slots__ = ("_file", "_write", "_flush", "_lock") 
  
     def __init__(self, file: BinaryIO | None = None): 
         self._file = file or sys.stdout.buffer 
         self._write = self._file.write 
         self._flush = self._file.flush 
  
         self._lock = _get_lock_for_file(self._file) 
  
     def __getstate__(self) -> str: 
         """ 
         Our __getattr__ magic makes this necessary. 
         """ 
         if self._file is sys.stdout.buffer: 
             return "stdout" 
  
         if self._file is sys.stderr.buffer: 
             return "stderr" 
  
         raise PicklingError( 
             "Only BytesLoggers to sys.stdout and sys.stderr can be pickled." 
         ) 
  
     def __setstate__(self, state: Any) -> None: 
         """ 
         Our __getattr__ magic makes this necessary. 
         """ 
         if state == "stdout": 
             self._file = sys.stdout.buffer 
         else: 
             self._file = sys.stderr.buffer 
  
         self._write = self._file.write 
         self._flush = self._file.flush 
         self._lock = _get_lock_for_file(self._file) 
  
     def __deepcopy__(self, memodict: dict[str, object]) -> BytesLogger: 
         """ 
         Create a new BytesLogger with the same attributes. Similar to pickling. 
         """ 
         if self._file not in (sys.stdout.buffer, sys.stderr.buffer): 
             raise copy.error( 
                 "Only BytesLoggers to sys.stdout and sys.stderr " 
                 "can be deepcopied." 
             ) 
  
         newself = self.__class__(self._file) 
  
         newself._write = newself._file.write 
         newself._flush = newself._file.flush 
         newself._lock = _get_lock_for_file(newself._file) 
  
         return newself 
  
     def __repr__(self) -> str: 
         return f"<BytesLogger(file={self._file!r})>" 
  
     def msg(self, message: bytes) -> None: 
         """ 
         Write *message*. 
         """ 
         with self._lock: 
             until_not_interrupted(self._write, message + b"\n") 
             until_not_interrupted(self._flush) 
  
     log = debug = info = warn = warning = msg 
     fatal = failure = err = error = critical = exception = msg 

[Drachenfels]

@hynek I wanted to say thank you, your response actually put me on track with what I have to do, and as I was wrapping my head around the solution I was thinking about how to improve documentation to avoid similar questions in the future. No idea really, it's truly well said in the documentation that structlog is fiddly if someone makes first-time ever migration to it from an old-school logging solution.