add_logger_name not working with many logger factories
Drachenfels opened this issue · comments
I guess it's not a bug in itself, more like a clarification needed polite request.
In some places of documentation, there is mention that there is add_logger_name, like here: https://www.structlog.org/en/stable/standard-library.html#processors
But upon adding it to my configuration I suddenly had an error like this:
def add_logger_name(
logger: logging.Logger, method_name: str, event_dict: EventDict
) -> EventDict:
"""
Add the logger name to the event dict.
"""
record = event_dict.get("_record")
if record is None:
> event_dict["logger"] = logger.name
E AttributeError: 'PrintLogger' object has no attribute 'name'
Upon testing a lot of stuff I figured out it's because logger factories I use, depends on context it is:
- None -> defaults to PrintLoggerFactory() [during development]
- structlog.BytesLoggerFactory() [once deployed]
Both of them fail because they do not have name, the only logger factory that works seems to be: structlog.stdlib.LoggerFactory()
It seems this gotcha crops up every now and then (after checking closed issues here on github). Maybe it makes sense to add Note or Warning next to https://www.structlog.org/en/stable/api.html#structlog.stdlib.add_logger_name and maybe even https://www.structlog.org/en/stable/standard-library.html#processors that lists which factories will work with this processor or that most won't work.
That brings me to a question.
I have module in my project:
app/tools/logging.py
import logging
import sys
from typing import Union
import orjson
import structlog
def _configure_logger(log_level: Union[str, int] = "INFO"):
shared_processors = [
structlog.contextvars.merge_contextvars,
structlog.processors.add_log_level,
structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S", utc=True),
structlog.dev.set_exc_info,
structlog.stdlib.add_logger_name,
]
if sys.stderr.isatty():
processors = shared_processors + [
structlog.processors.StackInfoRenderer(),
structlog.dev.ConsoleRenderer(),
]
logger_factory = None
else: # pragma: no cover
processors = shared_processors + [
structlog.processors.dict_tracebacks,
structlog.processors.JSONRenderer(serializer=orjson.dumps), # pylint: disable=no-member
]
logger_factory = structlog.stdlib.BytestLoggerFactory()
log_level = getattr(logging, log_level.upper()) if isinstance(log_level, str) else log_level
structlog.configure(
cache_logger_on_first_use=True,
wrapper_class=structlog.make_filtering_bound_logger(log_level),
processors=processors,
logger_factory=logger_factory,
)
def get_logger(name):
if not structlog.is_configured():
_configure_logger()
logger = structlog.get_logger(name)
return logger
Else in the code:
app/main.py
from app.tools.logger import get_logger
logger = get_logger(__name__)
def run():
logger.info("Hello world!")
What I find unclear is that we provide the name explicitly and as such BytesLoggerFactory could (should?) instantiate logger using it. It seems it does not, value is just swallowed and used just to load configuration (I guess). At this stage, I am confused. We have name = __name__
but no processor can access it unless we use one specific factory: structlog.strdlib.LoggerFactory.
I encountered this issue when I tried to add a processor that logs only every 10th message for 'app.main' only, other loggers run with standard configuration, in order to avoid merciless spamming logs but see some activity.
P.S.
I understand that another way of dealing with every 10th message is perhaps adding a filter to the standard python logging config and allow structlog to consume it this way, would that work?
So firstly: structlog.stdlib.add_logger_name()
(as the module name suggests 🤓) only works if you configure structlog to use stdlib logging. Which given you seem to care about performance, you probably won't want to.
IIRC, structlog's own loggers have no concept of logger names, but you can always do: log = structlog.get_logger(module=__name__)
to get that name in.
BytesLogger's job is only writing out bytes – nothing else:
structlog/src/structlog/_output.py
Lines 243 to 322 in 752b702
@hynek I wanted to say thank you, your response actually put me on track with what I have to do, and as I was wrapping my head around the solution I was thinking about how to improve documentation to avoid similar questions in the future. No idea really, it's truly well said in the documentation that structlog is fiddly if someone makes first-time ever migration to it from an old-school logging solution.