Distributed Logging Architecture in the Container Era

  • Logging is a cross-cutting concern in any application. For distributed application, it's better to have shared logging technology across all the services. Log aggregators are a solution for polyglot systems as they have connectors to most languages.
  • Logging infrastructure must be searchable. What's the point of logging everything without being able to answer queries like "Which service throws the most errors?"
  • Using a single correlation ID across services allows you to filter log messages from all the sources. This makes debugging a lot easier if you can view all the code paths that a request touched.
  • Include a lot more context in each log message. Data such as username, service name, timestamp etc are very handy to scan through logs quickly.
  • Network failure is an inevitability when dealing with log aggregators. Some of the ways to handle this are using local disk to log or a fallback service.
  • With privacy and GDPR in full force, be careful to not log personally identifiable information.

Full Post here, 9 mins read