Candidate: Jian Chen Zhao
Date: November 24, 2026
Time: 11:30 AM
Location: Online
Supervisor: Weiyi Shang
All are welcome!
Abstract:
Logs generated by software during runtime contain valuable information about the running system; however, logs are a mixture of natural language and semi-structured content, which requires them to be preprocessed such that the information can be extracted. Log parsing is one of such preprocessing techniques that have been adopted both in research and in practice. Log parsing turns raw log messages into identifiable event templates. Many parsing techniques have been proposed and compete to achieve the highest performance on common benchmarks. This talk presents two studies that question some of the assumptions behind those benchmarks.
In a first study we look at what counts as a log message. Benchmarks treat each line as one message, but console logs often contain messages that span multiple lines. In a qualitative study of console logs, we identify four structural types and ten common formats of multi-line segments. Only 3% of log messages are multiline, but they account for 66% of the content by line count. We propose a deep-learning preprocessing step to first split logs into structurally consistent segments and at the same time labels their format. It reaches an F1 score of 0.88 for segmentation and 0.84 for format classification.
In following studies, we look at what counts as a correct template. Across three surveys with 37 practitioners, we find no reliable agreement on which parser's templates to prefer (Krippendorff's α = 0.22) or on template granularity (α = 0.29). Individual respondents also show no consistent preference. Their choices became less random only when we showed them logs that share a context (normalized entropy dropped from 0.84 to 0.59). Respondents said they chose templates by picturing a specific downstream task.
Together, these results suggest that a log has no single right template, and that parsers should be evaluated by how well they serve downstream tasks, such as compression and anomaly detection, rather than by agreement with labeled templates.