EFFICIENCY GUIDED ADAPTIVE LOG PARSING USING LARGE LANGUAGE MODEL

Log data contains critical information about the runtime behavior of software services, making it highly valuable for research and application. Log parsing, as a core step in the log processing workflow, converts semi-structured data into structured data, significantly enhancing the analysis and utilization of log information. However, existing log parsing methods based on large language models face challenges such as cold start difficulties and inefficiency. The efficiency-guided adaptive parsing (EGAP) method introduced an online optimization strategy using large language models, based on traditional parsing methods, to effectively improve parsing accuracy and efficiency. EGAP used a template caching mechanism for rapid log template matching and employed an efficiency estimation mechanism to dynamically control the use of large language models, ensuring a balance between efficiency and accuracy in the parsing process. Experimental results demonstrate that EGAP significantly enhances log parsing accuracy while substantially improving parsing efficiency.