Every syslog message starts with the same <PRI>, but the text after it comes in two very different formats, and a lot of parsing pain comes from not knowing which you have.
RFC 3164: the BSD (Berkeley Software Distribution) format
The original format was documented, after the fact, in RFC 3164. It is loose: after the (priority value) comes a timestamp like Oct 1 14:23:05, a hostname, and then a free-form tag and message. The weaknesses are real:
- The timestamp has no year and no timezone, so a stored message is ambiguous about when it actually happened.
- The structure is only loosely specified, so the boundary between the tag and the message varies by sender.
- It predates any notion of structured fields, so everything interesting is buried in free text.
It is still extremely common on network gear and older systems, which is why parsers must handle it, quirks and all.
RFC 5424: the modern format
RFC 5424 replaced it with a precise, parseable structure: <PRI> then a version number, an ISO 8601 timestamp with timezone and fractional seconds, then hostname, app-name, procid, and msgid as distinct fields, optional structured data (typed key-value elements in brackets), and finally the message, which is UTF-8 with a byte-order mark when needed. The gains matter: unambiguous time, clear fields, machine-readable structured data, and proper Unicode.
Telling them apart and why it matters
The quickest tell is right after the PRI: a digit and a space (the version, almost always 1) means RFC 5424; a three-letter month means RFC 3164. This matters because the two carry different information. A 3164 message cannot tell you the year or timezone, so a collector has to add them and may get them wrong across a reboot or a zone change. A 5424 message carries that itself. When timestamps look wrong or fields land in the wrong place, the format mismatch is usually why, and modern deployments prefer 5424 precisely to avoid that ambiguity.
Two formats, and parsers that guess between them
RFC 3164 and RFC 5424 are different enough to break a parser and similar enough that one will often accept the other and produce nonsense — a timestamp read as a hostname, a structured-data block treated as free text.
The practical hazard is that the wrong-format parse usually succeeds. Fields land in the wrong columns, the message is indexed, and the search that would have found it looks in a field that never received the value. The event is present and unfindable, which during an investigation is worse than absent.
Pin the format explicitly at both ends rather than relying on detection, and verify by searching for a known test event rather than by confirming that lines are arriving.