About Me

My photo
Rohit is an investor, startup advisor and an Application Modernization Scale Specialist working at Google.

Tuesday, June 14, 2016

Multi-line Java stack traces out of order in Logstash and Splunk

Do you have customers frustrated by getting multi-line Java stack traces out of order? We're working on a clean solution in our enhanced metrics work, but here is a workaround courtesy @DavidLaing .
With the Java Logback library you can do this by adding 

"%replace(%xException){'\n','\u2028'}%nopex" 

to your logging config [1] , and then use the following logstash conf.[2]
Replace the unicode newline character \u2028 with \n, which Kibana will display as a new line.


mutate {
  gsub => [ "[@message]", '\u2028', "
"]
^^^ Seems that passing a string with an actual newline in it is the only way to make gsub work
}