AWS Logs Insights
Use AWS Logs Insights to make sense of complex logs
The basic parts of a Logs insights query is as follows.
Fields: These are
keys
of the AWS log message format. There are usually two values (when you look at the AWS console)timestamp
: This is pretty straight forward.message
: The entire logged message in JSON or plain text.logStream
: The CloudWatch Logstreamlog
: The specific log.
Parse: Parse the text of the message based on certain conditions. See below for details.
Filter: Use this to filter the output.
For example: filter using timestamp
filter @timestamp >= timestamp('2024-08-10T00:00:00Z')
Filter using text in the message.
filter @message like /ERROR|WARN/
Filter using Log stream.
filter @logStream like <log-stream-pattern>
Sort: Sort the output: For example; descending using timestamp
sort @timestamp desc
Limit: Limit the output to the first x results.
limit 30
Display: Display only certain fields
display @timestamp,level
Parsing
Since the entire second field of logs insights is message
, we can apply additional queries on the text (or JSON) message.
We’d need to apply a regex to parse the messages out of the logs.
{"level": "info", "grpc.method": "GET"}
| parse @message '"level":"*"' as level
References
- General: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html
- Querying: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-Parse.html