Logo menu close
Contact Me

AWS Logs Insights

Use AWS Logs Insights to make sense of complex logs

The basic parts of a Logs insights query is as follows.

  1. Fields: These are keys of the AWS log message format. There are usually two values (when you look at the AWS console)

    1. timestamp: This is pretty straight forward.
    2. message: The entire logged message in JSON or plain text.
    3. logStream: The CloudWatch Logstream
    4. log: The specific log.
  2. Parse: Parse the text of the message based on certain conditions. See below for details.

  3. Filter: Use this to filter the output.

    For example: filter using timestamp

    filter @timestamp >= timestamp('2024-08-10T00:00:00Z')
    

    Filter using text in the message.

    filter @message like /ERROR|WARN/
    

    Filter using Log stream.

    filter @logStream like <log-stream-pattern>
    
  4. Sort: Sort the output: For example; descending using timestamp

    sort @timestamp desc
    
  5. Limit: Limit the output to the first x results.

    limit 30
    
  6. Display: Display only certain fields

    display @timestamp,level
    

Parsing

Since the entire second field of logs insights is message , we can apply additional queries on the text (or JSON) message.

We’d need to apply a regex to parse the messages out of the logs.

{"level": "info", "grpc.method": "GET"}
| parse @message '"level":"*"' as level

References

  1. General: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html
  2. Querying: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-Parse.html


<- Back to blog