How to Query CloudWatch Logs Using Boto3 in Python?

To query AWS CloudWatch Logs in Python use the boto3 library with start_query and get_query_results. This is useful for getting log data programmatically.

Summarize with AI

Not enough time? get the key points instantly.

Amazon CloudWatch allows you to run log insights queries using the logs client in Boto3. Below is a step by step guide to querying logs using Boto3 in Python development.

Step 1: Install Boto3 (if not already)

Install the required AWS SDK package for Python to interact with CloudWatch.

bash

pip install boto3

Step 2: Set Up the Boto3 Client

Create a CloudWatch Logs client by specifying the correct AWS region.

python

import boto3client = boto3.client(“logs”, region_name=”us-east-1″) # Adjust region as needed

Step 3: Start the Query

Define the log group, time range and query string using start_query.

python

from datetime import datetime, timedelta

log_group = “/aws/lambda/my-function” # Replace with your log group

query = “fields @timestamp, @message | sort @timestamp desc | limit 10”

start_time = int((datetime.utcnow() – timedelta(hours=1)).timestamp())

end_time = int(datetime.utcnow().timestamp())

 

response = client.start_query(

    logGroupName=log_group,

    startTime=start_time,

    endTime=end_time,

    queryString=query

)

 

query_id = response[‘queryId’]

Also Read: Using Python For Finance

Step 4: Wait and Retrieve Results

Poll get_query_results until the query status is Complete.

python

import time

while True:

    result = client.get_query_results(queryId=query_id)

    if result[“status”] == “Complete”:

        break

    time.sleep(1)

 

# Print formatted results

for row in result[“results”]:

    print({field[‘field’]: field[‘value’] for field in row})

Also Read: Top Python Machine Learning Libraries

Tip

Use short time ranges and smaller limits to speed up queries. Also make sure your IAM role has logs:StartQuery and logs:GetQueryResults permissions.


Resources

Insights from our engineering leaders.

Our team shares practical insights on the technologies and engineering decisions shaping enterprise software today, grounded in real client work.

Python

How to Copy Files in Python

You can copy files in python using the shutil module which provides high level file and collection of files operations. You can also combine it…

Python

OpenCV Kalman Filter with Python

The Kalman Filter is an efficient recursive algorithm used to estimate the state of a system from noisy measurements. In computer vision, it’s commonly used…