Skip to main content
Time filter is based on UTC.
When Trace is enabled; collections named log_apiproxytraffic, log_apiproxytraffic_policy_execution, log_apiproxytraffic_routing record API traffic details. Even after Trace is disabled, this data continues to be kept in the database. Thanks to these records, in the trace screen, you can examine the state before and after policy(ies) are applied on the relevant proxy, traffic information, and routing information.

​
Record Cleanup Script

The following script is used to delete records older than the specified time from the log_apiproxytraffic, log_apiproxytraffic_policy_execution, log_apiproxytraffic_routing collections stored in MongoDB. This operation optimizes storage space by cleaning the database. 
#!/bin/bash
 
# Information required to connect to MongoDB server
HOST="<MONGODB_MASTER_IP_ADDRESS>"
PORT="<MONGODB_PORT>"
DB_NAME="apinizerdb"
USERNAME="apinizer"
PASSWORD="<PASSWORD>"
AUTH_DB="admin"
TARGET_COLLECTIONS=("log_apiproxytraffic_policy_execution" "log_apiproxytraffic" "log_apiproxytraffic_routing")
TARGET_LOG_LOCATION=$(pwd)/purge_trace_logs.log
 
echo "Script started on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION  
 
# Variable indicating how many hours/days ago records will be deleted
TIME_VALUE="1D"  #You can change this value as you wish.
 
QUERY_CONDITION=""
 
# Analyze TIME_VALUE and convert to milliseconds
if [[ ! -z $TIME_VALUE ]]; then
    if [[ $TIME_VALUE =~ [Hh]$ ]]; then
        HOURS_AGO=${TIME_VALUE%H*}
        TIME_IN_MILLISECONDS=$(($HOURS_AGO * 60 * 60 * 1000))
        QUERY_CONDITION="\"timestamp\": {\"\$lt\": new Date((new Date().getTime() - $TIME_IN_MILLISECONDS))}"
    elif [[ $TIME_VALUE =~ [Dd]$ ]]; then
        DAYS_AGO=${TIME_VALUE%D*}
        TIME_IN_MILLISECONDS=$(($DAYS_AGO * 24 * 60 * 60 * 1000))
        QUERY_CONDITION="\"timestamp\": {\"\$lt\": new Date((new Date().getTime() - $TIME_IN_MILLISECONDS))}"
    else            
        echo "Invalid time format. Please enter a value ending with H (hour) or D (day)." >> $TARGET_LOG_LOCATION
         exit 1
    fi
fi
 
if [[ -z $QUERY_CONDITION ]]; then
    echo "TIME_VALUE is not valid." >> $TARGET_LOG_LOCATION
    exit 1
fi
 
for COLLECTION in "${TARGET_COLLECTIONS[@]}"; do
MONGO_COMMANDS=$(cat <<EOF
var bulk = db.getCollection("$COLLECTION").initializeUnorderedBulkOp();
bulk.find({$QUERY_CONDITION}).remove();
bulk.execute();
EOF
)
 
    # Execute command and write output to log file
    {
        mongosh mongodb://$USERNAME:$PASSWORD@$HOST:$PORT/$DB_NAME --authenticationDatabase $AUTH_DB --eval "$MONGO_COMMANDS"
        # Check if command completed
        if [ $? -ne 0 ]; then
            echo "MongoDB command for collection $COLLECTION failed." >> $TARGET_LOG_LOCATION
            exit 1
        fi
    } >> $TARGET_LOG_LOCATION 2>&1
done  
 
echo "The command has been executed. Please check the content of the related collection." >> $TARGET_LOG_LOCATION
  
echo "Script finished on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION

​
How It Works

1

Defining MongoDB Connection Information

Information required to connect to MongoDB server is defined in relevant variables: HOST, PORT, DB_NAME, USERNAME, PASSWORD, and AUTH_DB.
2

Determining Time Period

The TIME_VALUE variable specifies how many hours ago records will be deleted. This value can be in hours (H) or days (D).
3

Looping Over Collections

The script loops over three collections (log_apiproxytraffic_policy_execution, log_apiproxytraffic, log_apiproxytraffic_routing) and performs cleanup for each.
4

Preparing and Executing MongoDB Commands

MongoDB commands are defined and executed for each collection to find and delete documents older than the specified time.
5

Checking Results

When the operation is completed, the message “The command has been executed. Please check the content of the related collection.” is printed to the screen.

​
Usage

Before running the script, enter your own information in the MongoDB variables. You can update the TIME_VALUE variable according to your desired hour/day value. For example:
  • To keep documents from the last 3 hours and delete the rest: TIME_VALUE=3H
  • To keep documents from the last 5 days and delete the rest: TIME_VALUE=5D
To run the script on a Linux-based operating system:
  1. Copy the script to a file using a text editor (vi, nano, etc.) and edit the variables inside
  2. Give execute permission to the file: chmod +x purge_trace_logs.sh
  3. Run the file: ./purge_trace_logs.sh
If you think the operation will take a long time, you can make it run in the background by adding a space and the & character at the end of the command: 
chmod +x /path/to/purge_trace_logs.sh
./path/to/purge_trace_logs.sh &
This operation can be done manually or repeated at certain intervals. To repeat it, this record must be added to linux cronjob settings.

​
CronJob Usage

1

Opening Cron Editor

Open the cron editor by running the following command in the terminal:
crontab -e
2

Adding Cron Job

Add a line to the opened editor according to how often you want to run the script.For example, to run it every day at 23:59:
59 23 * * * /path/to/purge_trace_logs.sh > /path/to/logfile.log 2>&1
3

Saving

Press Esc and type :wq and press Enter to save the line you added.
Data cleanup operation will be performed according to the time period you specified in the script. Therefore, it is important to ensure that the time period in the script and the trigger time you set with cron are consistent.