Cleaning audit_event Collection

uyarı

Since all changes made by people on Apinizer are kept in the audit_event collection, we absolutely do not recommend interfering with this collection.

uyarı

Automatic purge via Purge Jobs has been removed for this collection. Use this manual script only if absolutely necessary.

uyarı

Time filter is based on UTC.

Record Cleanup Script

The following script is used to delete records older than the specified time from the audit_event collection stored in MongoDB. This operation optimizes storage space by cleaning the database.

#!/bin/bash
 
# Information required to connect to MongoDB server
HOST="<MONGODB_MASTER_IP_ADDRESS>"
PORT="<MONGODB_PORT>"
DB_NAME="apinizerdb"
USERNAME="apinizer"
PASSWORD="<PASSWORD>"
AUTH_DB="admin"
TARGET_COLLECTION="audit_event"
TARGET_LOG_LOCATION=$(pwd)/purge_audit_event_collection.log
 
echo "Script started on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION  
 
# Variable indicating how many hours/days ago records will be deleted
TIME_VALUE="1D"  #You can change this value as you wish.
 
# Analyze TIME_VALUE and convert to milliseconds
if [[ $TIME_VALUE =~ [Hh]$ ]]; then
    HOURS_AGO=${TIME_VALUE%H*}
    TIME_IN_MILLISECONDS=$(($HOURS_AGO * 60 * 60 * 1000))
elif [[ $TIME_VALUE =~ [Dd]$ ]]; then
    DAYS_AGO=${TIME_VALUE%D*}
    TIME_IN_MILLISECONDS=$(($DAYS_AGO * 24 * 60 * 60 * 1000))
else
    echo "Invalid time format. Please enter a value ending with H (hour) or D (day)." >> $TARGET_LOG_LOCATION
    exit 1
fi
 
MONGO_COMMANDS=$(cat <<EOF
var dateToRemove=new Date((new Date().getTime() - $TIME_IN_MILLISECONDS));
var bulk = db.getCollection("$TARGET_COLLECTION").initializeUnorderedBulkOp();
bulk.find( {"auditEventDate":{"\$lt": dateToRemove}}).remove();
bulk.execute();
EOF
)
 
# Execute command and write output to log file
{
    mongosh mongodb://$USERNAME:$PASSWORD@$HOST:$PORT/$DB_NAME --authenticationDatabase $AUTH_DB --eval "$MONGO_COMMANDS"
    # Check if command completed
    if [ $? -ne 0 ]; then
        echo "MongoDB command failed." >> $TARGET_LOG_LOCATION
        exit 1
    fi
} >> $TARGET_LOG_LOCATION 2>&1
 
echo "The command has been executed. Please check the content of the related collection." >> $TARGET_LOG_LOCATION
 
echo "Script finished on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION

How It Works

Defining MongoDB Connection Information

Information required to connect to MongoDB server is defined in relevant variables: HOST, PORT, DB_NAME, USERNAME, PASSWORD, and AUTH_DB.

Determining Time Period

The TIME_VALUE variable specifies how many hours ago records will be deleted. This value can be in hours (H) or days (D).

Preparing MongoDB Commands

MongoDB commands are defined in the MONGO_COMMANDS variable to find and delete documents older than the specified time.

Executing Commands

Connect to MongoDB server using the mongosh command and execute the commands defined in the MONGO_COMMANDS variable.

Checking Results

When the operation is completed, the message "The command has been executed. Please check the content of the related collection." is printed to the screen.

Usage

Before running the script, enter your own information in the MongoDB variables. You can update the TIME_VALUE variable according to your desired time period. For example, to keep documents from the last 3 days and delete the rest: TIME_VALUE="3D".

To run the script on a Linux-based operating system:

1. Copy the script to a file using a text editor (vi, nano, etc.) and edit the variables inside
2. Give execute permission to the file: chmod +x purge_audit_event_collection.sh
3. Run the file: ./purge_audit_event_collection.sh

If you think the operation will take a long time, you can make it run in the background by adding a space and the & character at the end of the command:

chmod +x /path/to/purge_audit_event_collection.sh
./path/to/purge_audit_event_collection.sh &

This operation can be done manually or repeated at certain intervals. To repeat it, this record must be added to linux cronjob settings.

CronJob Usage

Opening Cron Editor

Open the cron editor by running the following command in the terminal:

crontab -e

Adding Cron Job

Add a line to the opened editor according to how often you want to run the script.

For example, to run it every day at 23:59:

59 23 * * * /path/to/purge_audit_event_collection.sh > /path/to/logfile.log 2>&1

Saving

Press Esc and type :wq and press Enter to save the line you added.

bilgi

Data cleanup operation will be performed according to the time period you specified in the script. Therefore, it is important to ensure that the time period in the script and the trigger time you set with cron are consistent.