Clearing the notification_user Collection

The notification_user  collection, there are records of user notifications stored.

Record Cleaning Script:

The following script is used to delete records from the notification_user collection in MongoDB that are older than the specified hour.

This operation helps optimize storage space by cleaning up the database.

#!/bin/bash
 
# Information required to connect to the MongoDB server
HOST="<MONGODB_MASTER_IP_ADDRESS>"
PORT="<MONGODB_PORT>"
DB_NAME="apinizerdb"
USERNAME="apinizer"
PASSWORD="<PASSWORD>"
AUTH_DB="admin"
TARGET_COLLECTION="notification_user"
TARGET_LOG_LOCATION=$(pwd)/purge_apinizer_log_collection.log
 
echo "Script started on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION
 
#The variable that specifies how many hours/days ago documents will be deleted up to.
TIME_VALUE="1D"  #You can change this value as you wish.

# TIME_VALUE analyzing the value and converting it to milliseconds.
if [[ $TIME_VALUE =~ [Hh]$ ]]; then
    HOURS_AGO=${TIME_VALUE%H*}
    TIME_IN_MILLISECONDS=$(($HOURS_AGO * 60 * 60 * 1000))
elif [[ $TIME_VALUE =~ [Dd]$ ]]; then
    DAYS_AGO=${TIME_VALUE%D*}
    TIME_IN_MILLISECONDS=$(($DAYS_AGO * 24 * 60 * 60 * 1000))
else
    echo "Invalid time format. Please enter a value ending with H (hour) or D (day)." >> $TARGET_LOG_LOCATION
    exit 1
fi
 
MONGO_COMMANDS=$(cat <<EOF
var dateToRemove=new Date((new Date().getTime() - $TIME_IN_MILLISECONDS));
var bulk = db.getCollection("$TARGET_COLLECTION").initializeUnorderedBulkOp();
bulk.find( {"createdTime":{"\$lt": dateToRemove}}).remove();
bulk.execute();
EOF
)
 
# Run the command and write its output to a log file.
{
    mongosh mongodb://$USERNAME:$PASSWORD@$HOST:$PORT/$DB_NAME --authenticationDatabase $AUTH_DB --eval "$MONGO_COMMANDS"
    # Check the completion of the command.
    if [ $? -ne 0 ]; then
        echo "MongoDB command failed." >> $TARGET_LOG_LOCATION
        exit 1
    fi
} >> $TARGET_LOG_LOCATION 2>&1
 
echo "The command has been executed. Please check the content of the related collection." >> $TARGET_LOG_LOCATION
 
echo "Script finished on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION

The time filter is applied based on UTC.

How It Works:

1. The necessary information to connect to the MongoDB server is defined in the respective variables: HOST, PORT, DB_NAME, USERNAME, PASSWORD, and AUTH_DB.
2. The TIME_VALUE variable indicates how many hours ago records will be deleted up to. This value can be in hours (H) or days (D).
3. The MONGO_COMMANDS variable defines MongoDB commands to find and delete documents up to the specified time value.
4. The mongosh command is used to connect to the MongoDB server and execute the commands defined in the MONGO_COMMANDS variable.
5. When the process is completed, the message "The command has been executed. Please check the content of the related collection." is displayed on the screen.

Usage:

• Before running the script, please enter your own information into the MongoDB variables. You can update the TIME_VALUE variable according to the desired hour/day value. For example, to retain documents from the last 3 hours and delete the rest, set TIME_VALUE=3H. To keep documents from the last 5 days and delete the rest, set TIME_VALUE=5D.
• To run the script on a Linux-based operating system, the script is first copied into a file using a text editor (such as vi, nano, etc.), and then the variables inside are edited.
• The command "chmod +x purge_apinizer_log_collection.sh "grants execution permission to the file.
• The file is executed by typing "./purge_apinizer_log_collection.sh"
• If you anticipate a lengthy operation, you can append a space followed by the '&' character to run it in the background: ./purge_apinizer_log_collection.sh &

chmod +x /path/to/purge_apinizer_log_collection.sh
./path/to/purge_apinizer_log_collection.sh &

CronJob Usage:

1) Open the cron editor by running the following command in the terminal:

crontab -e

2) In the opened editor, add a line based on how frequently you want to run the script.

For example, to run it every day at 23:59, you can write:

59 23 * * * /path/to/purge_anomaly_detector_result_collection.sh > /path/to/logfile.log 2>&1

To save the added line, press the Esc key, type :wq, and press the Enter key.

Reminder

The data clearing process in the script will occur according to the time zone specified within the script.

Therefore, it is important to ensure consistency between the time zone in the script and the triggering time set with cron.