The user_token_blackList  collection contains records of users who attempt multiple logins.


Record Cleaning Script:

The following script is used to delete records from the user_token_blackList collection in MongoDB that are older than the specified hour.

This operation helps optimize storage space by cleaning up the database.

#!/bin/bash
 
# The necessary information to connect to a MongoDB server.
HOST="<MONGODB_MASTER_IP_ADDRESS>"
PORT="<MONGODB_PORT>"
DB_NAME="apinizerdb"
USERNAME="apinizer"
PASSWORD="<PASSWORD>"
AUTH_DB="admin"
TARGET_COLLECTION="user_token_blackList"
TARGET_LOG_LOCATION=$(pwd)/purge_apinizer_log_collection.log
 
echo "Script started on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION
 
# The variable that determines how many hours/days ago documents will be deleted. 
TIME_VALUE="1D"  #You can change this value as you wish.

# TIME_VALUE analyzing the value and converting it into milliseconds.
if [[ $TIME_VALUE =~ [Hh]$ ]]; then
    HOURS_AGO=${TIME_VALUE%H*}
    TIME_IN_MILLISECONDS=$(($HOURS_AGO * 60 * 60 * 1000))
elif [[ $TIME_VALUE =~ [Dd]$ ]]; then
    DAYS_AGO=${TIME_VALUE%D*}
    TIME_IN_MILLISECONDS=$(($DAYS_AGO * 24 * 60 * 60 * 1000))
else
    echo "Invalid time format. Please enter a value ending with H (hour) or D (day)." >> $TARGET_LOG_LOCATION
    exit 1
fi
 
MONGO_COMMANDS=$(cat <<EOF
var dateToRemove=new Date((new Date().getTime() - $TIME_IN_MILLISECONDS));
var bulk = db.getCollection("$TARGET_COLLECTION").initializeUnorderedBulkOp();
bulk.find( {"createdDate":{"\$lt": dateToRemove}}).remove();
bulk.execute();
EOF
)
 
# Run the command and write its output to a log file.
{
    mongosh mongodb://$USERNAME:$PASSWORD@$HOST:$PORT/$DB_NAME --authenticationDatabase $AUTH_DB --eval "$MONGO_COMMANDS"
    # Check the completion of the command.
    if [ $? -ne 0 ]; then
        echo "MongoDB command failed." >> $TARGET_LOG_LOCATION
        exit 1
    fi
} >> $TARGET_LOG_LOCATION 2>&1
 
echo "The command has been executed. Please check the content of the related collection." >> $TARGET_LOG_LOCATION
 
echo "Script finished on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION
CODE

The time filter is applied based on UTC.

How It Works:

  1. The necessary information for connecting to the MongoDB server is defined in the relevant variables: HOST, PORT, DB_NAME, USERNAME, PASSWORD, and AUTH_DB.
  2. The TIME_VALUE variable specifies how many hours ago records will be deleted up to. This value can be in hours (H) or days (D).
  3. MONGO_COMMANDS  variable defines MongoDB commands to find and delete documents up to the specified time value.
  4. mongosh command is used to connect to the MongoDB server and execute the commands defined in the MONGO_COMMANDS variable.
  5. When the process is completed, the message "The command has been executed. Please check the content of the related collection."  is printed on the screen.

Usage:

  • Before running the script, please enter your own information into the MongoDB variables. You can update the TIME_VALUE variable according to the desired hour/day value. For instance, to retain documents from the last 3 hours and delete the rest, set TIME_VALUE=3H. To keep documents from the last 5 days and delete the rest, set TIME_VALUE=5D.
  • The sentence "The script is copied into a file using a text editor (such as vi, nano, etc.) and the variables inside it are edited." would be a suitable translation.
  • The file is given execution permission using the command: chmod +x purge_apinizer_log_collection.sh
  • The file is executed using the command: ./purge_apinizer_log_collection.sh

  • If you anticipate a lengthy operation, you can append a space followed by the '&' character to run it in the background. ./purge_apinizer_log_collection.sh &

chmod +x /path/to/purge_apinizer_log_collection.sh
./path/to/purge_apinizer_log_collection.sh &
CODE

CronJob Usage:

1) Open the cron editor by running the following command in the terminal:

crontab -e
CODE


2) In the opened editor, add a line based on how frequently you want to run the script.

For example, to run it every day at 23:59, you can write:

59 23 * * * /path/to/purge_anomaly_detector_result_collection.sh > /path/to/logfile.log 2>&1
CODE

To save the added line, press the Esc key, type :wq, and press the Enter key.


Reminder

The data clearing process in the script will occur according to the time zone specified within the script.

Therefore, it is important to ensure consistency between the time zone in the script and the triggering time set with cron.