Skip to main content
Time filter is based on UTC.

​
Record Cleanup Script

The following script is used to delete records older than the specified time from the apinizer_log collection stored in MongoDB. This operation optimizes storage space by cleaning the database. 
#!/bin/bash
 
# Information required to connect to MongoDB server
HOST="<MONGODB_MASTER_IP_ADDRESS>"
PORT="<MONGODB_PORT>"
DB_NAME="apinizerdb"
USERNAME="apinizer"
PASSWORD="<PASSWORD>"
AUTH_DB="admin"
TARGET_COLLECTION="apinizer_log"
TARGET_LOG_LOCATION=$(pwd)/purge_apinizer_log_collection.log
 
echo "Script started on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION
 
# Variable indicating how many hours/days ago records will be deleted
TIME_VALUE="1D"  #You can change this value as you wish.
 
# Analyze TIME_VALUE and convert to milliseconds
if [[ $TIME_VALUE =~ [Hh]$ ]]; then
    HOURS_AGO=${TIME_VALUE%H*}
    TIME_IN_MILLISECONDS=$(($HOURS_AGO * 60 * 60 * 1000))
elif [[ $TIME_VALUE =~ [Dd]$ ]]; then
    DAYS_AGO=${TIME_VALUE%D*}
    TIME_IN_MILLISECONDS=$(($DAYS_AGO * 24 * 60 * 60 * 1000))
else
    echo "Invalid time format. Please enter a value ending with H (hour) or D (day)." >> $TARGET_LOG_LOCATION
    exit 1
fi
 
MONGO_COMMANDS=$(cat <<EOF
var dateToRemove=new Date((new Date().getTime() - $TIME_IN_MILLISECONDS));
var bulk = db.getCollection("$TARGET_COLLECTION").initializeUnorderedBulkOp();
bulk.find( {"date":{"\$lt": dateToRemove}}).remove();
bulk.execute();
EOF
)
 
# Execute command and write output to log file
{
    mongosh mongodb://$USERNAME:$PASSWORD@$HOST:$PORT/$DB_NAME --authenticationDatabase $AUTH_DB --eval "$MONGO_COMMANDS"
    # Check if command completed
    if [ $? -ne 0 ]; then
        echo "MongoDB command failed." >> $TARGET_LOG_LOCATION
        exit 1
    fi
} >> $TARGET_LOG_LOCATION 2>&1
 
echo "The command has been executed. Please check the content of the related collection." >> $TARGET_LOG_LOCATION
 
echo "Script finished on $(date +"%Y-%m-%d %H:%M:%S")" >> $TARGET_LOG_LOCATION

​
How It Works

1

Defining MongoDB Connection Information

Information required to connect to MongoDB server is defined in relevant variables: HOST, PORT, DB_NAME, USERNAME, PASSWORD, and AUTH_DB.
2

Determining Time Period

The TIME_VALUE variable specifies how many hours ago records will be deleted. This value can be in hours (H) or days (D).
3

Preparing MongoDB Commands

MongoDB commands are defined in the MONGO_COMMANDS variable to find and delete documents older than the specified time.
4

Executing Commands

Connect to MongoDB server using the mongosh command and execute the commands defined in the MONGO_COMMANDS variable.
5

Checking Results

When the operation is completed, the message “The command has been executed. Please check the content of the related collection.” is printed to the screen.

​
Usage

Before running the script, enter your own information in the MongoDB variables. You can update the TIME_VALUE variable according to your desired hour/day value. For example:
  • To keep documents from the last 3 hours and delete the rest: TIME_VALUE=3H
  • To keep documents from the last 5 days and delete the rest: TIME_VALUE=5D
To run the script on a Linux-based operating system:
  1. Copy the script to a file using a text editor (vi, nano, etc.) and edit the variables inside
  2. Give execute permission to the file: chmod +x purge_apinizer_log_collection.sh
  3. Run the file: ./purge_apinizer_log_collection.sh
If you think the operation will take a long time, you can make it run in the background by adding a space and the & character at the end of the command: 
chmod +x /path/to/purge_apinizer_log_collection.sh
./path/to/purge_apinizer_log_collection.sh &
This operation can be done manually or repeated at certain intervals. To repeat it, this record must be added to linux cronjob settings.

​
CronJob Usage

1

Opening Cron Editor

Open the cron editor by running the following command in the terminal:
crontab -e
2

Adding Cron Job

Add a line to the opened editor according to how often you want to run the script.For example, to run it every day at 23:59:
59 23 * * * /path/to/purge_apinizer_log_collection.sh > /path/to/logfile.log 2>&1
3

Saving

Press Esc and type :wq and press Enter to save the line you added.
Data cleanup operation will be performed according to the time period you specified in the script. Therefore, it is important to ensure that the time period in the script and the trigger time you set with cron are consistent.