Skip to content

Commit

Permalink
updates
Browse files Browse the repository at this point in the history
  • Loading branch information
alexanderbazhenoff committed Jan 7, 2024
1 parent 8e39e18 commit d99ffae
Show file tree
Hide file tree
Showing 14 changed files with 95 additions and 139 deletions.
16 changes: 16 additions & 0 deletions backup/bareos_pool_operations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,22 @@ and run them via a Bareos Admin job without parameters pass:
Command = "/etc/bareos/bareos-dir.d/my_wrapper_script.sh"
```

### autoclean_job_script_wrapper.sh

Latest versions of Bareos [can't parse arguments](https://bugs.bareos.org/view.php?id=1587) in the job config `Command`
like `--arg1 param1 --arg2 param2`. So you can put all required arguments inside the script and use them as a
wrapper:

```text
Job {
...
RunScript {
...
Command = "/etc/bareos/bareos-dir.d/autoclean_job_script_wrapper.sh"
}
}
```

### batch_process_bareos_volumes.sh

Common-usage and the most multifunctional script for Bareos pool and volumes troubleshooting.
Expand Down
6 changes: 6 additions & 0 deletions backup/bareos_pool_operations/autoclean_job_script_wrapper.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash


# A wrapper for autoclean script because of new Bareos bug: you can't pass arguments like:
# --arg1 param1 --arg2 param2. See: https://bugs.bareos.org/view.php?id=1587
/./etc/bareos/bareos-dir.d/clean_expired_bareos_volumes.sh --action delete --expire 60 --name Full-
15 changes: 10 additions & 5 deletions backup/bareos_pool_operations/clean_expired_bareos_volumes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Clean expired volumes from Bareos storage pool script.
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024, Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -38,15 +38,15 @@

# Requirements:
# * permissions to run 'bconsole' command and access to $pool_path
# (don't mind if you run this script from bareos Admin Job, otherwise - you should
# (don't mind if you run this script from Bareos Admin Job, otherwise - you should
# edit /etc/sudoers or run from root)

# Usage:
# # ./clean_expired_bareos_volumes.sh --name Full- --action delete --expire 10 --filter Pruned
# or
# # ./clean_expired_bareos_volumes.sh --help
#
# Use "--test yes" key for testmode (only output, no actions).
# Use "--test yes" key for test mode (only output, no actions).
# For more information about volume status read Bareos manual:
# * http://doc.bareos.org/master/html/bareos-manual-main-reference.html

Expand All @@ -55,8 +55,8 @@
# Leave empty if you don't wish additional log file
LOG_PATH=""

# Path of the pool
POOL_PATH="/mnt/pool_path"
# Path of the pool, e.g.: /mnt/pool_path
POOL_PATH="/mnt/backup"


# Process volume function. In test mode prints only command (use for debug).
Expand Down Expand Up @@ -116,6 +116,11 @@ while [[ $# -gt 0 ]]; do
shift
shift
;;
-a | --action )
POOL_ACTION="$2"
shift
shift
;;
-e | --expiration )
POOL_EXPIRE="$2"
shift
Expand Down
10 changes: 6 additions & 4 deletions backup/bareos_pool_operations/clean_missing_volumes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Physically delete non-existent volumes from Pool in the Bareos database.
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024, Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,11 +35,13 @@
# that you know what you're doing. All actions with this script at your own risk.


# Set Pool path
POOL_PATH="/mnt/pool_path"
# Set Pool path, e.g.: /mnt/pool_path
POOL_PATH="/mnt/backup"

cd $POOL_PATH || exit
cd $POOL_PATH || exit 1
FILELIST=$(find . -maxdepth 1 -type f -printf "%f\n")
[[ -z $FILELIST ]] && echo "Nothing to do."

for I in $FILELIST; do
echo "list volume=$I" | bconsole | if grep --quiet "No results to list"; then
echo "$I is ready to be deleted"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Delete all volumes from Bareos storage pool (Bareos with MySQL installation).
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024, Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,8 +35,11 @@
# that you know what you're doing. All actions with this script at your own risk.


# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full"

VOLUMES=$(mysql -u root -B -e'select VolumeName from Media order by VolumeName;' bareos | tail -n+2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit

echo "This will delete all volumes in ${POOL_NAME}. Sleep 10 for sure."
sleep 10
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Delete all volumes from Bareos storage pool (Bareos installation with PostgreSQL).
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024, Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,11 +35,14 @@
# that you know what you're doing. All actions with this script at your own risk.


POOL_NAME="Incremental-"
# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full-"

PWD_R=$(pwd)
cd /var/lib/postgresql || exit 1
VOLUMES=$(sudo -u postgres -H -- psql -d bareos -c "SELECT volumename FROM media ORDER BY volumename" | tail -n+3 | \
head -n -2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit
cd "$PWD_R" || exit 1

echo "This will delete all volumes in ${POOL_NAME}. Sleep 10 for sure."
Expand Down
4 changes: 3 additions & 1 deletion backup/bareos_pool_operations/delete_range_of_volumes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,12 @@
# that you know what you're doing. All actions with this script at your own risk.


VOLUME_NAME="Full-"


echo "WARNING! This will remove selected range of volumes in pool."
echo "Sleep 10 for sure."
sleep 10
VOLUME_NAME="Full-"

for I in {8702..8771}
do
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# prune all volumes from pool (Bareos installation with MySQL).
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024. Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,8 +35,11 @@
# that you know what you're doing. All actions with this script at your own risk.


POOL_NAME="Incremental"
# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full"

VOLUMES=$(mysql -u root -B -e'select VolumeName from Media order by VolumeName;' bareos | tail -n+2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit

echo "This will prune all volumes in $POOL_NAME. Sleep 30 for sure."
sleep 30
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# prune all volumes from pool (Bareos installation with PostgreSQL)
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024. Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,11 +35,14 @@
# that you know what you're doing. All actions with this script at your own risk.


# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full"

PWD_R=$(pwd)
cd /var/lib/postgresql || exit 1
VOLUMES=$(sudo -u postgres -H -- psql -d bareos -c "SELECT volumename FROM media ORDER BY volumename" | tail -n+3 | \
head -n -2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit
cd "$PWD_R" || exit 1

echo "This will prune all volumes in $POOL_NAME. Sleep 30 for sure."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Set all volumes in the pool to "purged" state (Bareos MySQL installation).
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024. Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -35,10 +35,11 @@
# that you know what you're doing. All actions with this script at your own risk.


# Set your pool name here
POOL_NAME="Incremental"
# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full"

VOLUMES=$(mysql -u root -B -e'select VolumeName from Media order by VolumeName;' bareos | tail -n+2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit

echo "This will purge all volumes in $POOL_NAME. Sleep 30 for sure."
sleep 30
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


# Set all volumes in the pool to "purged" state (Bareos PostgreSQL installation).
# Copyright (c) December, 2018. Aleksandr Bazhenov. Updated December, 2021.
# Copyright (c) 2018-2024. Aleksandr Bazhenov.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
Expand Down Expand Up @@ -34,12 +34,14 @@
# that you know what you're doing. All actions with this script at your own risk.


# Set your pool name here
# Set pool name, e.g.: "Incremental" or "Full"
POOL_NAME="Full"

PWD_R=$(pwd)
cd /var/lib/postgresql || exit 1
VOLUMES=$(sudo -u postgres -H -- psql -d bareos -c "SELECT volumename FROM media ORDER BY volumename" | tail -n+3 | \
head -n -2 | grep $POOL_NAME)
[[ -z $VOLUMES ]] && echo "No volumes in the pool, nothing to do." && exit
cd "$PWD_R" || exit 1

echo "This will purge all volumes in $POOL_NAME. Sleep 30 for sure."
Expand Down
36 changes: 28 additions & 8 deletions backup/bareos_pool_operations/remove_purged_volumes.sh
Original file line number Diff line number Diff line change
@@ -1,18 +1,38 @@
#!/usr/bin/env bash


# ---------------------------------------------------------------------------- #
# Removes all volumes physically from the disk which are in 'purged' state #
# Written by Aleksandr Bazhenov December, 2018. Updated December, 2021. #
# ---------------------------------------------------------------------------- #
# Remove all volumes physically from the disk which are in 'purged' state.
# Written by Aleksandr Bazhenov December, 2018. Updated December, 2021.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# ------------------------------------------------------------------------------
# WARNING! Running this file may cause a potential data loss and assumes you accept
# that you know what you're doing. All actions with this script at your own risk.

# This Source Code Form is subject to the terms of the BSD 3-Clause License.
# If a copy of the source(s) distributed with this file, You can obtain one at:
# https://github.com/alexanderbazhenoff/data-scripts/blob/master/LICENSE


for F in $(echo "list volume" | bconsole | grep Purged | cut -d ' ' -f6)
do
Expand Down
20 changes: 0 additions & 20 deletions backup/mysql_dump/mysq_dump.sh

This file was deleted.

Loading

0 comments on commit d99ffae

Please sign in to comment.