Need help with cv4pve-autosnap?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Corsinvest
125 Stars 32 Forks Other 78 Commits 2 Opened issues

Description

Automatic snapshot tool for Proxmox VE

Services available

!
?

Need anything else?

Contributors list

# 203,656
C#
HTML
Shell
proxmox...
68 commits

cv4pve-autosnap

License Release GitHub All Releases AppVeyor branch Nuget

Proxmox VE automatic snapshot tool

More information about cv4pve-autosnap

More information about Qemu guest agent

The old bash version inside Proxmox VE is no longer supported because the Proxmox VE developers continue to change output. The risk of incompatibility is high. With the new version that uses native APIs, the problem no longer exists

    ______                _                      __
   / ____/___  __________(_)___ _   _____  _____/ /_
  / /   / __ \/ ___/ ___/ / __ \ | / / _ \/ ___/ __/
 / /___/ /_/ / /  (__  ) / / / / |/ /  __(__  ) /_
 \____/\____/_/  /____/_/_/ /_/|___/\___/____/\__/

Corsinvest for Proxmox VE Auto Snapshot (Made in Italy)

Usage: cv4pve-autosnap [options] [command]

Options: -?|-h|--help Show help information --version Show version information --host The host name host[:port],host1[:port],host2[:port] --api-token Api token format '[email protected]!TOKENID=UUID'. Require Proxmox VE 6.2 or later --username User name @ --password The password. Specify 'file:path_file' to store password in file. --vmid The id or name VM/CT comma separated (eg. 100,101,102,TestDebian) -vmid or -name exclude (e.g. -200,-TestUbuntu) 'all-???' for all VM/CT in specific host (e.g. all-pve1, all-$(hostname)), 'all' for all VM/CT in cluster --timeout Timeout operation in seconds --timestamp-format Specify different timestamp format. Default: yyMMddHHmmss

Commands: app-check-update Check update application app-upgrade Upgrade application clean Remove auto snapshots snap Will snap one time status Get list of all auto snapshots

Run 'cv4pve-autosnap [command] --help' for more information about a command.

cv4pve-autosnap is a part of suite cv4pve-tools. For more information visit https://www.cv4pve-tools.com

Copyright and License

Copyright: Corsinvest Srl For licensing details please visit LICENSE.md

Commercial Support

This software is part of a suite of tools called cv4pve-tools. If you want commercial support, visit the site

Tutorial

Tutorial

GUI Version Toolbox

Tutorial

Introduction

Automatic snapshot for Proxmox VE with retention.

In this version the tool works outside the Proxmox VE host using the API. The reasons are:

  • if the host does not work the tool does not work
  • using the API, future changes are guaranteed
  • Root access is not required, a user is required to perform the operation
  • use of standard https / https json technology

Main features

  • Completely written in C#
  • Use native Api REST Proxmox VE (library C#)
  • Independent os (Windows, Linux, Macosx)
  • Installation rapid, unzip file extract binary
  • Not require installation in Proxmox VE
  • Execute out side Proxmox VE
  • For KVM and LXC
  • Work for single node or cluster, automatically resolve VM/CT id/name
  • Use id or name with --vmid parameter
  • Can keep multiple snapshots --keep
  • Clean all snapshots
  • Multiple schedule VM/CT using --label (es. daily,monthly)
  • Hook script
  • Multiple VM/CT (100,102,ubuVm,debVm,pipperoVm,fagianoVm or all) in a single execution
  • Exclusion specific VM/CT using minus e.g --vmid=all,-100
  • Exclusion template from snapshot
  • Waiting for the snapshot process to finish
  • Alerting in QEMU, agent not enabled.
  • Save memory VM Qemu in snap using parameter --state.
  • No stop on error
  • Script hook for metrics
  • Timeout remove/create snapshot in --timeout parameter specify in second
  • Support multiple host for HA in --host parameter es. host[:port],host1[:port],host2[:port]
  • Multiple output text,unicode,unicodeAlt,markdown,html
  • Check-Update and Upgrade application
  • Use Api token --api-token parameter
  • Support range vmid 100:103,134,200:204,-102
  • Support different timestamp format with parameter --timestamp-format

Api token

From version 6.2 of Proxmox VE is possible to use Api token. This feature permit execute Api without using user and password. If using Privilege Separation when create api token remember specify in permission.

Custom timestamp format

Timestamp format is based on C# date and time format string.

Configuration and use

E.g. install on linux 64

Download last package e.g. Debian cv4pve-autosnap-linux-x64.zip, on your os and install:

[email protected]:~# unzip cv4pve-autosnap-linux-x64.zip

This tool need basically no configuration.

Snapshot a VM/CT one time

[email protected]:~# cv4pve-autosnap --host=192.168.0.100 [email protected] --password=fagiano --vmid=111 snap --label='daily' --keep=2

This command snap VM 111.

[email protected]:~# cv4pve-autosnap --host=192.168.0.100 [email protected] --password=fagiano --vmid="all,-111" snap --label='daily' --keep=2

This command snap all VMs except 111.

The --keep tells that it should be kept 2 snapshots, if there are more than 2 snapshots, the 3 one will be erased (sorted by creation time). The --state save memory VM. The --timeout specify the timeout remove/creation snapshot.

When create a snapshot the software add "auto" prefix of the name the snapshot.

Clean a VM/CT one time

[email protected]:~# cv4pve-autosnap --host=192.168.0.100 [email protected] --password=fagiano --vmid=111 clean --label='4hours' --keep=2
----- VM 100 QEMU -----
Remove snapshot: auto4hours190617080002
Remove snapshot: auto4hours190617120002
Remove snapshot: auto4hours190617160002
Remove snapshot: auto4hours190617200002

Status snapshots

[email protected]:~# cv4pve-autosnap --host=192.168.0.100 [email protected] --password=fagiano --vmid=100 status

┌──────┬─────┬───────────────────┬──────────────────────────────┬──────────────────────────────┬─────────────────┬─────┐ │ NODE │ VM │ TIME │ PARENT │ NAME │ DESCRIPTION │ RAM │ ├──────┼─────┼───────────────────┼──────────────────────────────┼──────────────────────────────┼─────────────────┼─────┤ │ pve1 │ 100 │ 19/08/28 09:21:35 │ no-parent │ auto4hours190828112133 │ cv4pve-autosnap │ │ │ pve1 │ 100 │ 19/08/28 12:23:09 │ auto4hours190828112133 │ auto4hours190828142307 │ cv4pve-autosnap │ │ │ pve1 │ 100 │ 19/08/29 06:50:23 │ auto4hours190828142307 │ auto4hours190829085021 │ cv4pve-autosnap │ │ │ pve1 │ 100 │ 19/08/29 07:32:15 │ auto4hours190829085021 │ auto4hours190829093214 │ cv4pve-autosnap │ │ └──────┴─────┴───────────────────┴──────────────────────────────┴──────────────────────────────┴─────────────────┴─────┘

Output type:

-o|--output   Type output (default: text) Text,Json,JsonPretty

Hook script

Before run hook script, program create environment variable:

CV4PVE_AUTOSNAP_PHASE
CV4PVE_AUTOSNAP_VMID
CV4PVE_AUTOSNAP_VMNAME
CV4PVE_AUTOSNAP_VMTYPE
CV4PVE_AUTOSNAP_LABEL
CV4PVE_AUTOSNAP_KEEP
CV4PVE_AUTOSNAP_SNAP_NAME
CV4PVE_AUTOSNAP_VMSTATE     #1/0
CV4PVE_AUTOSNAP_DEBUG       #1/0
CV4PVE_AUTOSNAP_DRY_RUN     #1/0
CV4PVE_AUTOSNAP_DURATION
CV4PVE_AUTOSNAP_STATE       #1/0

See example hook file script-hook.bat, script-hook.sh

Some words about Snapshot consistency and what qemu-guest-agent can do for you

Bear in mind, that when taking a snapshot of a running VM, it's basically like if you have a server which gets pulled away from the Power. Often this is not cathastrophic as the next fsck will try to fix Filesystem Issues, but in the worst case this could leave you with a severely damaged Filesystem, or even worse, half written Inodes which were in-flight when the power failed lead to silent data corruption. To overcome these things, we have the qemu-guest-agent to improve the consistency of the Filesystem while taking a snapshot. It won't leave you a clean filesystem, but it sync()'s outstanding writes and halts all i/o until the snapshot is complete. Still, there might me issues on the Application layer. Databases processes might have unwritten data in memory, which is the most common case. Here you have the opportunity to do additional tuning, and use hooks to tell your vital processes things to do prio and post freezes.

First, you want to make sure that your guest has the qemu-guest-agent running and is working properly. Now we use custom hooks to tell your services with volatile data, to flush all unwritten data to disk. On debian based linux systems the hook file can be set in

/etc/default/qemu-guest-agent
and could simply contain this line:
DAEMON_ARGS="-F/etc/qemu/fsfreeze-hook"

Create

/etc/qemu/fsfreeze-hook
and make ist look like:
#!/bin/sh

This script is executed when a guest agent receives fsfreeze-freeze and

fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)

option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).

When the agent receives fsfreeze-freeze request, this script is issued with

"freeze" argument before the filesystem is frozen. And for fsfreeze-thaw

request, it is issued with "thaw" argument after filesystem is thawed.

LOGFILE=/var/log/qga-fsfreeze-hook.log FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d

Check whether file $1 is a backup or rpm-generated file and should be ignored

is_ignored_file() { case "$1" in *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave | *.sample | *.dpkg-old | *.dpkg-new | *.dpkg-tmp | *.dpkg-dist | *.dpkg-bak | *.dpkg-backup | *.dpkg-remove) return 0 ;; esac return 1 }

Iterate executables in directory "fsfreeze-hook.d" with the specified args

[ ! -d "$FSFREEZE_D" ] && exit 0 for file in "$FSFREEZE_D"/* ; do is_ignored_file "$file" && continue [ -x "$file" ] || continue printf "$(date): execute $file [email protected]\n" >>$LOGFILE "$file" "[email protected]" >>$LOGFILE 2>&1 STATUS=$? printf "$(date): $file finished with status=$STATUS\n" >>$LOGFILE done

exit 0

For testing purposes place this into

/etc/qemu/fsfreeze-hook.d/10-info
:
#!/bin/bash
dt=$(date +%s)

case "$1" in freeze) echo "frozen on $dt" | tee >(cat >/tmp/fsfreeze) ;; thaw) echo "thawed on $dt" | tee >(cat >>/tmp/fsfreeze) ;; esac

Now you can place files for different Services in

/etc/qemu/fsfreeze-hook.d/
that tell those services what to to prior and post snapshots. A very common example is mysql. Create a file
/etc/qemu/fsfreeze-hook.d/20-mysql
containing
#!/bin/sh

Flush MySQL tables to the disk before the filesystem is frozen.

At the same time, this keeps a read lock in order to avoid write accesses

from the other clients until the filesystem is thawed.

MYSQL="/usr/bin/mysql" #MYSQL_OPTS="-uroot" #"-prootpassword" MYSQL_OPTS="--defaults-extra-file=/etc/mysql/debian.cnf" FIFO=/var/run/mysql-flush.fifo

Check mysql is installed and the server running

[ -x "$MYSQL" ] && "$MYSQL" $MYSQL_OPTS < /dev/null || exit 0

flush_and_wait() { printf "FLUSH TABLES WITH READ LOCK \G\n" trap 'printf "$(date): $0 is killed\n">&2' HUP INT QUIT ALRM TERM read < $FIFO printf "UNLOCK TABLES \G\n" rm -f $FIFO }

case "$1" in freeze) mkfifo $FIFO || exit 1 flush_and_wait | "$MYSQL" $MYSQL_OPTS & # wait until every block is flushed while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |
"$MYSQL" $MYSQL_OPTS | tail -1 | cut -f 2)" -gt 0 ]; do sleep 1 done # for InnoDB, wait until every log is flushed INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX) [ $? -ne 0 ] && exit 2 trap "rm -f $INNODB_STATUS; exit 1" HUP INT QUIT ALRM TERM while :; do printf "SHOW ENGINE INNODB STATUS \G" |
"$MYSQL" $MYSQL_OPTS > $INNODB_STATUS LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |
tr -s ' ' | cut -d' ' -f4) LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |
tr -s ' ' | cut -d' ' -f5) [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break sleep 1 done rm -f $INNODB_STATUS ;;

thaw)
    [ ! -p $FIFO ] &amp;&amp; exit 1
    echo &gt; $FIFO
    ;;

*)
    exit 1
    ;;

esac

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.