目前这个需求已经有人开发了工具,可以直接结合Proxmox VE 内部的备份工具,可以在Backup Log 中看到Rclone 的相关Log,搭配Proxmoe VE 内建CRON 自动备份到各种网路空间,非常方便:
安装Rclone
在Proxmox VE 中以root 身份登入,直接使用指令安装rclone
apt update apt install rclone -y
或到Rclone 的官网下载安装:
初始化 Rclone
请务必仔细跟着设定,或是你自己去研究rclone 的config
# rclone config 2021/10/08 02:03:49 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n name> gd-backup_crypt # 务必使用这个名字,或是之后改脚本 Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value ~~ 15 / Google Drive \ "drive" ~~ Storage> 15 Google Application Client Id Setting your own is recommended. See https://rclone.org/drive/#making-your-own-client-id for how to create your own. If you leave this blank, it will use an internal key which is low performance. Enter a string value. Press Enter for the default (""). client_id> 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com OAuth Client Secret Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Scope that rclone should use when requesting access from drive. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value 1 / Full access all files, excluding Application Data Folder. \ "drive" ~~ scope> 1 ID of the root folder Leave blank normally. Fill in to access "Computers" folders (see docs), or for rclone to use a non root folder as its starting point. Enter a string value. Press Enter for the default (""). root_folder_id> xxxx_123456789 # 在Google Drive 建一个资料夹,网址最后一串就是root_folder_id # https://drive.google.com/drive/u/1/folders/xxxx_123456789 Service Account Credentials JSON file path Leave blank normally. Needed only if you want use SA instead of interactive login. Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`. Enter a string value. Press Enter for the default (""). service_account_file> [Enter] Edit advanced config? y) Yes n) No (default) y/n> [Enter] Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes (default) n) No y/n> n Verification code Go to this URL, authenticate then paste the code here. https://accounts.google.com/o/oauth2/auth?xxxxxxxxxx # 复制上面网址到浏览器登入帐号,取得验证码填入下方 Enter a string value. Press Enter for the default (""). config_verification_code> 123454321 2021/10/08 02:16:42 NOTICE: Make sure your Redirect URL is set to "urn:ietf:wg:oauth:2.0:oob" in your custom config. Configure this as a Shared Drive (Team Drive)? y) Yes n) No (default) y/n> [Enter] -------------------- [gd-backup_crypt] type = drive client_id = 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com client_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx scope = drive root_folder_id = xxxx_123456789 token = {"access_token":"123454321","token_type":"Bearer","refresh_token":"123454321","expiry":"2021-10-08T03:16:42.034218011+08:00"} team_drive = -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> Current remotes: Name Type ==== ==== gd-backup_crypt drive e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
安装proxmox-vzbackup-rclone
下载
以root 身分执行,这是我修改过的版本:
apt-get install git cd /root git clone https://github.com/fuyuanli/proxmox-vzbackup-rclone.git git checkout pve7 chmod +x /root/proxmox-vzbackup-rclone/vzbackup-rclone.sh
修改vzbackup-rclone.sh
修改第5 行至你的备份储存路径
1 #!/bin/bash 2 # ./vzbackup-rclone.sh rehydrate YYYY/MM/DD file_name_encrypted.bin 3 4 ############ /START CONFIG 5 dumpdir="/data/pve/dump" # Set this to where your vzdump files are stored
备份保留的份数会依据你在Proxmox VE 的Backup 选项那边来删除,这个脚本只负责上传档案到云端空间
Proxmox VE 7 修正
参考Issue#7中提到的方法修正,否则会上传失败,我Fork 后修改的版本已经修正了:
修改第18 行 tarfile=${TARFILE}
改成 tarfile=${TARGET}
15 if [ ! -z "${3}" ];then 16 CMDARCHIVE=$(echo "/${3}" | sed -e 's/\(.bin\)*$//g') 17 fi 18 tarfile=${TARGET} 19 exten=${tarfile#*.} 20 filename=${tarfile%.*.*}
设定vzdump.conf
编辑 /etc/vzdump.conf 第14 行后加入
script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh
后存档
1 # vzdump default settings 2 3 #tmpdir: DIR 4 #dumpdir: DIR 5 #storage: STORAGE_ID 6 #mode: snapshot|suspend|stop 7 #bwlimit: KBPS 8 #ionice: PRI 9 #lockwait: MINUTES 10 #stopwait: MINUTES 11 #stdexcludes: BOOLEAN 12 #mailto: ADDRESSLIST 13 #prune-backups: keep-INTERVAL=N[,...] 14 #script: FILENAME 15 script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh 16 #exclude-path: PATHLIST 17 #pigz: N
设定完毕后就可以在Proxmox VE 的界面备份了,每次备份都会触发vzbackup-rclone.sh,再透过 rclone 上传,执行log 如下:
INFO: starting new backup job: vzdump 110 --storage local --node home --compress zstd --remove 0 --mode snapshot INFO: Deleting backups older than 3 days. INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp2067793_110 for temporary files INFO: Starting Backup of VM 110 (lxc) INFO: Backup started at 2021-10-08 18:42:05 INFO: status = running INFO: CT Name: pykms INFO: including mount point rootfs ('/') in backup INFO: backup mode: snapshot INFO: ionice priority: 7 INFO: create storage snapshot 'vzdump' INFO: creating vzdump archive '/var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst' INFO: Total bytes written: 962498560 (918MiB, 65MiB/s) INFO: archive file size: 274MB INFO: Backing up /var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst to remote storage INFO: rcloning /var/lib/vz/dump/rclone/2021/10 INFO: 2021/10/08 18:43:21 INFO : INFO: Transferred: 44.496Mi / 274.777 MiByte, 16%, 763.432 KiByte/s, ETA 5m8s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 1m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 16% /274.777Mi, 763.432Ki/s, 5m8s INFO: 2021/10/08 18:44:21 INFO : INFO: Transferred: 88.684Mi / 274.777 MiByte, 32%, 750.874 KiByte/s, ETA 4m13s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 2m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 32% /274.777Mi, 750.874Ki/s, 4m13s INFO: 2021/10/08 18:45:21 INFO : INFO: Transferred: 132.840Mi / 274.777 MiByte, 48%, 769.235 KiByte/s, ETA 3m8s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 3m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 48% /274.777Mi, 769.235Ki/s, 3m8s INFO: 2021/10/08 18:46:21 INFO : INFO: Transferred: 177.402Mi / 274.777 MiByte, 65%, 768.902 KiByte/s, ETA 2m9s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 4m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 64% /274.777Mi, 768.902Ki/s, 2m9s INFO: 2021/10/08 18:47:21 INFO : INFO: Transferred: 218.777Mi / 274.777 MiByte, 80%, 645.420 KiByte/s, ETA 1m28s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 5m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 79% /274.777Mi, 645.420Ki/s, 1m28s INFO: 2021/10/08 18:48:21 INFO : INFO: Transferred: 262.652Mi / 274.777 MiByte, 96%, 742.687 KiByte/s, ETA 16s INFO: Transferred: 0 / 1, 0% INFO: Elapsed time: 6m1.3s INFO: Transferring: INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 95% /274.777Mi, 742.687Ki/s, 16s INFO: 2021/10/08 18:48:40 INFO : vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: Copied (new) INFO: 2021/10/08 18:48:40 INFO : INFO: Transferred: 274.777Mi / 274.777 MiByte, 100%, 668.665 KiByte/s, ETA 0s INFO: Transferred: 1 / 1, 100% INFO: Elapsed time: 6m20.2s INFO: cleanup temporary 'vzdump' snapshot INFO: Finished Backup of VM 110 (00:06:35) INFO: Backup finished at 2021-10-08 18:48:40 INFO: Backing up main PVE configs INFO: Tar files INFO: rcloning /var/tmp/proxmox-cC0xj8WC/pveConfig.2021-10-08.18.48.40.tgz INFO: 2021/10/08 18:48:43 INFO : pveConfig.2021-10-08.18.48.40.tgz: Copied (new) INFO: 2021/10/08 18:48:43 INFO : pveConfig.2021-10-08.18.48.40.tgz: Deleted INFO: 2021/10/08 18:48:43 INFO : INFO: Transferred: 34.916Ki / 34.916 KiByte, 100%, 34.910 KiByte/s, ETA 0s INFO: Checks: 2 / 2, 100% INFO: Deleted: 1 (files), 0 (dirs) INFO: Renamed: 1 INFO: Transferred: 1 / 1, 100% INFO: Elapsed time: 3.0s INFO: Cleaning up INFO: Backup job finished successfully TASK OK
跑完的档案会在这两个位置
GoogleDrive/Year/Month/vzdump
:VM/LXC 备份
GoogleDrive/Year/Month/pveconfig
:/etc/pve/备份
档案还原
使用指令
# 拉取所有备份 ~/proxmox-vzbackup-rclone/vzbackup-rclone.sh rehydrate # 拉取指定文件 ~/proxmox-vzbackup-rclone/vzbackup-rclone.sh rehydrate YYYY/MM/DD file_name.tar.zst
会把上面的备份都下载下来,建议未来有需要时再手动下载就好,也可以直接使用rclone或fclone将指定文件下载到pve的dump目录,手动进行还原。
# 这里的lxc是rclone/fclone中配置的备份虚拟磁盘名称,后面指定了日期和文件 fclone --config /root/.config/rclone/rclone.conf --drive-chunk-size=128M copy lxc:/2022/10/14/vzdump-lxc-100-2022_10_14-08_17_43.tar.zst /var/lib/vz/dump -vP --stats=1s --transfers=16 --checkers=16 --check-first