Python 执行 Rsync 备份磁盘

本文最后更新于:2022年2月17日 晚上

Rsync 数据同步工具 中介绍了 rsync 的使用方法,本文记录:使用python语言执行备份磁盘命令,并记录日志。

实现目标

  • 自动执行对指定文件夹的 rsync 备份
  • 自动记录日志
  • 可以调整命令

功能实现

核心函数

  • 输入源文件夹目标文件夹日志路径是否删除作为参数
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
import os
from mtutils import log_init
from mtutils import OS_isdir
from mtutils import multi_process
from mtutils import vvd_round
from functools import partial
from tqdm import tqdm
from pathlib import Path
import random
import time


def rsync_folder(source_dir, target_dir, log_path, delete=False):
logger = log_init(log_path)
time.sleep(random.random() * 10)
try:
source_dir = str(source_dir)
target_dir = str(target_dir)

logger(f"@@ start to rsync {source_dir} to {target_dir}, delete {delete}")

assert OS_isdir(source_dir), f"source_dir {source_dir} is not a dir."
assert OS_isdir(target_dir), f"target_dir {target_dir} is not a dir."

if delete:
logger("@@ this is a rsync with backup target cleaning.")
delete_str = '--delete'
else:
delete_str = ''
logger("")
sync_cmd = "rsync -azrvhP " + delete_str + " " + source_dir + " " + target_dir + ' >> ' + log_path
logger("")
logger(f"@@ rsync command: {sync_cmd}")
logger("@@ rsync start !!")

start = time.time()

os.system(sync_cmd)

passed = time.time() - start

seconds = passed % 60
total_minutes = vvd_round((passed - seconds) / 60)
minusts = total_minutes % 60
hours = vvd_round((total_minutes - minusts) / 60)

logger("@@ rsync finish !!")
logger(f"@@ running time: {hours}h {minusts}m {seconds}s")
return 0

except Exception as e:
logger('@@ rsync failed!')
logger('@@ error message: {e}')

return -1

调用方法

  • 通过 single_infer 函数接收 task 作为输入文件夹参数进行封装
1
2
3
4
def single_infer(task, log_path, delete=False):
assert len(task) == 2
source, target = task
return rsync_folder(source, target, log_path, delete)

顺序执行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
if __name__ == '__main__':

log_path = 'log/rsync.log'

task_list = [
['source_dir1', 'target_dir1'],
['source_dir2', 'target_dir2'],
['source_dir3', 'target_dir3'],
]

# 每月 1 号删除已经删除的源文件在备份盘的备份资料
cur_time = time.localtime(time.time())
if cur_time.tm_mday == 1:
delete_arg = True
else:
delete_arg = False

rsync_infer = partial(single_infer, log_path=log_path, delete=delete_arg)

for task in tqdm(task_list):
rsync_infer(task)

多进程操作

1
multi_process(rsync_infer, lambda id: task_list[id], len(task_list))

本博客所有文章除特别声明外,均采用 CC BY-SA 4.0 协议 ,转载请注明出处!