P3-3: Flowable 5.13-SNAPSHOT→6.8.1 升级工具链产品化 + 演练全绿
一键链(tools/flowable-upgrade/): ①前置校验→②版本串规范化→③Flowable 5.23→④6.8.1 兼容层→⑤后置校验 全步骤幂等可重入;参数化 host/port/db/schema;JAVA_HOME 强制传 Maven。 完整重演(eip4flow_rehearsal,客户库只读口径): import_5x 63s | 一键链 21s | 后置全绿 1515 定义 / 9386 执行 / 9324 任务 / 26761 历史 全量就地保留 schema.version: 5.13-SNAPSHOT → 5.23.0.0 → 6.8.1.0 engineVer=v5 标记: 1515/1515 迁移手册新增坑6.4: Maven exec:java 在多 JDK 环境不走脚本 JAVA_HOME。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
58
tools/flowable-upgrade/rehearsal/bytearray_copy.py
Executable file
58
tools/flowable-upgrade/rehearsal/bytearray_copy.py
Executable file
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env python3
|
||||
"""bytearray_copy.py —— act_ge_bytearray 过滤拷贝(参数化版,源于 eip4-poc/eip4_bytearray_copy.py)。
|
||||
|
||||
口径:巨型 act_ge_bytearray(生产 ~1.2GB 全量)只拷:
|
||||
- 流程定义资源(deployment_id_ 非空) —— 升级链必需,流程定义/图/表单
|
||||
- 运行时变量引用的字节块(被 act_ru_variable.bytearray_id_ 引用) —— 续跑必需
|
||||
历史变量(act_hi_varinst)引用的 bytearray 不拷(历史表本身 schema-only)。
|
||||
|
||||
环境变量(全部有默认):
|
||||
SRC_HOST/SRC_PORT/SRC_DB/SRC_SCHEMA/SRC_USER/SRC_PASS 源(客户库,只读)
|
||||
TGT_HOST/TGT_PORT/TGT_DB/TGT_SCHEMA/TGT_USER/TGT_PASS 目标(本地新库)
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import psycopg2
|
||||
|
||||
|
||||
def env(k, d):
|
||||
v = os.environ.get(k)
|
||||
return v if v not in (None, "") else d
|
||||
|
||||
|
||||
SRC = dict(host=env("SRC_HOST", "192.168.20.68"), port=int(env("SRC_PORT", "5435")),
|
||||
dbname=env("SRC_DB", "postgres"), user=env("SRC_USER", "ceshi"),
|
||||
password=env("SRC_PASS", "ceshi"))
|
||||
SRC_SCHEMA = env("SRC_SCHEMA", "eip3")
|
||||
TGT = dict(host=env("TGT_HOST", "localhost"), port=int(env("TGT_PORT", "5433")),
|
||||
dbname=env("TGT_DB", "eip4flow_rehearsal"), user=env("TGT_USER", os.environ.get("USER", "")),
|
||||
password=env("TGT_PASS", ""))
|
||||
TGT_SCHEMA = env("TGT_SCHEMA", "eip3")
|
||||
|
||||
src = psycopg2.connect(connect_timeout=15, **SRC)
|
||||
src.set_session(readonly=True) # 客户库只读纪律
|
||||
dst = psycopg2.connect(**{k: v for k, v in TGT.items() if v != ""})
|
||||
scur, dcur = src.cursor(), dst.cursor()
|
||||
|
||||
scur.execute(f"""
|
||||
SELECT id_, rev_, name_, deployment_id_, bytes_, generated_
|
||||
FROM {SRC_SCHEMA}.act_ge_bytearray
|
||||
WHERE deployment_id_ IS NOT NULL
|
||||
OR id_ IN (SELECT bytearray_id_ FROM {SRC_SCHEMA}.act_ru_variable WHERE bytearray_id_ IS NOT NULL)
|
||||
""")
|
||||
n = 0
|
||||
while True:
|
||||
rows = scur.fetchmany(500)
|
||||
if not rows:
|
||||
break
|
||||
args = b','.join(dcur.mogrify('(%s,%s,%s,%s,%s,%s)', r) for r in rows)
|
||||
dcur.execute(
|
||||
f'INSERT INTO {TGT_SCHEMA}.act_ge_bytearray '
|
||||
'(id_,rev_,name_,deployment_id_,bytes_,generated_) VALUES '.encode() + args)
|
||||
n += len(rows)
|
||||
if n % 5000 == 0:
|
||||
print(n, flush=True)
|
||||
dst.commit()
|
||||
print('bytearray 过滤拷贝完成:', n, '(仅流程定义资源 + 运行时变量引用)')
|
||||
src.close()
|
||||
dst.close()
|
||||
64
tools/flowable-upgrade/rehearsal/import_5x.sh
Executable file
64
tools/flowable-upgrade/rehearsal/import_5x.sh
Executable file
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# import_5x.sh —— 按 eip4_flow_import 口径从客户库(只读)拉一份新鲜 5.13 副本到本地新库。
|
||||
# 源于 eip4-poc/tools/eip4_flow_import.sh,参数化 + 加 schema/库/bytearray 过滤步骤。
|
||||
#
|
||||
# 口径:
|
||||
# - 全量运行时表(ACT_FULL):表结构 + 数据全量
|
||||
# - 巨型历史/bytearray(ACT_SCHEMA_ONLY):仅表结构(act_hi 巨型表 schema-only)
|
||||
# - act_ge_bytearray:schema-only 建表后,用 bytearray_copy.py 按引用过滤拷贝
|
||||
#
|
||||
# 环境变量(源=客户库只读,默认值见下;目标=本地新库,由 common.sh 提供):
|
||||
# SRC_HOST/SRC_PORT/SRC_DB/SRC_SCHEMA/SRC_USER/SRC_PASS
|
||||
set -euo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "${HERE}/../lib/common.sh"
|
||||
|
||||
: "${SRC_HOST:=192.168.20.68}"
|
||||
: "${SRC_PORT:=5435}"
|
||||
: "${SRC_DB:=postgres}"
|
||||
: "${SRC_SCHEMA:=eip3}"
|
||||
: "${SRC_USER:=ceshi}"
|
||||
: "${SRC_PASS:=ceshi}"
|
||||
|
||||
log "重拉 5.13 副本: ${SRC_HOST}:${SRC_PORT}/${SRC_DB}.${SRC_SCHEMA}(只读) -> ${TGT_HOST}:${TGT_PORT}/${TGT_DB}.${TGT_SCHEMA}"
|
||||
begin_timer
|
||||
|
||||
# 源 pg_dump 用源口令;目标 psql 用 common.sh 已 export 的 PGPASSWORD(目标)
|
||||
spd() { PGPASSWORD="${SRC_PASS}" "${PGBIN}/pg_dump" -h "${SRC_HOST}" -p "${SRC_PORT}" -U "${SRC_USER}" -d "${SRC_DB}" "$@"; }
|
||||
dpsql() { "${PGBIN}/psql" -h "${TGT_HOST}" -p "${TGT_PORT}" -U "${TGT_USER}" -d "${TGT_DB}" "$@"; }
|
||||
|
||||
TARGS=""; for t in ${ACT_FULL}; do TARGS="${TARGS} -t ${SRC_SCHEMA}.${t}"; done
|
||||
SARGS=""; for t in ${ACT_SCHEMA_ONLY}; do SARGS="${SARGS} -t ${SRC_SCHEMA}.${t}"; done
|
||||
|
||||
# 目标 schema 重置(幂等:演练库每次重拉)
|
||||
dpsql -v ON_ERROR_STOP=1 -c "DROP SCHEMA IF EXISTS ${TGT_SCHEMA} CASCADE; CREATE SCHEMA ${TGT_SCHEMA};" >/dev/null
|
||||
ok "目标 schema ${TGT_SCHEMA} 已重建"
|
||||
|
||||
log "== 表结构(全量表 + schema-only 巨表)=="
|
||||
ERR=$(spd --schema-only --no-owner --no-privileges ${TARGS} ${SARGS} \
|
||||
| dpsql -q -v ON_ERROR_STOP=0 2>&1 | grep -c ERROR || true)
|
||||
[ "${ERR}" = "0" ] && ok "表结构导入 0 错误" || warn "表结构导入 ${ERR} 个 ERROR(检查)"
|
||||
|
||||
log "== 数据(仅全量运行时表;巨型历史表 schema-only 不带数据)=="
|
||||
ERR=$(spd --data-only --no-owner --no-privileges --disable-triggers ${TARGS} \
|
||||
| dpsql -q 2>&1 | grep -c ERROR || true)
|
||||
[ "${ERR}" = "0" ] && ok "运行时数据导入 0 错误" || warn "运行时数据导入 ${ERR} 个 ERROR(检查)"
|
||||
|
||||
log "== act_ge_bytearray 过滤拷贝(流程定义资源 + 运行时变量引用)=="
|
||||
SRC_HOST="${SRC_HOST}" SRC_PORT="${SRC_PORT}" SRC_DB="${SRC_DB}" SRC_SCHEMA="${SRC_SCHEMA}" \
|
||||
SRC_USER="${SRC_USER}" SRC_PASS="${SRC_PASS}" \
|
||||
TGT_HOST="${TGT_HOST}" TGT_PORT="${TGT_PORT}" TGT_DB="${TGT_DB}" TGT_SCHEMA="${TGT_SCHEMA}" \
|
||||
TGT_USER="${TGT_USER}" TGT_PASS="${TGT_PASS}" \
|
||||
python3 "${HERE}/bytearray_copy.py"
|
||||
|
||||
log "== 副本行数核对 =="
|
||||
dpsql -tc "SET search_path TO ${TGT_SCHEMA};
|
||||
select 'act_re_procdef', count(*) from act_re_procdef union all
|
||||
select 'act_ru_execution', count(*) from act_ru_execution union all
|
||||
select 'act_ru_task', count(*) from act_ru_task union all
|
||||
select 'act_ru_variable', count(*) from act_ru_variable union all
|
||||
select 'act_ge_bytearray', count(*) from act_ge_bytearray union all
|
||||
select 'act_hi_procinst', count(*) from act_hi_procinst" | sed 's/^/ /'
|
||||
|
||||
end_timer "import_5x"
|
||||
log "新鲜 5.13 副本就绪 @ ${TGT_DB}.${TGT_SCHEMA}"
|
||||
48
tools/flowable-upgrade/rehearsal/rehearse.sh
Executable file
48
tools/flowable-upgrade/rehearsal/rehearse.sh
Executable file
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env bash
|
||||
# rehearse.sh —— 完整重演:本地重拉新鲜 5.13 副本 + 跑一键升级链全程 + 后置全绿。
|
||||
#
|
||||
# 默认目标库 = eip4flow_rehearsal(不动现有 eip4flow,那是 P3-1 代理的验证场)。
|
||||
# 客户库只读纪律:import_5x.sh 用 readonly session 拉数;act_hi 巨型表 schema-only。
|
||||
#
|
||||
# 用法:
|
||||
# ./rehearse.sh # 全程:建库 -> 拉数 -> 一键链 -> 全链耗时
|
||||
# ./rehearse.sh --keep # 不 DROP 重建演练库(在已有副本上重跑链)
|
||||
set -euo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
export TGT_DB="${TGT_DB:-eip4flow_rehearsal}"
|
||||
source "${HERE}/../lib/common.sh"
|
||||
|
||||
KEEP=0
|
||||
[ "${1:-}" = "--keep" ] && KEEP=1
|
||||
|
||||
# 安全闸:绝不允许把演练目标指到 eip4flow(P3-1 验证场)
|
||||
[ "${TGT_DB}" = "eip4flow" ] && die "拒绝:目标库 eip4flow 是 P3-1 验证场,演练请用 eip4flow_rehearsal"
|
||||
|
||||
log "########## Flowable 升级完整重演 (目标库 ${TGT_DB}) ##########"
|
||||
: > "${WORK_DIR}/timings.tsv"
|
||||
|
||||
# 0) 演练库存在性(幂等:不存在则建)
|
||||
DB_EXISTS=$("${PGBIN}/psql" -h "${TGT_HOST}" -p "${TGT_PORT}" -U "${TGT_USER}" -d postgres -tAc \
|
||||
"SELECT 1 FROM pg_database WHERE datname='${TGT_DB}'" || echo "")
|
||||
if [ "${DB_EXISTS}" != "1" ]; then
|
||||
"${PGBIN}/createdb" -h "${TGT_HOST}" -p "${TGT_PORT}" -U "${TGT_USER}" -E UTF8 -T template0 "${TGT_DB}"
|
||||
ok "演练库 ${TGT_DB} 已创建"
|
||||
else
|
||||
ok "演练库 ${TGT_DB} 已存在"
|
||||
fi
|
||||
|
||||
# 1) 重拉新鲜 5.13 副本(--keep 时跳过)
|
||||
if [ "${KEEP}" = "0" ]; then
|
||||
bash "${HERE}/import_5x.sh"
|
||||
else
|
||||
warn "--keep:跳过重拉,在现有副本上重跑链"
|
||||
fi
|
||||
|
||||
# 2) 一键升级链(演练默认 assume-backed-up:副本即可丢弃重拉,无需额外备份)
|
||||
bash "${HERE}/../flowable_upgrade.sh" --db "${TGT_DB}" --schema "${TGT_SCHEMA}" --assume-backed-up
|
||||
|
||||
echo
|
||||
log "########## 重演完成 ##########"
|
||||
log "全程分步耗时(本次 timings.tsv):"
|
||||
column -t -s $'\t' "${WORK_DIR}/timings.tsv" | sed 's/^/ /'
|
||||
log "前置快照: ${WORK_DIR}/snapshot_pre.tsv 后置快照: ${WORK_DIR}/snapshot_post.tsv 对账: ${WORK_DIR}/reconcile.txt"
|
||||
Reference in New Issue
Block a user