Compare commits

..

7 Commits

Author SHA1 Message Date
vibe-code 10d3c7f88c fix(config): 为火山渠道白名单 reasoning_effort,修复思考强度被静默丢弃
LiteLLM 的 VolcEngineChatConfig.get_supported_openai_params() 白名单不含
reasoning_effort,客户端一传就抛 UnsupportedParamsError;叠加 drop_params
后更糟 —— Router._deployment_params_with_request_reasoning_override 会先剥掉
部署侧 extra_body/thinking,drop_params 再把客户端参数丢掉,两者同归于尽,
上游 body 皆无该字段,方舟回落默认 high(静默失效)。

改动:17 个 volcengine 条目全部加 allowed_openai_params: [reasoning_effort]。
- 透传组(huoshan/deepseek-v4-*、doubao-seed-2-1-*、glm-5.2):客户端显式指定生效
- 档位别名组(doubao-seed-nothink/-minimal/-low/-medium/-high、fanyi):
  extra_body 钉死值为默认档位,客户端显式传参可覆盖;不传则走钉死档位

语义为「默认档位 + 客户端可覆盖」,非硬锁:LiteLLM 无纯配置手段实现强制覆盖。
实测依据见 config.yaml 注释(本地 echo 抓上游请求体)。
2026-09-24 15:37:56 +08:00
vibe-code fafcf6e732 fix: DATABASE_URL 加 client_encoding=UTF8,修复 SQL_ASCII 库启动崩溃
新镜像 main-latest 自 2026-09-20 起默认启用 v2 migration resolver
(上游 fac518dbf7),v2 启动时调用 _warn_if_db_ahead_of_head() 读
_prisma_migrations。本库 litellm 为 SQL_ASCII(server/client_encoding
均 SQL_ASCII,datcollate=C),psycopg 的 TextLoader 对此直接返回 bytes
而非 str(psycopg/types/string.py:110-120),导致 utils.py:174 的
_MIGRATION_TS_RE.match(bytes) 抛 TypeError,容器启动即崩。

同一缺陷还潜伏在 repair_invalid_indexes/_index_repair
(utils.py:760,_TRANSIENT_INDEX_SUFFIX_RE.search(index.name))与
_failed_migration_logs(utils.py:1023,str in bytes),二者 v1/v2 双路径
均会经过,只在对应数据非空时触发(CREATE INDEX CONCURRENTLY 中断或
迁移死锁)。

根因修复:DATABASE_URL 追加 client_encoding=UTF8。libpq 将其作为
启动参数下发(fe-protocol3.c:2503),psycopg 按服务端确认值决定解码
(pq_ctypes.py:229-232),恢复返回 str。Prisma 侧无影响:quaint 丢弃
未知参数(url.rs:459),其驱动 tokio-postgres 本就固定发
client_encoding=UTF8。已全库校验 450 个文本列/43548 行/14.5MB 全部
为合法 UTF-8,无解码失败风险。

另保留 USE_V2_MIGRATION_RESOLVER=false 作独立兜底,沿用更新前的 v1
路径;根因已除后该项可移除以启用 v2。
2026-09-24 12:53:19 +08:00
root f0aa78c904 删除下线的模型 2026-09-12 15:26:49 +08:00
vibe-code 6dfaa3f34c fix: 幂等检查提前到 token_counter 之前,第二次 pre-hook 不打印 Pre-reserve 2026-08-18 11:05:48 +00:00
vibe-code 84828646bf docs: 幂等检查注释标注 data 引用假设与失效条件 2026-08-18 11:01:48 +00:00
vibe-code 5869d311c9 fix: pre-hook 幂等防重,litellm 双调用不重复预扣
litellm 对同一请求调用两次 async_pre_call_hook(proxy 级 +
deployment 级),第二次重复预扣 inflight 但结算只释放一次,
残留累积。已在日志确认(18:46:07 每个请求 Pre-reserve 打印两次)。

修复:pre-hook 入口检查 data["metadata"] 是否已有
PREBLOCK_RESERVE_META_KEY,有则跳过预扣直接放行(幂等)。

验证:新增场景7(幂等防重),7 场景 ALL PASS。
2026-08-18 11:00:18 +00:00
vibe-code 69d144ba34 fix: _release_inflight 改用 Lua 原子释放,floor 0 防幽灵误删
TTL 过期后迟到结算的 DECRBY 会把 inflight 打到负数,旧代码 DEL 整个
key → 误删别的请求的预留。改用 RELEASE_SCRIPT 在 Redis 内原子
DECRBY + floor 0:结果 <=0 则 DEL,否则 SET new_val + EXPIRE 保留 TTL。

_release_inflight eval 补第三个参数 PREBLOCK_INFLIGHT_TTL,与
RELEASE_SCRIPT 的 ARGV[2] 对应。

验证:新增场景6(幽灵释放),6 场景 ALL PASS。
2026-08-18 10:42:21 +00:00
3 changed files with 123 additions and 222 deletions
+61 -212
View File
@@ -36,40 +36,6 @@ model_list:
###
#日常使用ds v3.2 ,强制think
# - model_name: deepseek-v3.2
# litellm_params:
# <<: *token_pricing
# model: volcengine/deepseek-v3-2-251201
# api_base: os.environ/HUOSHAN_BASE_URL
# api_key: os.environ/HUOSHAN_API_KEY
# 说明中建议设置30分钟(1800秒)以上,避免深度思考过程中断连接
# timeout: 1800
# 关键配置2:启用思考模式
# extra_body 中的参数会被直接合并到请求的 JSON body 中
# extra_headers:
# Connection: "close"
# extra_body:
# thinking:
# type: "enabled"
# model_info:
# max_input_tokens: 128000 # 允许的最大输入长度(单位:token)
# max_context_length: 128000 # 上下文窗口上限
# - model_name: deepseek-v3.2-nothink
# litellm_params:
# <<: *token_pricing
# model: volcengine/deepseek-v3-2-251201
# api_base: os.environ/HUOSHAN_BASE_URL
# api_key: os.environ/HUOSHAN_API_KEY
# extra_headers:
# Connection: "close"
# extra_body:
# thinking:
# type: "disabled"
- model_name: huoshan/deepseek-v4-flash
@@ -78,6 +44,7 @@ model_list:
model: volcengine/deepseek-v4-flash-ga-260731
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
- model_name: huoshan/deepseek-v4-pro
@@ -86,101 +53,11 @@ model_list:
model: volcengine/deepseek-v4-pro-ga-260813
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
- model_name: doubao-seed-1-8
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-1-8-251228
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
- model_name: doubao-seed-1-8-nothink
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-1-8-251228
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
extra_headers:
Connection: "close"
drop_params: true
extra_body:
reasoning_effort: "minimal"
# - model_name: doubao-seed-2-0-pro
# litellm_params:
# <<: *token_pricing
# model: volcengine/doubao-seed-2-0-pro-260215
# api_base: os.environ/HUOSHAN_BASE_URL
# api_key: os.environ/HUOSHAN_API_KEY
# timeout: 1800
# drop_params: true
# extra_headers:
# Connection: "close"
# - model_name: doubao-seed-2-0-lite
# litellm_params:
# <<: *token_pricing
# model: volcengine/doubao-seed-2-0-lite-260215
# api_base: os.environ/HUOSHAN_BASE_URL
# api_key: os.environ/HUOSHAN_API_KEY
# drop_params: true
# extra_headers:
# Connection: "close"
- model_name: doubao-seed-2-0-code
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
model_info:
max_input_tokens: 224000 # 允许的最大输入长度(单位:token)
max_context_length: 256000 # 上下文窗口上限
#禁用思考,用于标题等
- model_name: doubao-seed-2-0-code-nothink
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
# thinking:
# type: "disable"
reasoning_effort: "minimal"
model_info:
max_input_tokens: 224000 # 允许的最大输入长度(单位:token)
max_context_length: 256000 # 上下文窗口上限
# doubao-seed-2-0-code 强制中等思考,用作回退
- model_name: doubao-seed-2-0-code-medium
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
timeout: 1800
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "medium"
model_info:
max_input_tokens: 224000 # 允许的最大输入长度(单位:token)
max_context_length: 256000 # 上下文窗口上限
- model_name: doubao-seed-2-1-turbo
litellm_params:
@@ -188,6 +65,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -201,6 +79,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -215,6 +94,17 @@ model_list:
model: volcengine/glm-5-2-260617
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
# 方舟 glm-5-2-260617 支持 reasoning_effort(none/minimal/high/max,默认 high;
# 官方参数页 + 实测 curl 均确认),但 LiteLLM 的 VolcEngineChatConfig
# get_supported_openai_params() 白名单里没有它 —— 它是 openai-like 的固定
# 参数表,不含 reasoning_effort —— 客户端一传就抛 UnsupportedParamsError。
# allowed_openai_params 是官方逃生口:get_optional_params 用它在校验前
# extend supported_params,随后 _apply_openai_param_overrides 把该参数从
# non_default_params 原样写回 optional_params,最终透传到请求体。
# 不用 drop_params:那是把该模型所有不支持参数静默丢弃,会掩盖其它真错。
# 注意 thinking.type 无需此项,VolcEngineChatConfig 已原生白名单。
allowed_openai_params:
- reasoning_effort
extra_headers:
Connection: "close"
model_info:
@@ -222,18 +112,22 @@ model_list:
max_context_length: 1024000 # 上下文窗口上限
#=======================豆包汇总nothink
# 【思考档位别名组】doubao-seed-nothink / -minimal / -low / -medium / -high / fanyi
# 语义:别名后缀即【默认】档位,非硬锁。
# - 客户端不传 reasoning_effort:按 extra_body 钉的档位发给方舟。
# - 客户端显式传 reasoning_effort:以客户端为准(覆盖默认档位)。
# 为什么钉不成硬锁(实测,本地 echo 抓上游 body):
# 请求带 reasoning_effort 时,Router._deployment_params_with_request_reasoning_override
# 会【无条件】剥掉部署侧 extra_body.reasoning_effort / thinking;
# 此时若没有 allowed_openai_params 放行,drop_params 又把客户端这个参数丢掉,
# 两者同归于尽 ⇒ 上游 body 两者都没有 ⇒ 方舟回落默认 high(静默失效)。
# 故必须加 allowed_openai_params:让客户端值能通过校验,使结果至少是
# 「客户端说了算」而不是「悄悄变回 high」。
# LiteLLM 当前无纯配置手段做到「不论客户端传什么一律覆盖」;要硬锁只能加
# pre-call hook(本仓库当前不采用)。
# 注:方舟 doubao 上 minimal 等价于关闭思考(实测 reasoning_tokens=0),
# 故 nothink/minimal/fanyi 三个别名默认档位统一用 minimal。
- model_name: doubao-seed-nothink
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "minimal"
- model_name: doubao-seed-nothink
litellm_params:
@@ -241,6 +135,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -253,6 +148,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -263,17 +159,6 @@ model_list:
#=======================豆包汇总思考等级minimal
- model_name: doubao-seed-minimal
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "minimal"
- model_name: doubao-seed-minimal
litellm_params:
@@ -281,6 +166,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -293,6 +179,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -302,29 +189,7 @@ model_list:
#=======================豆包汇总思考等级low
- model_name: doubao-seed-1-8
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-1-8-251228
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "high"
- model_name: doubao-seed-low
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "low"
- model_name: doubao-seed-low
litellm_params:
@@ -332,6 +197,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -344,6 +210,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -353,17 +220,6 @@ model_list:
#=======================豆包汇总思考等级medium
- model_name: doubao-seed-medium
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "medium"
- model_name: doubao-seed-medium
litellm_params:
@@ -371,6 +227,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -383,6 +240,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -392,17 +250,7 @@ model_list:
#=======================豆包汇总思考等级high
- model_name: doubao-seed-high
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-0-code-preview-260215
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "high"
- model_name: doubao-seed-high
litellm_params:
@@ -410,6 +258,7 @@ model_list:
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -422,6 +271,7 @@ model_list:
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
@@ -434,30 +284,29 @@ model_list:
- model_name: fanyi
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-1-8-251228
model: volcengine/doubao-seed-2-1-pro-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
timeout: 120
stream: false
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
Connection: "close"
extra_body:
reasoning_effort: "minimal"
reasoning_effort: "minimal"
- model_name: fanyi
litellm_params:
<<: *token_pricing
model: volcengine/doubao-seed-2-1-turbo-260628
api_base: os.environ/HUOSHAN_BASE_URL
api_key: os.environ/HUOSHAN_API_KEY
allowed_openai_params: ["reasoning_effort"]
drop_params: true
extra_headers:
Connection: "close"
extra_body:
reasoning_effort: "minimal"
# - model_name: fanyi
# litellm_params:
# <<: *token_pricing
# model: volcengine/doubao-seed-2-0-lite-260215
# api_base: os.environ/HUOSHAN_BASE_URL
# api_key: os.environ/HUOSHAN_API_KEY
# timeout: 120
# stream: false
# drop_params: true
# extra_headers:
# Connection: "close"
# extra_body:
# reasoning_effort: "minimal"
@@ -469,7 +318,7 @@ litellm_settings:
#drop_params: true #启用drop_params,这样LiteLLM会忽略不支持的参数,但参数仍然会传递给后端API
num_retries: 3 # retry call 3 times on each model_name (e.g. zephyr-beta)
fallbacks: [
{"doubao-seed-high": ["deepseek-v4-pro"]}
#{"doubao-seed-high": ["deepseek-v4-pro"]}
]
#default_fallbacks: ["deepseek-v4-flash"]
request_timeout: 120 # raise Timeout error if call takes longer than 10s. Sets litellm.request_timeout
+21 -1
View File
@@ -21,8 +21,28 @@ services:
TZ: Asia/Shanghai
# ⚠️ 必须设置这个环境变量,否则 Python 找不到你的脚本
PYTHONPATH: "/app"
DATABASE_URL: "postgresql://litellm_app:shriek-sixfold-ethanol@192.168.1.34:5432/litellm"
# ⚠️ 必须保留 client_encoding=UTF8。
# 本库 litellm 是 SQL_ASCII 编码(server/client_encoding 均为 SQL_ASCII,
# datcollate=C)。psycopg 对 SQL_ASCII 连接的 TextLoader 直接返回 bytes
# 而非 str(psycopg/types/string.py:110-120),于是 litellm_proxy_extras
# 里 _MIGRATION_TS_RE.match(migration_name)、
# _TRANSIENT_INDEX_SUFFIX_RE.search(index.name) 等对 bytes 用字符串正则,
# 抛 TypeError: cannot use a string pattern on a bytes-like object。
# 该参数经 libpq 作为启动参数下发(fe-protocol3.c:2503),使连接按 UTF8
# 协商;psycopg 读服务端确认值决定解码(pq_ctypes.py:229-232),
# 从而正常返回 str。Prisma 侧无影响:quaint 丢弃未知参数
# (prisma-engines quaint/src/connector/postgres/url.rs:459),
# 且其驱动 tokio-postgres 本就固定发送 client_encoding=UTF8。
# 已全库校验:450 个文本列 / 43548 行 / 14.5MB 全部是合法 UTF-8,
# 不存在解码失败风险。
DATABASE_URL: "postgresql://litellm_app:shriek-sixfold-ethanol@192.168.1.34:5432/litellm?client_encoding=UTF8"
STORE_MODEL_IN_DB: "True"
# v2 migration resolver 的额外兜底(可去掉以启用 v2):
# main-latest 自 2026-09-20 起默认启用 v2(上游 commit fac518dbf7),
# v2 会在启动时调用 _warn_if_db_ahead_of_head() —— 即上面那个 TypeError
# 的触发点。上面的 client_encoding 修复已从根因上解除该崩溃,
# 但 v2 还引入迁移协调锁等新机制;保守起见先沿用更新前的 v1 路径。
USE_V2_MIGRATION_RESOLVER: "false"
# --- Modified: 添加 Redis 配置 ---
REDIS_HOST: "redis" # 对应下方 redis 服务的名字
REDIS_PORT: "6379"
+41 -9
View File
@@ -138,6 +138,24 @@ redis.call('EXPIRE', KEYS[2], tonumber(ARGV[3]))
return 1
"""
# Lua 原子释放:结算时配对扣减 inflight,floor 0 防止迟到结算的幽灵释放
# 误删别的请求的预留(TTL 过期后迟到 DECRBY 导致负数 → DEL 全部)。
# SET 后补 EXPIRE,保留剩余预留的 TTL(不因 SET 重置为永久)。
# KEYS[1]=inflight_key, ARGV[1]=reserve, ARGV[2]=inflight TTL
# 返回: 扣减后剩余 inflight 值(>=0)
RELEASE_SCRIPT = """
local cur = tonumber(redis.call('GET', KEYS[1]) or '0')
local reserve = tonumber(ARGV[1])
local new_val = cur - reserve
if new_val <= 0 then
redis.call('DEL', KEYS[1])
return 0
end
redis.call('SET', KEYS[1], new_val)
redis.call('EXPIRE', KEYS[1], tonumber(ARGV[2]))
return new_val
"""
# stable_limit 缺失时(首次同步前/新增模型/TTL 过期/Redis 数据丢失)的保守兜底限额。
# 免费池常态 ~2M(首开赠送/昨日采多少返多少),若 fallback 裸奔到采集上限 2M,
# 余额不足或已耗尽的模型会被烧穿扣费。900k 对 0 余额模型仍是 9 倍优惠,
@@ -927,20 +945,18 @@ def _atomic_reserve(clean_model, api_base, effective_limit, reserve):
def _release_inflight(clean_model, api_base, reserve):
"""结算配对释放:请求完成后把 pre-hook 预扣的 reserve 从 inflight 账户扣除。
reserve<=0 或 inflight 已过期(TTL 兜底释放)时无操作。失败/丢失的预留
由 TTL 自动归零,这里只做正常结算路径的精确回收。
用 Lua 原子脚本做 DECRBY + floor 0,防止 TTL 过期后迟到结算的幽灵释放
把 DECRBY 打到负数 → 误删别的请求的预留。reserve<=0 或脚本失败时无操作,
inflight 残留由 TTL 兜底归零。
"""
if not redis_client or not reserve:
return
try:
inflight_key = get_inflight_key(clean_model, api_base)
cur = float(redis_client.get(inflight_key) or 0)
if cur <= 0:
return
# DECRBY 到负数说明 inflight 已部分过期,直接 DEL 归零即可
new_val = redis_client.decrby(inflight_key, int(reserve))
if int(new_val) <= 0:
redis_client.delete(inflight_key)
redis_client.eval(
RELEASE_SCRIPT, 1, inflight_key, int(reserve),
int(PREBLOCK_INFLIGHT_TTL),
)
except Exception as e:
log(f"❌ _release_inflight failed: {e}", level="ERROR")
@@ -1142,6 +1158,22 @@ class DailyLimitHandler(CustomLogger):
# 输出侧:思考模型 usage 含 reasoning tokens 且不受 max_tokens 限制,
# 故取 max(请求声明 max_tokens, THINKING_RESERVE_FLOOR) 兜底思考开销,
# 防短文本 + 小 max_tokens 请求 reserve 低估(实测 34 vs 实际 150+)。
# 幂等防重:litellm 对同一请求可能调用两次 async_pre_call_hook
# (proxy 级 + deployment 级),第二次预扣会重复占 inflight 且
# 结算只释放一次 → 残留累积。已有 reserve 标记时跳过预扣(含
# token_counter 精确计数等昂贵路径)直接放行。
# 假设:deployment-level hook 收到的是同一 data dict 引用(litellm
# pre_call_hook 循环体 data=result 赋值,就地修改会带过去)。若
# litellm 升级为深拷贝 data 传给第二次 hook,此检查失效,需改用
# 请求级唯一 ID(如 litellm_call_id)做去重。部署后留意日志确认
# 第二次 pre-hook 是否被跳过(Pre-reserve 只应打印一次)。
try:
_existing_meta = data.get("metadata") or {}
if isinstance(_existing_meta, dict) and PREBLOCK_RESERVE_META_KEY in _existing_meta:
return data
except Exception:
pass
try:
declared = (
data.get("max_tokens")