fix: streamline monitored model maintenance
This commit is contained in:
@@ -1,180 +1,252 @@
|
||||
# VolcSync - LiteLLM 火山引擎每日限制管理系统
|
||||
# VolcSync - LiteLLM 火山模型监控与限额说明
|
||||
|
||||
## 项目概述
|
||||
## 当前生效入口
|
||||
|
||||
VolcSync 是一个专为 LiteLLM 代理服务器设计的智能每日限制管理系统，专注于火山引擎 API 的使用限制控制和资源包同步。它提供了动态限制计算、自动模型路由管理和资源包余额实时同步功能。
|
||||
当前 `config.yaml` 使用的回调入口是：
|
||||
|
||||
## 核心功能
|
||||
```yaml
|
||||
callbacks: ["volc_limit_sync.proxy_handler_instance"]
|
||||
```
|
||||
|
||||
### 1. 智能限制管理
|
||||
- **动态限制计算**：基于资源包余额和使用模式自动调整每日限制
|
||||
- **硬性限制配置**：支持静态配置特定模型的每日硬限制
|
||||
- **北京时区优化**：完全按照北京时间 (UTC+8) 进行每日限制计算
|
||||
- **多层级限制策略**：支持精确匹配、模糊匹配和默认策略
|
||||
这意味着当前 Docker 容器里真正生效的是 `volc_limit_sync.py`。
|
||||
|
||||
### 2. 自动模型路由管理
|
||||
- **软驱逐机制**：当模型达到每日限制时自动禁用路由
|
||||
- **自动恢复机制**：定期检查已禁用模型的配额恢复情况
|
||||
- **恢复策略优化**：根据时间段动态调整检查频率
|
||||
`daily_limit.py` 目前 **未启用**，只保留作历史参考。除非你把 `callbacks` 改回 `daily_limit.py`，否则改它里面的 `MODEL_QUOTAS` 不会影响当前运行行为。
|
||||
|
||||
### 3. 资源包同步
|
||||
- **火山引擎 API 集成**：实时同步资源包余额
|
||||
- **智能缓存策略**：基于余额和使用模式动态调整同步频率
|
||||
- **突发流量检测**：自动检测异常流量并触发紧急同步
|
||||
- **资源包去重**：处理火山引擎 API 响应中的重复项
|
||||
---
|
||||
|
||||
### 4. 监控与日志
|
||||
- **详细使用统计**：实时跟踪每个模型的使用情况
|
||||
- **多级别日志**：支持 DEBUG、INFO、WARNING、ERROR 级别
|
||||
- **可视化指标**：使用 emoji 图标增强日志可读性
|
||||
- **配置自动发现**：自动从 config.yaml 中发现火山引擎模型
|
||||
## 术语约定
|
||||
|
||||
## 安装与配置
|
||||
为避免再把“白名单”这种不准确的说法带进来，这个仓库统一使用下面几组名称：
|
||||
|
||||
- **纳入监控模型**
|
||||
- 指 `config.yaml` 中启用了火山 `api_base` 的上游模型。
|
||||
- 这些模型会被 `volc_limit_sync.py` 自动发现，并参与资源包同步、用量统计和限额控制。
|
||||
|
||||
- **监控别名**
|
||||
- 指火山资源包返回的名字，经过归一化后，用来匹配“纳入监控模型”的名字。
|
||||
- 例如 `doubao-seed-2-0-pro-260215` 会自动推导出监控别名 `doubao-seed-2-0-pro`。
|
||||
|
||||
- **手工别名覆盖**
|
||||
- 指 `volc_limit_sync.py` 中 `MANUAL_MONITORED_MODEL_ALIASES` 里的特殊映射。
|
||||
- 只用于两类场景：
|
||||
- 资源包名和上游模型名不一致。
|
||||
- 一个资源包要映射到多个上游模型。
|
||||
|
||||
- **硬限额**
|
||||
- 指 `volc_limit_sync.py` 中的 `HARD_DAILY_LIMITS`。
|
||||
- 支持三种常见粒度：
|
||||
- `model:hash`
|
||||
- `model`
|
||||
- `__DEFAULT__`
|
||||
|
||||
---
|
||||
|
||||
## 维护入口
|
||||
|
||||
### 主入口：`config.yaml`
|
||||
|
||||
新增、删除“纳入监控模型”时，第一步永远是改 `config.yaml` 里的 `model_list`。
|
||||
|
||||
对大多数正常的版本号模型，做到这一步就已经足够让运行时识别它们进入监控链路了。现在的逻辑会自动把：
|
||||
|
||||
```text
|
||||
deepseek-v3-2-251201 -> deepseek-v3-2
|
||||
doubao-seed-2-0-pro-260215 -> doubao-seed-2-0-pro
|
||||
glm-4-7-251222 -> glm-4-7
|
||||
```
|
||||
|
||||
这类常规版本号模型推导成监控别名，不需要你再手工去补映射。
|
||||
|
||||
### 审计入口：`generate_keys.py`
|
||||
|
||||
每次改完 `config.yaml` 后，执行：
|
||||
|
||||
### 1. 依赖安装
|
||||
```bash
|
||||
pip install litellm redis requests
|
||||
python generate_keys.py
|
||||
```
|
||||
|
||||
### 2. 环境变量配置
|
||||
```bash
|
||||
# 火山引擎认证
|
||||
VOLC_ACCESS_KEY="your-access-key"
|
||||
VOLC_SECRET_KEY="your-secret-key"
|
||||
这个脚本现在不是单纯“生成 key”，而是正式的 **监控模型审计入口**。它会输出：
|
||||
|
||||
# Redis 连接
|
||||
REDIS_HOST="localhost"
|
||||
REDIS_PORT="6379"
|
||||
REDIS_PASSWORD="your-password"
|
||||
- 当前从 `config.yaml` 发现了哪些“纳入监控模型”
|
||||
- 每个上游模型对应哪些“监控别名”
|
||||
- 每个模型当前命中的限额来源
|
||||
- 自定义键
|
||||
- 模型级规则
|
||||
- 模糊规则
|
||||
- `__DEFAULT__`
|
||||
- 哪些模型当前仍在使用 `__DEFAULT__`
|
||||
- 如果你想把默认限额改成单模型硬限额，应复制哪些 key 到 `HARD_DAILY_LIMITS`
|
||||
|
||||
# 日志级别
|
||||
DAILY_LIMIT_LOG_LEVEL="INFO"
|
||||
```
|
||||
### 特殊入口：`volc_limit_sync.py`
|
||||
|
||||
### 3. 配置文件
|
||||
系统会自动读取 `config.yaml` 文件中的火山引擎模型配置。
|
||||
只有下面两种情况，才需要改 `volc_limit_sync.py`：
|
||||
|
||||
## 使用方法
|
||||
1. 资源包名和上游模型名不一致
|
||||
|
||||
示例：
|
||||
|
||||
### 1. 作为模块导入
|
||||
```python
|
||||
from volc_limit_sync import proxy_handler_instance
|
||||
"doubao-seed-code": "doubao-seed-code-preview-251028"
|
||||
```
|
||||
|
||||
### 2. 直接运行监控
|
||||
```bash
|
||||
python volc_limit_sync.py
|
||||
```
|
||||
2. 一个资源包要映射到多个上游模型
|
||||
|
||||
### 3. 集成到 LiteLLM 代理服务器
|
||||
系统会自动作为回调函数注册到 LiteLLM 的 input_callback 和 callbacks 中。
|
||||
示例：
|
||||
|
||||
## 架构设计
|
||||
|
||||
### 核心组件
|
||||
|
||||
1. **DailyLimitHandler**：使用记录和限制检查
|
||||
2. **QuotaInterceptor**：请求前的配额检查
|
||||
3. **SoftEvict/Restore**：模型路由管理
|
||||
4. **ActiveMonitorLoop**：资源包同步和动态限制计算
|
||||
5. **Util Functions**：工具函数集合（哈希、清理、签名）
|
||||
|
||||
### 关键策略
|
||||
|
||||
#### 动态限制算法
|
||||
```python
|
||||
# 基于资源包峰值 - 安全缓冲计算有效限制
|
||||
effective_limit = max(0, current_peak - safety_margin)
|
||||
"doubao-seed-1-6": ["doubao-seed-1-6-250615", "doubao-seed-1-6-251015"]
|
||||
```
|
||||
|
||||
#### 同步频率调整
|
||||
- 高余额时：低频同步（每小时）
|
||||
- 低余额时：高频同步（每2分钟）
|
||||
- 突发流量：立即同步
|
||||
如果只是新增一个普通的版本号模型，例如 `xxx-260215`，通常不需要改这里。
|
||||
|
||||
#### 北京时区处理
|
||||
```python
|
||||
beijing_time_ts = now + (8 * 3600)
|
||||
date_str = time.strftime("%Y-%m-%d", time.gmtime(beijing_time_ts))
|
||||
```
|
||||
---
|
||||
|
||||
## 配置说明
|
||||
## 推荐维护流程
|
||||
|
||||
### 新增监控模型
|
||||
|
||||
1. 在 `config.yaml` 的 `model_list` 里加入新的火山模型条目。
|
||||
2. 运行 `python generate_keys.py`。
|
||||
3. 先确认脚本输出中：
|
||||
- 该模型已经出现在“监控模型审计”里。
|
||||
- “监控别名”正确。
|
||||
- “限额来源”符合预期。
|
||||
4. 如果只是走默认限额，到这一步通常已经可以结束。
|
||||
5. 如果需要单模型硬限额，把脚本输出的 key 复制到 `volc_limit_sync.py` 的 `HARD_DAILY_LIMITS`。
|
||||
6. 只有当脚本显示资源包名无法靠常规版本号自动推导时，再去改 `MANUAL_MONITORED_MODEL_ALIASES`。
|
||||
7. 重启 LiteLLM 容器，让新配置生效。
|
||||
|
||||
### 删除监控模型
|
||||
|
||||
1. 从 `config.yaml` 里删掉对应模型条目。
|
||||
2. 运行 `python generate_keys.py`，确认它已经不在“监控模型审计”里。
|
||||
3. 如果该模型曾有单独的 `HARD_DAILY_LIMITS` 项，可以按需清理。
|
||||
4. 如果该模型对应的是特殊手工别名，且已经没有任何上游模型再用它，可以再清理 `MANUAL_MONITORED_MODEL_ALIASES`。
|
||||
5. 重启 LiteLLM 容器。
|
||||
|
||||
---
|
||||
|
||||
## 运行时行为概览
|
||||
|
||||
### 1. 自动发现纳入监控模型
|
||||
|
||||
`volc_limit_sync.py` 启动后会扫描 `config.yaml`，找出使用火山 `api_base` 的上游模型，并记录：
|
||||
|
||||
- 监控的上游模型名
|
||||
- `api_base` 对应的 hash
|
||||
- 由上游模型自动推导出来的监控别名
|
||||
|
||||
### 2. 资源包名归一化
|
||||
|
||||
火山接口返回资源包名后，脚本会先做归一化：
|
||||
|
||||
- 去掉资源包后缀
|
||||
- 统一小写
|
||||
- `_` 和 `.` 归一化成 `-`
|
||||
|
||||
之后再尝试通过：
|
||||
|
||||
- 自动版本号推导
|
||||
- 手工别名覆盖
|
||||
|
||||
把资源包归到实际纳入监控的上游模型。
|
||||
|
||||
### 3. 限额来源优先级
|
||||
|
||||
`HARD_DAILY_LIMITS` 的命中顺序是：
|
||||
|
||||
1. `model:hash`
|
||||
2. `model`
|
||||
3. 模糊规则
|
||||
4. `__DEFAULT__`
|
||||
|
||||
例如：
|
||||
|
||||
### 硬限制配置
|
||||
```python
|
||||
HARD_DAILY_LIMITS = {
|
||||
"doubao-seed-code-preview-251028": 4_800_000,
|
||||
"doubao-seed-2-0-pro-260215:ba0383d3": 1_900_000,
|
||||
"doubao-seed-2-0-lite-260215:ba0383d3": 1_900_000,
|
||||
"__DEFAULT__": 1_800_000,
|
||||
}
|
||||
```
|
||||
|
||||
### 模型映射
|
||||
```python
|
||||
VOLC_MODEL_MAP = {
|
||||
"doubao-seed-1-6": ["doubao-seed-1-6-250615", "doubao-seed-1-6-251015"],
|
||||
"deepseek-v3": "deepseek-v3-250324",
|
||||
}
|
||||
---
|
||||
|
||||
## 常见坑
|
||||
|
||||
### 1. 改了 `daily_limit.py` 但不生效
|
||||
|
||||
这是最容易踩的坑。
|
||||
|
||||
当前 `config.yaml` 没有启用 `daily_limit.py`，所以：
|
||||
|
||||
- 改 `daily_limit.py`
|
||||
- 改 `MODEL_QUOTAS`
|
||||
|
||||
都不会影响当前 LiteLLM 容器的启动和监控行为。
|
||||
|
||||
### 2. 只补了硬限额，但没有纳入监控
|
||||
|
||||
`HARD_DAILY_LIMITS` 只负责限额，不负责把模型纳入监控。
|
||||
|
||||
模型是否被监控，首先取决于：
|
||||
|
||||
- `config.yaml` 是否有该火山模型条目
|
||||
- 资源包名是否能被自动推导或手工别名覆盖正确归到该模型
|
||||
|
||||
### 3. 普通版本号模型不需要每次都手改映射
|
||||
|
||||
现在的运行逻辑已经支持从上游模型自动推导：
|
||||
|
||||
```text
|
||||
xxx-251201 -> xxx
|
||||
xxx-260215 -> xxx
|
||||
```
|
||||
|
||||
### 同步参数
|
||||
```python
|
||||
BUFFER = 300000 # 安全缓冲
|
||||
SYNC_INTERVAL = 60 # 基础同步间隔
|
||||
CHECK_INTERVAL = 10 # 检查间隔
|
||||
THRESHOLD = 300000 # 硬阈值
|
||||
```
|
||||
所以普通版本升级时，通常只需要改 `config.yaml`，再跑一遍 `python generate_keys.py` 审计确认。
|
||||
|
||||
## 高级功能
|
||||
---
|
||||
|
||||
### 1. 模型别名处理
|
||||
支持一个火山引擎模型映射到多个 LiteLLM 模型。
|
||||
## 常用命令
|
||||
|
||||
### 2. API Base 哈希
|
||||
通过对 API Base 进行哈希处理，支持多端点配置。
|
||||
### 审计当前纳入监控模型
|
||||
|
||||
### 3. 资源包清理
|
||||
自动清理响应中的后缀（如"_pack_data_collaboration"）。
|
||||
|
||||
### 4. 配置自动发现
|
||||
从 config.yaml 中自动发现火山引擎配置的模型。
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 常见问题
|
||||
|
||||
1. **Redis 连接失败**
|
||||
- 检查环境变量配置
|
||||
- 确保 Redis 服务正在运行
|
||||
|
||||
2. **模型未被识别**
|
||||
- 检查 config.yaml 中的模型配置
|
||||
- 确保 api_base 包含"volcengineapi"或"HUOSHAN_BASE_URL"
|
||||
|
||||
3. **同步失败**
|
||||
- 检查火山引擎 API 密钥
|
||||
- 验证网络连接
|
||||
|
||||
## 开发与维护
|
||||
|
||||
### 项目结构
|
||||
```
|
||||
litellm/
|
||||
├── volc_limit_sync.py # 主文件
|
||||
├── daily_limit.py # 每日限制计算
|
||||
├── debug_volc_sync.py # 调试工具
|
||||
├── debug_volc_sync_manual.py # 手动同步工具
|
||||
└── config.yaml # LiteLLM 配置
|
||||
```
|
||||
|
||||
### 调试模式
|
||||
```bash
|
||||
DAILY_LIMIT_LOG_LEVEL="DEBUG" python volc_limit_sync.py
|
||||
python generate_keys.py
|
||||
```
|
||||
|
||||
## 许可证
|
||||
### 直接运行同步监控
|
||||
|
||||
MIT License
|
||||
```bash
|
||||
python volc_limit_sync.py
|
||||
```
|
||||
|
||||
## 作者
|
||||
### LiteLLM 调试模式
|
||||
|
||||
OpenCode 社区
|
||||
```bash
|
||||
DAILY_LIMIT_LOG_LEVEL=DEBUG python volc_limit_sync.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 文件职责
|
||||
|
||||
```text
|
||||
litellm-config/
|
||||
├── config.yaml # 纳入监控模型的主入口
|
||||
├── volc_limit_sync.py # 当前生效的监控、同步、限额脚本
|
||||
├── generate_keys.py # 监控模型审计入口
|
||||
├── daily_limit.py # 旧脚本，当前 config 未启用
|
||||
├── debug_volc_sync.py # 资源包调试脚本
|
||||
└── debug_volc_sync_manual.py # 手动调试脚本
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 维护原则
|
||||
|
||||
- 先改 `config.yaml`，不要先改 `daily_limit.py`
|
||||
- 先跑 `python generate_keys.py` 审计，再决定要不要补 `HARD_DAILY_LIMITS`
|
||||
- 普通版本号模型优先依赖自动推导，不要手工补一堆重复映射
|
||||
- 只有特殊别名或一对多场景，才改 `MANUAL_MONITORED_MODEL_ALIASES`
|
||||
|
||||
+15
-7
@@ -3,11 +3,17 @@ x-definitions: &token_pricing
|
||||
input_cost_per_token: 1
|
||||
output_cost_per_token: 1
|
||||
|
||||
model_list:
|
||||
|
||||
|
||||
|
||||
#=======================火山平台，限时免费
|
||||
model_list:
|
||||
# ======================= 监控模型维护入口 =======================
|
||||
# 1. 这里的火山模型条目，就是“纳入监控模型”的主入口。
|
||||
# 2. 常规版本号模型（如 xxx-260215）会由 volc_limit_sync.py 自动推导成监控别名 xxx。
|
||||
# 3. 只有“资源包名与上游模型名不一致”或“一包多模型”时，才需要再改
|
||||
# volc_limit_sync.py 里的 MANUAL_MONITORED_MODEL_ALIASES。
|
||||
# 4. 改完后运行 `python generate_keys.py`，检查监控别名和 HARD_DAILY_LIMITS 的限额来源。
|
||||
|
||||
|
||||
|
||||
#=======================火山平台，限时免费
|
||||
|
||||
|
||||
|
||||
@@ -164,8 +170,10 @@ litellm_settings:
|
||||
# type: "redis" # 指定使用 Redis
|
||||
# supported_call_types: ["completion", "embedding", "acompletion"] # 支持的类型
|
||||
allowed_fails: 3 # cooldown model if it fails > 1 call in a minute.
|
||||
# 关键配置：注册自定义回调，实现单上游模型每日token限制的关键
|
||||
callbacks: ["volc_limit_sync.proxy_handler_instance"]
|
||||
# 当前生效入口：volc_limit_sync.py
|
||||
# 监控模型发现、监控别名推导、资源包同步、HARD_DAILY_LIMITS 都在这个脚本里。
|
||||
# daily_limit.py 当前未启用，仅保留作历史参考。
|
||||
callbacks: ["volc_limit_sync.proxy_handler_instance"]
|
||||
|
||||
router_settings: # router_settings are optional
|
||||
routing_strategy: simple-shuffle
|
||||
|
||||
+8
-1
@@ -8,6 +8,13 @@ from litellm.exceptions import ServiceUnavailableError, ContextWindowExceededErr
|
||||
import redis
|
||||
import traceback
|
||||
|
||||
# ==========================================
|
||||
# ⚠️ 入口说明
|
||||
# ==========================================
|
||||
# 当前 config.yaml 的 callbacks 使用的是 volc_limit_sync.proxy_handler_instance，
|
||||
# 不是本文件。除非你把 callbacks 切回 daily_limit.py，否则这里的 MODEL_QUOTAS
|
||||
# 修改不会影响当前 Docker 容器的监控与限额行为。
|
||||
|
||||
# ==========================================
|
||||
# 🛠️ 调试配置
|
||||
# ==========================================
|
||||
@@ -37,7 +44,7 @@ def dprint(msg):
|
||||
log(msg, level="INFO")
|
||||
|
||||
|
||||
dprint("Loading daily_limit.py module v20 (Robust Hash Fix)...")
|
||||
dprint("Loading daily_limit.py module v20 (legacy entry; current config uses volc_limit_sync.py)...")
|
||||
|
||||
# ==========================================
|
||||
# Redis 与 限额配置
|
||||
|
||||
@@ -35,6 +35,8 @@ SUFFIXES_TO_REMOVE = [
|
||||
VOLC_MODEL_MAP = {
|
||||
"doubao-seed-1-6": ["doubao-seed-1-6-250615", "doubao-seed-1-6-251015"],
|
||||
"doubao-seed-1-8": "doubao-seed-1-8-251228",
|
||||
"doubao-seed-2-0-pro": "doubao-seed-2-0-pro-260215",
|
||||
"doubao-seed-2-0-lite": "doubao-seed-2-0-lite-260215",
|
||||
"doubao-seed-code": "doubao-seed-code-preview-251028",
|
||||
"doubao-seed-translation": "doubao-seed-translation-250915",
|
||||
"deepseek-v3": "deepseek-v3-250324",
|
||||
|
||||
@@ -168,6 +168,8 @@ def fetch_all_packages():
|
||||
VOLC_MODEL_MAP = {
|
||||
"doubao-seed-1-6": ["doubao-seed-1-6-250615", "doubao-seed-1-6-251015"],
|
||||
"doubao-seed-1-8": "doubao-seed-1-8-251228",
|
||||
"doubao-seed-2-0-pro": "doubao-seed-2-0-pro-260215",
|
||||
"doubao-seed-2-0-lite": "doubao-seed-2-0-lite-260215",
|
||||
"doubao-seed-code": "doubao-seed-code-preview-251028",
|
||||
"doubao-seed-translation": "doubao-seed-translation-250915",
|
||||
"deepseek-v3": "deepseek-v3-250324",
|
||||
|
||||
+245
-76
@@ -1,18 +1,20 @@
|
||||
import os
|
||||
import yaml
|
||||
import ast
|
||||
import hashlib
|
||||
from urllib.parse import urlparse # ✅ 必须导入
|
||||
import os
|
||||
import re
|
||||
from typing import Dict, Any
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import yaml
|
||||
|
||||
# ==========================================
|
||||
# 🛠️ 配置
|
||||
# ==========================================
|
||||
CONFIG_FILE = "config.yaml"
|
||||
ENV_FILE = ".env"
|
||||
RUNTIME_FILE = "volc_limit_sync.py"
|
||||
|
||||
|
||||
def load_env(env_path: str) -> Dict[str, str]:
|
||||
def load_env(env_path):
|
||||
"""读取 .env 文件并返回字典"""
|
||||
env_vars = {}
|
||||
if not os.path.exists(env_path):
|
||||
@@ -32,7 +34,7 @@ def load_env(env_path: str) -> Dict[str, str]:
|
||||
return env_vars
|
||||
|
||||
|
||||
def resolve_env_var(value: str, env_vars: Dict[str, str]) -> str:
|
||||
def resolve_env_var(value, env_vars):
|
||||
"""
|
||||
处理环境变量替换，支持以下两种格式：
|
||||
1. ${VAR_NAME} - 标准格式
|
||||
@@ -41,17 +43,14 @@ def resolve_env_var(value: str, env_vars: Dict[str, str]) -> str:
|
||||
if not isinstance(value, str):
|
||||
return value
|
||||
|
||||
# 1. 处理 os.environ/VAR_NAME 格式
|
||||
if value.startswith("os.environ/"):
|
||||
var_name = value.replace("os.environ/", "")
|
||||
return env_vars.get(var_name, os.getenv(var_name, value))
|
||||
|
||||
# 2. 处理 ${VAR_NAME} 或 $VAR_NAME 格式
|
||||
pattern = re.compile(r"\$\{?([A-Z0-9_]+)\}?")
|
||||
|
||||
def replace_match(match):
|
||||
var_name = match.group(1)
|
||||
# 优先从 .env 读取，其次从系统环境变量读取
|
||||
return env_vars.get(var_name, os.getenv(var_name, match.group(0)))
|
||||
|
||||
return pattern.sub(replace_match, value)
|
||||
@@ -60,47 +59,253 @@ def resolve_env_var(value: str, env_vars: Dict[str, str]) -> str:
|
||||
def clean_model_name(name):
|
||||
if not name:
|
||||
return ""
|
||||
return name.replace("openai/", "").replace("azure/", "").replace("anthropic/", "")
|
||||
return (
|
||||
name.replace("openai/", "")
|
||||
.replace("azure/", "")
|
||||
.replace("anthropic/", "")
|
||||
.replace("volcengine/", "")
|
||||
)
|
||||
|
||||
|
||||
def generate_api_base_hash(api_base):
|
||||
"""
|
||||
[同步更新] 生成 api_base 的 Hash，强制忽略 URL 的路径部分。
|
||||
保持与 daily_limit.py v20 逻辑完全一致。
|
||||
保持与 volc_limit_sync.py 逻辑一致。
|
||||
"""
|
||||
if not api_base:
|
||||
return "default"
|
||||
|
||||
try:
|
||||
# 1. 预处理：去空格
|
||||
s = api_base.strip()
|
||||
|
||||
# 2. 解析 URL
|
||||
parsed = urlparse(s)
|
||||
|
||||
# 3. 提取 Scheme 和 Netloc (域名+端口)
|
||||
# 如果解析成功且包含 scheme 和 netloc，则重组
|
||||
if parsed.scheme and parsed.netloc:
|
||||
clean_base = f"{parsed.scheme}://{parsed.netloc}".lower()
|
||||
else:
|
||||
# 兜底：如果不是标准 URL，或者解析失败，尝试简单的去斜杠和小写化
|
||||
clean_base = s.rstrip('/').lower()
|
||||
|
||||
clean_base = s.rstrip("/").lower()
|
||||
except Exception:
|
||||
# 极端情况下的兜底
|
||||
clean_base = api_base.strip().rstrip('/').lower()
|
||||
clean_base = api_base.strip().rstrip("/").lower()
|
||||
|
||||
# 4. 计算 Hash
|
||||
return hashlib.md5(clean_base.encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
def main():
|
||||
print("🔍 Generating Daily Limit Keys from config.yaml (Robust Mode)...\n")
|
||||
def strip_model_version_suffix(model_name):
|
||||
"""Convert foo-bar-260215 -> foo-bar for common versioned upstream names."""
|
||||
if not model_name:
|
||||
return ""
|
||||
|
||||
parts = model_name.rsplit("-", 1)
|
||||
if len(parts) == 2 and len(parts[1]) == 6 and parts[1].isdigit():
|
||||
return parts[0]
|
||||
|
||||
return model_name
|
||||
|
||||
|
||||
def load_python_constant(file_path, constant_name, default=None):
|
||||
"""Safely read a literal constant from volc_limit_sync.py without importing it."""
|
||||
if not os.path.exists(file_path):
|
||||
return default
|
||||
|
||||
try:
|
||||
with open(file_path, "r", encoding="utf-8") as f:
|
||||
source = f.read()
|
||||
module = ast.parse(source, filename=file_path)
|
||||
except Exception as e:
|
||||
print(f"⚠️ Warning: failed to parse {file_path}: {e}")
|
||||
return default
|
||||
|
||||
for node in module.body:
|
||||
if not isinstance(node, ast.Assign):
|
||||
continue
|
||||
for target in node.targets:
|
||||
if isinstance(target, ast.Name) and target.id == constant_name:
|
||||
try:
|
||||
return ast.literal_eval(node.value)
|
||||
except Exception as e:
|
||||
print(
|
||||
f"⚠️ Warning: failed to read constant {constant_name} from {file_path}: {e}"
|
||||
)
|
||||
return default
|
||||
|
||||
return default
|
||||
|
||||
|
||||
def build_monitored_model_aliases(monitored_models, manual_aliases):
|
||||
"""Mirror the monitored-model alias logic used by volc_limit_sync.py."""
|
||||
alias_map = {}
|
||||
|
||||
def add_alias(alias_name, target_model):
|
||||
clean_alias = clean_model_name(alias_name)
|
||||
clean_target = clean_model_name(target_model)
|
||||
if not clean_alias or not clean_target:
|
||||
return
|
||||
alias_map.setdefault(clean_alias, [])
|
||||
if clean_target not in alias_map[clean_alias]:
|
||||
alias_map[clean_alias].append(clean_target)
|
||||
|
||||
for target_model in sorted(set(monitored_models)):
|
||||
add_alias(target_model, target_model)
|
||||
|
||||
base_alias = strip_model_version_suffix(target_model)
|
||||
if base_alias != target_model:
|
||||
add_alias(base_alias, target_model)
|
||||
|
||||
for alias_name, targets in (manual_aliases or {}).items():
|
||||
if isinstance(targets, list):
|
||||
for target_model in targets:
|
||||
add_alias(alias_name, target_model)
|
||||
else:
|
||||
add_alias(alias_name, targets)
|
||||
|
||||
return alias_map
|
||||
|
||||
|
||||
def is_monitored_volc_entry(api_base_raw, api_base_resolved):
|
||||
if not isinstance(api_base_raw, str):
|
||||
return False
|
||||
resolved_text = str(api_base_resolved or "").lower()
|
||||
return "HUOSHAN_BASE_URL" in api_base_raw or "volcengineapi" in resolved_text
|
||||
|
||||
|
||||
def collect_monitored_entries(config, env_vars):
|
||||
entries = {}
|
||||
model_list = config.get("model_list", [])
|
||||
|
||||
for model in model_list:
|
||||
litellm_params = model.get("litellm_params", {})
|
||||
api_base_raw = litellm_params.get("api_base", "")
|
||||
api_base_resolved = resolve_env_var(api_base_raw, env_vars)
|
||||
|
||||
if not is_monitored_volc_entry(api_base_raw, api_base_resolved):
|
||||
continue
|
||||
|
||||
model_alias = model.get("model_name", "unknown")
|
||||
upstream_model = clean_model_name(litellm_params.get("model", ""))
|
||||
if not upstream_model:
|
||||
continue
|
||||
|
||||
base_hash = generate_api_base_hash(api_base_resolved)
|
||||
final_key = f"{upstream_model}:{base_hash}"
|
||||
|
||||
entry = entries.setdefault(
|
||||
final_key,
|
||||
{
|
||||
"config_aliases": [],
|
||||
"upstream_model": upstream_model,
|
||||
"api_base_raw": api_base_raw,
|
||||
"api_base_resolved": api_base_resolved,
|
||||
"base_hash": base_hash,
|
||||
"final_key": final_key,
|
||||
},
|
||||
)
|
||||
|
||||
if model_alias not in entry["config_aliases"]:
|
||||
entry["config_aliases"].append(model_alias)
|
||||
|
||||
return list(entries.values())
|
||||
|
||||
|
||||
def get_limit_rule(clean_model, base_hash, hard_limits):
|
||||
specific_key = f"{clean_model}:{base_hash}"
|
||||
if specific_key in hard_limits:
|
||||
return specific_key, hard_limits[specific_key]
|
||||
|
||||
if clean_model in hard_limits:
|
||||
return clean_model, hard_limits[clean_model]
|
||||
|
||||
for quota_model, limit_val in hard_limits.items():
|
||||
if quota_model == "__DEFAULT__":
|
||||
continue
|
||||
if ":" not in quota_model and quota_model in clean_model:
|
||||
return quota_model, limit_val
|
||||
|
||||
if "__DEFAULT__" in hard_limits:
|
||||
return "__DEFAULT__", hard_limits["__DEFAULT__"]
|
||||
|
||||
return None, None
|
||||
|
||||
|
||||
def build_reverse_alias_map(alias_map):
|
||||
reverse_map = {}
|
||||
for alias_name, targets in alias_map.items():
|
||||
for target_model in targets:
|
||||
reverse_map.setdefault(target_model, [])
|
||||
if alias_name not in reverse_map[target_model]:
|
||||
reverse_map[target_model].append(alias_name)
|
||||
return reverse_map
|
||||
|
||||
|
||||
def format_limit_source(rule_name, limit_value, final_key, upstream_model):
|
||||
if rule_name is None:
|
||||
return "未配置"
|
||||
if rule_name == final_key:
|
||||
return f"自定义键 {rule_name} => {int(limit_value):,}"
|
||||
if rule_name == upstream_model:
|
||||
return f"模型级规则 {rule_name} => {int(limit_value):,}"
|
||||
if rule_name == "__DEFAULT__":
|
||||
return f"默认规则 __DEFAULT__ => {int(limit_value):,}"
|
||||
return f"模糊规则 {rule_name} => {int(limit_value):,}"
|
||||
|
||||
|
||||
def print_monitored_audit(entries, reverse_alias_map, hard_limits):
|
||||
print("=== 监控模型审计 / config.yaml -> volc_limit_sync.py ===")
|
||||
print()
|
||||
|
||||
for entry in entries:
|
||||
monitor_aliases = reverse_alias_map.get(
|
||||
entry["upstream_model"], [entry["upstream_model"]]
|
||||
)
|
||||
rule_name, limit_value = get_limit_rule(
|
||||
entry["upstream_model"], entry["base_hash"], hard_limits
|
||||
)
|
||||
|
||||
print(f"- Config alias(es): {', '.join(entry['config_aliases'])}")
|
||||
print(f" Upstream model : {entry['upstream_model']}")
|
||||
print(f" 监控别名 : {', '.join(monitor_aliases)}")
|
||||
print(f" Generated key : {entry['final_key']}")
|
||||
print(
|
||||
f" 限额来源 : {format_limit_source(rule_name, limit_value, entry['final_key'], entry['upstream_model'])}"
|
||||
)
|
||||
print()
|
||||
|
||||
|
||||
def print_maintenance_guide(entries, manual_aliases, hard_limits):
|
||||
default_limit = hard_limits.get("__DEFAULT__")
|
||||
|
||||
print("=== 维护入口 ===")
|
||||
print("1. 在 config.yaml 的 model_list 中增删火山模型，这是“纳入监控模型”的主入口。")
|
||||
print("2. 改完后运行 python generate_keys.py，先看上面的“监控别名”和“限额来源”。")
|
||||
print("3. 常规版本号模型（如 xxx-260215）会自动推导成监控别名 xxx，通常不用再改 volc_limit_sync.py。")
|
||||
print("4. 只有两种情况才需要改 volc_limit_sync.py 的 MANUAL_MONITORED_MODEL_ALIASES：")
|
||||
print(" - 资源包名和上游模型名不一致。")
|
||||
print(" - 一个资源包要映射到多个上游模型。")
|
||||
if manual_aliases:
|
||||
print(
|
||||
f"5. 当前手工别名覆盖: {', '.join(sorted(manual_aliases.keys()))}"
|
||||
)
|
||||
print()
|
||||
|
||||
defaulted_entries = []
|
||||
for entry in entries:
|
||||
rule_name, _ = get_limit_rule(
|
||||
entry["upstream_model"], entry["base_hash"], hard_limits
|
||||
)
|
||||
if rule_name == "__DEFAULT__":
|
||||
defaulted_entries.append(entry)
|
||||
|
||||
if defaulted_entries and default_limit is not None:
|
||||
print("📋 可选：以下监控模型当前走 __DEFAULT__，如需单模型硬限额可复制到 HARD_DAILY_LIMITS")
|
||||
print("HARD_DAILY_LIMITS = {")
|
||||
for entry in defaulted_entries:
|
||||
print(f' "{entry["final_key"]}": {int(default_limit)},')
|
||||
print("}")
|
||||
print()
|
||||
|
||||
|
||||
def main():
|
||||
print("🔍 Auditing monitored Volcengine models from config.yaml...\n")
|
||||
|
||||
# 1. Load Env
|
||||
env_vars = load_env(ENV_FILE)
|
||||
|
||||
# 2. Load Config
|
||||
if not os.path.exists(CONFIG_FILE):
|
||||
print(f"❌ Error: {CONFIG_FILE} not found!")
|
||||
return
|
||||
@@ -112,60 +317,24 @@ def main():
|
||||
print(f"❌ Error parsing yaml: {e}")
|
||||
return
|
||||
|
||||
model_list = config.get("model_list", [])
|
||||
if not model_list:
|
||||
print("⚠️ No models found in config.yaml")
|
||||
entries = collect_monitored_entries(config or {}, env_vars)
|
||||
if not entries:
|
||||
print("⚠️ No monitored Volcengine models found in config.yaml")
|
||||
return
|
||||
|
||||
results = []
|
||||
seen_keys = set()
|
||||
entries.sort(key=lambda item: item["upstream_model"])
|
||||
|
||||
# Table Header
|
||||
# Model Alias | Upstream Model | API Base (Env/URL) | Generated Key
|
||||
header = f"{'Model Name (Alias)':<25} | {'Upstream Model':<30} | {'API Base (Env/URL)':<40} | {'Generated Key'}"
|
||||
print(header)
|
||||
print("-" * len(header))
|
||||
manual_aliases = load_python_constant(
|
||||
RUNTIME_FILE, "MANUAL_MONITORED_MODEL_ALIASES", default={}
|
||||
) or {}
|
||||
hard_limits = load_python_constant(RUNTIME_FILE, "HARD_DAILY_LIMITS", default={}) or {}
|
||||
|
||||
for model in model_list:
|
||||
litellm_params = model.get("litellm_params", {})
|
||||
monitored_upstreams = [entry["upstream_model"] for entry in entries]
|
||||
alias_map = build_monitored_model_aliases(monitored_upstreams, manual_aliases)
|
||||
reverse_alias_map = build_reverse_alias_map(alias_map)
|
||||
|
||||
# Get raw values
|
||||
model_alias = model.get("model_name", "unknown")
|
||||
upstream_model = litellm_params.get("model", "")
|
||||
api_base_raw = litellm_params.get("api_base", "")
|
||||
|
||||
# Resolve env vars for hash generation
|
||||
# ⚠️ 这里非常重要：必须先解析出真实的 URL，再进行 Hash
|
||||
api_base_resolved = resolve_env_var(api_base_raw, env_vars)
|
||||
|
||||
# Generate components
|
||||
clean_upstream = clean_model_name(upstream_model)
|
||||
|
||||
# 使用同步后的 Hash 逻辑
|
||||
base_hash = generate_api_base_hash(api_base_resolved)
|
||||
|
||||
# Generate final key
|
||||
final_key = f"{clean_upstream}:{base_hash}"
|
||||
|
||||
# Deduplicate
|
||||
if final_key in seen_keys:
|
||||
continue
|
||||
seen_keys.add(final_key)
|
||||
|
||||
# Truncate api_base_raw if it's too long for display
|
||||
display_api_base = api_base_raw
|
||||
if len(display_api_base) > 37:
|
||||
display_api_base = display_api_base[:34] + "..."
|
||||
|
||||
print(f"{model_alias:<25} | {clean_upstream:<30} | {display_api_base:<40} | {final_key}")
|
||||
results.append(f'"{final_key}": 1500000')
|
||||
|
||||
print("\n📋 [Copy & Paste into daily_limit.py MODEL_QUOTAS]:")
|
||||
print("-" * 50)
|
||||
print("MODEL_QUOTAS = {")
|
||||
print(",\n".join(f" {item}" for item in results))
|
||||
print("}")
|
||||
print("-" * 50)
|
||||
print_monitored_audit(entries, reverse_alias_map, hard_limits)
|
||||
print_maintenance_guide(entries, manual_aliases, hard_limits)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+88
-42
@@ -91,24 +91,18 @@ SUFFIXES_TO_REMOVE = [
|
||||
"-free-infer-res-pack", # Matches: DeepSeek-V3-free-infer-res-pack
|
||||
]
|
||||
|
||||
# Volcengine Name -> LiteLLM Model Name Map
|
||||
# Key is the normalized (lowercase, hyphens) name from Volcengine API
|
||||
VOLC_MODEL_MAP = {
|
||||
# Manual package-name aliases for monitored models.
|
||||
# 常规版本号模型（如 xxx-260215）会根据 config.yaml 自动推导成 xxx，
|
||||
# 这里只保留特殊别名或一对多映射，减少新增/删除监控模型时需要手改的地方。
|
||||
MANUAL_MONITORED_MODEL_ALIASES = {
|
||||
"doubao-seed-1-6": ["doubao-seed-1-6-250615", "doubao-seed-1-6-251015"],
|
||||
"doubao-seed-1-8": "doubao-seed-1-8-251228",
|
||||
"doubao-seed-code": "doubao-seed-code-preview-251028",
|
||||
"doubao-seed-translation": "doubao-seed-translation-250915",
|
||||
"deepseek-v3": "deepseek-v3-250324",
|
||||
"deepseek-v3-1": "deepseek-v3-1-terminus",
|
||||
"deepseek-v3-2": "deepseek-v3-2-251201",
|
||||
"deepseek-r1": "deepseek-r1-250528",
|
||||
"kimi-k2": "kimi-k2-thinking-251104",
|
||||
"glm-4-7": "glm-4-7-251222",
|
||||
}
|
||||
|
||||
# Whitelist of (model_name, api_base_hash) managed by VolcSync
|
||||
# Populated by active_monitor_loop scanning config.yaml
|
||||
VOLC_MANAGED_PAIRS = set()
|
||||
# Auto-discovered monitored upstream models and endpoint pairs from config.yaml
|
||||
MONITORED_MODEL_PAIRS = set()
|
||||
MONITORED_MODEL_ALIASES = {}
|
||||
|
||||
# ==========================================
|
||||
# 🛑 Hard Daily Limits (Local Override / 本地每日硬限额)
|
||||
@@ -122,6 +116,9 @@ VOLC_MANAGED_PAIRS = set()
|
||||
HARD_DAILY_LIMITS = {
|
||||
# Specific Limit for Doubao Code
|
||||
"doubao-seed-code-preview-251028": 4_800_000,
|
||||
# Specific Limits for Doubao Seed 2.0 models on HUOSHAN_BASE_URL
|
||||
"doubao-seed-2-0-pro-260215:ba0383d3": 1_900_000,
|
||||
"doubao-seed-2-0-lite-260215:ba0383d3": 1_900_000,
|
||||
# Default Limit for all other models (Universal Fallback)
|
||||
"__DEFAULT__": 1_800_000,
|
||||
}
|
||||
@@ -189,8 +186,53 @@ def clean_model_name(name):
|
||||
)
|
||||
|
||||
|
||||
def strip_model_version_suffix(model_name):
|
||||
"""Convert foo-bar-260215 -> foo-bar for common versioned upstream names."""
|
||||
if not model_name:
|
||||
return ""
|
||||
|
||||
parts = model_name.rsplit("-", 1)
|
||||
if len(parts) == 2 and len(parts[1]) == 6 and parts[1].isdigit():
|
||||
return parts[0]
|
||||
|
||||
return model_name
|
||||
|
||||
|
||||
def build_monitored_model_aliases(monitored_models):
|
||||
"""
|
||||
Build alias -> monitored upstream model(s) mapping.
|
||||
Common versioned names are derived automatically from config.yaml.
|
||||
"""
|
||||
alias_map = {}
|
||||
|
||||
def add_alias(alias_name, target_model):
|
||||
clean_alias = clean_model_name(alias_name)
|
||||
clean_target = clean_model_name(target_model)
|
||||
if not clean_alias or not clean_target:
|
||||
return
|
||||
alias_map.setdefault(clean_alias, [])
|
||||
if clean_target not in alias_map[clean_alias]:
|
||||
alias_map[clean_alias].append(clean_target)
|
||||
|
||||
for target_model in sorted(monitored_models):
|
||||
add_alias(target_model, target_model)
|
||||
|
||||
base_alias = strip_model_version_suffix(target_model)
|
||||
if base_alias != target_model:
|
||||
add_alias(base_alias, target_model)
|
||||
|
||||
for alias_name, targets in MANUAL_MONITORED_MODEL_ALIASES.items():
|
||||
if isinstance(targets, list):
|
||||
for target_model in targets:
|
||||
add_alias(alias_name, target_model)
|
||||
else:
|
||||
add_alias(alias_name, targets)
|
||||
|
||||
return alias_map
|
||||
|
||||
|
||||
def normalize_volc_name(raw_name):
|
||||
"""Volcengine API response cleaning."""
|
||||
"""Normalize Volcengine package names before monitored-model resolution."""
|
||||
if not raw_name:
|
||||
return "unknown"
|
||||
clean_name = raw_name
|
||||
@@ -200,7 +242,7 @@ def normalize_volc_name(raw_name):
|
||||
break
|
||||
clean_name = clean_name.replace("_", "-").replace(".", "-")
|
||||
clean_name = clean_name.lower().strip()
|
||||
return VOLC_MODEL_MAP.get(clean_name, clean_name)
|
||||
return clean_name
|
||||
|
||||
|
||||
def sign(key, msg):
|
||||
@@ -573,8 +615,8 @@ class DailyLimitHandler(CustomLogger):
|
||||
# This prevents tracking NewAPI aliases that point to other providers
|
||||
current_hash = generate_api_base_hash(api_base)
|
||||
if (
|
||||
VOLC_MANAGED_PAIRS
|
||||
and (clean_model, current_hash) not in VOLC_MANAGED_PAIRS
|
||||
MONITORED_MODEL_PAIRS
|
||||
and (clean_model, current_hash) not in MONITORED_MODEL_PAIRS
|
||||
):
|
||||
return
|
||||
|
||||
@@ -726,12 +768,9 @@ def process_packages_breakdown(packages):
|
||||
available = pkg.get("AvailableAmount", 0)
|
||||
unit = pkg.get("Unit", "token")
|
||||
|
||||
# 2. Handle 1-to-many mapping (e.g. doubao-seed-1-6 -> [v1, v2])
|
||||
target_models = []
|
||||
if isinstance(norm_result, list):
|
||||
target_models = norm_result
|
||||
else:
|
||||
target_models = [norm_result]
|
||||
# 2. Resolve to monitored upstream models.
|
||||
# 普通版本号模型会自动从 config.yaml 推导；特殊别名走手工覆盖。
|
||||
target_models = MONITORED_MODEL_ALIASES.get(norm_result, [norm_result])
|
||||
|
||||
# Debug log
|
||||
if (
|
||||
@@ -740,7 +779,7 @@ def process_packages_breakdown(packages):
|
||||
or "glm" in str(target_models)
|
||||
):
|
||||
print(
|
||||
f"DEBUG: Pkg Raw='{target_raw}' -> Norm='{norm_result}', Avail={int(float(available)):,}",
|
||||
f"DEBUG: Pkg Raw='{target_raw}' -> Norm='{norm_result}' -> Targets={target_models}, Avail={int(float(available)):,}",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
@@ -768,9 +807,10 @@ def process_packages_breakdown(packages):
|
||||
|
||||
def active_monitor_loop():
|
||||
# Config Discovery
|
||||
global VOLC_MANAGED_PAIRS
|
||||
VOLC_MANAGED_PAIRS.clear()
|
||||
INTERESTED_MODELS = set()
|
||||
global MONITORED_MODEL_PAIRS, MONITORED_MODEL_ALIASES
|
||||
MONITORED_MODEL_PAIRS.clear()
|
||||
MONITORED_MODEL_ALIASES.clear()
|
||||
MONITORED_MODELS = set()
|
||||
try:
|
||||
if os.path.exists("config.yaml"):
|
||||
with open("config.yaml", "r", encoding="utf-8") as f:
|
||||
@@ -778,7 +818,7 @@ def active_monitor_loop():
|
||||
lines = content.split("\n")
|
||||
|
||||
print(
|
||||
"Auto-discovering Volcengine models from config.yaml / 正在从 config.yaml 自动发现火山引擎模型...",
|
||||
"Auto-discovering monitored Volcengine models from config.yaml / 正在从 config.yaml 自动发现纳入监控的火山模型...",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
@@ -800,12 +840,12 @@ def active_monitor_loop():
|
||||
and current_entry.get("upstream_model")
|
||||
and current_entry.get("api_base_val")
|
||||
):
|
||||
INTERESTED_MODELS.add(current_entry["upstream_model"])
|
||||
MONITORED_MODELS.add(current_entry["upstream_model"])
|
||||
# Compute Hash
|
||||
h = generate_api_base_hash(current_entry["api_base_val"])
|
||||
VOLC_MANAGED_PAIRS.add((current_entry["upstream_model"], h))
|
||||
MONITORED_MODEL_PAIRS.add((current_entry["upstream_model"], h))
|
||||
print(
|
||||
f"✅ [Config] Added Managed Pair: {current_entry['upstream_model']} | Hash: {h} | Base: {str(current_entry['api_base_val'])[:30]}...",
|
||||
f"✅ [Config] Added monitored pair: {current_entry['upstream_model']} | Hash: {h} | Base: {str(current_entry['api_base_val'])[:30]}...",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
@@ -866,22 +906,28 @@ def active_monitor_loop():
|
||||
and current_entry.get("upstream_model")
|
||||
and current_entry.get("api_base_val")
|
||||
):
|
||||
INTERESTED_MODELS.add(current_entry["upstream_model"])
|
||||
MONITORED_MODELS.add(current_entry["upstream_model"])
|
||||
h = generate_api_base_hash(current_entry["api_base_val"])
|
||||
VOLC_MANAGED_PAIRS.add((current_entry["upstream_model"], h))
|
||||
MONITORED_MODEL_PAIRS.add((current_entry["upstream_model"], h))
|
||||
if DEBUG_MODE:
|
||||
print(
|
||||
f"DEBUG: Added Managed Pair: {current_entry['upstream_model']} @ {h}"
|
||||
f"DEBUG: Added monitored pair: {current_entry['upstream_model']} @ {h}"
|
||||
)
|
||||
|
||||
MONITORED_MODEL_ALIASES = build_monitored_model_aliases(MONITORED_MODELS)
|
||||
print(
|
||||
f"Auto-discovered / 已自动发现 {len(INTERESTED_MODELS)} Active Volcengine models: {INTERESTED_MODELS}",
|
||||
f"Auto-discovered / 已自动发现 {len(MONITORED_MODELS)} 个纳入监控的火山上游模型: {MONITORED_MODELS}",
|
||||
flush=True,
|
||||
)
|
||||
print(
|
||||
f"Managed Pairs Hashes: {[p[1] for p in VOLC_MANAGED_PAIRS]}",
|
||||
f"Monitored model pair hashes / 监控模型端点哈希: {[p[1] for p in MONITORED_MODEL_PAIRS]}",
|
||||
flush=True,
|
||||
)
|
||||
if DEBUG_MODE:
|
||||
print(
|
||||
f"DEBUG: Monitored model aliases / 监控模型别名: {MONITORED_MODEL_ALIASES}",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Config Auto-discovery failed / 配置自动发现失败: {e}", flush=True)
|
||||
@@ -986,7 +1032,7 @@ def active_monitor_loop():
|
||||
burst_triggered = False
|
||||
burst_reason = ""
|
||||
|
||||
for model in INTERESTED_MODELS:
|
||||
for model in MONITORED_MODELS:
|
||||
# 1. Get Current Usage
|
||||
curr_usage_key = f"limit:dynamic:volcengine:usage:{model}"
|
||||
curr_usage_val = redis_client.get(curr_usage_key)
|
||||
@@ -1062,19 +1108,19 @@ def active_monitor_loop():
|
||||
for target in targets:
|
||||
# STRICT FILTER: Only sync models explicitly configured with VolcEngine Base URL
|
||||
# User Requirement: Do NOT sync or limit models that share names but use other providers (e.g. NewAPI)
|
||||
if target not in INTERESTED_MODELS:
|
||||
if target not in MONITORED_MODELS:
|
||||
if DEBUG_MODE:
|
||||
print(
|
||||
f"DEBUG: Skipping {target} (Not in INTERESTED_MODELS / 未在白名单)",
|
||||
f"DEBUG: Skipping {target} (Not in monitored models / 未纳入监控模型)",
|
||||
flush=True,
|
||||
)
|
||||
continue
|
||||
|
||||
# Fetch Current Usage from Redis (to calculate Total Pool)
|
||||
# We need to find the correct hash for this model from VOLC_MANAGED_PAIRS
|
||||
# We need to find the correct hash for this model from MONITORED_MODEL_PAIRS
|
||||
current_usage = 0
|
||||
target_hashes = [
|
||||
p[1] for p in VOLC_MANAGED_PAIRS if p[0] == target
|
||||
p[1] for p in MONITORED_MODEL_PAIRS if p[0] == target
|
||||
]
|
||||
|
||||
# Sum usage across all configured endpoints for this model (usually just one)
|
||||
@@ -1222,7 +1268,7 @@ def active_monitor_loop():
|
||||
|
||||
# Update snapshots for next iteration delta calculation
|
||||
current_usage_global_snapshot = 0
|
||||
for m in INTERESTED_MODELS:
|
||||
for m in MONITORED_MODELS:
|
||||
uk = f"limit:dynamic:volcengine:usage:{m}"
|
||||
uv = redis_client.get(uk)
|
||||
u_val = float(uv) if uv else 0.0
|
||||
|
||||
Reference in New Issue
Block a user