kim/litellm-config
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
112 Commits 1 Branch 0 Tags
ee1245621d4b2651d0b15d446f6175c00cae982d
Commit Graph

53 Commits

Author SHA1 Message Date
kim 2cdd1a3295 feat(volc_sync): optimize frequency intervals and logging 2026-01-29 16:12:26 +08:00
kim 3156bbe431 feat: merge daily_limit functionality into volc_limit_sync for single-file dynamic rate limiting 
This commit merges the interception and soft-eviction logic from daily_limit.py into volc_limit_sync.py.
It enables a unified solution where:
1. Running the script directly starts the sync daemon.
2. Importing 'proxy_handler_instance' provides the LiteLLM callback handler.
The handler now respects dynamic limits synced to Redis.
 2026-01-29 15:11:14 +08:00
kim 8630f287b4 feat: add dynamic rate limiting for Volcengine models 
This commit introduces a dual-layer rate limiting system:
1. volc_limit_sync.py: A daemon that adaptively syncs Volcengine API balances to Redis based on traffic and safety margins.
2. daily_limit.py: Updated to enforce dynamic limits by checking Redis (Current Usage + Remaining Balance) and handling soft evictions.
 2026-01-29 14:51:55 +08:00