[LLM] merge code from fastdeploy

[kevincheng2]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

Others

PR changes

Others

Description

merge code from fastdeploy

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.66%. Comparing base (d039ad2) to head (490e354).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #9791      +/-   ##
===========================================
+ Coverage    52.05%   52.66%   +0.60%     
===========================================
  Files          735      730       -5     
  Lines       116672   115270    -1402     
===========================================
- Hits         60738    60703      -35     
+ Misses       55934    54567    -1367     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM