合并自建歌曲源支线（selfhost-source）+ 版本号按约定修正为 1.6.0
版本号：
- 1.5.11 越界（规则 3 规定 patch ≤ 10）→ 修正为 1.6.0（进位到 minor、patch 归 0）
- versionCode 43 不变：43 从未对外发布，本次修正只是把命名拉回约定
合并支线（原在工作区根目录、App 仓库之外，DESIGN.md 自称"支线"）：
- selfhost-source/server/ 自建歌曲源服务器（纯 Python 标准库、零 pip 依赖）
- selfhost-source/deploy/、DESIGN.md、LIBRARY.md、build_inventory.py
- selfhost-source/inventory/ 小清单（playlists.csv、song-stats.md 等）
- ⚠️ inventory 生成物（songs-*.csv / singers-top.csv，共 84MB）不入库，
已写入 .gitignore；需要时用 build_inventory.py 依据 muse.db 重新生成
同时入库的 App 侧支线改动（此前为工作树未提交状态）：
- Whoami.kt（whoami 设备心跳）、DeviceIdentity.kt（设备身份伪装）
- PushDownload.kt / PushDownloadClient.kt（点歌推送）
- AdVideoFilter 带 query/fragment 的地址闸门；KTV_BRIDGE_API 11 → 13
- 修 ktv_api.js 的 global.KtvBridgeApiVersion（12 → 13，与注释及 Kotlin 两处校验一致）
- 对应单测：WhoamiTest / DeviceIdentityTest / PushDownloadTest / AdVideoFilterTest（共 39 例）
文档：
- DESIGN.md 的"在 App 仓库之外"约定改为"已并入仓库统一版本化"
- AGENTS.md 版本号行更新为 1.6.0/43，并新增支线并入说明
验证：assembleRelease + testReleaseUnitTest 全过（39 例）
This commit is contained in:
zijunchiang
@@ -0,0 +1,553 @@
|
||||
#!/usr/bin/env python3
|
||||
"""反向代理（IIS / ARR）支持自测：可信代理头、真实客户端 IP、HTTPS 地址、探活、413、HEAD/Range、IIS 配置。
|
||||
|
||||
覆盖：
|
||||
① `trusted_proxies` 判定（开关关闭 / 对端不可信 → **忽略** XFF；可信 → 取 XFF 最左侧非代理 IP）；
|
||||
② 推送限流按**真实客户端 IP** 分桶（反代后不再所有设备共用一份额度），且来源 IP 写进事件日志；
|
||||
③ `X-Forwarded-Proto/Host` **仅可信时**生效：`/music/do.php` 返回 https 公网地址；
|
||||
④ `/healthz` 微秒级返回、不查库、不碰媒体目录（打桩断言未被调用）；
|
||||
⑤ 请求体超 `max_body_bytes` → 413 + 中文说明；
|
||||
⑥ HTTP 层（真起本地端口、走 socket）：HEAD/Range 206 + Content-Length + Accept-Ranges、
|
||||
每个响应都有精确 Content-Length（keep-alive 的前提）、连接可复用；
|
||||
⑦ `deploy/iis/` 交付物静态校验：web.config 可解析、含 6677 反代规则、**不含 6678**、
|
||||
含 `push_download`/`healthz`/`media`。
|
||||
|
||||
不联网、不起外网连接（socket 用例只绑 127.0.0.1 的临时端口）。
|
||||
"""
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
import urllib.parse
|
||||
import xml.etree.ElementTree as ET
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from ktv import admin
|
||||
from ktv import health as health_mod
|
||||
from ktv import proxy
|
||||
from ktv import service
|
||||
from ktv import web
|
||||
from ktv.config import Config
|
||||
from ktv.store import Store
|
||||
from ktv.web import Request
|
||||
|
||||
HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
IIS_DIR = os.path.join(os.path.dirname(HERE), "deploy", "iis")
|
||||
TID = 7300123
|
||||
PUSH_URL = f"http://cdn.example.com/480p/{TID}.ts?token=abc&t=1700000000"
|
||||
|
||||
|
||||
def req(method, path, query=None, body=b"", headers=None, client_ip="", peer_ip="",
|
||||
via_proxy=False, proto="", host="", xff=""):
|
||||
"""构造 Request；反代场景用 `via_proxy=True` + `proto/host/xff`（= web.py 注入的口径）。"""
|
||||
return Request(method, path, query or {}, headers or {}, body, client_ip=client_ip,
|
||||
peer_ip=peer_ip or client_ip, via_proxy=via_proxy,
|
||||
proxy_source="x-forwarded-for" if (via_proxy and xff) else "tcp",
|
||||
forwarded_proto=proto, forwarded_host=host, forwarded_for=xff)
|
||||
|
||||
|
||||
def form(**kv) -> bytes:
|
||||
return urllib.parse.urlencode(kv).encode()
|
||||
|
||||
|
||||
# ══════════════════════ ① 可信代理判定 ══════════════════════
|
||||
|
||||
class TestTrustedProxies(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.cfg = Config(os.path.join(self.tmp, "c.json"))
|
||||
|
||||
def test_default_is_off(self):
|
||||
"""默认：开关关闭 + 可信列表为空 → 代理头一律不采信（安全默认）。"""
|
||||
self.assertFalse(self.cfg.get("trust_proxy_headers"))
|
||||
self.assertEqual(self.cfg.get("trusted_proxies"), [])
|
||||
info = proxy.resolve("203.0.113.7", {"X-Forwarded-For": "198.51.100.9"}, self.cfg)
|
||||
self.assertEqual(info.ip, "203.0.113.7")
|
||||
self.assertFalse(info.via_proxy)
|
||||
self.assertEqual(info.source, "tcp")
|
||||
|
||||
def test_switch_on_but_peer_not_trusted_ignores_headers(self):
|
||||
"""开关开着、但直连对端**不在** trusted_proxies → 仍然一个头都不看（防伪造）。"""
|
||||
self.cfg.update({"trust_proxy_headers": True, "trusted_proxies": ["192.168.100.10"]})
|
||||
info = proxy.resolve("203.0.113.7", {"X-Forwarded-For": "198.51.100.9",
|
||||
"X-Real-IP": "198.51.100.8",
|
||||
"X-Forwarded-Proto": "https",
|
||||
"X-Forwarded-Host": "evil.example.com"}, self.cfg)
|
||||
self.assertEqual(info.ip, "203.0.113.7")
|
||||
self.assertFalse(info.via_proxy)
|
||||
self.assertEqual(info.proto, "") # Proto/Host 也不采信
|
||||
self.assertEqual(info.host, "")
|
||||
|
||||
def test_trusted_peer_takes_leftmost_non_proxy(self):
|
||||
"""可信对端 → 取 XFF **最左侧非代理** IP（客户端伪造的可信代理 IP 会被跳过）。"""
|
||||
self.cfg.update({"trust_proxy_headers": True, "trusted_proxies": ["127.0.0.1"]})
|
||||
info = proxy.resolve("127.0.0.1",
|
||||
{"X-Forwarded-For": "127.0.0.1, 203.0.113.9, 127.0.0.1"}, self.cfg)
|
||||
self.assertEqual(info.ip, "203.0.113.9")
|
||||
self.assertTrue(info.via_proxy)
|
||||
self.assertEqual(info.source, "x-forwarded-for")
|
||||
self.assertEqual(info.peer_ip, "127.0.0.1")
|
||||
|
||||
def test_cidr_and_leftmost_fallback(self):
|
||||
"""CIDR 命中；整条链都是可信代理时退回最左侧（原始客户端），不退化成代理地址。"""
|
||||
self.cfg.update({"trust_proxy_headers": True,
|
||||
"trusted_proxies": ["127.0.0.1", "192.168.100.0/24"]})
|
||||
info = proxy.resolve("192.168.100.5", {"X-Forwarded-For": "10.1.2.3, 192.168.100.7"}, self.cfg)
|
||||
self.assertEqual(info.ip, "10.1.2.3")
|
||||
all_trusted = proxy.resolve("192.168.100.5", {"X-Forwarded-For": "127.0.0.1"}, self.cfg)
|
||||
self.assertEqual(all_trusted.ip, "127.0.0.1")
|
||||
self.assertTrue(all_trusted.via_proxy)
|
||||
|
||||
def test_falls_back_to_x_real_ip(self):
|
||||
"""没有 XFF 时用 X-Real-IP；两者都没有则退回 TCP 对端。"""
|
||||
self.cfg.update({"trust_proxy_headers": True, "trusted_proxies": ["127.0.0.1"]})
|
||||
info = proxy.resolve("127.0.0.1", {"X-Real-IP": "203.0.113.11"}, self.cfg)
|
||||
self.assertEqual(info.ip, "203.0.113.11")
|
||||
self.assertEqual(info.source, "x-real-ip")
|
||||
plain = proxy.resolve("127.0.0.1", {}, self.cfg)
|
||||
self.assertEqual(plain.ip, "127.0.0.1")
|
||||
self.assertTrue(plain.via_proxy) # 仍标记"经由可信代理"，只是没有头可用
|
||||
|
||||
def test_garbage_headers_ignored(self):
|
||||
"""垃圾/伪造头（unknown、带端口、非 IP、注入字符）不采信。"""
|
||||
self.cfg.update({"trust_proxy_headers": True, "trusted_proxies": ["127.0.0.1"]})
|
||||
info = proxy.resolve("127.0.0.1", {"X-Forwarded-For": "unknown", "X-Real-IP": "not-an-ip"}, self.cfg)
|
||||
self.assertEqual(info.ip, "127.0.0.1")
|
||||
self.assertEqual(proxy.norm_ip("203.0.113.9:8080"), "203.0.113.9")
|
||||
self.assertEqual(proxy.valid_host("evil/../x"), "")
|
||||
self.assertEqual(proxy.valid_host("ktv.example.com:8443"), "ktv.example.com:8443")
|
||||
self.assertEqual(proxy.valid_proto("javascript"), "")
|
||||
self.assertEqual(proxy.valid_proto("https"), "https")
|
||||
|
||||
def test_config_normalises_and_warns(self):
|
||||
"""配置层：字符串拆成列表、非法项丢弃并给警告、开关开着却没列表也要警告。"""
|
||||
warns = self.cfg.update({"trusted_proxies": "127.0.0.1, 192.168.100.0/24 , 不是IP"})
|
||||
self.assertEqual(self.cfg.get("trusted_proxies"), ["127.0.0.1", "192.168.100.0/24", "不是IP"])
|
||||
self.assertTrue(any("解析不了" in w for w in warns))
|
||||
warns2 = self.cfg.update({"trust_proxy_headers": True, "trusted_proxies": []})
|
||||
self.assertTrue(any("trusted_proxies 为空" in w for w in warns2))
|
||||
warns3 = self.cfg.update({"http_backlog": 1, "max_body_bytes": 10})
|
||||
self.assertEqual(self.cfg.get("http_backlog"), 5)
|
||||
self.assertEqual(self.cfg.get("max_body_bytes"), 4096)
|
||||
self.assertTrue(any("http_backlog" in w for w in warns3))
|
||||
|
||||
|
||||
# ══════════════════════ ② 推送限流按真实 IP ══════════════════════
|
||||
|
||||
class PushCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.cfg = Config(os.path.join(self.tmp, "c.json"))
|
||||
self.media = os.path.join(self.tmp, "media")
|
||||
os.makedirs(self.media, exist_ok=True)
|
||||
self.cfg.update({"media_dir": self.media, "data_dir": self.tmp, "push_token": "local",
|
||||
"push_rate_limit_per_min": 1, "trust_proxy_headers": True,
|
||||
"trusted_proxies": ["127.0.0.1"]})
|
||||
self.store = Store(os.path.join(self.tmp, "t.db"))
|
||||
self.router = service.build_router(self.cfg, self.store, None)
|
||||
|
||||
def tearDown(self):
|
||||
try:
|
||||
self.store.conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def push_from_proxy(self, tid, client_ip_in_xff):
|
||||
"""模拟 IIS 反代：TCP 对端 127.0.0.1（可信），XFF 带真实客户端 IP。"""
|
||||
info = proxy.resolve("127.0.0.1", {"X-Forwarded-For": client_ip_in_xff}, self.cfg)
|
||||
r = req("POST", "/push_download", body=form(tid=tid, url=f"http://cdn.example.com/480p/{tid}.ts",
|
||||
token="local"),
|
||||
client_ip=info.ip, peer_ip=info.peer_ip, via_proxy=info.via_proxy, xff=info.xff)
|
||||
return self.router.dispatch(r)
|
||||
|
||||
def test_rate_limit_separate_clients_behind_proxy(self):
|
||||
"""反代后两个设备各自限流，不再共用代理地址的一份额度。"""
|
||||
a = self.push_from_proxy(TID, "203.0.113.9")
|
||||
b = self.push_from_proxy(TID + 1, "203.0.113.10")
|
||||
self.assertEqual((a.status, b.status), (200, 200), (a.body, b.body))
|
||||
# 同一设备再来一条 → 触发限流（每分钟 1 条）
|
||||
again = self.push_from_proxy(TID + 2, "203.0.113.9")
|
||||
self.assertEqual(again.status, 429)
|
||||
self.assertIn("推送过于频繁", json.loads(again.body)["error"])
|
||||
self.assertIn("203.0.113.9", json.loads(again.body)["error"])
|
||||
|
||||
def test_rate_limit_logs_real_ip(self):
|
||||
"""来源 IP 写进事件日志：接受那条（即时）与限流汇总那条（分钟翻页后），都带真实 IP。"""
|
||||
self.push_from_proxy(TID, "203.0.113.9") # 接受 → 即时 INFO
|
||||
self.push_from_proxy(TID + 1, "203.0.113.9") # 第二条被限流 → 进"当前分钟桶"
|
||||
self.assertTrue(any("App 点歌推送：接受" in e["text"] and "203.0.113.9" in e["text"]
|
||||
for e in self.store.events(20)),
|
||||
[e["text"] for e in self.store.events(20)])
|
||||
st = json.loads(self.router.dispatch(req("GET", "/push_download/status")).body)
|
||||
self.assertEqual(st["rate_limited"], 1)
|
||||
self.assertIn("203.0.113.9", st["pending_log"].get("sample", ""))
|
||||
# 分钟翻页（把时间往前拨 2 分钟）→ 汇总成一条 WARN 事件，同样带来源 IP
|
||||
real_time = time.time
|
||||
with mock.patch("ktv.pushdl.time.time", side_effect=lambda: real_time() + 120):
|
||||
self.router.dispatch(req("GET", "/push_download/status"))
|
||||
self.assertTrue(any("触发限流" in e["text"] and "203.0.113.9" in e["text"]
|
||||
for e in self.store.events(30)),
|
||||
[e["text"] for e in self.store.events(30)])
|
||||
|
||||
def test_untrusted_peer_still_rate_limits_by_tcp_peer(self):
|
||||
"""对端不可信时：XFF 完全不同也只按 TCP 对端限流（防伪造，宁粗不松）。"""
|
||||
self.cfg.update({"trusted_proxies": ["192.168.100.10"]}) # 127.0.0.1 不再可信
|
||||
first = req("POST", "/push_download", body=form(tid=TID, url=f"http://cdn.example.com/{TID}.ts",
|
||||
token="local"),
|
||||
headers={"X-Forwarded-For": "203.0.113.1"}, client_ip="127.0.0.1")
|
||||
second = req("POST", "/push_download", body=form(tid=TID + 1, url=f"http://cdn.example.com/{TID + 1}.ts",
|
||||
token="local"),
|
||||
headers={"X-Forwarded-For": "203.0.113.2"}, client_ip="127.0.0.1")
|
||||
self.assertEqual(self.router.dispatch(first).status, 200)
|
||||
self.assertEqual(self.router.dispatch(second).status, 429)
|
||||
|
||||
def test_client_ip_debug_endpoint(self):
|
||||
"""`GET /api/client_ip`：返回认定结果与"为什么没用代理头"。"""
|
||||
info = proxy.resolve("127.0.0.1", {"X-Forwarded-For": "203.0.113.9",
|
||||
"X-Forwarded-Proto": "https",
|
||||
"X-Forwarded-Host": "ktv.example.com"}, self.cfg)
|
||||
r = req("GET", "/api/client_ip", client_ip=info.ip, peer_ip=info.peer_ip,
|
||||
via_proxy=info.via_proxy, proto=info.proto, host=info.host, xff=info.xff)
|
||||
d = json.loads(self.router.dispatch(r).body)
|
||||
self.assertEqual(d["client_ip"], "203.0.113.9")
|
||||
self.assertEqual(d["peer_ip"], "127.0.0.1")
|
||||
self.assertTrue(d["via_proxy"])
|
||||
self.assertEqual(d["source"], "x-forwarded-for")
|
||||
self.assertEqual(d["x_forwarded_proto"], "https")
|
||||
self.assertEqual(d["ignored_reason"], "")
|
||||
# 不可信时给出明确原因
|
||||
self.cfg.update({"trusted_proxies": []})
|
||||
d2 = json.loads(self.router.dispatch(req("GET", "/api/client_ip", client_ip="10.0.0.5")).body)
|
||||
self.assertFalse(d2["via_proxy"])
|
||||
self.assertIn("trusted_proxies 为空", d2["ignored_reason"])
|
||||
|
||||
|
||||
# ══════════════════════ ③ 协议/Host 感知的媒体地址 ══════════════════════
|
||||
|
||||
class TestForwardedSchemeHost(PushCase):
|
||||
def _ready(self):
|
||||
self.store.import_library([(TID, "测试歌", "国语", "", "歌手", f"{TID}.ts", 10, "")], "v")
|
||||
self.store.enqueue([TID])
|
||||
self.store.claim_one(TID)
|
||||
self.store.finish(TID, True, bytes_=7 << 20, path=os.path.join(self.media, f"{TID}.ts"))
|
||||
|
||||
def _do_php(self, **kw):
|
||||
r = req("GET", "/music/do.php", {"musicno": str(TID)}, **kw)
|
||||
return json.loads(self.router.dispatch(r).body)
|
||||
|
||||
def test_https_forwarded_when_trusted(self):
|
||||
self._ready()
|
||||
d = self._do_php(client_ip="203.0.113.9", peer_ip="127.0.0.1", via_proxy=True,
|
||||
proto="https", host="ktv.example.com")
|
||||
self.assertEqual(d["code"], 200)
|
||||
self.assertTrue(d["data"].startswith("https://ktv.example.com/media/"), d["data"])
|
||||
self.assertTrue(d["data"].endswith(f"/{TID}.ts"))
|
||||
|
||||
def test_forwarded_ignored_when_untrusted(self):
|
||||
"""不可信（没经代理）→ 用配置里的 public_media_base，绝不被头带偏。"""
|
||||
self._ready()
|
||||
self.cfg.update({"public_media_base": "http://ktv.example.com:6677/media"})
|
||||
d = self._do_php(client_ip="203.0.113.9", peer_ip="203.0.113.9", via_proxy=False,
|
||||
proto="https", host="evil.example.com")
|
||||
self.assertEqual(d["data"], f"http://ktv.example.com:6677/media/{TID}.ts")
|
||||
|
||||
def test_forwarded_ignored_when_switch_off(self):
|
||||
self._ready()
|
||||
self.cfg.update({"public_media_base": "http://ktv.example.com:6677/media",
|
||||
"trust_proxy_headers": False})
|
||||
# 走真实认定链路：开关关着 → via_proxy=False，头一个都不看
|
||||
info = proxy.resolve("127.0.0.1", {"X-Forwarded-For": "203.0.113.9",
|
||||
"X-Forwarded-Proto": "https",
|
||||
"X-Forwarded-Host": "ktv.example.com"}, self.cfg)
|
||||
self.assertFalse(info.via_proxy)
|
||||
d = self._do_php(client_ip=info.ip, peer_ip=info.peer_ip, via_proxy=info.via_proxy,
|
||||
proto=info.proto, host=info.host)
|
||||
self.assertTrue(d["data"].startswith("http://ktv.example.com:6677/"), d["data"])
|
||||
|
||||
def test_public_base_helper(self):
|
||||
self.cfg.update({"public_media_base": "http://ktv.example.com:6677/media"})
|
||||
r = req("GET", "/x", via_proxy=True, proto="https", host="ktv.example.com")
|
||||
self.assertEqual(proxy.public_base(self.cfg, r), "https://ktv.example.com/media")
|
||||
self.assertEqual(proxy.public_media_url(self.cfg, r, 42), "https://ktv.example.com/media/42.ts")
|
||||
self.assertEqual(proxy.absolute_url(self.cfg, r, "/healthz"), "https://ktv.example.com/healthz")
|
||||
# 只给 proto（没给 Host）→ 保留配置里的主机
|
||||
r2 = req("GET", "/x", via_proxy=True, proto="https")
|
||||
self.assertEqual(proxy.public_base(self.cfg, r2), "https://ktv.example.com:6677/media")
|
||||
# 不可信 → 原样
|
||||
self.assertEqual(proxy.public_base(self.cfg, req("GET", "/x")),
|
||||
"http://ktv.example.com:6677/media")
|
||||
|
||||
|
||||
# ══════════════════════ ④ /healthz ══════════════════════
|
||||
|
||||
class TestHealthz(PushCase):
|
||||
def test_healthz_is_fast_and_does_not_touch_db_or_media(self):
|
||||
"""探活必须微秒级返回：不查库、不列媒体目录（打桩后一旦被调用就失败）。"""
|
||||
with mock.patch.object(self.store, "stats", side_effect=AssertionError("healthz 不该查库")), \
|
||||
mock.patch.object(self.store, "keep_stats", side_effect=AssertionError("healthz 不该查库")), \
|
||||
mock.patch("os.listdir", side_effect=AssertionError("healthz 不该碰媒体目录")), \
|
||||
mock.patch("os.stat", side_effect=AssertionError("healthz 不该碰媒体目录")), \
|
||||
mock.patch("shutil.disk_usage", side_effect=AssertionError("healthz 不该碰磁盘")):
|
||||
t0 = time.perf_counter()
|
||||
resp = self.router.dispatch(req("GET", "/healthz"))
|
||||
elapsed = time.perf_counter() - t0
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertLess(elapsed, 0.05, f"healthz 太慢：{elapsed * 1000:.1f}ms")
|
||||
d = json.loads(resp.body)
|
||||
for key in ("ok", "version", "uptime_sec", "library", "keeps", "downloader"):
|
||||
self.assertIn(key, d)
|
||||
self.assertTrue(d["ok"])
|
||||
self.assertIn("snapshot_age_sec", d)
|
||||
|
||||
def test_healthz_snapshot_refresh_fills_numbers(self):
|
||||
"""后台刷新（refresh()）才会查库；刷完后探活就能看到数字。"""
|
||||
self.store.import_library([(1, "A", "国语", "", "s", "1.ts", 1, "")], "v")
|
||||
snap = health_mod.HealthSnapshot(self.cfg, self.store, None)
|
||||
self.assertEqual(snap.snapshot()["library"], 0) # 还没刷新 → 占位值
|
||||
snap.refresh()
|
||||
router = service.build_router(self.cfg, self.store, None, health=snap)
|
||||
d = json.loads(router.dispatch(req("GET", "/healthz")).body)
|
||||
self.assertEqual(d["library"], 1)
|
||||
self.assertIn("downloader", d)
|
||||
self.assertGreaterEqual(d["snapshot_age_sec"], 0)
|
||||
|
||||
def test_healthz_refresh_failure_keeps_serving(self):
|
||||
"""刷新失败（库暂时读不到）只记日志 + 沿用上次值，探活照样 200。"""
|
||||
snap = health_mod.HealthSnapshot(self.cfg, self.store, None)
|
||||
with mock.patch.object(self.store, "stats", side_effect=RuntimeError("db busy")):
|
||||
d = snap.refresh()
|
||||
self.assertEqual(d["library"], 0)
|
||||
self.assertTrue(snap.snapshot()["ok"])
|
||||
self.assertTrue(any("探活快照刷新失败" in e["text"] for e in self.store.events(10)))
|
||||
|
||||
|
||||
# ══════════════════════ ⑥ HTTP 层（真起本地端口） ══════════════════════
|
||||
|
||||
class TestHttpLayer(unittest.TestCase):
|
||||
"""socket 级验证：Content-Length / HEAD / Range / keep-alive / 413（IIS 关心的正是这些）。"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.cfg = Config(os.path.join(self.tmp, "c.json"))
|
||||
self.media = os.path.join(self.tmp, "media")
|
||||
self.logs = [] # 收集访问日志（校验里面用的是真实客户端 IP）
|
||||
os.makedirs(self.media, exist_ok=True)
|
||||
self.cfg.update({"media_dir": self.media, "data_dir": self.tmp, "push_token": "local",
|
||||
"trust_proxy_headers": True, "trusted_proxies": ["127.0.0.1"],
|
||||
"max_body_bytes": 4096}) # 配置里的最小值会被钳制到 4096
|
||||
self.store = Store(os.path.join(self.tmp, "t.db"))
|
||||
self.store.import_library([(TID, "测试歌", "国语", "", "歌手", f"{TID}.ts", 1, "")], "v")
|
||||
# 媒体文件：长度 300 字节（Range 用例断言精确 Content-Length）
|
||||
with open(os.path.join(self.media, f"{TID}.ts"), "wb") as fh:
|
||||
fh.write(b"\x47" * 300)
|
||||
# 服务端口路由 + 管理端口路由（`/api/keeps/import` 的 413 要在**真实路由**上验证）
|
||||
self.router = web.Router()
|
||||
self.router.routes.extend(service.build_router(self.cfg, self.store, None).routes)
|
||||
self.router.routes.extend(admin.build_router(self.cfg, self.store, None).routes)
|
||||
self.srv = web.start_server("127.0.0.1", 0, self.router, cfg=self.cfg, backlog=32,
|
||||
log_fn=self.logs.append)
|
||||
self.port = self.srv.server_address[1]
|
||||
|
||||
def tearDown(self):
|
||||
try:
|
||||
self.srv.shutdown()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
self.srv.server_close() # 关监听套接字，避免 ResourceWarning 噪音
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
self.store.conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
shutil.rmtree(self.tmp, ignore_errors=True)
|
||||
|
||||
def conn(self):
|
||||
return http.client.HTTPConnection("127.0.0.1", self.port, timeout=5)
|
||||
|
||||
def test_head_and_range_with_content_length(self):
|
||||
# 全部请求都带 XFF（= 经可信代理到达），验证代理场景下 HEAD/Range 与直连一致
|
||||
xff = {"X-Forwarded-For": "203.0.113.9, 127.0.0.1"}
|
||||
c = self.conn()
|
||||
try:
|
||||
c.request("HEAD", f"/media/{TID}.ts", headers=dict(xff))
|
||||
r = c.getresponse()
|
||||
self.assertEqual(r.status, 200)
|
||||
self.assertEqual(r.getheader("Content-Length"), "300")
|
||||
self.assertEqual(r.getheader("Accept-Ranges"), "bytes")
|
||||
self.assertEqual(r.read(), b"")
|
||||
# HEAD + Range 也要是 206 + 精确长度（App 的探测就是 HEAD）
|
||||
c.request("HEAD", f"/media/{TID}.ts", headers={**xff, "Range": "bytes=100-199"})
|
||||
rh = c.getresponse()
|
||||
self.assertEqual(rh.status, 206)
|
||||
self.assertEqual(rh.getheader("Content-Length"), "100")
|
||||
self.assertEqual(rh.getheader("Content-Range"), "bytes 100-199/300")
|
||||
self.assertEqual(rh.getheader("Accept-Ranges"), "bytes")
|
||||
self.assertEqual(rh.read(), b"")
|
||||
# 同一条连接继续发 Range（keep-alive 复用；若上一个响应长度不对这里会错乱/卡住）
|
||||
c.request("GET", f"/media/{TID}.ts", headers={**xff, "Range": "bytes=100-199"})
|
||||
r2 = c.getresponse()
|
||||
self.assertEqual(r2.status, 206)
|
||||
self.assertEqual(r2.getheader("Content-Length"), "100")
|
||||
self.assertEqual(r2.getheader("Accept-Ranges"), "bytes")
|
||||
self.assertEqual(r2.getheader("Content-Range"), "bytes 100-199/300")
|
||||
self.assertEqual(len(r2.read()), 100)
|
||||
# 再来一次，确认连接还能用
|
||||
c.request("GET", "/healthz", headers=dict(xff))
|
||||
r3 = c.getresponse()
|
||||
self.assertEqual(r3.status, 200)
|
||||
self.assertGreater(int(r3.getheader("Content-Length")), 0)
|
||||
self.assertTrue(json.loads(r3.read())["ok"])
|
||||
finally:
|
||||
c.close()
|
||||
|
||||
def test_keepalive_requires_exact_length_every_response(self):
|
||||
"""每个响应都必须有精确 Content-Length（缺了代理会挂住）。"""
|
||||
c = self.conn()
|
||||
try:
|
||||
for path, expect in ((f"/media/{TID}.ts", 300), ("/healthz", None),
|
||||
(f"/media/{TID}.ts", 300), ("/nope", None)):
|
||||
c.request("GET", path, headers={"X-Forwarded-For": "203.0.113.9"})
|
||||
r = c.getresponse()
|
||||
body = r.read()
|
||||
cl = r.getheader("Content-Length")
|
||||
self.assertIsNotNone(cl, f"{path} 没有 Content-Length")
|
||||
self.assertEqual(int(cl), len(body), f"{path} 的 Content-Length 与实际不符")
|
||||
if expect is not None:
|
||||
self.assertEqual(len(body), expect)
|
||||
finally:
|
||||
c.close()
|
||||
|
||||
def test_proxy_headers_resolve_client_ip_over_the_wire(self):
|
||||
"""经 socket 发 XFF（对端 127.0.0.1 可信）→ healthz 之外用 /api/client_ip 核对。"""
|
||||
c = self.conn()
|
||||
try:
|
||||
c.request("GET", "/api/client_ip",
|
||||
headers={"X-Forwarded-For": "203.0.113.9, 127.0.0.1",
|
||||
"X-Forwarded-Proto": "https",
|
||||
"X-Forwarded-Host": "ktv.example.com"})
|
||||
r = c.getresponse()
|
||||
d = json.loads(r.read())
|
||||
self.assertEqual(d["client_ip"], "203.0.113.9")
|
||||
self.assertEqual(d["peer_ip"], "127.0.0.1")
|
||||
self.assertTrue(d["via_proxy"])
|
||||
finally:
|
||||
c.close()
|
||||
|
||||
def test_access_log_uses_real_client_ip(self):
|
||||
"""访问日志里的客户端 IP 也必须是**真实客户端 IP**（反代后不能全是代理地址）。"""
|
||||
c = self.conn()
|
||||
try:
|
||||
c.request("GET", "/healthz", headers={"X-Forwarded-For": "203.0.113.77, 127.0.0.1"})
|
||||
r = c.getresponse()
|
||||
r.read()
|
||||
finally:
|
||||
c.close()
|
||||
self.assertTrue(any("203.0.113.77" in m for m in self.logs), self.logs[-3:])
|
||||
self.assertFalse(any('"GET /healthz' in m and m.startswith("127.0.0.1") for m in self.logs),
|
||||
self.logs[-3:])
|
||||
|
||||
def test_oversize_body_returns_413_chinese(self):
|
||||
c = self.conn()
|
||||
try:
|
||||
c.request("POST", "/api/keeps/import", body=b"x" * 6000,
|
||||
headers={"Content-Length": "6000"})
|
||||
r = c.getresponse()
|
||||
self.assertEqual(r.status, 413)
|
||||
d = json.loads(r.read())
|
||||
self.assertFalse(d["ok"])
|
||||
self.assertIn("请求体过大", d["error"])
|
||||
self.assertIn("max_body_bytes", d["error"])
|
||||
self.assertEqual(r.getheader("Connection"), "close")
|
||||
finally:
|
||||
c.close()
|
||||
|
||||
def test_body_within_limit_accepted(self):
|
||||
c = self.conn()
|
||||
try:
|
||||
csv = f"编号,歌名\n{TID},测试歌\n".encode("utf-8-sig")
|
||||
c.request("POST", "/api/keeps/import", body=csv,
|
||||
headers={"Content-Length": str(len(csv))})
|
||||
r = c.getresponse()
|
||||
self.assertEqual(r.status, 200, r.read())
|
||||
finally:
|
||||
c.close()
|
||||
|
||||
def test_chunked_request_body_rejected_with_close(self):
|
||||
"""分块请求体不支持：必须 411 + 关连接（否则 chunk 会被当成下一个请求，经代理时乱响应）。"""
|
||||
import socket
|
||||
s = socket.create_connection(("127.0.0.1", self.port), timeout=5)
|
||||
try:
|
||||
s.sendall(b"POST /api.php HTTP/1.1\r\nHost: 127.0.0.1\r\n"
|
||||
b"Transfer-Encoding: chunked\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n"
|
||||
b"5\r\nhello\r\n0\r\n\r\n")
|
||||
data = s.recv(65536)
|
||||
finally:
|
||||
s.close()
|
||||
head = data.split(b"\r\n", 1)[0]
|
||||
self.assertIn(b"411", head)
|
||||
self.assertIn(b"Connection: close", data)
|
||||
|
||||
def test_backlog_configurable(self):
|
||||
"""监听队列长度按 `http_backlog` 生效（配置写入 socket 的 listen backlog）。"""
|
||||
self.assertEqual(self.cfg.get("http_backlog"), 128)
|
||||
srv = web.start_server("127.0.0.1", 0, self.router, cfg=self.cfg, backlog=32)
|
||||
try:
|
||||
self.assertEqual(srv.request_queue_size, 32)
|
||||
finally:
|
||||
srv.shutdown()
|
||||
srv.server_close()
|
||||
|
||||
|
||||
# ══════════════════════ ⑦ IIS 交付物静态校验 ══════════════════════
|
||||
|
||||
class TestIisDeployFiles(unittest.TestCase):
|
||||
def test_web_config_is_index_pointing_to_two_sites(self):
|
||||
"""原「三合一」web.config 已拆成两个站点，本文件只作**索引**（不再含任何规则）。
|
||||
|
||||
两份站点配置的静态校验（可解析、各管各的、不含管理端口）见 `tests/test_app_token.py`
|
||||
的 `TestIisSplitConfigs`。
|
||||
"""
|
||||
path = os.path.join(IIS_DIR, "web.config")
|
||||
self.assertTrue(os.path.isfile(path), f"缺少 {path}")
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
root = ET.fromstring(text) # 必须能被 ElementTree 解析
|
||||
self.assertTrue(root.tag.endswith("configuration"))
|
||||
self.assertNotIn("6678", text) # 管理端口绝不出现在对外配置里
|
||||
self.assertNotIn("192.168.100.232", text) # 索引文件里不留后端地址（避免三份配置互相矛盾）
|
||||
self.assertEqual(root.findall(".//{*}rule"), [], "索引文件里不该再有反代规则")
|
||||
for needle in ("api-site/web.config", "media-site/web.config"):
|
||||
self.assertIn(needle, text, f"索引里应指向 {needle}")
|
||||
|
||||
def test_application_host_snippet_parses(self):
|
||||
path = os.path.join(IIS_DIR, "applicationHost.config.snippet.xml")
|
||||
self.assertTrue(os.path.isfile(path), f"缺少 {path}")
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
root = ET.fromstring(fh.read())
|
||||
proxy_el = root.find(".//{*}proxy")
|
||||
self.assertIsNotNone(proxy_el, "片段里必须有 ARR 的 <proxy> 配置")
|
||||
self.assertEqual(str(proxy_el.get("enabled")).lower(), "true")
|
||||
names = {e.get("name") for e in root.findall(".//{*}allowedServerVariables/{*}add")}
|
||||
for n in ("HTTP_X_FORWARDED_FOR", "HTTP_X_FORWARDED_PROTO", "HTTP_X_REAL_IP"):
|
||||
self.assertIn(n, names, f"未放行服务器变量 {n}")
|
||||
|
||||
def test_iis_readme_documents_steps_and_faq(self):
|
||||
path = os.path.join(IIS_DIR, "README.md")
|
||||
self.assertTrue(os.path.isfile(path), f"缺少 {path}")
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
for needle in ("URL Rewrite", "ARR", "Enable proxy", "healthz", "Range", "X-Forwarded-For",
|
||||
"HTTPS", "6678", "maxAllowedContentLength", "trust_proxy_headers"):
|
||||
self.assertIn(needle, text, f"IIS README 缺少 {needle}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Reference in New Issue
Block a user