优化小红书App短链与LivePhoto解析,与UI体验优化

This commit is contained in:
ucmao
2026-09-08 16:34:52 +08:00
parent aa2c615d7a
commit fd3ac0cd4c
15 changed files with 383 additions and 121 deletions
+2
View File
@@ -1,6 +1,7 @@
import os
import secrets
import fcntl
from datetime import timedelta
from flask import Flask
from src.api.parse import bp as api_bp
from src.web.views import bp as web_bp
@@ -43,6 +44,7 @@ def create_app(config=None):
'DATABASE_PATH', os.path.join(app.instance_path, 'media_parser.db')
)
app.config['MAX_CONTENT_LENGTH'] = 32 * 1024
app.config['PERMANENT_SESSION_LIFETIME'] = timedelta(days=30)
app.config['JSON_SORT_KEYS'] = False
app.config['TRUST_PROXY_HEADERS'] = os.getenv(
'TRUST_PROXY_HEADERS', ''
+1 -1
View File
@@ -30,7 +30,7 @@
| 序号 | 平台名称 | 支持媒体类型 | 无水印直链 | 配置门槛 (Cookie 依赖) | 逆向提取模式 | 对应指南 |
| :--- | :--- | :--- | :---: | :---: | :--- | :--- |
| 1 | **抖音** | 视频 / 图文 / LivePhoto / 音频 / 字幕 / 合集 | ✅ 支持 | 🟢 免配置 | a_bogus 签名 + 动态 ttwid + SSR 容灾兜底 | [查看指南](parsers/douyin.md) |
| 2 | **小红书** | 图文 / LivePhoto / 视频 | ✅ 支持 | ⚠️ 部分需 Cookie | SSR HTML 注入数据提取 | [查看指南](parsers/xiaohongshu.md) |
| 2 | **小红书** | 图文 / LivePhoto / 视频 | ✅ 支持 | 🟢 免配置 | SSR HTML 状态注入 + 双 UA 自适应回退 | [查看指南](parsers/xiaohongshu.md) |
| 3 | **快手** | 视频 / 图文 / 音频 | ✅ 支持 | 🟢 免配置 (内置游客凭证) | GraphQL / H5 双端多路由 Fallback | [查看指南](parsers/kuaishou.md) |
| 4 | **哔哩哔哩** | 视频 (MP4) / 音频分流 | ✅ 支持 | 🟢 免配置 | 官方 View + PlayURL API | [查看指南](parsers/bilibili.md) |
| 5 | **豆包 AI** | AI 视频生成直链 | ✅ 支持 | 🔐 需 `DOUBAO_COOKIE` | Web Session 维持 + 任务轮询 | [查看指南](parsers/doubao.md) |
+45 -22
View File
@@ -12,35 +12,47 @@
* 实况照片 (LivePhoto 动态视频流)
* 视频笔记 (H264 Master MP4 流)
* 笔记正文与作者信息
* 笔记封面 (自动从 cover 或首图提取)
* **常见链接形态**:
* 短链接:`http://xhslink.com/a/xxxx`
* 短链接:`http://xhslink.com/o/xxxx` 或 `http://xhslink.com/a/xxxx`
* 网页发现长链:`https://www.xiaohongshu.com/discovery/item/6a8fbccc000000002a03825a`
* 网页探索长链:`https://www.xiaohongshu.com/explore/6a8fbccc000000002a03825a`
* **Cookie 依赖**:公开普通笔记无需 Cookie;部分强受限笔记或高频调用建议在 `.env` 中提供 Cookie。
* **Cookie 依赖**:🟢 公开普通笔记及移动分享均免配置 Cookie。
---
## 2. 核心逆向方案 (SSR HTML 状态注入)
## 2. 核心逆向方案 (SSR HTML 状态注入 + 双 UA 自适应回退)
小红书 Web 页面采用 Nuxt / Vue SSR 服务端渲染。页面加载时,首屏所有笔记详情、高清媒体流及作者信息均已经内嵌在 HTML 的 `window.__INITIAL_STATE__` 变量中。
小红书 Web/H5 页面采用 Nuxt / Vue SSR 服务端渲染。页面加载时,首屏所有笔记详情、高清媒体流及作者信息均已经内嵌在 HTML 的 `window.__INITIAL_STATE__` 变量中。
### 2.1 请求配置
* **请求方式**:`GET` 目标长链接
* **必备请求头**:
* `User-Agent`: 现代 PC 浏览器 UA
* `Referer`: `https://www.xiaohongshu.com/`
### 2.1 请求配置与 URL 参数保留
* **请求方式**:`GET` 目标解析长链接
* **关键查询参数保留**:短链 302 重定向后必须保留 `xsec_token`、`xsec_source`、`source`、`xhsshare`、`app_platform` 等核心参数(在 [utils/web_fetcher.py](file:///Users/leo/Projects/media-parser/utils/web_fetcher.py) 的 `UrlParser.extract_video_address` 中维护),缺少来源签名参数会导致服务端返回 404 或拦截。
* **双 UA 自适应机制**:
* **App 端分享短链**(含 `app_platform=android` 等):优先使用 **Mobile User-Agent** 访问以获取移动端 H5 页面。
* **PC 桌面端长链**(含 `source=webshare` 等):优先使用 **PC User-Agent** 访问。
* **自动回退**:若首选 UA 遭遇 302/404 或拦截,解析器自动切换为备用 UA 再次发起请求。
### 2.2 正则状态提取
### 2.2 正则状态提取与结构兼容
```python
import re, json
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
json_str = BaseParser.parse_html_data(html_content, pattern)
# 1. 正则提取并清洗 :undefined 为 :null
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*?\})</script>', re.DOTALL)
match = pattern.search(html_content) or re.search(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', html_content, re.DOTALL)
if json_str:
if match:
json_str = re.sub(r':\s*undefined\b', ':null', match.group(1))
full_data = json.loads(json_str)
# 2. 兼容 PC 桌面端数据结构
first_note_id = full_data.get('note', {}).get('firstNoteId')
note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
if first_note_id:
note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
# 3. 兼容移动端 H5 数据结构
if not note_data:
note_data = full_data.get('noteData', {}).get('data', {}).get('noteData', {})
```
---
@@ -48,23 +60,30 @@ if json_str:
## 3. 字段提取与 LivePhoto 解析
### 3.1 视频笔记提取
* 视频直链位于:`note_data.video.media.stream.h264[0].masterUrl`
* 视频直链位于:`note_data.video.media.stream.h264[0].masterUrl` 或 `note_data.video.stream.h264[0].masterUrl`。
* 字符转义处理:将 Unicode 斜杠 `\\u002F` 替换为标准 `/`。
### 3.2 图文与 LivePhoto 提取
* 遍历 `note_data.imageList`:
* **原图直链**:提取 `urlDefault` 字段。
* **实况动图 (LivePhoto)**:检查 `image.get('livePhoto') == True`,若为真,则提取 `image.stream.h264[0].masterUrl` 作为实况动图关联的视频流。
* **原图直链**:按优先级提取 `image.urlDefault` -> `image.url` -> `image.infoList[-1].url`。
* **实况动图 (LivePhoto)**:检查 `image.get('livePhoto') == True`,若为真,则提取 `image.stream.h264[0].masterUrl` 作为实况动图关联的动态视频流。
### 3.3 封面与作者提取
* **封面**:优先读取 `note_data.cover.urlDefault` / `url`,若为空则以 `imageList[0]` 兜底。
* **作者**:兼容读取 `user.nickname` / `user.nickName` 及 `user.userId`。
---
## 4. 常见踩坑与反爬对抗 (Gotchas)
1. **重定向至登录页 (`/login`) 或 404**:
* *原因*:小红书对单一 IP 频次较高的请求,或者未带参数的特定私密笔记会强制要求登录。
* *解法*:在 [src/api/parse.py](file:///Users/leo/Projects/media-parser/src/api/parse.py) 中针对小红书内置了 3 次自适应重试机制(`_fetch_with_retry`);若持续失败,则友好提示用户配置 Cookie。
2. **字符转义问题**:
* JSON 字符串中经常包含 JSON 编码的斜杠 `\u002F`,必须统一格式化清洗。
1. **App 分享短链在 PC UA 下重定向到 `/explore?undertake_note_error=该内容暂时无法查看` (错误码 -510001)**:
* *原因*:小红书服务端对 App 端生成的分享链接(带有 `app_platform=android&xsec_source=app_share` 等参数)进行设备特征校验,PC UA 访问会被判定为非移动环境并重定向到探索页。
* *解法*:在 [src/parsers/xiaohongshu_parser.py](file:///Users/leo/Projects/media-parser/src/parsers/xiaohongshu_parser.py) 中启用双 UA 自动降级重试机制,对 App 分享链接自动采用移动端 User-Agent 请求,无需 Cookie 即可获取完整 H5 状态树。
2. **缺少 `xsec_source` 导致 H5 页面返回 404/安全校验**:
* *原因*:在 URL 清洗阶段若仅保留 `xsec_token` 而丢弃了 `xsec_source`,移动端 H5 接口会因缺少渠道来源标识而拒绝渲染笔记数据。
* *解法*:在 [utils/web_fetcher.py](file:///Users/leo/Projects/media-parser/utils/web_fetcher.py) 的 `UrlParser.extract_video_address` 中完整保留 `xsec_token`, `xsec_source`, `source`, `xhsshare`, `app_platform` 等核心参数。
3. **字符转义与 `undefined` 序列化**:
* HTML 内嵌的 JS 对象常包含 `:undefined`,直接 `json.loads` 会抛出异常,需统一正则替换为 `:null`。
---
@@ -74,3 +93,7 @@ if json_str:
```bash
python tests/manual_verify_parsers.py --platform 小红书
```
* **单元测试**:
```bash
python -m unittest tests/test_xiaohongshu_parser.py
```
+1
View File
@@ -202,6 +202,7 @@ def login():
else:
session.clear()
session["user_id"] = user["id"]
session.permanent = request.form.get("remember_me") in {"1", "on", "true"}
target = request.args.get("next", "")
if not target.startswith("/") or target.startswith("//"):
target = url_for("admin.dashboard") if user["role"] == "admin" else url_for("portal.dashboard")
+127 -42
View File
@@ -3,7 +3,9 @@ import re
import json
from src.parsers.base_parser import BaseParser
from configs.logging_config import get_logger
from configs.general_constants import USER_AGENT_PC, USER_AGENT_M
import requests
logger = get_logger(__name__)
@@ -11,45 +13,80 @@ logger = get_logger(__name__)
class XiaohongshuParser(BaseParser):
def __init__(self, real_url):
super().__init__(real_url)
self.headers = {
"content-type": "application/json; charset=UTF-8",
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36",
'referer': 'https://www.xiaohongshu.com/'
}
# 获取 HTML 并解析 JSON 状态
html_content = self.fetch_html_content()
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
json_str = BaseParser.parse_html_data(html_content, pattern)
# 初始化数据容器
self.note_data = {}
try:
if json_str:
full_data = json.loads(json_str)
first_note_id = full_data.get('note', {}).get('firstNoteId')
if first_note_id:
self.note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
except (json.JSONDecodeError, KeyError) as e:
logger.error(f"初始化解析数据失败: {e}")
self.fetch_and_parse()
def fetch_html_content(self):
try:
resp = self.session.get(self.real_url, headers=self.headers, timeout=5)
resp.raise_for_status()
if "xiaohongshu.com/login" in resp.url:
logger.error("小红书解析提示: 请求被重定向到了登录页面。")
elif "xiaohongshu.com/404" in resp.url:
logger.error("小红书解析提示: 遭遇安全拦截或页面未找到(404)。")
self.html_content = resp.text
return self.html_content
except requests.RequestException as e:
logger.error(f"Failed to get the page: {self.real_url}, Error: {e}")
def fetch_and_parse(self):
pc_ua = USER_AGENT_PC[0] if USER_AGENT_PC else "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
mobile_ua = USER_AGENT_M[0] if USER_AGENT_M else "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1"
# 根据链接特征选择优先尝试的 UA
if "app_platform" in self.real_url or "app_share" in self.real_url or "xhslink" in self.real_url:
candidate_uas = [mobile_ua, pc_ua]
else:
candidate_uas = [pc_ua, mobile_ua]
for ua in candidate_uas:
headers = {
"User-Agent": ua,
"Referer": "https://www.xiaohongshu.com/",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
}
try:
resp = self.session.get(self.real_url, headers=headers, timeout=6)
if resp.status_code != 200:
continue
if "xiaohongshu.com/login" in resp.url or "xiaohongshu.com/404" in resp.url or "undertake_note_error" in resp.url:
continue
self.html_content = resp.text
note = self._extract_note_from_html(self.html_content)
if note:
self.note_data = note
return
except requests.RequestException as e:
logger.warning(f"小红书请求失败 (UA: {ua[:30]}...): {e}")
continue
except Exception as e:
logger.error(f"解析小红书页面异常: {e}")
continue
if not self.note_data:
logger.error(f"未能解析出小红书笔记数据: {self.real_url}")
def _extract_note_from_html(self, html_content):
if not html_content:
return None
except Exception as e:
logger.error(f"An unexpected error occurred while fetching {self.real_url}: {e}")
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*?\})</script>', re.DOTALL)
match = pattern.search(html_content)
if not match:
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
match = pattern.search(html_content)
if not match:
return None
json_str = match.group(1)
json_str = re.sub(r':\s*undefined\b', ':null', json_str)
try:
full_data = json.loads(json_str)
except json.JSONDecodeError as e:
logger.error(f"小红书 JSON 解析失败: {e}")
return None
# 1. PC 端结构: full_data['note']['noteDetailMap'][first_note_id]['note']
first_note_id = full_data.get('note', {}).get('firstNoteId')
if first_note_id:
note = full_data.get('note', {}).get('noteDetailMap', {}).get(first_note_id, {}).get('note', {})
if note:
return note
# 2. 移动端 H5 结构: full_data['noteData']['data']['noteData']
note = full_data.get('noteData', {}).get('data', {}).get('noteData', {})
if note:
return note
# 3. 兜底回退结构
return full_data.get('noteData', {}).get('note', {}) or full_data.get('note', {}).get('note', {}) or None
def get_author_info(self):
"""
@@ -57,17 +94,30 @@ class XiaohongshuParser(BaseParser):
"""
user = self.note_data.get('user', {})
return {
'nickname': user.get('nickname', ''),
'author_id': user.get('userId', ''),
'nickname': user.get('nickname') or user.get('nickName', ''),
'author_id': user.get('userId') or user.get('author_id') or user.get('id', ''),
'avatar': user.get('avatar', '')
}
def get_real_video_url(self):
try:
video_info = self.note_data.get('video', {})
master_url = video_info['media']['stream']['h264'][0].get('masterUrl', '')
return master_url.replace("\\u002F", "/") if master_url else None
except (KeyError, IndexError):
if not isinstance(video_info, dict):
return None
stream = video_info.get('media', {}).get('stream', {}) or video_info.get('stream', {})
h264_data = stream.get('h264', [])
if h264_data and isinstance(h264_data, list):
master_url = h264_data[0].get('masterUrl', '')
if master_url:
return master_url.replace("\\u002F", "/")
for codec in ('h265', 'av1'):
codec_data = stream.get(codec, [])
if codec_data and isinstance(codec_data, list):
master_url = codec_data[0].get('masterUrl', '')
if master_url:
return master_url.replace("\\u002F", "/")
return None
except (KeyError, IndexError, AttributeError):
return None
def get_title_content(self):
@@ -77,20 +127,55 @@ class XiaohongshuParser(BaseParser):
return self.note_data.get('desc', '') or None
def get_cover_photo_url(self):
cover = self.note_data.get('cover')
if isinstance(cover, dict):
url = cover.get('urlDefault') or cover.get('url')
if url:
return url.replace("\\u002F", "/")
info_list = cover.get('infoList', [])
if info_list and isinstance(info_list, list):
url = info_list[0].get('url')
if url:
return url.replace("\\u002F", "/")
elif isinstance(cover, str) and cover:
return cover.replace("\\u002F", "/")
# 兜底从 imageList 中获取首图作为封面
image_list = self.note_data.get('imageList', [])
if image_list and isinstance(image_list, list):
first_img = image_list[0]
if isinstance(first_img, dict):
url = first_img.get('urlDefault') or first_img.get('url')
if url:
return url.replace("\\u002F", "/")
info_list = first_img.get('infoList', [])
if info_list and isinstance(info_list, list):
url = info_list[0].get('url')
if url:
return url.replace("\\u002F", "/")
return None
def get_image_list(self):
image_url_list = []
image_list = self.note_data.get('imageList', [])
if not isinstance(image_list, list):
return image_url_list
for image in image_list:
url = image.get('urlDefault', '')
if not isinstance(image, dict):
continue
url = image.get('urlDefault') or image.get('url')
if not url:
info_list = image.get('infoList', [])
if info_list and isinstance(info_list, list):
url = info_list[-1].get('url') or info_list[0].get('url', '')
if url:
img_data = url.replace("\\u002F", "/")
# 检查是否有livePhoto
# 检查是否有 livePhoto
if image.get('livePhoto', False):
stream = image.get('stream', {})
h264_data = stream.get('h264', [])
if h264_data:
if h264_data and isinstance(h264_data, list):
master_url = h264_data[0].get('masterUrl', '')
if master_url:
img_data = {
+13 -13
View File
@@ -1079,16 +1079,16 @@ table input[type="number"] {
.pie-wrapper {
display: flex;
flex-direction: column;
gap: 16px;
gap: 12px;
align-items: center;
justify-content: center;
padding: 10px 0;
padding: 6px 0;
}
.pie-svg-box {
position: relative;
width: 180px;
height: 180px;
width: 210px;
height: 210px;
flex-shrink: 0;
}
@@ -1099,7 +1099,7 @@ table input[type="number"] {
transform: translate(-50%, -50%);
text-align: center;
pointer-events: none;
width: 120px;
width: 130px;
display: flex;
flex-direction: column;
align-items: center;
@@ -1108,7 +1108,7 @@ table input[type="number"] {
}
.pie-center-lbl {
font-size: 12px;
font-size: 12.5px;
color: var(--ink-muted);
white-space: nowrap;
line-height: 1;
@@ -1116,7 +1116,7 @@ table input[type="number"] {
}
.pie-center-val {
font-size: 21px;
font-size: 24px;
font-weight: 800;
color: var(--ink);
line-height: 1.15;
@@ -1145,12 +1145,12 @@ table input[type="number"] {
flex-wrap: wrap;
justify-content: center;
align-items: center;
gap: 8px 16px;
margin-top: 14px;
gap: 5px 10px;
margin-top: 12px;
width: 100%;
max-height: 220px;
overflow-y: auto;
padding: 4px 8px;
padding: 2px 4px;
}
.pie-pill-grid::-webkit-scrollbar {
@@ -1164,13 +1164,13 @@ table input[type="number"] {
.pie-pill-item {
display: inline-flex;
align-items: center;
gap: 6px;
font-size: 12.5px;
gap: 5px;
font-size: 12px;
color: var(--ink-secondary);
cursor: pointer;
background: transparent;
border: none;
padding: 3px 6px;
padding: 2px 5px;
border-radius: var(--radius-sm);
transition: all 0.15s ease;
user-select: none;
+1
View File
@@ -20,4 +20,5 @@
<symbol id="icon-user" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="8" r="4"/><path d="M4 21a8 8 0 0 1 16 0"/></symbol>
<symbol id="icon-tag" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><path d="M20.6 13.6 13.6 20.6a2 2 0 0 1-2.8 0L3.4 13.2V4h9.2l8 8a1.1 1.1 0 0 1 0 1.6Z"/><circle cx="8" cy="8" r="1"/></symbol>
<symbol id="icon-lightbulb" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><path d="M9 18h6m-5 3h4"/><path d="M8.2 14.2A6 6 0 1 1 16 14.2c-.7.6-1 1.2-1 2.1H9.2c0-.9-.3-1.5-1-2.1Z"/></symbol>
<symbol id="icon-play" viewBox="0 0 24 24" fill="currentColor"><polygon points="6 3 20 12 6 21 6 3"></polygon></symbol>
</svg>

Before

Width:  |  Height:  |  Size: 5.5 KiB

After

Width:  |  Height:  |  Size: 5.7 KiB

+16 -12
View File
@@ -701,31 +701,35 @@
<table style="width:100%;min-width:1080px">
<thead>
<tr>
<th>时间</th>
<th>客户 / Key Prefix</th>
<th>平台</th>
<th style="white-space:nowrap;width:150px">时间</th>
<th style="white-space:nowrap;width:140px">客户 / Key Prefix</th>
<th style="white-space:nowrap;width:90px">平台</th>
<th>脱敏 URL</th>
<th>状态</th>
<th>耗时</th>
<th>错误码</th>
<th style="white-space:nowrap;width:70px">状态</th>
<th style="white-space:nowrap;width:90px">耗时</th>
<th style="white-space:nowrap;min-width:160px">错误码</th>
</tr>
</thead>
<tbody>
{% for log in logs %}
<tr>
<td>{{ format_log_time(log['created_at']) }}</td>
<td>{{ log['username'] or '在线体验' }} {{ log['key_prefix'] or '' }}</td>
<td>{{ log['platform'] or '-' }}</td>
<td style="white-space:nowrap">{{ format_log_time(log['created_at']) }}</td>
<td style="white-space:nowrap">{{ log['username'] or '在线体验' }} {{ log['key_prefix'] or '' }}</td>
<td style="white-space:nowrap">{{ log['platform'] or '-' }}</td>
<td style="max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap">
{% if log['input_url'] %}
<a href="{{ log['input_url'] }}" target="_blank" rel="noopener noreferrer"
title="{{ log['input_url'] }}">{{ log['input_url'] }}</a>
{% else %}-{% endif %}
</td>
<td><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
<td style="white-space:nowrap"><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
}}</span></td>
<td>{{ log['duration_ms'] }} ms</td>
<td>{{ log['error_code'] or '-' }}</td>
<td style="white-space:nowrap">{{ log['duration_ms'] }} ms</td>
<td style="white-space:nowrap">
{% if log['error_code'] %}
<code style="font-size:12px;font-weight:600;color:#dc2626;background:#fef2f2;padding:2px 6px;border-radius:4px;border:1px solid #fee2e2">{{ log['error_code'] }}</code>
{% else %}-{% endif %}
</td>
</tr>
{% else %}
<tr>
+7 -1
View File
@@ -32,7 +32,13 @@
<label>密码</label>
<input name="password" type="password" placeholder="请输入账户密码" autocomplete="current-password" required>
</div>
<div style="margin-top:24px">
<div style="margin-top:14px;display:flex;align-items:center;justify-content:space-between">
<label style="display:inline-flex;align-items:center;gap:7px;cursor:pointer;font-size:13px;color:var(--text-secondary);user-select:none">
<input type="checkbox" name="remember_me" value="1" checked style="accent-color:var(--brand);width:15px;height:15px;cursor:pointer;margin:0">
保持登录状态 (30天免登录)
</label>
</div>
<div style="margin-top:20px">
<button type="submit" class="btn btn-primary btn-block" style="height:40px;font-size:14px">立即登录</button>
</div>
</form>
+76 -16
View File
@@ -106,20 +106,39 @@
<div class="mb-3 flex flex-wrap items-center justify-between gap-3">
<h4 class="text-sm font-semibold text-slate-900 flex items-center gap-2">{{ icon('video', 'w-4 h-4 text-slate-500') }} 视频文件预览 (<span id="videoCount">0</span>)</h4>
<div class="flex items-center gap-2">
<a id="videoOpenDirectLink" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="inline-flex items-center gap-1.5 rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand transition shadow-sm" title="在新窗口中打开当前视频">{{ icon('link', 'w-3.5 h-3.5') }}新窗口打开</a>
<button type="button" onclick="copyCurrentVideoUrl()" class="inline-flex items-center gap-1.5 rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand transition shadow-sm" title="复制当前视频直链">{{ icon('clipboard', 'w-3.5 h-3.5') }}复制直链</button>
<button id="previousVideoBtn" type="button" onclick="stepVideo(-1)" class="hidden rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand disabled:cursor-not-allowed disabled:opacity-40 transition">← 上一个</button>
<button id="nextVideoBtn" type="button" onclick="stepVideo(1)" class="hidden rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand disabled:cursor-not-allowed disabled:opacity-40 transition">下一个 →</button>
</div>
</div>
<div class="relative aspect-video w-full max-h-[460px] overflow-hidden rounded-2xl bg-slate-950 shadow-sm">
<div class="relative aspect-video w-full overflow-hidden rounded-2xl bg-slate-950 shadow-sm">
<!-- 原生视频播放器 -->
<video id="videoPlayer" controls playsinline preload="metadata" referrerpolicy="no-referrer" class="absolute inset-0 h-full w-full bg-black object-contain"></video>
<img id="videoPlaceholder" src="{{ url_for('static', filename='images/video-placeholder.svg') }}" alt="已成功解析视频,但当前浏览器无法预览" class="absolute inset-0 hidden h-full w-full object-contain">
<div class="group absolute inset-x-0 top-0 bottom-32">
<div class="pointer-events-none absolute inset-x-0 top-0 -bottom-32 flex items-center justify-center bg-slate-950/45 opacity-0 transition duration-200 group-hover:opacity-100 group-focus-within:opacity-100">
<div class="pointer-events-auto flex items-center gap-2 p-4">
<a id="videoOpenDirectLink" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="inline-flex items-center gap-1.5 rounded-xl bg-white/95 px-3.5 py-2 text-xs font-bold text-slate-800 shadow-lg hover:bg-white transition">{{ icon('link', 'w-3.5 h-3.5') }}打开浏览器播放</a>
<button type="button" onclick="copyCurrentVideoUrl()" class="inline-flex items-center gap-1.5 rounded-xl bg-white/95 px-3.5 py-2 text-xs font-bold text-slate-800 shadow-lg hover:bg-white transition">{{ icon('clipboard', 'w-3.5 h-3.5') }}复制链接</button>
<!-- 播放受限 / 降级封面与占位容器 (极简现代风) -->
<div id="videoFallbackContainer" class="absolute inset-0 hidden select-none overflow-hidden bg-slate-950 group/fallback">
<!-- 封面图背景高斯模糊(填满 16:9 比例) -->
<img id="videoCoverBg" src="" alt="" aria-hidden="true" referrerpolicy="no-referrer" class="absolute inset-0 hidden h-full w-full object-cover scale-125 blur-2xl opacity-35 brightness-75 pointer-events-none">
<!-- 居中封面主体(保持原比例不失真) -->
<img id="videoCoverImg" src="" alt="视频封面" referrerpolicy="no-referrer" class="absolute inset-0 z-10 hidden h-full w-full object-contain pointer-events-none drop-shadow-2xl">
<!-- 兜底通用占位图(无封面或封面加载失败时) -->
<img id="videoDefaultPlaceholder" src="{{ url_for('static', filename='images/video-placeholder.svg') }}" alt="已成功解析视频,但当前浏览器无法预览" class="absolute inset-0 z-10 hidden h-full w-full object-cover pointer-events-none">
<!-- 半透明暗色微遮罩层 -->
<div class="absolute inset-0 z-20 bg-slate-950/30 backdrop-blur-[0.5px] pointer-events-none transition duration-300 group-hover/fallback:bg-slate-950/40"></div>
<!-- 中央核心交互区:大播放按钮 + 极简轻量文案 (点击整块区域在新窗口播放) -->
<a id="videoFallbackPlayBtn" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="group/btn absolute inset-0 z-30 flex flex-col items-center justify-center p-4 text-center cursor-pointer text-white no-underline" title="在新窗口中播放视频" aria-label="在新窗口中播放视频">
<div class="flex items-center justify-center w-16 h-16 sm:w-20 sm:h-20 rounded-full bg-white/25 backdrop-blur-md border border-white/40 shadow-2xl text-white transition-all duration-300 group-hover/btn:scale-110 group-hover/btn:bg-white/35 active:scale-95 mb-2.5">
<svg class="w-7 h-7 sm:w-8 sm:h-8 translate-x-0.5 fill-white drop-shadow" viewBox="0 0 24 24"><polygon points="6 3 20 12 6 21 6 3"></polygon></svg>
</div>
</div>
<span class="inline-flex items-center gap-1.5 rounded-full bg-black/40 backdrop-blur-md border border-white/20 px-3.5 py-1 text-xs font-medium text-white/95 shadow drop-shadow-sm group-hover/btn:bg-black/60 transition">
点击在新窗口播放 {{ icon('link', 'w-3 h-3') }}
</span>
</a>
</div>
</div>
<div id="videoStrip" class="hidden mt-3 overflow-x-auto custom-scroll scroll-smooth snap-x snap-mandatory pb-2">
@@ -332,6 +351,7 @@
let lastParsedData = null;
let activeLightboxUrl = '';
let currentVideoUrls = [];
let currentCoverUrl = '';
let activeVideoIndex = 0;
const iconMarkup = (name, className) => `<svg class="${className}" aria-hidden="true"><use href="{{ url_for('static', filename='icons.svg') }}#icon-${name}"></use></svg>`;
@@ -502,22 +522,58 @@
if(url) copyText(url, '已复制当前视频链接');
}
function showVideoFallback(selectedUrl){
const videoPlayer = document.getElementById('videoPlayer');
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
const videoCoverBg = document.getElementById('videoCoverBg');
const videoCoverImg = document.getElementById('videoCoverImg');
const videoDefaultPlaceholder = document.getElementById('videoDefaultPlaceholder');
const videoFallbackPlayBtn = document.getElementById('videoFallbackPlayBtn');
videoPlayer.pause();
videoPlayer.classList.add('hidden');
videoFallbackContainer.classList.remove('hidden');
if(videoFallbackPlayBtn) videoFallbackPlayBtn.href = selectedUrl;
if(currentCoverUrl){
videoDefaultPlaceholder.classList.add('hidden');
videoCoverImg.onload = () => {
videoCoverImg.classList.remove('hidden');
videoCoverBg.classList.remove('hidden');
};
videoCoverImg.onerror = () => {
videoCoverImg.classList.add('hidden');
videoCoverBg.classList.add('hidden');
videoDefaultPlaceholder.classList.remove('hidden');
};
videoCoverImg.src = currentCoverUrl;
videoCoverBg.src = currentCoverUrl;
if(videoCoverImg.complete && videoCoverImg.naturalWidth > 0){
videoCoverImg.classList.remove('hidden');
videoCoverBg.classList.remove('hidden');
}
} else {
videoCoverImg.classList.add('hidden');
videoCoverBg.classList.add('hidden');
videoDefaultPlaceholder.classList.remove('hidden');
}
}
function selectVideo(index, autoplay = false){
if(index < 0 || index >= currentVideoUrls.length) return;
activeVideoIndex = index;
const videoPlayer = document.getElementById('videoPlayer');
const videoPlaceholder = document.getElementById('videoPlaceholder');
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
const videoOpenDirectLink = document.getElementById('videoOpenDirectLink');
const selectedUrl = currentVideoUrls[index];
videoPlaceholder.classList.add('hidden');
videoFallbackContainer.classList.add('hidden');
videoPlayer.classList.remove('hidden');
videoOpenDirectLink.href = selectedUrl;
if(videoOpenDirectLink) videoOpenDirectLink.href = selectedUrl;
videoPlayer.onerror = () => {
if(currentVideoUrls[activeVideoIndex] !== selectedUrl) return;
videoPlayer.pause();
videoPlayer.classList.add('hidden');
videoPlaceholder.classList.remove('hidden');
showVideoFallback(selectedUrl);
};
videoPlayer.src = selectedUrl;
videoPlayer.load();
@@ -632,12 +688,16 @@
authorAvatar.classList.add('hidden');
}
currentCoverUrl = (typeof data.cover_url === 'string' && data.cover_url.trim())
|| (typeof data.cover === 'string' && data.cover.trim())
|| (Array.isArray(data.image_list) && data.image_list.length > 0 && typeof data.image_list[0] === 'string' ? data.image_list[0].trim() : '')
|| '';
currentVideoUrls = uniqueMediaUrls([data.video_url, ...(data.video_list || [])]);
// 1. 视频渲染
const videoContainer = document.getElementById('videoContainer');
const videoPlayer = document.getElementById('videoPlayer');
const videoPlaceholder = document.getElementById('videoPlaceholder');
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
const videoStrip = document.getElementById('videoStrip');
const videoTrack = document.getElementById('videoTrack');
const previousVideoBtn = document.getElementById('previousVideoBtn');
@@ -668,7 +728,7 @@
videoPlayer.removeAttribute('src');
videoPlayer.load();
videoPlayer.classList.remove('hidden');
videoPlaceholder.classList.add('hidden');
videoFallbackContainer.classList.add('hidden');
document.getElementById('videoOpenDirectLink').removeAttribute('href');
videoStrip.classList.add('hidden');
previousVideoBtn.classList.add('hidden');
+14 -10
View File
@@ -503,29 +503,33 @@ Authorization: Bearer mp_xxxxxxxxx</pre>
<table style="width:100%;min-width:920px">
<thead>
<tr>
<th style="width:28%">时间</th>
<th>平台</th>
<th style="white-space:nowrap;width:160px">时间</th>
<th style="white-space:nowrap;width:100px">平台</th>
<th>脱敏 URL</th>
<th>状态</th>
<th>耗时</th>
<th>错误码</th>
<th style="white-space:nowrap;width:70px">状态</th>
<th style="white-space:nowrap;width:90px">耗时</th>
<th style="white-space:nowrap;min-width:160px">错误码</th>
</tr>
</thead>
<tbody>
{% for log in logs %}
<tr>
<td>{{ format_log_time(log['created_at']) }}</td>
<td>{{ log['platform'] or '-' }}</td>
<td style="white-space:nowrap">{{ format_log_time(log['created_at']) }}</td>
<td style="white-space:nowrap">{{ log['platform'] or '-' }}</td>
<td style="max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap">
{% if log['input_url'] %}
<a href="{{ log['input_url'] }}" target="_blank" rel="noopener noreferrer"
title="{{ log['input_url'] }}">{{ log['input_url'] }}</a>
{% else %}-{% endif %}
</td>
<td><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
<td style="white-space:nowrap"><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
}}</span></td>
<td>{{ log['duration_ms'] }} ms</td>
<td>{{ log['error_code'] or '-' }}</td>
<td style="white-space:nowrap">{{ log['duration_ms'] }} ms</td>
<td style="white-space:nowrap">
{% if log['error_code'] %}
<code style="font-size:12px;font-weight:600;color:#dc2626;background:#fef2f2;padding:2px 6px;border-radius:4px;border:1px solid #fee2e2">{{ log['error_code'] }}</code>
{% else %}-{% endif %}
</td>
</tr>
{% else %}
<tr>
+18 -1
View File
@@ -1,7 +1,7 @@
{
"schema_version": 2,
"description": "50 平台多形态全景自动化回归测试用例库 (涵盖短链/长链/图集/单视频/AI生成等)",
"total_cases": 308,
"total_cases": 310,
"cases": [
{
"platform": "网易云音乐",
@@ -413,6 +413,23 @@
"note": "2026-08-28 已验证",
"pattern": "基础标准样例"
},
{
"platform": "小红书",
"url": "http://xhslink.com/o/5EY4JTCKXTr",
"pattern": "App 分享短链与 LivePhoto 实况图文",
"media_types": [
"images",
"live_media"
],
"expected_fields": [
"images",
"live_media",
"cover",
"title",
"author"
],
"note": "2026-09 已验证: App 端分享短链,含 6 张高清图集与 LivePhoto 实况动图流"
},
{
"platform": "视频号",
"url": "https://weixin.qq.com/sph/AzGrUgqzFv",
+23
View File
@@ -79,6 +79,29 @@ class ManagementTest(unittest.TestCase):
self.assertEqual(response.status_code, 302)
self.assertTrue(response.location.endswith("/admin"))
def test_login_remember_me_controls_session_permanence(self):
self.client.post(
"/auth/setup",
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123", "confirm_password": "password123"},
)
# Without remember_me
with self.client:
self.client.post(
"/auth/login",
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123"},
)
from flask import session
self.assertFalse(session.permanent)
# With remember_me
with self.client:
self.client.post(
"/auth/login",
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123", "remember_me": "1"},
)
from flask import session
self.assertTrue(session.permanent)
def test_setup_rejects_missing_csrf_token(self):
response = self.client.post(
"/auth/setup",
+32
View File
@@ -0,0 +1,32 @@
import unittest
from src.parsers.xiaohongshu_parser import XiaohongshuParser
class XiaohongshuParserTest(unittest.TestCase):
def test_extract_note_from_pc_initial_state(self):
html = '<script>window.__INITIAL_STATE__ = {"note": {"firstNoteId": "note123", "noteDetailMap": {"note123": {"note": {"title": "PC\u6807\u9898", "desc": "PC\u63cf\u8ff0", "user": {"nickname": "PC\u7528\u6237", "userId": "u123", "avatar": "http://avatar.jpg"}, "imageList": [{"urlDefault": "http://img1.jpg"}]}}}}}</script>'
parser = XiaohongshuParser.__new__(XiaohongshuParser)
note = parser._extract_note_from_html(html)
self.assertIsNotNone(note)
self.assertEqual(note.get("title"), "PC标题")
self.assertEqual(note.get("desc"), "PC描述")
parser.note_data = note
self.assertEqual(parser.get_author_info()["nickname"], "PC用户")
self.assertEqual(parser.get_cover_photo_url(), "http://img1.jpg")
def test_extract_note_from_mobile_initial_state(self):
html = '<script>window.__INITIAL_STATE__ = {"noteData": {"data": {"noteData": {"title": "\u79fb\u52a8\u7aef\u6807\u9898", "desc": "\u79fb\u52a8\u7aef\u63cf\u8ff0", "user": {"nickName": "\u79fb\u52a8\u7aef\u7528\u6237", "userId": "u456", "avatar": "http://m_avatar.jpg"}, "imageList": [{"url": "http://m_img1.jpg", "livePhoto": true, "stream": {"h264": [{"masterUrl": "http://live.mp4"}]}}]}}}}</script>'
parser = XiaohongshuParser.__new__(XiaohongshuParser)
note = parser._extract_note_from_html(html)
self.assertIsNotNone(note)
self.assertEqual(note.get("title"), "移动端标题")
parser.note_data = note
self.assertEqual(parser.get_author_info()["nickname"], "移动端用户")
self.assertEqual(parser.get_cover_photo_url(), "http://m_img1.jpg")
images = parser.get_image_list()
self.assertEqual(len(images), 1)
self.assertEqual(images[0]["live_photo_url"], "http://live.mp4")
if __name__ == "__main__":
unittest.main()
+7 -3
View File
@@ -172,9 +172,13 @@ class UrlParser:
address = f"{address}?id={vid}"
elif platform == "小红书":
query_params = parse_qs(parsed_url.query)
xsec_token = query_params.get('xsec_token', [None])[0] # 使用 get 方法避免 KeyError
if xsec_token:
address = f"{address}?xsec_token={xsec_token}"
preserved_params = []
for key in ('xsec_token', 'xsec_source', 'source', 'xhsshare', 'app_platform'):
value = query_params.get(key, [None])[0]
if value is not None:
preserved_params.append((key, value))
if preserved_params:
address = f"{address}?{urlencode(preserved_params)}"
elif platform == "快手":
address = address.replace('http://', 'https://')
elif platform == "抖音":