mirror of
https://github.com/ucmao/media-parser.git
synced 2026-11-20 08:04:56 +08:00
优化小红书App短链与LivePhoto解析,与UI体验优化
This commit is contained in:
@@ -1,6 +1,7 @@
|
||||
import os
|
||||
import secrets
|
||||
import fcntl
|
||||
from datetime import timedelta
|
||||
from flask import Flask
|
||||
from src.api.parse import bp as api_bp
|
||||
from src.web.views import bp as web_bp
|
||||
@@ -43,6 +44,7 @@ def create_app(config=None):
|
||||
'DATABASE_PATH', os.path.join(app.instance_path, 'media_parser.db')
|
||||
)
|
||||
app.config['MAX_CONTENT_LENGTH'] = 32 * 1024
|
||||
app.config['PERMANENT_SESSION_LIFETIME'] = timedelta(days=30)
|
||||
app.config['JSON_SORT_KEYS'] = False
|
||||
app.config['TRUST_PROXY_HEADERS'] = os.getenv(
|
||||
'TRUST_PROXY_HEADERS', ''
|
||||
|
||||
+1
-1
@@ -30,7 +30,7 @@
|
||||
| 序号 | 平台名称 | 支持媒体类型 | 无水印直链 | 配置门槛 (Cookie 依赖) | 逆向提取模式 | 对应指南 |
|
||||
| :--- | :--- | :--- | :---: | :---: | :--- | :--- |
|
||||
| 1 | **抖音** | 视频 / 图文 / LivePhoto / 音频 / 字幕 / 合集 | ✅ 支持 | 🟢 免配置 | a_bogus 签名 + 动态 ttwid + SSR 容灾兜底 | [查看指南](parsers/douyin.md) |
|
||||
| 2 | **小红书** | 图文 / LivePhoto / 视频 | ✅ 支持 | ⚠️ 部分需 Cookie | SSR HTML 注入数据提取 | [查看指南](parsers/xiaohongshu.md) |
|
||||
| 2 | **小红书** | 图文 / LivePhoto / 视频 | ✅ 支持 | 🟢 免配置 | SSR HTML 状态注入 + 双 UA 自适应回退 | [查看指南](parsers/xiaohongshu.md) |
|
||||
| 3 | **快手** | 视频 / 图文 / 音频 | ✅ 支持 | 🟢 免配置 (内置游客凭证) | GraphQL / H5 双端多路由 Fallback | [查看指南](parsers/kuaishou.md) |
|
||||
| 4 | **哔哩哔哩** | 视频 (MP4) / 音频分流 | ✅ 支持 | 🟢 免配置 | 官方 View + PlayURL API | [查看指南](parsers/bilibili.md) |
|
||||
| 5 | **豆包 AI** | AI 视频生成直链 | ✅ 支持 | 🔐 需 `DOUBAO_COOKIE` | Web Session 维持 + 任务轮询 | [查看指南](parsers/doubao.md) |
|
||||
|
||||
+45
-22
@@ -12,35 +12,47 @@
|
||||
* 实况照片 (LivePhoto 动态视频流)
|
||||
* 视频笔记 (H264 Master MP4 流)
|
||||
* 笔记正文与作者信息
|
||||
* 笔记封面 (自动从 cover 或首图提取)
|
||||
* **常见链接形态**:
|
||||
* 短链接:`http://xhslink.com/a/xxxx`
|
||||
* 短链接:`http://xhslink.com/o/xxxx` 或 `http://xhslink.com/a/xxxx`
|
||||
* 网页发现长链:`https://www.xiaohongshu.com/discovery/item/6a8fbccc000000002a03825a`
|
||||
* 网页探索长链:`https://www.xiaohongshu.com/explore/6a8fbccc000000002a03825a`
|
||||
* **Cookie 依赖**:公开普通笔记无需 Cookie;部分强受限笔记或高频调用建议在 `.env` 中提供 Cookie。
|
||||
* **Cookie 依赖**:🟢 公开普通笔记及移动分享均免配置 Cookie。
|
||||
|
||||
---
|
||||
|
||||
## 2. 核心逆向方案 (SSR HTML 状态注入)
|
||||
## 2. 核心逆向方案 (SSR HTML 状态注入 + 双 UA 自适应回退)
|
||||
|
||||
小红书 Web 页面采用 Nuxt / Vue SSR 服务端渲染。页面加载时,首屏所有笔记详情、高清媒体流及作者信息均已经内嵌在 HTML 的 `window.__INITIAL_STATE__` 变量中。
|
||||
小红书 Web/H5 页面采用 Nuxt / Vue SSR 服务端渲染。页面加载时,首屏所有笔记详情、高清媒体流及作者信息均已经内嵌在 HTML 的 `window.__INITIAL_STATE__` 变量中。
|
||||
|
||||
### 2.1 请求配置
|
||||
* **请求方式**:`GET` 目标长链接
|
||||
* **必备请求头**:
|
||||
* `User-Agent`: 现代 PC 浏览器 UA
|
||||
* `Referer`: `https://www.xiaohongshu.com/`
|
||||
### 2.1 请求配置与 URL 参数保留
|
||||
* **请求方式**:`GET` 目标解析长链接
|
||||
* **关键查询参数保留**:短链 302 重定向后必须保留 `xsec_token`、`xsec_source`、`source`、`xhsshare`、`app_platform` 等核心参数(在 [utils/web_fetcher.py](file:///Users/leo/Projects/media-parser/utils/web_fetcher.py) 的 `UrlParser.extract_video_address` 中维护),缺少来源签名参数会导致服务端返回 404 或拦截。
|
||||
* **双 UA 自适应机制**:
|
||||
* **App 端分享短链**(含 `app_platform=android` 等):优先使用 **Mobile User-Agent** 访问以获取移动端 H5 页面。
|
||||
* **PC 桌面端长链**(含 `source=webshare` 等):优先使用 **PC User-Agent** 访问。
|
||||
* **自动回退**:若首选 UA 遭遇 302/404 或拦截,解析器自动切换为备用 UA 再次发起请求。
|
||||
|
||||
### 2.2 正则状态提取
|
||||
### 2.2 正则状态提取与结构兼容
|
||||
```python
|
||||
import re, json
|
||||
|
||||
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
|
||||
json_str = BaseParser.parse_html_data(html_content, pattern)
|
||||
# 1. 正则提取并清洗 :undefined 为 :null
|
||||
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*?\})</script>', re.DOTALL)
|
||||
match = pattern.search(html_content) or re.search(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', html_content, re.DOTALL)
|
||||
|
||||
if json_str:
|
||||
if match:
|
||||
json_str = re.sub(r':\s*undefined\b', ':null', match.group(1))
|
||||
full_data = json.loads(json_str)
|
||||
|
||||
# 2. 兼容 PC 桌面端数据结构
|
||||
first_note_id = full_data.get('note', {}).get('firstNoteId')
|
||||
note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
|
||||
if first_note_id:
|
||||
note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
|
||||
|
||||
# 3. 兼容移动端 H5 数据结构
|
||||
if not note_data:
|
||||
note_data = full_data.get('noteData', {}).get('data', {}).get('noteData', {})
|
||||
```
|
||||
|
||||
---
|
||||
@@ -48,23 +60,30 @@ if json_str:
|
||||
## 3. 字段提取与 LivePhoto 解析
|
||||
|
||||
### 3.1 视频笔记提取
|
||||
* 视频直链位于:`note_data.video.media.stream.h264[0].masterUrl`
|
||||
* 视频直链位于:`note_data.video.media.stream.h264[0].masterUrl` 或 `note_data.video.stream.h264[0].masterUrl`。
|
||||
* 字符转义处理:将 Unicode 斜杠 `\\u002F` 替换为标准 `/`。
|
||||
|
||||
### 3.2 图文与 LivePhoto 提取
|
||||
* 遍历 `note_data.imageList`:
|
||||
* **原图直链**:提取 `urlDefault` 字段。
|
||||
* **实况动图 (LivePhoto)**:检查 `image.get('livePhoto') == True`,若为真,则提取 `image.stream.h264[0].masterUrl` 作为实况动图关联的视频流。
|
||||
* **原图直链**:按优先级提取 `image.urlDefault` -> `image.url` -> `image.infoList[-1].url`。
|
||||
* **实况动图 (LivePhoto)**:检查 `image.get('livePhoto') == True`,若为真,则提取 `image.stream.h264[0].masterUrl` 作为实况动图关联的动态视频流。
|
||||
|
||||
### 3.3 封面与作者提取
|
||||
* **封面**:优先读取 `note_data.cover.urlDefault` / `url`,若为空则以 `imageList[0]` 兜底。
|
||||
* **作者**:兼容读取 `user.nickname` / `user.nickName` 及 `user.userId`。
|
||||
|
||||
---
|
||||
|
||||
## 4. 常见踩坑与反爬对抗 (Gotchas)
|
||||
|
||||
1. **重定向至登录页 (`/login`) 或 404**:
|
||||
* *原因*:小红书对单一 IP 频次较高的请求,或者未带参数的特定私密笔记会强制要求登录。
|
||||
* *解法*:在 [src/api/parse.py](file:///Users/leo/Projects/media-parser/src/api/parse.py) 中针对小红书内置了 3 次自适应重试机制(`_fetch_with_retry`);若持续失败,则友好提示用户配置 Cookie。
|
||||
2. **字符转义问题**:
|
||||
* JSON 字符串中经常包含 JSON 编码的斜杠 `\u002F`,必须统一格式化清洗。
|
||||
1. **App 分享短链在 PC UA 下重定向到 `/explore?undertake_note_error=该内容暂时无法查看` (错误码 -510001)**:
|
||||
* *原因*:小红书服务端对 App 端生成的分享链接(带有 `app_platform=android&xsec_source=app_share` 等参数)进行设备特征校验,PC UA 访问会被判定为非移动环境并重定向到探索页。
|
||||
* *解法*:在 [src/parsers/xiaohongshu_parser.py](file:///Users/leo/Projects/media-parser/src/parsers/xiaohongshu_parser.py) 中启用双 UA 自动降级重试机制,对 App 分享链接自动采用移动端 User-Agent 请求,无需 Cookie 即可获取完整 H5 状态树。
|
||||
2. **缺少 `xsec_source` 导致 H5 页面返回 404/安全校验**:
|
||||
* *原因*:在 URL 清洗阶段若仅保留 `xsec_token` 而丢弃了 `xsec_source`,移动端 H5 接口会因缺少渠道来源标识而拒绝渲染笔记数据。
|
||||
* *解法*:在 [utils/web_fetcher.py](file:///Users/leo/Projects/media-parser/utils/web_fetcher.py) 的 `UrlParser.extract_video_address` 中完整保留 `xsec_token`, `xsec_source`, `source`, `xhsshare`, `app_platform` 等核心参数。
|
||||
3. **字符转义与 `undefined` 序列化**:
|
||||
* HTML 内嵌的 JS 对象常包含 `:undefined`,直接 `json.loads` 会抛出异常,需统一正则替换为 `:null`。
|
||||
|
||||
---
|
||||
|
||||
@@ -74,3 +93,7 @@ if json_str:
|
||||
```bash
|
||||
python tests/manual_verify_parsers.py --platform 小红书
|
||||
```
|
||||
* **单元测试**:
|
||||
```bash
|
||||
python -m unittest tests/test_xiaohongshu_parser.py
|
||||
```
|
||||
|
||||
@@ -202,6 +202,7 @@ def login():
|
||||
else:
|
||||
session.clear()
|
||||
session["user_id"] = user["id"]
|
||||
session.permanent = request.form.get("remember_me") in {"1", "on", "true"}
|
||||
target = request.args.get("next", "")
|
||||
if not target.startswith("/") or target.startswith("//"):
|
||||
target = url_for("admin.dashboard") if user["role"] == "admin" else url_for("portal.dashboard")
|
||||
|
||||
@@ -3,7 +3,9 @@ import re
|
||||
import json
|
||||
from src.parsers.base_parser import BaseParser
|
||||
from configs.logging_config import get_logger
|
||||
from configs.general_constants import USER_AGENT_PC, USER_AGENT_M
|
||||
import requests
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@@ -11,45 +13,80 @@ logger = get_logger(__name__)
|
||||
class XiaohongshuParser(BaseParser):
|
||||
def __init__(self, real_url):
|
||||
super().__init__(real_url)
|
||||
self.headers = {
|
||||
"content-type": "application/json; charset=UTF-8",
|
||||
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36",
|
||||
'referer': 'https://www.xiaohongshu.com/'
|
||||
}
|
||||
|
||||
# 获取 HTML 并解析 JSON 状态
|
||||
html_content = self.fetch_html_content()
|
||||
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
|
||||
json_str = BaseParser.parse_html_data(html_content, pattern)
|
||||
|
||||
# 初始化数据容器
|
||||
self.note_data = {}
|
||||
try:
|
||||
if json_str:
|
||||
full_data = json.loads(json_str)
|
||||
first_note_id = full_data.get('note', {}).get('firstNoteId')
|
||||
if first_note_id:
|
||||
self.note_data = full_data['note']['noteDetailMap'].get(first_note_id, {}).get('note', {})
|
||||
except (json.JSONDecodeError, KeyError) as e:
|
||||
logger.error(f"初始化解析数据失败: {e}")
|
||||
self.fetch_and_parse()
|
||||
|
||||
def fetch_html_content(self):
|
||||
try:
|
||||
resp = self.session.get(self.real_url, headers=self.headers, timeout=5)
|
||||
resp.raise_for_status()
|
||||
if "xiaohongshu.com/login" in resp.url:
|
||||
logger.error("小红书解析提示: 请求被重定向到了登录页面。")
|
||||
elif "xiaohongshu.com/404" in resp.url:
|
||||
logger.error("小红书解析提示: 遭遇安全拦截或页面未找到(404)。")
|
||||
self.html_content = resp.text
|
||||
return self.html_content
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"Failed to get the page: {self.real_url}, Error: {e}")
|
||||
def fetch_and_parse(self):
|
||||
pc_ua = USER_AGENT_PC[0] if USER_AGENT_PC else "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
|
||||
mobile_ua = USER_AGENT_M[0] if USER_AGENT_M else "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1"
|
||||
|
||||
# 根据链接特征选择优先尝试的 UA
|
||||
if "app_platform" in self.real_url or "app_share" in self.real_url or "xhslink" in self.real_url:
|
||||
candidate_uas = [mobile_ua, pc_ua]
|
||||
else:
|
||||
candidate_uas = [pc_ua, mobile_ua]
|
||||
|
||||
for ua in candidate_uas:
|
||||
headers = {
|
||||
"User-Agent": ua,
|
||||
"Referer": "https://www.xiaohongshu.com/",
|
||||
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
||||
}
|
||||
try:
|
||||
resp = self.session.get(self.real_url, headers=headers, timeout=6)
|
||||
if resp.status_code != 200:
|
||||
continue
|
||||
if "xiaohongshu.com/login" in resp.url or "xiaohongshu.com/404" in resp.url or "undertake_note_error" in resp.url:
|
||||
continue
|
||||
|
||||
self.html_content = resp.text
|
||||
note = self._extract_note_from_html(self.html_content)
|
||||
if note:
|
||||
self.note_data = note
|
||||
return
|
||||
except requests.RequestException as e:
|
||||
logger.warning(f"小红书请求失败 (UA: {ua[:30]}...): {e}")
|
||||
continue
|
||||
except Exception as e:
|
||||
logger.error(f"解析小红书页面异常: {e}")
|
||||
continue
|
||||
|
||||
if not self.note_data:
|
||||
logger.error(f"未能解析出小红书笔记数据: {self.real_url}")
|
||||
|
||||
def _extract_note_from_html(self, html_content):
|
||||
if not html_content:
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"An unexpected error occurred while fetching {self.real_url}: {e}")
|
||||
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*?\})</script>', re.DOTALL)
|
||||
match = pattern.search(html_content)
|
||||
if not match:
|
||||
pattern = re.compile(r'window\.__INITIAL_STATE__\s*=\s*(\{.*\})', re.DOTALL)
|
||||
match = pattern.search(html_content)
|
||||
if not match:
|
||||
return None
|
||||
|
||||
json_str = match.group(1)
|
||||
json_str = re.sub(r':\s*undefined\b', ':null', json_str)
|
||||
try:
|
||||
full_data = json.loads(json_str)
|
||||
except json.JSONDecodeError as e:
|
||||
logger.error(f"小红书 JSON 解析失败: {e}")
|
||||
return None
|
||||
|
||||
# 1. PC 端结构: full_data['note']['noteDetailMap'][first_note_id]['note']
|
||||
first_note_id = full_data.get('note', {}).get('firstNoteId')
|
||||
if first_note_id:
|
||||
note = full_data.get('note', {}).get('noteDetailMap', {}).get(first_note_id, {}).get('note', {})
|
||||
if note:
|
||||
return note
|
||||
|
||||
# 2. 移动端 H5 结构: full_data['noteData']['data']['noteData']
|
||||
note = full_data.get('noteData', {}).get('data', {}).get('noteData', {})
|
||||
if note:
|
||||
return note
|
||||
|
||||
# 3. 兜底回退结构
|
||||
return full_data.get('noteData', {}).get('note', {}) or full_data.get('note', {}).get('note', {}) or None
|
||||
|
||||
def get_author_info(self):
|
||||
"""
|
||||
@@ -57,17 +94,30 @@ class XiaohongshuParser(BaseParser):
|
||||
"""
|
||||
user = self.note_data.get('user', {})
|
||||
return {
|
||||
'nickname': user.get('nickname', ''),
|
||||
'author_id': user.get('userId', ''),
|
||||
'nickname': user.get('nickname') or user.get('nickName', ''),
|
||||
'author_id': user.get('userId') or user.get('author_id') or user.get('id', ''),
|
||||
'avatar': user.get('avatar', '')
|
||||
}
|
||||
|
||||
def get_real_video_url(self):
|
||||
try:
|
||||
video_info = self.note_data.get('video', {})
|
||||
master_url = video_info['media']['stream']['h264'][0].get('masterUrl', '')
|
||||
return master_url.replace("\\u002F", "/") if master_url else None
|
||||
except (KeyError, IndexError):
|
||||
if not isinstance(video_info, dict):
|
||||
return None
|
||||
stream = video_info.get('media', {}).get('stream', {}) or video_info.get('stream', {})
|
||||
h264_data = stream.get('h264', [])
|
||||
if h264_data and isinstance(h264_data, list):
|
||||
master_url = h264_data[0].get('masterUrl', '')
|
||||
if master_url:
|
||||
return master_url.replace("\\u002F", "/")
|
||||
for codec in ('h265', 'av1'):
|
||||
codec_data = stream.get(codec, [])
|
||||
if codec_data and isinstance(codec_data, list):
|
||||
master_url = codec_data[0].get('masterUrl', '')
|
||||
if master_url:
|
||||
return master_url.replace("\\u002F", "/")
|
||||
return None
|
||||
except (KeyError, IndexError, AttributeError):
|
||||
return None
|
||||
|
||||
def get_title_content(self):
|
||||
@@ -77,20 +127,55 @@ class XiaohongshuParser(BaseParser):
|
||||
return self.note_data.get('desc', '') or None
|
||||
|
||||
def get_cover_photo_url(self):
|
||||
cover = self.note_data.get('cover')
|
||||
if isinstance(cover, dict):
|
||||
url = cover.get('urlDefault') or cover.get('url')
|
||||
if url:
|
||||
return url.replace("\\u002F", "/")
|
||||
info_list = cover.get('infoList', [])
|
||||
if info_list and isinstance(info_list, list):
|
||||
url = info_list[0].get('url')
|
||||
if url:
|
||||
return url.replace("\\u002F", "/")
|
||||
elif isinstance(cover, str) and cover:
|
||||
return cover.replace("\\u002F", "/")
|
||||
|
||||
# 兜底从 imageList 中获取首图作为封面
|
||||
image_list = self.note_data.get('imageList', [])
|
||||
if image_list and isinstance(image_list, list):
|
||||
first_img = image_list[0]
|
||||
if isinstance(first_img, dict):
|
||||
url = first_img.get('urlDefault') or first_img.get('url')
|
||||
if url:
|
||||
return url.replace("\\u002F", "/")
|
||||
info_list = first_img.get('infoList', [])
|
||||
if info_list and isinstance(info_list, list):
|
||||
url = info_list[0].get('url')
|
||||
if url:
|
||||
return url.replace("\\u002F", "/")
|
||||
return None
|
||||
|
||||
def get_image_list(self):
|
||||
image_url_list = []
|
||||
image_list = self.note_data.get('imageList', [])
|
||||
if not isinstance(image_list, list):
|
||||
return image_url_list
|
||||
|
||||
for image in image_list:
|
||||
url = image.get('urlDefault', '')
|
||||
if not isinstance(image, dict):
|
||||
continue
|
||||
url = image.get('urlDefault') or image.get('url')
|
||||
if not url:
|
||||
info_list = image.get('infoList', [])
|
||||
if info_list and isinstance(info_list, list):
|
||||
url = info_list[-1].get('url') or info_list[0].get('url', '')
|
||||
if url:
|
||||
img_data = url.replace("\\u002F", "/")
|
||||
# 检查是否有livePhoto
|
||||
# 检查是否有 livePhoto
|
||||
if image.get('livePhoto', False):
|
||||
stream = image.get('stream', {})
|
||||
h264_data = stream.get('h264', [])
|
||||
if h264_data:
|
||||
if h264_data and isinstance(h264_data, list):
|
||||
master_url = h264_data[0].get('masterUrl', '')
|
||||
if master_url:
|
||||
img_data = {
|
||||
|
||||
+13
-13
@@ -1079,16 +1079,16 @@ table input[type="number"] {
|
||||
.pie-wrapper {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 16px;
|
||||
gap: 12px;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 10px 0;
|
||||
padding: 6px 0;
|
||||
}
|
||||
|
||||
.pie-svg-box {
|
||||
position: relative;
|
||||
width: 180px;
|
||||
height: 180px;
|
||||
width: 210px;
|
||||
height: 210px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
@@ -1099,7 +1099,7 @@ table input[type="number"] {
|
||||
transform: translate(-50%, -50%);
|
||||
text-align: center;
|
||||
pointer-events: none;
|
||||
width: 120px;
|
||||
width: 130px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
@@ -1108,7 +1108,7 @@ table input[type="number"] {
|
||||
}
|
||||
|
||||
.pie-center-lbl {
|
||||
font-size: 12px;
|
||||
font-size: 12.5px;
|
||||
color: var(--ink-muted);
|
||||
white-space: nowrap;
|
||||
line-height: 1;
|
||||
@@ -1116,7 +1116,7 @@ table input[type="number"] {
|
||||
}
|
||||
|
||||
.pie-center-val {
|
||||
font-size: 21px;
|
||||
font-size: 24px;
|
||||
font-weight: 800;
|
||||
color: var(--ink);
|
||||
line-height: 1.15;
|
||||
@@ -1145,12 +1145,12 @@ table input[type="number"] {
|
||||
flex-wrap: wrap;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
gap: 8px 16px;
|
||||
margin-top: 14px;
|
||||
gap: 5px 10px;
|
||||
margin-top: 12px;
|
||||
width: 100%;
|
||||
max-height: 220px;
|
||||
overflow-y: auto;
|
||||
padding: 4px 8px;
|
||||
padding: 2px 4px;
|
||||
}
|
||||
|
||||
.pie-pill-grid::-webkit-scrollbar {
|
||||
@@ -1164,13 +1164,13 @@ table input[type="number"] {
|
||||
.pie-pill-item {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 12.5px;
|
||||
gap: 5px;
|
||||
font-size: 12px;
|
||||
color: var(--ink-secondary);
|
||||
cursor: pointer;
|
||||
background: transparent;
|
||||
border: none;
|
||||
padding: 3px 6px;
|
||||
padding: 2px 5px;
|
||||
border-radius: var(--radius-sm);
|
||||
transition: all 0.15s ease;
|
||||
user-select: none;
|
||||
|
||||
@@ -20,4 +20,5 @@
|
||||
<symbol id="icon-user" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="8" r="4"/><path d="M4 21a8 8 0 0 1 16 0"/></symbol>
|
||||
<symbol id="icon-tag" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><path d="M20.6 13.6 13.6 20.6a2 2 0 0 1-2.8 0L3.4 13.2V4h9.2l8 8a1.1 1.1 0 0 1 0 1.6Z"/><circle cx="8" cy="8" r="1"/></symbol>
|
||||
<symbol id="icon-lightbulb" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round"><path d="M9 18h6m-5 3h4"/><path d="M8.2 14.2A6 6 0 1 1 16 14.2c-.7.6-1 1.2-1 2.1H9.2c0-.9-.3-1.5-1-2.1Z"/></symbol>
|
||||
<symbol id="icon-play" viewBox="0 0 24 24" fill="currentColor"><polygon points="6 3 20 12 6 21 6 3"></polygon></symbol>
|
||||
</svg>
|
||||
|
||||
|
Before Width: | Height: | Size: 5.5 KiB After Width: | Height: | Size: 5.7 KiB |
@@ -701,31 +701,35 @@
|
||||
<table style="width:100%;min-width:1080px">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>时间</th>
|
||||
<th>客户 / Key Prefix</th>
|
||||
<th>平台</th>
|
||||
<th style="white-space:nowrap;width:150px">时间</th>
|
||||
<th style="white-space:nowrap;width:140px">客户 / Key Prefix</th>
|
||||
<th style="white-space:nowrap;width:90px">平台</th>
|
||||
<th>脱敏 URL</th>
|
||||
<th>状态</th>
|
||||
<th>耗时</th>
|
||||
<th>错误码</th>
|
||||
<th style="white-space:nowrap;width:70px">状态</th>
|
||||
<th style="white-space:nowrap;width:90px">耗时</th>
|
||||
<th style="white-space:nowrap;min-width:160px">错误码</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for log in logs %}
|
||||
<tr>
|
||||
<td>{{ format_log_time(log['created_at']) }}</td>
|
||||
<td>{{ log['username'] or '在线体验' }} {{ log['key_prefix'] or '' }}</td>
|
||||
<td>{{ log['platform'] or '-' }}</td>
|
||||
<td style="white-space:nowrap">{{ format_log_time(log['created_at']) }}</td>
|
||||
<td style="white-space:nowrap">{{ log['username'] or '在线体验' }} {{ log['key_prefix'] or '' }}</td>
|
||||
<td style="white-space:nowrap">{{ log['platform'] or '-' }}</td>
|
||||
<td style="max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap">
|
||||
{% if log['input_url'] %}
|
||||
<a href="{{ log['input_url'] }}" target="_blank" rel="noopener noreferrer"
|
||||
title="{{ log['input_url'] }}">{{ log['input_url'] }}</a>
|
||||
{% else %}-{% endif %}
|
||||
</td>
|
||||
<td><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
|
||||
<td style="white-space:nowrap"><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
|
||||
}}</span></td>
|
||||
<td>{{ log['duration_ms'] }} ms</td>
|
||||
<td>{{ log['error_code'] or '-' }}</td>
|
||||
<td style="white-space:nowrap">{{ log['duration_ms'] }} ms</td>
|
||||
<td style="white-space:nowrap">
|
||||
{% if log['error_code'] %}
|
||||
<code style="font-size:12px;font-weight:600;color:#dc2626;background:#fef2f2;padding:2px 6px;border-radius:4px;border:1px solid #fee2e2">{{ log['error_code'] }}</code>
|
||||
{% else %}-{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% else %}
|
||||
<tr>
|
||||
|
||||
@@ -32,7 +32,13 @@
|
||||
<label>密码</label>
|
||||
<input name="password" type="password" placeholder="请输入账户密码" autocomplete="current-password" required>
|
||||
</div>
|
||||
<div style="margin-top:24px">
|
||||
<div style="margin-top:14px;display:flex;align-items:center;justify-content:space-between">
|
||||
<label style="display:inline-flex;align-items:center;gap:7px;cursor:pointer;font-size:13px;color:var(--text-secondary);user-select:none">
|
||||
<input type="checkbox" name="remember_me" value="1" checked style="accent-color:var(--brand);width:15px;height:15px;cursor:pointer;margin:0">
|
||||
保持登录状态 (30天免登录)
|
||||
</label>
|
||||
</div>
|
||||
<div style="margin-top:20px">
|
||||
<button type="submit" class="btn btn-primary btn-block" style="height:40px;font-size:14px">立即登录</button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
+76
-16
@@ -106,20 +106,39 @@
|
||||
<div class="mb-3 flex flex-wrap items-center justify-between gap-3">
|
||||
<h4 class="text-sm font-semibold text-slate-900 flex items-center gap-2">{{ icon('video', 'w-4 h-4 text-slate-500') }} 视频文件预览 (<span id="videoCount">0</span>)</h4>
|
||||
<div class="flex items-center gap-2">
|
||||
<a id="videoOpenDirectLink" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="inline-flex items-center gap-1.5 rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand transition shadow-sm" title="在新窗口中打开当前视频">{{ icon('link', 'w-3.5 h-3.5') }}新窗口打开</a>
|
||||
<button type="button" onclick="copyCurrentVideoUrl()" class="inline-flex items-center gap-1.5 rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand transition shadow-sm" title="复制当前视频直链">{{ icon('clipboard', 'w-3.5 h-3.5') }}复制直链</button>
|
||||
<button id="previousVideoBtn" type="button" onclick="stepVideo(-1)" class="hidden rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand disabled:cursor-not-allowed disabled:opacity-40 transition">← 上一个</button>
|
||||
<button id="nextVideoBtn" type="button" onclick="stepVideo(1)" class="hidden rounded-lg border border-slate-200 bg-white px-2.5 py-1.5 text-xs font-semibold text-slate-600 hover:border-indigo-300 hover:text-brand disabled:cursor-not-allowed disabled:opacity-40 transition">下一个 →</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="relative aspect-video w-full max-h-[460px] overflow-hidden rounded-2xl bg-slate-950 shadow-sm">
|
||||
<div class="relative aspect-video w-full overflow-hidden rounded-2xl bg-slate-950 shadow-sm">
|
||||
<!-- 原生视频播放器 -->
|
||||
<video id="videoPlayer" controls playsinline preload="metadata" referrerpolicy="no-referrer" class="absolute inset-0 h-full w-full bg-black object-contain"></video>
|
||||
<img id="videoPlaceholder" src="{{ url_for('static', filename='images/video-placeholder.svg') }}" alt="已成功解析视频,但当前浏览器无法预览" class="absolute inset-0 hidden h-full w-full object-contain">
|
||||
<div class="group absolute inset-x-0 top-0 bottom-32">
|
||||
<div class="pointer-events-none absolute inset-x-0 top-0 -bottom-32 flex items-center justify-center bg-slate-950/45 opacity-0 transition duration-200 group-hover:opacity-100 group-focus-within:opacity-100">
|
||||
<div class="pointer-events-auto flex items-center gap-2 p-4">
|
||||
<a id="videoOpenDirectLink" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="inline-flex items-center gap-1.5 rounded-xl bg-white/95 px-3.5 py-2 text-xs font-bold text-slate-800 shadow-lg hover:bg-white transition">{{ icon('link', 'w-3.5 h-3.5') }}打开浏览器播放</a>
|
||||
<button type="button" onclick="copyCurrentVideoUrl()" class="inline-flex items-center gap-1.5 rounded-xl bg-white/95 px-3.5 py-2 text-xs font-bold text-slate-800 shadow-lg hover:bg-white transition">{{ icon('clipboard', 'w-3.5 h-3.5') }}复制链接</button>
|
||||
|
||||
<!-- 播放受限 / 降级封面与占位容器 (极简现代风) -->
|
||||
<div id="videoFallbackContainer" class="absolute inset-0 hidden select-none overflow-hidden bg-slate-950 group/fallback">
|
||||
<!-- 封面图背景高斯模糊(填满 16:9 比例) -->
|
||||
<img id="videoCoverBg" src="" alt="" aria-hidden="true" referrerpolicy="no-referrer" class="absolute inset-0 hidden h-full w-full object-cover scale-125 blur-2xl opacity-35 brightness-75 pointer-events-none">
|
||||
|
||||
<!-- 居中封面主体(保持原比例不失真) -->
|
||||
<img id="videoCoverImg" src="" alt="视频封面" referrerpolicy="no-referrer" class="absolute inset-0 z-10 hidden h-full w-full object-contain pointer-events-none drop-shadow-2xl">
|
||||
|
||||
<!-- 兜底通用占位图(无封面或封面加载失败时) -->
|
||||
<img id="videoDefaultPlaceholder" src="{{ url_for('static', filename='images/video-placeholder.svg') }}" alt="已成功解析视频,但当前浏览器无法预览" class="absolute inset-0 z-10 hidden h-full w-full object-cover pointer-events-none">
|
||||
|
||||
<!-- 半透明暗色微遮罩层 -->
|
||||
<div class="absolute inset-0 z-20 bg-slate-950/30 backdrop-blur-[0.5px] pointer-events-none transition duration-300 group-hover/fallback:bg-slate-950/40"></div>
|
||||
|
||||
<!-- 中央核心交互区:大播放按钮 + 极简轻量文案 (点击整块区域在新窗口播放) -->
|
||||
<a id="videoFallbackPlayBtn" target="_blank" rel="noopener noreferrer" referrerpolicy="no-referrer" class="group/btn absolute inset-0 z-30 flex flex-col items-center justify-center p-4 text-center cursor-pointer text-white no-underline" title="在新窗口中播放视频" aria-label="在新窗口中播放视频">
|
||||
<div class="flex items-center justify-center w-16 h-16 sm:w-20 sm:h-20 rounded-full bg-white/25 backdrop-blur-md border border-white/40 shadow-2xl text-white transition-all duration-300 group-hover/btn:scale-110 group-hover/btn:bg-white/35 active:scale-95 mb-2.5">
|
||||
<svg class="w-7 h-7 sm:w-8 sm:h-8 translate-x-0.5 fill-white drop-shadow" viewBox="0 0 24 24"><polygon points="6 3 20 12 6 21 6 3"></polygon></svg>
|
||||
</div>
|
||||
</div>
|
||||
<span class="inline-flex items-center gap-1.5 rounded-full bg-black/40 backdrop-blur-md border border-white/20 px-3.5 py-1 text-xs font-medium text-white/95 shadow drop-shadow-sm group-hover/btn:bg-black/60 transition">
|
||||
点击在新窗口播放 {{ icon('link', 'w-3 h-3') }}
|
||||
</span>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
<div id="videoStrip" class="hidden mt-3 overflow-x-auto custom-scroll scroll-smooth snap-x snap-mandatory pb-2">
|
||||
@@ -332,6 +351,7 @@
|
||||
let lastParsedData = null;
|
||||
let activeLightboxUrl = '';
|
||||
let currentVideoUrls = [];
|
||||
let currentCoverUrl = '';
|
||||
let activeVideoIndex = 0;
|
||||
const iconMarkup = (name, className) => `<svg class="${className}" aria-hidden="true"><use href="{{ url_for('static', filename='icons.svg') }}#icon-${name}"></use></svg>`;
|
||||
|
||||
@@ -502,22 +522,58 @@
|
||||
if(url) copyText(url, '已复制当前视频链接');
|
||||
}
|
||||
|
||||
function showVideoFallback(selectedUrl){
|
||||
const videoPlayer = document.getElementById('videoPlayer');
|
||||
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
|
||||
const videoCoverBg = document.getElementById('videoCoverBg');
|
||||
const videoCoverImg = document.getElementById('videoCoverImg');
|
||||
const videoDefaultPlaceholder = document.getElementById('videoDefaultPlaceholder');
|
||||
const videoFallbackPlayBtn = document.getElementById('videoFallbackPlayBtn');
|
||||
|
||||
videoPlayer.pause();
|
||||
videoPlayer.classList.add('hidden');
|
||||
videoFallbackContainer.classList.remove('hidden');
|
||||
|
||||
if(videoFallbackPlayBtn) videoFallbackPlayBtn.href = selectedUrl;
|
||||
|
||||
if(currentCoverUrl){
|
||||
videoDefaultPlaceholder.classList.add('hidden');
|
||||
videoCoverImg.onload = () => {
|
||||
videoCoverImg.classList.remove('hidden');
|
||||
videoCoverBg.classList.remove('hidden');
|
||||
};
|
||||
videoCoverImg.onerror = () => {
|
||||
videoCoverImg.classList.add('hidden');
|
||||
videoCoverBg.classList.add('hidden');
|
||||
videoDefaultPlaceholder.classList.remove('hidden');
|
||||
};
|
||||
videoCoverImg.src = currentCoverUrl;
|
||||
videoCoverBg.src = currentCoverUrl;
|
||||
if(videoCoverImg.complete && videoCoverImg.naturalWidth > 0){
|
||||
videoCoverImg.classList.remove('hidden');
|
||||
videoCoverBg.classList.remove('hidden');
|
||||
}
|
||||
} else {
|
||||
videoCoverImg.classList.add('hidden');
|
||||
videoCoverBg.classList.add('hidden');
|
||||
videoDefaultPlaceholder.classList.remove('hidden');
|
||||
}
|
||||
}
|
||||
|
||||
function selectVideo(index, autoplay = false){
|
||||
if(index < 0 || index >= currentVideoUrls.length) return;
|
||||
activeVideoIndex = index;
|
||||
const videoPlayer = document.getElementById('videoPlayer');
|
||||
const videoPlaceholder = document.getElementById('videoPlaceholder');
|
||||
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
|
||||
const videoOpenDirectLink = document.getElementById('videoOpenDirectLink');
|
||||
const selectedUrl = currentVideoUrls[index];
|
||||
|
||||
videoPlaceholder.classList.add('hidden');
|
||||
videoFallbackContainer.classList.add('hidden');
|
||||
videoPlayer.classList.remove('hidden');
|
||||
videoOpenDirectLink.href = selectedUrl;
|
||||
if(videoOpenDirectLink) videoOpenDirectLink.href = selectedUrl;
|
||||
videoPlayer.onerror = () => {
|
||||
if(currentVideoUrls[activeVideoIndex] !== selectedUrl) return;
|
||||
videoPlayer.pause();
|
||||
videoPlayer.classList.add('hidden');
|
||||
videoPlaceholder.classList.remove('hidden');
|
||||
showVideoFallback(selectedUrl);
|
||||
};
|
||||
videoPlayer.src = selectedUrl;
|
||||
videoPlayer.load();
|
||||
@@ -632,12 +688,16 @@
|
||||
authorAvatar.classList.add('hidden');
|
||||
}
|
||||
|
||||
currentCoverUrl = (typeof data.cover_url === 'string' && data.cover_url.trim())
|
||||
|| (typeof data.cover === 'string' && data.cover.trim())
|
||||
|| (Array.isArray(data.image_list) && data.image_list.length > 0 && typeof data.image_list[0] === 'string' ? data.image_list[0].trim() : '')
|
||||
|| '';
|
||||
currentVideoUrls = uniqueMediaUrls([data.video_url, ...(data.video_list || [])]);
|
||||
|
||||
// 1. 视频渲染
|
||||
const videoContainer = document.getElementById('videoContainer');
|
||||
const videoPlayer = document.getElementById('videoPlayer');
|
||||
const videoPlaceholder = document.getElementById('videoPlaceholder');
|
||||
const videoFallbackContainer = document.getElementById('videoFallbackContainer');
|
||||
const videoStrip = document.getElementById('videoStrip');
|
||||
const videoTrack = document.getElementById('videoTrack');
|
||||
const previousVideoBtn = document.getElementById('previousVideoBtn');
|
||||
@@ -668,7 +728,7 @@
|
||||
videoPlayer.removeAttribute('src');
|
||||
videoPlayer.load();
|
||||
videoPlayer.classList.remove('hidden');
|
||||
videoPlaceholder.classList.add('hidden');
|
||||
videoFallbackContainer.classList.add('hidden');
|
||||
document.getElementById('videoOpenDirectLink').removeAttribute('href');
|
||||
videoStrip.classList.add('hidden');
|
||||
previousVideoBtn.classList.add('hidden');
|
||||
|
||||
@@ -503,29 +503,33 @@ Authorization: Bearer mp_xxxxxxxxx</pre>
|
||||
<table style="width:100%;min-width:920px">
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="width:28%">时间</th>
|
||||
<th>平台</th>
|
||||
<th style="white-space:nowrap;width:160px">时间</th>
|
||||
<th style="white-space:nowrap;width:100px">平台</th>
|
||||
<th>脱敏 URL</th>
|
||||
<th>状态</th>
|
||||
<th>耗时</th>
|
||||
<th>错误码</th>
|
||||
<th style="white-space:nowrap;width:70px">状态</th>
|
||||
<th style="white-space:nowrap;width:90px">耗时</th>
|
||||
<th style="white-space:nowrap;min-width:160px">错误码</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for log in logs %}
|
||||
<tr>
|
||||
<td>{{ format_log_time(log['created_at']) }}</td>
|
||||
<td>{{ log['platform'] or '-' }}</td>
|
||||
<td style="white-space:nowrap">{{ format_log_time(log['created_at']) }}</td>
|
||||
<td style="white-space:nowrap">{{ log['platform'] or '-' }}</td>
|
||||
<td style="max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap">
|
||||
{% if log['input_url'] %}
|
||||
<a href="{{ log['input_url'] }}" target="_blank" rel="noopener noreferrer"
|
||||
title="{{ log['input_url'] }}">{{ log['input_url'] }}</a>
|
||||
{% else %}-{% endif %}
|
||||
</td>
|
||||
<td><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
|
||||
<td style="white-space:nowrap"><span class="badge {{ '' if log['status_code']<400 else 'off' }}">{{ log['status_code']
|
||||
}}</span></td>
|
||||
<td>{{ log['duration_ms'] }} ms</td>
|
||||
<td>{{ log['error_code'] or '-' }}</td>
|
||||
<td style="white-space:nowrap">{{ log['duration_ms'] }} ms</td>
|
||||
<td style="white-space:nowrap">
|
||||
{% if log['error_code'] %}
|
||||
<code style="font-size:12px;font-weight:600;color:#dc2626;background:#fef2f2;padding:2px 6px;border-radius:4px;border:1px solid #fee2e2">{{ log['error_code'] }}</code>
|
||||
{% else %}-{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% else %}
|
||||
<tr>
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"schema_version": 2,
|
||||
"description": "50 平台多形态全景自动化回归测试用例库 (涵盖短链/长链/图集/单视频/AI生成等)",
|
||||
"total_cases": 308,
|
||||
"total_cases": 310,
|
||||
"cases": [
|
||||
{
|
||||
"platform": "网易云音乐",
|
||||
@@ -413,6 +413,23 @@
|
||||
"note": "2026-08-28 已验证",
|
||||
"pattern": "基础标准样例"
|
||||
},
|
||||
{
|
||||
"platform": "小红书",
|
||||
"url": "http://xhslink.com/o/5EY4JTCKXTr",
|
||||
"pattern": "App 分享短链与 LivePhoto 实况图文",
|
||||
"media_types": [
|
||||
"images",
|
||||
"live_media"
|
||||
],
|
||||
"expected_fields": [
|
||||
"images",
|
||||
"live_media",
|
||||
"cover",
|
||||
"title",
|
||||
"author"
|
||||
],
|
||||
"note": "2026-09 已验证: App 端分享短链,含 6 张高清图集与 LivePhoto 实况动图流"
|
||||
},
|
||||
{
|
||||
"platform": "视频号",
|
||||
"url": "https://weixin.qq.com/sph/AzGrUgqzFv",
|
||||
|
||||
@@ -79,6 +79,29 @@ class ManagementTest(unittest.TestCase):
|
||||
self.assertEqual(response.status_code, 302)
|
||||
self.assertTrue(response.location.endswith("/admin"))
|
||||
|
||||
def test_login_remember_me_controls_session_permanence(self):
|
||||
self.client.post(
|
||||
"/auth/setup",
|
||||
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123", "confirm_password": "password123"},
|
||||
)
|
||||
# Without remember_me
|
||||
with self.client:
|
||||
self.client.post(
|
||||
"/auth/login",
|
||||
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123"},
|
||||
)
|
||||
from flask import session
|
||||
self.assertFalse(session.permanent)
|
||||
|
||||
# With remember_me
|
||||
with self.client:
|
||||
self.client.post(
|
||||
"/auth/login",
|
||||
data={"csrf_token": self.csrf(), "username": "admin", "password": "password123", "remember_me": "1"},
|
||||
)
|
||||
from flask import session
|
||||
self.assertTrue(session.permanent)
|
||||
|
||||
def test_setup_rejects_missing_csrf_token(self):
|
||||
response = self.client.post(
|
||||
"/auth/setup",
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import unittest
|
||||
from src.parsers.xiaohongshu_parser import XiaohongshuParser
|
||||
|
||||
|
||||
class XiaohongshuParserTest(unittest.TestCase):
|
||||
def test_extract_note_from_pc_initial_state(self):
|
||||
html = '<script>window.__INITIAL_STATE__ = {"note": {"firstNoteId": "note123", "noteDetailMap": {"note123": {"note": {"title": "PC\u6807\u9898", "desc": "PC\u63cf\u8ff0", "user": {"nickname": "PC\u7528\u6237", "userId": "u123", "avatar": "http://avatar.jpg"}, "imageList": [{"urlDefault": "http://img1.jpg"}]}}}}}</script>'
|
||||
parser = XiaohongshuParser.__new__(XiaohongshuParser)
|
||||
note = parser._extract_note_from_html(html)
|
||||
self.assertIsNotNone(note)
|
||||
self.assertEqual(note.get("title"), "PC标题")
|
||||
self.assertEqual(note.get("desc"), "PC描述")
|
||||
parser.note_data = note
|
||||
self.assertEqual(parser.get_author_info()["nickname"], "PC用户")
|
||||
self.assertEqual(parser.get_cover_photo_url(), "http://img1.jpg")
|
||||
|
||||
def test_extract_note_from_mobile_initial_state(self):
|
||||
html = '<script>window.__INITIAL_STATE__ = {"noteData": {"data": {"noteData": {"title": "\u79fb\u52a8\u7aef\u6807\u9898", "desc": "\u79fb\u52a8\u7aef\u63cf\u8ff0", "user": {"nickName": "\u79fb\u52a8\u7aef\u7528\u6237", "userId": "u456", "avatar": "http://m_avatar.jpg"}, "imageList": [{"url": "http://m_img1.jpg", "livePhoto": true, "stream": {"h264": [{"masterUrl": "http://live.mp4"}]}}]}}}}</script>'
|
||||
parser = XiaohongshuParser.__new__(XiaohongshuParser)
|
||||
note = parser._extract_note_from_html(html)
|
||||
self.assertIsNotNone(note)
|
||||
self.assertEqual(note.get("title"), "移动端标题")
|
||||
parser.note_data = note
|
||||
self.assertEqual(parser.get_author_info()["nickname"], "移动端用户")
|
||||
self.assertEqual(parser.get_cover_photo_url(), "http://m_img1.jpg")
|
||||
images = parser.get_image_list()
|
||||
self.assertEqual(len(images), 1)
|
||||
self.assertEqual(images[0]["live_photo_url"], "http://live.mp4")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -172,9 +172,13 @@ class UrlParser:
|
||||
address = f"{address}?id={vid}"
|
||||
elif platform == "小红书":
|
||||
query_params = parse_qs(parsed_url.query)
|
||||
xsec_token = query_params.get('xsec_token', [None])[0] # 使用 get 方法避免 KeyError
|
||||
if xsec_token:
|
||||
address = f"{address}?xsec_token={xsec_token}"
|
||||
preserved_params = []
|
||||
for key in ('xsec_token', 'xsec_source', 'source', 'xhsshare', 'app_platform'):
|
||||
value = query_params.get(key, [None])[0]
|
||||
if value is not None:
|
||||
preserved_params.append((key, value))
|
||||
if preserved_params:
|
||||
address = f"{address}?{urlencode(preserved_params)}"
|
||||
elif platform == "快手":
|
||||
address = address.replace('http://', 'https://')
|
||||
elif platform == "抖音":
|
||||
|
||||
Reference in New Issue
Block a user