增强小云雀AI微短剧与灵感作品解析兼容性

This commit is contained in:
ucmao
2026-09-14 20:10:12 +08:00
parent 9116452f6a
commit 6b7d31025d
7 changed files with 244 additions and 30 deletions
+6 -6
View File
@@ -46,13 +46,13 @@ Media-Parser是一款专为短视频创作者与开发者打造的**100%原生
| **快手** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | ✓ | | [查看](docs/parsers/kuaishou.md) |
| **哔哩哔哩** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | [查看](docs/parsers/bilibili.md) |
| **豆包** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | [查看](docs/parsers/doubao.md) |
| **即梦AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | [查看](docs/parsers/jimeng.md) |
| **小云雀AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/xiaoyunque.md) |
| **即梦AI** | ✓ | ✓ | ✓ | ✓ | ✓* | ✓ | ✓ | | | | [查看](docs/parsers/jimeng.md) |
| **小云雀AI** | ✓ | ✓ | ✓ | ✓ | ✓* | ✓ | | | | | [查看](docs/parsers/xiaoyunque.md) |
| **可灵AI** | ✓ | ✓ | | ✓ | ✓ | | | | | | [查看](docs/parsers/kling.md) |
| **海螺AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/hailuo.md) |
| **夸克AI** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/quark-ai.md) |
| **通义千问** | ✓ | ✓ | | ✓ | ✓ | ✓ | ✓ | | | | [查看](docs/parsers/qianwen.md) |
| **腾讯元宝** | ✓ | ✓ | | ✓ | ✓ | ✓ | ✓ | | | | [查看](docs/parsers/yuanbao.md) |
| **腾讯元宝** | ✓ | ✓ | | ✓ | ✓* | ✓ | ✓ | | | | [查看](docs/parsers/yuanbao.md) |
| **闲鱼** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/xianyu.md) |
| **拼多多** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | ✓ | | [查看](docs/parsers/pinduoduo.md) |
| **Soul** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/soul.md) |
@@ -90,7 +90,7 @@ Media-Parser是一款专为短视频创作者与开发者打造的**100%原生
| **央视** | ✓ | ✓ | | ✓ | ✓ | | | | | | [查看](docs/parsers/cctv.md) |
| **央视频** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | [查看](docs/parsers/yangshipin.md) |
*注:腾讯元宝等极少数平台提取的是官方存储桶原画质直链,素材保留官方原生水印。
注:带 `*` 的项表示支持提取官方最高画质直链,但受平台生成或导出机制限制,该素材目前仍保留官方原生水印;未带 `*` 的项均为无水印素材。
---
@@ -235,10 +235,10 @@ http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/
## 🧪 自动化测试与健康自检
本项目拥有完备的双层测试体系(Mock 单元测试 + 50 平台真实在线样本库回归)。遇到解析异常或日常部署验证时,可一键运行健康自检:
本项目拥有完备的双层测试体系(Mock 单元测试 + 基于 [`tests/live_parser_samples.json`](tests/live_parser_samples.json) 的 300+ 条真实在线样本库回归)。遇到解析异常或日常部署验证时,可一键运行50 平台健康自检:
```bash
# 50 平台极速冒烟测试(每个平台测 1 条最具代表性的链接,秒级完成健康检查)
# 50 平台极速冒烟测试(读取样本库,每个平台测 1 条最具代表性的链接,秒级完成)
python3 tests/manual_verify_parsers.py --limit 1
# 仅验证单个或指定平台(如:小云雀AI / 抖音)
+5
View File
@@ -17,6 +17,11 @@
* 移动社区详情长链:`https://jimeng.jianying.com/ai-tool/share/item/7631885529415568665`
* 同款/草稿/生成记录回流长链:`https://jimeng.jianying.com/activities/reflux/mproject?id=7683715084807376153&search_keyword=...`
> [!NOTE]
> **水印与素材说明**:
> 1. **AI 生图图片**:提取的是官方最高清晰度原图(最高 4096 分辨率),**完全无水印**。
> 2. **AI 生成视频**:官方在生成视频时自带水印。解析器会优先选择仅在片尾带 logo 的版本(主体画面无右上角常驻水印)。
---
## 2. 核心架构与原生双轨解析流程
+2
View File
@@ -56,6 +56,8 @@ flowchart TD
* 新版 `qianwen.my.cn/share/chat/` 页面为纯客户端渲染,HTML 中无 `__INITIAL_PROPS__`,直接正则提取 HTML 会失败。必须调用 `chat2-api` 才能获取真实数据。
2. **`initialData` 双重编码**:
* 老版前端模板将 `initialData` 做了 `encodeURIComponent`,在 `json.loads` 之前必须进行 `urllib.parse.unquote`。
3. **无水印原图 vs 带水印导出图 (`url` vs `downloadUrl`)**:
* 在 `__INITIAL_PROPS__` 的 `images` / `image` 节点中,`url` 对应模型生成的 1024x1024 无水印原图,而 `downloadUrl` 对应网页端带右下角通义标识的导出图。解析器必须优先提取 `url`,以确保输出无水印纯净资源。
---
+96 -19
View File
@@ -1,6 +1,6 @@
# 小云雀 AI (Xiaoyunque) 逆向解析指南
本篇详细记录字节跳动剪映旗下 **小云雀 AI (xiaoyunque.jianying.com)** 分享作品的接口抓取机制与参数解析。
本篇详细记录字节跳动剪映旗下 **小云雀 AI (xiaoyunque.jianying.com / xyq.jianying.com)** 作品的接口抓取机制、底层 VOD 视频分发流、水印压制机制与双轨逆向解析算法。
---
@@ -8,38 +8,115 @@
* **平台标识**:`小云雀AI`
* **支持媒体类型**:
* AI 创作图集 (PNG/JPEG)
* AI 创作高清视频 (MP4)
* 提示词 Prompt、封面与作者信息
* **AI 生图 / 创作图集** (PNG/JPEG,100% 官方原始纯净无水印原图)
* **AI 创作高清视频** (MP4,最高 9.4 Mbps 原画,支持去除左上角 AI 标识)
* **提示词文案** (完整的 Prompt 提示词、标题、封面及创作者元数据)
* **常见链接形态**:
* 分享短链:`https://xiaoyunque.jianying.com/s/z_7nWGLGruM/`
* 灵感分享长链:`https://xiaoyunque.jianying.com/activities/pippit_share?inspiration_id=...`
* **Cookie 依赖**:公开短链**无需配置 Cookie**。
* 分享短链:`https://xiaoyunque.jianying.com/s/ebmxLzUc87I/`
* 落地页长链:`https://xiaoyunque.jianying.com/activities/pippit_share?inspiration_id=7647057288208977177&...`
* **Cookie 依赖**:公开短链**完全无需 Cookie** 即可匿名解析;配置 `XIAOYUNQUE_COOKIE` 可激活创作者超清去标通道。
> [!NOTE]
> **水印与素材状态说明**:
> 1. **AI 生图 / 图集**:从字节 TOS 存储桶中直接提取官方原始图(`origin_aigc`),**100% 完全无水印**。
> 2. **AI 生成视频**:平台在云端渲染合成时直接把水印压制在 MP4 画面像素中。解析器配置 Cookie 时自动优先提取已去除左上角「AI生成」黑框的 **9.4 Mbps 创作者原画流**(仅保留右下角小云雀微标);公开免登录模式下自动回退至 H5 落地页通道。
---
## 2. 核心逆向流程
## 2. 核心架构与双轨自适应解析流程
### 2.1 短链解析与落地页跳转
通过发起 GET 请求跟踪 302 跳转,从小云雀长链接中提取 Query 参数字典 `qdict`。
```mermaid
sequenceDiagram
autonumber
participant Client as 调用方
participant Parser as XiaoyunqueParser
participant Redirect as 302 重定向服务
participant BizAPI as 创作者详情接口 (get_inspiration_detail)
participant LandingAPI as 公开落地页 (landing_page)
### 2.2 核心数据接口
Client->>Parser: 输入分享短链 (如 /s/ebmxLzUc87I/)
Parser->>Redirect: HTTP GET 获取 302 Location
Redirect-->>Parser: 提取 query_params (含 inspiration_id, share_id 等)
alt 已配置 XIAOYUNQUE_COOKIE
Parser->>BizAPI: POST /api/biz/v1/inspiration/get_inspiration_detail
BizAPI-->>Parser: 返回 watermark_video_info
Note over Parser: 优先提取 brand_user_url (9.4 Mbps,无左上角 AI 标)
else 未配置 Cookie / 创作者接口请求失败
Parser->>LandingAPI: POST /luckycat/cn/jianying/campaign/v1/pippit/share/landing_page
LandingAPI-->>Parser: 返回 page_info (兼容 inspiration / generate / gugu 等模板)
end
Parser->>Client: 统一返回标准 JSON (视频、图集、Prompt、作者元数据)
```
---
## 3. 视频流类型与底层逆向分析
小云雀视频采用字节跳动 VOD (Video On Demand) 视频云分发体系,服务端在完成视频生成后会产出多路物理文件:
| 视频流字段 / 类型 | 码率 / 分辨率 | 水印状态 | 解析器处理策略 |
| :--- | :---: | :--- | :--- |
| **`brand_user_url`** (创作者流) | **9.4 Mbps (720P/1080P)** | **无左上角「AI生成」黑框**,仅保留右下角微标 | **Cookie 模式下作为默认主视频 (`video_url`) 输出** |
| **`share_url`** (分享流) | ~9.4 Mbps | 左上角带「AI生成」框,右下角无标 | 备选流,放入 `video_list` |
| **`download_url` / `brand_url`** | ~9.4 Mbps | 左上角「AI生成」+ 右下角「小云雀」双水印 | 备选流,放入 `video_list` |
| **`ai_url`** (预览流) | ~6.4 Mbps (720P) | 带官方基础预览水印 | 免登录兜底流 |
| **母带原始渲染流** (如 `7cef...`) | 原画 | 100% 纯净(无任何角标) | **受作者账号 UID 权限保护**(见下文说明) |
### 底层母带权限边界分析 (UID Security Boundary)
通过对 Web 前端源码(`main.e10fbb27b2.js`)及后端 RPC 服务逆向发现:
* 原始 100% 纯净母带渲染任务(`videogen/batch_get_render_task` / `get_published_inspiration_detail`)在后端 RPC 微服务(`pippit.business.editor`)层面进行了严格的用户归属鉴权:
```json
{"ret":"10007","errmsg":"RPCError{PSM:[pippit.business.editor] Method:[CheckRenderTask] BizStatusMessage:[uid not equal]}"}
{"ret":"2","errmsg":"uid not own inspiration"}
```
* **结论**:普通访客或跨账号请求无法直接获取母带播放签名。解析器通过 `get_inspiration_detail` 提取的 `brand_user_url` 是当前非作者本号跨端获取的最佳清晰度与最少水印版本。
---
## 4. 接口规范与参数清洗
### 4.1 创作者灵感详情接口(优先,需 Cookie)
* **接口地址**:
`POST https://xyq.jianying.com/api/biz/v1/inspiration/get_inspiration_detail`
* **请求头**:
```http
Accept: application/json, text/plain, */*
Content-Type: application/json
Appid: 795647
pf: 7
appvr: 5.8.0
Cookie: <XIAOYUNQUE_COOKIE>
```
* **请求体**:
```json
{
"inspiration_id_list": ["7647057288208977177"]
}
```
* **核心字段提取**:
* `data.inspiration_list[0].prompt`:完整 Prompt 提示词。
* `data.inspiration_list[0].watermark_video_info.brand_user_url`:无左上角 AI 标原画视频。
* `data.inspiration_list[0].watermark_image_info`:高清图片。
### 4.2 公开落地页接口(兜底,免 Cookie)
* **接口地址**:
`POST https://xiaoyunque.jianying.com/luckycat/cn/jianying/campaign/v1/pippit/share/landing_page`
* **请求头**:
* `User-Agent`: 现代浏览器 UA
* `Accept`: `application/json, text/plain, */*`
* `Content-Type`: `application/json`
* **提取数据**:
* 解析返回的 `data.page_info`,兼容 `generate_page`(生成页)、`inspiration_page`(灵感创作/壁纸)与 `template_page`(模板页);
* 从 `item_info` 中提取列表型 `video_info` 视频直链、`image_info` 高清图集及创作者(`nick_name`, `avatar_url`)元数据。
* **模板多态兼容**:
* `inspiration_page`:灵感与同款生成;
* `generate_page`:生图与生视频页面;
* `template_page`:官方创意模板;
* `gugu_page`:接龙片单与短剧播放列表(按 `current_index` 索引)。
* **图集无水印提取**:从 `item_info.image_info[].image_url` 中直接获取 `~tplv-wipe09lpyt-origin_aigc.image` 原图。
---
## 3. 测试与验证
## 5. 测试与验证
* **单元测试**:[tests/test_xiaoyunque_parser.py](file:///Users/leo/Projects/media-parser/tests/test_xiaoyunque_parser.py)
* **执行测试**:
```bash
python -m unittest tests/test_xiaoyunque_parser.py
python -m unittest discover tests/
```
+92 -5
View File
@@ -5,6 +5,7 @@ from urllib.parse import parse_qs, urlparse
from configs.logging_config import get_logger
from src.parsers.base_parser import BaseParser
from src.utils.cookie_manager import get_platform_cookie
logger = get_logger(__name__)
@@ -15,6 +16,7 @@ class XiaoyunqueParser(BaseParser):
"""通过小云雀官方接口解析 xiaoyunque.jianying.com 分享链接。"""
API_URL = "https://xiaoyunque.jianying.com/luckycat/cn/jianying/campaign/v1/pippit/share/landing_page"
BIZ_DETAIL_API = "https://xyq.jianying.com/api/biz/v1/inspiration/get_inspiration_detail"
USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
@@ -23,11 +25,14 @@ class XiaoyunqueParser(BaseParser):
def __init__(self, real_url):
super().__init__(real_url)
self.cookie = get_platform_cookie("xiaoyunque")
self.headers = {
"Accept": "application/json, text/plain, */*",
"Content-Type": "application/json",
"User-Agent": self.USER_AGENT,
}
if self.cookie:
self.headers["Cookie"] = self.cookie
self.data = {
"title": "",
"desc": None,
@@ -61,6 +66,28 @@ class XiaoyunqueParser(BaseParser):
logger.warning(f"Unable to extract Xiaoyunque query parameters from URL: {self.real_url}")
return
# 1. 若配置了 Cookie,优先尝试请求高码率/下载流接口
inspiration_id = qdict.get("inspiration_id") or qdict.get("template_id")
if self.cookie and inspiration_id:
try:
biz_resp = self.session.post(
self.BIZ_DETAIL_API,
headers=self.headers,
json={"inspiration_id_list": [str(inspiration_id)]},
timeout=15,
)
if biz_resp.status_code == 200:
biz_data = biz_resp.json().get("data") or {}
insp_list = biz_data.get("inspiration_list") or biz_data.get("inspirations") or []
if insp_list and isinstance(insp_list[0], dict):
formatted_biz = self._format_biz_data(insp_list[0])
if formatted_biz.get("video_url") or formatted_biz.get("image_list"):
self.data.update(formatted_biz)
return
except Exception as biz_exc:
logger.debug(f"Xiaoyunque biz detail query failed, falling back to landing_page: {biz_exc}")
# 2. 公开免登录 H5 landing_page 接口兜底
response = self.session.post(
self.API_URL,
headers=self.headers,
@@ -76,24 +103,77 @@ class XiaoyunqueParser(BaseParser):
except Exception as exc:
logger.exception(f"Failed to parse Xiaoyunque share: {exc}")
@classmethod
def _format_biz_data(cls, insp):
author_info = insp.get("author_info") or {}
wm_video = insp.get("watermark_video_info") or {}
primary_video = (
wm_video.get("brand_user_url")
or wm_video.get("download_url")
or wm_video.get("brand_url")
or wm_video.get("share_url")
or wm_video.get("ai_url")
)
video_list = []
for key in ("brand_user_url", "download_url", "brand_url", "share_url", "ai_url"):
val = wm_video.get(key)
if val and val not in video_list:
video_list.append(val)
title = insp.get("title") or insp.get("prompt")
desc = insp.get("prompt") or insp.get("title")
# 图片信息
wm_image = insp.get("watermark_image_info") or {}
image_list = []
for key in ("download_url", "brand_url", "share_url", "ai_url", "origin_url"):
val = wm_image.get(key)
if val and val not in image_list:
image_list.append(val)
cover_info = insp.get("cover_info") or {}
cover_url = cover_info.get("cover_image_url") or insp.get("cover_url") or insp.get("cover")
return {
"title": title,
"desc": desc,
"video_url": primary_video,
"video_list": video_list,
"cover_url": cover_url,
"author": {
"nickname": author_info.get("nickName") or author_info.get("nickname") or "",
"author_id": str(author_info.get("id") or insp.get("uid") or ""),
"avatar": author_info.get("avatar") or author_info.get("avatar_url") or "",
},
"image_list": image_list,
}
@classmethod
def _format_data(cls, data):
page_info = data.get("page_info") or {}
# 寻找命中的有效页面节点(兼容 generate_page, inspiration_page, template_page 等)
# 寻找命中的有效页面节点(兼容 generate_page, inspiration_page, template_page, gugu_page 等)
target_page = {}
for key in ("generate_page", "inspiration_page", "template_page", "share_page"):
for key in ("generate_page", "inspiration_page", "template_page", "share_page", "gugu_page"):
if isinstance(page_info.get(key), dict):
target_page = page_info[key]
break
if not target_page:
for val in page_info.values():
if isinstance(val, dict) and ("item_info" in val or "user_info" in val):
if isinstance(val, dict) and ("item_info" in val or "user_info" in val or "item_list" in val):
target_page = val
break
user_info = target_page.get("user_info") or {}
item_info = target_page.get("item_info") or {}
item_list = target_page.get("item_list")
if isinstance(item_list, list) and item_list:
curr_idx = target_page.get("current_index")
if not isinstance(curr_idx, int) or curr_idx < 0 or curr_idx >= len(item_list):
curr_idx = 0
item_info = item_list[curr_idx]
user_info = item_info.get("author_info") or item_info.get("user_info") or target_page.get("user_info") or {}
else:
user_info = target_page.get("user_info") or target_page.get("author_info") or {}
item_info = target_page.get("item_info") or {}
title = item_info.get("title") or None
desc = item_info.get("desc") or None
@@ -137,6 +217,13 @@ class XiaoyunqueParser(BaseParser):
# 封面图
cover_url = item_info.get("cover_url") or item_info.get("poster") or item_info.get("cover")
if not cover_url and isinstance(video_info, list):
for v in video_info:
if isinstance(v, dict) and (v.get("cover_url") or v.get("poster") or v.get("cover")):
cover_url = v.get("cover_url") or v.get("poster") or v.get("cover")
break
elif not cover_url and isinstance(video_info, dict):
cover_url = video_info.get("cover_url") or video_info.get("poster") or video_info.get("cover")
author = {
"nickname": user_info.get("nick_name") or user_info.get("nickname") or "",
+2
View File
@@ -13,6 +13,8 @@ PLATFORM_COOKIE_ALIASES = {
"bilibili": ["BILIBILI_COOKIE", "BILI_COOKIE"],
"weibo": ["WEIBO_COOKIE"],
"kuaishou": ["KUAISHOU_COOKIE", "KS_COOKIE"],
"xiaoyunque": ["XIAOYUNQUE_COOKIE", "XYQ_COOKIE", "PIPPIT_COOKIE"],
"xyq": ["XIAOYUNQUE_COOKIE", "XYQ_COOKIE", "PIPPIT_COOKIE"],
}
+41
View File
@@ -78,6 +78,47 @@ class XiaoyunqueParserTest(unittest.TestCase):
self.assertEqual(parser.get_real_video_url(), "https://v11-xyq-video.jianying.com/video.mp4")
self.assertEqual(parser.get_author_info()["nickname"], "茉莱")
def test_gugu_page_response(self):
response = Mock()
response.raise_for_status.return_value = None
response.json.return_value = {
"err_no": 0,
"err_tips": "success",
"data": {
"page_info": {
"gugu_page": {
"current_index": 0,
"item_list": [
{
"title": "F级废物给鬼号脉",
"desc": "首映剧情",
"author_info": {
"nick_name": "浅酱雨禾",
"avatar_url": "https://avatar.example.com/qian.jpg",
},
"video_info": [
{
"video_url": "https://v11-xyq-video.jianying.com/gugu.mp4",
"cover_url": "https://cover.example.com/gugu.jpg",
}
],
}
],
}
}
},
}
url = "https://xiaoyunque.jianying.com/activities/pippit_share?gugu_id=12345"
with patch("requests.Session.post", return_value=response):
parser = XiaoyunqueParser(url)
self.assertEqual(parser.get_title_content(), "F级废物给鬼号脉")
self.assertEqual(parser.get_description(), "首映剧情")
self.assertEqual(parser.get_real_video_url(), "https://v11-xyq-video.jianying.com/gugu.mp4")
self.assertEqual(parser.get_cover_photo_url(), "https://cover.example.com/gugu.jpg")
self.assertEqual(parser.get_author_info()["nickname"], "浅酱雨禾")
def test_url_parser_recognizes_xiaoyunque(self):
url = "https://xiaoyunque.jianying.com/s/z_7nWGLGruM/"
self.assertEqual(UrlParser.get_platform(url), "小云雀AI")