mirror of
https://github.com/ucmao/media-parser.git
synced 2026-11-20 08:04:56 +08:00
添加网易云音乐平台解析支持
This commit is contained in:
@@ -3,7 +3,7 @@
|
||||
|
||||
**基于 Python 的多平台媒体原生本地解析系统**
|
||||
|
||||
[](LICENSE) [](https://www.python.org/) [](https://flask.palletsprojects.com/) [](#-部署指南) [](#-支持的平台矩阵)
|
||||
[](LICENSE) [](https://www.python.org/) [](https://flask.palletsprojects.com/) [](#-部署指南) [](#-支持的平台矩阵)
|
||||
|
||||
<p align="center">
|
||||
<a href="#-核心解析逻辑">解析逻辑</a> •
|
||||
@@ -63,6 +63,7 @@
|
||||
| **Soul** | ✓ | ✓ | ✓ | ✓ | ✓ | | | |
|
||||
| **汽水音乐** | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | |
|
||||
| **QQ音乐** | ✓ | ✓ | ✓ | ✓ | | | | |
|
||||
| **网易云音乐** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
|
||||
| **腾讯频道** | ✓ | ✓ | ✓ | ✓ | | | | |
|
||||
| **剪映 / CapCut** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | |
|
||||
| **快影** | ✓ | ✓ | ✓ | ✓ | | ✓ | | |
|
||||
@@ -94,7 +95,7 @@
|
||||
|
||||
### Cookie 配置(Docker 与 Python 环境运行通用)
|
||||
|
||||
大多数平台无需登录态即可解析;如需按需增强部分特定功能(如抖音放映厅长片、豆包无水印视频、视频号等),请先复制环境变量示例文件:
|
||||
大多数平台无需登录态即可解析;如需按需增强部分特定功能(如豆包无水印视频、微信视频号等),请先复制环境变量示例文件:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
@@ -104,7 +105,7 @@ cp .env.example .env
|
||||
|
||||
1. **抖音放映厅长视频 (`DOUYIN_COOKIE`)**:
|
||||
* 仅在解析放映厅/影视长片/演唱会大片等强风控内容时才需要配置。推荐精简 Cookie 配置 `s_v_web_id=xxx; __ac_nonce=xxx`,该字段**不是个人账号登录信息**,仅为字节安全 SDK 的人机风控通行证。日常 99% 的普通短视频、图文笔记、LivePhoto、原声音乐等**完全免 Cookie 匿名解析**。
|
||||
2. **豆包视频 (`DOUBAO_COOKIE`)**:
|
||||
2. **豆包无水印视频 (`DOUBAO_COOKIE`)**:
|
||||
* 用于解密获取 1080P 原始纯净无水印视频。由于该字段包含豆包的个人账号登录会话凭证,**建议使用闲置小号**。推荐精简 Cookie 配置 `sessionid_ss=xxx`。未配置时仍可解析公开图片与带水印预览切片。
|
||||
3. **微信视频号视频 (`YUANBAO_COOKIE`)**:
|
||||
* 通过腾讯元宝接口提取视频号原始流。由于该接口绑定了腾讯元宝的账号会话(属于个人账号登录凭证),**建议使用闲置小号**。推荐精简 Cookie 配置 `hy_user=xxx; hy_token=xxx`。未配置时仅可获取除视频外的其他信息。
|
||||
@@ -153,17 +154,17 @@ media-parser/
|
||||
│ ├── architecture.md # 系统分层架构与生命周期设计
|
||||
│ ├── reverse-guide.md # 通用逆向方法论 (抓包/SSR/JS签名提取)
|
||||
│ ├── testing.md # 完整测试规范与回归手册
|
||||
│ └── parsers/ # 38 份逆向分析文档,覆盖 41 个平台
|
||||
│ └── parsers/ # 39 份逆向分析文档,覆盖 42 个平台
|
||||
├── src/ # 核心业务逻辑
|
||||
│ ├── api/ # RESTful API 路由 (/api/parse, /api/health)
|
||||
│ ├── web/ # Demo 体验页与交互蓝图
|
||||
│ ├── parsers/ # 38 个解析器模块,支持 41 个平台 (核心解析逻辑)
|
||||
│ ├── parsers/ # 39 个解析器模块,支持 42 个平台 (核心解析逻辑)
|
||||
│ └── parser_factory.py # 工厂分发器 (解析器动态发现与自动注册)
|
||||
├── utils/ # 底层工具库与逆向支持
|
||||
│ ├── signer/ # JS 签名沙箱引擎 (a_bogus, x_bogus 等算法执行)
|
||||
│ └── web_fetcher.py # 智能 URL 识别、重定向追踪与请求封装
|
||||
├── tests/ # 完备的双层测试体系
|
||||
│ ├── live_parser_samples.json # 41 平台真实多形态在线样本库
|
||||
│ ├── live_parser_samples.json # 42 平台真实多形态在线样本库
|
||||
│ ├── manual_verify_parsers.py # 命令行交互式冒烟与健康检查工具
|
||||
│ └── test_*_parser.py # 各平台 Mock 自动化单元测试
|
||||
├── static/ & templates/ # Web 演示页面前端静态资源
|
||||
@@ -177,7 +178,7 @@ media-parser/
|
||||
* 🏗️ **[系统架构与生命周期设计](docs/architecture.md)**:分层设计、302 追踪与 `ParserFactory` 动态发现机制。
|
||||
* 🔍 **[通用逆向方法论](docs/reverse-guide.md)**:SSR 数据提取、H5 接口伪装、JS 签名沙箱及抓包 SOP。
|
||||
* 🧪 **[测试体系与回归验证](docs/testing.md)**:Pytest 单元测试、Mock 与线上真实样例回归。
|
||||
* 📚 **[平台逆向指南索引](docs/index.md)**:包含抖音、快手、小红书、B站等全部 41 个平台的技术文档。
|
||||
* 📚 **[平台逆向指南索引](docs/index.md)**:包含抖音、快手、小红书、B站等全部 42 个平台的技术文档。
|
||||
|
||||
---
|
||||
|
||||
@@ -186,7 +187,7 @@ media-parser/
|
||||
本项目拥有完备的测试体系,包括基础单元测试与基于真实样本库的在线回归测试:
|
||||
|
||||
### 1. 真实样本交互式验证
|
||||
样例测试库见 [`tests/live_parser_samples.json`](tests/live_parser_samples.json),用于实时验证 41 个平台的解析有效性与多形态覆盖:
|
||||
样例测试库见 [`tests/live_parser_samples.json`](tests/live_parser_samples.json),用于实时验证 42 个平台的解析有效性与多形态覆盖:
|
||||
|
||||
```bash
|
||||
# 快速冒烟测试(每个平台测 1 条最具代表性的链接,极速完成健康检查)
|
||||
|
||||
@@ -88,6 +88,11 @@
|
||||
"i2.y.qq.com": "QQ音乐",
|
||||
"y.qq.com": "QQ音乐",
|
||||
"www.y.qq.com": "QQ音乐",
|
||||
"163cn.tv": "网易云音乐",
|
||||
"music.163.com": "网易云音乐",
|
||||
"www.music.163.com": "网易云音乐",
|
||||
"y.music.163.com": "网易云音乐",
|
||||
"fn.music.163.com": "网易云音乐",
|
||||
"pd.qq.com": "腾讯频道",
|
||||
"lv.ulikecam.com": "剪映",
|
||||
"www.capcut.cn": "剪映",
|
||||
|
||||
+3
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
欢迎查阅 **Media Parser** 开发者文档与逆向工程知识库。
|
||||
|
||||
本项目是一个高性能、模块化、支持 **41 个主流媒体与 AI 内容平台** 的无水印音视频、图文及 LivePhoto 结构化解析引擎。
|
||||
本项目是一个高性能、模块化、支持 **42 个主流媒体与 AI 内容平台** 的无水印音视频、图文及 LivePhoto 结构化解析引擎。
|
||||
|
||||
---
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
* **图文与综合社区**:[小红书](parsers/xiaohongshu.md) | [微信公众号](parsers/wechat-mp.md) | [微博](parsers/weibo.md) | [知乎](parsers/zhihu.md) | [闲鱼](parsers/xianyu.md) | [Soul](parsers/soul.md)
|
||||
* **长视频与弹幕/创作**:[哔哩哔哩](parsers/bilibili.md) | [AcFun](parsers/acfun.md) | [新片场](parsers/xinpianchang.md) | [好看视频](parsers/haokan.md) | [西瓜视频](parsers/xigua.md) | [剪映](parsers/jianying.md) | [快影](parsers/kwaiying.md)
|
||||
* **AI 生成与大模型**:[豆包 AI](parsers/doubao.md) | [腾讯元宝](parsers/yuanbao.md) | [即梦 AI](parsers/jimeng.md) | [可灵 AI](parsers/kling.md) | [海螺 AI](parsers/hailuo.md) | [通义千问](parsers/qianwen.md) | [夸克 AI](parsers/quark-ai.md) | [小云雀 AI](parsers/xiaoyunque.md)
|
||||
* **音频与垂直频道**:[汽水音乐](parsers/qsmusic.md) | [QQ音乐](parsers/qqmusic.md) | [全民K歌](parsers/quanminkge.md) | [虎牙](parsers/huya.md) | [梨视频](parsers/lishipin.md) | [微信视频号](parsers/wechat-channels.md) | [腾讯频道](parsers/tencent-channel.md)
|
||||
* **音频与垂直频道**:[汽水音乐](parsers/qsmusic.md) | [QQ音乐](parsers/qqmusic.md) | [网易云音乐](parsers/netease-music.md) | [全民K歌](parsers/quanminkge.md) | [虎牙](parsers/huya.md) | [梨视频](parsers/lishipin.md) | [微信视频号](parsers/wechat-channels.md) | [腾讯频道](parsers/tencent-channel.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -70,6 +70,7 @@
|
||||
| 39 | **红果短剧** | 短剧视频 | ✅ 支持 | 🟢 免配置 | H5 推广页 HTML 提取 | [查看指南](parsers/fanqie.md) |
|
||||
| 40 | **红果漫剧** | 漫剧/短剧视频 | ✅ 支持 | 🟢 免配置 | H5 推广页 HTML 提取 | [查看指南](parsers/fanqie.md) |
|
||||
| 41 | **QQ音乐** | MV / 分享视频 / 多档 MP4 | ✅ 支持 | 🟢 免配置 | SSR 元数据 + MusicU 播放接口 | [查看指南](parsers/qqmusic.md) |
|
||||
| 42 | **网易云音乐** | MV / Mlog / 歌曲 / 动态图集与 LivePhoto | ✅ 支持 | 🟢 公开内容免配置 | SSR / Event 数据 + 公开播放接口 | [查看指南](parsers/netease-music.md) |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# 网易云音乐解析指南
|
||||
|
||||
## 支持范围
|
||||
|
||||
- `163cn.tv` 分享短链
|
||||
- `fn.music.163.com` 的 MV 与 Mlog 分享页
|
||||
- `music.163.com/mv?id=...` 与 `music.163.com/song?id=...`
|
||||
- `music.163.com/event?id=...` 和 `y.music.163.com/m/event?id=...`
|
||||
- MV / 动态视频多清晰度、免费歌曲完整音频、歌词、图集与 LivePhoto
|
||||
|
||||
## 分类解析
|
||||
|
||||
### MV
|
||||
|
||||
通过公开的 MV 详情接口获取标题、封面、歌手及带时效签名的 `brs` 播放地址,按 1080P、720P、480P 等清晰度降序输出。
|
||||
|
||||
### Mlog 动态视频
|
||||
|
||||
读取分享页的 `window.__INITIAL_PROPS__`,从 `mlogInfo.resource.content.video.urlInfos` 获取多档签名视频,并提取发布者、封面和正文。
|
||||
|
||||
### Event 动态
|
||||
|
||||
旧版 Event 分享需要访问桌面落地页并解析 `event-data`:
|
||||
|
||||
- `pics[].originUrl` 作为原图;
|
||||
- `videoOriginalUrl` / `videoUrl` 作为 LivePhoto 动态部分;
|
||||
- 关联免费歌曲继续请求完整音频与 LRC 歌词;
|
||||
- Event 内的视频资源进入 `video_url` / `video_list`。
|
||||
|
||||
### 普通歌曲
|
||||
|
||||
歌曲详情通过公开接口获取;只有接口明确返回完整播放 URL 时才填写 `audio_url`。会员、付费、下架或区域受限歌曲不会把试听片段伪装成完整音频。
|
||||
|
||||
## 返回字段
|
||||
|
||||
| 字段 | 内容 |
|
||||
| --- | --- |
|
||||
| `video_id` | MV、Mlog、Event 或歌曲 ID |
|
||||
| `title` | MV 名、动态标题或歌曲名 |
|
||||
| `author` | 歌手或动态发布者 |
|
||||
| `cover_url` | MV 封面、视频帧、专辑图或动态首图 |
|
||||
| `video_url` | 最高可用清晰度视频 |
|
||||
| `video_list` | 多档视频地址 |
|
||||
| `audio_url` | 可公开播放的完整歌曲音频 |
|
||||
| `subtitles` | LRC 转换后的时间轴歌词 |
|
||||
| `image_list` | 动态原图及可选 `live_photo_url` |
|
||||
|
||||
## 限制
|
||||
|
||||
- 音频和视频 URL 带短期签名,应在解析后及时使用。
|
||||
- 会员、付费、下架、私密或区域受限内容可能仅返回元数据。
|
||||
- 已删除的旧 Event 无法恢复其媒体内容。
|
||||
|
||||
## 验证
|
||||
|
||||
```bash
|
||||
python3 -m unittest tests.test_netease_music_parser
|
||||
python3 tests/manual_verify_parsers.py --platform "网易云音乐"
|
||||
```
|
||||
+3
-3
@@ -13,7 +13,7 @@ tests/
|
||||
├── test_parser_factory.py # 解析器自动注册与工厂路由测试
|
||||
├── test_web_fetcher.py # 302跳转与URL正则提取测试
|
||||
├── test_*_parser.py # 各平台专属的单元测试
|
||||
├── live_parser_samples.json # 真实线上验证样本库 (41+ 平台活跃样例)
|
||||
├── live_parser_samples.json # 真实线上验证样本库 (42+ 平台活跃样例)
|
||||
├── test_live_parser_samples.py # 自动化执行样本库回归测试
|
||||
└── manual_verify_parsers.py # 命令行交互式/全量人工排查脚本
|
||||
```
|
||||
@@ -48,10 +48,10 @@ pytest tests/test_live_parser_samples.py -s
|
||||
```
|
||||
|
||||
### 4. 运行交互式真实样本回归验证工具
|
||||
[tests/manual_verify_parsers.py](file:///Users/leo/Projects/media-parser/tests/manual_verify_parsers.py) 支持对 41+ 个平台的多种形态(短链、长链、视频、图集等)发起真实网络请求与字段契约校验:
|
||||
[tests/manual_verify_parsers.py](file:///Users/leo/Projects/media-parser/tests/manual_verify_parsers.py) 支持对 42+ 个平台的多种形态(短链、长链、视频、图集等)发起真实网络请求与字段契约校验:
|
||||
|
||||
```bash
|
||||
# 1. 快速冒烟测试 (每个平台只测 1 条最具代表性的链接,极速完成 41 平台基准健康检查)
|
||||
# 1. 快速冒烟测试 (每个平台只测 1 条最具代表性的链接,极速完成 42 平台基准健康检查)
|
||||
python3 tests/manual_verify_parsers.py --limit 1
|
||||
|
||||
# 2. 验证指定平台的多形态覆盖 (支持重复传入多个 --platform 参数)
|
||||
|
||||
@@ -0,0 +1,362 @@
|
||||
"""网易云音乐歌曲、MV、Mlog 与动态分享解析器。"""
|
||||
|
||||
import html
|
||||
import json
|
||||
import re
|
||||
from urllib.parse import parse_qs, urlencode, urlparse
|
||||
|
||||
from configs.logging_config import get_logger
|
||||
from src.parser_factory import register_parser
|
||||
from src.parsers.base_parser import BaseParser
|
||||
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@register_parser("网易云音乐")
|
||||
class NeteaseMusicParser(BaseParser):
|
||||
MOBILE_UA = (
|
||||
"Mozilla/5.0 (iPhone; CPU iPhone OS 18_0 like Mac OS X) "
|
||||
"AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0 Mobile/15E148 Safari/604.1"
|
||||
)
|
||||
DESKTOP_UA = (
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0 Safari/537.36"
|
||||
)
|
||||
|
||||
def __init__(self, real_url):
|
||||
super().__init__(real_url)
|
||||
self.headers = {"User-Agent": self.MOBILE_UA, "Referer": "https://music.163.com/"}
|
||||
self.title = ""
|
||||
self.cover_url = None
|
||||
self.video_list = []
|
||||
self.audio_url = None
|
||||
self.image_list = []
|
||||
self.subtitles = None
|
||||
self.author = {"nickname": "", "author_id": "", "avatar": ""}
|
||||
self.media_type, self.media_id = self._detect_media(real_url)
|
||||
self._parse()
|
||||
|
||||
@staticmethod
|
||||
def _detect_media(url):
|
||||
parsed = urlparse(url or "")
|
||||
params = parse_qs(parsed.query)
|
||||
path = parsed.path.lower()
|
||||
media_id = (params.get("id") or [None])[0]
|
||||
if "/landing/mlog" in path:
|
||||
return "mlog", media_id
|
||||
if "/landing/mv" in path or re.search(r"/(?:mv|mvdetail)(?:/|$)", path):
|
||||
return "mv", media_id or parsed.path.rstrip("/").split("/")[-1]
|
||||
if "/event" in path:
|
||||
return "event", media_id
|
||||
if "/song" in path:
|
||||
return "song", media_id
|
||||
return None, media_id
|
||||
|
||||
def _parse(self):
|
||||
if self.media_type == "mv":
|
||||
self._parse_mv()
|
||||
elif self.media_type == "mlog":
|
||||
self._parse_mlog()
|
||||
elif self.media_type == "event":
|
||||
self._parse_event()
|
||||
elif self.media_type == "song":
|
||||
self._parse_song(self.media_id)
|
||||
|
||||
def _request_json(self, url, params=None):
|
||||
try:
|
||||
response = self.session.get(url, params=params, headers=self.headers, timeout=10)
|
||||
response.raise_for_status()
|
||||
payload = response.json()
|
||||
return payload if isinstance(payload, dict) else {}
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to fetch NetEase Music API %s: %s", url, exc)
|
||||
return {}
|
||||
|
||||
def _parse_mv(self):
|
||||
if not self.media_id:
|
||||
return
|
||||
payload = self._request_json(
|
||||
"https://music.163.com/api/mv/detail",
|
||||
{"id": self.media_id},
|
||||
)
|
||||
data = payload.get("data") or {}
|
||||
self.title = data.get("name") or ""
|
||||
self.cover_url = data.get("cover")
|
||||
artists = data.get("artists") or []
|
||||
if artists and isinstance(artists[0], dict):
|
||||
artist = artists[0]
|
||||
self.author = {
|
||||
"nickname": artist.get("name") or data.get("artistName") or "",
|
||||
"author_id": str(artist.get("id") or data.get("artistId") or ""),
|
||||
"avatar": artist.get("img1v1Url") or "",
|
||||
}
|
||||
elif data.get("artistName"):
|
||||
self.author["nickname"] = data["artistName"]
|
||||
self.author["author_id"] = str(data.get("artistId") or "")
|
||||
|
||||
streams = []
|
||||
for quality, url in (data.get("brs") or {}).items():
|
||||
if self._valid_url(url):
|
||||
try:
|
||||
rank = int(quality)
|
||||
except (TypeError, ValueError):
|
||||
rank = 0
|
||||
streams.append((rank, url))
|
||||
streams.sort(key=lambda item: item[0], reverse=True)
|
||||
self.video_list = self._unique(url for _, url in streams)
|
||||
|
||||
def _parse_mlog(self):
|
||||
payload = self._fetch_initial_props(self.real_url, self.MOBILE_UA)
|
||||
info = payload.get("mlogInfo") or {}
|
||||
resource = info.get("resource") or {}
|
||||
content = resource.get("content") or {}
|
||||
profile = resource.get("profile") or {}
|
||||
video = content.get("video") or {}
|
||||
|
||||
self._set_author(profile)
|
||||
self.title = (
|
||||
content.get("title")
|
||||
or content.get("text")
|
||||
or (f"{self.author['nickname']}的音乐动态" if self.author["nickname"] else "网易云音乐动态")
|
||||
)
|
||||
self.cover_url = (
|
||||
video.get("coverUrl")
|
||||
or video.get("frameUrl")
|
||||
or ((video.get("frameImage") or {}).get("imageUrl"))
|
||||
)
|
||||
self.video_list = self._ranked_urls(video.get("urlInfos") or [])
|
||||
if not self.video_list:
|
||||
fallback = ((video.get("urlInfo") or {}).get("url"))
|
||||
self.video_list = [fallback] if self._valid_url(fallback) else []
|
||||
self.image_list = self._extract_images(content.get("image") or [])
|
||||
|
||||
song = content.get("song") or resource.get("song") or {}
|
||||
if song.get("id") and not self.video_list:
|
||||
self._parse_song(song["id"], fallback=song)
|
||||
|
||||
def _parse_event(self):
|
||||
if not self.media_id:
|
||||
return
|
||||
parsed = urlparse(self.real_url)
|
||||
params = parse_qs(parsed.query)
|
||||
query = {"id": self.media_id}
|
||||
if uid := (params.get("uid") or params.get("userid") or [None])[0]:
|
||||
query["uid"] = uid
|
||||
url = f"https://music.163.com/event?{urlencode(query)}"
|
||||
payload = self._fetch_event_data(url)
|
||||
if not payload:
|
||||
return
|
||||
|
||||
user = payload.get("user") or {}
|
||||
self._set_author(user)
|
||||
try:
|
||||
content = json.loads(payload.get("json") or "{}")
|
||||
except json.JSONDecodeError:
|
||||
content = {}
|
||||
|
||||
song = content.get("song") or {}
|
||||
self.title = content.get("title") or content.get("msg") or song.get("name") or ""
|
||||
self.image_list = self._extract_event_images(payload.get("pics") or [])
|
||||
if self.image_list:
|
||||
first = self.image_list[0]
|
||||
self.cover_url = first.get("url") if isinstance(first, dict) else first
|
||||
|
||||
video_data = content.get("videoData") or {}
|
||||
self.video_list = self._extract_nested_video_urls(video_data)
|
||||
if song.get("id"):
|
||||
self._parse_song(song["id"], fallback=song, preserve_title=True)
|
||||
|
||||
def _parse_song(self, song_id, fallback=None, preserve_title=False):
|
||||
if not song_id:
|
||||
return
|
||||
try:
|
||||
numeric_id = int(song_id)
|
||||
except (TypeError, ValueError):
|
||||
return
|
||||
detail = self._request_json(
|
||||
"https://music.163.com/api/song/detail/",
|
||||
{"id": numeric_id, "ids": json.dumps([numeric_id])},
|
||||
)
|
||||
songs = detail.get("songs") or []
|
||||
song = songs[0] if songs and isinstance(songs[0], dict) else (fallback or {})
|
||||
if not preserve_title or not self.title:
|
||||
self.title = song.get("name") or self.title
|
||||
album = song.get("album") or {}
|
||||
self.cover_url = self.cover_url or album.get("picUrl") or album.get("blurPicUrl")
|
||||
artists = song.get("artists") or []
|
||||
if artists and isinstance(artists[0], dict) and not self.author["nickname"]:
|
||||
artist = artists[0]
|
||||
self.author = {
|
||||
"nickname": artist.get("name") or "",
|
||||
"author_id": str(artist.get("id") or ""),
|
||||
"avatar": artist.get("img1v1Url") or artist.get("picUrl") or "",
|
||||
}
|
||||
|
||||
player = self._request_json(
|
||||
"https://music.163.com/api/song/enhance/player/url",
|
||||
{"id": numeric_id, "ids": json.dumps([numeric_id]), "br": 320000},
|
||||
)
|
||||
entries = player.get("data") or []
|
||||
if entries and isinstance(entries[0], dict) and self._valid_url(entries[0].get("url")):
|
||||
self.audio_url = entries[0]["url"]
|
||||
self._parse_lyrics(numeric_id)
|
||||
|
||||
def _parse_lyrics(self, song_id):
|
||||
payload = self._request_json(
|
||||
"https://music.163.com/api/song/lyric",
|
||||
{"id": song_id, "lv": 1, "kv": 1, "tv": -1},
|
||||
)
|
||||
lyric = ((payload.get("lrc") or {}).get("lyric"))
|
||||
if not isinstance(lyric, str):
|
||||
return
|
||||
subtitles = []
|
||||
for line in lyric.splitlines():
|
||||
match = re.match(r"\[(\d+):(\d+(?:\.\d+)?)\](.*)", line.strip())
|
||||
if match and match.group(3).strip():
|
||||
start = int(match.group(1)) * 60 + float(match.group(2))
|
||||
subtitles.append({"start": round(start, 3), "text": match.group(3).strip()})
|
||||
self.subtitles = subtitles or None
|
||||
|
||||
def _fetch_initial_props(self, url, user_agent):
|
||||
try:
|
||||
response = self.session.get(
|
||||
url,
|
||||
headers={"User-Agent": user_agent, "Referer": "https://music.163.com/"},
|
||||
allow_redirects=True,
|
||||
timeout=10,
|
||||
)
|
||||
response.raise_for_status()
|
||||
self.html_content = response.text
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to fetch NetEase Music share page: %s", exc)
|
||||
return {}
|
||||
match = re.search(
|
||||
r"window\.__INITIAL_PROPS__\s*=\s*(\{.*?\})\s*</script>",
|
||||
self.html_content or "",
|
||||
re.DOTALL,
|
||||
)
|
||||
if not match:
|
||||
return {}
|
||||
try:
|
||||
return json.loads(match.group(1))
|
||||
except json.JSONDecodeError as exc:
|
||||
logger.warning("Failed to parse NetEase Music initial props: %s", exc)
|
||||
return {}
|
||||
|
||||
def _fetch_event_data(self, url):
|
||||
try:
|
||||
response = self.session.get(
|
||||
url,
|
||||
headers={"User-Agent": self.DESKTOP_UA, "Referer": "https://music.163.com/"},
|
||||
timeout=10,
|
||||
)
|
||||
response.raise_for_status()
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to fetch NetEase Music event: %s", exc)
|
||||
return {}
|
||||
match = re.search(
|
||||
r'<textarea[^>]+id="event-data"[^>]*>\s*(.*?)\s*</textarea>',
|
||||
response.text or "",
|
||||
re.DOTALL,
|
||||
)
|
||||
if not match:
|
||||
return {}
|
||||
try:
|
||||
return json.loads(html.unescape(match.group(1)))
|
||||
except json.JSONDecodeError as exc:
|
||||
logger.warning("Failed to parse NetEase Music event data: %s", exc)
|
||||
return {}
|
||||
|
||||
def _set_author(self, profile):
|
||||
if not isinstance(profile, dict):
|
||||
return
|
||||
self.author = {
|
||||
"nickname": profile.get("nickname") or profile.get("name") or "",
|
||||
"author_id": str(profile.get("userId") or profile.get("id") or ""),
|
||||
"avatar": profile.get("avatarUrl") or profile.get("avatar") or "",
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def _ranked_urls(cls, entries):
|
||||
ranked = []
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict) or not cls._valid_url(entry.get("url")):
|
||||
continue
|
||||
quality = entry.get("resolution") or entry.get("r") or 0
|
||||
ranked.append((quality if isinstance(quality, (int, float)) else 0, entry["url"]))
|
||||
ranked.sort(key=lambda item: item[0], reverse=True)
|
||||
return cls._unique(url for _, url in ranked)
|
||||
|
||||
@classmethod
|
||||
def _extract_images(cls, entries):
|
||||
images = []
|
||||
for entry in entries:
|
||||
if isinstance(entry, str) and cls._valid_url(entry):
|
||||
images.append(entry)
|
||||
elif isinstance(entry, dict):
|
||||
url = entry.get("originUrl") or entry.get("url") or entry.get("imageUrl") or entry.get("picUrl")
|
||||
if cls._valid_url(url):
|
||||
images.append(url)
|
||||
return cls._unique(images)
|
||||
|
||||
@classmethod
|
||||
def _extract_event_images(cls, entries):
|
||||
images = []
|
||||
for entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
url = entry.get("originUrl") or entry.get("pcRectangleUrl") or entry.get("rectangleUrl")
|
||||
if not cls._valid_url(url):
|
||||
continue
|
||||
live_url = entry.get("videoOriginalUrl") or entry.get("videoUrl")
|
||||
images.append(
|
||||
{"url": url, "live_photo_url": live_url if cls._valid_url(live_url) else None}
|
||||
)
|
||||
return images
|
||||
|
||||
@classmethod
|
||||
def _extract_nested_video_urls(cls, value):
|
||||
found = []
|
||||
if isinstance(value, dict):
|
||||
for key, child in value.items():
|
||||
if "url" in key.lower() and cls._valid_url(child) and child.lower().split("?", 1)[0].endswith((".mp4", ".mov", ".m3u8")):
|
||||
found.append(child)
|
||||
else:
|
||||
found.extend(cls._extract_nested_video_urls(child))
|
||||
elif isinstance(value, list):
|
||||
for child in value:
|
||||
found.extend(cls._extract_nested_video_urls(child))
|
||||
return cls._unique(found)
|
||||
|
||||
@staticmethod
|
||||
def _valid_url(value):
|
||||
return isinstance(value, str) and value.startswith(("http://", "https://"))
|
||||
|
||||
@staticmethod
|
||||
def _unique(values):
|
||||
return list(dict.fromkeys(value for value in values if value))
|
||||
|
||||
def get_real_video_url(self):
|
||||
return self.video_list[0] if self.video_list else None
|
||||
|
||||
def get_video_list(self):
|
||||
return self.video_list
|
||||
|
||||
def get_audio_url(self):
|
||||
return self.audio_url
|
||||
|
||||
def get_title_content(self):
|
||||
return self.title
|
||||
|
||||
def get_cover_photo_url(self):
|
||||
return self.cover_url
|
||||
|
||||
def get_author_info(self):
|
||||
return self.author
|
||||
|
||||
def get_image_list(self):
|
||||
return self.image_list
|
||||
|
||||
def get_subtitles(self):
|
||||
return self.subtitles
|
||||
@@ -1,8 +1,57 @@
|
||||
{
|
||||
"schema_version": 2,
|
||||
"description": "41 平台多形态全景自动化回归测试用例库 (涵盖短链/长链/图集/单视频/AI生成等)",
|
||||
"total_cases": 290,
|
||||
"description": "42 平台多形态全景自动化回归测试用例库 (涵盖短链/长链/图集/单视频/AI生成等)",
|
||||
"total_cases": 293,
|
||||
"cases": [
|
||||
{
|
||||
"platform": "网易云音乐",
|
||||
"url": "https://163cn.tv/beBkdACV",
|
||||
"media_types": [
|
||||
"audio",
|
||||
"images",
|
||||
"live_media"
|
||||
],
|
||||
"expected_fields": [
|
||||
"audio",
|
||||
"images",
|
||||
"live_media",
|
||||
"cover",
|
||||
"title",
|
||||
"author"
|
||||
],
|
||||
"note": "2026-09-01 已验证: 网易云短链 Event,包含歌曲、图集、LivePhoto 与歌词",
|
||||
"pattern": "Event 音乐图文与实况"
|
||||
},
|
||||
{
|
||||
"platform": "网易云音乐",
|
||||
"url": "https://163cn.tv/ba6qK0Lv",
|
||||
"media_types": [
|
||||
"video"
|
||||
],
|
||||
"expected_fields": [
|
||||
"video",
|
||||
"cover",
|
||||
"title",
|
||||
"author"
|
||||
],
|
||||
"note": "2026-09-01 已验证: 网易云 Mlog 分享短链,多档签名视频",
|
||||
"pattern": "Mlog 动态视频"
|
||||
},
|
||||
{
|
||||
"platform": "网易云音乐",
|
||||
"url": "https://fn.music.163.com/g/mlog/mlog-mobile/landing/mv?id=34781314&userid=12112389178",
|
||||
"media_types": [
|
||||
"video"
|
||||
],
|
||||
"expected_fields": [
|
||||
"video",
|
||||
"cover",
|
||||
"title",
|
||||
"author"
|
||||
],
|
||||
"note": "2026-09-01 已验证: 网易云 MV 分享,可提取 1080P 至 240P 多档 MP4",
|
||||
"pattern": "MV 多清晰度"
|
||||
},
|
||||
{
|
||||
"platform": "QQ音乐",
|
||||
"url": "https://c6.y.qq.com/base/fcgi-bin/u?__=eZmYFaYs9yHY",
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
import html
|
||||
import json
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from src.parsers.netease_music_parser import NeteaseMusicParser
|
||||
|
||||
|
||||
class NeteaseMusicParserTest(unittest.TestCase):
|
||||
@staticmethod
|
||||
def response(*, text="", payload=None, url="https://music.163.com/"):
|
||||
response = Mock()
|
||||
response.text = text
|
||||
response.url = url
|
||||
response.json.return_value = payload or {}
|
||||
response.raise_for_status.return_value = None
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def initial_html(key, value):
|
||||
payload = {key: value, "isServerError": False}
|
||||
return f"<script>window.__INITIAL_PROPS__ = {json.dumps(payload, ensure_ascii=False)}\n</script>"
|
||||
|
||||
@patch("src.parsers.base_parser.requests.Session.get")
|
||||
def test_parses_mv_with_highest_quality_first(self, get):
|
||||
get.return_value = self.response(
|
||||
payload={
|
||||
"code": 200,
|
||||
"data": {
|
||||
"id": 34781314,
|
||||
"name": "透明爱人",
|
||||
"cover": "http://p1.music.126.net/cover.jpg",
|
||||
"artistName": "白鹿",
|
||||
"artistId": 12603258,
|
||||
"artists": [{"id": 12603258, "name": "白鹿"}],
|
||||
"brs": {
|
||||
"480": "http://vod.126.net/480.mp4",
|
||||
"1080": "http://vod.126.net/1080.mp4",
|
||||
"720": "http://vod.126.net/720.mp4",
|
||||
},
|
||||
},
|
||||
}
|
||||
)
|
||||
parser = NeteaseMusicParser(
|
||||
"https://fn.music.163.com/g/mlog/mlog-mobile/landing/mv?id=34781314"
|
||||
)
|
||||
self.assertEqual(parser.get_title_content(), "透明爱人")
|
||||
self.assertEqual(parser.get_author_info()["nickname"], "白鹿")
|
||||
self.assertEqual(parser.get_real_video_url(), "http://vod.126.net/1080.mp4")
|
||||
self.assertEqual(len(parser.get_video_list()), 3)
|
||||
|
||||
@patch("src.parsers.base_parser.requests.Session.get")
|
||||
def test_parses_mlog_signed_video_and_profile(self, get):
|
||||
mlog = {
|
||||
"resource": {
|
||||
"content": {
|
||||
"title": "动态视频",
|
||||
"image": [],
|
||||
"video": {
|
||||
"coverUrl": "http://p1.music.126.net/mlog-cover.jpg",
|
||||
"urlInfos": [
|
||||
{"resolution": 360, "url": "http://vod.126.net/360.mp4"},
|
||||
{"resolution": 1080, "url": "http://vod.126.net/1080.mp4"},
|
||||
],
|
||||
},
|
||||
},
|
||||
"profile": {
|
||||
"userId": 123,
|
||||
"nickname": "云村用户",
|
||||
"avatarUrl": "http://p1.music.126.net/avatar.jpg",
|
||||
},
|
||||
}
|
||||
}
|
||||
get.return_value = self.response(text=self.initial_html("mlogInfo", mlog))
|
||||
parser = NeteaseMusicParser(
|
||||
"https://fn.music.163.com/g/mlog/mlog-mobile/landing/mlog?id=a123&type=2"
|
||||
)
|
||||
self.assertEqual(parser.get_title_content(), "动态视频")
|
||||
self.assertEqual(parser.get_author_info()["author_id"], "123")
|
||||
self.assertEqual(parser.get_real_video_url(), "http://vod.126.net/1080.mp4")
|
||||
|
||||
@patch("src.parsers.base_parser.requests.Session.get")
|
||||
def test_parses_event_images_livephotos_and_free_song(self, get):
|
||||
event = {
|
||||
"user": {
|
||||
"userId": 456,
|
||||
"nickname": "动态作者",
|
||||
"avatarUrl": "http://p1.music.126.net/event-avatar.jpg",
|
||||
},
|
||||
"json": json.dumps(
|
||||
{
|
||||
"title": "动态标题",
|
||||
"msg": "动态正文",
|
||||
"song": {"id": 3383347615, "name": "念张师"},
|
||||
"videoData": None,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
"pics": [
|
||||
{
|
||||
"originUrl": "http://p1.music.126.net/photo.jpg",
|
||||
"videoOriginalUrl": "http://vod.126.net/live.mov",
|
||||
}
|
||||
],
|
||||
}
|
||||
event_html = (
|
||||
'<textarea name="txt" id="event-data" style="display:none;">'
|
||||
+ html.escape(json.dumps(event, ensure_ascii=False))
|
||||
+ "</textarea>"
|
||||
)
|
||||
|
||||
def side_effect(url, **kwargs):
|
||||
if "/event" in url:
|
||||
return self.response(text=event_html)
|
||||
if "/song/detail/" in url:
|
||||
return self.response(
|
||||
payload={
|
||||
"songs": [
|
||||
{
|
||||
"id": 3383347615,
|
||||
"name": "念张师",
|
||||
"album": {"picUrl": "http://p1.music.126.net/album.jpg"},
|
||||
"artists": [{"id": 1, "name": "歌手"}],
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
if "/player/url" in url:
|
||||
return self.response(payload={"data": [{"url": "http://m801.music.126.net/song.mp3"}]})
|
||||
return self.response(payload={"lrc": {"lyric": "[00:01.50]第一句\n[00:03.00]第二句"}})
|
||||
|
||||
get.side_effect = side_effect
|
||||
parser = NeteaseMusicParser(
|
||||
"https://music.163.com/event?id=37613855054&uid=13685426555"
|
||||
)
|
||||
self.assertEqual(parser.get_title_content(), "动态标题")
|
||||
self.assertEqual(parser.get_audio_url(), "http://m801.music.126.net/song.mp3")
|
||||
self.assertEqual(parser.get_image_list()[0]["live_photo_url"], "http://vod.126.net/live.mov")
|
||||
self.assertEqual(parser.get_subtitles()[0], {"start": 1.5, "text": "第一句"})
|
||||
|
||||
@patch("src.parsers.base_parser.requests.Session.get")
|
||||
def test_paid_song_does_not_return_fake_audio(self, get):
|
||||
def side_effect(url, **kwargs):
|
||||
if "/song/detail/" in url:
|
||||
return self.response(
|
||||
payload={
|
||||
"songs": [
|
||||
{
|
||||
"id": 472137906,
|
||||
"name": "时差",
|
||||
"album": {"picUrl": "http://p1.music.126.net/album.jpg"},
|
||||
"artists": [{"id": 1, "name": "鹿晗"}],
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
return self.response(payload={"data": [{"url": None}], "lrc": {}})
|
||||
|
||||
get.side_effect = side_effect
|
||||
parser = NeteaseMusicParser("https://music.163.com/song?id=472137906")
|
||||
self.assertEqual(parser.get_title_content(), "时差")
|
||||
self.assertIsNone(parser.get_audio_url())
|
||||
|
||||
@patch("src.parsers.base_parser.requests.Session.get")
|
||||
def test_invalid_song_id_fails_cleanly(self, get):
|
||||
parser = NeteaseMusicParser("https://music.163.com/song?id=invalid")
|
||||
get.assert_not_called()
|
||||
self.assertIsNone(parser.get_audio_url())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -43,6 +43,14 @@ class UrlParserTest(unittest.TestCase):
|
||||
"https://i2.y.qq.com/n3/other/pages/details/mv.html?vid=012XViNT0znYUR&noise=x",
|
||||
"https://i2.y.qq.com/n3/other/pages/details/mv.html?vid=012XViNT0znYUR",
|
||||
),
|
||||
(
|
||||
"https://fn.music.163.com/g/mlog/mlog-mobile/landing/mlog?id=a123&type=2&userid=456&noise=x",
|
||||
"https://fn.music.163.com/g/mlog/mlog-mobile/landing/mlog?id=a123&userid=456&type=2",
|
||||
),
|
||||
(
|
||||
"https://music.163.com/#/song?id=3383347615&noise=x",
|
||||
"https://music.163.com/song?id=3383347615",
|
||||
),
|
||||
("https://pd.qq.com/s/code?b=2&noise=x", "https://pd.qq.com/s/code?b=2"),
|
||||
("https://video.weibo.com/show?fid=1034:123&noise=x", "https://video.weibo.com/show?fid=1034%3A123"),
|
||||
(
|
||||
@@ -177,6 +185,11 @@ class UrlParserTest(unittest.TestCase):
|
||||
self.assertEqual(UrlParser.get_platform("https://i2.y.qq.com/n3/other/pages/details/mv.html?vid=abc"), "QQ音乐")
|
||||
self.assertEqual(UrlParser.get_platform("https://y.qq.com/n/ryqq/mv/abc"), "QQ音乐")
|
||||
|
||||
def test_recognizes_netease_music_domains(self):
|
||||
self.assertEqual(UrlParser.get_platform("https://163cn.tv/abc123"), "网易云音乐")
|
||||
self.assertEqual(UrlParser.get_platform("https://music.163.com/song?id=1"), "网易云音乐")
|
||||
self.assertEqual(UrlParser.get_platform("https://fn.music.163.com/g/mlog/x"), "网易云音乐")
|
||||
|
||||
def test_recognizes_xigua_video_on_iesdouyin_domain(self):
|
||||
self.assertEqual(
|
||||
UrlParser.get_platform("https://www.iesdouyin.com/xg/video/7676450021063735414/"),
|
||||
@@ -282,6 +295,18 @@ class WebFetcherTest(unittest.TestCase):
|
||||
"https://i2.y.qq.com/n3/other/pages/details/mv.html?vid=012XViNT0znYUR",
|
||||
)
|
||||
|
||||
def test_follows_netease_short_link_and_preserves_event_id(self):
|
||||
redirect_url = "https://y.music.163.com/m/event?id=37826361829&uid=5153433584&dlt=0846"
|
||||
with patch(
|
||||
"utils.web_fetcher.requests.get",
|
||||
return_value=self.response(redirect_url),
|
||||
):
|
||||
result = WebFetcher.fetch_redirect_url("https://163cn.tv/beBkdACV")
|
||||
self.assertEqual(
|
||||
result,
|
||||
"https://y.music.163.com/m/event?id=37826361829&uid=5153433584",
|
||||
)
|
||||
|
||||
def test_stops_before_login_or_verification_page(self):
|
||||
for blocked_path in ("/login", "/404", "/captcha", "/verify", "/error"):
|
||||
with self.subTest(blocked_path=blocked_path):
|
||||
|
||||
@@ -221,6 +221,21 @@ class UrlParser:
|
||||
preserved_params.append((key, value))
|
||||
if preserved_params:
|
||||
address = f"{address}?{urlencode(preserved_params)}"
|
||||
elif platform == "网易云音乐":
|
||||
query_params = parse_qs(parsed_url.query)
|
||||
fragment = parsed_url.fragment.lstrip("/")
|
||||
if "?" in fragment:
|
||||
fragment_path, fragment_query = fragment.split("?", 1)
|
||||
if fragment_path in {"song", "mv", "event"}:
|
||||
address = f"{parsed_url.scheme}://{domain}/{fragment_path}"
|
||||
query_params = parse_qs(fragment_query)
|
||||
preserved_params = []
|
||||
for key in ("id", "uid", "userid", "type", "eventId", "songId"):
|
||||
value = query_params.get(key, [None])[0]
|
||||
if value is not None and value != "null":
|
||||
preserved_params.append((key, value))
|
||||
if preserved_params:
|
||||
address = f"{address}?{urlencode(preserved_params)}"
|
||||
elif platform == "腾讯频道":
|
||||
query_params = parse_qs(parsed_url.query)
|
||||
if value := query_params.get("b", [None])[0]:
|
||||
|
||||
Reference in New Issue
Block a user