diff --git a/README.md b/README.md index 8a04a16..ab2fcfd 100644 --- a/README.md +++ b/README.md @@ -37,58 +37,58 @@ Media-Parser是一款专为短视频创作者与开发者打造的**100%原生 ## 💾 支持的平台矩阵 -| 平台名称 | 作者 | 标题 | 封面 | 视频 | 图集 | 多视频 | 实况 | 音频 | 字幕 | -| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| **抖音** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| **小红书** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | | -| **视频号** | ✓ | ✓ | ✓ | ✓ | ✓ | | | ✓ | | -| **微信公众号** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | -| **快手** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | | -| **哔哩哔哩** | ✓ | ✓ | ✓ | ✓ | | ✓ | | ✓ | | -| **豆包** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | -| **即梦AI** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **小云雀AI** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **可灵AI** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **海螺AI** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **夸克AI** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **通义千问** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | -| **腾讯元宝** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | -| **闲鱼** | | ✓ | ✓ | | ✓ | | | | | -| **拼多多** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **Soul** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **汽水音乐** | ✓ | ✓ | ✓ | ✓ | | | | ✓ | ✓ | -| **QQ音乐** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **网易云音乐** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | -| **酷狗音乐** | ✓ | ✓ | ✓ | ✓ | | | | ✓ | | -| **配音秀** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **松果时刻** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | -| **腾讯频道** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **剪映 / CapCut** | ✓ | ✓ | ✓ | ✓ | ✓ | | | ✓ | | -| **快影** | ✓ | ✓ | ✓ | ✓ | | | | ✓ | | -| **皮皮搞笑** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **微视** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **AcFun** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **西瓜视频** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **今日头条** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **绿洲** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **皮皮虾** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **全民K歌** | | ✓ | ✓ | ✓ | | | | | | -| **新片场** | ✓ | ✓ | ✓ | ✓ | | ✓ | | | | -| **好看视频** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **梨视频** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **微博** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **知乎** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **虎牙** | | ✓ | ✓ | ✓ | | | | | | -| **美拍** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **最右** | ✓ | ✓ | | ✓ | | | | | | -| **番茄小说** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **红果短剧** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **红果漫剧** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **得物** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **网易LOFTER** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **星绘AI** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | -| **央视** | ✓ | ✓ | ✓ | ✓ | | | | | | -| **央视频** | ✓ | ✓ | ✓ | | ✓ | | | | | +| 平台名称 | 作者 | 标题 | 文案 | 封面 | 视频 | 图集 | 多视频 | 实况 | 音频 | 字幕 | +| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| **抖音** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| **小红书** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | | | +| **视频号** | ✓ | ✓ | ✓ | ✓ | ✓ | | | ✓ | | | +| **微信公众号** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | | +| **快手** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | | +| **哔哩哔哩** | ✓ | ✓ | ✓ | ✓ | ✓ | | ✓ | | ✓ | | +| **豆包** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | +| **即梦AI** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | | +| **小云雀AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **可灵AI** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **海螺AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **夸克AI** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | +| **通义千问** | ✓ | ✓ | | ✓ | ✓ | ✓ | ✓ | | | | +| **腾讯元宝** | ✓ | ✓ | | ✓ | ✓ | ✓ | ✓ | | | | +| **闲鱼** | | ✓ | | ✓ | | ✓ | | | | | +| **拼多多** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | +| **Soul** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **汽水音乐** | ✓ | ✓ | | ✓ | ✓ | | | | ✓ | ✓ | +| **QQ音乐** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **网易云音乐** | ✓ | ✓ | | ✓ | ✓ | ✓ | | ✓ | ✓ | ✓ | +| **酷狗音乐** | ✓ | ✓ | | ✓ | ✓ | | | | ✓ | | +| **配音秀** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **松果时刻** | ✓ | ✓ | | ✓ | ✓ | ✓ | ✓ | | ✓ | | +| **腾讯频道** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **剪映 / CapCut** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | ✓ | | +| **快影** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | ✓ | | +| **皮皮搞笑** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | | +| **微视** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **AcFun** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **西瓜视频** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | | +| **今日头条** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | | +| **绿洲** | ✓ | ✓ | | ✓ | ✓ | ✓ | | | | | +| **皮皮虾** | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | | +| **全民K歌** | | ✓ | | ✓ | ✓ | | | | | | +| **新片场** | ✓ | ✓ | | ✓ | ✓ | | ✓ | | | | +| **好看视频** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **梨视频** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **微博** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **知乎** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **虎牙** | | ✓ | | ✓ | ✓ | | | | | | +| **美拍** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **最右** | ✓ | ✓ | ✓ | | ✓ | | | | | | +| **番茄小说** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **红果短剧** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **红果漫剧** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **得物** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **网易LOFTER** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **星绘AI** | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | +| **央视** | ✓ | ✓ | | ✓ | ✓ | | | | | | +| **央视频** | ✓ | ✓ | | ✓ | | ✓ | | | | | *注:腾讯元宝等极少数平台提取的是官方存储桶原画质直链,素材保留官方原生水印。 @@ -151,13 +151,13 @@ python app.py --- -## 🔌 API 核心接口说明 +## 🔌 API 接口 -### 客户 GET 简单接口(推荐) +### 解析媒体 -* **接口路径**:`GET /api/v1/parse` -* **参数**:`url`,视频或图文分享链接 -* **鉴权**:推荐 `Authorization: Bearer mp_xxx`;为方便小白,也兼容查询参数 `key=mp_xxx` +- **接口**:`GET /api/v1/parse` +- **参数**:`url`,视频或图文分享链接 +- **鉴权**:`Authorization: Bearer mp_xxx` ```bash curl --get 'http://localhost:8051/api/v1/parse' \ @@ -165,7 +165,7 @@ curl --get 'http://localhost:8051/api/v1/parse' \ --data-urlencode 'url=https://v.douyin.com/xxx/' ``` -纯 URL 兼容写法如下。查询参数中的 Key 可能被浏览器或代理日志记录,不建议在正式项目中使用: +也支持通过 `key` 查询参数传递密钥,但生产环境不推荐,因为密钥可能被浏览器或代理日志记录: ```text http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/ @@ -173,14 +173,10 @@ http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/ 客户可在 `/console` 创建和停用 API Key;管理员可在 `/admin` 配置客户有效期、积分、QPS 与平台开关。 -### 在线体验兼容接口 +### 返回数据 -`POST /api/parse` 仅供首页在线体验功能内部使用,可由管理员单独关闭,不作为客户接入接口。 +接口以 JSON 格式返回标题、正文、作者及解析出的媒体资源。部分平台可能没有独立标题、封面或主视频;为兼容已有调用方,对应字段会使用正文、首图或首视频兜底。原始正文和图集仍分别保留在 `desc`、`image_list` 中,多视频内容则通过 `video_list` 返回。 -### 通用返回说明(Response) -格式: `application/json` - -成功响应示例: ```json { "retcode": 200, @@ -189,6 +185,7 @@ http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/ "video_id": "7123...", "platform": "抖音", "title": "视频标题内容", + "desc": "视频文案内容", "video_url": "https://... (主视频地址)", "video_list": [ "https://... (仅多视频/合集内容额外返回,首项与 video_url 相同)" @@ -215,7 +212,8 @@ http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/ } ``` -失败响应示例: +失败时会返回稳定的错误码: + ```json { "retcode": 400, @@ -228,6 +226,8 @@ http://localhost:8051/api/v1/parse?key=mp_xxx&url=https://v.douyin.com/xxx/ 默认响应还会附带可由管理员配置或关闭的 `_tip` 服务信息字段。 +> `POST /api/parse` 仅供网站首页在线体验使用,不作为客户接入接口。 + --- ## 🧪 自动化测试与健康自检 diff --git a/src/api/parse.py b/src/api/parse.py index ca13775..03842be 100644 --- a/src/api/parse.py +++ b/src/api/parse.py @@ -155,18 +155,31 @@ def _execute_parse(text, access): ] processed_video_list = list(dict.fromkeys(processed_video_list)) primary_video_url = UrlParser.convert_to_https(content_data['video_url']) + if not primary_video_url and processed_video_list: + primary_video_url = processed_video_list[0] if primary_video_url and primary_video_url in processed_video_list: processed_video_list.remove(primary_video_url) processed_video_list.insert(0, primary_video_url) - # 4. 统一转换 HTTPS + # 4. 统一转换 HTTPS,并维持 v1 既有主字段的兜底语义。 + title = content_data['title'] + desc = content_data['desc'] + if not title and desc: + title = desc + + cover_url = UrlParser.convert_to_https(content_data['cover_url']) + if not cover_url and processed_image_list: + first_image = processed_image_list[0] + cover_url = first_image.get('url') if isinstance(first_image, dict) else first_image + data_dict = { 'video_id': UrlParser.get_video_id(redirect_url), 'platform': platform, - 'title': content_data['title'], + 'title': title, + 'desc': desc, 'video_url': primary_video_url, 'audio_url': UrlParser.convert_to_https(content_data.get('audio_url')), - 'cover_url': UrlParser.convert_to_https(content_data['cover_url']), + 'cover_url': cover_url, 'author': content_data['author'], 'image_list': processed_image_list } @@ -212,6 +225,7 @@ def _fetch_with_retry(parser, platform): for i in range(max_attempts): res = { 'title': parser.get_title_content(), + 'desc': safe_execute(getattr(parser, 'get_description', None)), 'video_url': parser.get_real_video_url(), 'video_list': safe_execute(getattr(parser, 'get_video_list', None), default=[]), 'cover_url': parser.get_cover_photo_url(), @@ -220,8 +234,6 @@ def _fetch_with_retry(parser, platform): 'audio_url': safe_execute(getattr(parser, 'get_audio_url', None)), 'subtitles': safe_execute(getattr(parser, 'get_subtitles', None)) } - if not res['video_url'] and res['video_list']: - res['video_url'] = res['video_list'][0] if res['video_url'] or res['video_list'] or res['image_list'] or res['audio_url']: return res diff --git a/src/parsers/base_parser.py b/src/parsers/base_parser.py index ac3979e..37a86d0 100644 --- a/src/parsers/base_parser.py +++ b/src/parsers/base_parser.py @@ -17,6 +17,10 @@ class BaseParser: def get_title_content(self): raise NotImplementedError + def get_description(self): + """获取作品文案或描述;平台没有独立描述时返回 None。""" + return None + def get_cover_photo_url(self): raise NotImplementedError diff --git a/src/parsers/bilibili_parser.py b/src/parsers/bilibili_parser.py index 9bf7976..76b036b 100644 --- a/src/parsers/bilibili_parser.py +++ b/src/parsers/bilibili_parser.py @@ -194,19 +194,25 @@ class BilibiliParser(BaseParser): def get_title_content(self): if self.video_info: - return self.video_info.get("title", "") + return self.video_info.get("title") or None if self.dynamic_info: modules = self.dynamic_info.get("modules", {}) dynamic_mod = modules.get("module_dynamic", {}) major = dynamic_mod.get("major", {}) if opus := major.get("opus"): - if title := opus.get("title"): - return title - if summary := opus.get("summary", {}).get("text"): - return summary - if desc := dynamic_mod.get("desc", {}).get("text"): - return desc - return "" + return opus.get("title") or None + return None + + def get_description(self): + if self.video_info: + return self.video_info.get("desc") or None + if self.dynamic_info: + dynamic_mod = self.dynamic_info.get("modules", {}).get("module_dynamic", {}) + major = dynamic_mod.get("major", {}) + if opus := major.get("opus"): + return opus.get("summary", {}).get("text") or dynamic_mod.get("desc", {}).get("text") or None + return dynamic_mod.get("desc", {}).get("text") or None + return None def get_cover_photo_url(self): if self.video_info: diff --git a/src/parsers/butterflyai_parser.py b/src/parsers/butterflyai_parser.py index d8647bb..092711c 100644 --- a/src/parsers/butterflyai_parser.py +++ b/src/parsers/butterflyai_parser.py @@ -21,6 +21,7 @@ class ButterflyAIParser(BaseParser): "Accept": "application/json, text/plain, */*", } self.title = "" + self.description = None self.cover_url = None self.video_url = None self.image_list = [] @@ -85,13 +86,8 @@ class ButterflyAIParser(BaseParser): creator = share_record.get("creator") or {} # 提取标题与生成 Prompt - self.title = ( - artwork.get("title") - or artwork.get("description") - or share_record.get("prompt") - or share_record.get("show_info", {}).get("effect_title") - or "" - ).strip() + self.title = (artwork.get("title") or share_record.get("show_info", {}).get("effect_title") or "").strip() + self.description = (artwork.get("description") or share_record.get("prompt") or "").strip() or None if not self.title and share_record.get("effect_list"): first_eff = share_record["effect_list"][0] self.title = ( @@ -144,15 +140,15 @@ class ButterflyAIParser(BaseParser): self.cover_url = artwork.get("cover_image_url") or artwork.get("cover_url") if not self.cover_url and rendering_video: self.cover_url = rendering_video.get("cover_image_url") - if not self.cover_url and self.image_list: - self.cover_url = self.image_list[0] - def get_real_video_url(self): return self.video_url def get_title_content(self): return self.title or "" + def get_description(self): + return self.description + def get_cover_photo_url(self): return self.cover_url diff --git a/src/parsers/dewu_parser.py b/src/parsers/dewu_parser.py index 0d22a73..a531ebb 100644 --- a/src/parsers/dewu_parser.py +++ b/src/parsers/dewu_parser.py @@ -21,6 +21,7 @@ class DewuParser(BaseParser): "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", } self.title = "" + self.description = None self.cover_url = None self.video_url = None self.image_list = [] @@ -53,7 +54,8 @@ class DewuParser(BaseParser): user_data = target_data.get("userInfo", {}) or {} # 提取标题与文案 - self.title = (content_data.get("title") or content_data.get("content") or "").strip() + self.title = (content_data.get("title") or "").strip() + self.description = (content_data.get("content") or "").strip() or None # 提取封面 cover_info = content_data.get("cover") or {} @@ -119,6 +121,9 @@ class DewuParser(BaseParser): def get_title_content(self): return self.title or "" + def get_description(self): + return self.description + def get_cover_photo_url(self): return self.cover_url diff --git a/src/parsers/douyin_parser.py b/src/parsers/douyin_parser.py index e9bc563..712972f 100644 --- a/src/parsers/douyin_parser.py +++ b/src/parsers/douyin_parser.py @@ -950,10 +950,7 @@ class DouyinParser(BaseParser): ep_info = data_dict.get('episode_info') or (data_dict.get('lvideo_detail') or {}).get('episode_info') or {} if not ep_info and data_dict.get('episode_list'): ep_info = data_dict['episode_list'][0] - ep_title = ep_info.get('episode_name') or ep_info.get('title') or ep_info.get('desc') or '' - - if not album_title and not ep_title and data_dict.get('aweme_detail'): - ep_title = data_dict['aweme_detail'].get('desc', '') or data_dict['aweme_detail'].get('itemTitle', '') + ep_title = ep_info.get('episode_name') or ep_info.get('title') or '' if album_title and ep_title and album_title != ep_title: return f"【放映厅】{album_title} - {ep_title}" @@ -964,12 +961,25 @@ class DouyinParser(BaseParser): if not data_dict.get('aweme_detail'): return None - title_content = data_dict['aweme_detail'].get('desc', '') - return title_content + aweme = data_dict['aweme_detail'] + title = aweme.get('itemTitle') or None + desc = aweme.get('desc') or None + # itemTitle 与 desc 相同通常只是接口冗余字段,不将文案伪装成标题。 + if title and desc and title.strip() == desc.strip(): + return None + return title except (KeyError, json.JSONDecodeError, TypeError) as e: logger.warning(f"Failed to parse title content: {e}") return None + def get_description(self): + """返回抖音原始作品文案,不用标题字段回填。""" + try: + return (self.data.get('aweme_detail') or {}).get('desc') or None + except (AttributeError, TypeError) as e: + logger.warning(f"Failed to parse Douyin description: {e}") + return None + def get_cover_photo_url(self): try: data_dict = self.data @@ -1208,4 +1218,3 @@ class DouyinParser(BaseParser): except Exception as e: logger.warning(f"Failed to parse image list: {e}") return [] - diff --git a/src/parsers/hailuo_parser.py b/src/parsers/hailuo_parser.py index 4319490..7262b76 100644 --- a/src/parsers/hailuo_parser.py +++ b/src/parsers/hailuo_parser.py @@ -27,6 +27,7 @@ class HailuoParser(BaseParser): "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", } self.title = None + self.description = None self.cover_url = None self.video_url = None self.video_list = [] @@ -100,7 +101,8 @@ class HailuoParser(BaseParser): def _apply_video_asset(self, asset): # 1. 提取标题 / 描述 - self.title = asset.get("title") or asset.get("desc") or "海螺AI 作品" + self.title = asset.get("title") or None + self.description = asset.get("desc") or None # 2. 提取视频直链(优先提取去除品牌大标的直链) video_urls_obj = asset.get("videoURLs") or {} @@ -164,7 +166,8 @@ class HailuoParser(BaseParser): return None def _apply_ld_video(self, ld_video): - self.title = ld_video.get("description") or ld_video.get("name") or "海螺AI 作品" + self.title = ld_video.get("name") or None + self.description = ld_video.get("description") or None self.video_url = ld_video.get("contentUrl") if self.video_url: self.video_list = [self.video_url] @@ -182,7 +185,10 @@ class HailuoParser(BaseParser): return self.video_url def get_title_content(self): - return self.title or "海螺AI 作品" + return self.title + + def get_description(self): + return self.description def get_cover_photo_url(self): return self.cover_url diff --git a/src/parsers/jimeng_parser.py b/src/parsers/jimeng_parser.py index 55ce5a4..5b371e7 100644 --- a/src/parsers/jimeng_parser.py +++ b/src/parsers/jimeng_parser.py @@ -101,7 +101,8 @@ class JimengParser(BaseParser): author_id = author.get("uid") or author.get("sec_uid") or "" return { - "title": common.get("description") or "即梦AI 视频", + "title": common.get("title") or None, + "desc": common.get("description") or None, "video_url": primary_video, "video_list": [primary_video] if primary_video else [], "cover_url": cover_url, @@ -149,7 +150,10 @@ class JimengParser(BaseParser): return self.data.get("video_list") or [] def get_title_content(self): - return self.data.get("title") or "" + return self.data.get("title") or None + + def get_description(self): + return self.data.get("desc") or None def get_cover_photo_url(self): return self.data.get("cover_url") diff --git a/src/parsers/kuaishou_parser.py b/src/parsers/kuaishou_parser.py index 5c19746..cc170d5 100644 --- a/src/parsers/kuaishou_parser.py +++ b/src/parsers/kuaishou_parser.py @@ -358,19 +358,21 @@ class KuaishouParser(BaseParser): return None def get_title_content(self): + return None + + def get_description(self): try: photo_key = f"VisionVideoDetailPhoto:{self.video_id}" if self.page_type == "VIDEO": - title = self.client.get(photo_key, {}).get('caption', '') - if title: - return title + caption = self.client.get(photo_key, {}).get('caption', '') + if caption: + return caption if self.page_type in ("ATLAS", "VIDEO"): payload = self._get_atlas_payload() - return payload.get("photo", {}).get("caption", "") + return payload.get("photo", {}).get("caption") or None except Exception as e: - logger.warning(f"Failed to parse title content: {e}") - pass - return "" + logger.warning(f"Failed to parse Kuaishou description: {e}") + return None def get_cover_photo_url(self): try: diff --git a/src/parsers/kugou_music_parser.py b/src/parsers/kugou_music_parser.py index 64ad807..0d84260 100644 --- a/src/parsers/kugou_music_parser.py +++ b/src/parsers/kugou_music_parser.py @@ -153,7 +153,7 @@ class KugouMusicParser(BaseParser): return isinstance(url, str) and url.startswith(("http://", "https://")) def get_real_video_url(self): - return self.video_list[0] if self.video_list else None + return None def get_video_list(self): return self.video_list diff --git a/src/parsers/kwaiying_parser.py b/src/parsers/kwaiying_parser.py index 80b1e25..7cbf948 100644 --- a/src/parsers/kwaiying_parser.py +++ b/src/parsers/kwaiying_parser.py @@ -94,12 +94,11 @@ class KwaiyingParser(BaseParser): return self.resource_data.get("videoUrl") def get_title_content(self): - """提取模板标题或文案描述。""" - return ( - self.resource_data.get("name") - or (self.resource_data.get("templateBean") or {}).get("description") - or "快影模板" - ) + """提取模板独立标题。""" + return self.resource_data.get("name") or None + + def get_description(self): + return (self.resource_data.get("templateBean") or {}).get("description") or None def get_cover_photo_url(self): """提取封面图地址。""" diff --git a/src/parsers/lofter_parser.py b/src/parsers/lofter_parser.py index 76bde46..65e7a7b 100644 --- a/src/parsers/lofter_parser.py +++ b/src/parsers/lofter_parser.py @@ -21,6 +21,7 @@ class LofterParser(BaseParser): "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", } self.title = "" + self.description = None self.cover_url = None self.video_url = None self.image_list = [] @@ -87,8 +88,6 @@ class LofterParser(BaseParser): img_url = img.get("orign") or img.get("raw") if img_url: self.image_list.append(img_url) - if self.image_list: - self.cover_url = self.image_list[0] return # 2. 标准博客文章详情 (postData) @@ -130,16 +129,12 @@ class LofterParser(BaseParser): first_img = photo_post_view.get("firstImage") or post_view.get("firstImage") if isinstance(first_img, dict): self.cover_url = first_img.get("raw") or first_img.get("orign") - elif not self.cover_url and self.image_list: - self.cover_url = self.image_list[0] if not caption: caption = photo_post_view.get("caption") or "" # 标题整理 - if not title and caption: - clean_caption = re.sub(r'<[^>]+>', '', caption).strip() - title = clean_caption + self.description = re.sub(r'<[^>]+>', '', caption).strip() or None self.title = title.strip() @@ -171,15 +166,15 @@ class LofterParser(BaseParser): if src and "lf127.net" in src and "ava" not in src: self.image_list.append(src) - if not self.cover_url and self.image_list: - self.cover_url = self.image_list[0] - def get_real_video_url(self): return self.video_url def get_title_content(self): return self.title or "" + def get_description(self): + return self.description + def get_cover_photo_url(self): return self.cover_url diff --git a/src/parsers/netease_music_parser.py b/src/parsers/netease_music_parser.py index 7c99481..531d43c 100644 --- a/src/parsers/netease_music_parser.py +++ b/src/parsers/netease_music_parser.py @@ -158,10 +158,6 @@ class NeteaseMusicParser(BaseParser): song = content.get("song") or {} self.title = content.get("title") or content.get("msg") or song.get("name") or "" self.image_list = self._extract_event_images(payload.get("pics") or []) - if self.image_list: - first = self.image_list[0] - self.cover_url = first.get("url") if isinstance(first, dict) else first - video_data = content.get("videoData") or {} self.video_list = self._extract_nested_video_urls(video_data) if song.get("id"): @@ -338,7 +334,7 @@ class NeteaseMusicParser(BaseParser): return list(dict.fromkeys(value for value in values if value)) def get_real_video_url(self): - return self.video_list[0] if self.video_list else None + return None def get_video_list(self): return self.video_list diff --git a/src/parsers/peiyinxiu_parser.py b/src/parsers/peiyinxiu_parser.py index ec29989..26ba29d 100644 --- a/src/parsers/peiyinxiu_parser.py +++ b/src/parsers/peiyinxiu_parser.py @@ -85,7 +85,7 @@ class PeiyinxiuParser(BaseParser): return [url] if url else [] def get_real_video_url(self): - return self.video_list[0] if self.video_list else None + return None def get_video_list(self): return self.video_list diff --git a/src/parsers/pinduoduo_parser.py b/src/parsers/pinduoduo_parser.py index e71301e..3404d05 100644 --- a/src/parsers/pinduoduo_parser.py +++ b/src/parsers/pinduoduo_parser.py @@ -86,9 +86,6 @@ class PinduoduoParser(BaseParser): self._parse_goods_detail() # 4. 保障默认兜底字段 - if self.image_list and not self.cover_url: - self.cover_url = self.image_list[0] - if not self.title: if self.goods_id: if self.review_id: diff --git a/src/parsers/pinecone_moment_parser.py b/src/parsers/pinecone_moment_parser.py index beb5ea8..65b2e30 100644 --- a/src/parsers/pinecone_moment_parser.py +++ b/src/parsers/pinecone_moment_parser.py @@ -97,7 +97,7 @@ class PineconeMomentParser(BaseParser): return list(dict.fromkeys(url for url in urls if cls._valid_url(url))) def get_real_video_url(self): - return self.video_list[0] if self.video_list else None + return None def get_video_list(self): return self.video_list diff --git a/src/parsers/pipigaoxiao_parser.py b/src/parsers/pipigaoxiao_parser.py index 80a3c9c..efab694 100644 --- a/src/parsers/pipigaoxiao_parser.py +++ b/src/parsers/pipigaoxiao_parser.py @@ -54,12 +54,15 @@ class PipigaoxiaoParser(BaseParser): return None def get_title_content(self): + return None + + def get_description(self): try: - if not self.data: return "" - return self.data.get('data', {}).get('post', {}).get('content', '') + if not self.data: return None + return self.data.get('data', {}).get('post', {}).get('content', '') or None except Exception as e: - logger.warning(f"Failed to parse Pipigaoxiao title content: {e}") - return "" + logger.warning(f"Failed to parse Pipigaoxiao description: {e}") + return None def get_cover_photo_url(self): try: diff --git a/src/parsers/pipixia_parser.py b/src/parsers/pipixia_parser.py index a3839b7..09b0622 100644 --- a/src/parsers/pipixia_parser.py +++ b/src/parsers/pipixia_parser.py @@ -75,11 +75,14 @@ class PipixiaParser(BaseParser): try: return self.data["data"]["cell_comments"][0]["comment_info"]["item"]["cover"]["url_list"][0]["url"] except: return None def get_title_content(self): + return self.page_title or None + + def get_description(self): try: content = self.data["data"]["cell_comments"][0]["comment_info"]["item"].get("content", "") - return content or self.page_title + return content or None except (KeyError, TypeError, IndexError): - return self.page_title + return None def get_author_info(self): try: author = self.data["data"]["cell_comments"][0]["comment_info"]["item"]["author"] diff --git a/src/parsers/qianwen_parser.py b/src/parsers/qianwen_parser.py index 531bf17..113e68b 100644 --- a/src/parsers/qianwen_parser.py +++ b/src/parsers/qianwen_parser.py @@ -114,12 +114,7 @@ class QianwenParser(BaseParser): img_urls.extend(deep_images) self.image_list = list(dict.fromkeys(img_urls)) - if self.image_list: - self.cover_url = self.image_list[0] - self.video_list = list(dict.fromkeys(video_urls)) - if self.video_list: - self.video_url = self.video_list[0] except Exception as exc: logger.warning("Failed to parse Qianwen share page: %s", exc) diff --git a/src/parsers/qqmusic_parser.py b/src/parsers/qqmusic_parser.py index 0235b7b..57e48a2 100644 --- a/src/parsers/qqmusic_parser.py +++ b/src/parsers/qqmusic_parser.py @@ -188,7 +188,7 @@ class QQMusicParser(BaseParser): return list(dict.fromkeys(url for _, url in streams)) def get_real_video_url(self): - return self.video_list[0] if self.video_list else None + return None def get_video_list(self): return self.video_list diff --git a/src/parsers/soul_parser.py b/src/parsers/soul_parser.py index e3b02f1..17994fb 100644 --- a/src/parsers/soul_parser.py +++ b/src/parsers/soul_parser.py @@ -95,7 +95,10 @@ class SoulParser(BaseParser): return self._normalize_url(self._get_primary_video().get("fileUrl")) def get_title_content(self): - return self.post.get("content") or "Soul 帖子" + return self.post.get("title") or None + + def get_description(self): + return self.post.get("content") or None def get_cover_photo_url(self): attachment = self._get_primary_video() diff --git a/src/parsers/toutiao_parser.py b/src/parsers/toutiao_parser.py index 46ab918..dfa15d8 100644 --- a/src/parsers/toutiao_parser.py +++ b/src/parsers/toutiao_parser.py @@ -108,10 +108,15 @@ class ToutiaoParser(DouyinParser): def get_title_content(self): if self.data and isinstance(self.data, dict) and "toutiao_article_info" in self.data: article_info = self.data["toutiao_article_info"] - return article_info.get("title") or article_info.get("content") or "" + return article_info.get("title") or None return super().get_title_content() + def get_description(self): + if self.data and isinstance(self.data, dict) and "toutiao_article_info" in self.data: + return self.data["toutiao_article_info"].get("content") or None + return super().get_description() + def get_cover_photo_url(self): if self.data and isinstance(self.data, dict) and "toutiao_article_info" in self.data: article_info = self.data["toutiao_article_info"] diff --git a/src/parsers/wechat_channels_parser.py b/src/parsers/wechat_channels_parser.py index 57306a4..e255811 100644 --- a/src/parsers/wechat_channels_parser.py +++ b/src/parsers/wechat_channels_parser.py @@ -164,7 +164,8 @@ class WeChatChannelsParser(BaseParser): audio_url = (feed.get("bgmInfo") or {}).get("bgmUrl") return { - "title": feed.get("description") or "视频号", + "title": feed.get("title") or None, + "desc": feed.get("description") or None, "video_url": video_url, "video_list": [video_url] if video_url else [], "cover_url": feed.get("coverUrl"), @@ -186,6 +187,9 @@ class WeChatChannelsParser(BaseParser): def get_title_content(self): return self.data["title"] + def get_description(self): + return self.data.get("desc") or None + def get_cover_photo_url(self): return self.data["cover_url"] diff --git a/src/parsers/wechat_mp_parser.py b/src/parsers/wechat_mp_parser.py index b127928..6be162d 100644 --- a/src/parsers/wechat_mp_parser.py +++ b/src/parsers/wechat_mp_parser.py @@ -84,6 +84,7 @@ class WechatMpParser(BaseParser): # 4. 正文高清全量插图 (将 /640? 或 /300? 升级为原始画质 /0?) image_list = [] content_box = soup.find("div", id="js_content") or soup + description = content_box.get_text("\n", strip=True) or None for img in content_box.find_all("img"): src = img.get("data-src") or img.get("src") if src and src.startswith("http") and "qpic.cn" in src: @@ -178,6 +179,7 @@ class WechatMpParser(BaseParser): return { "title": title, + "desc": description, "author": { "nickname": author_name, "author_id": author_id, @@ -201,6 +203,10 @@ class WechatMpParser(BaseParser): """提取文章标题。""" return self.article_data.get("title") or "微信公众号文章" + def get_description(self): + """提取已去除 HTML 标记的文章正文。""" + return self.article_data.get("desc") or None + def get_cover_photo_url(self): """提取文章封面图。""" return self.article_data.get("cover_url") diff --git a/src/parsers/weibo_parser.py b/src/parsers/weibo_parser.py index fb61335..08b5f66 100644 --- a/src/parsers/weibo_parser.py +++ b/src/parsers/weibo_parser.py @@ -257,15 +257,17 @@ class WeiboParser(BaseParser): return None def get_title_content(self): + return self.post_data.get('title') or None + + def get_description(self): content = ( self.post_data.get('text_raw', '') or self.post_data.get('text', '') or self.post_data.get('content', '') - or self.post_data.get('title', '') ) # Simple cleanup if there is HTML content = re.sub(r'<[^>]+>', '', content) - return content + return content or None def get_cover_photo_url(self): try: diff --git a/src/parsers/xiaohongshu_parser.py b/src/parsers/xiaohongshu_parser.py index 77c80e3..55fdf70 100644 --- a/src/parsers/xiaohongshu_parser.py +++ b/src/parsers/xiaohongshu_parser.py @@ -71,19 +71,13 @@ class XiaohongshuParser(BaseParser): return None def get_title_content(self): - title = self.note_data.get('title', '') - desc = self.note_data.get('desc', '') - return f"{title}\n{desc}".strip() + return self.note_data.get('title', '') or None + + def get_description(self): + return self.note_data.get('desc', '') or None def get_cover_photo_url(self): - try: - image_list = self.note_data.get('imageList', []) - if image_list: - cover_url = image_list[0].get('urlDefault', '') - return cover_url.replace("\\u002F", "/") - return None - except (KeyError, IndexError): - return None + return None def get_image_list(self): image_url_list = [] diff --git a/src/parsers/xiaoyunque_parser.py b/src/parsers/xiaoyunque_parser.py index e78e8e6..a87f15f 100644 --- a/src/parsers/xiaoyunque_parser.py +++ b/src/parsers/xiaoyunque_parser.py @@ -30,6 +30,7 @@ class XiaoyunqueParser(BaseParser): } self.data = { "title": "", + "desc": None, "video_url": None, "video_list": [], "cover_url": None, @@ -81,7 +82,8 @@ class XiaoyunqueParser(BaseParser): user_info = generate_page.get("user_info") or {} item_info = generate_page.get("item_info") or {} - title = item_info.get("desc") or item_info.get("title") or "小云雀AI 作品" + title = item_info.get("title") or None + desc = item_info.get("desc") or None # 图片列表 image_info = item_info.get("image_info") or [] @@ -102,9 +104,6 @@ class XiaoyunqueParser(BaseParser): # 封面图 cover_url = item_info.get("cover_url") - if not cover_url and image_list: - cover_url = image_list[0] - author = { "nickname": user_info.get("nick_name") or "", "author_id": str(user_info.get("user_id") or user_info.get("sec_uid") or ""), @@ -113,6 +112,7 @@ class XiaoyunqueParser(BaseParser): return { "title": title, + "desc": desc, "video_url": video_url, "video_list": video_list, "cover_url": cover_url, @@ -127,7 +127,10 @@ class XiaoyunqueParser(BaseParser): return self.data.get("video_list") or [] def get_title_content(self): - return self.data.get("title") or "" + return self.data.get("title") or None + + def get_description(self): + return self.data.get("desc") or None def get_cover_photo_url(self): return self.data.get("cover_url") diff --git a/src/parsers/xigua_parser.py b/src/parsers/xigua_parser.py index 84b5d8a..0b2335a 100644 --- a/src/parsers/xigua_parser.py +++ b/src/parsers/xigua_parser.py @@ -107,10 +107,15 @@ class XiguaParser(DouyinParser): def get_title_content(self): if self.data and isinstance(self.data, dict) and "xigua_article_info" in self.data: article_info = self.data["xigua_article_info"] - return article_info.get("title") or article_info.get("content") or "" + return article_info.get("title") or None return super().get_title_content() + def get_description(self): + if self.data and isinstance(self.data, dict) and "xigua_article_info" in self.data: + return self.data["xigua_article_info"].get("content") or None + return super().get_description() + def get_cover_photo_url(self): if self.data and isinstance(self.data, dict) and "xigua_article_info" in self.data: article_info = self.data["xigua_article_info"] diff --git a/src/parsers/xinpianchang_parser.py b/src/parsers/xinpianchang_parser.py index b93c4f8..5c7fd4a 100644 --- a/src/parsers/xinpianchang_parser.py +++ b/src/parsers/xinpianchang_parser.py @@ -87,8 +87,6 @@ class XinpianchangParser(BaseParser): if isinstance(p, dict) and p.get("url") and p["url"].startswith("http") ] self.video_list = list(dict.fromkeys(urls)) - if self.video_list: - self.video_url = self.video_list[0] except Exception as exc: logger.warning("Failed to fetch Xinpianchang data: %s", exc) diff --git a/src/parsers/yuanbao_parser.py b/src/parsers/yuanbao_parser.py index 2ac0f51..56be586 100644 --- a/src/parsers/yuanbao_parser.py +++ b/src/parsers/yuanbao_parser.py @@ -95,9 +95,6 @@ class YuanbaoParser(BaseParser): self.video_list = self._unique(self.video_list) self.image_list = self._unique(self.image_list) - self.video_url = self.video_list[0] if self.video_list else None - if not self.cover_url: - self.cover_url = self.image_list[0] if self.image_list else None def _parse_chat_share(self, payload): chat = payload.get("chat") or {} diff --git a/src/parsers/zhihu_parser.py b/src/parsers/zhihu_parser.py index 4a8ed30..e13b572 100644 --- a/src/parsers/zhihu_parser.py +++ b/src/parsers/zhihu_parser.py @@ -120,16 +120,18 @@ class ZhihuParser(BaseParser): def get_title_content(self): question = self.data.get("question") or {} - title = question.get("title") or self.data.get("title") or self.data.get("excerpt_title") or "" + return question.get("title") or self.data.get("title") or self.data.get("excerpt_title") or None + + def get_description(self): excerpt = self.data.get("excerpt") or "" - if title or excerpt: - return "\n".join(part for part in (title, excerpt) if part).strip() + if excerpt: + return excerpt content_html = self.data.get("content_html") or self.data.get("content") or "" if isinstance(content_html, str): text = BeautifulSoup(content_html, "html.parser").get_text(" ", strip=True) if text: return text - return "知乎视频" if self._pin_video() else "知乎内容" + return None def get_cover_photo_url(self): pin_video = self._pin_video() diff --git a/src/parsers/zuiyou_parser.py b/src/parsers/zuiyou_parser.py index 76841ac..b8e086d 100644 --- a/src/parsers/zuiyou_parser.py +++ b/src/parsers/zuiyou_parser.py @@ -40,8 +40,9 @@ class ZuiyouParser(BaseParser): return data["videos"][video_key]["url"] except: return None def get_cover_photo_url(self): return None - def get_title_content(self): - try: return self.data["data"]["post"]["content"] + def get_title_content(self): return None + def get_description(self): + try: return self.data["data"]["post"]["content"] or None except: return None def get_author_info(self): try: diff --git a/templates/landing.html b/templates/landing.html index cb7fc0f..c17890e 100644 --- a/templates/landing.html +++ b/templates/landing.html @@ -88,6 +88,7 @@