Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ feat: 元数据支持指定premiered的格式 #145

Merged
merged 7 commits into from
Jun 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ yutto <url> -c "d8bc7493%2C2843925707%2C08c3e*81"
|username|UP 主用户名|个人空间、收藏夹、稍后再看、合集、视频列表下载|
|series_title|合集标题|收藏夹、视频合集、视频列表下载|
|pubdate🕛|投稿日期|仅投稿视频|
|download_date|下载日期|全部|
|download_date🕛|下载日期|全部|
|owner_uid|UP 主UID|个人空间、收藏夹、稍后再看、合集、视频列表下载|

> **Note**
Expand Down Expand Up @@ -410,6 +410,14 @@ cat ~/.yutto_alias | yutto tensura-nikki --batch --alias-file -
- 参数 `--metadata-only`
- 默认值 `False`

#### 指定媒体元数据值的格式

当前仅支持 `premiered`

- 参数 `--metadata-format-premiered`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lc4t 我这边不太了解 metadata 相关的,请问我直接改成 --metadata-date-format 是否合适呢?因为还有一个 dateadded 字段,只为其中一个添加 format 是不是不太合适?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lc4t 我这边不太了解 metadata 相关的,请问我直接改成 --metadata-date-format 是否合适呢?因为还有一个 dateadded 字段,只为其中一个添加 format 是不是不太合适?

目前争议在于premiered,是因为不同APP中识别premiered的格式不同,至于dateadded,这个字段的含义是添加日期,不太确定是否会遇到争议(现在没遇到);
更稳妥的做法是整个metadata格式都开放自定义,例如--metadata-{field}-format={formatter}这样子;
另外未来可能遇到需要添加metadata自定义字段,比如用户需要手动添加一个tag=bilibili

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的了解~那就不改了

麻烦看一下这个 PR 修改后有没有什么问题~没有的话我就 merge 啦~

- 默认值 `"%Y-%m-%d"`
- 常用值 `"%Y-%m-%d %H:%M:%S"`
Copy link
Contributor Author

@lc4t lc4t Jun 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

奇怪..为什么要带上引号.. 虽然用户输入确实一般是--metadata-format-premiered="%Y-%m-%d %H:%M:%S"但是双引号本身并不是值的一部分

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

唔,这里目前是对齐现有的文档,是用来表示 Python 里表示字面量的方法,不是用户在传入时的写法,不过你说的也有道理,也许默认值使用用户的写法是更为友好的,但整体上都需要修改了

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

都ok。我是担心有用户复制的时候..多复制了引号(


#### 严格校验大会员状态有效

- 参数 `--vip-strict`
Expand Down
8 changes: 8 additions & 0 deletions yutto/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
from yutto.utils.console.logger import Badge, Logger
from yutto.utils.fetcher import Fetcher
from yutto.utils.funcutils import as_sync
from yutto.utils.time import TIME_DATE_FMT, TIME_FULL_FMT
from yutto.validator import (
initial_validation,
validate_basic_arguments,
Expand Down Expand Up @@ -109,6 +110,9 @@ def cli() -> argparse.ArgumentParser:
group_common.add_argument(
"-af", "--alias-file", type=argparse.FileType("r", encoding="utf-8"), help="设置 url 别名文件路径"
)
group_common.add_argument(
"--metadata-format-premiered", default=TIME_DATE_FMT, help="专用于 metadata 文件中 premiered 字段的日期格式"
)

# 资源选择
group_common.add_argument(
Expand Down Expand Up @@ -286,6 +290,10 @@ async def run(args_list: list[argparse.Namespace]):
"overwrite": args.overwrite,
"block_size": int(args.block_size * 1024 * 1024),
"num_workers": args.num_workers,
"metadata_format": {
"premiered": args.metadata_format_premiered,
"dateadded": TIME_FULL_FMT,
},
},
)
Logger.new_line()
Expand Down
1 change: 1 addition & 0 deletions yutto/_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ class DownloaderOptions(TypedDict):
overwrite: bool
block_size: int
num_workers: int
metadata_format: dict[str, str]


class FavouriteMetaData(TypedDict):
Expand Down
6 changes: 3 additions & 3 deletions yutto/api/bangumi.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from yutto.utils.console.logger import Logger
from yutto.utils.fetcher import Fetcher
from yutto.utils.metadata import MetaData
from yutto.utils.time import get_time_str_by_now, get_time_str_by_stamp
from yutto.utils.time import get_time_stamp_by_now


class BangumiListItem(TypedDict):
Expand Down Expand Up @@ -165,8 +165,8 @@ def _parse_bangumi_metadata(item: dict[str, Any]) -> MetaData:
show_title=item["share_copy"],
plot=item["share_copy"],
thumb=item["cover"],
premiered=get_time_str_by_stamp(item["pub_time"], "%Y-%m-%d"),
dateadded=get_time_str_by_now(),
premiered=item["pub_time"],
dateadded=get_time_stamp_by_now(),
source="", # TODO
actor=[], # TODO
genre=[], # TODO
Expand Down
6 changes: 3 additions & 3 deletions yutto/api/cheese.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
from yutto.utils.console.logger import Logger
from yutto.utils.fetcher import Fetcher
from yutto.utils.metadata import MetaData
from yutto.utils.time import get_time_str_by_now, get_time_str_by_stamp
from yutto.utils.time import get_time_stamp_by_now


class CheeseListItem(TypedDict):
Expand Down Expand Up @@ -138,8 +138,8 @@ def _parse_cheese_metadata(item: dict[str, Any]) -> MetaData:
show_title=item["title"], # 无此字段,用 title 代替
plot=item["title"], # 无此字段,用 title 代替
thumb=item["cover"],
premiered=get_time_str_by_stamp(item["release_date"], "%Y-%m-%d"),
dateadded=get_time_str_by_now(),
premiered=item["release_date"],
dateadded=get_time_stamp_by_now(),
source="", # TODO
actor=[], # TODO
genre=[], # TODO
Expand Down
6 changes: 3 additions & 3 deletions yutto/api/ugc_video.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from yutto.utils.console.logger import Logger
from yutto.utils.fetcher import Fetcher
from yutto.utils.metadata import Actor, MetaData
from yutto.utils.time import get_time_str_by_now, get_time_str_by_stamp
from yutto.utils.time import get_time_stamp_by_now


class _UgcVideoPageInfo(TypedDict):
Expand Down Expand Up @@ -271,8 +271,8 @@ def _parse_ugc_video_metadata(
show_title=page_info["part"],
plot=video_info["description"],
thumb=page_info["first_frame"] if page_info["first_frame"] is not None else video_info["picture"],
premiered=get_time_str_by_stamp(video_info["pubdate"], "%Y-%m-%d"),
dateadded=get_time_str_by_now(),
premiered=video_info["pubdate"],
dateadded=get_time_stamp_by_now(),
actor=video_info["actor"],
genre=video_info["genre"],
tag=video_info["tag"],
Expand Down
3 changes: 2 additions & 1 deletion yutto/processor/downloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,7 @@ async def start_downloader(
filename = episode_data["filename"]
require_video = options["require_video"]
require_audio = options["require_audio"]
metadata_format = options["metadata_format"]

Logger.info(f"开始处理视频 {filename}")
tmp_dir.mkdir(parents=True, exist_ok=True)
Expand Down Expand Up @@ -260,7 +261,7 @@ async def start_downloader(

# 保存媒体描述文件
if metadata is not None:
write_metadata(metadata, output_path)
write_metadata(metadata, output_path, metadata_format)
Logger.custom("NFO 媒体描述文件已生成", badge=Badge("描述文件", fore="black", back="cyan"))

if output_path.exists():
Expand Down
2 changes: 1 addition & 1 deletion yutto/processor/path_resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def resolve_path_template(
subpath_variables[key] = repair_filename(value)

# 将时间变量转换为对应的时间格式
time_vars: list[PathTemplateVariable] = ["pubdate"] # TODO: add download_date
time_vars: list[PathTemplateVariable] = ["pubdate", "download_date"]
for var in time_vars:
value = subpath_variables.pop(var)
if value == UNKNOWN:
Expand Down
27 changes: 20 additions & 7 deletions yutto/utils/metadata.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
from __future__ import annotations

from pathlib import Path
from typing import TypedDict
from xml.dom.minidom import parseString # type: ignore
from typing import Any, TypedDict

from dict2xml import dict2xml # type: ignore

from yutto.utils.time import get_time_str_by_stamp


class Actor(TypedDict):
name: str
Expand All @@ -20,8 +21,8 @@ class MetaData(TypedDict):
show_title: str
plot: str
thumb: str
premiered: str
dateadded: str
premiered: int
dateadded: int
actor: list[Actor]
genre: list[str]
tag: list[str]
Expand All @@ -30,9 +31,21 @@ class MetaData(TypedDict):
website: str


def write_metadata(metadata: MetaData, video_path: Path):
def metadata_value_format(metadata: MetaData, metadata_format: dict[str, str]) -> dict[str, Any]:
formatted_metadata: dict[str, Any] = {}
for key, value in metadata.items():
if key in metadata_format:
assert isinstance(value, int)
value = get_time_str_by_stamp(value, metadata_format[key])
formatted_metadata[key] = value
return formatted_metadata


def write_metadata(metadata: MetaData, video_path: Path, metadata_format: dict[str, str]):
metadata_path = video_path.with_suffix(".nfo")
custom_root = "episodedetails" # TODO: 不同视频类型使用不同的root name
xml_content = dict2xml(metadata, wrap=custom_root, indent=" ") # type: ignore
custom_root = "episodedetails" # TODO: 不同视频类型使用不同的 root name
# 增加字段格式化内容,后续如果需要调整可以继续调整
user_formatted_metadata = metadata_value_format(metadata, metadata_format) if metadata_format else metadata
xml_content = dict2xml(user_formatted_metadata, wrap=custom_root, indent=" ") # type: ignore
with metadata_path.open("w", encoding="utf-8") as f: # type: ignore
f.write(xml_content) # type: ignore
15 changes: 12 additions & 3 deletions yutto/utils/time.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,23 @@

import time

TIME_FMT = "%Y-%m-%d %H:%M:%S"
TIME_FULL_FMT = "%Y-%m-%d %H:%M:%S"
TIME_DATE_FMT = "%Y-%m-%d"


def get_time_str_by_now(fmt: str = TIME_FMT):
def get_time_stamp_by_now() -> int:
return int(time.time())


def get_time_str_by_now(fmt: str = TIME_FULL_FMT):
time_stamp_now = time.time()
return get_time_str_by_stamp(time_stamp_now, fmt)


def get_time_str_by_stamp(stamp: float, fmt: str = TIME_FMT):
def get_time_str_by_stamp(stamp: float, fmt: str = TIME_FULL_FMT):
local_time = time.localtime(stamp)
return time.strftime(fmt, local_time)


def get_time_struct_by_stamp(stamp: float):
return time.localtime(stamp)