v1.25.0

Changes

• [e621] split e621 extractors from danbooru module (#3425)
• [deviantart] remove mature scraps warning (#3691)
• [deviantart] use /collections/all endpoint for favorites (#3666 ,#3668)
• [newgrounds] update default image and audio archive IDs to prevent ID overlap (#3681)
• rename --ignore-config to --config-ignore

Extractors

• [catbox] add file extractor (#3570)
• [deviantart] add search extractor (#538, #1264, #2954, #2970, #3577)
• [deviantart] add gallery-search extractor (#1695)
• [deviantart] support fxdeviantart.com URLs (##3740)
• [e621] implement notes and pools metadata extraction (#3425)
• [gelbooru] add favorite extractor (#3704)
• [imagetwist] support phun.imagetwist.com and imagehaha.com domains (#3622)
• [instagram] add user metadata field (#3107)
• [manganelo] update and fix metadata extraction
• [manganelo] support mobile-only chapters
• [mangasee] extract author and genre metadata (#3703)
• [misskey] add misskey extractors (#3717)
• [pornpics] add gallery and search extractors (#263, #3544, #3654)
• [redgifs] support v3 URLs (#3588. #3589)
• [redgifs] add collection extractors (#3427, #3662)
• [shopify] support ohpolly.com (#440, #3596)
• [szurubooru] add tag and post extractors (#3583, #3713)
• [twitter] add transform option

Options

• [postprocessor:metadata] add sort and separators options
• [postprocessor:exec] implement archive options (#3584)
• add --config-create command-line option (#2333)
• add --config-toml command-line option to load config files in TOML format
• add output.stdout, output.stdin, and output.stderr options (#1621, #2152, #2529)
• add hash_md5 and hash_sha1 functions (#3679)
• implement globals option to enable defining custom functions for eval statements
• implement archive-pragma option to use SQLite PRAGMA statements
• implement actions to trigger events on logging messages (#3338, #3630)
• implement ability to load external extractor classes
  • -X/--extractors command-line options
  • extractor.modules-sources config option

Fixes

• [bunkr] fix extraction (#3636, #3655)
• [danbooru] send gallery-dl User-Agent (#3665)
• [deviantart] fix crash when handling deleted deviations in status updates (#3656)
• [fanbox] fix crash with missing images (#3673)
• [imagefap] update gallery URLs (#3595)
• [imagefap] fix infinite pagination loop (#3594)
• [imagefap] fix metadata extraction
• [oauth] use default name for browsers without name attribute
• [pinterest] unescape search terms (#3621)
• [pixiv] fix --write-tags for "tags": "original" (#3675)
• [poipiku] warn about incorrect passwords (#3646)
• [reddit] update videos option (#3712)
• [soundgasm] rewrite (#3578)
• [telegraph] fix extraction when images are not in <figure> elements (#3590)
• [tumblr] raise more detailed errors for dashboard-only blogs (#3628)
• [twitter] fix some original retweets not downloading (#3744)
• [ytdl] fix --parse-metadata (#3663)
• [downloader:ytdl] prevent exception on empty results

Improvements

• [downloader:http] use time.monotonic()
• [downloader:http] update _http_retry to accept a Python function (#3569)
• [postprocessor:metadata] speed up JSON encoding
• replace json.loads/dumps with direct calls to JSONDecoder.decode/JSONEncoder.encode
• improve option.Formatter performance

Removals

• [nitter] remove nitter.pussthecat.org

v1.24.5

Additions

• [booru] add url option
• [danbooru] extend metadata option (#3505)
• [deviantart] add extractor for status updates (#3539, #3541)
• [deviantart] add support for /deviation/ and fav.me URLs (#3558, #3560)
• [kemonoparty] extract hash metadata for discord files (#3531)
• [lexica] add search extractor (#3567)
• [mastodon] add num and count metadata fields (#3517)
• [nudecollect] add image and album extractors (#2430, #2818, #3575)
• [wikifeet] add gallery extractor (#519, #3537)
• [downloader:http] add signature checks for .blend, .obj, and .clip files (#3535)
• add extractor.retry-codes option
• add -O/--postprocessor-option command-line option (#3565)
• improve write-pages output

Fixes

• [bunkr] fix downloading .mkv and .ts files (#3571)
• [fantia] send X-CSRF-Token headers (#3576)
• [generic] fix regex for non-src image URLs (#3555)
• [hiperdex] update domain (#3572)
• [hotleak] fix video URLs (#3516, #3525, #3563, #3581)
• [instagram] always show cursor value after errors (#3440)
• [instagram] update API domain, headers, and csrf token handling
• [oauth] show client-id/api-key values (#3518)
• [philomena] match URLs with www subdomain
• [sankaku] update URL pattern (#3523)
• [twitter] refresh guest tokens (#3445, #3458)
• [twitter] fix search pagination (#3536, #3534, #3549)
• [twitter] use "browser": "firefox" by default (#3522)

v1.24.4

Additions

• [downloader:http] add validate option

Fixes

• [kemonoparty] fix regression from commit 473bd38 (#3519)

v1.24.3

Additions

• [danbooru] extract uploader metadata (#3457)
• [deviantart] initial implementation of username & password login for scraps (#1029)
• [fanleaks] add post and model extractors (#3468, #3474)
• [imagefap] add folder extractor (#3504)
• [lynxchan] support bbw-chan.nl (#3456, #3463)
• [pinterest] support All Pins boards (#2855, #3484)
• [pinterest] add domain option (#3484)
• [pixiv] implement metadata-bookmark option (#3417)
• [tcbscans] add chapter and manga extractors (#3189)
• [twitter] implement syndication=extended (#3483)
• implement slice notation for range options (#918, #2865)
• allow filter options to be a list of expressions

Fixes

• [behance] use delay between requests (#2507)
• [bunkr] fix URLs returned by API (#3481)
• [fanbox] return imageMap files in order (#2718)
• [imagefap] use delay between requests (#1140)
• [imagefap] warn about redirects to /human-verification (#1140)
• [kemonoparty] reject invalid/empty files (#3510)
• [myhentaigallery] handle whitespace before title tag (#3503)
• [poipiku] fix extraction for a different warning button style (#3493, #3460)
• [poipiku] warn about login requirements
• [telegraph] fix file URLs (#3506)
• [twitter] fix crash when using expand and syndication (#3473)
• [twitter] apply tweet type checks before uniqueness check (#3439, #3455)
• [twitter] force https:// for TwitPic URLs (#3449)
• [ytdl] adapt to yt-dlp changes
• update and improve documentation (#3453, #3462, #3496)

v1.24.2

Additions

• [2chen] support .club URLs (#3406)
• [deviantart] extract sta.sh URLs from text_content (#3366)
• [deviantart] add /view URL support (#3367)
• [e621] implement threshold option to control pagination (#3413)
• [fapello] add post, user and path extractors (#3065, #3360, #3415)
• [imgur] add support for imgur.io URLs (#3419)
• [lynxchan] add generic extractors for lynxchan imageboards (#3389, #3394)
• [mangafox] extract more metadata (#3167)
• [pixiv] extract date_url metadata (#3405)
• [soundgasm] add audio and user extractors (#3384, #3388)
• [webmshare] add video extractor (#2410)
• support Firefox containers for --cookies-from-browser (#3346)

Fixes

• [2chen] fix file URLs
• [bunkr] update domain (#3391)
• [exhentai] fix pagination
• [imagetwist] fix extraction
• [imgth] rewrite
• [instagram] prevent post date overwriting file date (#3392)
• [khinsider] fix metadata extraction
• [komikcast] update domain and fix extraction
• [reddit] increase id-max default value (#3397)
• [seiga] raise error when redirected to login page (#3401)
• [sexcom] fix video URLs (#3408, #3414)
• [twitter] update search pagination (#544)
• [warosu] fix and update
• [zerochan] update for layout v3
• restore paths for archived files (#3362, #3377)
• use util.NONE as keyword-default default value (#3334)

Removals

• [foolslide] remove kireicake
• [kissgoddess] remove module

v1.24.1

Additions

• [artstation] add pro-first option (#3273)
• [artstation] add max-posts option (#3270)
• [fapachi] add post and user extractors (#3339, #3347)
• [inkbunny] provide additional metadata (#3274)
• [nitter] add retweets option (#3278)
• [nitter] add videos option (#3279)
• [nitter] support /i/web/ and /i/user/ URLs (#3310)
• [pixhost] add gallery support (#3336, #3353)
• [weibo] add count metadata field (#3305)
• [downloader:http] add retry-codes option (#3313)
• [formatter] implement S format specifier to sort lists (#3266)
• implement version-metadata option (#3201)

Fixes

• [2chen] fix extraction (#3354, #3356)
• [bcy] fix JSONDecodeError (#3321)
• [bunkr] fix video downloads (#3326, #3335)
• [bunkr] use media-files servers for more file types
• [itaku] remove Extreme rating (#3285, #3287)
• [hitomi] apply format check for every image (#3280)
• [hotleak] fix UnboundLocalError (#3288, #3293)
• [nitter] sanitize filenames (#3294)
• [nitter] retry downloads on 404 (#3313)
• [nitter] set hlsPlayback cookie
• [patreon] fix 403 Forbidden errors (#3341)
• [patreon] improve campaign_id extraction (#3235)
• [patreon] update API query parameters
• [pixiv] preserve tags order (#3266)
• [reddit] use dash_url for videos (#3258, #3306)
• [twitter] fix error when using user IDs for suspended accounts
• [weibo] fix bug with empty playback_list (#3301)
• [downloader:http] fix potential ZeroDivisionError (#3328)

Removals

• [lolisafe] remove zz.ht

v1.24.0

Additions

• [exhentai] add metadata to search results (#3181)
• [gelbooru_v02] implement notes extraction
• [instagram] add guide extractor (#3192)
• [lolisafe] add support for xbunkr (#3153, #3156)
• [mastodon] add instance_remote metadata field (#3119)
• [nitter] add extractors for Nitter instances (#2415, #2696)
• [pixiv] add support for new daily AI rankings category (#3214, #3221)
• [twitter] add avatar and background extractors (#349, #3023)
• [uploadir] add support for uploadir.com (#3162)
• [wallhaven] add user extractor (#3212, #3213, #3226)
• [downloader:http] add chunk-size option (#3143)
• [downloader:http] add file signature check for .mp4 files
• [downloader:http] add file signature check and MIME type for .avif files
• [postprocessor] implement post-after event (#3117)
• [postprocessor:metadata] implement "mode": "jsonl"
• [postprocessor:metadata] add open, encoding, and private options
• add --chunk-size command-line option (#3143)
• add --user-agent command-line option
• implement http-metadata option
• implement "user-agent": "browser" (#2636)

Changes

• [deviantart] restore cookies warning for mature scraps (#3129)
• [instagram] use REST API for unauthenticated users by default
• [downloader:http] increase default chunk-size to 32768 bytes (#3143)
• build Windows executables using py2exe's new freeze() API
• build executables on GitHub Actions with Python 3.11
• reword error text for unsupported URLs

Fixes

• [exhentai] fix pagination (#3181)
• [khinsider] fix extraction (#3215, #3219)
• [realbooru] fix download URLs (#2530)
• [realbooru] fix tags extraction (#2530)
• [tumblr] fall back to gifv when possible (#3095, #3159)
• [twitter] fix login (#3220)
• [twitter] update URL for syndication API (#3160)
• [weibo] send Referer headers (#3188)
• [ytdl] update parse_bytes location (#3256)

Improvements

• [imxto] extract additional metadata (#3118, #3175)
• [instagram] allow downloading avatars for private profiles (#3255)
• [pixiv] raise error for invalid search/ranking parameters (#3214)
• [twitter] update bookmarks pagination (#3172)
• [downloader:http] refactor file signature checks
• [downloader:http] improve -r/--limit-rate accuracy (#3143)
• add loaded config files to debug output
• improve -K output for lists

Removals

• [instagram] remove login support (#3139, #3141, #3191)
• [instagram] remove channel extractor
• [ngomik] remove module

v1.23.5

Fixes

• [instagram] fix AttributeError on user stories extraction (#3123)

v1.23.4

Additions

• [aibooru] add support for aibooru.online (#3075)
• [instagram] add avatar extractor (#929, #1097, #2992)
• [instagram] support instagram.com/s/ highlight URLs (#3076)
• [instagram] extract coauthors metadata (#3107)
• [mangasee] add support for mangalife (#3086)
• [mastodon] add bookmark extractor (#3109)
• [mastodon] support cross-instance user references and /web/ URLs (#3109)
• [moebooru] implement notes extraction (#3094)
• [pixiv] extend metadata option (#3057)
• [reactor] match best, new, all URLs (#3073)
• [smugloli] add smugloli extractors (#3060)
• [tumblr] add fallback-delay and fallback-retries options (#2957)
• [vichan] add generic extractors for vichan imageboards

Fixes

• [bcy] fix extraction (#3103)
• [gelbooru] support alternate parameter order in post URLs (#2821)
• [hentai2read] support minor versions in chapter URLs (#3089)
• [hentaihere] support minor versions in chapter URLs
• [kemonoparty] fix dms extraction (#3106)
• [kemonoparty] update pagination offset
• [manganelo] update domain to chapmanganato.com (#3097)
• [pixiv] use exact_match_for_tags as default search mode (#3092)
• [redgifs] fix token extraction (#3080, #3081)
• [skeb] fix extraction (#3112)
• improve compatibility of DownloadArchive (#3078)

v1.23.3

Additions

• [2chen] Add 2chen.moe extractor (#2707)
• [8chan] add thread and board extractors (#2938)
• [deviantart] add group option (#3018)
• [fanbox] add content metadata field (#3020)
• [instagram] restore cursor functionality (#2991)
• [instagram] restore warnings for private profiles (#3004, #3045)
• [nana] add nana extractors (#2967)
• [nijie] add feed and followed extractors (#3048)
• [tumblr] support https://www.tumblr.com/BLOGNAME URLs (#3034)
• [tumblr] add offset option
• [vk] add tagged extractor (#2997)
• add path-extended option (#3021)
• emit debug logging messages before calling time.sleep() (#2982)

Changes

• [postprocessor:metadata] assume "mode": "custom" when format is given

Fixes

• [artstation] skip missing projects (#3016)
• [danbooru] fix ugoira metadata extraction (#3056)
• [deviantart] fix deviation extraction (#2981)
• [hitomi] fall back to webp when selected format is not available (#3030)
• [imagefap] fix and improve folder extraction and gallery pagination (#3013)
• [instagram] fix login (#3011, #3015)
• [nozomi] fix extraction (#3051)
• [redgifs] fix extraction (#3037)
• [tumblr] sleep between fallback retries (#2957)
• [vk] unescape error messages
• fix duplicated metadata bug with -j (#3033)
• fix bug when processing input file comments (#2808)