Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format Selection Bug: -f 137+136+140+139 should be -f 137/136+140/139 #680

Open
tsk-2222 opened this issue Jan 27, 2025 · 17 comments
Open
Labels
bug Something isn't working

Comments

@tsk-2222
Copy link

tsk-2222 commented Jan 27, 2025

Windows 10 Tartube: 2.5.062

I believe tartube is using the wrong format for the format selection option.

If I pick JUST these 4 formats, tartube will form this command '-f 137+136+140+139'
With that command, YT-DLP wants to find and download all 4 formats.
This causes downloads to fail or non-optimal formats to be downloaded depending on the exact list of formats.

As I understand the manual: The correct format should be '-f 137/136+140/139'
This will download one from the first set and 1 from the second set. Then combine them.

Here it a link to the appropriate section of the documents: (ctrl-f '137' to find a good example in the doc)

To reproduce the issue:
Start with this video: https://www.youtube.com/watch?v=SkF1U4vrlhU
Note that this video will be missing the Video 137 and Audio 139 format.

Put ONLY these 4 formats (137,136,140,139) in the "List of preferred formats" and attempt a download.
The download will use this in the command line: "-f 137+136+140+139" and then it will fail with this error:
"ERROR: [youtube] SkF1U4vrlhU: Requested format is not available. Use --list-formats for a list of available formats"

Now remove those 4 formats in the "List of preferred formats" so it is EMPTY and use this in the command line box: -f 137/136+140/139
Tartube will download the first found video and audio and merge them. Incidentally, this command syntax still has some issues and as near as I can figure something like this works much better: -f "(137/136)+(140/139)"

Thanks for all your hard work and I hope this helps.

PS. I have noticed 2 other issues while looking into this that you may encounter:

  1. If I click to download a video again, and it fails, then the video will still be added to the archive list but since it doesn't really 'exist' anymore, I cannot click to download it again until I manually edit the archive list to remove it.

  2. There are other formats which I do not think are handled properly by tartube. For example, the 140 format also has 140-drc, 140-1, 140-2, 140-3, 140-4, 140-5

You can see those formats here: https://www.youtube.com/watch?v=ewfn6qKUe48 https://www.youtube.com/watch?v=BGhrTC0o-qg

@tsk-2222 tsk-2222 added the bug Something isn't working label Jan 27, 2025
@axcore
Copy link
Owner

axcore commented Jan 30, 2025

Any yt-dlp option containing brackets must be surrounded by quotes, so this will not work:

-f (137/136)+(140/139)

But this will:

-f "(137/136)+(140/139)"

Tartube assumes that everything added to the Additional download options box should be passed on to yt-dlp unaltered, so it's up to users to familiarise themselves with yt-dlp documentation. (There are good reasons for not interfering, which I can explain if necessary.)

About the rest of your problem, I will write more shortly...

@axcore
Copy link
Owner

axcore commented Jan 30, 2025

I have fixed the problem. I will try to finish the new release this weekend; the new release will include the fix.

The problem was a holdover from the original youtube-dl-gui.

I have changed the code, so that by default, this is generated:

-f 101/102/103

...instead of this, which is clearly wrong:

-f 101+102+103

In addition, if the selected formats are exclusively video-only and audio-only, then Tartube will generate this instead:

-f 137/136+140/139

This special case is enabled for new installations by default; existing users must enable it by clicking Edit > General download options... > Formats > Advanced > For each video, combine the first video and first audio format from the preferred list

@axcore
Copy link
Owner

axcore commented Jan 30, 2025

Regarding your other issues:

  1. If you need to re-download a video, the "correct" way to do it is to right-click the video and select Re-download this video.

During this new operation, the archive file is ignored, so it shouldn't be necessary to edit it manually. (If that does not solve your problem, then you can tell me.)

  1. Unfortunately, YouTube has not published an exhaustive list of its video/audio formats, so Tartube's list is probably incomplete. At the moment I'm using this page as my "official" list. If it doesn't contain the format you want, you can still specify that format in the Additional download options box.

@tsk-2222
Copy link
Author

1. If you need to re-download a video, the "correct" way to do it is to right-click the video and select **Re-download this video**.

During this new operation, the archive file is ignored, so it shouldn't be necessary to edit it manually. (If that does not solve your problem, then you can tell me.)

Yes I am aware of that. This specific problem happens if you click Re-Download and that download fails or is stopped for whatever reason. At that point, the entry is still in the archive.txt file so that blocks downloading the file again, and (I presume) the entry has already been removed from the tartube database so Re-Download is no longer an available menu option.

In addition, if the selected formats are exclusively video-only and audio-only, then Tartube will generate this instead:
-f 137/136+140/139

As near as I can figure, that is suppose to work but in reality, it does not always work.

Here is a video that is missing only 139 : https://www.youtube.com/watch?v=8VLX_dqybpA
Using this: "-f 137/136+140/139" causes yt-dlp to only test for the existence of format 137 and thus it will only download 137. Your guess will be better than mine on why yp-dlp is skipping the 140 format that does exist.

The bottom line, is that the following does seem to work: -f "(137/136)+(140/139)"

(Incidentally, relating to #1 above. If you were to stop the re-download at the point where you see it is only going to download video and no sound, then you will see that you can no longer re-download that file without editing the archive.txt file)

Unfortunately, YouTube has not published an exhaustive list of its video/audio formats, so Tartube's list is probably incomplete. At the moment I'm using this page as my "official" list. If it doesn't contain the format you want, you can still specify that format in the Additional download options box.

These other formats are minor subtypes of each main numbered format. Would it be possible to use only the first 3 numbers for tartube logic but allow the full format code to be used in the "List of preferred formats" and/or the "Type extractor code" box?

@axcore
Copy link
Owner

axcore commented Jan 31, 2025

Yes I am aware of that. This specific problem happens if you click Re-Download and that download fails or is stopped for whatever reason. At that point, the entry is still in the archive.txt file so that blocks downloading the file again, and (I presume) the entry has already been removed from the tartube database so Re-Download is no longer an available menu option.

The re-download menu option is only visible if Tartube's database thinks the video has been successfully downloaded. When you click this option, the following things happen:

  1. In the database, the video is marked as not downloaded (yet)
  2. The old video files (including the thumbnail) are deleted
  3. The video is then downloaded again

Because the thumbnail is no longer visible in the Videos tab, it does look like the video has been removed from the database. But it's still there and is still visible in the list of videos. You can't "re-download" the video, but you can just download it as normal.

In this case, I think the best strategy is to temporarily disable the archive file (Edit > System preferences > Operations > Archive > Allow downloader to create its own archive file) while you get the videos you want. I definitely don't want Tartube to make changes to the archive file; that would create a whole new bunch of problems.

@axcore
Copy link
Owner

axcore commented Jan 31, 2025

The bottom line, is that the following does seem to work: -f "(137/136)+(140/139)"

Understood. Perhaps you could ask the yt-dlp people to fix that particular problem; there is no reason why it should not behave the same as -f "137/136+140/139".

@axcore
Copy link
Owner

axcore commented Jan 31, 2025

Would it be possible to use only the first 3 numbers for tartube logic but allow the full format code to be used in the "List of preferred formats" and/or the "Type extractor code" box?

I have modified the "Type extractor code" box so that it will accept anything; the change will be in this weekend's release.

@tsk-2222
Copy link
Author

The bottom line, is that the following does seem to work: -f "(137/136)+(140/139)"

Understood. Perhaps you could ask the yt-dlp people to fix that particular problem; there is no reason why it should not behave the same as -f "137/136+140/139".

The issue was reported, replied to, and closed....before I even finished my coffee. :)

Maybe I missed it in the documentation, but the official yt-dlp reply was:

Thats just how -f works. The operstors have different precedence, so you should parenthesize the operations here instead

@axcore
Copy link
Owner

axcore commented Feb 2, 2025

Great, I'll amend Tartube code to use -f "(137/136)+(140/139)" instead of -f "137/136+140/139", in this situation.

@axcore
Copy link
Owner

axcore commented Feb 2, 2025

All changes are available in v2.5.100, released today.

This release includes a change to re-checked and re-downloaded videos. If the re-download fails, the old files are restored, rather than being deleted immediately.

This was a long thread and maybe I've forgotten something, but feel free to close it yourself if not.

@tsk-2222
Copy link
Author

tsk-2222 commented Feb 2, 2025

I noticed a couple things:

If I manually add a format like 140-drc, then the logic breaks and the command reverts to: -f 137/136/135/134/140/139/140-drc

Otherwise the command is: -f (137/136/135/134)+(140/139)
It seems to work but does it need quotes?

I fear that people might not find the combine checkbox on the Advanced tab. Should there be something closer to where the formats are selected and/or should the default be to combine when audio and video are downloaded? or maybe a popup box???

@axcore
Copy link
Owner

axcore commented Feb 3, 2025

If I manually add a format like 140-drc, then the logic breaks and the command reverts to: -f 137/136/135/134/140/139/140-drc

Yes, because Tartube doesn't know what 140-drc is. The -f (A/B/C) + (D/E/F) only applies when Tartube knows that A/B/C/D/E/F are video-only or audio-only formats.

It seems to work but does it need quotes?

No, because of the shell environment, quotes are not needed and in fact they won't work.

I fear that people might not find the combine checkbox on the Advanced tab.

I think the layout of the "Preferred" tab is already too complicated for normal users. People who care about 140-drc are the same people who will find the Advanced tab anyway.

@tsk-2222
Copy link
Author

tsk-2222 commented Feb 3, 2025

Yes, because Tartube doesn't know what 140-drc is. The -f (A/B/C) + (D/E/F) only applies when Tartube knows that A/B/C/D/E/F are video-only or audio-only formats.

Couldn't any sub-formats with extensions be treated as the base format? For example, 140-0 is always going to be sound with the same codec and bitrate, etc. It's just going to be German rather than English.

I think the layout of the "Preferred" tab is already too complicated for normal users. People who care about 140-drc are the same people who will find the Advanced tab anyway.

To be clear, my comment had nothing to do with adding sub-formats like 140-drc. This will be relevant to everyone who combines audio and video. In my testing, the download will appear to succeed but only the first found file will be downloaded which is typically going to have the appearance of a video with broken sound.

Thanks for all your hard work. I know myself and a lot of people really appreciate it.

@axcore
Copy link
Owner

axcore commented Feb 3, 2025

Couldn't any sub-formats with extensions be treated as the base format? or example, 140-0 is always going to be sound with the same codec and bitrate, etc.

A possibility, but is there any guarantee that A-B will be the same as A? If we had an official list of YouTube formats, this decision would be easier, but we don't.

@tsk-2222
Copy link
Author

tsk-2222 commented Feb 5, 2025

It's your call. I can't find an official list either. Whatever might be available seems to be behind a 'wall' for channel owners/uploaders.

Before you move on, please allow me one last indulgence. Oh btw, close this issue at your option.

The following video is only a couple weeks old and contains only audio sub-formats. There is not a single standard audio format available to download. I don't think there is even a combined format.
https://www.youtube.com/watch?v=BGhrTC0o-qg

(I included the full format listing at the bottom of this message)

Since these formats were added by YT, I would consider them official. Based on the audio format descriptions, there is always a labeled default format which we can assume would typically be the native language. Then the other audio formats are typically AI generated dubbing in other popular languages. The only exception I have seen is the xxx-drc which is audio dynamic range balancing.

So, like I said, it's your call. It certainly looks to me like YT is rolling out language preference. Now that I see them doing it, I'm actually surprised they haven't done this sooner. (Or maybe they have and I just haven't noticed it)

As a side note, if you were to ever include a language preference option in the future, the actual format code numbers are meaningless as far as language it concerned. IOW, 140-0 is not always German. The formats would have to be filtered based on what is in the format description brackets. For example [de-DE] is German obviously, but could 140-0 or 249-2.

I hope that helps, and thanks again.

(Sorry if this listing is broken for you. I don't know how to make it display better)


133 mp4 426x240 30 | 4.07MiB 74k https | avc1.4d4015 74k video only 240p, mp4_dash
134 mp4 640x360 30 | 7.53MiB 137k https | avc1.4d401e 137k video only 360p, mp4_dash
135 mp4 854x480 30 | 12.13MiB 220k https | avc1.4d401f 220k video only 480p, mp4_dash
136 mp4 1280x720 30 | 19.86MiB 360k https | avc1.4d401f 360k video only 720p, mp4_dash
140-0 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [de-DE] German (Germany), medium, m4a_dash
140-1 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [es-US] Spanish (United States), medium, m4a_dash
140-2 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [fr-FR] French (France), medium, m4a_dash
140-3 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [it] Italian, medium, m4a_dash
140-4 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [pt-BR] Portuguese (Brazil), medium, m4a_dash
140-5 m4a audio only 2 | 7.14MiB 129k https | audio only mp4a.40.2 129k 44k [en-US] English (United States) original (default), medium, m4a_dash
160 mp4 256x144 30 | 1.90MiB 35k https | avc1.4d400c 35k video only 144p, mp4_dash
18 mp4 640x360 30 2 | 25.48MiB 462k https | avc1.42001E mp4a.40.2 44k [en] 360p
229 mp4 426x240 30 | ~ 17.09MiB 310k m3u8 | avc1.4D4015 310k video only
230 mp4 640x360 30 | ~ 41.23MiB 749k m3u8 | avc1.4D401E 749k video only
231 mp4 854x480 30 | ~ 61.38MiB 1115k m3u8 | avc1.4D401F 1115k video only
232 mp4 1280x720 30 | ~ 86.06MiB 1563k m3u8 | avc1.4D401F 1563k video only
233-0 mp4 audio only | m3u8 | audio only unknown [de-DE] Deutsch (Deutschland) - dubbed-auto
233-1 mp4 audio only | m3u8 | audio only unknown [es-US] es-US - dubbed-auto
233-2 mp4 audio only | m3u8 | audio only unknown [fr-FR] Fran�ais (France) - dubbed-auto
233-3 mp4 audio only | m3u8 | audio only unknown [it] Italiano - dubbed-auto
233-4 mp4 audio only | m3u8 | audio only unknown [pt-BR] Portugu�s (Brasil) - dubbed-auto
233-5 mp4 audio only | m3u8 | audio only unknown [en-US] American English - original (default)
234-0 mp4 audio only | m3u8 | audio only unknown [de-DE] Deutsch (Deutschland) - dubbed-auto
234-1 mp4 audio only | m3u8 | audio only unknown [es-US] es-US - dubbed-auto
234-2 mp4 audio only | m3u8 | audio only unknown [fr-FR] Fran�ais (France) - dubbed-auto
234-3 mp4 audio only | m3u8 | audio only unknown [it] Italiano - dubbed-auto
234-4 mp4 audio only | m3u8 | audio only unknown [pt-BR] Portugu�s (Brasil) - dubbed-auto
234-5 mp4 audio only | m3u8 | audio only unknown [en-US] American English - original (default)
242 webm 426x240 30 | 4.64MiB 84k https | vp9 84k video only 240p, webm_dash
243 webm 640x360 30 | 10.68MiB 194k https | vp9 194k video only 360p, webm_dash
244 webm 854x480 30 | 14.60MiB 265k https | vp9 265k video only 480p, webm_dash
249-0 webm audio only 2 | 3.32MiB 60k https | audio only opus 60k 48k [fr-FR] French (France), low, webm_dash
249-1 webm audio only 2 | 3.42MiB 62k https | audio only opus 62k 48k [es-US] Spanish (United States), low, webm_dash
249-2 webm audio only 2 | 3.52MiB 64k https | audio only opus 64k 48k [de-DE] German (Germany), low, webm_dash
249-3 webm audio only 2 | 3.53MiB 64k https | audio only opus 64k 48k [pt-BR] Portuguese (Brazil), low, webm_dash
249-4 webm audio only 2 | 3.54MiB 64k https | audio only opus 64k 48k [it] Italian, low, webm_dash
249-5 webm audio only 2 | 2.82MiB 51k https | audio only opus 51k 48k [en-US] English (United States) original (default), low, webm_dash
250-0 webm audio only 2 | 4.31MiB 78k https | audio only opus 78k 48k [fr-FR] French (France), low, webm_dash
250-1 webm audio only 2 | 4.43MiB 80k https | audio only opus 80k 48k [es-US] Spanish (United States), low, webm_dash
250-2 webm audio only 2 | 4.57MiB 83k https | audio only opus 83k 48k [it] Italian, low, webm_dash
250-3 webm audio only 2 | 4.58MiB 83k https | audio only opus 83k 48k [pt-BR] Portuguese (Brazil), low, webm_dash
250-4 webm audio only 2 | 4.60MiB 83k https | audio only opus 83k 48k [de-DE] German (Germany), low, webm_dash
250-5 webm audio only 2 | 4.12MiB 75k https | audio only opus 75k 48k [en-US] English (United States) original (default), low, webm_dash
251-0 webm audio only 2 | 8.33MiB 151k https | audio only opus 151k 48k [fr-FR] French (France), medium, webm_dash
251-1 webm audio only 2 | 8.55MiB 155k https | audio only opus 155k 48k [es-US] Spanish (United States), medium, webm_dash
251-2 webm audio only 2 | 8.73MiB 158k https | audio only opus 158k 48k [it] Italian, medium, webm_dash
251-3 webm audio only 2 | 8.87MiB 161k https | audio only opus 161k 48k [pt-BR] Portuguese (Brazil), medium, webm_dash
251-4 webm audio only 2 | 8.98MiB 163k https | audio only opus 163k 48k [de-DE] German (Germany), medium, webm_dash
251-5 webm audio only 2 | 7.90MiB 143k https | audio only opus 143k 48k [en-US] English (United States) original (default), medium, webm_dash
269 mp4 256x144 30 | ~ 9.62MiB 175k m3u8 | avc1.4D400C 175k video only
278 webm 256x144 30 | 2.65MiB 48k https | vp9 48k video only 144p, webm_dash
394 mp4 256x144 30 | 1.73MiB 31k https | av01.0.00M.08 31k video only 144p, mp4_dash
395 mp4 426x240 30 | 3.04MiB 55k https | av01.0.00M.08 55k video only 240p, mp4_dash
396 mp4 640x360 30 | 6.00MiB 109k https | av01.0.01M.08 109k video only 360p, mp4_dash
397 mp4 854x480 30 | 9.76MiB 177k https | av01.0.04M.08 177k video only 480p, mp4_dash
602 mp4 256x144 15 | ~ 4.56MiB 83k m3u8 | vp09.00.10.08 83k video only
603 mp4 256x144 30 | ~ 9.36MiB 170k m3u8 | vp09.00.11.08 170k video only
604 mp4 426x240 30 | ~ 16.08MiB 292k m3u8 | vp09.00.20.08 292k video only
605 mp4 640x360 30 | ~ 34.36MiB 624k m3u8 | vp09.00.21.08 624k video only
606 mp4 854x480 30 | ~ 54.72MiB 994k m3u8 | vp09.00.30.08 994k video only
ID EXT RESOLUTION FPS CH | FILESIZE TBR PROTO | VCODEC VBR ACODEC ABR ASR MORE INFO
[info] Available formats for BGhrTC0o-qg:
sb0 mhtml 320x180 0 | mhtml | images storyboard
sb1 mhtml 160x90 0 | mhtml | images storyboard
sb2 mhtml 80x45 0 | mhtml | images storyboard
sb3 mhtml 48x27 0 | mhtml | images storyboard
137 mp4 1920x1080 30 | 59.86MiB 1086k https | avc1.640028 1086k video only 1080p, mp4_dash
247 webm 1280x720 30 | 24.73MiB 449k https | vp9 449k video only 720p, webm_dash
248 webm 1920x1080 30 | 39.19MiB 711k https | vp9 711k video only 1080p, webm_dash
270 mp4 1920x1080 30 | ~258.18MiB 4688k m3u8 | avc1.640028 4688k video only
398 mp4 1280x720 30 | 16.74MiB 304k https | av01.0.05M.08 304k video only 720p, mp4_dash
399 mp4 1920x1080 30 | 26.55MiB 482k https | av01.0.08M.08 482k video only 1080p, mp4_dash
609 mp4 1280x720 30 | ~ 80.25MiB 1457k m3u8 | vp09.00.31.08 1457k video only
614 mp4 1920x1080 30 | ~135.31MiB 2457k m3u8 | vp09.00.40.08 2457k video only
616 mp4 1920x1080 30 | ~283.48MiB 5147k m3u8 | vp09.00.40.08 5147k video only Premium
Info operation finished

@tsk-2222
Copy link
Author

tsk-2222 commented Feb 5, 2025

Here's a couple more very recent videos I've come across with dubbing:
https://www.youtube.com/watch?v=yhPy4y2_EMc
https://www.youtube.com/watch?v=AqzJ58bM2rs

I asked Grok AI about it and received this reply for what it's worth:

Yes, YouTube has started implementing AI-powered auto-dubbing for videos in different languages. This feature automatically translates and dubs video content into various languages, including but not limited to English, Spanish, Portuguese, German, French, Italian, Hindi, Indonesian, and Japanese. As of the information available:

The auto-dubbing feature was announced to be in testing with hundreds of creators in June 2023, and by December 2024, it was expanded to "hundreds of thousands of channels" within the YouTube Partner Program, particularly those focused on knowledge and information content.

Creators don't need to do anything special; when they upload a video, YouTube automatically detects the original language and generates dubbed versions. However, creators have the option to review these dubs before publishing or to unpublish/delete them if they're not satisfied.

Viewers can select the dubbed language from the video settings (gear icon, then "Audio Track") if the feature is available for that video. A video with auto-dubbing is marked as "auto-dubbed" in its description.

@axcore
Copy link
Owner

axcore commented Feb 13, 2025

I think we will have to wait for yt-dlp to handle this situation, perhaps by introducing a new download option that looks something like this:

--format mp4 --format-language de_DE

I can't think of a convenient way for Tartube to handle language selections at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants