Support for adding audio streams via youtube-dl #379

lenerd · 2020-04-09T23:30:51Z

Hi all,

I have added a small feature to ncmpcpp which allows adding audio streams from websites via youtube-dl. So I would like to ask if there is interest in adding such a feature to the official sources (a similar suggestion appeared in issue #53). Otherwise I would just maintain this as a patch for myself.

"youtube-dl is a command-line program to download videos from YouTube.com". It also supports a lot of other sources, e.g., websites with podcasts, and media libraries of public broadcasters. While the main functionality of this tool is the download of videos, youtube-dl is also able to extract the URLs of the corresponding audio and audio/video streams, which can then be added to MPD's playlist.

This change adds a new action to ncmpcpp's playlist view: Pressing 'D' (in the default configuration), prompts for the URL of a website. The input is passed to youtube-dl to retrieve the URL of the audio stream and available metadata. If successful, the URL is added to the MPD playlist.

I use this to listen to radio programs or podcasts while having the controls of my usual music player. So I don't have to find some browser tab to pause the playback, and there is also no need to download/play/delete the audio files manually.

No new dependencies are introduced to compile ncmpcpp. To use the feature, youtube-dl is needed to be installed at run time (available in many distributions repositories or via pip). It is executed as a child process using Boost.Process and Boost.PropertyTree is used to parse its JSON output.

So, what do you think of this?

Of course, I am open for any suggestions, and I also have some questions:

I guess it would be a good idea to have some sort of timeout in case youtube-dl should hang (has not happen to me so far even without a network connection). Implementing this should be doable, but I am unsure of a good default value. Adding a long YouTube playlist can take a while.
Currently, the interface is blocked until the action is completed which might be a bit annoying. In principle, it could be run in a background thread, but I have no idea what parts of ncmpcpp are thread-safe.
Is there a way to "refresh" the playlist (view) during the execution of an action? In this way, progress could be shown when adding playlists. In a second ncmpcpp instance I can see the items being added one after another, but in the executing instance the new items appear only after the action has been completed.

Thanks!

CamilleScholtz · 2020-04-13T19:31:34Z

Does it actually pre-download the video/audio, or does it stream it on the go? That would maybe fix the playlist timeout thing

lenerd · 2020-04-13T22:04:26Z

Does it actually pre-download the video/audio, or does it stream it on the go?

The call to youtube-dl just extracts the URL(s) from the webpage. The download happens by MPD on the server-side.

Edit: Even though the audio itself is not downloaded on the client-side, a number of HTTP requests are made which can take a while for large playlists.

Salastil · 2020-11-02T21:35:45Z

There has been a general slowdown of work done on the main ytdl project and a few forks have popped up, one of which is some 700 commits ahead of the main project. Perhaps it would be prudent to put a config option to select the name of the ytdl binary? There seems to be 3 competing projects at the moment. I have aliased youtube-dlc to youtube-dl but there may come a point where users may have two competing versions of the project installed at once.

gottaeat · 2020-11-20T20:06:03Z

the patch applies and builds fine, but i have to set $LANG to C otherwise it prompts locale::facet::_s_create_c_locale name not valid.

lenerd · 2020-11-20T22:29:32Z

Hi,
@mssx86 I cannot reproduce this with any of the locales I have installed. Which locale are you usually using, and what did you do exactly to trigger the error message?

lenerd · 2020-11-20T22:33:28Z

@Salastil I suppose one could add an option is there is demand for this. Otherwise one could always adjust PATH. Do all of the projects maintain the same command line interface?

Salastil · 2020-11-21T10:16:39Z

@lenerd As it stands right now youtube-dlc: https://github.com/blackjack4494/yt-dlc just symbolically links youtube-dl to the youtube-dlc binary. Youtube-dl appears to have returned from its DMCA strikedown status and a quick glance at the diff for the two youtube extractors shows that the two projects are beginning to diverge. youtube-dlc at the moment is superior for the wider community of sites like Dailymotion, Peertube, Soundcloud etc as its some 700 commits ahead but some users may end up with both youtube-dl and youtube-dlc installed at the same time which would break the symbolic link and the option to just set it via PATH would be impossible. I don't believe that youtube-dlc has changed any command line flags, just updated the extractors to be functional.

arybczak · 2020-12-24T01:37:47Z

src/actions.cpp

+		if (id == -1) {
+			return;
+		}
+		if (title.has_value()) {


Why does it try to add multiple things from the database?

youtube-dl has support for playlists and outputs one line of JSON per title. Each of those is parsed and added to the MPD playlist.

Right, but why the output of youtube-dl is used to add items from the database to queue? It seems unexpected that it would do anything else than add youtube links to queue.

arybczak · 2020-12-24T01:38:29Z

src/actions.cpp

+	// search the youtube-dl executable in the PATH
+	auto ydl_path = bp::search_path("youtube-dl");
+	if (ydl_path.empty()) {
+		Statusbar::ScopedLock slock;


If you want to show a message for a while, use Statusbar::print(f). Statusbar::put is for prompts (with ScopedLock) or for showing a message while input is blocked.

Thanks, good to know!

lenerd · 2020-12-28T19:53:08Z

@arybczak Thanks for your comments!

I adjusted and extended the status messages.

arybczak · 2021-01-24T18:40:29Z

How is it supposed to be used? Can you show a few examples?

youtube-dl [0] is a "command-line program to download videos from YouTube.com" and many other websites, e.g. media libraries of public broadcasters. This commit adds a new action to ncmpcpp's playlist view: Pressing 'D', prompts for the URL of a website. The input is passed to youtube-dl to retrieve the URL of the audio stream and available metadata. If successful, the URL is added to the MPD playlist. [0]: https://github.com/ytdl-org/youtube-dl

- use `Statusbar::print{,f}` instead of `Statusbar::put` for non-blocking status messages - display number of items added - display of non-zero exit codes of youtube-dl

Salastil · 2023-08-26T23:39:04Z

src/actions.cpp

+	// -f bestaudio/best: selects the best available audio-only stream, or
+	//                    alternatively the best audio+video stream
+	bp::ipstream output;
+	bp::child child_process(ydl_path, url, "-j", "-f", "bestaudio/best", bp::std_out > output,


bp::child child_process(ydl_path, url, "-j", "-f", "bestaudio/best","--playlist-end", "100" , bp::std_o ++ut > output,

Just a quality of life change here, when adding autogenerated playlist it will add 1000 items to ncmpcpp. Adding in --playlist end 100 will trim that number down to a more sensible 100. The videos are only good for something like 3-6 hours before they need to expire and need to be regenerated so it is effectively impossible to listen to the full 1000 item playlist anyway and it takes a good bit of time for it to generate such a huge playlist.

lenerd force-pushed the lenerd/youtube-dl branch from 279ee83 to f37fd5c Compare June 30, 2020 13:57

lenerd force-pushed the lenerd/youtube-dl branch from f37fd5c to 2b0bdb4 Compare November 20, 2020 22:23

lenerd force-pushed the lenerd/youtube-dl branch 2 times, most recently from 1c298e6 to fb95546 Compare December 14, 2020 07:59

lenerd force-pushed the lenerd/youtube-dl branch 2 times, most recently from ac9340c to 5e58fdc Compare December 23, 2020 17:34

arybczak reviewed Dec 24, 2020

View reviewed changes

lenerd force-pushed the lenerd/youtube-dl branch 2 times, most recently from c8bfa27 to ecd4f16 Compare December 28, 2020 19:18

lenerd added 2 commits April 22, 2021 21:03

Update status messages of youtube-dl action

0847e59

- use `Statusbar::print{,f}` instead of `Statusbar::put` for non-blocking status messages - display number of items added - display of non-zero exit codes of youtube-dl

lenerd force-pushed the lenerd/youtube-dl branch from ecd4f16 to 0847e59 Compare April 22, 2021 19:03

Salastil reviewed Aug 26, 2023

View reviewed changes

Support for adding audio streams via youtube-dl #379

Are you sure you want to change the base?

Support for adding audio streams via youtube-dl #379

Uh oh!

Conversation

lenerd commented Apr 9, 2020

Uh oh!

CamilleScholtz commented Apr 13, 2020

Uh oh!

lenerd commented Apr 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Salastil commented Nov 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gottaeat commented Nov 20, 2020

Uh oh!

lenerd commented Nov 20, 2020

Uh oh!

lenerd commented Nov 20, 2020

Uh oh!

Salastil commented Nov 21, 2020

Uh oh!

arybczak Dec 24, 2020

Choose a reason for hiding this comment

Uh oh!

lenerd Dec 26, 2020

Choose a reason for hiding this comment

Uh oh!

arybczak Jan 24, 2021

Choose a reason for hiding this comment

Uh oh!

arybczak Dec 24, 2020

Choose a reason for hiding this comment

Uh oh!

lenerd Dec 26, 2020

Choose a reason for hiding this comment

Uh oh!

lenerd commented Dec 28, 2020

Uh oh!

arybczak commented Jan 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Salastil Aug 26, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lenerd commented Apr 13, 2020 •

edited

Loading

Salastil commented Nov 2, 2020 •

edited

Loading

arybczak commented Jan 24, 2021 •

edited

Loading