Episode 827 transcript

FLOSS-827

Jonathan: Hey folks. This week I talk with Bash only about Y-T-D-L-P. That is the cutting edge, YouTube and every other video service imaginable downloader. It's a lot of fun. You don't wanna miss it, so stay tuned. This is Floss Weekly, episode 827 recorded April 1st, Y-T-D-L-P. Sometimes you can't see the tail.

Hey folks, it's time for Floss Weekly Lead. That's the show about Free Libre and open source software. I'm your host, Jonathan Bennett, and we've got something a little different. Today we're talking with Bash only. Yes. First name Bash, last name only about Y-T-D-L-P. That is. One of the YouTube downloaders.

And as far as I could tell, it's sort of the premier YouTube downloader these days since that old project YouTube DL sort of went away or went on hiatus. There's a lot of history here but we've got the right person to talk with about it. So I'm gonna go ahead and bring on Mr. Bash. Mr. Only Bash only.

Welcome to the show.

Bashonly: Hello. Thanks for having me.

Jonathan: Hey, it is, it is great to have you here. I am. I'm excited. I know we've got at least one listener in the chat room that's excited to talk about Y-T-D-L-P because it's a tool that so many of us find ourselves using a lot for. You know, the things I've been using it for recently is to download the HDR version of certain videos to be able to play it in MPV to do HDR testing.

And you know, there's a whole bunch of those different like oddball use cases that you wouldn't ever think of that people end up using. Tools like this, this one in particular for that's gotta be kind of surprising, isn't it? Do you, do you have kind of a, a line to the, the way people use it? You get told stories?

Bashonly: You know, I came into the project in 2022 mm-hmm. And it was forked from YouTube dl, which we can get into later. Mm-hmm. But you know, it's, it's been around for over a decade, you know, in one form or another. And it just, it supports so many sites. That every week there's, you know, some new issue reported of a site that I didn't even know we supported.

And then I find out like thousands of people used Y-T-D-L-P just for that site.

Jonathan: Mm-hmm.

Bashonly: And, you know, I had never even heard of it before, so it really is sort of like the, the Swiss Army knife of downloading.

Jonathan: Yes. Which is interesting because I think it's, it's primarily based on ffm peg, which is the, the Swiss Army knife of anything related with media.

Right.

Bashonly: It uses FFM, peg it, it's, it's one of the libraries.

Jonathan: It depends. Right, right, right. That's, that's fair. Right. Okay, so let's, let's start with the elephant in the room then. And I got, I got permission beforehand to ask this. What is the, what is kind of the, the primary reason that you are Mr.

Bash only today and only a a logo and not going by, you know, your real name on the internet and putting your, your picture out there like some of us have chosen to What's what, what's behind that decision? I.

Bashonly: I mean, who says Bash only isn't my legal name?

Jonathan: It's on the, here's your, here's my social security card.

It's on the driver's license. Miss, that's Mr. Only to you.

Bashonly: Yeah, exactly. I was thinking about, you know, PowerShell only, but it didn't really have the same ring to it. Oh. But it's, and I also, I don't think I would agree with that name, but yeah, I just don't really want my name out there for reasons.

Mm-hmm. Like if you look at what happened to YouTube DL about five years ago there are people that want to take this project down. Mm-hmm. And I just don't know what angle they're gonna. Come at that from, right. So just to protect myself, I just, you know, rather stay semi-anonymous.

Jonathan: Mm-hmm. Yeah, that makes sense.

It, it's hard, it's hard to be served with paperwork if your real name and your address isn't out there anywhere. Yeah,

Bashonly: exactly.

Jonathan: So let's, let's talk about kind of the, I guess the origin story of Y-T-D-L-P. Honestly, I, I did not know for sure that it was a fork of YouTube dl, but like, what's the, what's the history there?

What was the, kind of the original problem that somebody wanted to solve and, and the, the original project got started.

Bashonly: I think the original intent with it was.

To sort of create a digital VCR, and by that I, I don't mean, you know, like DVR mm-hmm. You know, A DVR is owned by, you know, your service provider. You don't actually own the content. You know, it's encrypted.

Jonathan: Mm-hmm.

Bashonly: A digital VCR is, you know, you see a video on the internet, you can record it, you can watch it offline, you can, you know, use it for.

For personal use and I think with video streaming becoming more and more complex, that had be, become impossible without a tool like YouTube dl.

Jonathan: Mm-hmm. That's a, that's an interesting analogy too. This is something I was thinking about in, in preparing for the show. You know, there's gotta be legal questions around doing this.

You know, can you download videos? Let's just take YouTube. Like the, the question has to be, can you legally download videos from YouTube? And I think the, the VHS player is a really good analogy because here in the United States, we had a Supreme Court case about this. I mean, Mr. Rogers went and testified about this, right?

That it makes sense to be able to, and the term they use is time shifting, right? That's, that's the official term for what we do here. We time shift things. I. You, you, you take it and you make a local copy of it so you can watch it at another time. You can watch it repeatedly if you want to. And that's that's pretty apt for what, at least, you know, as far as I is what I do with it with, with Y-T-D-L-P and, and previously with YouTube dl.

You just, you get a, you get a local copy of it so you can watch it whenever you want to. But that's sort of been disputed over the years, hasn't it? By, by maybe Google to some extent, but other people like the RIAA, not, not everybody shares that same legal theory. Do they?

Bashonly: Well, the RIAA, they've, they have their own view on this issue and.

You know, GitHub apparently disagrees with the RIA. Right. 'cause they rein, they reinstated YouTube dl. I think I'm, I'm trying to be careful with my words here. Sure, sure.

Jonathan: So we, we are not lawyers, right? Neither of us have legal, as far as I know. Besh only may have a legal degree and just isn't telling anybody. I certainly do not. So this is not legal advice. This is us being internet lawyers based on what we can read, right?

Bashonly: So essentially the way I view it is. You know, your web browser visits a website.

Mm-hmm. It streams the video. You are stream the content. Mm-hmm. You receive the data of the video. It's just not viewable outside of your web browser. Right. What tools like YouTube, DL and Y-T-D-L-P do is they act essentially like a web browser. Mm-hmm. The way that, that they. Interact with the websites is they basically recreate everything the browser does, but then it saves it to your hard disk instead of, you know, just an ural stream.

Mm-hmm. So I don't think, you know what the RIA said in the the DMCA take down notice for YouTube dl. I don't think that applies here. Because what YouTube DL was doing really wasn't anything different than what your web browser was doing.

Jonathan: Right, right. And, and so there's a, and I think this was one of the things that, like the RIAA got totally wrong.

They talked about DRM and when I looked into it at the time, it was the case that YouTube dl and I'm, I'm assuming this is the case with Y-T-T-L-P as well, explicitly and intentionally does not break. DRM isn't, isn't that, isn't that accurate? Correct. Yeah. Yep. Yeah. So

Bashonly: yeah, if a, if a website only serves DRM content, then Y-T-D-L-P doesn't support it.

Yeah. And you know, if the, if there are DRM protected formats offered by a website, Y-T-D-L-P does not download those.

Jonathan: Mm-hmm.

Bashonly: So we deliberately avoid anything DRM protected. Because, because that would be unquestionably illegal. Yeah.

Jonathan: Yeah. Does YouTube still recommend that like Y-T-T-L-P for archiving?

That was, that was the funny thing at the time when it got taken down, there was still like active YouTube documentation saying, you know, if you need to download a copy of your video, here's this great command line tool that you can use.

Bashonly: I actually wasn't aware of that.

Jonathan: Yeah, that was, that was part of the document, part of the, part of the argument, the kind, like the counter argument people made about YouTube dl and it's like, look, Google, here's your own documentation where you tell people that this is the tool to use to do downloads.

It was, it was wild. It was a wild time. I'll have to,

Bashonly: I'll have to look into that. I wasn't aware of that.

Jonathan: Yeah. I'll see if I can, I'll see if I can go back and find some of those sources. Okay, so lots of, lots of directions I could potentially go here, but I'm real curious about, about the fork. So how did we get from YouTube DL and it got the take down notice and then eventually got reinstated.

To now it got forked to Y-T-D-L-P and as far as I can tell, like if you go to install things in say Fedora, Y-T-D-L-P is the one that's available. Not, not YouTube, dl. And in doing the prep for the show, I was kind of surprised when I went back and saw that YouTube DL does actually have a little bit of activity back in, its in its distro or in its GitHub again.

But what, what led to the fork and what led to where we're at now?

Bashonly: Sure. So, well, YouTube DL started I think sometime in the early 2010s. I think it might have been 2010 even. And it started out as just YouTube as a simple downloading tool. Mm-hmm. And then, you know, the scope started expanding. It supported more sites and more and more features were added.

Mm-hmm. More maintainers were added on, and I think the original maintainers sort of. Bit off more than they could, they could chew maybe the, you know, the, there was a lot of churn through the maintainers, right. And and. The maintainers before the take down. I think they were understandably overwhelmed by the sheer volume of issues and pull requests mm-hmm.

That they had on their tracker. You know, even I'm overwhelmed with what Y-T-D-L-P has and that's a, a fraction of what YouTube DL had at the time. Mm-hmm. And, they just, they weren't merging pull requests, you know, they, they had standards, which was good. Yeah. They had high standards for code quality and coding conventions and whatever.

Mm-hmm. But stuff wasn't getting merged. So then the first significant fork from YouTube B was called YouTube, DLC. And that happened, that happened I think a few months before the take down. And basically the idea was it'll be YouTube dl, except we'll merge the poll requests that should be merged mm-hmm.

That aren't getting merged. And then the takedown happened a few months after that. I think it was October, November, 2020. Sounds about right. And so when YouTube DL was taken down, all of its forks were also taken down. Anything that was marked as forked from YouTube dl mm-hmm. Was also got taken down.

So then YouTube, DLC, they started new repository called Y-T-D-L-C, which was. To get around the mm-hmm. The, the forking problem. Yes. And then I think the maintainer of Y-T-D-L-C eventually got burned out, or there was, you know, other life priorities there. Sure. And so then, this is actually the first time I've had to say his name out loud, so I'm not, I'm not sure I'm gonna pronounce it right here, but Pandan.

Mm-hmm. P Pandan he, you know, the torture of pass to him and he started Y-T-D-L-P, which the P in Y-T-D-L-P is his name. His name, yeah. So it's you, you, yeah. That's basically how we got here, how, where we are today.

Jonathan: Mm-hmm. Yeah. It's a, so YouTube go. No, go ahead. Go ahead.

Bashonly: YouTube, DL still does exist.

So after, after the reinstatement development had still pretty much frozen, the maintainers weren't really doing anything and it's, it's understandable. They were already overwhelmed and then they got hit with this, you know, legal nonsense. Yeah. And sort of, sort of sours

Jonathan: you wanting to work on the project anymore.

Bashonly: Yeah. Who wouldn't be burned out at that point, you know? So they basically removed themselves from the project and they, the reins were handed over to a new maintainer. And he's pretty much just been going it alone ever since. Mm. So development is slow over at YouTube dl. But YouTube downloading is still getting fixed, updated.

You know it if you use the. The master branch of U2 BL, it still works. Mm-hmm. There just hasn't been a proper release since I think 2021. Ah, so that's why it's no longer in package repositories. And, you know, the, the version you download with PIP doesn't work. But if you run from source or you, you know, you build it yourself.

Mm-hmm. It, it still works.

Jonathan: Yeah. Interesting stuff. So what, what, what is it like, are you the, are you the sole maintainer of Y-T-D-L-P or do you have a a, a group of people that work with you on it?

Bashonly: Oh, no, I'm not the sole maintainer. I would lose my mind if I was the sole maintainer.

Jonathan: That's, that's good. Yeah.

Bashonly: Yeah. Yeah, there's a team of us, there's about five or six of us, depending on who's active at any given moment. Mm-hmm. And it's sort of a, a democracy. There's not really a leader right now. Okay. We all just sort of, you know, go by consensus and mm-hmm. You know, work together.

Jonathan: Yeah. What's it, what's it like being part of that team?

Like how, how crazy is it? How many, how many sites do you support and how crazy is it to try to stay on top of it?

Bashonly: To be honest with you, I don't even know how many sites we support. I know we have over a thousand modules for extractors. Oh my. But then each of those modules can support more than one site.

And then Y-T-D-L-P also has what's called a generic extractor, which, can be used on any site we don't support. And it looks for generic embedding techniques in the webpage to see if there's any video or audio that can be extracted. Mm-hmm. So it's thousands of sites and I, oh my goodness, I couldn't tell you the exact number.

Jonathan: It's just, it's just drinking from the fire hose all the time then, isn't it?

Bashonly: Yeah. Yeah. I don't know if you've looked at our issue tracker, but I think we're up to 1.5 K.

Jonathan: Mm-hmm.

Bashonly: And most of those are, you know, the site's broken. Can you support this site? And yeah. It's, it's fun.

Jonathan: Yeah. Yeah. Do, do you get any support from anywhere?

Do you, like, does anybody have a Patreon account or does. Does YouTube give you any money to make this work?

Bashonly: Oh yeah. YouTube funds those all the way.

Jonathan: Got those big YouTube bucks. Google Bucks.

Bashonly: Yeah. I don't even work. YouTube just, you know, funds my lifestyle.

Oh no. We have a. We have a donations page on our GitHub that you can check out. Mm-hmm. I don't accept donations just 'cause I don't, you know, again, I, I don't wanna de anonymize myself or Right. Yeah. I wanna avoid that as much as possible.

Jonathan: I was just thinking about that there might be some legal ramifications for taking payment for the work.

Bashonly: Exactly. Yeah.

Jonathan: Mm-hmm. Yeah. So why. To get into a little bit more technical question, why is it, why is this so hard? Like, okay, so I remember years and years ago where back when everything was Flash video somebody came up with this, I don't remember if it was a browser plugin or it ran on Windows. I'm like, this was, this was like 20 years ago at this point.

Somebody had this little tool that would just, it would watch your network connection for incoming flash shock wave video files, and when it saw one, it would just download it right off the wire. We've come, we've come a very long way since then. I know. But why, why is this such a hard problem? Why isn't there just, why isn't there just one extension that works on all of the sites?

Bashonly: Well, that'd be nice, wouldn't it? It would be yeah. You know, back then, you know, I think that was probably around like. 2007, 2010, somewhere around that. Yeah. Time range. Streaming video was mostly just like an FLE file or an MP four file mm-hmm. Or something that you could just, you know, find the URL and download it.

And so for a browser extension to do that, it was simple. And then, you know, then there was HLS dash and. You know, YouTube is experimenting with its own new protocol. What fun. And now you, you know, you can't just download a file anymore. You need FFM peg, you need, you know, something that can stitch together the segments.

Something that can, you know, even find the URLs of the segments now, you know, parse the dash manifest and whatnot.

Jonathan: Mm-hmm. The is, is there quite a bit of churn? Like, does YouTube constantly change stuff to where you guys have to go in and, and ref fiddle with whatever? I mean, I don't, I don't even know, honestly, like, what is the, what is the process for downloading a YouTube video?

You, you've mentioned that you kind of have to act like a browser. So do you go and grab some huge obfuscated JavaScript file and try to de obfuscate it and, and run through it and like what, what does that process even look like?

Bashonly: It's. Similar to what you described. So to answer the first question about breakage and churn it depends I guess, on how upset Google is about.

People downloading and bypassing their ads at any given moment? Like I would say about two years ago, YouTube downing was, was pretty stable. Mm-hmm. It was only every few months something broke and we'd have to fix it. I think for about the past year now, it's been a weekly struggle. Something has been breaking.

At least every two weeks, if not every few days.

Jonathan: Hmm.

Bashonly: We have done about, I think, five releases just in March, just to fix YouTube support. So then to get into this second part of your question about how YouTube extraction works there's a lot of steps to it. And it does involve JavaScript, which has been the problem this month.

Jonathan: One, one might argue that JavaScript is always the problem, but that aside.

Bashonly: Yeah. Yeah. It's sort of me and the maintainers, some of the maintainers have a, a phrase called JavaScript hell, which is what we're in when this happens. Yeah. So there's a. JavaScript player file that, that basically, you know, that runs the whole operation when you're watching a video on YouTube.

Mm-hmm. And within that file, there's there's some functions used to descramble, a query parameter that you need in order to. Actually access the video stream without it, you'll get a 4 0 3 error. Or, you know, in the past you would get throttled so that it was basically unwatchable or downloadable.

Jonathan: I'm familiar, I'm familiar with both of those problems.

Bashonly: I, yeah, yeah. But now then I think about. Six to eight months ago, they changed it. So now you just get a 4 0 3. You can't watch or download anything unless you scramble this query parameter properly. Mm-hmm. And for a while, the, the player JavaScript was stable and then about a year ago they started changing it.

Very frequently and they would try new obfuscation techniques. They would, you know, add, recently they added an array that has all the strings in it. So you can't find the function by searching for the strings. 'cause YT DLP is written in Python. We don't use JavaScript. So, you know, we don't have access to an A ST or anything like that.

We just, I mean we, we essentially first parse it with. RegX. Okay. So we find the function names like that, and then we have our own little JavaScript interpreter module you've built, which is

Jonathan: you built a JavaScript parser in Python.

Bashonly: Yeah. Well, it, it's very primitive, right? It just in, it just interprets what it needs to interpret.

Sure. And it returns a python function. Mm-hmm. So that's been breaking all the time, but

yeah, well there's, there's two parameters like that, that need to be decoded. Mm-hmm. Anyways I mean, does that answer your question? Did you want me to get into more technical details?

Jonathan: No, I, I think so. I think that pretty much gets us into the into the ballpark. Yes. I, I, I remember when everything was throttle to be very slow and so, you know, if you had a, if you had a video you wanted to test something with, you would start it and then go to bed for the night and hope that it didn't time out and you got it by the morning time.

It was very, it was very reminiscent of using the internet 25, 30 years ago. Right, right. Yeah. Oh, fun times. There are some other projects one in particular that I know of that's kind of in this same space is a smart tube. I. And that's a, a replacement YouTube player for like Android tv. Is that, is that something you're aware of and is there any cross pollination?

You guys are obviously solving some of the same problems.

Bashonly: I don't know much about Smart Tube off the top of my head. Okay. I don't keep up with many of the other projects. There's a few of them that I do keep up with because they help us out. Mm-hmm. Quite a bit. There's YouTube js, which is sort of like the reference implementation for Yeah.

All things having to do with the YouTube API. Right. And I'm very aware of that and, you know, I, I watch their changes and everything. But I'm not, I'm not that familiar with Smart Tube.

Jonathan: One of our live listeners mashed potato points out that you, you may have similar problems to New Pipe as well.

And that's another, that's another app. That's an Android app particularly that I'm, I'm aware of. I never got it to work for me, but that's fine.

Bashonly: Yeah. Yeah. New, new Pipe. I'm aware of. New Pipe. And they help us out too. That's a good project. I, I believe they, it's an Android. Project? Yes,

Jonathan: I think so. I shall have to, I shall have to try this one again.

Last time I tried it, it just did not work for me at all. But

Bashonly: yeah, I mean, they're having the similar kinds of problems that we're having, right. They don't really have to deal with the JavaScript part of it that much because it's the Android side of things. Mm-hmm. But the Android side of things has its own set of problems, so

Jonathan: Of course.

Bashonly: Yeah.

Jonathan: In notably. Okay. So surely, surely, your, your career path, your education path, you, you studied software development or computer science, right? And that's what your day job is. That's how you're so good at this.

Bashonly: No. Nope. My, I did not go to school for computer science, software engineering, any of that.

My my real job has nothing to do with this. I'm just a pure hobby developer and pretty much self-taught.

Jonathan: It amazes it. I am, I am basically in that same boat. I didn't even finish college. It amazes me how many people, particularly in the open source world, although. In corporate world in some cases too, sort of took that same path.

It's like we didn't go to college for it. We found it, you know, maybe sometimes in high school, some of us earlier than that, some of us later than that, but just, it's like, oh, this is, this is the thing that I wanna do. Maybe not for a job, but I, I won't, I won't ever really be able to get away from this like, writing, writing code for some project somewhere.

It's just, it's just part of who we are. I find that real fascinating that that's such a, a common story.

Bashonly: Yeah. You know, when I was a kid, I was a huge nerd. I would I would write my own webpages in Notepad, you know, write h tm l by hand. Mm-hmm. JavaScript, and I, you know, I, I, I started out writing batch files.

Mm-hmm. And yep. And I didn't really keep up with that that much. You know, I, I wasn't like an early adopter or Linux or anything like that. And then, you know, I had, you know, sort of a, a busy time in my twenties and then life slowed down after that. Mm-hmm. And I had more time on my hands and I thought.

You know, why don't I, I get back into that. 'cause I, I thought I was pretty good at, you know, programming and Yeah. Doing that sort of thing. And then Covid happened and then I had a whole bunch, I had a whole bunch of time on my hands.

Jonathan: Yes.

Bashonly: So I started teaching myself. You know, I, I started with Bash 'cause I had started using Linux a few years before that.

And I thought. You know, there's so many things that I, I just wish I could just press a button and it would happen.

Jonathan: Mm-hmm.

Bashonly: So I started writing BA scripts for everything that I need to do. And then eventually I taught myself Python. 'cause I thought, you know, that's the logical next step here. Yeah. And I, before I got into Python, I was.

I sort of wrote my own YouTube DL in Bash. Hmm. 'cause I, I wanted to, I wanted to just, I, I wrote like a, a live stream, not downloader, but something that would pass the live stream to MPV. Mm-hmm. So this was back before YouTube became. As difficult as it was today. I don't think I could write this in Bash today, but Right.

And then I thought, you know, this is something I'm good at. And, you know, I was aware of YouTube DL and then Y-T-D-L-P wrote rolled along and I, I figured, why don't I just contribute to this instead of wasting my time on my silly little strips? So I opened a poll request and. I just kept going with it and that's how I ended up where I am today.

Jonathan: That's how they get you that first pull request. Yeah, the first one's free and then you're hooked. Yep. Oh, I'm involved in multiple projects with the, with exactly the same thing. Oh, there's this one thing that annoys me about this. I'm gonna go fix it. And the next thing you know, you've written a quarter of their code base and.

Yeah. That's just how it works. Yep. I must admit, I do still maintain a little bit of HTML by hand. I use Nano, not notepad, but that, that part of your story really spoke to me. I still have one website that I go in and do by hand.

Bashonly: I mean, there's something to be said for that. Sure. So, yeah. Not Notepad.

I don't use Notepad anymore.

Jonathan: Not notepad. No, no. HTML by hand. Yeah, for sure. For sure. Okay, so what what does the, what's the future look like for Y-T-D-L-P? Do you guys have any long-term plans? Are you going to get rid of some of the plugin, some of the modules to be able to make it more maintainable?

Or is it, is it just gas to the floor all the way out?

Bashonly: So probably the biggest undertaking that we're planning right now is to separate the extractor modules from the core project. Mm-hmm. We eventually wanna move all of the extractors, you know, that, that deal with the web, the specific websites. We wanna move those into their own repository or repositories, and then have the the core in its own repo, and it would basically treat every extractor as if it is a plugin.

Mm-hmm. You know, you can plug them in, unplug them. And that would sort of, you know, first it would, it would shield the project. You know, if, if any one site causes trouble for us, then that extractor repository would get, you know, taken down or, yeah. Had to deal with the repercussions of that or the core project would be fine.

That's still a ways out I think, because

Jonathan: that kind of a code reorganization is not true real.

Bashonly: Yeah, I mean, we're dealing with a code base that goes back like about 15 years. Yes. And compatibility has sort of been our enemy lately. Backwards compatibility.

Jonathan: Yeah.

Bashonly: It, it, it's sort of the, the enemy of new ideas and it's sort of been holding us back.

So it's one of the struggles we face is sort of balancing backwards compatibility with. Implementing these new ideas. Yeah. You know, we have to find a way.

Jonathan: What, what isn't, sorry, go ahead. What, what wouldn't be backwards compatible? So like, are you talking like internal API changes that break some of your extractors?

Bashonly: Well it on the, the user side of things, so. We tend to, you know, never remove command line options. Ah, okay. Okay. They're just deprecated and, you know,

Jonathan: forever deprecated. Yes.

Bashonly: Yeah. Yeah. And just I think the way that you would have to operate Y-T-D-L-P would be slightly different because it wouldn't necessarily come with all sites supported outta the box like it does now.

Jonathan: Mm-hmm.

Bashonly: You would have to, you know, pick and choose which extractor repositories you wanna add. I don't want to get into too much, too many specifics here 'cause That's fine. We haven't really hammered down the details Yeah. Of how it's gonna work ourselves. So I don't, I don't wanna give anyone the wrong idea, but

Jonathan: Sure, sure.

That makes sense. So I have, I have noticed something when you go to Google and you type in YouTube downloader or, well, yeah, let's, we'll just go with that. That's what most people are gonna search for. You have to go quite a few pages before you actually get to something like Y-T-T-L-P and you first find a whole bunch of websites.

That will download YouTube videos for you and very likely run the Y-T-D-L-P source in the background on their server. I assume you've noticed that.

Bashonly: Yeah. That's interesting, isn't it?

Jonathan: That's interesting. That's, that's gotta kind of annoy you guys.

Bashonly: It, it makes me wonder why, I mean, I think. Y-T-D-L-P being deprioritized in the search results has gotta be intentional. Unless it's just some, some sort of, you know, there these other sites are somehow gaming the, the search engine optimization, what have you, but you just think you search YouTube downloader, at least YouTube DL would be the top result.

If not Y-T-D-L-P. Yeah.

So I think we've been de boosted most likely, but it just makes you wonder why would they want these other sites,

Jonathan: some of which are certain malware, some of which are certain malware and adware and all kinds of garbage on them.

Bashonly: And I mean, yeah, it just makes you wonder. Just makes you wonder. And yeah, most of those sites, they are either.

You know, powered by YouTube dl or, you know, they're not,

Jonathan: I don't know, third page. You're on the third page. Which third page?

Bashonly: Wow.

Jonathan: I mean, it could be worse, but Yeah. On this site, on, on this particular search for YouTube downloader, the, the GitHub eight is on the third page. And above you is a whole bunch of websites that I would prefer not to mess with.

I mean, I'm sure some of 'em are legitimate and, and aren't going to try to eat your babies, but I'm sure some of them are serving malware if they can.

Bashonly: I know a, a big one right now is Cobalt. And I think they're, they're fairly legit. I don't know if they show up in the search results or if they're.

Yeah. Also down on the third page of this also

Jonathan: I, I think some of it is just that the vast majority of people don't wanna run a Python script, and so YouTube knows that, or excuse me, Google sort of knows that and is going to direct them to the services.

Bashonly: I just think that they wouldn't want people using those sites at all, though. Well, that's why it's confusing to me is you'd think, you know, yeah, let's send 'em to a GitHub page where they have to figure out how to, how to install this thing and run it from the command line instead of, you know, go to a webpage where they type in the URL and click a button.

Jonathan: Yeah. Yeah. That's fair. I guess I have to wonder. Are, are people at Google and people at YouTube sort of in the back of their mind thinking about like antitrust. And so if they do, if they manipulate the results too much, then you know, at some point they might get hit with some kind of antitrust lawsuit.

I don't know,

Bashonly: maybe, but I, I don't think they're worried about that from Y-T-D-L-P, though.

Jonathan: That's probably true. That's probably way, way down. That's on the third page of their list of concerns.

Bashonly: Yeah. Yeah, though, I mean, I think, I think we are on the radar though.

Jonathan: Oh, I'm sure. I tell you what else you are, you know, I, I what I can guarantee you, you are, you guys are used internally at Google.

I can guarantee you, you guys are used internally at Google. Absolutely.

Bashonly: I wouldn't, I wouldn't doubt that

Jonathan: By, by somebody. Probably multiple people. 'cause it's such a useful tool. Yeah. Which is, which is nuts to think about.

Bashonly: Yeah. I think we're, we're definitely used internally when they write their new versions of their player JavaScript to see if they can, they can break it.

Jonathan: Yeah. That's likely true. So if somebody goes to use Y-T-D-L-P. Whatever they're trying to get to doesn't work. Where should they go? Complain slash send patches? What's the process?

Bashonly: They can always add another issue onto the issue tracker. We're 1500 going strong. We can always use some more. As long as they're willing to submit complete, verbose output.

If they can give us a log with all the details we need, yeah, then feel free to open an issue on the issue tracker. That's, that's all we ask. 'cause just follow our issue template checklist and give us the details we need. If they have more general questions or they don't feel like. You know, if they, if they feel like they can't even get to the point where they have a verbose log to give us, then they can join our Discord and ask us on there.

Jonathan: Hmm. Yeah. We'll have to make sure and get the link to the discord and put it in the show notes. 'cause that'll, that'll be very interesting for folks to follow. How difficult is it to submit a patch and get it picked up?

Bashonly: It's, we accept pull requests from anybody. We do ask that you follow recording conventions and we ask that you understand that, that we have a high volume of pull requests and it might take a while for us to get to them mm-hmm. To review them. But yeah, anyone, if they wanna add support for a site and it's not, you know, it's not a sketchy site, nothing, you know, DRM protected, nothing piracy related, then, you know, we'll, we'll review it and, you know, merge it if it's acceptable.

Jonathan: Yeah. That's interesting. So you, you guys do have kind of a list of sites that you're just not willing to support. Not piracy stuff. Ma mashed potato, mashed potato points out. People might not want to admit that they're downloading furry videos.

Bashonly: Yeah. Oh, well, I mean, you can make a burner account on GitHub and open the, open the poll request.

As you know, anonymous, furry four 20.

Jonathan: Oh, that's you. You probably have that actually. People doing burner accounts just to complain about a site not working. That's probably a real thing.

Bashonly: Oh yeah,

Jonathan: yeah.

Bashonly: Oh yeah. And then they turn into ghost two days later. Mm-hmm. So, you know, the, the account's deleted, never be seen again.

Jonathan: Yes,

Bashonly: exactly. Yep.

Jonathan: Some, someone somewhere is living in shame that they had to do that,

but thus is the internet. This is what we live with. So we talked about what's coming. What do you, what do you see? Like what, what's, what worries you about this project? Right? I know, I know some of the plans you have, but what's your what do you, what do you see coming that's gonna be hard to deal with?

Bashonly: YouTube? Just

Jonathan: YouTube and mostly YouTube. Yeah. Yeah. Are they gonna, there's a, there's a lot of, are they gonna ever roll something out like DRM for all videos that we just sort of put it into it?

Bashonly: Well, they partially did already. It's not something that's been applied to all forms of all videos, but, so there are different clients that can access the YouTube, API, there's, you know, the Android client, the web client, and then there's the TV client or TV HTML five as they call it. Mm-hmm. And that's what Y-T-D-L-P is primarily using right now because it's not subject to some of the restrictions and what have you that the other clients are.

Okay. And if someone is downloading too much, if they're making too many requests with the TVH TM L client, then they will get DRM only formats right now. Or at least,

Jonathan: Hmm.

Bashonly: It's, it's, it's like an AB test. It's not that widespread, but you know, if they. Cool it on the downloading for a few days, everything will go back to normal.

So it's something they're testing out right now. We don't think that it's gonna be applied to any other client other than the TV HTML five client. Mm-hmm. But we don't know, you know, it's not like YouTube lets us know anything.

Jonathan: You don't have the inside line.

Bashonly: Yeah, exactly. But you know. That's sort of like their nuclear option.

If they do that, then YouTube support is gone. Mm-hmm. In Y-T-D-L-P and YouTube dl. I don't think they're gonna do that though, so I'm not that worried about that. I am worried though, that it'll make the TV client useless for us and we'll have to find another way of doing things.

Jonathan: Hmm.

Bashonly: There's something else happening right now called.

Proof of origin tokens, and this is what your web browser generates to prove that it's a real web browser. Hmm. And this is, it's it generated with JavaScript and it goes beyond what we're capable of, capable of doing, and Python. And so if we have to move to the web client, then we would need to add.

An actual,

Jonathan: more like a full blown added, a full-blown JavaScript parser.

Bashonly: Yeah. We, and you know, we're not gonna write that in Python, so we'd need to add a dependency on like Dino. Mm-hmm. So that could be something that happens where if you want to download from YouTube, you also need to have Dino installed.

Mm-hmm. Yeah. Yeah, I can see that. Those are the biggest concerns right now.

Jonathan: Yeah. We've, we've talked a lot about YouTube and rightfully so, right? It's, it's the, as far as I know, it's the most popular video site in the world, but you guys have a bunch of other sort of backends that work. Do, do you have any favorites?

Like is there anything out there that you, you, you really think people should know about? Like, Hey, we can download from crunchy roll so that you can archive anime, or, you know, is there, is there anything like that that, that should be highlighted? I

Bashonly: well, crunch roll, we can't download from anymore. Oh, it went full DRM So that, well, that got removed. That got removed a couple months ago. It

Jonathan: used to work. Interesting, interesting. What I would, interesting example, I pulled outta the hat then.

Bashonly: Well, what I would say is, you know, we get these questions on discord and on the issue tracker all the time of, you know, does it work with this site?

I would say just. Get the URL and plug it in and see if it works. Mm-hmm. And I think more often than not, it will work or it'll start to work and then it'll break and then you can report the bug to us and, you know, it'll get fixed eventually. Right. But yeah, not, not any one particular site. But you know, like I said at the beginning of the show, I'm surprised, you know, at least every month.

It that there's this site that dozens, hundreds, thousands of people use Y-T-D-L-P for that I'd never even heard of before. Mm-hmm. And then it breaks and all these people come outta the woodwork and it's like, wow, you know, I've never even looked at this code before and now I gotta fix it. So yeah, just try, just try it with anything.

Mm-hmm. Is what I'd say.

Jonathan: Yeah. Yeah. Was, did the code base start in like Python two? Was there a, was there a big Python two to Python three transition for this?

Bashonly: Yeah. And that is one of the big differences between Y-T-D-L-P and YouTube. DL is YouTube. DL started in, I don't know if it started before Python three, but they have maintained compatibility going back to Python 2.6 this entire time. So they have all these compatibility modules that. With, you know, compact functions that you know will, will work the same in Python two and Python three and Y-T-D-L-P.

After it was forked it dropped Python two support pretty soon after that. So we don't have to worry about that anymore. Mm-hmm. Thank God. Yeah. But again, that's, that's actually one of the reasons where I decided I'm gonna contribute to Y-T-D-L-P instead of YouTube dl. 'cause I didn't want to have to deal with, you know, maintaining compatibility, going back to 2.6,

Jonathan: the, the mess that is trying to work in both versions of Python.

Yeah. Right, right. That's kind of ugly. Are there, are there any skeletons like that still hidden in the code base?

Bashonly: Probably we've gotten rid a lot, gotten rid of a lot of 'em and we've been sort of keeping on the same timeline as the PSF itself with, you know, if they drop support for. You know, the, the end of life of, you know, Python 3.7, 3.8.

We've dropped support for 3.7, 3.8. Mm-hmm. Right now, I think we only support 3.9 and above. So every time that happens, then we go and we try to, you know. Purge a bunch of old code, right. That we don't need anymore, but I'm sure there's still some lingering around that we haven't found.

Jonathan: Yeah. So I, I'm kind of remiss I should have asked you earlier about this.

We, this is open source. What, what is the license that Y-T-D-L-P is under?

Bashonly: So it's under the unlicensed, which is basically a fancy way of saying public domain. So when we. You know, push code. It's just out there for anyone to use however they want. Which sort of explains the proliferation of all those YouTube download sites that just use Yes, YouTube, DL, my DLP.

'cause I mean, they're free to do that if they want, but you know, I wouldn't use those.

Jonathan: Oh. I'm looking forward to the inevitable fork of this into a source available licensed business clause. Something, something, something. Yeah. Probably, probably not on the roadmap.

Bashonly: Probably not. No. We've, you know, we've thought about relicensing, but there's really not, there's just really not a great solution to that problem, I think.

Yeah. And I, I, I actually kind of like that it's public domain. Sure. I feel like, you know, anyone can use the tower they want. It's sort of, you know, takes responsibility off of me, I feel like, is, you know. Mm-hmm. This code just exists is, is what I say. Yeah. You know, it's not really my code, you know, it's not our code, it's just, it's out there so anyone can use it.

Yeah.

Jonathan: So I, I was speaking of course, very tongue in cheek about the idea of the, the relicensing and the, the point with that is like the worst of these sites that do this, that use your code, the ones that you would really like to stop, those are the ones that don't care what the license is, right? So it would, it would be a, a pointless gesture.

Bashonly: Yeah, exactly. You know, and it's not like they're, they're saying like, oh, we're powered by Y-T-D-L-P, you know, it'd be kind of nice, but no, all these sites are like, look at what we can do, you know? So.

Jonathan: Our proprietary YouTube support plugin.

Yeah, yeah,

yeah. I I've seen it. I know how that goes. Alright.

Hey, so is there anything I didn't ask you about that I should have? I know this is a hard question, just gotta do, set math in your head. Think about all the things we talked about and what your, your list of what you want to is there anything we didn't cover if we didn't get to?

Bashonly: No, I think we, we touched on a lot of stuff.

Jonathan: Okay. You got to plug your discord. You told people where to go to open issues and where to go to submit code. Those are, those are the big things, I think.

Bashonly: Yeah. github.com/yt. DLP slash yt dlp, as long as you can remember those letters, it's pretty easy. Yep. We got the link to the discord on our GitHub.

Pretty much, you know, that that's where you can download from. That's where you can get the source code from. That's our, that's our hub.

Jonathan: Yeah. Is it fair to say, let me ask you this before I get into the final two questions. If, if Y-T-D-L-P like permanently breaks with YouTube, I, I kind of have this feeling that there are gonna be a lot of repercussions from that, that people don't immediately think of.

Is, is that kind of your feeling too? Like there's a lot of, obviously these, these fly by night website services, they're gonna go down, but I, I feel like this probably gets used in a lot of other tools and, and there's a lot of places that are suddenly going to stop working if, if this goes down for a long period of time.

Bashonly: Yeah. Well, I think probably the most noticeable thing is I think a lot of creators on YouTube use it. To, you know, clip, clip other videos and include them in their videos. Yeah. So I think if YouTube support goes away, I think there will be a lot of annoyed creators that they have to find another way, or that, you know, they, they can't make these videos that clip other videos and stuff like that.

I think it's, I know it's used by journalists. Yeah. And it's, you know, to,

I think it's, it's used, yeah. Like you said, it's used for a lot more applications than, than you'd think. It's, you know, it's not just. It's not just, you know, yeah. Someone at home, you know, downloading a, downloading a funny video. They like, you know. Yeah. Right. So it's,

Jonathan: it is that too, but there's some, right, right.

I mean, yeah. Yeah. It is something we find with a lot of these open source projects that we talk about, there's a tail to them. Right. And, and not everybody can see the tail sometimes, you know, we're all, it touches things as hidden. You, you know, you guys are not in quite as many places as like lib curl.

Where there's probably a billion installs, but there's a lot, there's a lot of things that would, that would stop work. You would eventually notice that things would go away if, if Y-T-T-L-P went down or even if just YouTube shut it down. Completely interesting stuff. Oh,

Bashonly: well hopefully that doesn't happen.

Jonathan: Hopefully indeed. So I've gotta ask you two final questions before we let you go, and that is, what is your favorite scripting language and text editor?

I mean you Well, can I say

Bashonly: anything o other than Bash,

Jonathan: based on the introduction, I thought you were gonna say Notepad and HTML.

Bashonly: I, I think, well, I think I've gotta say Python. That's, that's what I write every day. So Python an editor was the other question. Yeah, a text editor. I like Genie. Or Genie, I pronounce it Genie.

G-E-A-N-Y.

Jonathan: I am not familiar with this one actually.

Bashonly: It's, it's really lightweight, which is why I like it. I don't like, like the IDs and stuff like that. Mm-hmm. And it has, it has syntax highlighting. That's pretty good. That's all I really ask for. So.

Jonathan: Cool. That is, that is not one that was on my on my radar.

I will definitely have to look into that. Very cool. Alright Mr. Bash, only, thank you so much for being here with us today. I sure appreciate you taking the time to do it and we learned a lot about Y-T-T-L-P and kind of the history and some of the things that go on, on the background that we didn't know about.

So thank you. Appreciate it very much.

Bashonly: Yeah. Thanks for having me on.

Jonathan: Yeah. Alright. That was bash only of Y-T-D-L-P and definitely a lot of fun. And like we said, there's a, there's a tail to some of these projects that you don't really think about, but they get used a lot, may or may not realize it. We've got something really fun coming up next week.

We're talking with Han Graver about LXC, the Linux containers project. Definitely looking forward to that. And then we have a couple of weeks open and then we'll be talking about. Kuna, which is a Linux security product. And excited about that one too. Appreciate everybody that's here, both those that watch us live and get us on the download, and we will see you next week on Floss Weekly.