FLOSS Weekly

Jonathan: Hey folks, this week Dan joins me and we talk with Stefano Zaccaroli about Debian, but also Software Heritage. That's the source code archiving project that you probably haven't heard of, but really should know about. You don't want to miss it, so stay tuned. This is Floss Weekly, episode 817, recorded Tuesday, January the 21st.

Incompatible with reality. It's time for Floss Weekly. That's a show about free, libre, and open source software. I'm your host, Jonathan Bennett, and we've got something very fun today. First off, we've got Dan the man, Method Dan, the original Linux outlaw. I like that intro, Dan. It's, it's fun to introduce you that way.

Dan: Thank you very much. I like that. It's fun to be introduced that way. I have to say. Yeah, it's great to be back. Good to see you, Jonathan. Hello everyone.

Jonathan: Yeah, you know one of the things I enjoy we talked briefly in the in the pre show But one of the things I enjoy about doing this show is like the group of people that we get to hang out with You know, we've got we've got dan.

We've got simon aaron I've brought some guys on in the form of you know, rob and jeff and those guys But we also will have randall and even we're having the The plan is for next week doc to come back as co host, right? And so just the fact that we get to hang out with all of, all of these guys as part of the hosting team, I enjoy.

And then, you know, there's the fact that we have all of these guests that are just, you know, every one of them are great, have, have super interesting things to talk about. We've got another one of those today. And I understand that you've got you've got some connection with Stefano Zach Zaccaroli.

What's, what's your history with, with Zach, with Stefano? I interviewed I interviewed Stefano a few times for Ledex Outlaws back in those days. Once we were just discussing earlier before we got started once when we were both pressed against the window of a bus, quite literally, physically in Brussels on, at Fosdem because Fosdem is very busy.

Dan: Busy and those buses are not built to take that many people. So it was quite funny. We were just next to each other. And then I re I didn't actually realize who he was at first. And then we got chatting and it was like, Oh, can I interview you? So we, we did it right there. And then

Jonathan: Foster must be great for that because like almost every, basically everybody that is there is potential interview material.

Dan: Yeah, yeah, you need to take a lot of spare SD cards or whatever it is You use to record on when you go to somewhere like Fosdome.

Jonathan: Yeah All right. Well, let's go ahead and bring him on So Stefano, I know that he has been involved in Debian over the years Probably still involved to some extent and the other real no Not the only other.

Been involved in OSI but the thing that really interests me is software archive, because I, I am a, I am a fan of the idea of archiving software, and I'm also aware of some of the challenges, and so I want to, I want to pick Stefano's brain about all of that but first, just Stefano's Welcome. We are absolutely delighted to have you here.

Stefano: Hello. Welcome. Well, thanks a lot. Thanks Jonathan. Thanks Dan. And thanks also for the memory, Dan. Well, that was a long time ago. It was a really fun interview to actually do.

Jonathan: So what, give us your background. Like, so if somebody asks you, what, what, what do you do? What open source stuff do you do? What's, what's your answer?

Stefano: Yeah, so I've been around for a while, so I'm myself in let's say in my real life, I'm a computer science professor at the Polytechnic Institute of Paris, but I've been a geek for my, a computer geek for my entire life. So I started getting into free software when I was a computer science undergraduate at the university.

And we were, we had the lab at the university that was managed by the faculty itself, and it was some proprietary Unix at the time. And we have a second lab, which was managed by volunteer students. And of course it was Linux. And so that. Things intrigued me from the very beginning. And I had one of my first objective was becoming one of those, you know, volunteers is that mean that was helping out students and that's, that's how I started.

Yeah. It was that yet, of course. And then the rest of history, as you say,

Jonathan: there's an interesting crossover. And I think there always has been between that idea of political science and open source. And, you know, we, On the show, one of our rules is that we don't dive into partisan politics because nobody, nobody's happy about that.

But with that set aside, there is an interesting crossover and it is a thing that it is important to have people that sort of are aware of both of those realms, particularly when you start talking about legislation that's going to deal with cybersecurity and copyright law and things like that. Is that something that you, you focused on where those two overlap?

Stefano: Oh, no, I was actually doing computer science. Oh, computer science! But, since then, especially in my days at Debian, I've been involved mostly in policy work, and so, yeah, what you say, I completely agree. So software and technology is really political, and free software, if possible, is even more political.

So, yeah.

Jonathan: Well, yes, but actually no, but actually yes. Yeah.

Stefano: It's complicated.

Jonathan: Yes. Yes. So how, how did the how did the Debian piece come along? I've, I've always thought of Debian as difficult to get started with. Like you, from what I understand, to even be able to publish packages on Debian, you've got to go into like physically beat one of the other packagers to get your keys signed.

And it's just, you know, there's some, there, there's some challenges.

Stefano: Yeah, that was part of the thrill and of the challenge at the time. So the technically, the reason why I started looking into that was that, so I was passionate about for now, nowadays, somewhat obscure programming language, which is OCaml, Objective, Objective Camel, which is a functional programming language.

And I was really passionate about functional programming. And we were using Debian, and we were using GoCamo. And as you start coding, you, you discover that, well, some library is missing, in the sense that you cannot just, you know, apt get install that library and say, Hmm, let's look at this. How could I make this package, this library, which exists in this free software available to myself, my, you know, students, colleagues, and everyone else in the world?

And that's why, that's why I started looking into this Debian thing. thing, which was indeed quite intimidating at the time. But then, you know, I find friends that were already collaborating. A friend of mine, a student as well, was already a Debian developer. So it basically got me started into the process.

That must have been 98 something. So yeah, at some point you need to meet people and then the chance to meet a lot of Debian developers that were, you know. Traveling around and, you know, getting to my city at a time that was Bologna and finally get my keys signed.

Jonathan: It was pretty

Stefano: amazing.

Jonathan: Now you, you did, you were not just content as being a Debian developer.

You actually became a, you became the head of Debian for a bit, right? The Debian project leader for a while?

Stefano: Yeah, I, I, I've been dev project leaders for three terms in between 2010 and 2013, I think. Mm-hmm . And that was, so at the time I was a postdoc researcher. I already moved to, to Paris at the time, and I was really, really involved in the project and I, I was realizing that there is a technical part, but the, I think the most interesting innovation, innovation of Dion is actually the development model.

It was the first distribution of being actually a community driven distribution. And it's also a community that essentially self determined itself. So it's a project with a constitution that developers themselves created that was before my time, but it was not really common at the time to think of self determination of online communities.

Later, it was a notion that started to become more discussed at least, but at the time it was really, really an innovation. And so I was really passionate about that. Thing. And I say, okay, what can I do to help? And beca I did a bunch of things before , you know nominating myself for for the DBL election.

But I was really passionate about improving the processes and, you know, try to see how we can make the community give. The best of themselves.

Jonathan: And you're still involved with Debbie and I assume

Stefano: so. I'm still formally a, a developer, technically it's called a non uploader developer because I, I'm not doing technical work anymore in Debbie and this day, so I, I announced the, you know, the, the, the right, the upload right associated to my GPG key to avoid, you know, reducing the tax surface.

But i'm still a voting member of the project and i'm still following The yearly election the policy decision the political discussion on the project and some legal work work That's the kind of stuff i'm still involved with

Jonathan: well while we're on the topic of debian If you if you have your sort of finger on the pulse of the project still something i'm curious about is What's what does the health of debian look like these days and i'm thinking, you know, primarily in terms of Are there still developers coming on board?

Are there enough developers that are there to handle the workload? What direction is that trending? And like, is there a, is there a funding model there? Does there need to be? Are people getting paid to work on Debian? I honestly don't know how all that works inside Debian land.

Stefano: Okay. So many questions there?

Yes. So Debian as up. So starting from the, the funding project, Uhhuh Debian as a project is actually a nonprofit project in the sense that the Deion project doesn't pay developers or maintainers or other technical roles or even non-technical roles to, to, to actually do stuff. But there are a number of companies that are.

present in the broader let's say Debian ecosystem. A big example is of course Canonical that the, that the company behind Ubuntu, which is a commercial distribution based on Debian. And so all these sorts of realities, not only Canonical pays people for making something happen either downstream distribution or in Debian itself.

So that is part of the, of the Debian, let's say sustainability model. But what's beautiful about the Debian model is that a lot of people are just. You know, volunteers to the project there might be paid by research labs or or other institutions, but they might also be just working in their spare time.

And that's correct. And it poses some challenges because when you start mixing the work of volunteers with the work of people pay to do something, you might have some sort of tensions. But it's I think it's really working well for that. And regarding the turnover question, I think it's still pretty healthy.

So probably I don't have the exact number numbers in mind, but it might be a bit lower than it used to be in the in the early days. But it's still quite healthy. People come and go and and actually. You know, making packages is becoming more efficient. So with team maintenance and with the number of automation tools, the workload is actually cheaper in terms of the number of packages you can maintain and you can act on than what it was like 10 years or 15 years ago.

Dan: Yeah. Pretty cool. Wow. So did they still have to Debbie in kilts by any chance? Cause that's something that you used to be able to identify people at conferences. I certainly do.

Stefano: I still have my Debbie in kilts and I even wore it at my wedding. Not for the entire time of the ceremony. My wife wasn't really okay with that, but at some point at my wedding party, I did wear the Debian kilt.

And that was created for the DEBCOV in Edinburgh and by an actual Scottish Debian developer who registered a Tartan. So we actually have a Tartan registered in the name of Debian in. Whatever official Tartan registry there is out there.

Dan: It's probably a bit like software heritage. There's a Tartan heritage somewhere where they've got it all archived.

And, and they're very serious about that kind of stuff. Don't get

Stefano: me started about the trademarks of Tartans because I have no idea. I'm sure there is.

Dan: Yeah. Yeah. And so sticking to the Debian kind of subject, a lot of people listening to this I know we have a lot of very technical listeners, but we have some who may be less technical might be thinking I've never used Debian.

Why does that matter to me? But you mentioned it just there about the fact that the upstream thing. So Debian to me feels like the upstream of so many distributions that I suppose I'm interested in like the relationships that Debian has with say Ubuntu, Canonical is a big one that we all know, but also things like Linux Mint, and I'm not going to go into listing how many Debian derivatives there are so many.

So how does that relationship work? Do they give back to the Debian project?

Stefano: Yeah, so that's a very interesting question, because one of the things that I I. I sit down and thought about a lot when I, when I first become DPL, Debian Project Leader, was actually understanding the role of Debian in the ecosystem of distribution at the time.

And I realized that a lot of the impact that Debian had was not directly in the sense that people were directly or primarily using Debian. There were a lot of people that were benefiting from Debian. Indirectly, possibly not even knowing that Debian exists. And so I started measuring at the time, essentially the tree of derivative distribution.

And it was hundreds of downstream distributions, meaning that people have installed those distribution. They know the name of that distribution. They don't necessarily know it comes from Debian, but they're still benefiting from all the work that Debian people were doing and are still doing today. So, It's free software, right?

So there is no strict obligation to give back changes. The license says that you can just use the code for whatever you want, as long as you respect the license. But there is no obligation to contribute back. So you have some distribution that do contribute back and that benefits Debian a lot. And then from there, it also goes down again to other distribution.

And there are others that Contribute less, maybe because they don't have the means, maybe because they are not interested, and that's fine. So it starts to get, you know, tense and complicated when people in Debian expect contributions that are not coming. And so that was part of the time of the debates between Debian, Ubuntu, and Canonical.

Dan: Yeah, it seems to me that there's a lot of scope for discussion. I hesitate to quote argument, but discussion about things like system D, for example, when Debian brought system D in Jonathan's a Fedora guy. So he's looking at us a little both like everybody knows about that. Oh, I'm aware. I am

Jonathan: aware.

Dan: So there was arguments about system D or no system D. Is it difficult to wrangle? Is it difficult to get like a clear purpose? Is it it's a lot of debate going on all the time?

Stefano: So it's I mean, I think it's the the reverse of the coin of you know, having a large community of maintainers in the sense that in theory the Debian model and the Debian Constitution actually it's it's designed so that everyone can work on their own packages and essentially they can work independently from others but you can only work independently from others As long as there are no strict technical interactions between packages.

And when you start touching like low level packages that are pre installed and they deal with the operating system and they manage services like systemd, you can have very complicated technical decisions and you can have maintainers that are on opposite side of the decision you need to take. So this is the example was a big one and the way that Debian has to govern all that is that it has a body which is called the technical committee, which is some sort of judge it's not a single person, it's multiple people, but it's a committee that can actually make decision when they are not in the scope of a single package and they are like they have broad implication on the on the technical setup of the distribution and the system, the one at the time was, was a big one.

Dan: Yeah, that was a big fallout at the time. So while I'm kind of on this tack, I need to ask a bit, but as well about the relationship with FSF and all of that kind of stuff, because am I right in thinking that Debian isn't on the approved list of, of FSF distributions?

Stefano: That, that is correct. So still to this day.

So the, I guess we, the, the relationship within Debian and FSF went through many different periods. So in the very beginning, the FSF actually funded. The work that then led to the creation of Debian. So Ian Murdoch, the founder of Debian, for a time was funded by the FSF to essentially create a distribution on top of the free software that the FSF was providing.

Either developing themselves or funding to achieve the GNU system and the GNU operating system. So that was something at the time. Then I would say that culturally and even before my time, Debian went through a big, to put it simple, open source moment. So let's say we are here for the tech part. We are here for the collaboration.

We are not here for the political part of the software and for specifically for user freedoms. Then, and that was Around my time as a Debian project leader, the two projects got closed together again. So I, I insisted a lot on the discourse that Debian is primarily about free software and one of the main goal of Debian is creating a completely free operating system, but Debian also has some non free parts that needs to be explicitly enabled by user to be used.

And the FSF didn't like that part. So this was one of the reasons. And then there were another big reason. For the FSF for not listing Debian as a free distribution is firmware. So Debian historically took a decision that essentially the core firmware that you need to make some of your devices run was okay in some part of the archive and the FSF was not okay with that.

We got closer at the time, and at some point, FSF sent out a statement saying, well, Debian is not a community free distribution yet, but they made a lot of progress toward that goal, and that was a big achievement at the time.

Jonathan: I am personally not a fan of the line in the sand in the place that FSF has drawn it there.

Years ago, we had somebody from FSF on the show, and I tried to make the point It's actually rather ironic because all of these devices have firmware on them And it's actually a little bit more user friendly if you give the user the ability to replace that firmware as opposed to just baking It in forever and the FSF has the exact opposite take on that that no No, if the If the user has to touch the firmware, that's when it's a problem.

But if the firmware is baked in forever, that's, that respects your freedom. And I just think that's the weirdest take.

Stefano: Yeah, I think I personally agree, but I, so one point that I made to the FSF specifically, so I attended LibrePlanet, the main FSF conference as Debian the time. And one argument I use is that knowing that there is a problem, Part of software, which is completely free software, and then, you know, a red line separating that from a part of non free software is actually an opportunity to teach users about the importance of their user freedoms.

So saying, okay, there is software that is not enabled by default, and it's firmware that you need. To have your specific device run. You can enable it, but we take that moment in which you need to enable it as a teaching moment. We teach users that, well, you can enable it, but then here you don't know what the software is doing or what the firmware is doing.

You cannot change it or that kind of stuff. It can be something useful for the free software movement to, you know, to use rather than just pretending that non free software does not exist. Because essentially the results of some of the takes of the FSF is just pretending that. Yes,

Jonathan: sad but true. Have things changed very recently in Debian?

I, I, I don't remember the details, but I've, I've had at least one person tell me that there are some things that are now a little bit easier in Debian, like getting, I think it was getting NVIDIA. Drivers working that that recently changed in it.

Stefano: So what they changed recently is that essentially before there was a Essentially a big part of the archive which contains either known free software or software depending on non free software And essentially to get for instance just a non free firmware you had to enable all that software So that means that you enable that to get your firmware, but then suddenly you are exposed to all the rest of the software Non free software that's there like I don't know, Acrobat Reader or you name it.

And what has changed recently is that there is a specific section of the archive to just enable the non free firmware part.

Jonathan: Which I

Stefano: think is a good improvement.

Jonathan: Yeah, yeah.

Stefano: I mean, I also have here, unfortunately, a laptop with an Nvidia chipset. So I could say that technically installing that kind of proprietary model, it's technically easy.

But I don't know if it has changed recently or not.

Jonathan: Yeah, does that change and some of the other things that Debian has done recently to make it easier to use? Does that steal some of the thunder you think from? Distros like Ubuntu and Pop! OS?

Stefano: I think it's different targets. So I, so I, I haven't been using Ubuntu for, for many, many years.

So I don't know what's the state of, you know, usability and and of those, let's say desktop distributions, but I think Debian really is a different target. So it's, it's a perfectly fine distribution for the desktop. I'm using it myself here and it works just fine. But I think the main target is being like a foundational distribution for, well, servers, of course, everything that is in the cloud.

A while ago, I was checking and in the major public, so called public clouds, Debian was the most installed VM.

Dan: It

Stefano: was a few years back, but, and that's really, you know, that's a key thing. Because to get your. Autonomy, for instance, if you want to self post stuff, being very easy to use there is really, really important.

Jonathan: Makes sense. All right. I know I am itching to, and Dan is as well, that is to get into the idea of software heritage. And so what, give us the, give us the overview to start with. What, what is this project? What is it looking to accomplish?

Stefano: Okay, so it's a project that started 10 years ago now. I co founded it with my colleague Roberto Di Cosmo and it was initially supported by INRIA, which is a national research center here in France for computer science and UNESCO for, you know, preserving important stuff for humanity.

And the key idea for this software is to retrieve Preserve in the very long term and share with everyone the entire body of software that is available in source code form that of course is a super set. So it's a larger set of all of free software. And the starting idea was that essentially. asking ourselves, well, is anyone archiving free software so that it does not get lost for future generations?

And when we first asked ourselves that question, we, we thought, well, of course, someone should be doing it. It would be crazy if nobody is right. And actually nobody was doing that. And that was a huge surprise for us. So we, we've, there were a number of other initiatives, including by UNESCO for preserving the executable form of software, like for doing retro gaming.

Or also for making sure that images stored in some proprietary formats, which are only implemented by some software 20 years ago, will still be viewable 20 years from now. So that kind of initiative existed already, but the preservation of source code was not a thing, at least not at the scale that we were imagining.

And so we set to actually create this this project. And as it happens, while we were thinking about that, Turned out that it can be useful for many different use cases. You have preservation for just, you know, not losing all the important knowledge that is embedded in source code. Do you have a bunch of research use cases?

Like I'm a researcher myself, and I've always been envious of physicians that create these? Yeah. amazing infrastructures like, you know the CERN or the very large telescope in the, in the Atacama desert. And I wanted to create something similar for computer scientists tuning software. A place where scientists with an hypothesis about how software is developed can go, run their experiment.

And then go away and do something else. And then finally, you have also a bunch of important industrial applications like create, helping with creating software bill of materials, of indexing all the software components that are in your smartphone or in your IoT devices. It's something that was not easy before and that's something that software editors can help with.

Jonathan: So, I'm going to ask a bit of a troll question, but it might be one that people are actually wondering. And that is, why do we need this? We have GitHub.

Stefano: That's such a serious question. You couldn't imagine. So another big field of use for software is open science. So, open science is the idea that all artifacts related to science should be open.

The papers themselves. Software created for scientific experiments, data, should all be open. And it's amazing how many scientific papers out there, when they want to tell you where the software used for that experiment is, they just add a GitHub link. Which is, which is a great development platform, aside from the fact that it's not free software itself, of course.

But The repositories on GitHub disappear

Dan: with

Stefano: very, very quickly. There are a number of scientific papers that that studied that that try to retrieve software associated to papers in any scientific fields. And you literally get the answer like, I'm sorry, the dog ate. The the homework that kind of answer and say, well, that was a good time owned by a student that is no longer it's been longer with us for the past five years.

We don't know where the code is. So that's why you need that sort of archive.

Jonathan: So when, when software, when you guys archive a piece of software, do you get like the entire get history? Or is it just a snapshot of where something is at?

Stefano: Yeah, so the short pitch I mentioned before is that we archive the source code, but in reality, the form of code that we archive is indeed development history.

And that is really, really important. So it means that essentially, if a Git repository disappears, we can recreate it from the archive, obtaining a hash that matches the one of the repositories that disappeared. And it's actually not only Git, it's multiple version control systems, and it's not only GitHub, for sure, All the different software platforms out there, and there are a number of very interesting use cases.

We are seeing these days maintainers of our repository hosted on public get rewriting the history to retroactively change the license and to make sure that the code that today is free software will no longer be advertised as free software in the future. So when this kind of stuff starts to happen, the importance of an archive is even greater.

Because you need to, you know, to empower users to show that that code was under the GPL one month ago or two years ago. So it means it's still under the GPL today.

Dan: So

Stefano: that are a number of use cases we are seeing. So people are really using the archive for defending themselves. Against, against these strategies of, you know, stripping rights that were there in the past.

Jonathan: So there is a, this is something that, that really concerns me with all of these data archiving things. And that is that there are laws out there, especially in the European Union, that give a right to be, I, I'll, I'll, I'll use the term right to be forgotten. That's not technically the right term, but that's one that's easy to be understood.

How does that work? When you have, well, for one thing, when you're trying to archive all of these things, but also, like, just mechanically speaking, when we're talking about Git, how does it work if someone says, I want to be removed from your database, but it's like, but your name is on half of these Git commits on this particular project.

Like, that's gotta be a nightmare.

Stefano: I mean, that is a tricky issue for all archives out there. And of course, all the applicable laws in this case in Europe, because the data, the data owner is in the world, the data, the DPO, the Data Protection Office is the one from India. And of course, if there are takedown requests, that request to remove code from the archive, that should not remain public anymore.

Oh,

Jonathan: so you, you could go in and you know, this person is now John Doe as opposed to whatever, you know, a real name. If someone wants their real name to be forgotten.

Stefano: people for reason for removing code from the archive that are legitimate under the law, they are implemented and people can remove their code.

And then ideally, if it is free software, the ideal way would be people republishing the code under some other format. And we are kind of like, yeah,

Jonathan: it's, it's such a, it's such a tricky thing. I, I guess the, the place that I come down on this is if something was made intentionally made public, I don't think there should be a right to be forgotten.

I think that is incompatible with reality.

Stefano: I mean, I don't have a specific opinion on that topic. What I say is that in some cases there is, there is to consider also the right of, you know, free software users to not lose access to some piece of free software. Yes. So, and how to balance those two rights in some cases can be very complicated.

Jonathan: Yeah. Dan, did you want to jump in here? I was about to send you a message on the back chat and he beat me to it.

Dan: That's okay. We're, we're revealing the inner workings of the show. Much like maybe we can archive them in future. Maybe Stefano can archive the inner workings of the show. Yeah. So I'm curious about how some of this works technically, really.

I suppose I've been reading through your documentation, which is excellent, by the way. And I've been reading about some really cool acronyms like sword got me. I was like, Ooh, they've got a tool called sword. That sounds really cool. So can you tell us a little bit about the kind of actual physical process of doing this?

So how does it work?

Stefano: So essentially what Git does, and I'm taking the example of Git because essentially models of most other version control system out there can be mapped to Git, is essentially creating a big graph. So your Git repository is itself a graph and each object that you store in Git as an identifier can be a file, can be a directory, can be a commit, can be a release.

And essentially what we're doing is that We are either crawling, if they're available from some public development platform, or receiving requests for archiving specific repositories, as you would do for a web page on the Internet Archive, for instance. And essentially, we are storing all the objects we retrieve from the version control system into a unified model.

This way, if a file that is in your repository is also present in a gazillion different repositories out there, we store it only once. Same thing with a single commit, and same thing if you have one million forks. Of the exact same version of the Linux kernel git repository while you store it on the watch.

You don't need to store it 1, 000, 000, 000 times. And so essentially it's a fully duplicated draft data model keeping information about where stuff you have archived. Come from in the in the public Internet and something else. So it's a protocol that we did not create ourselves. It's a protocol used by open access and platforms for store papers.

It's an interchange format so that if you deposit a paper on a specific open access archive, it can be pushed. To a different archive, and we support that for integration with open access platforms, for instance, scientists, depositing paper on those platforms can say, hey, by the way, I have an associated software in source code form for this paper.

And when it is recognized as being. source code, it's also pushed to the Software Heritage Archive via this work protocol. That's super useful because you, you can know that some specific software you have archived is associated to some scientific paper and then keep the link between the two.

Dan: Oh wow, that's excellent.

Is there a mass import of stuff from GitHub? How do projects get, do they have to request to be included in the archive? Or do you proactively kind of go out and say, these are all public? In some platforms, we archive all publicly available Packages from pipe high or indeed repository from the public guitar.

Stefano: In other cases, users request that we explicitly archive some specific forges like a GitLab instance, also operated by a public administration or operated by a research institution. People from those institutions or third party users of those repositories can request that we archive all those GitLab instances, that GitLab instance in full.

Dan: Wow, excellent. And where, I imagine physically there's quite a lot of data here, so do you have masses of cloud servers and stuff out there that you put the stuff on? Are you duplicating it all over the place so it can never be lost?

Stefano: So in terms of data size, it's about two petabytes of data. So it's big, but it's not, you know, a video of video archive big.

And we have, of course, multiple copies because the only way you can keep something safe is to have multiple copies of it. So we, we make a distinction between copies that we operate ourselves, like to protect against the virus. Discs failing or that kind of stuff and independent mirrors that even if we wanted to, we couldn't destroy ourselves.

So we have three copies of ourselves that we operate. One is on bare metal, also here in France. The Inria data center is a couple of racks full of, completely full, I think, to this day. And Two other copies on public cloud that offer in kind storage space. One is Azure from Microsoft and one is Amazon S3.

And then we have independent mirrors. So one full mirror is already operating, operating from Italy. And we're in the process of setting up another mirror in Germany and another one in Spain.

Dan: Oh, wow. Excellent. And talking about the legal side of this as well, you've mentioned a bit about licensing and stuff.

That really interests me. What what set of licenses do you accept? Is it the stuff that relates to, say, the OSI approved licenses and things like that?

Stefano: So right now is really is much larger than that. So everything that is public is archived. And in some cases you can archive stuff for Public interest, even if the license is not speaking for software, but in addition to that, we do some, let's say, mining of the stuff.

We have archives that we recognize the license of the software. We have archives. So you can actually query that guy. Not in batch mode, but for a specific project, you can see what the license is of that software. And if you only care about, you know, OSI compliant licenses or FSF compliant licenses. We have the metadata to actually support that kind of searches.

Dan: Oh, wow. That sounds, that sounds brilliant. So how many people are involved in this? Because obviously, I know you started off with just two people, I believe. So is there, is there now a growing team?

Stefano: Yeah, that is correct. So in, in terms of the different sort of roles, but so we are about 20 people operating on this, and I should mention also that is a completely nonprofit initiative.

So there is no company associated with that. It's really for the public good. And you have all sorts of different roles. So you have software developers. So we have operators, system administrators. We have a bunch of researchers that are also doing research on this kind of stuff. And that is the part I'm mostly focusing on these days.

Okay.

Dan: And how do you keep it going financially then? Is it through donations and so on?

Stefano: So the the source of funding is diversified because it's the best strategy for financing any non profit initiative So we do accept donations, but the bulk of the funding comes from sponsors that are in part public sponsors like I've already mentioned inria, but there are also a bunch of other institutions that can be specific universities or public bodies around europe And we also have a few of the tech sponsors that finance a part of the project, but it's quite diversified.

So we are not dominated by any single sponsor and that's a very healthy position to be.

Dan: And on the subject of diversity, sorry, I'm jumping in again, but on the subject of diversity you've got some stuff to do with DEI. Dave, do you want Diversity, equality, and inclusion and trends and so on that you've been looking at through the research.

Can you tell us any more about that?

Stefano: Yeah. So as part of my, so here changing hat a little bit. So I'm a researcher myself and I'm a lot of the stuff I'm researching is related to free software. And of course I'm using the a lot of the public data that software heritage is producing for, for my kind of research, which tend to be very large scale.

So some of the studies I've done in the past are related to diversity in at least a couple of sense, gender diversity, and. Geographic origin, diversity, and essentially you can mine the software heritage archive to see who is committing code and seeing what is the ratio of male contributors, female contributors, and doubt that is evolving over time.

And you have a very long window of observation because we have not, well, we started archiving in 2015, but of course, version control system preserve history. So, and you have the timestamps. And so you can observe essentially. Something like 50 years of development history as recorded by version control systems.

So in one of the study I did a few years ago, we observed the ratio of yearly contribution coming from women. And as you can imagine, in the entire body of public code out there, it's pretty abysmal. So it essentially, overall, overall, 5%. But the trend. was actually growing yearly, and it reached for the first time, just before the pandemic, to 10 percent of the yearly contribution coming from women.

And unfortunately, the COVID pandemic changed the trend. And we have actually proved in a recent paper that it was not just a coincidence. And actually, the COVID pandemic caused the inability of women to contribute to free software, or at least to public code in general. This is not surprising, because essentially, The pandemic has heightened some of the interesting discrimination that exists in households and whatnot.

But we've actually proven that it has been the case also for public local contributions.

Jonathan: So in, in that particular realm, one of the things that's always interested me is, and I kind of see this as a strength of open source and the way things are done, but like a lot of commits just come with an email address and that is sort of inherently non gendered, right?

I, I'm just. Sort of curious, how do you go from all I have is an email address or even all I have is a name? and And it's it's sort of dangerous these days to even make an assumption about gender from a name how do you how do you go from that to determining whether a commit comes from a man or a woman?

Stefano: Yeah, absolutely. So there is essentially a spectrum of the kind of methods you can apply to this kind of studies Which depends on the size of your sample. So if your sample is small enough The right way to do is just go and interview people and you ask themselves which gender, what gender you identify with.

But here we're talking about tens of millions of authors, so you cannot possibly go to ask tens of millions of authors and you shouldn't do that either because they've not opted in to be interviewed or that kind of stuff. And so the approach that you use, you actually recognize not from the email, because it's usually not very informative, but from the full name, you recognize their there's a number of characteristics can be the gender, or it can be geographic origin, and you validate that with external data sets.

For instance, people can self declare. The gender on the top metadata or their own pages, but you also have some ground truth data set. For instance, we obtain a data set from the Olympic committee, essentially, with all the statistic of all the athletes that have ever participated into the Olympics with their name, their origin, their gender, the country that we're coming from, and you can use that to essentially.

Test if your automatic tools for detecting gender or geographic origin is correct or not. Of course, it's not as good as interviewing people, but for analyzing such huge data set in aggregate form is, is good.

Jonathan: Yeah. Interesting. What, what are some of the other fascinating sort of scientific queries that people are doing?

What, what, so we kind of dug into gender and you've mentioned a geographic area, which is super interesting to think about. So if you want to speak on that briefly, you can, but then I'm curious, like, what, what else is there that people are looking for? In, in the status set.

Stefano: A quick thing about geographic origin, because it was really fascinating that we were being able to observe essentially the history of technology through the lens of geographic origin, like in the early sixties, you see a lot of contributions coming from North America.

That was the early year, next day. And then you see later in Europe coming up and in more recent years China and other countries coming up again. So this is really, really fascinating, but so more than that, there are a bunch of other things. So one thing is software evolution. Like, you would be sort of surprising, or maybe not, to know that the ratio at which new code is published has been exponentially growing for more than 20 years now.

And is stable in both the number of new commits that you see coming up, or of new files that get published into public code. So there is exponential growth of code that is, at the very least, Publicly available and a big part of that is also called is actually free software. So that's great. So the humanity is producing more and more Digital commons and releasing it to the public for others to see that's another thing and you also have a bunch of Studies related to security for example You can observe how vulnerabilities propagate from one repository to another.

You can observe vulnerabilities that are fixed in an upstream repository, but not integrated into downstream forks. That's another big example. And then you have a bunch of technical advancements that you need to create, essentially, to be able to analyze data at this scale. And so we've been working a lot on compression techniques.

So, can we make the entire body of software that we have archived fit on your laptop one day using specific compression techniques? Maybe yes, maybe no. Can we make the entire development graph fit in the memory of your single server that you have at home? The answer to this is yes, for instance. We have developed techniques to do that.

Based on compression techniques. So graph compression techniques. Sorry. So you see, it's really a huge ramification of thing that you can do. You can study using this data and you have to develop yourself to make studying these data actually possible.

Jonathan: Yeah. Super interesting. So what, what about.

Documentations like I'm curious about the things that are that are included in this kind of repository and is documentation in scope. So obviously, in some cases, documentation is just part of the source code, you know, it's part of the get repository. And I would imagine that that sort of gets hoovered up automatically, but brought more broadly than that is is documentation.

And I'll get to a more specific question about this in a second. But like website content, is that is that in scope? Is that something you're thinking about?

Stefano: Yeah, so the essentially the choice in software heritage was to at least initially archive what is stored in version control system repositories that are publicly available and we select development platforms that are usually used to develop software, but you're absolutely right that incidentally archive a lot more than that per capita documentation, archive essentially taking the example of guitar, Everything that people could put on GitHub.

And there is a lot of trash in there. Okay, you have, of course, all your random website. You have people using GitHub just to create their portfolio of contributions to look good on, you know, on the market. Yeah, that sort of thing. So the solution there that we well, our approach is being we archive everything.

And then we develop techniques to recognize what's what. For instance, we develop classifiers to decide whether automatically whether a repository is a website, is a data set, is a is some source code. And you get from there. So downstream users of this data can search what they care about. And only consider that in your analysis, for instance.

Jonathan: So you could, someone could put together a query and say, show me all of the websites that are you know, hosted on GitHub pages, you know, or show me all of the places where people use GitHub to manage their website as a, you know, as you just do a Git pull to be able to do updates to their website.

Stefano: So that is correct.

That said, we are not operating ourselves a cloud service that can, you know, at the CPU capacity or memory capacity or storage capacity to execute queries. So we're essentially the storage and computing resources of the project are usually mostly for their cutting part, but we create data sets, for instance, that.

Researchers can actually use and do these kind of queries on their own hardware. Sure.

Jonathan: Sure makes sense So this this leads into sort of an obvious next question and that is Are you do you anticipate are you planning on changes for what exactly? The the archive holds, you know, are you going to expand into we want to archive more web pages?

Stefano: Web page is not really but we are getting into the territory of for instance Issues, bugs discussions in pull request, because essentially if you take like an archivist or a museum curator, curator approach to rebuild the history of a specific software project, especially a community one, all that kind of things are really, really relevant.

We decided not to archive that kind of stuff in the beginning because a common data model for that stuff looked way more complicated than the common data model just for the code. And but we are starting to look into that right now. So archiving issues are kind of a pull request. Discussions is something we're looking into right now.

Jonathan: So something something along the lines of archiving the Linux kernel mailing list.

Stefano: Well, yes, I guess so. Not really. We're not really specifically looking into mailing list. We're first focusing on what is Part of the common development platforms out there, and if you look at most of the Gitforges out there the GitHub one, but also all the GitLab or GTI instances usually support at least issues, discussion in issues and discussion associated to pull requests.

So that's what we're starting to look into.

Jonathan: Yeah, I imagine at some point though that's going to have to include emails because there are still Quite a few projects actually that their discussion and their issues and their pull requests all happen over email and their mailing list

Stefano: Yeah, they are correct.

Jonathan: Yeah,

Stefano: but to be honest, that's easier to handle than you know All the different pull request discussion and whatnot, but they are more difficult to link To be interesting objects in the archive. More difficult to link to the commits and what not.

Jonathan: Right, we already have tools to be able to to collect emails all in one place.

Stefano: Yeah, and so actually Jemaine, that was a, I don't know, has it been restarted? Because Jemaine was a precious resource, but at some point it was discontinued. I don't know if it has been restarted since. Okay, anyway

Jonathan: so in the kind of in the pre show, maybe before Stefano joined even I, I was talking to Dan about the internet archive and, you know, that is a super useful resource, but they are also sort of seeing some trouble and, and it, it kind of occurs to me that you guys are in, in a very small slice of it are a backup to what the internet archive does.

Stefano: Yeah, so the archival model is pretty different in the sense that the Internet Archive takes a traditional archivist model in which you archive stuff and you put the stuff you archive in a box, and you can retrieve that box to obtain this website, this webpage, or this zip file that you have archived, while we take a more unified model in which everything is intertwined.

So that's the big difference in terms of archival model. But other than that, yes, it's a precious resource. I've seen some of their str technical struggle due to attacks and whatnot in recent years, and that was really scary

Jonathan: even for us. Well, yeah, so I, I genuinely, I, I hope so. Like, I hope you guys are looking at that and taking notes of there are trolls out there, there are these problems out there, and it's probably going to come to your door at some point.

Stefano: Yeah, I mean, it's, it's, it's only an uphill battle, right? Because we're, we're, we're fairly size. Team, like 20 people is not nothing, but it's not the fire capacity of the huge companies out there that have, you know, if they have an attack, they can just deploy 50 people to defend against that attack. So, but, so we do what we can and we take, of course, security very seriously.

Jonathan: Yeah. What do you think the history, excuse me, the, the future of archiving looks like? Like, so this is obviously something that's very important, I would say to humanity. Both the internet archive and software heritage, both of what you're doing. And there are, there are other archives out there. I'm, I'm not sure of the details of all of them, but I know there are some other ones out there.

What is, what does this look like in the future? You know, are we, are we, are we getting to the point to where the things that are being archived are just going to become so overwhelmingly huge that it's not, it's not practical for a small organization like yours to be able to do it are we looking towards a future where maybe governments need to step in and.

Budget towards this.

Stefano: The digital preservation community is something really fascinating. So there is a huge community of both practitioners, companies working on that researchers doing the groundwork for how to do that properly. And it's fascinating. Everyone focuses on different, let's say, kind of artifacts.

And the question of size is really interesting. And I was surprised that one of the first. questions we got from the digital preservation community when we present a software heritage was indeed about sustainability. And sustainability is not only funding that we have already discussed, but there's also sustainability in terms of sizing.

So is it doable to archive what you're archiving? I think with software source code, we are in a kind of a sweet spot in the sense that very precious human knowledge, Takes a lot of time to produce, usually brain time, and then is, let's say, serialized into a fairly compact representation. Like, every developer knows that they can spend a day finding a bug, then result in a single line change in a piece of code.

So in comparison to, like, archiving videos, public videos, where you can do a 10 second HD video of a cat and get a gigabyte of data, I don't know, maybe not a gigabyte, but a lot of data. We are in a safer spot than, well, let's say archiving the other kind of format. But still there is a question of how you will find something interesting into the trove of the 300 million projects that we have already archived today.

And we're working with archivists, curators, and to actually enable this kind of this kind of research. But we see ourselves as a necessary building block for doing that kind of work in the future, rather than being the ones that should, you know, answer that right now before, instead of letting the expert of that part work on it later.

Dan: So I'm interested in security as well. You talked about. The security of the archive itself, but I'm thinking of the useful, there'd be really good use case for the archive would be tracing security vulnerabilities through different pieces of software and how they've affected it over time. Is anyone doing that kind of thing with it?

Stefano: Yeah, as a matter of fact, we're working on that right now. And so it's something that is already done via other techniques. So for instance, you already have a bunch of. even, you know industry products that do that based on package metadata. The way we work on that is looking at the commit graph. And we're, we're trying to see if the increased granularity, which you can do can help finding vulnerabilities in projects that might be unaware of that.

But there are a bunch of other secure, relevant security use cases for the archive. Like there are people that are trying to reproduce vulnerabilities. So there was a vulnerability 10 years ago in a specific piece of open source software. Can we reproduce it today to see if the resultant time were sound?

Or even just for historical reasons, like the previous talk, right? And that are kind of activities in which a software archive, a comprehensive software archive like Software Heritage can help. Because you need the vulnerability, you need all its dependencies. And, and, and only with a comprehensive archive, you can know finally all you need for that.

Dan: And I heard you say 300 million projects there. I mean, I'm sure that's a ballpark figure, but I was going to ask, do you know how many how many projects are actually being archived right now?

Stefano: So we do know. And so project is kind of a blurry term that we use in, in for software a lot without Defining it properly because it's very hard to define.

And so the figure I've mentioned is if you want the number of URLs that we archive, where a URL can be the URL of a specific Git repository, or the identifier of a specific package in some repository. Package manager repository like pipe or NPM or or or whatnot. And we have 300, I think 30 millions such software origins that, the way we call them archived right now.

Dan: Wow, that's amazing. Is there a a target that you'd like to get to at some point? Like is, you know, we want to get to a billion or we, no,

Stefano: not really. There is no target question. We, we get a lot, and it's really hard to answer is that, can you estimate how much you're missing? And the answer is that noise.

That's very hard to do.

Jonathan: So I am, I am listening to this and I'm, I'm, there's a thought that's going through my head. And that is, you guys saw this need to archive source code. And you jumped in and it's there. And so it's, it's taken care of now. What is not taken care of, right? And so I'm thinking of things like you know, the old GeoCities.

Lots of people had little websites put on GeoCities, and there were some efforts to archive those. And I think Perhaps all of them were archived, or at least the majority of them. You've got the internet archive archiving things that they can get to from the internet, but then you have places like YouTube where people are archiving things to YouTube.

You can imagine a future where. Google pulls the plug on YouTube because they, they do that sort of thing. And then I'm thinking of even things like, you know, there are, there are TV shows and movies from years and years and years ago that never got digitized and there were fires in the you know, in the place where the studio was storing their master copy of it.

And there are some of those things that, you know, as far as anybody knows, they're just gone forever. And it brings to mind this question of like, what's the next frontier or, or what, what are we missing? Is there some place out there that. There needs to be an archiving effort that's, that's not there. You know, do, do you have any, any feeling of something that, boy, it would be nice if the, if this was archived in a, in a safer way than it is now.

Stefano: Yeah. So let me stick to my comfort zone or focusing on computer stuff. And what we discussed thus far is essentially. Public code that we archive, but there is a lot of important code that is not public, not because it's proprietary, but just because maybe it's not even digitized, it can be on a punch card.

Okay. And we, in computer science, we're in computing in general, we are still in a pretty lucky let's say period in which a lot of the funding fathers and funding mothers of computer science are still around and it's the time to go see them and see if they have that floppy disk or that punch card or that.

magnetic tape still in a drawer and work with them to digitize that stuff and archive it. We have done this for a number, for a very small number of historical software, but it's really time we do that for essentially all the important historic software that have made the history of the world before it gets lost because People just are no longer with us anymore, and nobody else knows that there is something important in that doorway.

Jonathan: Yeah, so you talk about not proprietary software. I think there's a, an argument to be made that proprietary software, really, it would be ideal to have archives of that source code as well. And just, you mentioned earlier on retro computing and retro gaming, like, I just, I kind of shudder to think, but I wonder, like, how many of the 8 bit and 16 bit era games is the source code just irrevocably lost?

Stefano: Yeah, and I've been talking with a lot of people developing retro gaming emulators, so free software retro gaming emulators, but they have to deal with, okay, how do we support this specific game that is, you know, From this specific realm, and they get a lot of issues in which in some cases they cannot do that.

And it's even hard in most cases to identify who are the right owners of that piece of code. So maybe you have the bytes, but you don't know legally who has the rights on that piece of software. And I mean, don't get me started on proprietary software. So for me, we could even as well expropriate all the proprietary software, make it free software and archive it all.

But it doesn't look like we're in a world where this is going to happen anytime soon. But the day it happens I want to archive it for sure.

Jonathan: Yeah. I don't know.

Stefano: On the other hand, it's more complicated for companies, but for code written by individuals, if it is publicly available at some point, the copyright on it will expire.

It will take decades, but at some point it will expire. So it is indeed important to archive that as well. You usually cannot do that with a public archive like Software Heritage, but you can imagine an escrow schemes in which it is deposited somewhere and the day it becomes the day copyright expires is automatically published and automatically archived.

Jonathan: Yeah, you know, I think, I think a lot of people are with the resurgence of retrocomputing and retro gaming. I think a lot of people are sort of starting to think about that. And every time there's another story about, you know, how something is just lost to time. I think more and more people are sort of becoming aware of the problem.

And I don't know that we're, I don't know that we're ever going to completely solve it, but just having more people paying attention to it is, is a win at least.

Stefano: Absolutely. I mean, people think about that. I mean, people think that, you know, GitHub is not an archive or YouTube is not an archive, you know, sending out that message, at least thinking about where I should put this stuff to maximize my chances that it will be available in the long run.

This is a very important message to send out.

Jonathan: And even, even the fact that you know, in the United States, when things started entering public domain again, that the, the, the companies, the, the, the usual suspects of companies that don't like that, they did not try to pass another, you know, get another law passed that would extend copyright, you know, another 20 years.

And I, I did some reading on, I've done some reading on that several times, and the statement that they made, that those companies made was, There is enough awareness of the issue now that they didn't feel like that they could make it happen. That there would be so much pushback from the, so much outcry from the public that it wasn't worth even attempting it.

And that's, it's, it's a huge win.

Stefano: Okay, that's interesting. So I, I think we all have in mind that there's a diagram on, on Wikipedia on the evolution, the periodic evolution of copyright terms over time. And so you're saying the, pulling this out this day would be more difficult than in the past? Well, I hope you're right.

I'm skeptical, but I hope you're right.

Jonathan: Well, so in the United States, at least things are entering public domain. And so we have public domain and it was not that very many years ago that nothing was entering public domain. Right. And, and so I, I saw an interview where one of the CEOs from one of the big companies that was behind a lot of those laws, in fact, The, the law was, was named after them, not, you know, not on the, on the actual law, but people were referred to it as the, the nickname that it was named after this company.

And there was an interview made where they, they made the statement that there would be enough public outcry against trying to do this again, that they opted not to. They just didn't think it could happen. Very interesting. So, yeah. All right. We are. At the bottom of the hour. And there's a couple of questions that I am required to ask you or else I get in trouble.

And you, as a computer science major, I'm sure you've got answers for these and this, what's your favorite text editor and scripting language. So I made the news when at the time I was the Debian Vim maintainer and they switched to Emacs. So I'm still on Emacs these days, but with evil mode. So with the VI key bindings into Emacs.

Stefano: And

Jonathan: scripting language

Stefano: scripting language I would say Python.

Jonathan: Yes. Yeah. That's, that is very, very typical of these days. Do you do anything with OCaml anymore?

Stefano: No, but so that place in my heart has been replaced by Rust. So I'm a big Rust fan. So for everything that needs to really scale, I ditched Python and switched to Rust.

Both of them with bindings and whatnot.

Jonathan: Yeah, yeah, I I finally sat down with the this year's Advent of last year's Advent of code. I finally sat down and started doing some rust work. I made it about 3 days in before everything else started happening and I had to shelve that again, but I will eventually get back to it.

Stefano: I got started with Rust. I also decided to do Advent of Code. It was a few years back.

Jonathan: Yeah. It's a, it's a good way to get started on it. It kind of gives you an excuse. Yeah, exactly. Exactly. All right. Stefano, thank you so much for being here. It was an absolute pleasure. And we will definitely have to have you back in a year or two maybe six months.

I don't know when talk about when things have changed because things are obviously always changing in this particular realm, but we sure appreciate you being here.

Stefano: Thank

Jonathan: you both for the invitation. Yes. Bye. All right. What do you think, Dan?

Dan: A great discussion and such an important project as well, because as, as Stefano was saying there, you know, so much of this stuff could be lost otherwise.

And it's really important that we have, you know, a good archive and they're doing a great job of it, so it seems.

Jonathan: Yeah, you know, and we, we mentioned this a couple of times, but there are, there are things that you read about that have been lost to time. And it's just, it's, it kills me every time I come across one of those, you know, like you, you have a memory from childhood.

Oh, I remember this, this TV show we used to watch when we were little kids. And sometimes you, you go and you search for it and here's all the, here's all the episodes. They've done a re release on DVD, or here's a bunch of the episodes on YouTube. But sometimes you go and you look one of those up and it's, yeah.

Those were never digitized and they were lost in a fire in 1987 and it's like, oh man, just forever gone.

Dan: In the case of the BBC and in the UK, where I am, they actually ran out of tape and started to record over old episodes of things. So they recorded over lots of episodes of Dr. Who. So if you're a Whovian, you know, you'd be upset by that.

They, yeah, they recorded. new episodes of soap operas and things like that over these classic episodes of sci fi and all kinds of stuff just because they were like, we're not going to buy more tape. We've got lots of tape here. We're just going to reuse it, which is such a shame. Yeah, it is. It is.

Jonathan: Yeah. We could, we could dive into that more.

I, I have questions. I've heard that before, but there are, there are just questions that come to mind, but we don't have time for that. So there, is there anything that you want to plug before we let the folks go?

Dan: Yeah, so I'm, I'm heavily involved with Liverpool Makefest, which we've talked about in the past.

It's the biggest maker kind of celebration in the UK. We have about three, 4, 000 people come along. Sometimes we've had up to 6, 000 in the past. Not all at once because they tend to pass through during the day, but I'm just in the early process of sorting stuff out for that this year with my colleagues who run that.

So if you go to liverpoolmakefest. org, you can find all the information on there. And I'd encourage people to to give that a look. And we're coming up on the, it's going to be in July. It's the first Saturday in July. So you've got quite a bit of time. So if anybody fancies coming over from other countries, we have people coming from Canada, we have people coming from all over.

So a distance is not an excuse, Jonathan.

Jonathan: I am, I am making it out to one convention this year and assuming that goes well, maybe next year we'll expand a little bit more, but I don't, I don't know what the future holds for me. So we will just have to see. But yeah, I, I, I've got several friends now that are going to be at FOSDEM.

So that'll, that'll be fun. I'll get reports from everybody on how it went. Alright, I got a couple things that I would like to plug as well. Of course, we have Hackaday, my work there. You got the security column goes live every Friday morning. Make sure and check that out. We appreciate Hackaday as the home of Floss Weekly.

Also have the Untitled Linux Show, which is over at Twit. Dot TV, make sure and check that out as well. If you want more of my ramblings, as we said today. But yeah, we sure appreciate everybody that watches the show and listens, both those that get us live and on the download next week, as we said, we're going to have Doc Searls back as the co host and we're going to talk with someone from CIQ and that's actually the company behind Rocky Linux.

I am very much looking forward to that. We've had Alma lakes on the show. It's only fair that we have rocky links as well. So make sure and be back for that next week. And again, thank you to everybody. And we will see you next week on Floss Weekly.

This week Jonathan and Dan chat with Stefano Zacchiroli about Debian and Software Heritage!

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week Aaron joins me and we talk with Simon and Stefano about the open source AI definition. That's something that was just minted and did not come a moment too soon. It's a super interesting conversation and you don't want to miss it, so stay tuned. This is Floss Weekly, episode 816, recorded Tuesday, January the 14th.

Open source AI. Hey folks, it's time for Floss Weekly. That's the show about free, libre, and open source software. I'm your host, Jonathan Bennett, and we've got something that we've been teasing, it seems like, for months now. We're actually going to do a deep dive on the open source AI definition, and I've got a co host that is not also part of the team that wrote the definition, and you'll get this joke here in just a second.

But I've got Aaron with me. Welcome, Aaron. Hey, thanks. Thanks for having me and Aaron is going to be sort of our AI optimist. Is that how you described yourself?

Aaron: Yeah, a little bit of an optimist. I don't think it's taking over the world as quickly as people think, although it's doing a lot of crazy things these days.

It seems like But, but I, I don't, especially since I do use it on a daily basis for work and for other stuff I'm working on. I, I, I tend to be, if there's one person in the room, that's a little bit more optimistic, I tend to be that person. So,

Jonathan: yeah. And I've, I've explained this on the show before my, my theory, working theory as of now, is it artificial intelligence falls into.

Sort of the same place that the crypto bubble did, but also the same place that the dot com bubble did. And that is that it's a bubble, people are doing dumb things with it, trying to figure out where it makes sense, but once the bubble bursts, it's going to stick around, and it's going to be something that sort of changes the way the world works, indelibly.

And that's obvious with dot com, and you know, with cryptocurrency, we're still sort of in the process of figuring out what that looks like. I think it's a good analogy. Yeah.

Aaron: 10, 10 years from now, we'll, we'll kind of have to think back like what life was like before we had AI just auto, you know, just assuming that it was there and that we could ask it to do things.

Yeah.

Jonathan: It's, it's the same, it's the same difference of, of worldview that I have with my children. Because I remember before the internet was just always there, and my kids don't. It is always, they've always been connected, you know, in some way, the household has been at least. So it's just totally different.

All right, well, let's let's bring the guys on. So we have Both Simon Phipps, which that is, that is the, the joke that I was making that one of our, one of our co-hosts is also a guest. But we've also got Stefano, Mali, and welcome guys. And you two are the absolute experts, from what I can tell in , what it means for an AI or for an LLM to be open source.

And let's, I guess, go to Stefano first and sort of taking that as the prompt to use an LLM term. Tell us what, what is all of this about?

Stefano: What's oh, thanks for being thanks for

having me. It's it's a pleasure to be here. Yes, sir. It's so, so what is open source AI? Yes, let's start there.

It's. Yeah, it's really not that different from other open and, and open materials or or artifacts that we've been thinking of. We need to have, we want to have freely, free access, freely available access. to all the components and all the pieces that have made that artifact that you have received.

So it sounds, sounded really simple and almost trivial at the very beginning of the conversation that we had almost three years ago, but it was quickly, we quickly realized that All the paradigms that we were used to apply, like the term source just to get started with, didn't really match the technology.

And so we had to study a little bit the issue and we had a long process and the end result that I can today, I can simply say that an open source AI is an AI is a system. It's, it's it's a system that makes you. gives you a free availability to all the pieces that made it. That includes the, the parameters, so the, the results of the training, the, the code that does all the, all the training, the code that produces the training data set and the data itself, when it's possible to distribute that.

And that's, that's it. Sounds simple.

Jonathan: I'm, I'm kind of thinking of the, and I know this is not exactly the same thing, the free software and the OSI, those are two like overlapping but at the same time separate things. But I can't help but think of the, the five freedoms that the Free Software Foundation talks about, and that's the freedom to, when it comes to software, to run, to copy, to distribute, to study, and then to change and improve a piece of software.

Are, are we kind of talking about the same thing when it comes to open source ai or is it, is it a similar sort of overlapping definition? The way that the OSI and the free software definition is l

Stefano: right. Let, lemme start from, from scratch because maybe, maybe the whole concept of open source AI definition throws, throws people off balance because what we have done.

The principles are really the same. And in fact, if you look the, at the document that we have published the OSI published at the end of last year, the, the document really lists all those freedoms that you are talking about, like an AI, and we talk about systems, I can talk about it later, but the, the, an AI pieces, in other words, something that something that produces an output infers an output based on an input is, needs to be made available.

Needs to be able, you as a recipient of that system, need to be able to study, to share it, to understand to to run it you know, to execute, to have those to have those outputs generated for you. And you have to have also the freedom to modify it, change it, change how it works so that others can enjoy the same freedoms that you have received.

So those principles are really the same. What is really missing when you look at the definition of free software? It is that sentence that says the precondition to exercise the freedom to study and the freedom to modify the software is access to the source code. And that's the, the words, those are the words that we were missing when we started looking into AI specifically.

We didn't have a very good way, we didn't have any way of understanding how an AI system can be really studied deeply. Modified to change its behavior, right? That was the, that that's what we have researched for the past almost three years and what we came out with is a, is a way to describe. The equivalent of source code for software and, and which in open the, in the open source definition, it's a definition point number two, it's called the preferred form of making modifications to the, to the code, right?

It's not just the source code, but also instructions on how to build dependencies. Of libraries and versions and, and of course, language compilers and things like that. All those things, knowledge about those pieces need to be shared in order to make, to have access to the source code.

Jonathan: Yeah, that's interesting that you mentioned that.

I'm trying to find the exact my notes on this, but there was a, there was a court case in I believe it was in Germany here recently, where a router manufacturer used some code that was under the GNU, the lesser public license of the LGPL, and they got sued, not because the code itself was missing, but that extra stuff, like the scripts needed for compilation and installation.

were missing from, you know, their, their source code repository. And someone sued them and said, no, you've got to, you've got to provide this under the LGPL as well. And I, I've commented before that every time, like, the GPL, the LGPL, or any of the other, but particularly those two, because they have, like, the strongest copy left protections in them, every time they go to court, I'm kind of, like, I hope this turns out okay Because the there's sort of this nightmare scenario that a court says no no No, this this is a this is a contract not a license and it's not a valid contractor You know, there's various ways that that could go poorly but the german court found that yes, not you know, not only is it a valid is it a valid agreement, but they they It looks like, you know, reading the, between the lines here, it looks like a conclusion was come to before the court case happened, but then the court case kind of rubber stamped it.

But the extra stuff was shared. And so it was it was confirmed that yes, you can force someone to share your compilation steps as a part of the, the source code license. So that's kind of a, it's, it's good to see that confirmed in

Stefano: court, right? It, it's really, it's a really a, a fundamental component.

A fundamental piece of, of the whole movement is to have access to the source code as defined as the, the preferred form in which a programmer would modify the program.

Simon: Mm-hmm .

Stefano: You, you, you know, obfuscated source code is not source code in this, in the context of the open source definition. Missing information about.

Like, build scripts is really not open source code.

Jonathan: Yeah, so, in the, what OSI done, has done, is they've put together a definition for what open source AI, what it has to be, like the minimum requirements. I guess the next step then, or maybe this has already been done, is to begin to produce OSI compliant licenses for AI models?

Is that, is that what's sort of next on the horizon? I'm jumping way to the end with this question, but it just comes to mind and I'm very curious.

Stefano: We, we are, as OSI, we are ready to start evaluating licenses that do not cover squarely or exclusively software. Like historically, the OSI has never Taken into consideration licenses that cover, for example, content music or.

Databases or other things that, that are not necessarily software. But with we are ready to evaluate other sorts of licenses against the, the open source definition, the original one, the 10 points, the 10 points one, are there, there are efforts that we are aware of. Of groups that are writing new documents.

They're not licenses, technically, or they're not necessarily going to be licenses, but they're, they're terms of use and distributions. There are other legal terms that are squarely new and, and cover only specifically parameters data and data sets and, and code also. And, and they're all comprehensive.

That's it. put together. Yes, we're ready to do it. Yeah. To review that

Jonathan: has in, and this is a, this is another sort of strange tangential question, but when we talk about open source beyond just software, what immediately comes to mind is open source hardware. Has OSI been involved with any of the open source hardware efforts, like defining what that looks like, or has that been left to you?

You want to jump in Simon? Yeah. So you have, you have thoughts about open source hardware,

Simon: don't you? Yeah, well, it's, so this came up before Steph's time. There, there is another organization called Open Source Hardware Association, or Oshawa.

Jonathan: Mm-hmm .

Simon: And Oshawa you know, did what frequently happens in open source communities.

They gave us the great. Using our logo as the basis for their logo and in trademark law, that's a big problem because that means you have to ask them very politely not to do that. And and so at that time, which was about 10 years ago, 12 years ago OSI and Oshawa had to reach a legal agreement agreeing that they would deal with open source hardware and we would deal with open source software.

And as a consequence, OSI has never done that. Actually got into defining what open source means in the world of hardware in the same way that the Open Data Institute, ODI, talked about what open data was and OSI again has never got into defining what open data would be. So the open source hardware definition, sorry, the open source AI definition is something of a departure because it's really OSI's first move into something which is not open source.

But I think it was a necessary thing to do because the, the boundary is such a a series of dotted lines that it's very, whereas with hardware, you can tell, well, you know, that's, that's fairly clear, even actually open source hardware is fairly unclear. One of the things I did when I was at Sun in 2006 was release all of the Spark designs for the Spark silicon chips as open source under the GPLV2.

We called it OpenSpark and we released all of the Verilog designs because it turns out that silicon chips are actually software as well. They're just software that's compiled to silicon. So the dividing lines are kind of hazy there as well, but OSI, as an organization doesn't doesn't harbor an opinion about open source hardware and does not produce a definition in that region.

Stefano: Can I jump in because I, I want to augment a little bit this, this part, like, why did we jump in? Why did we feel the urge when software is not we haven't done the same for hardware? I think what Simon just said hardware is programmed by a human. In, in, in some sense, like the chip design is done by a human.

And then it is compiled into Silicon. So you can see the mapping is pretty easily translated into source code being written by human compiled by a compiler into executable code for, for AI. What we noticed was that these systems look a lot like software, but they're not programmed by humans. They.

They apprehend, they learned by themselves, like they, they, they they have capabilities that emerge semi randomly. I don't want to get, I mean, I'm not a technician, I don't understand exactly why, but what I've been told is that these things just start to execute and become they have new capabilities.

They're not programmed. And, and the question. is therefore, how do you fix it? If it's consistently creating issues or spitting out the wrong answers, or, you know, or whatever, you, you know, you want to fix it, you want to change it, you want to give it to others, what is it that you actually want to ask?

It was easy for the hardware piece, it's not that simple, it wasn't easy, it wasn't immediate for the AI piece, that's what triggered it.

Jonathan: Yeah, that makes sense. Before we get any further into this, we really ought to stop and define who we are talking to. And I know that both Stefano and Simon are involved with OSI, and I, I could, honestly could not tell you what each of your exact roles are.

So let's let's go there next. Stefano, I guess. Where, where are you in the OSI org chart?

Stefano: I'm the executive director of the open source initiative. I started three years ago and I'm in Italy now. You're at the top

Jonathan: of the

Stefano: chart. No, no, the board is actually at the top. Well,

Jonathan: okay, that's fair. Simon, where

Simon: do you fall in this?

Well, I was the president of OSI for about a decade, give or take the odd year here or there, and when I quit the board of directors, I had foolishly started some work on open standards and public policy. And there was nobody to carry that work on if I just ran away and disappeared. So OSI hired me to be their director of policy.

And so for the last three years, I've been OSI's director of policy and standards, and then we hired somebody else to do, to look after us policy. So I've been the director. Director of EU policy and standards for the last year or so. And I, I, I actually don't do anything at all to do with AI. I, I, I look after making sure that things like the cyber resilience act doesn't break open source.

I'm making sure that the standards organizations create standards as if open source was real. Those are the two things that I actually do as the day job. But one of the things I'm going to have to do now, Steph made a A very clear statement when he took over as executive director that OSI needed to do something about defining open source in the context of AI.

And I'm now going to be taking that definition forward into the the, the, into Brussels and performing the necessary education to help people understand that, for example, Meta's LLAMA AI system is not open source. because the licensing includes field of use restrictions and other insights drawn from understanding what AI is and working out how we should legislate.

And that's necessary because people have already started writing legislation about AI. I mean, it, it. It may be really young, but there is already on the statute books in Europe, the the artificial intelligence act. And it contains within it an exception for open source AI. So somebody had to define what open source AI means.

And that was, that was Steph.

Stefano: Yeah. No, no, no, no. Wait a second. Wait a second. It wasn't me who defined it. I mean, that's.

Well, that's crucial because, because different from the free software definition and the open source definition itself, this is not the work of a lone person you know, smart or otherwise the coming out of the, of their, of their garage with, with with the sacred text. This was the. This process had to come from the community of AI developers, researchers, lawyers, copyright holders subjects of, of of AI systems, all of these different, different stakeholders had to be consulted and we needed to find a definition that matched what was actually happening.

In the space, providing some, some guidance, of course, and bringing our expertise and experience from 30 plus years of free software. But it was definitely not the work of my work.

Jonathan: We, we absolutely need now to go ask one of the one of the AI image generators to generate the image of Stefano carrying the tablets of stone out of his garage.

Simon: You know, I think that the point Steph's making there is actually really important. Yes. Because some, there has been the question asked, you know, what writers OSI got to define what open source AI is. Sure. And, and this then reads back to the open source definition itself, you know, what writers OSI got to define what open source means.

And the answer is that it's actually, The OSI's role is to collect together the consensus on what it means. That's what we did with the open source definition. There was a definition that came out of Debian that became the open source definition. And over a fairly short space of time it became obvious that That the consensus of the global community was that the open source definition was the the, the, the canonical explanation of how you can tell that a license is an open source license.

And now what Steph has done for the last few years is he has run this exhausting global program where he's held public meetings. He's hired facilitators. He's hired authors and writers. And what's happened is he's asked AI experts and open source experts and people who, who work in, in social development.

He said to them, what, what is open source AI? And over that time, he's gradually evolved what the answer is not by being clever and working it out for himself, but by. Identifying the consensus of this huge crowd of people. And that means that the, the, the, the definition is really a consensus definition in many ways.

Now because the, the field is younger, there are some dissenting voices. You go out and ask anybody who actually works in AI on open source nine out of 10 of them, and I can tell you the name of them. The 10th, but nine out of 10 of them will tell you that this definition is, it's pretty much right.

They might want to nuance some of the words or some of the concepts. And indeed OSI is putting in place a plan to evolve the definition to a, you know, a V 1. 1 or a V2 in the future. But this really is a consensus definition rather than an imposed definition from Steph. There you go. He's

Jonathan: got it.

There's Steph with the open source definition coming out of his garage, that's great. I love just that hair. So, I think, I think we can take a moment and just say, like, on some level, that, that is impressive, right? Like, that's pretty amazing that we could do that. That we could go, hey, here's a cool idea.

Let's, let's have a machine draw a picture for us. And it comes out that well. Like that, it is cool that we live in these times where we could do that. Yeah, can

Stefano: you imagine, can, do you

Jonathan: remember when you had to

Stefano: go to a bookstore to order a book?

Jonathan: I still enjoy going to bookstores, but yes. Me too. Take it. We take it.

Aaron: Yeah. Hey, I wanted to dig in if I could a little bit more into the nuances of what we're dealing with, because one of the things that really interested me two things you know, one is that there could be a dependency on something that lives outside the source code for the thing to work right as a problem statement.

And then the other, er, That it may have self generation capabilities and how do you license them? So I'm just kind of curious if what made me think initially was about yesterday, I discovered and started playing around with whisper from open AI. I'm not sure if you're familiar with it, but it's a.

text to speech translation tool. And you can, you can, it's, it's under the MIT license. So you can go, you know, do whatever you want with it under that license. But I'm just kind of curious, like if there's some examples of things that are already out there where those two things would be problematic or are problematic already.

Stefano: Two things that are problematic. Sorry, I, I wasn't, I'm not sure what, What do you mean? An

Aaron: example of some, of, of some AI tool or, or AI LLM or something that's out there that is. Either self generating, do you have an example of, of something like that, that could be problematic or the other one is where it's totally dependent on things that are outside of the, what could be covered under a license or what could be considered open source and thereby kind of invalidates the idea.

Stefano: Self generate, do you mean self generating as in Skynet? I don't know if

Aaron: I'm going that far, but something where I think you were talking about it, Simon, where it would be difficult to categorize the what was covered under there because it wasn't necessarily programmed by a human.

Stefano: All right, okay, now I understand.

So the case of whisper is interesting because the license. is, is extremely permissive. We're very familiar with it and it covers the parameters, like the weights, the train, the trained weights of that engine. What we don't know about Whisper is how it's been built, how, what kind of training, how, how the training happened, what kind of data sources they use to, to, to, you know, to, to have those results, those pieces are missing.

And that is why we don't consider, we wouldn't consider. Whisper and Open Source AI, despite the fact that the weights are openly and freely available. Because what we have defined in the document, the Open Source AI definition, what's defined in there that is brand new is the preferred form, the definition of preferred form of making modifications to an AI system.

If you want to change the behavior of Whisper, you can, you can, sure, you can fine tune it. You can, you can add layers to it. Or you can modify manually some of the, or randomly poke at at the weights themselves inside the, inside the matrix, you will not have, you will have a much harder, easier time if you knew what kind of data went in there, have a full list of it, the, if you had the training, the training code in order to understand how that training was done, you had all the code that was used to Change the, to generate the training data set because training data, I mean, the original data needs to be massaged, needs to be filtered, needs to be duplicated, tokenized, etc.

Before it can be fed into the training machine. So all of these pieces are required to understand to, to have a fully access to preferred form of making modifications to the, to the, to the code. Sorry, to the system, to the AI. That's the new thing that's in the definition.

Jonathan: So would it be fair to say then that that particular model, some of it is still a black box?

You don't know where it came from, it's just you have this black box kind of artifact that's part of it. And, and to make something in open source AI, we want to get rid of all of those black boxes and be able to see inside of all of them.

Stefano: Yep, that's exactly. Or

Simon: at least know where they came from. You know, so, so you can't, because some of the things that you do to make an AI are ephemeral.

They are transient. What matters more is that you've got a, an adequate description of how it was, how the AI acquired its knowledge that somebody else who is sufficiently experienced could take your recipe and do the same thing. They may not need exactly the same data. But they do need the same recipe that you had for how to train it on, you know, the health data of everyone in an emergency room for a month with these, with, you know, the, these inputs from the equipment and from with these sign offs from the patients and so on.

An expert can take that description and. produce the same transparent box. They don't necessarily need all the exact same data to do that, but that that's a corner case as the general case. Yes, indeed. You know, we don't want any black boxes. We want to know what was, what they were shown in order to be populated.

Jonathan: And I suppose that's sort of a hinge point here, right? Like there are, there are certain to medical is a really good example, right? There are certain times where you would want an LLM. That has been trained on medical data. And the idea of releasing all of that medical source data is just, it's a complete non starter.

Like, legally, you just cannot do it. But you would want that LLM to still be as open as possible and ideally be able to fall under the open source definition of AI. And so that's, that's kind of the corner case that's been the, the sticking point maybe through all of this?

Stefano: Honestly, a lot of the corner cases that have been, that have been, that are circulating We must wait and see, like, I, a lot of the conversations that we're having about these corner cases require new science or they may, they may, they may become obsolete tomorrow. Like a lot of the, a lot of the ideas that we had two or three years ago about the technology, specifically LLMs are starting to go away, like, or, or are becoming less relevant.

So, before we talk about corner cases, I would really love to see more work and more analysis of the actual good examples that we have today of, of groups, research institutions, nonprofit. Hackers that are really releasing datasets with with full instructions on how to build them full code releases.

They're making attempts at, at creating platforms, including hardware descriptions of clusters, training clusters. To, to build AI, AI themselves, like all of these examples, the, the virtuous examples are the ones that we are losing track of. Everyone talks about Lama and then on the other side there is a lot of other groups that are releasing much, very, very compelling technology.

With full access to all of the underlying components and pieces. So respecting that, the concept of preferred form of making modifications to an AI system.

Jonathan: Have you, have you gotten a decent bit of a contact from like the industry? People saying we, we acknowledge the work that you've done and we would like to make changes to make our license or model or, you know, whatever.

Open source compliant. Has there been some reach out?

Stefano: There has rather than the industry, the most the, the most productive and conversations we had in collaboration, we had So we had collaboration with industry, large corporations, small corporations, startups, and research institutions, and non profit groups.

The non profit groups are the ones who have endorsed more happily the definition as it came out, because it really supports that idea of creating a framework for collaboration, a shared understanding of what are the principles of furthering the science. furthering the knowledge on how systems have been built, how you train and therefore how to improve without having to reinvent the wheel.

But from the industry perspective, because of the technology, the way it is built, the way it's of its complexity, the fact that it has To take into account multiple layers of the companies themselves, like they go from the data scientists, but even on the legal departments, just to give you an example, in the legal department for software, copyright experts and maybe patent experts are sufficient, all of a sudden for AI components and pieces, you end up having to involve the whole Expertise of the firm plus consulting consultants from outside, because you have to have export regulation, the, the, the privacy regulation across multiple, multiple countries.

And world like it becoming, it becomes a lot more complex. Companies generally haven't been very happy with the way the definition came out. It's too restrictive for their point of view. Yeah.

Jonathan: And then you've got people on the other side that don't think it's restrictive enough, I'm sure.

Stefano: Right. Yes, there, there are some, some groups who say, yes, we should be asking for more.

Jonathan: The thing that comes to mind with that is that there's nothing in this that would prevent someone from writing a more restrictive license. Right? And I would imagine that you could write a more restrictive license and it would still be considered an open source AI license under these guidelines. So someone could come along and try to, you know, bring the idea of copyleft.

Into into this, it's the, the AI, the open source AI definition. Someone could come along and try to, you know, bring in an, a Fero, Genie, Genie public license. To where, you know, if anyone touches it, even on a website, they need to have a way to be able to get the source. Like, I would imagine that it would be possible to write these now, they're not going to get a whole lot of, of uptake.

But it would be possible to write these license and to to release something, to release an LLM under, you know, a less restrictive or a copy left license, right? Like that's an option, isn't it?

Stefano: It is an option and it's a conscious one. I don't think it's a negative one, to be honest. Like if you. And I don't classify the GNU GPL or the GPL as restrictive licenses.

They're permissive. They just add requirements that you may want or not. They're not really restrictive. So in the same vein, I do think that we It's a good idea to have the possibility to have legal frameworks, legal documents that would let someone like a user downstream to, to say, Hey, you're coming up with this system is spitting out this, this this output like I'm asking for a mortgage and it's consistently telling me that I'm not qualified.

To, to, to have it like, but why can I get access to all the instructions of how it's been built, can I have my experts review it, you know, that kind of stuff is not, is, is good for society and in general let me put it also on the other way. Like, think about the fact that you. We, collectively, have created a lot of content that has been crawled and, and and spidered and archived into, into repositories like Common Crawl, or the Internet Archive, or our code is into Software Heritage and GitHub repositories, right?

That content is now, can be used and, and is being used to train wonderful machines that create images and, and and spit out code. Right? Do we want, as a society, to have the possibility to say, Hey, you're using my code, I want the parameters to go back to me, and under the free, same conditions I gave the code to you, or my pictures.

I don't think it's wrong to think about it that way. It's a choice that we need to allow.

Jonathan: Yeah. And that, that actually kind of touches on a, a much bigger question with, with the, the, the way AI works right now, and there's this sort of legal theory that AI is put, putting data into a large language model, it is so transformative that it pretty much removes the original copyright, right?

So you, you put it, you put it in. And the, the language model trains on it, but does not inherit the copyright of the training data. That's essentially the way that it's being used. I mean, you look at something like Copilot on GitHub. It is trained on a whole bunch of GPL code. But then you can say to Copilot, write me code, and there's no expectation that the code that Copilot writes carries the, the GPL license.

So just as an example, that's the sort of thing that I mean. And I know that there are some there are some legal theories out there that that's not going to survive now We we talked last week with a lawyer and his comment was the genie is out of the bottle And I don't think we can ever put the genie back in the bottle but it is interesting to think that there is there is sort of this push for We should inherit some of the copyright of the original training data.

And I'm, what, I guess, what do you, what is your thought on that? Do you think that's a reasonable thing to think about? Or is the genie just entirely out of the bottle and there's no way to go back?

Stefano: It's, it's not, it's not a, I don't have an easy answer. It's a complicated, complicated and nuanced conversation because, and I have a dual, dual approach to this.

Do that on one hand. First of all, these theories, they are being proven in in American courts. And I'm, I'm saying American courts specifically because there are different in law. The copyright law is fairly uniformly applied around the world. But some things are different in the United States, in Europe, in Great Britain, China, et cetera, et cetera.

So, yes, the, the genie, I, I guess the lawyer was may, may have been referring to the the theory that has been proven by Google Scholar Google Books, actually the when, when if you remember, was it 10 years plus 10 plus years ago, Google started scanning books and making them available. Search available through the content of the books.

They were buying the books and paper, scanning them, doing character recognition and offering those as search engine products. And and they got sued. Google got sued by the American Publisher Association and the publisher, the Google won that case because the judge said basically that what Google was doing was not damaging the copyright holders.

And also being was transformative work. And, and so I believe that Copilot and GitHub and Microsoft are building their cases, their defenses against the lawsuits pretty much on the similar theories. We'll see if that survives the courts. But there is a, there is a different, there is another angle that I keep on thinking.

That the fact that. Collectively, we have created content. Collectively, the society has created content. Every blogger, every podcaster, we are creating content. We're making it available on their permissive licenses, like Creative Commons licenses and others. And we're telling the world, use it, do whatever you want with it.

And now, if we start saying individually, like, you can use it, but not for training, or you can use it not for training under these conditions, Then all of a sudden, the larger copyright hold, the larger corporations who can license content from content providers, large aggregators, et cetera, or they will, they will have an edge.

They will have an advantage because they have the money and the resources. To get access to large quantities of data and groups like that are building their, their, their systems openly in 3d and they want to have access to freely available content, they will have to jump through hoops and license individually from millions of people and copyright holders.

So we, it's a, it's a more complicated. Question in my mind than, than what it looks like at surface.

Jonathan: With some of those being more complicated questions and the way that all of this is still being developed and changing so much, do you foresee the possibility of changing the open source AI definition?

Do you think there's going to be some updates to it where you go back and you say, this needed to be stronger or this doesn't need to be in there?

Stefano: Absolutely. I think we need to, we need to really pay attention to what's happening in, in the space, like in the field, what, what are not only the technologies, how they're evolving and changing, but also how the developers and developers of AI, builders of AI, builders of datasets, how they're behaving, how they're adjusting their.

Their, their tooling, their expectations, how are they, they're, they're releasing. What, where is the collaboration happening and how or what's the potential for collaboration? Where is the safety coming from? All of these habits, we need to watch them. Like, consider the fact that the open source definition appeared more than a decade, almost two decades after the free software definition appeared.

The, and the free software. Definition was basically a statement of principles and in a manifesto that the open source definition is more like a checklist what we have today, a checklist that was based on 20 years of experience, right? So we're basically watching the space as it evolves. More systems are released.

More data sets are released, more, more experience we gain, and we can generalize from there. What we have right now in the open source AI definition version one is a stake in the ground. It's basically like, like a conversation starter, if you want. The place that we can use, like someone was saying, to have conversations with policymakers, with researchers and developers, with corporations.

To say, this is, these are the principles that came out of the, of a large conversation. Where do you disagree? What do you think is coming, is, is wrong? You know, we'll, we'll keep collecting that information. And we're also going to keep watching the data space because that's where I think that we, We need to pay more attention to like we have right now Really been talking about open data as if it's the only and most important thing that that we when we have But the open data alone is not sufficient to describe the complexity of training data sets and training data.

There's more nuance to that.

Aaron: There's so I've got a kind of a reverse question. We talked about this a little bit when Jonathan was talking about protecting creators rights or protecting the rights of the people who's Data, whether it was images or whatever, ever kind of data that was used for training.

Is there, I'm reading through the definition now. Is, is there, because one of the nice things about open source licenses, and we've got probably also distinguished between the definition and the license, but you know, the license does provide protection. At various levels, depending on which license you choose for the creator of the software, in that case to make sure that they get credit, for example, as the code is, is copied by others or used in other projects, et cetera.

Is there also that same protection in the definition, or is that left up to the various licenses that will come that meet the, The, the definition to provide that type of protection.

Stefano: I think it will have to be coming from the legal terms as they get developed.

Simon: So keep in mind that that, that the the credit to the author is a license term. It's one that's permitted by the open source definition, but it's not actually required. Actually a little while ago, somebody, I think it was Jonathan talked about restrictive licenses.

There are no open, there are no restrictive open source licenses. I

Jonathan: knew that was coming. Simon's had his B in his bonnet ever since I said that. And look at him go. There

Simon: are no restrictive open source licenses because a restriction is something that you have to negotiate removal of. And open source licenses do include conditions.

And so I, I say you can use this license if, blah. But that's not a restriction, that's a condition. A restriction is you can't, no, you, you, you guys with, with, who have, don't have white hair, you can't use this software. That's a restriction. And the only way you can get rid of that restriction is to go to the person who owns the copyright and ask them to waive the restriction, probably in exchange for some money.

So open, no all open source licenses are permissive in the, because they contain no restrictions. They only contain conditions and credit to the authors is a condition. And it's an optional condition. There are open source licenses that don't include that condition. And so if there are going to be those sorts of conditions placed on open source AIs, they will have to be licensed terms because they're not predicated by the open source AI definition itself.

Aaron: That's a good point. I the other question I had as you guys were talking, was there ever any thought put into this or maybe it's just outside the bounds and it doesn't apply because it's too high level of of intelligence itself, right? So what happens when we have an AI that, I don't want to go all the way to sentience, but I mean, as, as, as these AI tools and things become more and more intelligent and able to do things on their own, was there any thought in terms of the definition into we need to somehow Accommodate that, or think about that, or do the rights change at that point?

When, when AIs become or, or approach sentience?

Simon: That's got to be your question, because I don't believe that can ever happen. I'm, I'm a weak AI guy. I read Marvin Minsky in the 80s. And I, I believe that the society of mind does not lead to emergent intelligence. I believe that that evil, the evolution of, of machine learning only goes to just below that point.

But, so it's gotta be his question,

Stefano: right? My question, your question. I'll take it. I, I, I like we have this, this this sticker that says Skynet won't be open source . So I love, I love that. But it's just a working theory right now.

Jonathan: Oh, Aaron, I think that is a great question. But I think that's probably also a great answer. Yeah, that's pretty funny. Oh, all right. Let's see. Where, where do we, where do we want to go from here? So, the Do it. Simon told me where one of the bodies was buried. Do I want to do? I want to dig around in that.

Somebody's like, I don't care. I know, Simon. So there's this, there's this thought that in the, so there, there are exceptions for where, you know, sometimes all of the data for whatever reason cannot be provided. And we talked about the medical exception. And there are some others. And is it sort of a challenge to just to explain the data.

Open source definition, because those exceptions are so prominent and are we going to maybe see a version in the future that puts the exceptions sort of in the, in the back half of that section?

Stefano: We might. So, okay, let me put it that way. If I started today, rewriting everything from scratch. With with the help of a few people without having to, you know, go through the whole process and,

and interviewing hundreds of people, et cetera and traveling the world that blah, blah, blah probably the, the wording might.

I think that could change, but honestly, I, I think that we are panicking a little bit and, and we need to, we need to take a little bit of a step back, like really, we're going to release next week, a paper on about data specifically about the issue of data. And because if you put things away, like I said, the open source, an open source AI is one that gives you all the means to understand exactly how it's been built and be able to build something that is.

working exactly like the one that you have received. Knowing that software is, we have decades of experience with, we have understood it well enough that we have now reproducible builds, or at least a theory of it. Like, we can really take source code and rebuild a binary bit by bit exactly the same as exactly equivalent to the one that we have received.

It took us decades to get there with, with AI, with LLMs, with, with with this sort of neural networks technology, we still don't have the science to to do the same thing, to take I've been told by a notable AI developer that releases everything in open source, that they tried to Take the same data set.

Okay. Take one data set, feed it through. I

had an ambulance. I live near a hospital. So ambulances every now and then, I forgot about this issue. So they, they had this one data set and they were feeding the same data set to through two different pipelines into the same cluster split into halves. And they would put these two streams. would produce two different models.

Okay. So it's not, it's not easy. If not, it's not easy. It's not a solved problem on how to replicate a training so that you get an exact identical model model weights. So knowing that this is different than software, and this is a very young discipline, we need to take a little bit of a step back and, and we need to really understand the issue of data.

is that the issue of data is not going to be solved quickly. It requires much deeper understanding that what I was talking about, like we're now thinking of open data versus not. And I give you a very quick example. I was talking to a developer who's been thinking about building a data set that is purely made of.

Copyright free or unencumbered material content. And they were adamant on using public domain movies only. They step into the public domain issue, which is in the United States, public domain for a movie is 70 years after the death of the director in France. A movie goes into public domain 70 years after.

The last person who worked on the movie dies. It's impossible to calculate. So a lot of the, so where is a data set that looks like it's, it's built on public domain material in the United States may not be in public domain in, in in Europe or, or in France, right, or in German. Completely different story.

We need to get a little bit, take a step back and say, okay, well, this is different than source code. We need, we need more. We need to talk about changing policies maybe. Have new laws or build new habits, or change the technology, right? All of these questions are all open, all of these possibilities are open.

Jonathan: One of the, one of the things as we, as we get towards wrapping up, this is something that's kind of been in the back of my mind the whole time. When we talk about open source software, generally all it takes is a computer to be able to make changes and compile. When it comes to open source AI, like the, just the barrier to entry to really play with these things is much higher, you know, in some, in some cases you can do it on a CPU, in some cases a GPU, but like these, these rather on the edge AI models, you, multiple GPUs to be able to do anything interesting with it and I guess that doesn't really change anything as far as what the open source definition says for it, but surely that changes things for how accessible, like, On a practical level, how accessible it is to be able to actually get into and mess with some of these things.

Stefano: Yeah, there is definitely something that changed, like the scale of some of these training is, is prohibitive. But again, in a couple of years, I've seen there's been some difference. And small language models and other technologies. They seem to be making things more accessible. Like if before, if open AI, the company trains on clusters that are worth billions of dollars there are smaller groups that are doing something on tens of thousands of dollars, like, and getting similar results.

And I'm not saying that, you know, they can build something as big as Powerful of OpenAI's GPT and being able to respond that quickly. But, you know, these models, like Mozilla is doing something extremely fascinating with with, with smaller models running inside the browser, even a mobile. So, you know, training versus execution, that's another big conversation.

It's early. It's really early. We need, we need to give it time.

Jonathan: Oh, yeah. We are, we are definitely in the well, even, even faster than Moore's Law, right? So, like, there's this idea with early computing that every, what was it 16 months, it, it, the computing power would double. Some, something like that.

And we're, we're moving even faster than that with AI at this point. So, you know, give it a year or two and just who knows where we'll be as far as the accessibility part of it goes.

Aaron: And it definitely speaks to the reason why something like this is needed in my opinion, even though even though I'm an optimist, I'm not a skeptic.

I'm not, I'm not saying we don't need any, any of these protections and definitions for things. Speaking of definitions, I'm kind of curious, like, you know, you've been saying it's early days, this is, you know, You know, 1. 0, the definition, right, is, is what's out today. By the way, if people are looking for it, it's on open, I think it's opensource.

org slash AI. If they want to check it out so definitely go there and have a read as I've been doing, as we've been talking here. But it's still early days. I'm kind of curious, like, for example, you know, there's, I only see really two definitions in version 1. 0. One is of an AI system and the other one These are definitions in the definition, , but one is of a, of an AI system.

And then the other one is for machine learning. Mm-hmm . And of course AI is is lots of different things, right? Like LLMs and, and different types of things. So I'm kind of curious where, where do you go from here? Will there, where will there be other definitions that you know, you already have to add?

Are there things that, you know, what's the roadmap here of, of the definition? Where do we go?

Stefano: Yeah. Very good point. So. So why did we include the definition of an AI system in there is because we needed to have an anchor to understand what we were talking about, what we were defining three years ago, it wasn't really clear what we were talking about, like, what is it, why, what's different in here.

And so we use the definition of the. OECD, the Organization for Economic Cooperation and Development, which is what's been used also in the Artificial Intelligence Act, very similar definition. And why we're targeting machine learning specifically to talk about the preferred form to make modifications is because the new LLMs, the ones that require training, that have the dependency of data, etc.,

the ones that learn how to magically, those are machine learning systems. So we We said, okay, we don't want to define. The preferred form for everything all at forever at at every time, let's focus on what we know today that needs clarifications. And that's what we, we got it from what we take it from, I mean, what, where do we go from here?

We need in the next year, year and a half, we will need to understand better how groups like LLM 360, LL3i, The Allen Institute for AI, Falcon Foundation, TII, and other groups like this are LLM France. These groups that are releasing software parameters and data sets. As much as possible to the, to the comments into the group, into the, into the, the open source communities to understand how they operate and how they work and what they need in terms of legal frameworks, legal documents, opinions and, and generalize from there.

Jonathan: How soon do you think we'll see the first license that is OSI approved specifically for AI?

Stefano: I know that the Linux Foundation is working on a new license specifically for that. I would love to see even the, the current ones that are, would not pass definition, but I would love to see the debate, like the ones, like the responsible AI license.

I would love to see one of those being submitted, but also would love to see submitted the data licenses, for example, the, like the CD, CDLA, I think it's called. From the Linux Foundation, they have, they have developed a couple of licenses that are suitable for, for, for data sets. It would be a nice exercise to, to start getting, you know, start getting the, the habits but also understanding where the open source definition terms like require clarifications.

Jonathan: All right. Very interesting. Good stuff. We have basically reached the bottom of the hour. Aaron, was there anything that you. Desperately wanted to ask before we let him or we wrap.

Aaron: Yeah, I've got two and they're kind of, they, they, they weren't immediately applicable to the discussion. But I'm curious about their thoughts on this.

So I'm kind of curious if you have any thoughts on what other open source projects can do besides getting more. Developers involved to incorporate some AI tools into other open source programs or existing open source programs. And the ones that come to mind for me, cause I use them all the time, of course, are like Inkscape and GIMP.

I find myself using Photoshop more than I want to these days because I just need a tool that can quickly remove the background and generate a forest behind my image or something. And they do that really well. And I'm just concerned that. People will stop using these great tools that have been in the community for so long because they're now they're missing these AI function, this AI functionality because they don't have the development bench to go build it.

Any, any thoughts on that?

Jonathan: That's interesting.

Stefano: Yeah, it's I, I love to see Colabora. And, and LibreOffice get some summarization stuff. Like Mozilla is doing some very interesting work on that front. I, I think that it's just a matter of time to get the, to get on one hand, developers with skills and understanding these, these tools and how to do things like reducing the size of a model, like something that looks really gigantic that works only on a hardcore platform.

And high level GPUs to make it run on CPUs, for example. Yeah, so that it can be distributed freely by Debian. But also we need a little bit,

speaking of Debian, right? We need to also I think that we need to understand a little bit better what the legal frameworks are. Because I, I don't think that Debian is going to be very happily distributing Whisper. For example, or something based on a whisper, right? So we will have to come to, we will have to have more conversations about, about that, this reluctance, or maybe let me put it that way.

Maybe we need to think about how we're going to be solving that, that challenge between. Oh my God, this thing is stealing my content, it's stealing my stuff. And, and, oh my God, this, this is useful. I would use it or my friends are using it. So there's that tension in there that we need to resolve.

Jonathan: And you had one more Aaron.

Aaron: Yeah. One, one more quick one. And maybe this can be part of the wrap up instead of our usual questions, but I'm kind of curious what your favorite AI tool is that you're using at the moment.

Jonathan: Oh, that's a really good one.

Stefano: Yeah. I really don't. Don't use them that much, but we do use Google workspace at, at USI and Gemini is included in it.

So every now and then the temptation is to, to go and check what Gemini does.

Simon: Simon? You know, I'm not knowingly using any AIs at the moment. I, I do go and kick the tires on them every now and then. So like Steph, I, I kick the tires on Gemini. Because I can, you do that without needing to buy tokens to put in the slot machine.

Right. I, I'm going to be quite interested, so to go back to Aaron's earlier question, you know, the biggest challenges with doing that are indeed the, the the belief that the copyleft is, is carried into the statistical model. And I, I, I, personally, I am of the view that the courts are going to find that ridiculous.

They have done so far in each attempt in the U. S. We've, we've really not seen very much happening elsewhere. But until we get over the idea that the copyleft has been carried into the model, we're not going to see anybody then using the model in some software. And I think that's a big obstacle to seeing AI tools, AI ending up in open source tools.

I, I also think that we're going to see another generational change in AI coming about, where the technology is going to be made differently and deployed differently to the way that it is now. And I think that's, that's a big obstacle. Quite likely to wait for those, those cool things to make their way in to open source tools is going to need to wait for that.

Having said that, there are already AI hooks in an awful lot of places. I don't know if you've realized this, you know, you look in home assistant, for example, so I do use home assistant. There's a great big AI hook in the middle of home assistant so that it can do voice recognition. There are great big hooks in Mastodon for going and doing AI translation.

So actually, the, the, the question you asked, you know, when are we going to see open source supporting AI, it's already happening, but the way it's happening at the moment is by providing hooks to go use external systems rather than by building the capability into the product itself.

Jonathan: And

Simon: there's a lot of that already happening.

But as a community or a community of communities, we've got some very hard conversations to have sometime soon, if we're going to see freedom respecting software, do AI rather than just call out to freedom, non respecting software that does AI.

Yeah,

Aaron: right. To your, the, the agentic model, I guess, as it's known, it seems like that's a big word.

That's thrown around a lot these days.

Jonathan: To your point, Simon, there, about whether, and it's the same question that I asked about the kind of inherent, the inherence of the copyleft when it's part of the training data and how that's gone in the courts. I imagine what we're eventually going to see is essentially a test.

Right? There's going to be a legal test that says, you know, if, and I don't know for sure what it's going to look like, but like if you can create a prompt that gives you this many words in a row that matches the the input, then you have inherited the copyright from the input, you know, it's, and it may not be exactly that, but like, I have to assume that there's going to be some court case that's going to give us a test that sort of everybody can agree on.

Okay. Because, obviously, like, you could just copy a file through a black box AI that doesn't do anything, call it AI, and

Simon: I mean, there was a cartoon to that effect, wasn't there? Well, someone

Jonathan: tried to do that with Beatles music, years ago. And they, they ran the Beatles records through their sonic maximizer.

And I don't know if they even use the AI buzzword or not, but they're, they tried to make the point that, Oh, this transforms it so much. It's a completely new work. And of course the courts completely shot that down and, but there's got to be a happy medium in there somewhere. So I, I expect that at some point there's going to be an agreed upon test of some sort that here's the, here's the guideline for where your AI is transformative enough.

that you're not inheriting that copyright.

Simon: Yeah. Well, I'm not going to second guess it. You know, but I, as an accident of what I do for a living, I know a lot of lawyers. And I have yet to meet one that thinks that that the license that the, the the statistical model is a derivative work of the source data.

Now that maybe that's going to change sooner or later, but at the moment If you, if you want to go make a court case that the some AI has copied your work, it's going to be a really tough case to even get legal counsel to defend you on, and I draw your attention to the fact there's a lot of no win, no fee work going on in this area.

But Steph disagrees. I can see him waving. No, I, I don't disagree.

Stefano: I, I think that it's the, it's a less interesting question for me because the more interesting question is, should it consider a derivative or not? And I, I think that we should be running that exercise a little bit more consciously and think about the consequences of either way.

Like, either way, what happens and what's the worst outcome possible in, in either parts? And and maybe, maybe we can even influence courts. But we should do it with consciousness, right? It's not just the gut reaction. Say, Hey, that code is mine. I should be getting something remuneration for, for copilots capabilities, for example,

Simon: or,

Stefano: or Dolly, the fact that it, you know, reproduces works that look like mine, like I need to get compensated.

All right. Should you, what happens if we do?

Jonathan: Yeah, there's a hole and we do not have time to go down this rabbit hole, but there is an entire rabbit hole about whether it's a good thing that we can have a machine produce an image and we're no longer paying an artist to do it, right? Like, that's its own entire conversation that, yeah, I think is important to think about, but as I said, we are not going down that rabbit hole today.

Maybe someday in the future. I do, I am required to get a couple of final questions in myself. And that is, what are each of your favorite text editor and scripting language? Stefano! Tsk.

Stefano: Yeah, it's passed on the scripting language. I really don't code anymore. You know, I played with Bash and Python.

But I, right now, I'd probably ask a co pilot at JGPT or something to do it for me. Is there a text editor in there? Text

editors VI, VIM is one that I usually fire up when I do some quick stuff, but I'm not really a text editor guy. Yeah.

Simon: And I'm very disloyal to my text editors. I've been using different ones throughout my career.

Yep. The one I fire up most often at the moment is Nano. Sure.

Yep.

But hey, you know when, when I was IBM, we were using e and I, I used to work on, I used to work on, on a word processor. I used, I worked with WordStar and Word Perfect back in the day.

Jonathan: Yeah.

Simon: And I've got everything loaded on a, on various computers around here.

Scripting language is more interesting though. I'm doing it all in YAML at the moment because I'm doing all this home assistant stuff. Yeah. You

Stefano: consider that a

Simon: scripting

Stefano: language?

Jonathan: I'm not sure that, that yet another markup language really counts as a scripting language, but

Simon: And yet I'm programming the whole of my home assistant deployment using YAML, using YAML pages.

Jonathan: All right. That's fair. I suppose. All right. Thank you guys both so much for being here. We appreciate it. And it's a fascinating dive into the, some of the questions and some of the answers about open source AI. Appreciate it very much. Thank you. All right. Man. What do you think?

Aaron: Yeah, I mean, it is, I'm, I'm glad that it's being done, I feel like mm-hmm

With other areas in tech, we kind of missed the boat. Open source hardware and some of those other things. Yeah. Social media, for example. Yeah. You know, it, it, it feels like now there's all this concern around social media. It's like, well, where were you five, 10 years ago? Right.

Simon: Yeah.

Aaron: When people could see some of the problems that it was gonna have, and at least with AI moving so fast, I'm glad that.

Somebody at least is thinking about these things and coming up with these definitions because otherwise, you know you who knows what what could happen? Right? And like Stefano said, we need to have the discussion, right? Even if you don't like 1. 0 of the definition that's out today. Talk about it. Have the discussion.

It's what it's what we've always done going. Back to early days of networking, for example, think of all the discussions and all of the groups that were around to try to define how we're going to make this thing work, right? Without discussing it, you're, you're, you're just opening the door for not necessarily bad things to happen, but unwanted things to happen.

And so yeah, I'm glad it's taking place.

Jonathan: So I think it was Simon that told us something that absolutely terrifies me. And that is that in Europe, there are laws that refer to open source AI, and they referred to that with there being no definition for what open source AI is. And it's like, that encapsulates the central problem that many of us have with like over regulation and, and just government's, Stepping into things that they really don't understand.

That's, that's, that's humorous. And all of that to say, I'm glad that there are people working on this and, you know, people that sort of have an idea of what they're doing.

Aaron: Exactly. People that understand these things instead of. Politicians who don't just coming up with, you know, random things that they throw darts on the wall and come up with sometimes.

Jonathan: Yes. Yes. All right. You have anything?

Aaron: What we, you know, we should, we should talk about our favorite AI tools that were you, what's your favorite, Jonathan, what are you using?

Jonathan: About the only one that I use these days is when you Google something. Sometimes you'll get like the, the, and I guess it's Gemini, right?

So you'll get like the AI answer to your question. And that is getting to the point to where it's useful. I, I do not consider it trustworthy, but it is useful. And then I've done a little bit similar to what you did with like here's a prompt, give me an image that looks like this. I've, I've done some playing around with that.

With varying degrees of success. Some, sometimes the prompts can be, or if you're good at writing the prompts, sometimes you can get really good results. And then sometimes you ask for something a little esoteric or off the beaten path, and you just get weird results, which I guess is kind of fun too, but not really what you were looking for.

What, what tool do you make you most use of?

Aaron: I've probably got to pick JAT GPT. I mean, I do pay for the plus license and I use it multiple times a day. Okay. For personal and business use. And. It's, it's, it, it impresses me more than it disappoints me, that's for sure. So I use it to summarize notes from, from meetings.

So, you know, I'll have like hour and a half long meetings. We get the transcription, feed it into chat GPT, say, can you summarize this? And it does a great job. And there are other tools, of course, that do that. I use it as part of my YouTube channel for sometimes creating thumbnails. Images, if I just can't find anything that I'm allowed to use sometimes.

I'll use it for that. And of course I use it for you know, like I said, removing background on things, just things, shortcuts that, that, that I can get it to do that would take me. You know, 10 times as long, I can say, look, just like generating that image today, for example of the guy coming out of the garage with the open source tablets, right?

I mean, yeah, that was

Jonathan: pretty good.

Aaron: Yeah. So things like that, that you can do just as time savers really is the biggest way that I use it today.

Jonathan: I think, I think I'm a little gun shy on using AI because I write for Hackaday. And like, I do not want there to ever be the even the appearance of crossing those streams.

Right. So every word that I write for hackaday always comes directly from me. The, the most in AI is ever involved with is Google will tell me that I misspelled something and that's it. And we've had some conversations internally at hackaday and that's pretty much the conclusion we've come to as well.

You know, there, I think there's, was. Once one of our writers used an AI image as like the headline image and we kind of got shot down internally it's like let's not do this unless we decide that we're gonna be okay with this and So I'm I'm pretty careful to not use much AI just kind of because of that even as a

Aaron: research tool though for for the background for the For the piece that you're writing

Jonathan: about the only thing is if I google If I Google something, you know, it'll, it'll come up with the the, the, the Google summary of what it thinks the answer is.

And again, I've kind of found out that that's not necessarily always trustworthy. Sometimes it'll tell you the exact, now it's getting better, obviously, but sometimes it'll tell you the exact opposite, or it'll pull something from an article that's not really about the thing that you asked about. So even, even then I've, I'm pretty careful to go in and try to find actual hard sources.

I use

Aaron: it. I use it for research and I tell it to give me sources and then I can go take a look at the sources and validate.

Jonathan: Yeah. Yeah. And that's, that's apparently a really useful hack. Like you make your, you make your AI assistant. Way more accurate. If you tell it, it's, it's kind of like with people, you ask them, Hey, what's the answer to this?

And give me your source. Well, we're going to work a little bit harder to make sure we give you the right answer. If we also have to give you a source, apparently that works for AI too.

Aaron: Exactly. Exactly. Yeah.

Jonathan: All right. What do you want to plug if we let folks go?

Aaron: Well, I mentioned the YouTube channel.

Check it out. I've got two channels. That's important because I've actually been publishing more on the second channel than on the first. So there's a RetroHackShack, of course, which is the main channel. I do more history videos there, more big projects there, and they come out a little bit less frequency, less frequently.

And then there's also RetroHackShack After Hours. And the last two videos I did there, one was a repair on a 1990s motherboard where the keyboard didn't work. I couldn't get any keyboards to work, so that was a short one, and you can go watch that one and figure out what happened. And then the one before that was oh, I do a lot of e waste stuff on the second channel.

I find stuff at e waste and just, just bring it home and start the camera, and I go through and figure out what happened. Like on the fly, like what it is, what's working, what's not. And so that's, that's a lot of fun too. So check out both channels and you know, hopefully there's, there's still people out there that like vintage computers.

Jonathan: Oh, definitely. I know. I watched that, that last one about the the keyboard not working and I won't give it away, but I will say the component that you replaced, I would have looked at and probably just thought it was a weird resistor. So that's actually, that's actually pretty, a pretty useful bit of knowledge for me to add to my own.

Fix it toolkit.

Aaron: Right. Right. And I would say probably it wasn't long ago, you know, 10 years ago, I would have been in the same boat. I would have been like, Oh, what's that thing? So, yeah, it is, it's, it's fun. And that's, what's fun about it is you learn about what things are and how they work and how they used to work.

So yes,

Jonathan: absolutely. Absolutely. All right. So next week we've actually got something really interesting. I believe, yeah, we're talking with according to the calendar. Hopefully it's right. We have another Stefano. We're talking with Stefano Zaccaroli about the Software Heritage group. And that's going to be a lot of fun.

And then the week after that, hopefully we're going to have someone from CIQ to talk about Rocky, which we've talked to all Malinux. We figure we might as well talk Rocky. They were there at the very beginning as well. And then the week after that, we're talking Thunderbolt. Thunderbird, so all kinds of fun stuff coming up.

You don't want to miss it. If you want to follow me and my work, of course, there is Hackaday. We appreciate Hackaday being the home of Floss Weekly. I've got the security column that goes live pretty much every Friday morning there, and you can check that out. Got a YouTube channel that you can find.

There's also the Untitled Linux Show still over at. Twit, that's twit. tv. I think it's twit. tv slash ULS. But we have a lot of fun there talking about what's going on with Linux and lots of open source stuff, but more news of the week over there as opposed to the long interview form here. Yeah, we appreciate everyone that watches that get us both live and on the download, and we will see you next week on Floss Weekly.

This week Jonathan and Aaron chat with Simon Phipps and Stefano Maffulli about Open Source AI. Why did we need a new definition? Has it been controversial? And why did OSI step into this particular conversation?

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week, Randall joins me. We talked with Matthias Schakla about open source and the law. We can talk about, uh, contributor license agreements, software licenses themselves, and lots more. You don't want to miss it. So stay tuned. This is Floss Weekly episode 815 recorded Tuesday, January 7th.

You win some.

It's time for Floss Weekly. That's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and we've got something a lot of fun today, but also really, really important. And, uh, if you are, but let's see, let's count. If you're part of an open source project, If you're doing any security research at all, uh, if you're an employee anywhere in the IT sector, you probably need to pay attention to this because we're talking about the, the intersection between the law and open source software and probably other technology questions.

And, uh, it's not just me, of course, I've got a lovely and talented co host, as we like to say, Mr. Randall Schwartz. Welcome.

Randal: Hey, Jonathan, thanks for having me back on this show. This is great.

Jonathan: Yeah. Thanks for jumping in kind of at the last minute. My scheduling got further behind that I wanted it to. And so last night I was just messaging a few people like, I know it's like 10 PM where you're at.

Could you please come on the show tomorrow? And I asked Randall, he's like, yeah, yeah, yeah, sure. I'll do it.

Randal: Yeah, uh, well, I was looking at, uh, at this week's schedule and I said, Oh, Tuesday, I'm blocking out FOSS Weekly. I wonder if I'm needed for that. And that's when I contacted you at 10 o'clock last night saying, I still have an open slot tomorrow morning.

What do you think? Yeah, it worked out.

Jonathan: It worked out. We are. It worked out real well. Um, so our guest is Mattia, Mattia Cioche. And I, I'm sure I killed that last name. I did not pronounce that correctly. It sounded right in my brain and then my lips started moving and it was terrible. Um, but he is a legal expert and sort of a geek.

And, uh, he said it's been a while since he has compiled the kernel, but used to do it all the time back when he was running Gentoo. And, uh, I think that makes him one of our people, you know, at least, uh, one of my people, I think tangential to being one of Randall's people, right? Have you ever compiled the Linux kernel, Randall?

Randal: I don't believe I ever have. It's true.

Jonathan: That's all right. We still love you.

Randal: I've installed it a few times, but it's not my primary install when I pick up a computer. So yeah, sure.

Jonathan: So let's bring it. Mattia on welcome, sir, to the show. Hello. Uh, it's an honor to be here. We are absolutely thrilled to have you.

Um, so what, let's start, let's start with your background. Because you, you come at this from sort of an interesting, uh, direction. Your, your background is mainly the law and the, the IT computer stuff was sort of a hobby that got picked up. And then from what I understand, it's sort of become part of your career now.

Matija: Um, yeah, that's the long, long short version of it. Um, so. The longer, slightly longer version is, I've been interested in computers since forever. Um, me and my brother Noah, we had, we got some really old books on computers, um, which, that included like, uh, you know, the whole page is full of basic, um, that you could, you know, type in to your computer to get, uh, something that looked like Tetris out of it.

It never worked. Because that was some other basic and we tried it on DOS. So the QBasic didn't work, but we spent like days doing that. So I then got infected by the Linux virus, mid nineties at, uh, uh, high school. Um, because there was like all these computers in the hallway, um, that you could use to go online and all had, you know, nine, Windows 95 or 98 or something like that.

But in the middle there was this one with black screen, white text.

Jonathan: But what I

Matija: saw is like, they changed, like on every other computer, it took like, because we had like the whole high school had a dual ISDN line for the whole school. So somebody knows what that meant, right? Uh, I see nodding, which meant that when you logged into your Yahoo mail, which was usually what you had back then.

And you know, the Netscape times, um, it took like the whole break before you managed to log in. And so you, you saw you had new mail. So you, then you had to go through into the next break and try to be the first one at the computer to manage, to click on the first mail, to see what it is. But then on the.

That black screen computer I saw like people rotating and it turned out that was a Linux machine and people had mails on the machine itself. And I think we used the pine or something like that. I think it was already pine. And so we just learned from each other how to use this weird contraption. Um, so, you know, we learned how to use, I apologize, the program was actually called like that, uh, Bitch X, which is an awesome IRC client back then, um, especially with the ASCII art.

Yeah. Um, and we learned together how to use that machine. And then through that I got into the, into the early essays, uh, by Stallman and, uh, uh, from the esr Cathed, yeah. Sr. Yeah. Um, and that's kind how I got into the whole thing. Um, but I always want, I always knew law was what interested me in general. Um, and that's another.

You know, interesting story why, but that was kind of how it went. It's like always while I was studying law, um, I was also part of the local hackerspace, um, so, and the Linux user group. Um, so, you know, I don't have any IT, um, training official, but, um, I've dabbled with it for a while. Many years. Far from being a good coder.

Far from being a coder.

Jonathan: I do some tech support for some lawyers, and I just have to say, The fact that you know what Linux is, puts you so far ahead of some of that crowd. Yeah.

Matija: I mean, I probably run around a different crowd of lawyers than the average crowd of lawyers. Yeah. But yes, in general, I do agree.

Like, when I remember when, when it was, when, um, at university, yeah, it was, turning on the PC was a challenge for some people.

Jonathan: And so did that, did that kind of affect the, uh, the trajectory of your career, your legal career that you knew sort of this, this whole. computer and open source side of things existed.

Um, is, is that sort of where you, where you went with your legal career?

Matija: Um, kind of maybe, but it was also a bit of a chance. So, while I was, um, working, uh, with the hackerspace and, uh, and the Linux user group, um, we did, you know, we clearly, we did some advocating like in the late 90s and, you know, against software patents in Europe, et cetera, et cetera.

Um, I did. You know, through that, through advocating for more FOSS and, uh, and all that, um, I found out that there's actually a pan European, um, organization that does that, uh, the Free Software Foundation Europe, which is like the sister, uh, organization of the USFSF. Um, and I messaged them, just said, well, you know, you know, do you have anything in Slovenia, which, which is where I'm from?

Um, and if not, can we do anything? And they said, yeah, you just, you know, there you are, you know, you should start a chapter. And so we did. Um, and then at some point, um, when I was volunteering for the FSV as a, you know, running the Slovenian chapter, um, at some point, In my studies, um, there was a, I had an option to go study abroad, um, and, uh, in the U.

S. actually. And, uh, I, um, asked the president back then, of the FSFE if that was okay, um, because there was some, you know, then I wouldn't be in Europe anymore, et cetera, et cetera. And uh, he messaged me back like saying, I'll, um, I'll give you a call and I'm like, oh crap. What did I do now? Um, and so I run out of the law, uh, faculty's library and I, you know, pick up the phone and, you know, that's still when, you know, even in Europe, you know, the.

And he was in Germany, I was in Slovenia, that was still like kind of a costly call. It wasn't super expensive, but it wasn't local yet, as it is nowadays. So, um, I pick it up and he's like, well, we actually wanted to, we wanted to employ you. Like, what? So, that's how I became, uh, FSFE's, um, What was it called back then?

Legal coordinator. So I was responsible for open source or free software, uh, legal matters within FSFE. And that kind of, I'll be honest, I mean, if that didn't happen, I don't know what I would have been nowadays, because that, you know, not just, you know, open doors, et cetera, but introduced me, part of the, part of the job was, um, there is this, uh, legal network.

Um, that, uh, FSFE facilitates, which is the largest, uh, network of, uh, FOSS specialists, lawyers, and technical people who have a lot of knowledge when it comes to, like, license compliance, etc. Um, and, uh, for many years, I was in charge of that network. So that's clearly, you know, you learn a lot on that by that, doing that by.

Organizing conferences for, and workshops and all that great stuff. Um, so that's, that's how it happened. And I'm very grateful for that because otherwise. I don't know, I would probably as a, just, you know, a box standard attorney, I probably wouldn't be great, pretty good at what I do, but, you know, when, when I was looking for a job and I was looking at my grades, um, in school, because I know I was, you know, Helping with the hackerspace, you know, it took me forever to finish law school.

It was, you know, on paper, this was not looking good, man.

Jonathan: Yes, I very much get that. My, uh, my college career went very similarly to that. Uh, so one of the things that Randall and I were wondering before the show, Is, are you a, and I don't, honestly I don't even know how this works for sure, are you an American lawyer, a Slovenian lawyer, are you licensed in both places?

How, how does that, that bit of international law work?

Matija: Um, I'm a lawyer, I'm not an attorney. That's the first thing. So I am, I am, I finished Master of Laws. Okay.

Jonathan: So

Matija: I have, um, it's because it was the old system. It's, um, we're not allowed to call it LLM, but it translates to LLM. Um, not the new LLM. Not the new, yeah, yeah, yeah.

We're not talking about artificial intelligence here. We're talking like the title. Right, right. And, um, but then if you want to be an attorney, so attorney at law, that means you can represent people in court. I mean, there's to, to some extent, to some level of court, anybody can represent. Right. But from a certain point onward, you have to have an attorney.

Um, and that is, and different places have different, uh, exams, how they do that. Um, and, um, Um, when you do an exam, you're only allowed to do it in that jurisdiction. So even in, within the EU, if I did mine in Slovenia, I couldn't be practicing in Austria, which is across the border. I would need to take, I would need to take at least the partial exam to get it re evaluated from just for Austrian law.

Because the, the thing is like, Counter to how, you know, tech and, and natural sciences work is like jurisdictions, like every country has their own laws and their own history of laws. So you need to be trained in that specific history. Um, so I'm not admitted to any bar. So attorney at law means also like.

In English, it would be a barrister, um, so that is, you know, when you take the bar exam and you're admitted to the bar. Got it. I

Jonathan: appreciate the, uh, the clarification there. Um, Randall, this seems like this would be a good, a good time for you to, uh, fill us in on your, your history with legal issues, and maybe we can kind of jump off from there.

Randal: Yeah, so I also wanted to jump in with like maybe what you're closer to and say the U. S. would be like a paralegal, which is somebody who isn't a lawyer because lawyer here actually means probably the same thing that attorney means to you. If I say somebody is a lawyer in the U. S., they, they are somebody who can represent me in a court, in all court actions, things like that.

So, um, so I think, I think our terminology is a little different here in the U. S. So I would say you're probably closer to someone who can do filings on behalf of people and, and, uh, and, and, and understand the law in an area and have been at least certified to a certain degree. That, that would be paralegal here.

I don't think so,

Matija: honestly. I think paralegal is somebody who just has, uh, who just has an exam. It doesn't have like the whole schooling.

Randal: Oh, okay. Okay. Maybe, maybe not, but here, here, a lawyer and attorney mean exactly the same thing. There's not a distinction for those.

Jonathan: Although that may be, that may be one of those cases where if you're a lawyer, there is a distinction, but just for the rest of us, we don't know what the distinction is.

Maybe that might be what's going on there too.

Randal: A software engineer, not a programmer. Okay. Okay, here we go. It's probably the same sort of thing. I'm not a coder. I'm actually a software engineer, right? Like that means anything really come on guys Um, so just uh, just the brief version many people in our audience are familiar with my legal case But just to bring you up to speed a little bit on that Uh, I, uh, I am, uh, I am a three times over felon as a result of doing my job for one of my clients with a bit too much enthusiasm, as in I was doing, uh, unsolicited pen testing.

Jonathan: Uh,

Randal: against a machine that, uh, was no longer in my jurisdiction, but had been earlier and I was still concerned about the new people that had taken my job over there. So, uh, so I just got crazy and started doing some pen testing without asking my boss. And, uh, that eventually led to cops showing up at my door and, uh, 270, 000 later, um, I'm still ultimately restricted from going to Canada or Australia.

Although I've been in 68 countries, I can't go back to Canada or Australia ever again, uh, which is really just weird. But, um, uh, I think from that, I actually want to start asking questions about liability, uh, in an open source project. So here I am contributing to an open source project. How careful do I need to be as an individual contributor, contributing to a business project that the project's not going to get in trouble somehow, but also like, I'm not going to be individually liable somehow as I'm contributing.

That was kind of my first volley of a question.

Matija: Oh, that's an ungrateful question because that like literally I just started reading there's there's you guys in the US have the executive order number, whatever, whatever, uh, national security thing, uh, that was a big thing like five years ago or a few years ago.

Um, And I did read through that one, uh, because of work, um, and I'm just right now reading the European kind of variation of that, which is the Cyber Resilience Act, um, and I haven't gone through it yet, um, so I know just, you know, Secondhand information, um, but from what I gather so far, at least it, it depends on jurisdiction to jurisdiction, but in this case, it would be like EU will be pretty stable.

Standardized between itself, uh, within itself and, uh, U. S. will have its own thing. And then I think Brazil has something already cooking if it's not done yet. Um, so there is legislation happening on that case, uh, on that side. Um, I don't know about the rest, but from what I seen. And, uh, in the CRA, so the Cyber Resilience Act in the EU, um, I noticed that the, uh, the, the likes of, uh, the FSFE and, uh, OSI, so the Open Source Initiative, and I think the Eclipse Foundation was involved as well.

Um, they were very involved with the European Union, uh, when it came to, you know, Honestly, to lobbying to make sure that FOSS gets excluded or, you know, gets preferential treatment. Um, and to some extent that's what happened. Um, so, in your case, I mean, you were pen testing, that's different. That's not contributing to an open source project, let's be honest.

Um, And I don't know enough about that case to, you know, make it sure, make a statement, what I think about it. Um, but, um, yeah, it sucks to be you.

Randal: I have said that frequently. Yes. Yeah. Uh, but, but, but we, there, there are issues, I think, both from the level of civilian penalties, as well as criminal penalties, if you cross certain lines, even being an individual contributor to an open source project. Right.

Matija: Um, the question of an individual contributor is a good one.

Um, because from, again, I haven't finished reading it yet. Um, but from what I've read in the CRA, um, they do distinguish between Manufacturer which may also include open source project or components or, you know, maybe the whole thing is an open source thing and then they just make money, um, shipping it on, you know, Computers or, or offering as a SaaS or whatever, and, um, open source stewards, that is actually like a defined term in the, in that law and, um, open source stewards is defined to encapsulate things like, um, like the Eclipse foundation or, you know, you're Linux Foundation is already kind of huge.

Um, they do a lot of stuff, but you know, kind of like those, like GNOME Foundation, KDEV and stuff like that. So you have like, it's a preferably non profit or not even necessarily an organization that is a steward of an open source project. Um, they may have, they may get some money from donations or otherwise to work on it, but they're not.

You know, they're making the software, you know, publicly available and they're not necessarily primarily monetizing it. Um, and there they got like preferential treatment according to the CRA. So this is what is happening, at least in Europe is that we've seen now that the trend is that. for years now, they, they, they had preferential treatment for small and medium enterprises and, you know, even micro enterprises, of course, um, that they're now also are starting, like starting to, um, make exceptions also for FOSS, uh, how that works in the U S I don't know.

I've followed the, that executive order and, um, That's as far as I got, um, because that was, uh, relevant for work. Um, but, the, and, and here's a, as I said, this is what is interesting when you point it out from an individual contributor is,

Here, in this case, it could be like a company is using, you know, some foundations, open source software, the company would need to make sure that it's secure. And it has, like, depending on how serious the use case is, um, whether it's critical or, or just severe or just You know, everyday stuff, um, depending on the severity of the use case, they have different levels of what they need to, how much they need to take into account that the use of that software that they, you know, manufacture, you know, this is actually secure to use, uh, safe to use.

Um, and then the foundation, the open source foundation. The project has a much, much, much, much, much lower, um, barrier they need to meet, meet is, um, I, I mean, there's one of the obligations is, for example, if you see as a manufacturer, if you see a, if you find, uh, security hole, you have to report it to the EU agency.

That's, you know, for security of software. I forgot what the name was. It's a new one. Um, and also if there's an upstream also reported to the upstream, and if you have a fix, ideally also ship the fix. So in this use case, it would be if the, if the company finds a bug in the open source component and they patched it, they have to report it.

And then they also have to submit the patch. The patch with the report to the upstream, uh, open source, uh, project as well. Um, and there's, you know, the open source project doesn't have as many obligations. I don't think there's, I think they need to, again, I haven't gone that far. Uh, so, so far it seems like they might need to report an issue if they found a serious issue, but they don't have to do it in 24 hours because there's, you If you're critical infrastructure, you need to report it within 24 or 48 hours.

Um, and I don't think I've seen individual contributors mentioned yet in the text. So that's an interesting question. It is a good question who they would be, who they would be liable to. I mean, most Pretty much all open source licenses have, you know, the, as honestly, pretty much all software licenses. We don't, there's no warranty to the extent, to the extent permitted by law, et cetera, et cetera.

So it's going to be, I would say, following on that, I would say that, building on that, I would say that the contributor wouldn't be liable Unless they would be liable under just general law, period. For example, like, if they would be, if they were malicious interacting. So, and a malicious actor would be liable regardless.

Um, uh, but a, you know, if a contributor just, Made a boo boo. I don't think that would be a big problem. Um, but again, haven't finished reading it yet. So just got feelings so far.

Randal: As a follow up to that, I'm looking at the angle for individual contributor to an open source project. I think one of the other obligations as an individual contributor I have is that.

Uh, the, the chain of, of creation is proper. In other words, I created this software, therefore I have the rights to contribute it, but we can get into trouble by, uh, taking things that actually have proprietary licenses or incompatible licenses and contributing them to an open source project. And I just, that's one point that I want to look at, but I also, I'm thinking about.

Like licenses, like say the Mac, Mac OS user licenses, you have to be running on Apple hardware. Does that mean that if I'm using a hackintosh to make software that I'm contributing to an open source project that relates to a Mac M M where, where did that cross the line and is the project liable for accepting my stuff?

Or am I the only one liable here? Cause I'm the one that violated Apple's contract. I mean, where, where do we go there? Wow.

Matija: I mean, you'd be the one who violated the Apple's contract. So that, that would be on you. I mean, how, how would, and, and, I mean, there's also what you, what the output of your work here.

And I'm slightly oversimplifying here, of course, but it's also like. If I understand the situation here is, it's kind of like, if I use a, if I use a cracked pirated version of, for example, Microsoft Office to write a document, who's, who's liable for using that document? They're like, I don't care. I mean, the document is a separate piece of work.

It just happened to be, you know, I managed, I, I did something illegal to get it. done, but, um, the document itself should be fine. There's no, you know, if, if that was a problem, we'd be in all sorts of hell.

Jonathan: I I'm, I'm curious with this idea of, uh, the individual contributors and liability for open source, uh, in the United States, we've seen, I don't think it's a Supreme court precedent, but we've seen a court precedent that. Source code is First Amendment protected speech. And I'm wondering if that helps. Like, does that help protect the individual contributors?

When we're talking about these projects.

Matija: I'm not U. S. trained, so I do know a bit of U. S. legislation. I do have a Harvard online course in copyright, in U. S. copyright, apart from my own master's, but, um, I think like the First Amendment gets thrown around quite a bit. Um, and, uh, From the history I've heard about the First Amendment, it's, um, it's been stretched in different ways and different forms through the decades, so I don't know how that would work.

Um, but

I'm not sure. I'm not sure. I'd be, I'd be, literally, I'd be just pulling something. Well, ding,

Jonathan: ding, ding, ding. We've

Matija: stumped the expert. No, I mean, I can theorize on it, just that this is not a jurisdiction I'm familiar with. Familiar with enough, you know, specifically the first amendment. I'm not as familiar with the first amendment to make it I can make an educated guess but right well I don't know if an educated guess would be a fair answer at this point

Jonathan: So I mean we we see things the United States like there are there are open source projects that are essentially Malicious software, right?

And so, you know, someone will publish it and throw on there, you know, this is for uh Entertainment not even entertainment education education purposes only gets published under an open source project license And then the next thing you know that piece of software is used in an attack somewhere and I don't know that i've ever seen someone get Prosecuted just for writing the software and my understanding has been that that's always basically been because they are protected because that is protected speech, even though it's code.

Matija: Eh, I think it's more like if, if it's, like it's similar to the, um, to the CRA here, right? When I said, I mean, if, if it's an open source project, they're not liable, but as soon as you, you know, use it, the, the one who's using it is liable. Um, and I mean, from the first amendment, I mean, here's the way I would approach the first amendment question is, um, if I.

Violated, if I copied like half of, you know, let's, let's, let's use a blatant, um, examples. If I copied like half of a Disney movie, um, and remixed it a bit, maybe, and then release that as my own. Would the First Amendment protect me? I mean, it would probably protect me that I can say that, but I'm still gonna get into trouble.

I mean, nobody's preventing me. Nobody's telling me I'm not, I can't do that. It's just that, you know, I will face the consequences. So I think it would be similar in this case. But again, I'm not an expert on the U. S. constitutional law and its amendments, but Sure. So, um, but you're, you're the other part of, uh, Randall's question was, um, if I understood you correctly, was the liability for introducing not a security issue or a bug, but, uh, introducing a license issue or copyright infringement.

Um, and that's something I'm usually working with. So that's more of a home territory for me. But if you, but, um, if you had a. Further question, just please continue and we can get back to that one later.

Jonathan: No, I think that's an interesting, an interesting direction to go. Um, and one of the, one of the things I saw that you were involved in is some, some, uh, some CLAs.

The, uh, the, what is it, the CLA 2. 0? What, um, Fiduciary License Agreement. The FLA 2. 0. Yeah. Okay. So that's, that's kind of a, it's, it's one of the solutions, I think, to this question about, you know, licenses and whether they're compatible, at least it's something that I've seen businesses try to use as a solution is, well, we'll just have everybody sign a CLA and then if we find a license incompatibility in the future, well, then we can just relicense it.

And, uh, I, I'm, yeah. I'm very curious, like, kind of your perspective, maybe even your background on the idea of CLAs and what the FLA 2. 0 is and what the purpose is.

Matija: Ooh, oh yeah, that's a, that's a pet project of mine. I actually, my master's thesis was specifically about the FLA, so the versions 1. x and the 2.

0 was a result of my thesis. Um, but you know, if we just go back to the basics, as if, you know, as Randall said, like, what you're, what you're As a project, you're getting in code, you have to understand there's like this concept, you know, you, you understand the concept of inbound and outbound code or, you know, upstream, downstream, right?

Um, so the same is with rights. So, you know, rights have to flow with the code. Otherwise, you're not allowed to do anything with it. Because of copyright and it being automatic and all that stuff. So you also have like inbound licenses and outbound licenses. So the outbound license of a project would be, you know, what you would typically find in the readme or the license file.

So this is everybody out there to the public. This is the license the project is published under. It, you know, in practice, it gets more complicated. There's usually like a dozen of them in there. But, you know, let's keep it simple. Um, and, you know, typically a. Open source project pretty,

I'm not going to say easily, maybe sometimes too easily decides what their outbound license is and then they try to backpedal later on. But, um, at least, you know, that's a relatively, you know, at least you have like one main license that you choose as a project. But then, you know, if every piece of code is protected by copyright, By default, um, then what do you do with all the contributions?

So the contributors have to agree to give it to you under a license. So for a long time, and, you know, still, um, the assumption. Was, and it is an assumption, you know, this has to be said. If it's not explicit, it's an assumption that contributors are contributing the code under the same license that the project is under.

Um, but that's an assumption, right? Um, and that is usually called like the inbound equals outbound, um, licensing situation, which is, uh, I think a term that Richard Fontana coined. Um, he's a really smart lawyer, uh, with red hats. And you can formulate, the best way would be that you formalize that. You put somewhere in the README or some other documentation that's easy to find, a contributing file or something like that, where you explicitly say that, you know, every contribution you make is going to be under the same license as our outbound license.

So if the project is under GPL2, then whatever you contribute is under the GPL2. Like, for example, for the Linux kernel. Then, as you guys alluded to, there's, there's, there's The CLAs and CAAs. So, and there's a distinction, a small, but big distinction between the two that people mix up because, um, a CLA is a contributor licensing agreement, um, which means that it is an agreement on what the licensing will be for contributions or between contributors, and that can be whatever.

That is a very, very broad spectrum. What you can write in there. Um, It's just an umbrella term. It's like, you know, just basically, you know, this is our inbound licensing policy. Um, But then you also have, um, the CAAs, so which are copyright assignment agreements, which, you know, means, you know, if you sign that one, you're assigning all your copyrights to, you know, this project or this company, typically.

And this is usually, you know, people conflate the two, uh, you know, some companies explain especially make sure that they, they're vague about that. They call it the CLA, but in practice, it's a copyright assignment. So you actually need to read the text to figure out the difference. Um, and that's, you know, that's as Randall said, I was like, that's, you know, the basis of it is if, you know, the company has, if, if, They only allow contributions that are, that have signed a CAA or CLA, all the contributions are covered by those.

And then, you know, the, the upside is that the company or open source project then has all the rights it wants. or needs to change the license if they need to do it later on. For companies, there may be different reasons for, you know, open source foundations or projects. There could be different reasons to do that.

Um, CIA is the, the copyright assignment is like the Most extreme way, but then you have like with the CLA, as I said, there's a big spectrum and the fiduciary license agreement is one of those. Um, and the fiduciary license agreement specifically What it does differently than most CLAs is that it ensures that, you know, first of all, you know, if I signed the FLA, um, you know, for example, like even with a company, like, let's say I signed the FLA with a company.

And that means I give all my exclusive rights. Um, and, you know, we can discuss this, there's, you know, jurisdiction reasons why you cannot assign full copyright in most jurisdictions, um, outside of US, um, mainly most of Europe, you cannot assign copyright, period. You can just assign exclusive rights.

Effectively, same crap, but different and for practical purposes, this is like 99 percent the same. So with the FLA, I would assign all my rights to the code that I contributed to the, you know, foundation, um, company, whatever. Um, but the company foundation would at the same time contribute, uh, give me back a non exclusive rights of the exact same kind, like everything that I could you know, gave them the rights to, I get all the rights back.

They keep the exclusivity, but I get non exclusive rights and I can do with my code, whatever I want. So I can still, I can run my own code as a closed source. If I want to, I can contribute to a different project under a different condition if I want to. The only difference is that they keep the exclusivity, which is a bit not really because I get a full copy of the rights as well.

Um, so that's. One part of the balance and the other part of the balance that the FLA brings is crucial and that is that as soon as, um, that it includes the clause that the fiduciary, so the company or foundation that received my rights, they are under the obligation by the FLA. They're under the obligation to keep the code that I contributed to them also under an open source license.

So they can, they can run a dual license situation where they have a proprietary version that they monetize or whatever. But my code, if there's code in the, proprietary version that I wrote, that, that code also has to be present in an open source version. And if they don't do that, I can just say, I can automatically say, well, I returned, I, you know, you broke the FLA.

Which means you don't have my rights anymore. So they lose, automatically lose the rights to my contribution. So that's the, that's the interesting part of the FA.

Jonathan: Yeah. And so I, I'm, I'm, I'm super curious that that bit right there, that they lose their rights if they break it. Like that's, that's in a lot of other open source licenses, I believe, like that's part of the GPL and, and all of that.

I, I think, uh, something, something sort of to that effect. Yeah. Um, it, it in that it has, it's, it, it is. Ideally, it has some teeth, right? Of some sort. Um, has, has that ever actually been tested in court?

Matija: Yes. How did that go? Um, I don't think I'm allowed to say because I was. Part of the, I was part of the, it was not part of the court case, but, um, there was a situation where a company and a different company, um, both held, uh, used code, a lot of code, um, that was contributed, that was, uh, under the FLA to a different entity, to a foundation.

Um, and they, interestingly enough, they sued each other. So the company and the company, the other company sued each other for copyright infringement, which was interesting because most copyright, like most exclusive rights were held by the foundation. Right. So the foundation had to go, you know, had to, it wasn't directly involved in the court case, but, You know, they had to bring in statements, et cetera, et cetera.

Um, and because that's when I was kind of involved with the foundation. I don't think I'm allowed to say too much, but it did it, I can tell you, it does have teeth. Both companies are still alive and there's a, there had to be a three way agreement on how to settle that. And it was settled in a way where, um, both companies and both projects are alive and they both have to also be open source.

Jonathan: And so that was the, the FLA. And so it, it survived because I know this is a,

Matija: It was in court, but it was, it was settled. So it wasn't,

Jonathan: I know this is, I know this is one of the things that it's kind of like a nightmare scenario to some open source people. It was pretty bad. Well, so the, the nightmare scenario is that the, say the GPL, Gets pulled up in a court case.

You've got a, you've got a court case and the GPL is the center star. And in that court case, the judge looks at it and goes, for reason XYZ, this is not binding, right? This is, this is not a valid license agreement. It doesn't have the teeth that you thought it did. Um, and that's, that's kind of the for, for some of us that are all in on open source, like that is the scenario that keeps us up at night.

Matija: But I mean, here's the thing is like on the other hand, um, and there's the distinction, this is going to be similar distinction as with the CIA and the CLA is what's the distinction between a license, like an open source license, um, and, uh, and a EULA. And if, you know, by default, copyright law allows you to run a piece of software as long as you legally obtained it.

So if, you know, I get some code from Randall, um, and he will willingly gives it to me, but there's no license attached, you know, nothing. Um, and I run that code. Um, he can't sue me for running the code if he didn't explicitly provide you know, prevent me from running the code. He gave me the code, I illegally obtained it.

And if I downloaded from, you know, if I found it on the internet, um, I knew that I don't have the rights in it, then I'm, you know, I'm probably not allowed to run it, but, you know, like pirated games and stuff, you know, you didn't legally obtain it. But if you did, Did obtain a legal copy, um, then, then it's fine.

So the thing is like a license is by definition something that gives you more rights than what you get by law. So an open source license is a license by legal definition because it gives you, you know, the right to use, study, share, and approve. Mm, the code. Whereas, you know, by, by copyright law, you only get the right to use it.

You're not allowed to copy it. I mean, it's copy left or right, right? Mm-hmm . So you're not allowed to copy it. So also the other things don't fall off. I mean, there's some exceptions for interoperability reasons, et cetera, et cetera. But so whatever gives you more rights as you know, the receiver, then you don't need to agree with it because.

You know, you did notice you don't have to click on I agree when you download a piece of when you install a piece of software with GPL or BSD or whatever, because you're not agreeing to less, which is opposed to the EULA, which is, you know, it does say end user license agreement, but the trick is it's not a license, it's an agreement, it just happens to be called the EULA, but it's technically it's an agreement because typically in a EULA, Um, and they differ wildly, but typically in a EULA, uh, when you read it, you notice that they tell you you're not allowed, you're only allowed to use it on X amount of CPUs, you're only allowed to use it for X amount of time.

You're only allowed to use it for certain purposes. Um, your. You're not allowed to, to, to share it. You're not allowed to do copies. You're not allowed to do basically anything. So you agree because you get less rights, uh, through the EULA, you actively have to agree with it as you know, the other party. Um, because it's not in your benefit.

It's, it's, you know, compared to just copyright law, it is, you know, potentially to your detriment.

Jonathan: Interesting. Are there, are there any cases where like, and this is a question from David Ruggles, one of our live audience members, uh, are there any cases or what would happen if different nations? Treat the same open source license differently.

Are there is that happens? Yeah, so what what what do you what do we do? Like what's the that just has to make everything more complex, right?

Matija: I mean it does but so coming back to to finish that thought on the GPL where you said what happens if If if a court says well the GPL is you know, it's not it's worth this piece of paper piece of text It's what happens then so that means that the GPL You Didn't give you all the four rights or the four freedoms.

So, you know, the cop, the copyright owners, uh, are the ones who still have all the rights. And, um, so depends on what you're looking at. If you're, you know, if you're. The court cases rarely go into the point of are you allowed to use this as an end user. They typically go into the, into, oh, somebody violated the GPL because they didn't provide the source code or they didn't provide the, You know, provide the full source code or they didn't mention it's under the GPL to the, to their users, et cetera.

And in that case, if the GPL went away, you are still completely violating the copyright. So, so, eh.

When it comes to treating licenses differently, interpreting licenses differently in different jurisdictions, that actually does happen. But that's also, you know, it's, it's a question of, What is, again, what is the issue here is like, I know that I think there was a Chinese or a Korean court case that was a bit different than what you would expect.

Um, but ultimately it didn't, you know, it's usually the court cases, um, in most jurisdictions out there, of course. You know, U. S. and the U. K., et cetera, do have this common law system where you have to take into account legal proceedings, et cetera. In most other jurisdictions, the law is the law, and the court cases are court cases.

They're between the parties. Um, yes, you know, in If it comes to a, you know, very high instance, you know, it would be odd for a lower court to, to say otherwise than the higher court in an exact same or a very similar case. Um, but they would just need to, you know, argument why this is not the same as it, you know, the, you know, 10 years ago, um, uh, Supreme Court, uh, ruled.

Um, but yeah, potentially. You know, potentially it could introduce some ambiguity and the ecosystem, but in practice, it doesn't affect that much. I mean, there's very, for how long we've known free and open source software and for how, in how many places it's used https: otter. ai

And, uh, in most of those court cases, it was more like a between the parties thing. And a lot of them just settle out of court anyway. So there's what is,

Jonathan: ah, now I lost the thought. Are you, are you watching, have you been watching the, uh, the software Freedom Conservancy versus Vizio out in California?

Matija: I, I'm following on the sidelines.

I'm not deeply following. Um, personally, I mean, from, if, if I did that in Europe, I would Take a different approach, but this is also probably a bit of a, you know, jurisdiction difference. Um, but, um, yeah, it's an interesting one. It's an interesting one. It's let's see happens. Um, it's is it going to like, what, whether it goes one way or the other.

Will it defect the whole world? I don't think so. I mean, it's, you know, we we've been People do, in other jurisdictions, do sometimes quote, you know, courts from other jurisdictions. But that's, I mean, if, if quoting your local court isn't binding, this is very much not binding. This is just like, somebody looked at this in their own jurisdiction, and this is what they found out.

And they're like, okay, that's an interesting thought. But I just had that thought again and then I lost it. Oh, yeah, so an example of You know different jurisdictions and different courts interpreting it differently on the mailing list of the Open source legal experts that I mentioned there's been like an evergreen discussion Or argument was whether the GPL is a license or a contract

Jonathan: mm-hmm

Matija: And, uh, in the US that's actually, you know, it, it depends on which court it goes to, et cetera. Right. Um, and then also, you know, which, what kind of damages you can expect or what kind of outcome is possible at all. Uh, and eu, that's a nonsense question because it's both. It's a license in a contract, and the U.

S. situation here is, it cannot be a contract, or at least used to be, that it's, it's, it could be weird to be a contract because a contract needs to have, um, consideration. So it needs to have, like, both parties need to have something to exchange. And that was the main argument for a lot of U. S. lawyers to say that, you know, it's not a contract.

Um, I think it was like a few years ago that it came to a, it was another court case. I don't remember. I remember which one it was, where in the U. S. where it actually did say, well, it's also a contract. But the, you know, the consideration is like all the worth that The software brings, right? Like all the rights are actually a consideration.

Jonathan: Yeah. I'm, I'm looking for this. It was a, uh, a California district court. Um, she ruled that the GNU General public License is an enforceable legal contract and I'm not seeing exactly why. Yeah. Yeah. Uh, south Korean company hand com. Yeah. Hand com. And what was the

Matija: artifacts thing?

Jonathan: Yes. Yeah. Yeah. Yeah. The artifacts thing.

Yeah. It's hand combed artifacts. Yeah. Yeah. So that, uh, that's, that's one of those, that's one of those examples of, uh, you know, it, it actually got tested and the results of that sort of changed things for, well, I guess didn't change in this case, didn't change anything, just clarified. Um, but yeah, super interesting.

Matija: Yeah. They did change, you know, the things in a lot of U S lawyers mindsets when it comes to, yeah. This situation, but yeah, it just clarified it.

Jonathan: Yeah, yeah. Okay, so I would be, I would be remiss if I didn't ask you about this, and that is, If an open source project or an open source contributor has a concern, and I can even give you, I can demonstrate one for you that, that is not very real for us, Um, I've, I'm part of an open source project and we run an MQTT server and we let people upload data to it.

And one of the things that gets uploaded is location information, totally opt in. You know, we're not collecting that, but still it gets uploaded there sometimes. And the question that we have then is, well, what about the PII laws? You know, the, the California consumer protection act and all of that. Uh, What sort of, um, you know, like what, what's our liability under that?

And I know that you haven't researched like this particular question, so you don't have an answer for me at the moment. But the bigger question is, though, when an open source project has a concern like this, where is there a place that they can turn to? Is there a, you know, a panel of experts out there?

Is there a legal group that we can say, Hey, we don't, we don't really have a whole lot of money, but we need, You know, we need opinion, but we also need like the equivalent of a lawyer on retainer just in case this blows up in our face. Is, is there any kind of solution like that out there?

Matija: Oh, um, back when I was, when I was the legal coordinator for FSFP, I would have had the answer.

Um, um, so we did use to have the, the, FSFE used to have a legal team, uh, which did comprise several, um, mostly European though. It was actually only European, but very, very experienced lawyers in these things. Um, and we did answer, like, if it was something that you can just send an email and it's not a thing that you need an attorney for, so you don't need to actually go to court or have some, you know, attorney privilege, et cetera, uh, somebody would just answer.

Um, So that was a mailing list for that. I don't know if, I think they started something similar with FSV again. Um, so that's one way I would, I would look at, uh, FSV. org and then search on their, um, legal team or whatever they call it nowadays, I think they have a. Place where you can ask questions. Um, and if it's something that's like a deep problem, like a very specialized problem, they have access to the, to the legal network.

So there's like, I think like 400 or 500, uh, legal experts on that mailing list, um, that they can just, you know, ask, um, if there's any questions. There's, I know OSI, uh, the open source initiative has, um, uh, license mailing list, which is public, um, and there's also a license review, but it's, uh, for a different use.

Um, so that one is a, that is one where you can ask potentially questions about licensing, not maybe personal information, uh, liability laws, but, um, there's a lot of lawyers there as well. Um, and, um, probably FSF has something as well set up, I don't know. Um, I was never actively involved with the FSF, so I don't know what they have, but they, I, probably they have something.

Um, so when it comes to GPL, that they might have something. Um, so that would be options. I mean, you're, it's not 100 percent sure you're going to get an answer. Um, because you have like the attorney privilege is. is a very important thing when it comes to lawyers. Um, and there's also liability. You know, if you're acting as a lawyer for somebody or acting as an attorney for somebody and you're not, especially in the U.

S., there's serious liability for that, for the lawyer in question. Um, so So, but if any, it, at least it's going to be, it's, I would say that would be the best foot in the door. Um, so it could peak, you could pick, pick some, uh, somebody's interest and they'll just, you know, answer your off list and say, well, you know, you didn't hear it from me, but maybe you should read this and this for more information.

Um, oh yeah. And the, yeah, yeah, of course. And this SFLC as well.

Jonathan: So that's the, those would be. Yeah. The Electronic Frontier Foundation and the Software Freedom Law Center.

Matija: I don't know how active they are. Right now anymore, but yes, FSFLC was, uh, had a thing going for a while. And there's also the Software Freedom Conservancy, um, that's also an option.

And clearly if you're near or if you're a member or near one of the big, Open source foundation. So like, you know, Apache, Eclipse, um, Linux foundation, et cetera. All of those have, um, legal people on board or have access to teams. So if you have a problem, especially if you're a member, you know, you ask it there and you might, you know, you probably will either get an answer or get, you know, put Told where to go.

Jonathan: Yeah. Oh, that's, that's interesting. I hadn't, I had not thought of the Linux foundation as one of the places to turn to, that's actually really useful. Um, all right, Randall, did you have any, any final questions that you wanted to get in?

Randal: It was, uh, I was just going to point out that, uh, when my case first started, the people of the EFF were very helpful in contacting me with, uh, Mike Godwin.

As sort of initial contact, and he was their point guy for years on, uh, where to go to about legal issues relating to look to your foundation's sort of charter. Of course, he couldn't represent me, so he got me directions to get to the local guys, and he also supported my local guys. And, uh, and helping me defend against that, uh, the pen testing shouldn't make you a triple felon.

But, you know, um, it was, uh, it was really interesting, um, uh, being able to work with them as an outsider. Uh, it was kind of fun.

Matija: Yeah, but maybe an important thing here also to mention is if we're not talking about the big Foundations that have like a very wide net You know EFF is is it's relatively big and pretty important, but it's also very u.

s. Centric. Yeah And the same would be for a lot of others. So As, as, as I said, like, you know, the jurisdiction by jurisdiction, you're going to have different issues. And as Randall said, you're, you know, if you get, if you get into a court case, it's going to be a local thing. So, you know, if you're going to come, if you're going to go have a, have something in Canada, you know, a U.

S. lawyer is not going to do you much. Um, they could be really smart and everything, but, you know, they're, they're different. There's, you know, they're not allowed to, they're not admitted to the bar. And if they're not admitted to the bar there, then they're not admitted to the bar. Um, so maybe, you know, depending on, you know, who listens to this podcast as also, you know, make sure that you search for something in your jurisdictions.

Like, you know, for the EFF equivalent in Europe would be something in France. It would be something like, like what the, or April. For open source or, uh, Edry, so European digital rights initiative is maybe something that if you don't have an open source issue or a free software issue, but you have like something else, like EFF ish, um, Edry would be one of those that I would go to if I had a question that was, you know, maybe PII related or something like that.

And because the, the really good thing about Edry is also their network. Um, Um, they're not just an organization, so they know who to send you to, so that would be

Randal: the, this might be a 15 minute question, but let me at least get it started and see how far we can get in a short amount of time, um, over the years, there have been.

Sort of misunderstandings and gaps in interpretations of what legislators think software is about and what we all know software is about leading to some really weird laws like the DMCA. Um, is that getting better? You think over time, or is that getting worse? Is it getting worse because it's international?

I mean, what, what's, what's your take on how well. Legislation is keeping up with the technologies that we have.

Matija: I mean, legislation will always, almost always. So that, that's come, that brings me back to my, you know, something I mentioned, like at the very start is why I started studying law is, um, I found. I find law interesting because on one hand, it has two functions.

On one hand, it has the function of, it has to be, it has to represent the situation, um, that we have. So it has to be applicable to the real world. Um, And if it doesn't, there's a legal theory, I think it was by Jelinek, an important figure in legal theory, uh, who said that if it doesn't, if law doesn't correspond with, you know, real life anymore, it becomes unlaw, and therefore ignored.

Um, and on the other hand, you have the other side is law Not always, um, but sometimes also has the function of pushing society in a specific direction. So it predicts the future in a way. Um, so this is something that you would have, like, when you know, you know, where you have things like, um, when, uh, women were given voting rights, where they didn't have them before, that was definitely something that, it didn't just happen, you know, you had to enact a law for that.

Um, and this is, you know, the interesting, this bipolarity of, um, the functions is what I find interesting. And I think, I mean, the cardinal sin of, of. Intellectual property in general is that it's not something that's very new. It's like, you know, 200, depends on how you look at it, 200, 400 years old. Um, so like most of the stuff that was written before or drawn before or composed before doesn't fall under copyright at all.

It never did. It didn't exist.

Jonathan: Right.

Matija: Um, And now then we suddenly have this thing, which started depending on, you know, where you look at it, if you look at it from the UK, or which then, you know, developed into the US as well, copyright law, or you'd look at it from the, from the French authors rights, which developed to most of continental Europe, you have different spins on how, what was important.

Um, and why, uh, copyright happened, uh, roughly ish at the same time. Um, so that's already a cardinal sin in IP is that it exists. Um, and then it's how do you shoehorn something that is software into this? Um, and early there was a debate whether software is, you know, before, before I, I'm gonna, it, it, interestingly, I like, there was a time where software wasn't copyrightable.

It was public domain, period. And it was actually, I think IBM, actually, software licenses predate software copyright because IBM was slapping on licenses on their software, on their source code, because back then it was just source code. Um, on source code. With the intent of, you know, this is either going to be a contract, um, uh, or ignored or might happen to be something.

Um, so

It was always, I think copyright was always a bad option, um, for software because software, like part of, there's a, you have like the technical distinction is like, there's a test of originality, um, in software, uh, and in, in, in copyright in general, and the thing is like the more formal something is, Okay, now, coming back, um, copyright doesn't, copyright protects the expression, it doesn't express an idea.

So, If the expression and the idea, you know, if there's only one way how you can express an idea, then it becomes a fact, then it's not protected by copyright. Because that is, for example, would be, you know, arguably is the case with, uh, config files. Um, because there's your, you know, if you want to set up a config configuration in a file to do this, you know, the idea, very, very little you can do, you can be completely wrong.

interesting things around there. Um, so it's, it's a mismatch already. And then, you know, the length of the term for copyright protection is, you know, for in general, is, you know, A bit silly, you know, who gives a, I mean, if I die, I mean, now I need to, now, now, now, now people need to wait another 70 years or 90 years after I croak that they can use my, that my stuff gets into public domain.

I mean, what, how does that benefit me? It doesn't mean theory benefits my heirs, et cetera, et cetera. Um, but it's also, um,

It's also a mismatch in the sense of, I lost it. Um, it's, it's late here. So just so you, just so you know, there's a, there's a, it's, it's not dark because I'm in a basement.

Jonathan: It, it, it reminds me of the, uh, the recipe book, right? Like the actual recipes in a recipe book. Yeah. You can't copyright because they're just, they're mechanical.

Matija: Yeah. So, and, and this is why there's a mismatch already in the start. Um, you know, you could argue that. You know, patents would be a better option, but then, you know, we get into a completely different problem. Patents have their own problems. Yes, everything has its own problems. So the question originally was, is it getting better or worse?

Both.

Randal: Wonderful answer.

Matija: That's fair. So it's, it's like, it's similar is like with the CRAs as an example or the security legislation that we have on even in also in the U. S. Um, There's on one hand, you know, legislators seem to understand software better from the point of view. Oh, there's Security issues there.

Somebody needs to make sure it's it's it's done properly and then you know, how do you do that? So, I don't know. I'm

I have mixed feelings on this. I wouldn't say I would I wouldn't say I'm cautiously optimistic. I wouldn't say I'm pessimistic. I'm just, um, it's, it's the way it is. And, um, you win some, you lose some. Yeah. I think, I think they understand more than they used to. Um, but whether they understand enough to make everything better is a good question.

Yeah. Yeah. And the problem with law is always, I mean, it's not gonna, the law is typically Um, written by politicians and you have a lot of lobbying and a lot of, uh, stakeholders involved in pushing the agenda here and there. And this is maybe an important call through for all the open source, uh, you know, software advocates out there that you need to make yourself heard because, you know, the, those who have an interest to have, you know, to keep copyright as difficult to use in software Those guys typically also have a lot of money, and they do lobby.

It's not like they don't.

Jonathan: All right. Well, we have hit the bottom of the hour again, and I've, there's a couple of questions that I've absolutely got to ask you, and that is, what's your favorite, and it's totally different, what's your favorite text editor and scripting language?

Matija: Ooh, um, I mean, the geeky answer is Vim and Python, but in practice, I mean, uh, I use most of the time I use Kate.

So the Katie's, uh, advanced, uh, text editor. Um, I did try Helix and I liked it a lot. Um, there's one or two things that I don't like about it. I, if you like the idea of VIM or VI, but you're not in, you know, you don't have the muscle memory yet. Helix makes a lot of sense. Um, it does things in a more logical way than VIM does.

Um, But I just, when I, when I don't use it for a while, I just start doing Vim things and then they don't work. It's like, ah, um, but, and same with, I'm, I'm not really good at programming. I did, you know, ages ago I did in school. I did try to learn, uh, I did go to a course on, uh, Pascal. Uh, I was never good at it.

I just know, I just still remember a lot of semicolons and read lines and write lines. Um, I was, at some point I knew, I learned how to program in Python, I, but, um, in practice, I use fish a lot. I have a lot of scripts in fish. I love fish. Cool. Fish is great.

Jonathan: Yeah.

All right. Uh, we appreciate you being here.

Uh, very, very much. That is, uh, Mattia Mattia Shukla. Yes. Uh, that's, that's what it sounded like in my brain. I just couldn't get it to come out of my lips. We appreciate you being here very much talking about some legal issues and, uh, it was a lot of fun. I appreciate it.

Matija: Likewise. Thanks for having me.

Jonathan: All right.

Uh, so yes, thanks for staying up. We appreciate it. Yeah, no problem. All right. What do you think, Randall?

Randal: Well, uh, covered a lot of areas. Uh, it's, uh, it's, it's never uninteresting to talk about computers in the law. Uh, particularly for those of us who have been at the, Short end of the pointy stick. So it's like, okay, what's going on now with how these are all working out?

A couple of subjects that we didn't get into that would have been interesting, but you know, for a matter of time, I didn't want to bring up anything else. But, um, the idea of, uh, my personal liability is an individual contributor contributor using LLM based solutions to create my code that are later found Themselves liable for reproducing copyrighted code without disclosure, and I'm wondering at what point Who's we haven't tested this yet in court like we were talking about earlier It says it hasn't been tested yet It's going to be interesting to see what the first test case actually comes out to be where Uh, uh, I'm using something like co pilot and it inserts GPL code into my code.

I then submit it to a BSD style project, uh, which would not have the same GPL protections. Uh, what, where do the fingers point to in that chain?

Jonathan: Yeah.

Randal: And

Jonathan: that is interesting. It is, it is such a big question that several projects have simply said, we will not accept any code that's written by AI or LLM,

Randal: right?

Jonathan: So

Randal: that's, uh, it's interesting. And that's also why I stopped using Copilot once I sort of discovered that was happening. Uh, Codium, uh, C O D E I U M promises that they're training only on licensed appropriate material. So that does help a bit. Uh, and I'm hoping that Gemini also trains only that, because I'm also using Gemini quite a lot these days for that sort of stuff, so.

Yeah, Gemini is getting pretty smart. Uh, for those of you who haven't tried it yet, check it out. They're doing some really cool stuff.

Jonathan: That is an interesting segue, because next week, the plan is to have the guys on and talk about the open source AI definition. And that has been, yeah, that's a big deal.

That has been kind of a, uh, a brouhaha because everybody has a little bit different idea of what open source should mean when you're talking about artificial intelligence. Yeah.

Randal: So

Jonathan: OSI has put their oar in and they've got a definition and we're going to talk to them about it. Hopefully. That's the plan.

Awesome.

Randal: Awesome. That sounds good. I may watch that show.

Jonathan: Yeah. Did you have anything you wanted to plug, Randall, before we let folks go? Ooh.

Randal: Just in general. I am doing, uh, these days, I'm not doing anything with Pearl. I am, uh, for those of you that know me as the Pearl guy, uh, I'm doing everything with, as you can see, by reading the hat.

Flutter, flutter and dart are my, uh, primary things. I'm one of the 10. Uh, Google developer experts in the United States about the Dart and Flutter arena. So I spent a lot of my time teaching online, uh, making presentations, uh, answering questions on things like Stack Overflow and Discord and stuff. And, uh, I'm also always looking for commercial contracts for that and have to keep me off the street and out of jails.

I like to say that was more literal than before.

I would say it's a joke before all that happened and then I started to say, oh, that's a little too real now, but, um, but yeah, uh, and, uh, other than that, uh, I'm, uh, I'm, I'm happy. I'm getting a lot of stuff done in my life. Uh, I'm, uh, I've taken up karaoke, uh, running a karaoke show on Friday nights now, so I'm having a great time helping other people sing and choose songs, stuff like that.

It's a lot of fun. So yeah, it's all good. And, and thanks for having me back on the show as always. Cause I. I do kind of miss the hosting role for this,

Jonathan: but

Randal: not enough, not enough to actually do it, except when asked on temporarily. I, I, I am very happy to have the five hours a week back that I was putting into this show for 13 years.

So I'm very happy that you're doing that now. And all y'all do is tell me. Minutes before we go on air that here's the button. I gotta press so i'm much happier with that now

Jonathan: Yeah, well, we we appreciate having you it's always fun to have you back and we'll have you again in a few weeks All right If uh, if folks want to find me, of course Pack a day is one of the places is where the security column glows live on friday morning That is the home of floss weekly now.

We appreciate them for that And then I still have the untitled linux show over at twitch. tv Twitter. tv and we have a lot of fun with that covering the news and the tips and all the things with Linux. So you can check that out as well. We appreciate everybody that caught us that watch us live and those that get us on the download and we will see you next week on Floss Weekly.

This week Jonathan and Randal chat with Matija Šuklje about Open Source and the Law! How do Open Source projects handle liability, what should a CLA look like, and where can an individual or project turn for legal help?

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, it's time for Floss Weekly and this week, Rob joins me. We talk with Alistair Woodman about FR routing, but also the future of open source, open source business, software liability, and all kinds of other stuff. It's a lot of fun. You don't want to miss it. So stay tuned. This is Floss Weekly, episode 814, recorded December 31st.

The Banksy situation.

It's time for Floss Weekly. That's it. It's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and we're back after a couple of weeks break for the holidays, which is kind of ironic considering that today is December 31st, so we're not quite through all of the holidays yet, but that's okay.

Uh, we've got a guest, we've got a co host speaking of which, uh, let's go ahead and bring Rob on our other resident, uh, networking engineer, networking experts. And, uh, Rob, I think we're going to have to take our networking expert hats off for today because we've got, we've got a, a real expert, or, uh, he was telling us before the show that he doesn't do a whole lot of coding these days.

So, so maybe he's the business manager for the real experts. We'll, uh, we'll ask him

Rob: about that. Yeah. I'm interested to see what he has to say about, uh, about the business side of things.

Jonathan: Yeah, yeah, that's something we're very interested in around here. Um, so our our guest today is Alistair Woodman and he he has an interesting background Uh, he says by education.

He's a physicist And by inclination he is into Product management and business development. He's done apparently some coding in the past, but he said he retired from that. He hung his head up on that one. Um, but he's involved with some really neat things here, like, uh, FR routing, which I've been telling people this week is going to be about like sort of the, the deep internet protocols because they do, uh, they, they do some of the, the routing things that you've never heard of, or maybe you've heard of once or twice, but you don't have like, BGP, the Border Gateway Protocol.

I know, I've never messed with BGP. Uh, I've written about it because every once in a while you'll find a story where, uh, somebody advertised a bad BGP route and we accidentally routed the entire internet through Brazil for a few hours. It's like, I'm sure nothing malicious happened as associated with that, right?

It'll be fun. All right, let's go ahead and bring him on. Mr. Alistair Woodstock. Welcome to the show, sir.

Alistair: Hello, good morning, good evening, whatever good next year for those of you, uh further afield in the east So,

Jonathan: oh,

Alistair: yeah.

Jonathan: Yes. I've i've already gotten some well wishes for 2025 in my various discord channels and that's always fun Yeah All right.

So What, what, what all do you have your fingers in? What all are you involved in? Um, are you a wizard of the deep internet magic? What, uh, what are you up to these days, sir?

Alistair: Well, so, well, I, I left, uh, formal, you know, working business stuff in 2011, uh, after putting in a solid 25 years in high tech. Um, for companies being well paid.

Um, so I took early retirement and then I pretty much figured out that I didn't enjoy golfing. Um, so I think the last time I did play golf was as an 11 year old. So I pretty much figured that I wasn't going to enjoy it. So I decided what else am I going to do? And I actually figured out that I really like hanging out with nerds anyway so, um, I still hang out with nerds and I work on open source projects and And hanging out with nerds is either a terrible thing for many people and uh, but I actually enjoy dealing with people who are sort of uh, uh, very diverse very focused on what they're interested in and um, I mean It's sort of a popular meme here in silicon valley that we have a lot of you know on spectrum Uh type individuals and I I like that Working with them and I like getting, you know, good results out of them.

And I like helping them, um, interface with, uh, business issues because people in the tech community tend to be very focused on what they're doing. And of course the thing that they've done. Is super important and super useful. And the question is how are you, you know, keeping a roof over your head and paying your bills?

So a lot of the time, um, I spend my time running around, making sure that that part of the process is working for open source projects, unfortunately, I can only do a couple. Um, but I'm heavily engaged in the two that I'm active in at the moment.

Jonathan: Okay, so there is, there's a lot, there's a lot in just that opening statement and uh, I didn't realize we were gonna go down this route but it is super interesting to talk about.

Um, and that is that it's quite a, it's quite a skill set to be able to talk to nerds. And, and I, I very much appreciate that thought of, Some of these people are on the spectrum and I think that's true because I've known some of them over the years and It sort of is a Useful, but also important skill set to be able to I I call it translating.

I refer to it I translate for nerds and I think that is that is super useful. Um, I'm i'm thinking of a there's a there's a youtuber, uh, david plumber, I think um that I follow he worked at microsoft for years and he's like at some point people told me enough times that I ought to go get tested to see whether it's on the spectrum and I'm like You know after laughing them off enough times I finally went and did it and then I started reading about being on the autism spectrum And he goes so many things about my life started making sense.

I think it's that's super fascinating I think it is a real thing. I think it definitely is.

Alistair: Okay. Well what I think it attracts people with um, special Gifts and skills, right? So some of it can be just pure IQ related stuff, but a lot of the requirements for being a very good coder are sort of very good memory, understanding of structural things across space and time, being able to sort of figure out what that really is.

means in terms of where to find something in code. Um, and these are very interesting skill sets. Um, you know, people who, you know, study the Hebrew Bible, um, also have those types of skill sets, right? It's, it's very focused a lot on the written word and, and what that means, right. And it attracts a very gifted set of people.

People and sometimes they also have, you know, um, other proclivities, right? And as we all do, and Uh, they sometimes want things to happen in a particular way and get frustrated when you know, it doesn't happen Um, or the rest of the universe doesn't appreciate that. They aren't really the center of the universe So

Rob: I feel like I feel like a key strength may be the ability to hyper focus on something Yeah, whereas me when I got into it and technology I feel like I went the opposite way and I just so many things are flying at me from so many directions that I'm all over the place and and if I had maybe more of a hyper focus, I could really focus and go down one of those paths, but I'm more of a jack of a lot of things

Alistair: and that is an interesting point because when you building larger scale.

Communities and interrelationship you can't afford to be hyper focused on You need to be able to Frequency shift from being very focused on a particular thing at the time being and then try to build connectivity or causal relationships to other things and and that's um, That's an interesting challenge that that one faces when you're doing business development, right trying to find the Yeah The way of, of connecting something that a project's doing with a need that somebody else has somewhere else.

Jonathan: Yeah, so one of the, one of the interesting things that I've been faced with a few times, honestly, trying to make money writing code. Um, when, when you, when we're writing code, things are very, um, binary, you could say. You know, either this code is written correctly and it works, or this code is not written correctly, and either the compiler tells me, or I get, you know, runtime errors and things break.

Uh, the business world is not quite that, uh, cut and dried. And oftentimes when we are convinced, you know, our internal compiler is telling us that we've got just the awesome, the best business idea. This is going to make us millions. And then you go to launch it and nobody cares. Uh, it's, it's quite a different world.

Um, I have found in trying to take my coding acumen and move it into the business world.

Alistair: Uh, that's very true. Um, and both in the business world and even just people using hobby projects from one another, right, amongst open source stuff, right, the new mousetrap or somebody's done something that already exists somewhere else and everybody goes, I don't care, right, the marginal improvement isn't good enough.

So. You know, have a nice day, right?

Jonathan: Or, or the idea is so niche, there's, you know, all of five people around the world that are ever going to care about it. Yes. Yes. Been there. Um, All right. So let's, let's get into, um, a little bit of maybe, maybe background. So you're involved from what I understand heavily with, uh, the FR routing.

Project is that is that the only one you said maybe there was two projects that you you kind of have a hand in

Alistair: Well, so the other thing that I'm involved in is the Erlang ecosystem foundation, which is a group Of projects so, um, the the two are different in scope and scale um, but if we talk about the free range routing project or frr as most people refer to it as um, you probably Um, well certainly people listening to this will be getting their traffic over, um, routers that are running FRR.

Um, we're usually, typically in OpenWRT, um, releases. So anybody who's running on a lot of the embedded hardware for home routing stuff, it's got OpenWRT in some flavor underneath and with a routing stack on that. Most people don't turn on all the different daemons that we have and support. But, uh, some people do run, um, decent routing topologies in their own home.

Um, so some people turn OSPF and ISIS and other type of stuff on and have relatively large, um, home networks already, especially if they've got a lot of IoT devices. So, um, that can be. Um, n not the typical home. I've just got a wifi and I need need an uplink type of thing. Um, which is still what open WRT will serve, but many people turn naturally a lot more knobs on.

Um, the FR R is also used at the other end of the scale on Sonic, um, which is a white box nos, and there were a couple of white box noss out there. So via, um, BISD analytics. Um, that use FRR for general routing purposes. So the forwarding plane and control plane stuff is not part of our project, but as many things in the open source software go, we're, they're downstream of us in terms of integrating FRR into their projects.

And they, Then talk to the appropriate hardware, silicon board support packages and whatever it is to build the embedded system, um, installations. So we've got a lot of code, um, on different code bases, uh, usually. As far as the users are concerned, there's a binary, although many home hobbyists will compile their own and put it on their hardware, but most people will be just downloading something and running it there.

Um, so that's what FRR does. It's also used internally, um, By telcos and some of the hyperscalers for um, sometimes purely just operational backup But other people are using it for more interesting projects so it's um, it's been around for over 20 odd years now because it used to start out. It was quagga and then it got forked Well zebra quagga and then free range routing.

So we stuck with the animals, but we went from Uh, we went from the individual animal to the the paddock basically when we changed the name to free range routing Yeah,

Jonathan: yeah, that's that's funny Um, one of the other protocols that I saw that is part of frr is uh, BGP, the board of gateway protocol. Um, I'm curious.

Are there, are there any of the, like the big level one ISPs that use any of the FR routing stack in their sort of big iron routers?

Alistair: The answer to that is yes, and then you'll have to go figure out the details yourself. So, so yes.

Jonathan: That's fair. That's fair. So it is, it is very, very likely, if not certain, that our, our bits that are being broadcast right now are going through the FRR stack. At least

Alistair: in one of the nodes, yes. At one of the nodes, yeah.

So somewhere along the path, yeah.

Rob: Is that because it's built into hardware they're buying, or are they building stuff upon this, are you suggesting?

Alistair: Um, I would suggest that from purely a percentage likelihood, it's mostly people have got a binary that they've got that came bundled with the hardware, and this will typically be the home router of most cases.

But if there are more. FRR on that link and possibly further up towards the data center or hyperscaler or some service that you're accessing, um, then they probably built it themselves on, on white boxes and possibly be tinkering with that. Um, but it is possible to just download the NOS, uh, you know, from, from folks and then just turn a binary on as well.

Jonathan: What's that people

Rob: have.

Jonathan: What's the, what's the license? What's the license FRR is under? Then we'll give it back to Rob.

Alistair: GPL. Okay.

Rob: So that sort

Alistair: of makes a, okay. Sorry, Rob. Go ahead. Go ahead.

Rob: Go ahead on the license.

Alistair: Right. So, so the interesting thing about licenses, I mean, I think it makes a hell of a lot of sense that routing is under a GPL licensing model because if people make modifications to it, they should be making modifications to a standard routing protocol that you really don't want routing protocols that can't interoperate with one another.

So, um, you sort of really, from a community perspective, it's a better solution from a licensing perspective than to allow um, People to just fork their own and then do what they want. Um, there are some hyperscalers who actually do use it internally and have built their own flavors of routing and because it's, uh, they're running it as an internal, uh, software, they do not have to give those, uh, modifications back, but many of them do.

Um, and it, but it makes sense if you want to interoperate with a router on the outside that you better be, you know, capable of doing that. So, forking and, and not reconverging is a bad thing in networking.

Rob: So, if people are using these, uh, higher end routing protocols on, on their Linux system, something they're building or something they bought, Um, Is, is this, you know, this being around for 20 years, is this what they are using or are there other, um, alternatives out there that people use?

Alistair: Oh, there are several other open source alternatives. Uh, BIRD's a BGP, um, code base that, that is quite popular amongst, um, service providers and inter exchange carriers. Um, the, um, The, if you want to be doing multi protocol routing stuff, uh, you're basically in the FRR domain, or you're going to be using, you know, Juniper or Cisco because you, you know, you, nobody got fired for buying Cisco or Juniper products, right?

That problem, right? But if you are willing to, you know, be fired and want to run multi protocol routing, then you'll typically be running FRR.

Jonathan: Yeah, so if, uh, if, if I do actually finally pony up the money and, uh, buy, buy into and get an ASN, an autonomous system number to be able to truly own my own IP addresses, uh, it sounds like FR routing is the, uh, is the project that I would want to go with because I don't have Cisco or Juniper money.

Alistair: Well, so you're certainly being a bit of a piker if you haven't bought your own AS, right? I mean, that's relatively cheap. It's like 500 a year. Exactly, right? That's chump change, right? So you should at least go get the AS. Now, whether you do anything with it is a separate topic.

Jonathan: I look forward to being successful enough that I can say the 500 a year is chump change.

Okay.

Rob: Question from Discord. Does FRR support MPLS?

Alistair: Um, yeah, that's an interesting question. Um, we do. We're on boxes that support MPLS. Um, but the, the routing it, so it, so the answer is partially we don't get in the way, but, uh, we don't typically, um, It usually requires people to have done something else or want to do something else.

The Linux stack does actually support MPLS and you can turn it on. And it will work under those, uh, circumstances. If you're running on dedicated hardware, you'll need the drivers and other types of things to do MPLS on the hardware. So, the answer is maybe, uh, probably, if you just want to run it on Linux by yourself.

It doesn't work on FreeBSD.

Rob: I imagine MPLS being, being that path, um, FRR would support the routing over the MPLS.

Jonathan: Correct. Makes sense. Um, so it sounds like this is a, a fairly complicated project, FRR, and it supports some fairly complicated protocols. Uh, How did, how did we get, how did we get to this point?

Like how, first off, how did, how did FRR get started? And then at what point did it kind of turn into, cause it sounds like it's been commercialized now. It's been used in a lot of places. Hopefully there is some money being paid to help keep it supported. And so how did we get to the point we're at now?

Alistair: Um, so as I said, it was started over, uh, 20 years ago as, um, By in Japan to actually try building routing and as a self study, uh, type of project it as, as it was. Under the Quagga flagship thing, it was already being used and was a part of, so Quagga still was and is used in some of the embedded systems.

So it's probably still out there in, in old routers that somebody hasn't updated for a long time. Um, which is one of the other topics that we might get into is, you know, how to end of life and, and maintain software over time. Um, it's. So when we, when we forked it to FRR, there were, um, business interests.

So there was, um, couple of startups that at the time, if you remember software defined networking and a whole white boxes were a thing. Um, so it became, there was enough startup money being put into the space to actually make it from a very good. Um, code base, but we're still not really competitive when you looked at, you know, Cisco and Juniper and, you know, people would use it, but many people would scoff at it, um, as not being sort of industry ready.

And certainly if you wanted to run networks, the old adage about, you know, You're sleeping at night and your page are not going off. Most people will buy Cisco and Juniper gear. But with the advent of white box, uh, switching, um, there was enough money flowing into that that startups, um, worked on making the code a lot better.

So Cumulus Networks was, um, around at that particular period of time. They no longer, Uh, exist as a separate entity. They were acquired by Mellanox and Mellanox was acquired by NVIDIA at about the same time. So there are folks over at NVIDIA now working on the project and they're a, um, major contributor to FRR.

Jonathan: And just for, for anyone listening that might not know what, what exactly do we mean when we say something is a white box, like a white box switch?

Alistair: Um, it basically a, uh, uh, Vendors have built something to spec, um, so this happened as well with, uh, what happened in the hyperscalers, where they, where they basically said, we don't need to be buying all these pieces of equipment, we can have servers that we will specify, please don't put these extra chips in there, so there's a lot of, you know, specifications for white box servers, and then there were also specifications associated with um, switching infrastructure, we can just have layer two on layer three switching, which doesn't need all the bells and whistles, um, or rather, they were so focused on chip count and hardware costs that they would specify this is all you need to put in this, you know, don't, don't over Egg the hardware.

This is all we want. Please put some open source software on this and we're off to the races. So um white box white box anything is a Statement about the fact that it wants to be relatively generic um and not particularly complex, although Now they are actually sometimes quite complex, right? Uh, but, but you know, they're, they're trying to create a lot of the interest in the white box switching thing was to remove, um, to, to restructure the business in the sense of having a vendor specific, um, my things better than your thing to actually, this is just a utility thing.

It needs to do these basic things. Please just build them at scale cheaply.

Jonathan: Yeah, definitely. Um, okay, so back to FR routing, and I think, I think back into the kind of the business side of things. Maybe the first thing I want to ask is how did, how, how did you go about, so I don't know if you were there at the time, how did they, whoever was, was doing FR routing, um, How did they get in touch with and start getting that startup money to flow into the open source project?

Uh, we see sometimes that open source projects get used for startups, and sometimes it's difficult to get that money to flow back to the actual people working on the code. So

Alistair: that is actually a very interesting question, and I do think we're a relatively rare bird, and if we segue later in the conversation to other examples, I think we are, we are Accidentally rare and that it ended up with this good symbiosis and so, I mean you you Yeah, you were talking about the whole concept of curl a couple of weeks ago, right and just how well supported curl is correct Uh, how, how well used it is and now that it is almost has a business model, uh, associated with it.

Right. And, um, that's just astounding to me, right? I mean that, that we've got something that's, that well used and that's still struggling to get, uh, a viable business model. Yeah. And so when I look at that and some of the other. Uh, projects which obviously do struggle, um, the, the reason that I think we got lucky is that routing is a complicated problem and back to this other issue, it needs a lot of interworking support.

So, when, uh, Where the F are an active project, there are probably 30 very active committers to the project and there are probably another hundred that commit, um, more than 10% of their work time to adding features. And then there's a long tail of. People who just do some feature or bug fix, you know, that show, show up, but because this all has to interoperate with other people's equipment, specifically Juniper, Cisco, and the installed base out there, um, you have to be able to prove that you've not in, you've not just added code, but you haven't put any entropy into the system.

Uh, And so there's a lot of testing necessary with those things to, to ensure that. That you've not broken the code and to your observation earlier, that you think it's a bit of a miracle that networking sort of works and you've got all these things that can connect to things, but it's not quite clear and some of them are in the process of going down or being rebooted and the network, you know, continues to work.

That's because there's all this complicated state machine machinery in there, which can easily get broken by neophytes. And even the experienced folks can break it too. So we have a very, very large CICD. Pipeline and we throw a huge amount of resources at this And so the our budget for doing testing on this type of thing is, you know over You know half a million dollars Uh, a year in terms of testing, both in terms of people and CPU utilization to keep the project around.

And I think it's obvious to anybody who works in the networking space that that's the cost of doing that business. So, The the pool of companies that are involved using the software understand that it's that hard because Everybody has worked in those companies Basically used to work at cisco or juniper in the you know sometime in their career Um, or they might have worked at you know, some of the other guys like alcatel lucent or whatever it is, but people know What the cost is?

And and how much work you need to do to avoid regressions creeping into the system So It it's not too much of us of a hard sell. I still need to go talk to people about this um, but it's it's it's a communicable problem amongst a very small number of Consumers that allow the business development process to occur Um, and we sort of all know one another that helps.

Um, so this is not very we're not an anonymous project So if you if you thought of stacking um some relationship relationship diagram or sort of two by two matrix of how how well people so Daniel, for instance as he Set on the program, right? Doesn't know most of knows nobody who uses his code.

Everybody uses it, but he doesn't know what they're using it for and how they're using it. Right. And we're the opposite end of that spectrum. We have a reasonably small number of. People who use the code seriously, and they also know why they're using it and they know who we are. So we can, we can forge business relationships in a much tighter manner.

So that's how we manage to run an, an, a system which is, um, which is working from a business perspective.

Jonathan: Yeah. What, what is the, what is the actual, um, like funding model? There's a couple of different ways I can imagine going about this. So do you, do you go to all of those white boxers that include your code and say, Hey, why don't you pay us 2 percent of your MSRP on everything you sell?

Because we're in there and you know, we need support. Or do you go to, you know, one of the companies and say the, the amount of, of support we're giving you, we think is worth a million dollars a year. Why don't you write us that check each year? How does that part of it work? I'm very curious about the nuts and bolts.

So

Alistair: it's it's certainly not on a per copy basis um So it's much more along the lines of the this is our support load this is how much it costs to run this infrastructure and um, and then just saying what a fair fair cost payment would be To those people and and some of it scales by the size of the entity in the organization um, and It's it's just a usual sales negotiation thing, uh, because the the quantum size the number of Number of parties that i'm having conversations with is as I said reasonably small.

It could be larger Uh, there is a group of people who still fall into the free rider You know user category that I can't get to or I don't know how to get to them in organizations um, but it's very much You You know, I basically tell them how much it's costing to do the job and then discussing, you know Whether they're a bigger or a smaller user than somebody else and and what does the pecking order look like?

Yeah, and again, this is a tractable problem because it's small right So those

Rob: aren't those, uh, fees or whatever you discuss with them, that's not backed by say a support contract. Is it just complete voluntary?

Alistair: Well, we do also provide support to, to companies who want support. We do that too, but we do get a, a set of donations, which are pure donations to run the CICD.

So we do both.

Jonathan: Yeah, that's interesting. Um, How, I guess the first thing that comes to mind is not every, not, that doesn't make sense for every project,

Alistair: right? Well, that's, that's my observation, right? We are, I think we are quite peculiar or special in that particular regard. The, the other class, similar class that looks like this is, is projects which were started by some very large, Entity that then get put into the public domain, uh, and still have large support from those entities, right?

So if you think about something like Kafka or any of those types of projects, um, they typically, um, Have a similar model where people understand how that's going to work and people continue to contribute. Um, as I said, I think we're relatively is that we organically grew around a type of thing. The code itself did not come out of a large entity as a, an open sourced project.

It, it organically grew but has been used because of networking's focus by larger entities. Um, so we certainly, Are at an opposite end of other some spectrum than say daniel right as he's definitely at the other end um, and and I think we're not typical so if I go and look at if I put my Erlang ecosystem hat on for a while, right?

That, that, that area encompasses multiple different, uh, projects. And so the, because there's a virtual machine underneath that, that is the Erlang virtual machine, which is run by People who want to program in Erlang, or Elixir, or Gleam, or any of the other languages that run on, um, the virtual machine.

There's a much larger tail of people doing stuff, from the very, from hobbyists that are just doing stuff, uh, in their own area, to the other end of the scale. Um, Ericsson and, uh, Facebook with WhatsApp. that use this at, at industrial scale, um, in networks, but we have a much longer tail of people and it's much more difficult to, there are a multiple projects that run on top of the VM.

So for instance, Phoenix, uh, you entered, you had a conversation with Lars recently about the nerves project. Um, so there's a whole bunch of different projects and they all have their own scaling. Systems and they're much broader from a scaling system. So it's much harder for us as a community to reach out to everybody to encourage them to support because it's diverse, right?

So we we have in that space, I think over 1000 commercial users of the technology, probably more. And most of them are dark Uh, and many of them are

Jonathan: free riders There's there's something that's sort of changing that I I am hopeful is going to help with the open source funding problem Um, and and that is there's been some laws passed and there's laws being passed Um, I don't remember the name.

I don't remember what project it was But I saw this statement where someone from a project was was saying We had a business reach out to us and say we need this documentation And for us to be, you know, to, to comply with such and such law. I think it was, was something related to a, you know, software bill of materials.

And it's like, so this, this company reached out to us and said, we need you to do this work for us. And the person running the project was like, I couldn't believe that they had the nerve to do that. And my take on that is no, no, no, no, no, no, no, no. You don't get it. That's a good thing. And your response needs to be.

I will do that for you. Here's my fee.

Alistair: Right. So, so you're entirely articulating the level of excitement that I had when I first saw the Cyber Resiliency Act over two years ago. So, um, most of the people When I first started engaging with the European Union Cyber Resiliency Act wanted to put their libertarian hats on and run away screaming and say, you know, keep, keep these bureaucrats out of my, you know, code base foo, leave me alone.

And I have,

Jonathan: I have a lot of sympathy for that point of view. I sometimes fall into that myself. I have one of those hats and I occasionally wear it.

Alistair: And, and I did it for like all of about three years. Three seconds before the penny dropped that's like, oh Oh This law so the interesting thing is not the law What it says you must do that's not the interesting bit.

There's a whole there's Hundreds of pages telling you what you must do and there will be much more Specifications written about you follow this standard and do these types of things from an implementation standpoint the interesting thing is what happens if you don't do these things and the The law sees that companies will be accountable for Not following business best Practices so you can be held, you know culpable as a company and there's even um One of the other european directives the product liability directive will see criminal Um law associated with some of these types of things if companies and their senior executives Do not follow the law So the interesting question is, oh, that allows me to talk to people about what they need to do to follow the law and the Um, I had a very useful conversation with the Apache Foundation, uh, again about 18, well actually almost 2 years ago on this very topic, is that their observation was that if you, if you just look at the law the way it's written, As you're a company, right?

You'd say to yourself, okay, fine. I've got 10 software devs in my company and I'm shipping something at the moment. And if I look at all the requirements, I'm now going to have to put in place to manage this software, then, um, I probably need 30 percent more resources to do the testing at the backend and the documentation and all this other type of stuff.

So how hard can that be? Right. Um, I just need to hire three people. But. Then your VP of engineering turns around and says, uh, actually 90 percent of our code is open sourced. Yes. And. The, the executive team sits around and goes, okay, so we have to hang on. We have 10 people, but they're only writing 10 percent of the code.

So that really means we've got a virtual engineering team of a hundred. So we now need to hire 30 people. To do the testing and verification That's where we don't want to do that, right? So that's why Everybody thinks that the government's interfering with their you know, their business plans But the obvious other solution to this problem is well Turn around and talk to the folks who are actually maintaining this stuff, right?

Um,

Jonathan: yeah, it it seems like a a real golden opportunity for a lot of open source projects, and I'm, I'm, I'm very thankful that when the law was written and then amended and they worked on it, um, they, they did go in and acknowledge that not everyone publishing source code. Is making a product right because from what I understand in one of the early, uh drafts of the law Um individual open source developers were going to be liable and that was terrifying and a terrible idea Um, so thankfully that got that got yeah,

Alistair: I don't think it was ever I think they clarified their position.

I don't think it was ever their intent to do that, right? I I do think that um that they So i've had the I I discussed I describe this as the Banksy, um, situation, right? So, if somebody just goes and posts something on a GitHub repo of the, you know, I've done the best bubble sort algorithm, you know, look at this and, you know, weep, right?

In terms of, you know, how efficient this is, right? And that's it, right? Everybody looks at it. It's a work of art, right? This is not a product. Product. This is a a political statement. This certainly should fall under the category of freedom of speech and um Fundamentally somebody does pure work like that.

It's just matter of algorithmic genius, so people will be able to do that and you know, I tend to I did talk to many people and they were they were throwing their toys out of the pram to start with and I said look They're not going after the banksy Situation, right? The interesting question is are you actually then maintaining it and do people use it?

Jonathan: And

Alistair: this gets to uh, toma de de pierre's Conversation about well, i'm not your supplier, right? And at some point Teams need to figure out whether they are a supplier And they just don't know their consumers or that they are not a supplier the law will create a bifurcation in thinking and um, you know in the In the conversations that you had recently, right?

I mean it would look like curl is already You know, in the situation of, of accepting and looking for maintenance, um, money, right? So, that has become, that's moved that side of the boundary line in terms of that. And I think a lot of projects are going to have to ask themselves, which side of the boundary line do we want to be on?

Yes. Yes. And the other problem is, is if you want to be on the You know the support In some way shape or form side of the maintenance line, right? How do people this gets to your other question is like, how do you how do you figure out how to pay? That right. So I think the ethos of the communities are most people Don't want to have a per unit License fee for maintenance that I mean that would go back to the you know, the the times before we had You know open source stuff, but people need to cover the costs of the maintenance and That's We need mechanisms of expressing what the costs of the project will be and what a reasonable apportionment is.

Uh, and I, I don't like the idea of just one benefactor turning up and saying, Oh, we'll take care of this cost, right? I think that's also bad from a relationship perspective. You actually want Um, major users who are using it commercially to actually be able to put their hand up and say, yes, I need to have a, I need to know something about these people, especially if they're going to have to fill in S bombs, figure out when CVS are going to get fixed and all these things.

And this is, this is all new territory at the moment. So, um, to Thomas point. The regulators want to see a tighter supply chain, um, by definition, open source and Open sources doesn't have that connection built into it. And we sort of, we need to introduce that reverse path into the system, which is an interesting problem.

Jonathan: Yeah. And you have other people that are working on this, right? So like, um, uh, post open source, Bruce Perens is working on the, the idea of post open source. And it's, it's intended to be sort of an addendum to all of the open source licenses that says you get to pay for your software. And his, his idea is let's make a central clearing house that handles this.

And I think it's really fascinating. I'm hoping to have him on the show to talk about it. Um, but I, I think there's been just sort of a awakening in a lot of different places to this idea that there's gotta be some money that flow back into these projects because developers need to eat. And we expect open source projects to have, um, quality standards.

Of commercial offerings, but and honestly a lot of times the quality standards open source projects are higher than commercial offerings But you find out, you know after a while, that's not necessarily sustainable unless somebody's paying the bills

Alistair: Well, well, it's it's that and more right? Um, the So so let's take the I I you know, I I'm not in it.

So I I spoke To daniel recently in a couple of months ago in stockholm on this topic, right? Um, it's not it's good that he's now in a situation where he has Uh, he has a sustainable business model and he can actually be paid to work on the thing that he enjoys working on Oh, yeah, right, but what happens if he gets hit by a bus, right?

We need In principle some a project like that should have You Couple of three apprentices working on, you know, a sustaining plan for when he decides he wants to, you know, hang up his clogs and do something else. Yeah, right. And there is, if you think about this, this is sort of in the Middle Ages, they had better plans for these types of things.

Yeah. You know, the, the, the famous artists would have people in their studios who were helping them, you know, and learning to do stuff. And when they died, right, the, the guys took over and it took us a long time to even be able to figure out that some of the works of art were done by the apprentice, right?

And, and we need a system like that for, uh, You know, artisanal software, which is of the highest quality, right? I mean, it really is, right? Daniel's Daniel stuff is the highest quality. Yes. And so there are systemically. At least a thousand projects, possibly 10, 000 that should have that same level of coverage, right?

I mean, the NTP keeps coming up as the topic, right? Um, I'm pretty sure that that's the one that's the, you know, the one maintainer in, you know, um, that's exactly right. So, so you've got all these things that if they were not managed and, and what's the succession plan? Nobody's thinking about this stuff and we need to figure out how to make this happen Because otherwise something's terrible is going to happen and then we end up with these Oh, let's just throw a large amount of money at it from the linux foundation.

It's like We could Plan better than this, people, right? We could have, we don't need to have, you know, Y2K bug problems just randomly turn up on a Friday because somebody got hit by a bug. Bus, right? I mean, we can work better than this.

Rob: Yeah, and that's a problem. That's a problem just all over the place in commercial endeavors where you have one person who knows or does everything and you don't know what tomorrow's gonna bring with that one person.

Yep. Yep. Right. Well, which

Alistair: is one of the reasons that industry has said, well, this is why we don't want to use open source. But then, unbeknownst to them, from the executive perspective, their engineering teams have been, you know, borrowing the code because it got them away, got them to their finish line faster and better by doing that, right?

So they've inherited this problem, but they don't want to face the fact that they've had all this benefit and what do you do about it? Yeah.

Rob: And that really could be quite the contrary, where in a small if you have one or two people doing the development and something happens to them.

Jonathan: Yeah.

Rob: You know, you've got to start from scratch.

Whereas open source, you have people at least looking at someone who can probably step in.

Alistair: Well, yeah probably step in but let's be let's be real about this right what we would really like Is somebody people who were actually wing wing people? On a regular basis and that they were actually capable and knew how to step in right?

And and then you can avoid The sort of xz exploit problem, right? Because the the we're talking about The same thing that whoever came up with the cunning plan Of doing xz looked at the way open source works and go there's a poor maintainer over there who needs help So guess what? We'll provide him with the help and this time it was with malicious hats on right?

But why don't we look at this and go well? That's a systemic problem. We should be providing the help so that there's, there's a general, I mean, doctors, for instance, when they go on holiday, they have backup. Other doctors that you can call. My dentist says, you know, call this other person, and they have access to my records, right?

Other industries have figured these types of things out, and we haven't.

Rob: You know, in your notes, you, uh, you mentioned abandon, abandon wear and, you know, even myself, I, I, I don't do a lot of programming these days, but I used to do quite a bit, a little bit of, uh, open source and, and what I found personally, I mean, I never grew to the heights of really anybody, but I'd start to build something.

I'd put it out there in the hopes that, okay, this is going to get some attraction. Some people will come along and help me. Um, maybe I'll get some, some other contributors to it, and then I just Get tired and burned out because I'm doing it myself and it's like well Nobody's interested and I am banning it myself So I mean

Jonathan: so it's interesting to me you have some of these projects where people just that Sounds like fr routing was one of these to go back to that as an example It starts as a single person sort of a labor of love And you started throwing code together.

And, you know, Rob says he's tried this a couple of times and never, never gotten anywhere with it. But what happens when you do get to like critical mass and there are some people out there to depend upon your code? I guess the real question that so many of us have is how do you go from that single person project where nobody really cares and it's okay if you get hit by a bus to there are people out there depending upon it and we suddenly have a budget, how does a How does a project get from, from that point to the point of having a budget and having sponsors and is there, is there a blueprint out there for this?

Alistair: Well, at the moment, it's entirely luck, right? And a lot of that luck is based on, as I mentioned earlier, the circumstance of the connectivity between the producers and the consumers of the code, right? And whether they know one another. If you end up in these situations where there is Coupling between things like people are using.

So in the, the Erlang space, right? We have a web web server on, on Elixir called Phoenix, which if you just want to do live view, Phoenix stuff. It's it's excessively popular with people who want to use web services stuff, right? But if you're just doing web content, you don't really need to talk to the guys who are keeping the html Um machinery working underneath you use a bunch of features you do turn up right?

but in principle you're You're writing, you know, your web service stuff on top. So there are a lot of web servers that have been run by, uh, very large companies that are on top of the Phoenix framework stuff, but their connectivity, I would guess less than 10 percent of those users are actually reaching back into the community other than, you know, to hang around a mailing lists and figure out how to get stuff done and, you know, Bitch about stuff, but they're not actually there is no formal way of setting up that relationship um, you know, so it's it's a difficult problem and and I think This gets back to the the topic of if you do now need to verify your supply chain It's it creates a bit of a magnetic field on the businesses that they want to start to have that conversation

Jonathan: So the gpl specifically I think most licenses have this but I know the gpl does i'm familiar with its text It has a big section in there.

This code includes no warranty, you know express or written Or is express or implied it's the term it uses And so it's, it's very expressly part of the GPL that you do not get a warranty. The, I am not your supplier is basically built into the GPL. I do not take any liability for this code. Um, do we just need an addendum that some open source projects can add that sits sort of on top of the GPL that says, if you need a warranty, if you need me to be your supplier, there is going to be a cost involved and here's how we handle that.

Alistair: So that's a way of solving the problem right sort of the essentially the dual license Model and that might be the way that folks decide to go. I don't think it has to be the only way That that you could do that, right? I mean you can think about things in a very different manner of of Um, showing support and sponsorship or any of those other types of things.

So, it doesn't have to be that way, right? It could be that way, and I'm relatively neutral on that topic. Um, so, What what is obvious is that we need a better way than the one we have at the moment, right? That's clearly not working, right? And I think you know getting people to sit down and talk about What the appropriate ways of making things work would be better, right?

Um You know, I, I, the, it, it amazes me that, that we are at this particular stage that, you know, there's so much dependency on this technology from a societal perspective, and that we as communities haven't quite figured out what the appropriate, uh, ways of, of, Running this are now. I'm not saying that the conversations are not happening, right?

Because I've been on many of these calls You know with the folks in European Union spaces as well as folks in the u. s Domain, and I think things are move that the the pressure is there To reach solutions. What's not clear is that I think we as Community activists are not actually sitting down and saying okay Well, there are four or five different ways of doing this.

Well, how do we think this could work? Right. Sometimes we need to actually talk to industry itself and ask them how they want to play. Yeah

Jonathan: Oh, yeah. So I've heard, I've heard stories of like a big business wants to sponsor an open source project and they're like, but we need you to send us an invoice and the open source project goes, we don't have any kind of business things set up.

We can't even send you an invoice and the business walks away and it's just like, well, we wanted to support them, but I guess we can't. Yes.

Alistair: Yeah. So, so there is, um, there's certainly that, and this leads to these perverse situations that either a project gets over. Supported by some company that thinks it's being like beneficial and in fact, we could talk about the you know So it's it's great when those things happen, right?

but the We need something that's going to work generically across scales and works for small and medium business customer companies that are consuming software and works for larger business and and and I think tomodipierre was very free He made a very cogent point about one of the reasons that open source has become So successful is that it's a one way graph of People downstream of you do not know What's upstream from a commercial perspective?

Yeah, right So the business, you know a the vp of engineering, you know doesn't even tell The CFO that they're bringing in a, you know, a project and nobody even knows all the branches and twigs that are associated with that project when they bring it in, right? Which, uh, led to the log4j surprise for everybody, which is like, oh, but we don't have that in our, oh shit we do.

Right? So, so, so that. We need and and this is what I think is certainly happening about the supply chain software bill of materials Technology. So all the the work that's going in there about generating s bombs will allow people to identify what that back path Looks like right and then the interesting question is like, okay fine.

Now, we know what you've got in there How what's an appropriate way? You Of providing funding down that path, right? So for instance, if i'm a consumer of a project should I also be somehow trying to make a micropayment to the log4j guys or Should the project that's using the log4j thing be making a larger donation and said Where is it turtles all the way down?

Yeah, right or and and Those are the types of questions Questions that I don't think we're we're sensibly asking ourselves because the solution to many of these things is oh Let's just throw a lot of money at the log 4j guys and that problem went away and it's like well No, you just you avoided Making the connectivity work and you've just fixed.

That's a point. It's a diving catch Solution which is not very good

Jonathan: Yeah, and I know there are groups out trying to trying to work on these problems like we years ago We interviewed the guys from Tidelift And so they were kind of, they were kind of way ahead of this. Um, we need to have them back on, because I'm very curious to hear how that has gone for them, whether they've had continued success or not.

Um, so I've got to ask, I've got to ask, this is kind of a, maybe a weird question, but I think you'll understand where I'm coming from. As, as we try, open source, historically, obviously, has been lightning in a bottle. Right? It has just been wildly successful. It has made things possible that were impossible before.

It has literally changed the world. As we try to sort of solve these problems and come up with blueprints for open source projects to solve them, do you think we have a danger that we, uh, we let the lightning out of the bottle? That we change open source in such a way that it becomes less impactful than it was?

Alistair: No, I th I

Jonathan: th So,

Alistair: I think if you

There are ways that you could Over engineer it and break the system, right? But on balance, it's working. It's working incredibly well in the downstream mechanism. The thing that isn't working. Is we're not getting the nutrients back up in the upstream mechanism and we don't have, you know, these longer to, you know, who's the, who's the apprentice for the master, right?

Who, what's the replacement, the backup strategy, those types of things are not happening and they, there needs to be a structural mechanism and making that happening. And to your point, this requires. organization on behalf of the, the open source projects. So good example is that in the EF, um, we've, we're in the process at the moment of applying for a CNA.

Um, and you, I think talk to Daniel at length about the CVE problem associated with this. So, um, I had the same conversation with him as well. And, um, it was pretty, it'd been pretty obvious to me for the last 12 months that we, as a, uh, an ecosystem wanted to have. Uh, you know, uh, autonomy over CVE, um, registration stuff just to avoid some of this frivolous stuff and be able to understand what was going on.

Right? So, um, I've articulated this as running your, you know, local fire station. Right. I mean, you need to have a volunteer fire station and you, and we don't want to avoid, we want to be able to manage that process and you need to set up your resources. That means we're actually hiring somebody to, to do this full time.

It means that we also need people to support this and we need to be able to talk to our community about why a fire station is necessary. So it's an interesting problem about how. Open source projects are going to be able to manage that back office infrastructure, right? And to your point, if somebody doesn't know how to write an invoice to somebody, right?

They probably need to bulk up. So, maybe we need group collectives. Maybe we need things that look like the mini Apache Foundations. Because, I mean, the Apache Foundation doesn't want to take on, um, lots of small stuff, right? But, you know, Um, that they take on big substantive projects, and I think they're doing a great job, but we need something that manages and looks like that for smaller projects.

Fortunately, we can do that for our language ecosystem, and I think you're seeing that happening in the other language ecosystems, like the Ruby project, the Rust project, they're all focused on doing things in their particular realm. Um, you know, there's, there's a whole bunch of stuff that, that needs to understand that it needs to be even thinking like this, right?

Jonathan: Yeah. Uh, is there, is there a resource somewhere that, uh, if someone is part of an open source project and they're kind of stuck in the middle of this, they're, they feel like the project has gotten too big for its own good and they've got to make a change. Is, is there someplace that you would recommend people go to, to kind of start learning about the options or to get help?

Alistair: Ooh, that's a very good question. Um. So actually that, uh, so coming up in my future is I'm off to FOSDEM, um, at the beginning of, um, the first weekend in February. Um, there are a lot of folks out there. I mean, one of the reasons talking to you guys, I think from this perspective is, uh, Is um, a lot of my knowledge about what people think and feel at the moment comes around about listening to podcasts So we are communicating over radio um Related to you know, what what people think and care about so I don't know I mean, maybe you want to put your hand up and and say i'm certainly happy to take redirections, uh Anytime you want to send them my way, but um, yeah Not clear who's trying to work on this at a total global scale.

I think it's probably too big for one person to work on. Yes.

Jonathan: Yes. Um, I, I do know of a couple of places that are, that are sort of thinking about this and trying to help. Uh, the Open Source Collective is one of them. I believe they're based in the United States. Uh, our friend Simon Phipps helps with a couple of companies, or a couple of projects, excuse me, and trying to, to Solve some of these same problems.

Um, and there, there are people out there, but I, I, I can't help but think that it would be useful. And, uh, I'm sure people are working on this, but it would be useful to have a blueprint where. You know, maybe, maybe we publish it as its own open source project. And it just, here, here's the documentation.

Here's what we suggest you put on your GitHub repository. Um, here's the way that you, you form a company to be able to take donations or, or what have you. It just seems like there, it would be useful to have a, a blueprint document out there for people to try to bootstrap from. I'm a single person. Open source project, but people depend upon me too.

Okay. Now I'm a healthy open source project with a succession plan, all of that.

Alistair: Uh, I couldn't agree with you more. Um, I've been on a couple of calls with Simon, um, possibly earlier on this year related to the EU cyber resiliency act and, um, um, and that there are obviously folks in some of the larger foundations that are, that are worried about.

This class of problem everywhere from the Linux Foundation to the Apache Foundation to the Eclipse Foundation. Um, what's, what is missing at the moment is something that scales. Across all the domains and, and, and to a great extent, even thinks about the really hard problems of the apprentice ship and survival.

So that's my usual test question. When somebody said, Oh yes, we're thinking about these things, right. Related to the, you know, cyber resiliency act. Then I say, okay, so if you've thought about them, what's the plan for that? And everybody goes, Oh, we hadn't thought about that bit. So, I mean, the, the, the, We sort of need to have a sort of discussion about how ambitious We want this to be and how you would think about stuff, right?

I mean at one level um You know other fields of endeavor have fixed this right? So they have things like the fields medal and they have whatever it is, right? I could imagine Somewhere where you actually decided and said, okay, we view these things as systemically interesting. We're not going to give um You Say a boatload of cash to um the curl project because They might just all then goof off and you know buy a boat and you know, hang out in the stockholm harbor Um, but we will we will pay You know the apprenticeship salary for three or four years or something like that, right?

There is there needs to be something that's going to go work on that level of sustaining. Yeah um So Yeah, don't know yeah interesting stuff

Jonathan: All right, we could go on for I think another hour chatting about all of this it is super fascinating Rob Is there anything you want to ask before we before we let Alistair go?

Rob: Um, no, I don't have any final questions. I

Jonathan: want to make sure and let you get, let you get the last word in if you want to do. Uh, Alistair, thank you so much for being here. The hour is just absolutely flown by and we will have to have you back here before too long because again, there's so much more to cover here and more thoughts to be had.

Um, but for now, I just want to say thanks. Thank you for coming and talking about FR routing. And then also this, obviously your passion is about having healthy open source projects. Yep, my pleasure. Before I let you go, I do have to ask two final questions. Uh, and then I guess I'll get a third one in too.

Um, so the, the, the third one that we usually ask, is there anything that you wanted to cover that we didn't ask you about?

Alistair: I think that, that, well actually, that, that you, you, that the one shout out that I would put out there is for the, you know, the first robotics, uh, programs. So, um, uh, I'm, Whole of january is going to be lined up doing uh, both Judging as well as inspector management for the local competitions in the bay area And for those people who haven't looked into first robotics and what they're doing.

It is absolutely wonderful um you You hear so many people especially my generation, right? But maybe even your generation like to complain about the kids are today And whatever it is, and i'm sure that there are problems With some of the kids are today, but the self selecting subgroup that actually turn up to these robotic competitions are absolutely wonderful.

And so it's a pleasure to be engaged in that activity.

Jonathan: Yeah, this is things like battle bots and line following robots and just all of that?

Alistair: Yeah, they don't. So the interesting thing about first is that they don't battle it's um, There is a competition. There is a scoring. It's a bit like doubles tennis that you play with other team members and it really Uh, it's not a zero sum game there.

There there is a Point system and you're trying to beat the other team, but it's not at their At the disadvantage of the others and there's you know, highly refereed and curated type of stuff and it it's a very interesting gaming theory uh process that whoever thought that up i'd like to go and you know have dinner with them to figure out whether it Came out as whole cloth or whether it took them several years to figure out how to make this process work.

So well, but um, But yeah, it they they're always striving to do some particular thing right as opposed to beat the other robot so so it's uh That

Rob: sounds similar to uh, we have a vex robotics in in our area

Alistair: Yes, it, it, yes, same thing, yes. Very cool. Yep.

Jonathan: Alright, and then Alistair, I've got to ask, what is, and this, this may be a historical question for you, but your favorite text editor and scripting language?

Alistair: Um, so certainly, uh, text editor and stuff, I would use nano, um, and I think the argument's been made before that it's just so ubiquitous. And I think that's, that's a good answer. Certainly if I want to get something done and I'll use bash if I have to do scripting stuff. Sure. Sure.

Jonathan: All right. Well, again, thank you so much for being here and, uh, we, we sure appreciate it.

Rob: Okay. Talk to you.

Jonathan: All right. Rob, what do you think?

Rob: Well, I definitely approve of his, uh, last two, uh, answers, nano and batch. But, uh, as for, you know, the, the open source projects he's involved in and FRR, um, routing, I was really surprised how ubiquitous that was. And definitely, uh, you know, in agreement with all the, the The needs we have to fund and, and have apprenticeships for open source and all that.

Jonathan: Yeah. So he, he threw out the numbers, uh, a thousand mission critical open source projects, maybe as many as 10, 000. I think 10, 000 is a low ball estimate. Of the number of open source projects that are mission critical to various things Um, they're probably there may be 10 times that many that if any one of them were to go down it would be It would well it'd be the left pad situation right when when the guy behind the left pad Uh did javascript like library, you know all one line of it Deleted it in protest and half of the internet broke.

I mean there are so many of those even In

Rob: one large piece of software they're Yeah, maybe I'm stretching it, but I was going to say there could be a thousand different little open source projects. It's possible, depending upon

Jonathan: what language it is. If you, if you expand that and say in a single server, there's going to be at least a thousand open source projects.

You probably have at least that many installed binaries. Um, yeah, there's a, there's a lot of them out there and, uh, it is, uh, it's quite a, Problem, but also an opportunity to really think through some of these things about how do you What does it look like for a project to be healthy? And then how do you get to that point?

Definitely interesting. We're going to be talking about more of this sort of thing next week as we have uh, Matija Silke Of FOSS, well, it's about FOSS legal. We're going to be talking about free and open source software, legal issues. Um, and he is involved with a couple of different organizations where I believe open source projects can reach out to and have a legal representation, get opinions and such as needed.

And I am super interested in that because one of the projects that I'm involved with, we have legal concerns because we occasionally deal with people's personal, uh, inter. Personal, identifiable information. And, uh, that is, that is sort of a hairy issue these days. So I'm gonna try to pick his brain next week.

Not sure who's gonna be with me as a co host. I need to figure that out. Um, but we will be back next week on the 7th to talk about that. Legal issues and open source. Be a lot of fun. Uh, Rob, is there anything that you want to plug before we let everybody go?

Rob: Well, you guys can catch me on the Untitled Lenox Show with Jonathan Bennett.

Yeah, and if you want to connect with me directly, you could find my website, Robert p camp bell.com, and there'll be links there to, uh, various social medias and things and information about me.

Jonathan: Yep. Great. I appreciate you being here. All right. As Rob said, you've got my work over at twit tv, the Untitled Linux Show, and then there's also Hack a Day.

Where you can find the security column goes live every Friday morning when it's not a holiday or I'm not sick Which was a very interesting past week or two for me Um, but we're back. We're gonna be back at the grind for a while at least Um, and yeah, make sure to check that out I've also got a youtube channel where you find a smorgasbord of things talk about meshtastic talk about modular synthesis That video did well, so we're gonna have to do more of that.

Uh, that's uh, that's you can just You search for me, Jonathan Bennett, on YouTube and find that. Um, but anyway, we will, uh, we'll be back next week with more Foss and Floss goodness. We'll see you then. We appreciate everybody that listens, that watches both live and on the download, and we'll see you next week on Floss Weekly.

This week Jonathan and Rob talk with Alistair Woodman about FRRouting, the Internet routing suite that helps make all this possible. But also business, and how an open source project turns the corner into a successful way to support programmers.

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

This week and next we take off for the holidays! We have an exciting schedule after the break!

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week, Simon Phipps and Aaron Newcomb joined me and we talk about an interesting smattering of topics. We start with talking about legacy computing and retro hardware and how open source still makes that work. And then end up with politics. It's actually a lot of fun and you don't want to miss it.

So stay tuned. This is Floss Weekly episode 813 recorded Tuesday, December the 10th. Turn off the internet. It's time for Floss Weekly. That's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and sometimes we talk about a little bit of hardware here. And today is interesting because I was absolutely certain that we had a guest schedule to talk about the open source AI definition, and I sent an email yesterday, just checking up to make sure that everything was good.

And I got sent back, you never confirmed that we were going to be here. So we don't have it on our calendars. And I went, Oh no, yeah, that's right. That's one. So. Thankfully we have a couple of willing volunteers. Simon and Aaron jumped in sort of at the last moment and they're here and we're going to talk about a couple of things.

First off. Welcome guys. Thank you so much for being here.

Aaron: Absolutely. It's all your fault, Jonathan. It is.

Jonathan: It's all my fault. So this, this was basically the crew that was supposed to be here. We were just supposed to have one more, sort of another expert in the field of open source AI to talk about, for one thing, what that even means.

And we may get into that a little bit here towards the end of the conversation. But I've got, I've got here two experts in retrocomputing. For two very different reasons. ,

Aaron: well, I don't know, is, I don't know that they're that different. I mean experts because we lived through it. Is that what you're getting at?

Because we're so old 1, 1,

Jonathan: 1 being an expert because he lived through it and the other being an expert because you've got the YouTube channel on it. Yes. That was sort of what I was getting at. Okay. Yeah. Okay.

Simon: You know, I have to say that I, I, the, one of the things that makes you feel old is going to the Computer History Museum and walking around and going.

Hey, that's my computer.

Aaron: Yeah, I worked

Simon: on that. Yes. And I, I, my, my SWH 68, 000 is indeed in the computer history museum. So you know, that's retro computing. Yeah. We, we call that a live systems production. That's cool. Yeah,

Aaron: I get asked, I get asked a lot by the younger crowd, like, you know, the question always starts with back in your day.

I was like, all right, were Atari joysticks really like this when you, yeah. Okay. Yeah, I get it now.

Jonathan: I'm old. Yeah. Yeah. I'm starting to feel that too. I had somebody the other day. We've got some really cool retro computers. You should come and take them, take a look at it. And you know, I'm expecting a certain era of machines and I get in there and I'm like, yeah, here, it's this Windows 95 machine.

I'm like, okay,

Aaron: That's not right. I try not to be too. I try not to be too. Yeah, because there is starting to be That nostalgia for things even out of the 2000s Now, you know and it's like well, yeah, it was 20 years ago. So I I get it. I it's not my thing, but I understand, you know So that's what I usually tell people.

It's about a 20 year window, right, where people start to feel nostalgic. Until then, it's just old crap, you know, and then all of a sudden it's like, oh, I remember how long that took to do.

Simon: Yeah. Yeah. So one of the curious consequences of always assuming that Microsoft was going to audit me I've, I've kept all of the original media for, I have it, I have it for Windows 3.

I have Windows for workgroups, I have Word version 1, I have all the original media and original packaging all down in the office, so I'm waiting for a museum to call, or I need an auction house to give me a call.

Jonathan: So one of the, one of the things that I have seen both on Aaron's channel on RetroHackShack and some other people's channels is that we're starting to see like open source, both software and hardware get applied to some of these old retro computing things.

And like just a couple of things that come to mind. I've seen, I've seen people take like Commodore 64s. I am the age that I consider a Commodore 64 to be retro. Simon, that's probably. That's probably just old to you, right? ?

Simon: Yeah. I, so I never, I never had a Commodore 64. I, I, I worked on Sinclair qls.

Mm-hmm. They were, they were my, my thing. 'cause they had built in networking and everything. But yeah. That's good. You know, I've got friends who've, who've got those on a shelf somewhere.

Aaron: Yeah. And the, the QL was the one with that little tiny tape drive, right? Yeah. With

Simon: the micro drive. Oh yeah, yeah.

Aaron: The micro drive.

Yeah. Yeah.

Simon: But I, so I actually used to run a network of them for an office. And because the QL had had built in networking in the hardware. So you just ran a cable from QL to QL, and they could all share each other's drives and resources. And so I added a hard drive on, and so that we effectively had a hard drive in the cluster.

And you add a printer on, and everyone can access that. It was actually really great computing. It was much easier than anything I ever encountered subsequently on PCs.

Aaron: And I think that, I think that one thing that lends itself to open source in retrocomputing is there was always an element of, even though they didn't call it open source back then, but certainly hacking right on these things.

And those became projects that were published in a magazine that anybody could pick up and do. So there, there is some, for me anyway, there is some root, some kernel that was going on back in the seventies and early eighties. That was akin to open source to an open source project where someone would come up with something It would get published in bite or it would get published in popular electronics or something like that and maybe they got paid for their article, but the point was they were sharing that information Yeah with the community and anybody could pick that up and do whatever they want with it if you go back all the way to The TV typewriter, for example, and different people will say, Oh, the TV typewriter, I think like Waz, for example, said the TV typewriter wasn't an influence in the Apple one, but it was, there was some, some sort of DNA there because what was used in the Apple one, which led to the Apple two, which led to.

Apple, as we know it today used all the same elements that were in Don Lancaster's TV typewriter. There was some memory there. There was a character generator. There was a keyboard. There was the way that it interacted with the rest of the circuitry was there. And that was published in a, in a, you know, one of those early magazines.

So I really do think that there's some open source DNA in there somewhere to the whole. Computing back when it was actually happening in the seventies and eighties.

Jonathan: Well, so something, something important on that, on that note, something important to keep in mind is that when, when, when RMS came along and came up with free software, he wasn't inventing the idea of let's make software freely available.

Like he was, he was in fact saying, this is the way it was. And this is the way it should still be and then they came up with the licensing hack that is the gpl And it's the same thing with open source when when the open source definition first was was kind of developed They were not they were not inventing open source.

They were looking at this is this is what people have been doing This is what's worked and let's try to codify that. So yeah It's, it is a much longer tradition than, you know, just the, even the free software foundation or the, the, even just the OSI, it has existed longer than that. It's just, those are, those are the groups that we sort of have to thank for, for taking all of this nebulous idea of let's share the code, let's share the information and sort of distilling it down into, let's use this as a definition for that.

Simon: Yeah, I mean, it all comes from a real long time ago. I've been working with somebody on a short book recently, and you'll likely love it, you know, because he's written a book about why open source is capitalist, and he started out by talking about how Stallman invented free software, and then he looked back at history and discovered that Stallman was really quite late to the game.

Yeah.

the folks over it to Berkeley University were there ahead of him. Were it's just, they, they had a different way of expressing their expectation. Their expectation was that this was before the IBM consent decree. And so it was before. The productization of software. And so there was no need for a GPL because there was no value to software.

People just gave it away. And but the bill joy was there creating BSD and then going on and creating the world's first open source software company, which was some microsystems and people look, they, they, that's so far back that people assume that it all happened in the nineties, you know,

Jonathan: It's interesting. You mentioned that book, you know, Steve Levy's book, Hacker's Heroes of the Computer Revolution. That was sort of the introduction for, at least for me. And I think for a lot of us to some of this history and he makes the exact same point, but with Stallman and I think it was the, the, the hacker club at MIT where they were playing with the PDP machines and Stallman came again.

He was late to that game. He was towards the end of it and he saw things. Changing away from the, let's just all share this code so that we can all make it better to again, this idea of productizing it and the free software foundation and the GPL itself was sort of a response to that. And so, yeah, it's super interesting to see the, you can sort of trace that line down through history to where we are today.

Aaron: Yeah, so the, so the, so the nuts and bolts have been there for a long time. But now it's kind of come full circle where, you know, people are creating these, these cool hardware and software projects that can then bring these retro computers and give them more functionality or make it easier to do things for people that, that maybe aren't you know, that don't want to do it on their own.

And there's some great examples of that. Of course you know, the one that comes first to my mind is RGB. Yeah. To HDMI which is an open source project. And I sell some of those boards on my site, shameless plug. But you can also, it's an open source project. So you can go get, I had a guy this morning that emailed me saying, are there DIY versions of this thing that you sell that I can just go do on my own?

And of course the answer is yes, you can. But yeah, this project allows you to take an old. Sinclair QL, for example, or TRS 80 or whatever, you know, computer you, you want to use that had some of these older protocols and allows you to hook them up to your modern HDMI. Monitor. So you take the output, convert it to HTMI and, you know, cause a lot of these old monitors aren't available or are very expensive.

I mean, try finding a IBM 5154 EGA monitor these days, it's going to cost you three or 400 bucks and hopefully it's working. It probably isn't, or probably won't be for long.

Jonathan: Or it's dim, or, yeah.

Aaron: Yeah, and not everybody can fix those. They're awfully hard to work on. So yeah. So that's a great example that, that first comes to mind is, is that project because it works with so many retro computers and is so accessible for people.

But there are also a lot of projects that are coming up right now around the Raspberry Pi Pico. So, and of course there's tons of things going on, but, but especially in the retro community, people are taking the Raspberry Pi Pico and making it do things inside retro computers as well. So there's two projects there that come to mind.

One is the PicoMem. Which adds memory for older IBM computers with a ISA slot. You can put this in there and the Raspberry Pi Pico will emulate the memory and give you, you know, two, four megs even. Can you believe it? Of memory for your old IBM computer, right? And what people found, I think, partly based on that product and based on other products, once they had the ISA bus figured out, is that they could do all sorts of other things.

So there's another project out there called the PicoGus and that project has gotten a lot of momentum lately, which actually takes the Raspberry Pi Pico and emulates a Gravis ultrasound. That's what it is. I was trying to think, what does G U S stand for? Gravis ultrasound, again, incredibly hard to find these days.

Not very many people had them cause they were expensive when they came out. But you know, this is an a sound card that was that worked with a lot of games. You added it onto your Sound Blaster and it did incredible, incredible things, but they're super hard to find these days, but it'll emulate a Gravis ultrasound.

Or a sound blaster or a a Roland system or a Tandy three voice system. It'll do all of that just based on this little, I don't know what they cost these days, 4, 5, a little open source or open hardware microcontroller. That's, that's cheap and easy to use. So there really is a lot of, a lot of great open source projects out there.

And I, I, I think it speaks to the legacy of open source. The fact that open source is established so much now that when people create these projects, the software that they create. Of course, it's going to be open. Of course, I'm going to give it a GPL license or whatever. It's not like a question where you have to convince people, Oh, can you please, like, just make that open source?

No, it's just like by default, right? They're choosing to make this open source and it's for the betterment of the community.

Jonathan: Yeah, every once in a while I come across a project where it's like somebody obviously, like they'll put it on GitHub and everything. They obviously intend for people to just grab it and use it, but they won't put a license on it.

Aaron: Yeah,

Jonathan: like you you realize that you're all rights reserved by default, right? Please pick one of the open source licenses I don't care which one you pick just one of the OSI approved licenses and just to add the file like here I'll even make the pull request for you. Just accept the pull request put a file on there.

That way we can use it Yeah,

Aaron: yeah. Yeah for people that don't know I think github does a pretty good job though of pushing people into choosing a license when you start a project Which is good. They try to at least Yeah Yeah. Yeah. Now , so there's a lot of there's a, there's a lot of great projects. One last project I'll, I'll mention is one that I just found and I'm super excited about it.

It's called The HID Man. Hid Man. Mm-hmm . And you might know HID from USB HID devices. You know that that's what makes USB so ubiquitous and easy to use. You can kind of like plug. A USB mouse into anything and it just works. And so there's a guy, he goes by the name of Rastiri on GitHub, but he he also has a YouTube channel and he's been working on this device that takes.

USB devices and converts them into either PS2 compatible or AT or even XT compatible protocols so that you can take, if you don't have one of these old keyboards or whatever, or you just want to use, like in my case, I have a fancy new mechanical keyboard that sounds wonderful. I love to type on it. But I want to use that on my retro devices.

But it's a USB keyboard. You can now take that plug it into this device and then plug from that device into your computer and use, you know, a mouse or, or a keyboard on your, your retro computer. And it's just a little tiny device that does a wonderful job. He made it open source. And I talked to him about it.

I'm going to be carrying the hardware portion of that on my, the hardware is all open source as well, but I'm going to be going ahead and making some of those and putting them on my shop as well. Because. I think it's just a, it's just a great, it does such a great job. One of the things I love about it is the, in order to do the configuration, because he has in, in the software, he has control over the PS2 keyboard.

Let's say you're using, when you want to change the configuration, you open up any text editor. Maybe it's just edit in DOS, right? And you push a button on the device and it prints out in the editor, what the configuration is. So instead of having to open a file or go in and change something, you know, it just prints out the configuration and it says, what do you want to change?

And you hit one. And of course he can interpret that he knows you hit one. So then he says, okay, I'll change that configuration to this. You know, this is like a brilliant in my mind is something that would have been done back in the early days. And. He's using this to actually allow people to change the configuration on this device.

And I was like, that's, that's cool. That's cool. And it's brilliant. And I love it. It's

Jonathan: kind of like using a teletype machine almost. Yeah, exactly.

Aaron: Exactly. That's fine. So

Jonathan: one of the things that fascinates me about this, and I've got the same bug and it's this idea of, of so someone, someone might ask.

Why don't you just emulate? Right? Like, so if you, if you want to play with a Commodore 64, well, there's Commodore 64 emulators. Why bother with the real hardware? Why bother with any of these adapters? You know, why would you use a, a machine that actually has an AT keyboard or an XT keyboard plug? Why not just emulate it all?

And, I don't know that I know the exact answer, but there's just something special about running the real hardware. And, and, Why, why is that?

Aaron: Yeah, there is something about, about running on actual hardware. It's, it's a great test bed. I use emulators to test stuff all the time. Testing code. If you're writing code, for example, for some of these old things, it's, it's a lot easier to pull out an emulator and just do it that way.

But even what I find interesting is even the younger crowd, right? Kids to me, anybody younger than 20 still has that same response to the actual hardware, you know, and you would think they would be like, I'm just going to run this on my steam deck or on my phone or something, but no, when you. When you give them the actual hardware, it like something clicks in their brain.

It's like, Oh, it's tactile. I can use it. I see how that worked. I understand like how the cartridges work and how the joysticks works and how that all came together. In, in a special way, I think because all of those systems at the time were, you know, there wasn't a lot of standards back then, you mentioned the Commodore 64, you know, that was as different from an IBM PC as it could be.

Right. But. you know, learning how the systems work and how the hard drives work. And you had to kind of be your own expert on some of those things. I think there is just an appeal to people that like to learn about old systems of having the physical cartridge, the physical hardware to play with, seeing how the keyboard felt in those days, how it was either bushy or clicky, or, you know, there's just something, something about that tactile response to those systems.

I mean, Simon, you mentioned those, the micro drive, right? Yep. For the Sinclair QL or the keyboard on the Sinclair QL, I know was quite a bit different. Of course, it wasn't the the, the old plasticky one of the original Sinclair, you know, ZX Spectrum, but

Simon: the membrane keyboard, only one keyboard. And so with the key, with the, the space on the top of the case with the keys attached to the, the the, the printed circuit board that was underneath, I mean, the whole thing was a single, was a single board computer.

And that's obviously a desirable thing because I, you know, Raspberry Pi are coming out with the Pi 500 at the moment, which is just the same. And it looks just the same. Just like what I remember my Sinclair QL looking. I mean, it's white, but it's very similar kind of experience and they expose all the ports on the back.

The GPIO pins are all out there. I think that one of the things that makes it compelling. Knowing that you can

yeah,

I think that the the difficulty with with lots of stuff, you know I've got it. I've got a device here that I picked up from a well known Scandinavian furniture shop that's for controlling my Sonos speakers And and it's it's all sealed And I can't do anything to it.

And if I take it apart, what's inside is sealed and the box that it's controlling over there is sealed and I can't mess with it. And all this stuff is stuff you can, you can get your hands on and not because anyone who made it intended you to do that, but because the people who wanted to stop you doing it are all dead now.

And there is that sense of, I've got this. This, you know, ancestral artifact here and I, and I can, I can touch it and I can, and nobody is compelling me to do things in a particular way and there's problems to solve, but I'm not going to find myself getting a cease and desist letter from Sony for doing it.

And, and I think that part of that is what's compelling, and I think it's, that's the same thing that's compelling about Raspberry Pi. It, it is that you can, you know, it's got all the GPI pins are, are on the back. You, you know, you can take it apart. They sell the whole thing as a chip you can put on your own circuit board, and people do.

And I think that's, that's always been, you know, the things that have always driven me in computing are the fact that I can and also the idea that I can do things here that make something happen over there. Those have been the two things that have always driven me in computing is action at a distance and being in control of my own destiny with my hardware.

This when when people try and take those things away from me, that's when I feel diminished

Jonathan: This is why simon and I get along so well We we have those two very core core to our being things in common All right. So simon, do you find yourself or maybe you have this? You mentioned the the Sinclair. Do you, do you have some of those still kicking around that you, you boot?

Is that something that even appeals to you?

Simon: Yeah, so I do have a box up in the attic that's got a couple of Sinclair QLs and I've got, I do have a, a, I do have some other Sinclair hardware up there and we have a dinner guest who comes every weekend for, for Sunday lunch who keeps a compute library.

He's, so he's got his old systems and we have our eight inch floppy drives and things, but I found with the QLs that the QLs are very

mechanical

and their mechanicalness is also very what we would call in this country, Heath Robinson. I forget what your term is for it over there. It, it, it, you know, it, it does involve rubber bands and, and it does involve little thin pieces of cassette tape in a, in a little plastic box.

And they've all stopped working and getting them back to being, working again is gonna be really quiet. A challenge. Yeah. And and I know I could, I've got a degree in electronic engineering. I could get them working again.

Yeah.

But there's other things that are more interesting to me at the moment.

Sure. So. They stay up in the loft, really.

Jonathan: And that's something, you know, kind of coming at things from the, like, even the Hackaday perspective. That's something I find fascinating about some of these retro machines, you know, in the right era. The parts inside are just the right size that you can actually get in and work on.

And so they're, they're super useful for young people. Teenagers, people in their 20s that are like working, maybe working to get their electronics degree or just got it Or they're in high school and they find all this stuff really fascinating You can actually get inside of them and work on them you can actually take these old machines and fix them and We're still thankfully in a kind of at a time period in history where if someone has a you know a not working Again, I keep coming back to the Commodore 64.

It's the best selling computer in history, so I guess that's why. But like, you can still find not working Commodore 64s on eBay and places like that without having to spend a fortune on them. And so it is, it's really fascinating for, for all people of all ages, but specifically young people. You can grab them and really get into the hardware, and it's a good, it's a good learning platform, I think, among, among other things.

Aaron: Oh, it's a tremendous learning platform, especially since in the olden days people would actually have schematics for these things as well, right? So if you get a bug in your, under your whatever, and you want to actually go in and fix that thing, or you want to learn why it's not working, you can pull out a schematic usually.

Or a service manual, you know, I mean, to Simon's point about things being sealed up and not people don't give you any information about what it is or how it works anymore, but they're used to and give you a full service manual with a theory of operations and a schematic and you could go in and you could learn to your point, Jonathan, about how it works.

You could learn about electronics or any of those things just by, you know, breaking open your home computer and getting in there with a screwdriver and a soldering iron. So it is a tremendous learning experience just because those things were available at one point in history.

Jonathan: I've got a couple of devices, old devices that You can tell like you pull you pull it apart and you look at the pcb And you can tell someone wrote this pcb out by hand Right, like there was a time where that was that was the way things were done and then you know, you would get a you You would replicate it, but it would the first thing it would be someone would literally draw it out by hand And on the other extreme of that today, you know you talk about You get five layer PCBs, you get eight layer PCBs made to be able to fit all the things into a tiny little space.

And so, you know, on on one hand, we get enormously more powerful devices. And and so, you know, we're we're not luddites. We appreciate the the enormously more powerful devices. But on the other hand, that means that it's so much more difficult to work on, to understand, and The, the old stuff, the old stuff is pretty neat, because, again, you can, you can see inside of it.

You can understand how it works on the inside. That, maybe maybe not by accident, is an interesting segue into something else that is enormously powerful that is very difficult to see inside of.

So we were going to talk about open source AI and the open source AI definition, and we've got half of our expert crew here. And so

Simon: You got the skeptic department here.

Jonathan: Well, that's interesting because I find myself being sort of skeptical about a lot of these things too. So maybe, maybe Aaron will hold down the, the AI enthusiast side of the conversation for us.

Sure.

Simon: So we, we, we, we've worked on the open source AI definition at OSI. And I was the staff skeptic. I stayed out of the process completely because somebody had to do the other stuff that wasn't AI, so I got on with it. With you know, looking at the legislative schedule of the European Union and influencing that for two years and, and I kept on throwing rocks at the AI stuff, you know, I, I would make rude comments like, isn't it strange how all those people who were into crypto last year into AI this year?

No. And but nonetheless, you know, I've, I've got a reasonable amount of respect for where we've got to. We do have open source AI definition. It exists for excellent reasons and it's the subject of a program that you'll be doing in January, I think.

Jonathan: Looking forward. I am very much looking forward to that.

You guys, you guys have gotten some pushback on that though. People don't like your definition.

Simon: Well, actually, when you, when you look at what people are saying about it, you find that Generally speaking, free and open source software people analyze the problem the same way. You look at what the, the, the way that the Software Freedom Conservancy has analyzed the problem.

You look at the way the Free Software Foundation has analyzed the problem. And they've, they, you know, we look at it the same way. The, the, we, we recognize that it's a, A system that involves software and data that it involves life, life cycle phases. We, we, we assume that for something to be open or free, it has to include all the source code to the software under an approved license, that it must include all the code, all the, the, the parameters that you need to configure the system, and that it must include all the data that you need to populate the model.

And, and like, like all good high detail movements what we're arguing about is one of the tiny details, which is, well, how much of the data do you actually need to trade and train the model? Yeah. And the answer to that question is quite complicated. The answer is that you don't actually need all the data to train the model.

The problem you have is that you don't know which of the data you do need to train the model. And so you give the model all the data, and it It then trains itself with the bits you needed, and if you could magically know which bits it needed, or if you could artificially synthesize those bits so that the model could be primed, that would be great, but we keep on being told by AI experts, and it's important to talk to AI experts as well as free software experts about this subject.

We keep on being told the data doesn't really matter. You can use any old data because the model will end up roughly the same. And also that they, the other thing that they say, which is really head scratching, is they say that if they run the training process twice, they get a different, a different model at the end with even using the same data.

And so the things that free software people want is they want the, The full corresponding source, you know, they want the, the, the form normally used for making modifications to use the software language about it. And you talk to the AI experts and they say, well, there isn't really one of those. And so all the arguments we're having have been up have been from free software and open source experts trying to force fit the free software concepts.

Yeah. into the world of A. I. And the answer is you can't do it. And we then disagree with each other about about the compromises that have to be made to try and do it. Now, I think the A. I. Definition we've got is is perfectly adequate for the next step, which is stopping meta taking the market over by claiming that their llama is open source, even though every any fool can see that it's not.

That's the big presenting problem. The big presenting problem isn't, you know, is it okay to train a model on transient data that you can't give to people? The real presenting problem is we've got these completely closed AIs that you can't see anything inside that people are saying are open AI.

I mean, it's even in the name of the company.

Or they're saying is open source AI. They say about LLAMA and neither of them is in any way. Well, LLAMA is actually, you know, less closed actually. But the real reason we need the definition now is to, to make sure that we've got time to have this conversation, because if we hadn't made this definition now, the conversation would be as moot as the conversation about you know, about cloud is the free software and open source community completely blue cloud.

Yeah, we made no ethical directive statements that helped. And consequently, the cloud industry has gone whoosh off in that direction. It's completely out of our reach. And the same thing was going to happen with AI. We were going to have an argument for five years about, you know, whether we should call it AI or machine learning.

And meanwhile, Meta was going to completely control the market and tax everybody who innovated. So you know, i'm i'm glad we've got the definition. I don't necessarily agree with all of it but I'm glad

Jonathan: we've got it. And it's, I guess, it's important to point out that like, you guys are not writing the licenses.

You're writing the things that must be in those licenses, like the minimum baseline requirements for them to be considered an open source license, right?

Simon: Yeah, we don't, so ISI has never written an open source license. What, what we've, what we have in the open source definition is a you've heard people say, you know, I don't know, I don't know what the, I don't know what one of those is, but I know it when I see it.

Well, the open source definition is the list of things to check to make sure that it's one of those. And that's AI definition is as well. It's a, it's a list of characteristics that let you know, Whether your freedoms are going to be respect respected that whether you're going to be able to take that AI and do the thing that you wanted to do rather than the thing that somebody else wants you to do.

And I, I think the definition we've got is pretty good for that. I think it's a little bit over generous with what it allows in terms of availability of data. But I think it's got it absolutely right on the source code. I think it's got it absolutely right on the model weights. I think it's got it, Absolutely right.

All the way down to this continuum of how much code do we need. And no one has been able to say how much code you need. And so there's some people say, well, so to be on the safe side, we want all the code. That means we'll never have an open source medical AI. Because, you know, medical AI is never going to ship with all the original data.

Aaron: Right.

Simon: And that's, that's the AI I want most is an open source medical AI because I can never get an appointment with a doctor. So, you know, I want to be able to ask my computer what that rash is down there. Why I can't bend this arm. And yeah, here's a,

Aaron: here's a picture. What, what, what is this? Yeah, yeah, yeah, exactly.

Simon: Do you recognize this rash?

Aaron: Yeah. Yeah. Here's my symptoms. Here's a picture, diagnose it for me. And there's already been examples of that happening, right. Where people have used AI to get a diagnosis that, that many doctors couldn't figure out. And then when they see it, the answer that they got back, like, Oh, of course you put these things together and it means you have this, this thing.

But, you know, AI did it in. A couple of seconds.

Simon: Thank God it got it right. I mean, the big problem with all these AI models. So, you know, you're getting into my skeptic self now.

Aaron: You

Simon: know, the generative AI models, you have to remember that their purpose is to persuade you that they have responded to your prompt.

Their purpose is not to tell you the truth.

Yeah.

And sometimes in persuading you, They accidentally tell the truth, and that's really handy. But the people who should be interpreting the output are the people who don't need the AI. The AI is there to help them get to an answer sooner, not to help them get to an answer without an expert.

Aaron: And,

Simon: I hear so much of this discussion being about, oh, you know, the AI hallucinated. The AI made an error. No, it didn't make an error. It was trying to persuade you that it was right. It has no idea what right actually is. Yeah. What it does for a living is persuade you. And, and it did a great job because you were persuaded.

And and the fact that you're persuaded about something that was completely false is beside the point because it's not here to tell you the truth.

Jonathan: Yeah. Yeah. You've got even, you've got even another, like, angle, another point to dig into with this, and people call it trusted AI, which sometimes boils down to we always want the AI to give you the answer that we've deemed appropriate, and there's a whole, like, Boy, that's a whole ball of wax in and of itself, right?

But then on the other side, you have things like one of the car manufacturers here in the United States, you know, Ford or Chevy, one of those guys, they put put an AI chat bot on their website and someone convinced it to sell them a car for a dollar. And apparently the courts upheld it and said you put the AI out there as your official spokesman So you you sold him a car for a dollar have fun with that.

Yeah

Aaron: That's funny.

Jonathan: I I've I've made the point repeatedly that I am I am just I am waiting for the next big thing So simon mentioned this with the idea of all the cryptocurrency guys are now into AI I'm waiting for that crowd to move off to the next big thing. Whatever it may be Maybe it'll be back to cryptocurrency or whatever like the the AI bubble will kind of pop And then we'll actually be able to see what it'll be useful for as a tool, once people stop trying to fit it into everything.

I'm looking forward to that.

Aaron: Yeah. That'd be nice. But again, that speaks to why this type of definition is so important right now, right? And even regulation. I'm not a big fan of regulation. But if you're going to be looking at putting regulation in, now's the time to do it. Because as fast as cloud moved, AI is moving a hundred, a thousand times faster, maybe, maybe even faster.

So if you don't do it now. It's, it's over, right? It's gonna be very hard to put the genie back in the bottle. Yeah,

Simon: I'm, I'm curiously a fan of that. I, you know, regulate early, regulate often is the, is the, the watchword. I think that what happened in Europe is we saw the, the absolute need to have privacy regulation on the internet.

Which was which became that became necessary like 15 years ago and GDPR was way late to the To the to the scene and as a result the scene that it was way late to was already extremely important economically, and so they compromised and they, they got it, they got it wrong. It's, it's an awful piece of legislation with that, that, you know, if anything, strengthens the arm of the people that it was trying to control.

And that's why I quite like the AI act in Europe, you know, it's not, it isn't perfect, but it's come out early enough that you can change it. And it can actually address the real problems rather than coming out in 10 years time when it, when it, when I is already a trillion dollar industry and you can't possibly regulate it because everybody will simply remove your political funding, regulate, regulate early and then you Change it every time you find it's wrong and that way you end up with a regulation that fits.

So I'm, I'm a big fan of regulate early, regulate often. I think that's the way to go.

Jonathan: You know, when you first said that I bristled and I'm like, ah, regulation, it's terrible. We shouldn't be doing it. And then you actually explain what you mean. It's like, okay, that's a reasonable approach.

Aaron: As long as there's a mechanism to fix it, right?

As long as it's not.

Jonathan: Yes.

Aaron: You know, a U. S. Amendment to the Constitution, which never happens anymore, right? But as long as there's a mechanism in there where we can say, go get, you know, go, go fix this thing and make it better, then it's fine. What is that? I'm not familiar with the European. What is it called?

The act or the act?

Simon: Yes. Yeah. So it's it's it's what it's it's It's been quite widely criticized by, by tech utopians. But it, it puts in place some regulations that will stop harms happening and they may be over conservative, but I think that it's being held reasonably lightly in the hands of the regulators.

And when you look at. In, in Europe regulations, you find they get changed, they get amended quite often. And so you look at a regulation like GDPR, you find it's, it's getting modified many times a year. By other political instruments. So by, by delegated powers from other pieces of legislation or from new pieces of legislation that modify the early legislation.

And the problem with GDPR is there's, there's all that awful money associated with spying on us and regulating it harms the people who are getting rich out of all of your and my private information. And so I, you know, what was needed was to stop them from creating the advertising surveillance industry in the first place, rather than to try and regulate it after it existed, because you know, when you try and regulate something evil, you have something evil and regulated.

Jonathan: And the other, the other side of that that we see here in the States from time to time is when you add regulation onto an industry, it is a minor burden. For the established players, but is a nearly insurmountable burden for anyone coming along trying to do anything new And so, you know when when you hear people in in my country decrying regulation, that's sort of the thing that they're getting at and again, it's one of those where when We try not to get political on here on the show, but I think we're going to, we're going to get into it just a little bit today by the very nature of the same things we're talking about.

So when you hear like a conservative talking about, we got to get rid of regulation, that's what they're getting at. We've got some of these regulations that are just because there's so much of it and because they're written to the benefit of the established players, it, it can be a problem. And again, it's one of those deals where, If you, if you just, oh, let's see, how should I put this?

If you just come at it from the kind of the partisan viewpoint, well, you have an an A versus B and, and, you know, the other side is wrong. But if, if you have something to where you can kind of cut across that and I think open source is an interesting tool for doing this and, and listen to some of the points that are being made, there, there are some things that we have in common here that people on the very far left and the very far right, if you actually have a, like a A well intentioned, good faith conversation.

There are some points that you can come and actually agree about. And that's sort of a useful thing.

Simon: Yeah. It's a, it's a, it's a turning fine line to walk. So, you know, before, before we, before we, you pressed record, we were talking a little bit about that. Yes. And What we found at OSI over the last 25 years is it's really important that we focus on the, on our swim lane,

you

know, and our swim lane is open source licensing.

And the reason that swim lane is so important is because every side. And there is, there is at least three sides to this conversation. Every, every side recognizes the need for a copyright anchored license that gives you the rights to exercise your freedoms. And. So as long as we focus on that, everyone's happy with that because, you know, there are some people who are, you know, they're very keen to have that happen so that their freedom to make sure everybody can use the software under copyright on the copyleft terms is protected, there's other people who are very keen that there's The license makes it available to everybody without restriction.

There's other people who are very keen that the license protects their, their rights in a way that is, is, is, is lightly touching the software, like Mozilla license does. And So that means that within the community of people who are wanting this to happen, we can have people who are, you know, over at the libertarian front or over the, the, the social good front or at the, Over at the intensely personal.

Leave me alone front. And and they can all be served and we don't have to argue about the things that we disagree about. And that's where we're going to run on the rocks with AI is because already in the conversation there is all of this is being woven together with ethical purpose and acceptable use policies.

And if there's one thing I've learned over the last decade, it's that what is an acceptable use to one person is an invasion of freedom to another person.

Jonathan: Yep. Or, or, One of the,

Aaron: one of the,

Jonathan: oh, go ahead, Jonathan. Or, so, or, it falls into their definition of, of evil. Like, that's not a thing that everybody agrees on.

And there've been some, there've been some brouhaha's in various open source projects, like the X, Y, or Z military uses some of this source code. And so, and to, to one group, that's the U. S. military uses it and therefore it's evil. And to another group, well, the Russian military uses it and therefore it's evil.

And I don't know that I could say that either of those. are like illegitimate points of view. They don't work well together when you try to add the, except the idea of acceptable use into the conversation, which I think, I think that's why it was, it was brilliant that when the open source definition was first founded, it was explicitly stated that we are not going to include.

Definitions of even no, no acceptable use policy because even back then they realized, and they could see it even in culture and, and, and all of that, that nobody's, we are not going to get on the same page on this.

Simon: That doesn't mean there's no place for them. So so I think it's absolutely fine for a project community to say that we are here to achieve these objectives.

Jonathan: Sure.

Simon: And the, and we're going to make sure that there's no one in our community that doesn't want to achieve these objectives. And when you find that excludes you, you fork, because you're free to do that, because the software license makes you free to fork. And then if there's a critical mass of people who share the other view, you get two pieces of software that are serving.

But you, that, that is a, the, that is a role for the governance of the community, not for the license of the software. very much. And so that's, that's the line that I, I think I'm still trying to work out how we draw on AI. Yeah. Because we're being encouraged by legislators to fold acceptable use policies into the core licensing.

And that isn't gonna work. Right. That's That's going to result in the fragmentation of the community down to individual company sizes. And that's going to result in there not being open source AI and thus not the, we won't get the social good of the network effect of massive collaboration.

Aaron: Yeah, I think there is a, I'm hoping this is true and maybe you guys can corroborate this or not, but I think that what you were alluding to there is this idea, I always worry, right, about open source, that it's going to be somehow influenced by, The conspiracy theorist, right?

It's going to be given a bad name. Just the reputation of open source could be affected or something. But I remember to your point earlier, one of the most interesting times I ever had at Sun was when we were down at Fizzle in Puerto Alegre. And I was giving a talk, so I was in the speaker room, and I'm sitting there, and there's two guys talking, right, in the corner.

And one of them is Richard Stallman. The other one is Peter Sund. I don't know if you guys know who Peter Sund is. I have a

Simon: photograph of that conversation. Do you? That's awesome. I do, yes.

Aaron: Wasn't that crazy? That was like That was reasonably

Simon: wild. You know, Michael Tiemann was also involved in that conversation.

Aaron: Yes! Like, how do these people get on the same page, right? Like, you would think they would be diametrically opposed or in some way, you know, how does the guy that created, or one of the guys that created Pirate Bay, sit down and have this philosophical conversation with Richard Stallman? But they were having that conversation, and I think it was, you know, my point is, can open source be more of a unifying factor?

And of course, this is again, why we need to get there with AI. But can it be the leverage that we need between all of these different political groups with different opinions to say, look, That's the nature of it. If you don't like it, fork it and do your own thing and see if other people like it. You don't have to have this conversation like this is, we need to change this or it's the end of the world because you can just go change it.

Jonathan: Yeah. So I'm, I'm, I'm involved in a, to, to kind of tie into this, I'm involved in a project, it's mischastic. And we, we have kind of a, because of what it is, it's off grid radio. We kind of have this like higher than usual population that are I do not, I do not want to insult them. So how shall I put this?

Well, they're, I guess, prepper community, right? We could, we could just say that. And we were talking again before the show, like I, I have some level of very much empathy towards that. If nor the reason, then I live in Tornado Alley. And so I definitely get the idea of let's try to be prepared. Let's try to prep.

And Then, of course, you have some people that just go way off into the deep end. And I, I've made the, I've made the point here before also that you, you have sometimes these people that, like, they will have reasonable points to make, but it's so wrapped up in the, sort of, the tinfoil hat language. So, there was a There was a deal where we, we founded a corporation to, to try to, you know, do some things around this, this particular open source project.

And we had a guy that goes, Oh, well, that's it. They're getting the Rothschild money now. And, you know, it was just sort of, it was off, it was off the rails. But when I actually read what he was saying, the, the point that he was trying to make was essentially I'm worried that now that there's a corporation, there is a leverage point for, you know, the, the government or whoever to be able to come in and say, we would like back doors in this encryption as well.

And like, I don't know if you've been watching the news, but that is not a that is not a tinfoil hat theory. That is a thing that government officials are actively trying to do. That's not a conspiracy theory, that's being done out in the open. And so like, yes, he was off the rails and he was wearing the tinfoil hat in the way that he presented it.

But there was a legitimate kernel of concern that you may not agree with but is a legitimate point to make And so this is something I spent a lot of time thinking about like for one thing How have we come to the point to where people just cannot make their legitimate? Concerns known without getting into this ridiculous language.

And then how do we how do we help? How do we help somebody get back from that point? It's all the internet's fault, you know yeah Yeah, that's probably true social media

Simon: you know, seriously, I think it is because I think that the The ability to compromise involves having a small number of opinions that you can think through rather than having a constant echo chamber reinforcing your first, your, your, your first surmise.

And I think that we've, the, the internet has produced this this dynamic for human beings that results in an inability to, to to, to empathize with. People from, people that you only know virtually. And I, I, I have no idea how we fix it. You know, the most obvious fix is to turn off the internet.

Bring

Aaron: back modems. Modems were underrated.

Jonathan: Media types.

Aaron: Not from a hardware perspective, but from a social fabric perspective. Yeah.

Simon: But I can't see that working out very well in the current environment. So I, I've no idea what we do, but and then the social media has then put that that effect on steroids and allowed people to create divisively boundaried communities for profit.

Yeah. And and you know, so what do you do about that? Well the best, very best thing you can do about that, if you're, if I'm honest with you is regulate it. Right.

Aaron: Right.

Simon: And and I don't see a great deal of that effectively happening just at the moment. So it's

Aaron: especially hard here. Cause we already have regulation that says you can't sue the social media companies, right?

So, you know, that's the thing that's like, I go back and forth on that one. I was like, well, I like that regulate. Cause I like technology and I don't want them to be sued. And I don't want to lose my. Whatever the latest social media thing is. Right. But at the same time, it's like, well, yeah, but the outcome of that could be this.

Other bad thing which we see happening right now, which is they can't be held responsible for what people say On their on their platforms.

Jonathan: Yeah, that is very much a two sided sword So I I fall into the camp of and I I acknowledge this is not working all that Well at the moment, but I very much fall into the camp of rather than solving it with regulation Let's solve it with open source technology This is why i've been trying to push people to mastodon because it it it cuts out all of these problems You can run it yourself.

You can, you can do, you know, whatever you want to with it because you can host it yourself. The problem really with it is two problems. One, there's not quite the critical mass that enough people are on it. Everybody is on, everybody's still on Twitter and then everybody's moving to blue sky and then you have a few people on threads.

But the other problem with that is that people just don't want to host their own mastodon server. Like the majority of people don't want to. And that. That sort of dries up a lot of that Advantage to it. So it's a it is it's a it's a tricky problem So simon, I see that I see the wheels turn and the smoke pouring out.

Yeah

Simon: so, you know that does concern me because I last I checked the the little islands of of isolationism Were running on mastodon Yeah, you know truth social is

mastodon

for example And I actually don't think that it with that that solves the problem because the problem is not a software problem.

The problem is a social problem and that social problem is being exacerbated by the technology and applying more technology to the social problem just gives you a bigger social problem. And I, so I. I don't really think that's the answer. I tell you what does happen though, when you get lots of open source software forming the basis of the economy like it does now, like whatever it is, I forget what the most recent research was, it's between 80 and 95 percent of all software systems are open source.

Yeah. And the what does happen is it's driving sufficient economic value to need regulating. And so that's what's happened over here in Europe. The European Union has decided to, to the Cyber Resilience Act is applying product liability to the whole product, including the software portion of it.

And so manufacturers are now liable for what their software does as well as what their hardware does. And that's going to change everything in Europe. I can't see you being able to get away without doing the same thing in the U. S., honestly. But Europe is a big enough block that it can, it can freely do that and not need everyone else to come along too.

And then we're going to see more of this. We're going to see more of people saying, hang on a minute. What is it that's driving this trillion dollar economy, economic force here? Shouldn't we be taxing it? Shouldn't we be regulating it? Shouldn't, shouldn't we, or, or, or alternatively, shouldn't, shouldn't we give, be giving it pork out of our barrel?

And those things are going to happen because it's big enough now to be visible from space. And as a consequence, the regulators are coming. And there may be regulators that, that pork barrel things that they took to, to, to help their electors. There may be regulators who want to stop people having fun.

There may be all sorts of different regulators, but now that open source software can be seen from space, the regulators are coming for it and you can't stop it.

Jonathan: And so there there is an interesting little quirk here and it's it's a difference between the united states and sort of the rest of the world And this is this is one of the things that was one of the latest kernel dust ups really had me livid in the united states we have the first amendment that that like enshrines freedom of speech as As one of the core principles of our entire existence And not to say that other countries don't have freedom of speech, but it's just it's so central To to the u.

s. Government to what the u. s. Government is supposed to be about and then you have some court rulings over here that basically say that code is a form of speech And so, there is a, there is a very interesting blocker to regulating, particularly open source code written by individuals, because that code is an expression of speech, and the freedom to be able to do, you know, to say what you believe is so bedrock to the United States.

It, it, it's just, it's interesting. I, I, I don't know that I have anything more to say at the moment other than that, but it's, it's really interesting what that's going to do when you try to get into regulation and all the other things.

Simon: I don't think that's going to cause too many problems for regulating it, because, you know, you regulate banks and people talk in banks.

And so you're regulating a modem and the software in the modem is what makes the modem do its things. I think you can regulate the behavior of the system. And when you recognize that the, that you may well have freedom of speech to create whatever system you want, you have freedom of speech to say whatever you want in the U.

S. And it's only when you start acting on it that you get, that you get restrained by the state. And that's, that's all this regulation is doing. It's, it isn't going to infringe anybody's freedom to write the software that they want. What it is going to do is remove their ability to be able to avoid the consequences of having written that software.

Jonathan: So isn't, isn't this though one of the things that, that you particularly had to fight for in the European Union and the laws there, the idea that the liability for the systems would be reflected back to the individual open source developers?

Simon: Well, well, in actual fact, the, the European Commissioner had done a pretty good job of understanding that, that, that there should not be mm-hmm

Liability imposed on people who were not benefiting from the availability on the market. What they hadn't understood was that the language that they used accidentally affected the people who were in the supply chain.

Mm-hmm .

Because they've, they, they'd had a view of the supply chain that, that there was always somebody placing the software on the market and that there was always a company and they were always an entity that was able to accept liability and they hadn't really considered some of the way that they had phrased things meant that You know, a guy in a barn in Omaha maintaining an open source library might well discover that he is a party to a liability claim.

Yeah, and that was what we had to fix. We had to to make sure that that their original intent of the legislation, which was to make parties placing software on the market in Europe liable for the consequences of doing so, which is a very fair proposition in my view, and we need to make sure that that intent Was the, was both real and bounded to the people who had actually placed it on the market and not the people whose work they had exploited to do so.

And, and we succeeded at that. We've, we largely succeeded at that. There's a couple of corner cases still. And so it's the, I, I believe the CRA is in pretty good shape in that regard.

Jonathan: And I would, I would, I would actually agree with you that I think similar legislation is coming to the U. S. You know, eventually, even, even if it's not like a federal legislation, probably what we'll see here is California will introduce it first.

Maybe. Because that's, that's the way things tend to go in this country, but

Simon: It can be you know, I, I also, I don't think you should underestimate the will of certain political actors to to cut the tech sector down to size. This, this is true? And I think you might well see some regulation that that has effects that are not unlike the CRA, but are there to make sure that the Googles and Facebooks and Microsofts of this world are not the the political forces that they have been for the last 15 years.

Jonathan: Isn't it, isn't it interesting how different groups and different political forces act in surprising ways sometimes? Yes. Well, I, so, I think something that we would all agree on with that is we hope they don't mess it up.

Simon: Well, yes. And, you know, that was, that was where, you know, I'm allowed to make political statements about the UK, aren't I?

That was where we, we really screwed up with, with Brexit, was we did something that was not Not realistically reversible still at the moment in the U. S. everything that's happening is still you know, conceivably reversible and it's not obvious that there's anything that's going to be different about that in this political, this, this upcoming political presidency.

And I think what you've got to watch out for is people erecting no return barriers. And you know, the place that that happens is actually in, in, in much more subtle places like the appointment of judges than the adjustment of electoral laws. I think that's where you want to be watching very carefully to make sure that your freedom isn't being abridged by anybody of whatever color.

Jonathan: Yeah, yeah, that's a, I think that's a reasonable statement to make. Let's see, we are, we are just about at an hour. Is there anything else that you guys want to touch on that we haven't gotten to? We, we talked about retrocomputing and politics.

Simon: I mean, you know, we, we, we could talk about sex or religion. We've never had shows on those. No, I think, I think we have avoided that on

Jonathan: purpose. I think I just try

Aaron: to imagine how that conversation would go. I can't even like.

Jonathan: So I've got, I've got somebody that follows me on, I think on Mastodon that will occasionally like and make comments and it's like an app to be able to get to some religious text.

So yes, there are, there are ways that that, that circle can be squared and that we could pull it in, but I, nope, not interested. We

Simon: can all, we've all seen the Saint Ignatius photographs.

I will say that so over here in Europe if you're going to be in Europe at the end of January. FOSDEM is happening. FOSDEM is coming, folks. You need to get yourselves over to Brussels for the first weekend in February, where more open source and free software developers than you've ever seen in a single place will all be trying to squeeze into too small a gap.

And we'll all be geeking out and probably drinking rather too much beer and pizza. And that is a completely free event. So FOSDEM you can attend completely free of charge. All you have to do is get yourself there. I think it's probably the highlight of the event year for me. The event that I, that I enjoy most from a health and welfare position is the SFS con that happened in Italy in November, but the one for simply seeing everybody and making the most progress ready for the new year that's got to be forced them.

So come and come and come, come on over. It's really easy to fly over to Europe these days.

Aaron: And when is that event again? I'm sorry I missed it.

Simon: That's the first weekend in February. So that's February the 1st and

Aaron: 2nd.

Simon: And there are actually quite a lot of adjacent events now. So they, it's now being called EU Tech Week.

And there are events as early as 29th of January. And there's events carrying on as late as 4th of February. So you can quite easily fill your calendar up there. And the best way to get to Brussels if you're flying from America is to fly to Schiphol, to Amsterdam airport and take the train.

There's a high speed train from Schiphol to Brussels. So you don't have to work out how to fly yourself to Brussels. And you get to try a really good high speed train.

Jonathan: Yeah, it's a lot of fun. Insider tips. Yeah. Aaron, anything you want to cover?

Aaron: Well, just the usual, you know, go check out my YouTube channels on Retro computing, vintage computers, all that kind of good stuff.

I just released an episode this morning covering the controllers that Atari put out the CX 40 plus and the CX 78 plus, which are reproduction Atari controllers but they are wireless. So, and, and they have dongles nine pin dongles, so you can actually connect them to your original hardware and get a wireless experience with a, with a controller that looks ju identical to the original, but it's wireless.

So, but then you miss out, they're pretty cool.

Jonathan: Then you miss out on the experience of playing your game and trying to dodge something and ripping your entire console off the shelf, ripping the controller out.

Aaron: Yeah, exactly. I did find a bug inadvertently actually, because, you know, me being me, the first thing I did was like, okay, well, let's really put this to the test.

So I grabbed my, my original heavy sixer, right? The first system that Atari made, you can't get any older than that and plugged it into there. And when I turned on the game, all of a sudden I was hearing all of these, all this new music that wasn't in the original game. And there's something actually in the wireless Controller that doesn't interact correctly with the heavy sixer and the light sixer It's only when you get to the one the four switch model that that problem goes away So there's something that was connected differently in those original ataris that causes the wireless.

Noise essentially that's going on in the circuit to bleed through Into the system and so it was really weird. I was playing with my son. We're like, I don't remember there being a soundtrack to joust on the atari 2600 And then we disconnected the dongle and it went away. I was like, oh So I wrote atari and said yeah, I think you need to be testing this stuff with the even older equipment than you're using but anyway, fun episode, check it out.

And there's a bunch of other Atari 10 things you didn't know about the Atari 2600. If you like Atari stuff, go check out my channel, RetroHackShack on YouTube.

Jonathan: Aaron, that's, that's not a bug. That's a feature. It, it introduces generative music to games. They tried to tell me that. Yeah, they were joking.

Aaron: They were joking.

It was all tongue in cheek, but yeah.

Jonathan: They

Aaron: said, Oh, I think I'd love to tell you that we planned it that way. But I think I'll forward this to our engineers.

Jonathan: All right. Anything else you wanted to plug Simon, you didn't get a, is there, is there a radio

Simon: I

Jonathan: just,

Simon: Put a new post up on, if people go to blog.

opensource. org, they can see my my article today all about the role that open standards are playing in the new regulatory environment and some unexpected consequences of it. I would love Focom Mastodon to, To to follow the OSI blog and anyone that finds any of the outrageous things that I'm saying interesting can follow me on Mastodon.

I'm webmink at mesh dot cloud. To follow along, say hi. I even reply sometimes.

Jonathan: All right. Very good. Thank you guys both for being here. I sure appreciate it.

Aaron: Absolutely. Anytime.

Jonathan: Yep. All right. So you can find my work of course, at Hackaday. We appreciate Hackaday being the home for Floss Weekly these days.

We've also got the Untitled Linux Show over at Twit still, and you can you can find my security column talking about sort of the other side of the coin there at Hackaday on every Friday morning and have a lot of fun with that. We appreciate everybody that watch both live and on the download, and we will see you next week on Floss Weekly.

This week Jonathan, Simon, and Aaron chat about Open Source Retro-computing, Open Source AI, and ... politics?

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week David Ruggles joins me and we talk with Sylvester and Brian about Firefox, the original open source browser. Something of a perpetual underdog, but maybe it's time to take another look at. You don't want to miss it, so stay tuned. This is Floss Weekly Episode 812, recorded Tuesday, December the 3rd.

Firefox and the future. It's time for Floss Weekly. That's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and we're going to have a lot of fun today. First off, we've got the one, the only David Ruggles, the David Factor, the, the original David Ruggles, as opposed to all of those clones running around out there.

Hey, David, welcome.

David: You're just trying to bring in every single one of my handles from every possible source. Oh, I'm excited to be here today. In proper form, I am calling in from a Firefox based browser, so this is, this is awesome. I'm excited. In fact, I plan to do a lot of listening and not a whole lot of talking.

Jonathan: Oh, well, but I mean, you're kind of the, you're the, you're the expert co host this time. You're, you're the Firefox user. So you gotta, you gotta be on the ball to be able to jump in and ask questions. All the things that I wouldn't think to ask. I can do that. Yeah. So for those that don't know, that are not watching the schedule the way that we are, our, our Guests today are Sylvester and Brian of the Firefox project.

So from Mozilla and boy, there's a lot, there's a lot going on with browsers these days, and I am just thrilled that these guys are willing to be here and chat about it. There's. There's some things to talk about. So let's, let's go ahead and we will just go ahead and bring them on. So Sylvester and Brian, both welcome.

Glad to have you both. Thank you.

Brian: Glad to be here. Thanks. Thanks for having us.

Jonathan: Yeah. Let me start with Sylvester and just kind of give us the rundown of like where, where are you in Firefox? Where are you in Mozilla? Well, how do you, how do you fit into this, this thing that we all know and love?

Sylvestre: So I am, I am.

I started at Mozilla 11 years ago, and now I'm a director of engineering. I'm managing an organization of about 50 people. We do release management, release engineers, OS integration, engineering workflow, and some security work also. So we do a lot of things, and we touch a lot of different parts of the project.

All

Jonathan: right. And then, Brian, same question for you. Kind of help us get the mental map. Where do you fit into the organization?

Brian: Yeah, I'm a senior principal engineer working on Firefox and Gecko, the web engine that powers it. I joined Mozilla in 2013 and I've worked on a whole bunch of stuff from kind of dev tools to the desktop front end to platform and recently performance and a whole bunch of other fundamental stuff.

Jonathan: Yeah. Okay, perfect. Perfect. So both both very senior engineering types, which we love. We'd love to be able to ask the engineering questions. But before we get to engineering, I think there's some, there's some interesting history stuff to talk about. I don't think we've ever had like Mozilla or Firefox on the show.

So let's, let's talk about some ancient history and Firefox and Mozilla used to be Netscape. Is that, is that fair to say there's at least a connection to Netscape?

Sylvestre: Oh yeah. In a while, right? So we just celebrated the 20th birthday of Firefox a few weeks ago. Basically Firefox started on the ashes of Netscape.

So, you know, Microsoft decided to ship IE with Windows for free, and back in the day, we, Netscape was sold. Two users. So obviously it Netscape took a hit and some people decided to refactor on Netscape into what we call back then the Matia project. And then the project was renamed to Firefox and here 20 years after.

So, but it's also a, Firefox was really a revolution. We think about it like most of us in that call are pretty old. And remember what it in the day when had popups everywhere. And you didn't have tabs. So if you wanted to look three different websites, you have to open three windows. Now we have tabs and a proper blogger.

Firefox is a browser that created that.

Jonathan: Yeah. Boy, there's some, there's some cool history there. I remember Firefox 4. It's kind of a big memory I have because I was just getting active like in things and on the internet. And I remember when Firefox 4 was sort of the big push and I don't remember when that was.

Early, early 2000s I guess. And that was, that was a big deal because you know, there were cool things happening in Firefox and it's been a long time ago now. And I was, I was talking with David before the show and, you know, you kind of look at the, the, the current browser wars and the, the how shall we put this, the market share of Firefox is maybe not quite as big as what everybody would like.

And it, it occurred to me, Firefox has been in this position before. This is, this is not the first time that Firefox is, is approaching the the market as the underdog. And this in some ways is a comfortable position for you guys. And one of the things I think that we can dig into maybe today is like what, what cool things are coming because you know, when you're, when you're a bit of an underdog, that kind of gives you a bit of freedom, I think.

to to experiment and try things out. And so I'm, I'm hoping that you guys will have, have some neat things to talk about that you're kind of looking at with that, that you know, things coming, things down the pike where Firefox is going to once again, be sort of shaking up the entire ecosystem.

Brian: Yeah. Yeah. It's, it's a, it's a, tough position to be sort of the independent choice and not be the default on on most, most devices. But I think at the same time, in terms of, I think that's important for the platform in terms of having an open interoperable platform and then being able to compete on, you know, basic browsing fundamentals, performance, security, stability you know, managing web compatibility, and then you know, adding a bunch of, a bunch of new features to just make getting through that.

Getting through the day better, you know, people use their browsers for all sorts of things today. And so there's a bunch of stuff coming down the pike in the next year that we'd love to get into.

Jonathan: Yeah, but before we, before we dive into sort of what's coming, maybe let's, let's talk about some more of the like fundamentals.

So Firefox has been around for 20 years and If, if I forget this, David, you particularly, I'm gonna put a pin in this and you help me remember, if I forget, I want to talk about Rust too, because that's something big that Mozilla has been involved in. But I guess first, what, how, how big of a project is Firefox?

Like what, what does the line of code look like? How about, do you know, just off the top of your head, like in the, in the ballpark, how many commits there are? Let's, let's chat about the size of the thing.

Sylvestre: Sure. So it's one of the biggest project that we can find on the planet. Like we we love the expression.

It's not rocket science, but here it might be more complex at rocket science at time Because when you think about a brother you have to run untrusted code all day long and you have to keep the user safe And secure so for firefox to give you a scale of what we have i'm looking at the number We have more than 10 000 Contributor over the history.

We have about 1000 contributor per year. We have about 31 million of code. Of course, some of them are platform specific. So you don't always get all those code in your binary. Some of the code is Android specific. Some of them are Mac specific, et cetera. And we are close to 1 million commits in the code base.

So when you think about it software that has been alive for 20 years, how many of those are still alive? The Linux kernel.

Jonathan: Yeah.

Sylvestre: So windows. And Photoshop, there's only a few. So yeah, it's crazy the scale of what we do.

Jonathan: Yeah. You know, we, we had, we've interviewed some of the people that have been around for 20 years and there's, there's projects that you wouldn't necessarily think of, you know, like the core utils, you know, some of those have been around for a long time, Emacs and VI, but.

Most of those have gone through, like, big shakeups, and you know, we're not still using the same Unix core util source code from the Unix days, right? People have re implemented that now a couple of times. And there's now, you know, this interesting move to re implement it in Rust, which is really fascinating.

I have put my foot in my mouth a few times and said that Mozilla created Rust, and I've been corrected each time. No, no, no, no, no, that was this professor guy. And Mozilla just came along and used it. But, is it fair to say that Mozilla is responsible for a big part of where Rust is today? I think that's probably fair to say.

Sylvestre: I think Graydon was a Mozilla employee when he started that project. Oh, so I think it's a, I think it is a Mozilla product. Like it is, maybe I'm showing off, but it is something that Mozilla created and we had like 10 or 10 employees at some point working full time on Rust. And we are still contributing to Rust in many different places.

So to me, it's a Mozilla product. Correct me if I'm wrong, Brian.

Brian: Yeah. And to, to add on to that, I think nowadays it's a separate sort of spun off into an independent foundation and has huge traction in the ecosystem from tons of, you know, companies and open source supporters and, and and, and whatnot.

But I think in terms of what it was sort of designed to do was to be a systems programming language that could build a browser and certainly to, to my knowledge it was sort of incubated and started at Mozilla.

Jonathan: Yeah, does that put Mozilla in an interesting place as kind of still well? So is Mozilla still sort of the main stakeholder?

Are they are they sort of the The the place where rust still lives and does that put Mozilla kind of in an interesting place as like the engineering experts?

Brian: So now it lives in in an independent foundation separate from Mozilla. It was sort of spun out To, you know, to, to broaden the ownership and sort of governance of the project some years ago, but we still have a user still involved in in the, in the core language and we are landing new Rust, you know, every day.

So we're certainly a big fan to continue to be a stakeholder.

Jonathan: Yeah. Are, are you just adding new features in rust or is there kind of an, an active development process to port things over from the old C code? Or is it c plus plus? Honestly, not sure. Are you actively porting old bits of the code base to rust?

Brian: Yeah. So it's mostly c plus plus. Mm-Hmm. , we are not, we don't have a concerted effort to kind of port you know, the millions of lines of existing code. But what we do is when we have a chance to redo a component. Or some sort of nice encapsulated portion of code. We tend to reach for that just for the security and sort of ability to parallelize code and so on.

So we had a really big project where we took the Stylo CSS engine out of Servo, the engine that we had sort of incubated. Within Mozilla and put it into, into a Gecko. And so that's a huge sort of rust component that handles all of the CSS, you know, parsing and matching and interaction with the layout engine.

That's entirely in rust. And we've seen amazing performance out of that over the years. And we've done the same with. Parts of the rendering stack, I think, crash reporter and, and other components.

Jonathan: Yeah, very cool. I, I, I have been saying for months, if not years now that, oh, well, I really just need to sit down and learn rust at some point.

And I have finally with the the 2024 advent of code, it's a, it's a fun little project where you get a tiny little code challenge. For every day for December up until the 25th. And, and I got to looking at that and they go, Oh, we support every single language. I was like, okay, it's time to learn Rust then.

So I've done the first couple of days of the advent of code and Rust. And I now understand why people, when they first start learning Rust say that it drives them nuts, but then it makes them better programs. I'm beginning to understand what they mean by that.

Brian: You'll have to ask Sylvester you know, about that one as well.

I'm sure he's got some stories from that.

Jonathan: Oh, I'm sure. All

Sylvestre: right. It's not always easy to start with Rust for sure. Yeah.

Jonathan: Well, it's just different. It's a different, it's a different paradigm, really. Because, you know, with C, you return an object, like you return a string, or a value, or whatever. And with Rust, you have this idea of, we're going to return either a value, or We're not going to return anything.

And Russ doesn't have an idea of a null. So you've, you've got this, like this week. So coming at it as a C plus plus developer, you've got this weird thing that you've got to handle. And then, you know, the first time you start using it, it's like, do I need an ampersand? Do I need to do a dot unwrap? How do I actually get it?

The thing,

Brian: I'll be interested to hear how the rest of the advent goes for you and sort of, as you get through, I assume it gets more challenging as it goes. So. Maybe it'll be a good ramp up.

Jonathan: That, that was, that was my hope. That was my hope that it would be a, a good sort of soft introduction. And there, the, the nice thing about it's they're bite-sized problems, right?

Like the first one was you're give, or let's see, the second one, I don't remember exactly what the first one was, but the second one was like, you're given. five numbers, and you have, and it gives you a very simple pattern, like each of the numbers have to be within two, and they all have to be either incrementing or decrementing, and so you've got to do some things like, all right, so we're going to take this, this text file, we're going to split it by new lines, and then we're going to Split it up by whitespace, and then we're going to convert each of those into an integer, and then we're going to do some simple math on the integers to make sure.

And then, you know, return a number and then do a, you know, a running calculation. It's all very, very simple stuff, but the fact that you're doing it in a new language, for me at least, I'm doing it in a whole new language to me. So it's like, over half the time I'm spending there on Google trying to figure out, alright, how does, How does one convert from a string to an integer in Rust?

And what kind of thing is that actually going to give me? But it's, it's, it's been fun. It's been a lot of fun. But it is very, it is different for this, this old C programmer.

Sylvestre: I can, I can share an anecdote about Rust. Given the scale of Firefox, we have, we need to develop plenty of tools around the product itself.

So for example the crash management is one of them. So we used to use bpa, and BPA was not very maintained and upstream was not really accepting our patches. Mm-Hmm. And it was quite old and hard to maintain and we decided to be right. It in rust. And and I, one, one of the anecdotes is that we, we wew wrote a pt B, so the Microsoft par Microsoft debugging format.

We wrote that, we rewrote the parser in Rust and it significantly decreased the load on the server, which were doing the parsing of the crashes, to the point that I think that some Microsoft team are now using our crate that is doing PDB parsing for their own format they created. And we have those kind of anecdotes with Rust everywhere.

Rewriting in Rust sometimes can be expensive, so it's not something that we always want to do, but the return on the investment can be huge.

Jonathan: Yeah, yeah, definitely makes sense. Okay. So what about contributions is, are there still quite a few contributions that come from outside of Mozilla? And what does that process look like?

Like how difficult is it to land something in Firefox?

Brian: Yeah, I think Sylvester may have the numbers pulled up on sort of the, the outside contributions. We, we definitely get a lot of outside contributions. I think in terms of difficulty. It's, you know, it's a browser. So I think finding, we try to identify sort of good bite sized places to start. I think we put a lot into the tool chain to make it easy to, you know, pull down, bootstrap all the dependencies and do the build.

I was just looking, you know, at some of the sort of. Depending on how fast your machine is, you could get built as fast as sort of four or five minutes or, you know, 20 plus minutes or 30. It just depends on the much. It's actually improved since I started at Mozilla a lot. We've invested a lot in that and then it's, it's open source.

So it's MPL licensed. Anybody can, can kind of pull it down and make contributions. And so I think finding the right. You know, we have, we have pages and stuff and a chat server if you want to hop in and get involved with the project, but we've gotten a lot of great contributions from people outside of Mozilla and continue to.

Jonathan: Yeah. So you mentioned the MPL license. Let me cover this real quick and then I'll toss it to David. What, what kind of license is MPL? Is that based on the GPL? Is it a BSD style? You know, how permissive is it?

Sylvestre: It's very close from the Apache. License. It was written by Mozilla. I think it's one of the first licenses of that style.

So it's a, it's, I wouldn't say that, but maybe to take a shortcut about license, not to base it about license it's between GPL and MIT. So we get some protection. It's very flexible. So you can fork Firefox and do what you want with it.

David: And David, feel free. I know that this is the Firefox show, not the Rust show, but one question with the public contributions that you're seeing, are you seeing a uptick in Rust contributions over C It's

Jonathan: a good question.

Sylvestre: I'm not sure. I, I'm not. You know, when I'm bored, I'm going between two meetings. Most of my life is spent in meetings. And sometimes between two meetings, you want to do something concrete and writing docs or talking with colleagues. So sometimes I'm running good first bug. And I know that I'm opening Python and C and Rust.

And usually, so, So those good first bug are taken very quickly by a contributor. Most of them are clippy fixes, that kind of stuff. With clippy, it's a static analyzer in Rust.

Brian: Yeah, and I don't know within the tree. That would be an interesting query to run, I think. We also have a lot of projects that are outside of Mozilla Central.

So the main sort of Mercurial source tree. That get vendored in that we still maintain. So, for example, we're working on a WebGPU backend which is a new API in JavaScript that gives you sort of platform independent graphics programming. And that is a separate GitHub repository called WGPU. That's, that's Rust based and is used by other projects as well.

And so we have that there's a quick stack. There's a lot of and then there's a lot of code sort of that's, that's very independent from the browser in terms of the repository itself that I think have also picked up some good outside contributions.

Jonathan: Yeah. So kind of in this vein of talking about the the, the various contributions and what gets worked on how, how does Firefox, how do you guys make the call?

Like what, what do we spend our, our development time on and kind of related to this? One of the I guess I could call it a criticism, suggestion, scuttlebutt. Something people say not just about Firefox, but about a lot of different projects, is I wish they would spend more time on the fundamentals and not work on and then X, Y, or Z, whatever that person is frustrated about at the time.

And so sometimes that's, I wish they wouldn't waste their time doing AI integration or, you know. And what's ironic about this is people have said that about different things over the years and Sometimes those criticisms turned out to be well founded and sometimes those criticisms were very not well founded.

And the thing that people were, you know, wasting their time on turned out to be the next killer feature. And so that's something I found is that it's very hard to know. But inside of Mozilla, like what, what does that process look like? How do you guys come to that conclusion of, okay, this is the thing we're going to work on.

And I leave that to the two of you to figure out which one is, is more appropriate to answer the question. Both of us, we have different

Sylvestre: perspectives. Brian, you want to start? Well, I think,

Brian: yeah, in terms of, in terms of priority, an enormous amount of effort goes into fundamentals. I think that's been true over the years.

I think. We've had a huge focus in the last couple of years on performance in particular that we could we could talk about sort of speedometer three benchmark and a lot of wins we've seen on that. But, you know, performance security stability. We put Mozilla puts an enormous amount of the investment into Firefox into those things.

They're not always super visible which is 11 difficulty with that. But I think they're they're super important. Like, in terms of performance. Advancing the web platform, both for the, you know, millions of Firefox users. And then also just for the competition in the, in the ecosystem, it's important that you're sort of never slow down on that stuff.

And that can be basically an infinite amount of resources you can take onto those things. And so, of course, you have to make decisions. On priorities and prioritization, especially for a new feature development. So I think in the, in the coming year, you know, there's a lot of investment planned in just improvements for getting through the day on the browser.

So things like tab groups vertical tabs for people who want that with the sidebar that has kind of more rich. Capabilities to interact with the web page, better you. I for managing user profiles. So for completely independent profiles, you could always kind of switch between those in Firefox, but making that really pleasant to use and then, yeah, building features that are using you know, newer technologies, AI inference, you know, such as local translations for language translation alternative text captioning and so on.

And so I think that would be an area interesting to, you know, to dig more into. But I'd also, you know, love to talk more about the performance end of things. And also maybe I'll give Sylvester a chance to jump in there before we go deep on that. Yeah, sure. Yeah.

Sylvestre: One of, one of the things that we, we have to keep in mind when we talk about browser roadmap and prioritization is that a significant part of our work is defined by the startup bodies.

So when there is a new, when there is a new startup coming, we, If it makes sense and if our competitors and friends are implementing it, we have to implement it. Otherwise, you are going to get some window on some website saying not supported on Firefox of things that we used to have back in the day. So we, we have some of our roadmap is driven by, by those standards.

So it's always a trade off. And to be honest, when, when you, you mentioned some comments that we can read online, we read those comments and we have also the tools to Those discussion internally, like, is it the best use of our time? Is it the best way to advance Mod Mission? So, as you know maybe not always the business know, but Modia is driven their mission with, and our goal is to keep internet healthy available to everyone and the public resource.

And all decisions that we make on Firefox and at Madia in general are, are based on that mission statement. So Brian mentioned performances. We have web compact, we have offline translation, privacy processing translation. So those kinds of things are clearly what what is advancing the web.

Jonathan: So Brian obviously wants to tell us about performance.

And so I'm going to, I'm going to let him, I'm going to give him a little space. Brian, what, what is, what's a recent performance win that you are proud of? What's something that's coming down the pike when it comes to performance?

Brian: Yeah. Yeah. Thanks. Thanks for that. Yeah, I think. So we had a, a project actually with, with with all of the browser vendors.

It's, it's not, not the standards based, but kind of standards adjacent project to ship a cross browser benchmark. So obviously people have benchmark browsers for a really long time. We got together and worked on a new benchmark called Speedometer 3 in the last couple of years and have shipped that shipped that this year.

That really focused on kind of end to end user journeys. And so one hazard of benchmarking as, as people probably are aware is you can create these little tests that are not indicative of the real world in any way. And then you optimize those tests to no end and it actually doesn't really help anybody.

And so we wanted to build a benchmark that was really end to end. Like it, it opens a webpage, it renders a chart, you measure kind of how long it takes to do. These different things. So I identified a bunch of use cases, vetted all the code work together with the other vendors to put that together and then sort of set off individually to go compete on that you know, a solid kind of.

framework. And so we, we have an amazing set of performance tools internally. I think that, that lets you sort of take profiles, share them, a great sort of web UI and user experience to share and pinpoint exactly what's slow here and there. And it's really satisfying work, I think, to kind of go and you take a thing, you see exactly where it's slow and how you make it faster and you go you know, work on the code to make it faster.

So a bunch of examples, I think. One that stands out is there's a JavaScript feature called proxies, which is kind of a weird metaprogramming feature. Where you can wrap an object and then sort of intercept any call into that object and like customize it before it happens. And we, when it shipped, it was not used really.

And so all we did was just focus on correctness for the feature and we had never noticed any usage in the wild. We had never heard complaints about our performance. As part of the benchmark, we updated to the latest view JavaScript framework, and it turned out we were like very slow at this test and it was, you know, very concerning.

We're landing these tests that were really bad at, and we started to look at it. Well, they started to use the proxy feature because it's kind of convenient for this, like, reactive programming model to, to intercept calls and know when some state changed. And so that was like evidence that this thing is used in the wild.

Then we went and we, you know, made sure everything happened in the JIT and made it way faster and. You know, land of the change and move made the benchmark faster, and we start to see improvements on real pages, you know, real user metrics and so on from stuff like that. And so there's just. You know, hundreds and hundreds of bugs like that, that, that we've done in the past couple of years.

And I think you may not notice any individual one as a user, but all in all, you know, when you press a key and the text appears quickly, or you click a link and the page is still responsive, you can feel it. And it makes a difference. I think in a, in a in a, in an incremental way, when not, not just Firefox, but everybody, all the browsers are really pushing against that.

Jonathan: Yeah. So something comes to mind with that is. What about on the browser? Like how much of the, how much of the code base is and I know there's some interesting questions with the browser. We'll get into that in a second or excuse me, with, with mobile how much of the browser code base is shared on the mobile platforms, like on Android and then even on iOS.

Brian: Yeah. So we split the the kind of, when we talk about the browser, we split it into two parts. One is called content and one is called Chrome. Kind of confusingly, the browser Chrome is like the tabs and the URL bar. And so. The content is on on desktop and Android is rendered by Gecko. The Chrome, so like the tabs and URL bar on desktop is actually also rendered by Gecko.

So it's, it's a web app. This actually kind of blew my mind when I started and joined Mozilla. Is I was like, okay, so how are we at a feature here? And it's like, oh, well, it's HTML and CSS, basically. And so, so when you're like typing in the URL bar, there's pop ups. That's all just like JavaScript. It's very nice to write as a front end web developer.

On Android, it used to be done that way. And then we switched to use the native UI controls for the browser Chrome on Android. And then on iOS, we use the native UI controls for the Chrome and we use WebKit for the content. Due to, you know, app store policy.

Jonathan: Right. Now, is that, is that changing? I know at least in, and so here, here's the, the background for this.

I, I don't know how many people know this. Am I going to get myself in trouble? Yes. Apple is terrible in my opinion. And so one of the things that in, in fact, I'm, I'm not sure why it is company A that we have an antitrust suit going on with and not Apple but that's just me. I think David agrees.

Anyway so what Apple has done, as I said, you know, if your browser is going to be in the app store, you're actually going to use our browser code. You don't get to bring your own code. for the actual rendering of the webpages. So for the longest time, when you, when someone runs Firefox on an iPad or an iO, any, any iOS device, you're actually using the Safari code to render the webpage.

And in Europe, there has recently been a, a new law passed that basically says, no, no, no, Apple, you don't actually get to do this. You need to support it. Third party app stores. And so the obvious question that I have about this is, does that change anything for Firefox and are we going to see another version of Firefox show up in one of those third party app stores?

It's

Sylvestre: a good question. So we, we, we are investigating. How much work would it be to have a part of Gecko? So what Brian described, so the core part of Firefox. How much work would it be to port Gecko and to maintain it on iOS? But as you can imagine, it's a completely different platform. So the constraint in terms of performances and integration CI are completely different.

So we are spending time on it. Maybe we will ship it, maybe we won't. It's still TBD, it depends on the ecosystem, the work that we have to do and the next step in the process.

Jonathan: Yeah,

Brian: I'd say we're definitely fans of the direction here. We've been working, you know, with with speaking with regulators, with Apple on the proposal and really digging into the technical fundamentals to make sure that it's sort of suitable for being able to run, run on a device, being able capable for the architectural differences between the engines.

And so there's a lot of work there that we're very interested in and continuing to pursue.

Jonathan: I, as an aside, I am extremely hopeful that the third party app stores will come to other countries outside of Europe. I, I do have a single iOS device and it is I am not using it to its full potential because of that, because there are apps that I would like to be able to use.

There are just. Not available inside of the official app store. So it is what it is. David, you want to jump in for a while and get some of these questions off your chest.

David: Oh, yeah, I've got lots of burning questions. So to back up a little bit when we were talking about your focus and what you're able to focus on, one of the things about Firefox that I love and as I mentioned at the beginning, I am using Firefox right now to have this conversation is your extension ecosystem.

And so, when you talk about, you know, the Chrome of Firefox and some of the, some of the other features, even ad blocking and stuff and you, we can see, like, the moves that the Chromium side of the house has moved away from ad blocking and, and that sort of stuff. So, that has actually driven some interest in Firefox.

But does that ecosystem, of extensions, allow you to focus more on that core speed and that core rendering engine and everything, and kind of watch the extensions to say, Hey. This is where people, there's interest in stuff and maybe it makes sense to like do the tabs on the side in the future, but you can kind of maintain your focus while letting the extension community experiment with Chrome and the other attributes around that.

Brian: Yeah, I think it's a great point. I think just to reiterate, like we, we support the web extension APIs, including the you know, previous or the newest version, the previous version, we continue to support that, you know, ad blockers, all of the existing add ons in the store on both desktop and android. Now that's a somewhat newer development that took quite a bit of engineering work.

And so we big fans of that. And people use that for all sorts of things. Things I think in terms of using it as a way to sort of identify features that people want the direction things are going. I think it's a good point. And there's times where that extension APIs are a bit too limited. I think like we've seen interest in, you know, vertical tabs, tree tabs and so on.

And there's some really good extensions to help manage that. But to get something that's like really tightly integrated into the UI that supports all of the different features, you know, we talked about grouping tab groups, making sure that's a first class thing that works in kind of both modes. It's just really hard to do that when you're sort of here's a surface, render what you want and through the just like anything with an extension API, it's always a bit harder to kind of coordinate changes.

So I think it's kind of a case by case thing, but to the extent that we can. Let the extension ecosystem sort of pave a cow path or show a direction forward on, Hey, this is a useful feature. It's getting a lot of traction. I think that's a great way to go.

Sylvestre: I was looking at the numbers and we have 600 extension on Android. Now you can look at how many Chrome extension you will find on Android. Probably zero. Yeah,

David: definitely. Let's see,

Jonathan: I didn't know, I didn't know that there were extensions on Android. Actually. I'm, I, I definitely need to go look into that because that's, that's a thing that has been missing on, I think all of the browsers for quite a while now is able to do extensions on mobile and there are some times that that would be extremely handy.

So that's, that's a win.

Brian: Yeah. Yeah. Check it out. I think especially, you know, you're concerned about, about bandwidth performance, maybe there's productivity things you want to add into the browser to customize that, you know, download Firefox on Android and check it out and see, see if it see if it works for you,

David: David.

Yeah, not to rehash it, but that's where I feel like if you are able to be successful in bringing Gecko to iOS, that extension ecosystem will be a huge win over there as well. Well, it's

Sylvestre: not only an extension, right? It's limiting our capability to innovate on iOS because we are constrained by WebKit and WebKit is not shipping as Often as as chrome, for example rs and it is limiting the innovation and we have so everybody Like if you as jonathan said earlier if you use firefox or chrome or brave on ios You always get the same bugs as the same bug of the platform.

So Shipping on this platform. We can do some nice, cool stuff with web assembly with we were talking about offline translation using was some advanced security things that you don't have with WebKit. So have plenty of things that you can do when you control the full stack.

David: I actually wanted to ask about WebAssembly and the effort that it took to integrate that and kind of where you see that going because that's becoming very hot in the development ecosystem.

Not

Sylvestre: an easy question. I think it's one of the things where Mozilla was at the beginning of that technology like 10 years ago, we had different technologies. So Google was pushing PNaCl Microsoft at some point was pushing ActiveX. We, we were pushing a a subset of JavaScript called ASM ts, which was high performance JavaScript and then with, with our partners, we coordinated to ship web assembly to a product origin.

So we have been, part of the people who created that technology. Brian, Brian can probably say more than myself about the next step in that space.

Brian: Well, yeah, it's really there's a lot of momentum in that space. And the interesting thing is, it's not all in the browser. You know, you think about it, it's web assembly.

It feels like a browser feature. But I think there's a lot of interest in sort of server side you know performance secure kind of language independent runtime. And so one of the, one of the challenges to my understanding is sort of making sure that the features are roughly aligned and make sense sort of in both of those context.

And then also just honestly, there's a lot of low hanging fruit still to adopt it, even with the capabilities that exist today. Like the, the translations infrastructure I didn't know if that would work when we started the, the project, but the way that works is we, we you start Firefox, we don't ship any code related to translations other than like a small little WASM thing to check and quickly guess what the language of the page is.

And then if it appears that the page is a different language than your Firefox is, then we prompt the user. Hey, do you want to translate this page? It will actually go and download the WASM program off the, off the internet. And so you're not shipping, you know, a couple megabytes or whatever size of the inference itself with the binary until the user chooses to.

And then it will go down with sort of language pair weights, maybe 10, 20 megabytes, something like this, and then you can integrate it really tightly with the browser, because, as I was mentioning, the browser has, you know, a javascript and wasm runtime, both for the front end and sort of under the hood.

And so then we can integrate it, translate the page, you know english to french, whatever it is very performantly, like, it took quite a bit of work to do this, but we've even been able to start shipping that on android devices. And yeah, when I started, I was thinking, okay, doing this sort of an abstraction layer above C Are you really going to be able to run this?

It kind of. In a way that makes sense for a user and they're looking at a page and it's sort of you can, you can see it very quickly kind of work through the content of the page and translate it. So it's really exciting. And I think there's lots of other use cases. You can just build on from there. And it's very secure, portable.

You know, it's not it's not. Included in the binary and so on. So just a lot of benefits to it.

Jonathan: I just, I just went and looked at Firefox installed on mobile on my Android. And I just went and looked at the extensions available. And one of the ones that's there is tamper monkey. And I am now entirely sidetracked because the ability to write little snippets of JavaScript and change websites on mobile is just been forever out of, out of reach.

And I'm, I'm super excited about that. Oh, that'll be fun. Let's see. We, we, we touched, we touched briefly on this and I think it might be worth kind of digging into that, that other browser. I don't know. Is it the browser that should not be named? Over and over in Chromium land, they are, they are changing the way extensions work.

And you mentioned briefly that you guys are supporting both the, the new version of extensions and the previous version of extensions. And I know there are some people that are really up in arms about that change over in the Chromium code base. Is this, is this something you guys foresee happening, like long term is supporting both of those versions of extensions and like, what does, what does that look like going forwards?

Brian: I, I'm not super familiar with this area of the, of the code itself, but I think in terms of extension support in Firefox, yeah, I don't think there's any. Intent to, to change. We sort of support the current set of APIs. A lot of add ons are programmed to those. I'm not quite sure the sort of deprecation plan at what point would it be if you're going to.

Ship it. It's only supported at Firefox. And if so, how does that change people's mind on kind of shipping it? And, you know, I think on some, you know, on things like ad blockers, I think there's some cases where the new model is maybe a little bit less resource hungry. And maybe some people would prefer that.

Whereas the older model is more feature rich and people prefer that. I think in terms of, you know, I think just the users could choose. Which is their preference. So long as the extension is supported.

Jonathan: In the, again, this may be outside of y'all's expertise, but in the kind of extension store for Firefox I assume there's, there's gotta be like some, some checking done for malicious code.

Has that ever been a problem? And, and how, like, how does that get sorted? How often do you see malicious extensions get uploaded and do they get caught pretty much right away before they get offered to users? I mean, I would assume that that's the goal.

Sylvestre: So we have a team who is in charge of doing moderation.

So we, we have a lot of tooling, we have a lot of static analyzer and just kind of tool to verify that nothing bad is happening to the user. But yeah, we, we have precedent of some, some extension being used against the user at the end. Yeah, fortunately, but yeah, we are, we are very careful. So that, that's why sometimes it takes, a few days for some new extension to be approved. It's because we need to make sure that every new extension is safe and is not going to do some crypto mining on the system of the user, those kind of things. Yeah,

Jonathan: and crypto mining is really the least of our worries, right? That's, that's the starting point.

And it just gets worse from there. What about, what about detecting those sorts of things in, in websites? Because like a website gets to run basically arbitrary JavaScript. Does, does Firefox do things like trying to detect crypto miners and JS in a website? Or I know obviously there are some things that are malicious that you guys have to, to stay on top of.

What, what does that process look like?

Brian: Yeah, we there's a built in features called tracking protection that has sort of support for each of those classes of sort of nasty code, you know, crypto miners and cross site trackers and that sort of thing. That's you know, there's a sort of curation process. It's tough because you're sort of on defense with that.

So you have to identify. Which, you know which scripts are doing the tracking, add them to a list, deploy that list and notify the clients. And so it's really a hard problem. A lot of challenges with the browser is trying to sort of understand and navigate, you know, unknown web content. And so it would be great to come up with some kind of heuristic.

Where we could say, okay, we have some confidence that this script is doing something like crypto mining as far as I know, that's sort of in the domain of research at this point. And we're starting to, you know, I would be interested to I know we're doing research on fingerprinting and other things, but it's just so hard because.

The APIs that you might want to use for doing something malicious or the same APIs that you would want to do for something legitimate. And so it just gets into really messy heuristics of, well, if you call it this many times and this long, is there some kind of budgeting thing? It's really messy. It's very hard to do in a reliable way.

That's part of what makes browsers challenging.

Jonathan: Yeah, and even a step beyond that, like, so what would you do? Let's just, you know, kind of game theory this up for a second. You know, say you had a heuristic that could detect crypto mining. Okay, so we're going to block that in the browser. Well, what do you do when someone wants to run a website that that is the point of it?

You have an account, pull this page up whenever you're not using a computer and make a few cents an hour doing crypto mining with us, right? And so then you're in this, you know, you're in this weird position of, okay, well, we have to go and approve these URLs to be able to do it. And oh, that, that gets very, very hairy and complicated very quickly.

And. I mean, I guess that's what that's what it is, though, running a browser. It is hairy and complicated because you have all that stuff all the time.

Brian: Yeah. And then every prompt that you're kind of showing the user at some overhead of sort of explaining and making sure they understand what they're consenting to or not.

And so, you know, obviously, ideally, you would, you would set defaults that are like, right for what you feel is right for users and give them an easy way to change it. But it's super, super hard to do that. In a way when the, when the content is not sort of deterministic or controlled by us, it's sort of decentralized content.

So yes, you're right. Yes.

Jonathan: And then, and then, you know, someone passes a law to try to help it. And the next thing you know, every website you visit has a cookie banner. Click here to accept cookies. Yeah. I, I would love, I would love a, a feature in Firefox could do it. Maybe maybe we make this a web API that says I actually know how cookies work and websites are allowed to do local cookies and please don't show me any more banners.

Could we make that feature work? Well, I

David: think that's going to wind up being a legislative question more than a technical question.

Sylvestre: Well, we are back to the web extension point, right? You have web extension doing that for you if you want. Yeah, there you go. There you go. It's a tricky. It's a tricky thing to do that one.

David: Yeah Continuing on the security questions, though. How How does mozilla which I mean, I know you've got the security bug bounty program Maybe you want to talk a little bit about that But there was actually a news article in the last week or so about rom com out of russia that took advantage Of a zero day in firefox that had actually been patched two months ago So obviously Some of the issue is end users not updating.

So I guess I have two questions that you can answer however you feel. Number one, how, how are you guys staying ahead of those evolving threats and issues in code base? And then two, we just talked about, you know, prompts to the user and stuff. Other than the automated updating, which is built in, unless you build it.

Disable it. Are there any other prompts or things that you're pushing the user saying? Hey, you really need to update

Sylvestre: Well as brian said earlier You have to be careful what you are going to prompt to the user because it can be an annoyance very quickly So you have to be very careful. So we do everything that we can to to help the user to update So for example, you will find firefox on the windows store we have a Debian repo an official one where our binary is going to update the user on every platform on Google Play.

The platform is going to update you. So we are trying to update the user as fast as we can, but there is so much we can do. So what we are doing in terms of security, we are, we are doing a lot of things. So we, the bounty program, as you mentioned, is one of the best way. One of the silver bullet that we have been using for a long time that if one day you want to talk about that into details it's an amazing subjects of fuzzing part of what it takes to fuzz a web browser.

So fuzzing is a technology where you send some correct or incorrectly formatted data into the various endpoints. So sometimes it can be just. It's by spider monkey. So JavaScript, but sometimes it's going to be the API or the IPC or Firefox. So you send that and you see what happens and you manage your crashes.

It is one of the silver bullets that web browser have been using for more than 10 years now And we are using static analysis. We are also experimenting with LLM. Like if an LLM can help us identifying some security issue, but we are investing a lot of money to keep our users safe.

Jonathan: Yeah. I think I'm going to jump in right there for a second.

I think one of the most interesting kind of frontiers and security research right now is essentially LLM guided fuzzing. I think that is extremely fascinating, because, and I'm curious about this, whether you've had this problem as well, we've heard from like the Curl project, where they've had people that have asked the LLM, you know, ChatGPT, find me a vulnerability in Curl, because Curl has a bug bounty and some people like I make money by getting chat GPT to find me bug bounties Well, and of course the person doing it doesn't actually understand what they're doing And so they say, you know, find me a vulnerability and chat GPT will happily hallucinate a vulnerability for them And so I am curious whether the bug bounty project at Firefox has seen some of that But there's been an uptick in you know, sort of fake fake vulnerabilities.

And then on the other side, is that something that you're, you're looking into is this idea of LLM guided fuzzing, because I think that is absolutely fascinating.

Sylvestre: So we do monitor those bugs. I don't know the percentage of of those kind of bug report, but we have a few people that are working on, on those kind of bug report full time. So, but given the complexity and the investment that we have done in security, there is, there are no longer, no hindering foods anymore.

The bugs are usually quite complex when you want to find one. Then for fuzzing, we are investigating because we know that it is the next big thing, and we know that the attackers are listening to your podcast and they know now that it is one of the, one of the tricks to find exploits. No, more seriously, it's obvious that it is a big deal.

I guess currently with Firefox, given the scale, it would be hard. But if you know how it works very well, you can probably find a few. And to be honest, we found a few security issues playing with LLM in Firefox. Sure.

Brian: Yeah, and I think there's a lot of interesting stuff you could do by hooking this technology up to a browser, an automated browser.

You know, when you think about looking for compatibility issues, you know, a site that says, oops, we don't support Firefox, you know. Is there some kind of vision analysis? Could you even for more complex workflows? Like you get a bug that says, here's the steps to reproduce. You know, if somebody does a really nice job filing the bug and they give you a very detailed bullet list of exactly what to do, can you sort of wire up a tool to help you know, validate, capture a performance profile, these sorts of things.

I think there's a lot of things to still explore in that, in that space, in addition to security.

Jonathan: Yeah, yeah, very much. So so I, I have said, I have said before, and I will continue to say that I am eagerly looking forward to the flash in the pan of AI going away because like with every other sort of bubble kind of technology, it's only when the bubble bursts that you really start to see the usefulness of the thing, right?

So like you have the debt, the. com bubble. Which when it burst the internet did not go away It's just people started using it more as a real tool rather than doing dumb things like renaming their city city. com and I I see something very very very similar happening with ai it is it is going to stick around We're going to use it for things but you know, hopefully at some point every company will will get the It will get the pat point past the point where every company feels like they need to make AI do everything for them.

And at that point, we get to see where it's, where it's going to be really useful.

Brian: Yeah, I felt I think the local translation thing changed my mentality a little bit on it. Like, I think I was pretty default skeptical. And when I saw that we with an example, I, I remember having a conversation with the accessibility team at Mozilla, maybe five years ago about the idea of doing auto image captioning.

So if you're a screen reader user, you have a vision impairment. If you're building a website, you're supposed to put alternative text on an image. So this is in the HTML tag, you write alt equals and then you describe what it is. In case the image doesn't load or if the user can't see it. But the problem is most authors don't do that.

And so most images don't have it. And so if you're using a screen reader, You're reading the paragraph, you're reading it, and then you get to the image, and it's like, oh, there's an image there, and then you keep going, which sucks.

Jonathan: And

Brian: so, we had looked into, some browsers had started to explore doing, using a cloud service to sort of send the image up, describe it, and send the text back down, which, it was not something that we were, you know, considering or wanting to do, we wanted to do it locally.

And I spoke with the team at the time, and it was like, no, the stuff is, It's just too bad. Like it's just not helpful. It's worse. It's kind of harmful for users because the descriptions are bad. It requires the super advanced hardware to do effectively. Maybe that was maybe five years ago. And then when we looked at it a year ago, it was like, Oh, my God, you can actually do this now.

And, you know, we want sort of controls around it. Like the user should be informed that this is machine generated, and they should opt into it. But if they do, it's actually a huge service to users, people who are using screen readers and like we've, we've sort of taken the steps to generalize the translations platform, basically.

Into more of a general inference platform within Firefox and started to dip our toes into that image captioning task a bit starting with the PDF editor where it's more like an authoring tool right now, but as we sort of validate the performance and quality of those captions, we're hoping to bring that into the main content for first accessibility tooling.

And so those types of things where it's a very clear user value to the feature, we can do it sort of locally performantly. It's to me, it's just like a no brainer from a product standpoint because it's a good feature. You know, regardless of what tech is sort of underneath it.

Jonathan: Yeah.

Sylvestre: And I would add to that we talked about fancy brand new technology with Rust and WebAssembly, and now I'm going to talk about all the boring tech with PDF.

We in Firefox are not showing off. We have the best PDF reader in Firefox currently. You can do editing of pages and it is fully written in JavaScript. We had only a couple security issue over the last 15 years with it. And if you look at our competitor, It sounds right. And in PDF, one of the cool things that we have been working on is you know, when you, when you, when you have a form and you need to put your signature, you can do it with your touchpad or your mouse, but it's not very good.

And usually what you want to do is you take a picture of your written signature with a pen and But it's hard to do, you know doing image recognition and to, to do the background removal and with with a simple model with LLM with a simple model with AI inside the browser, you can easily.

Identify the signature and convert that into an SVG locally without sending your signature to third party and then you will have a very nice signature That you can insert into your tax form for example, and it is that kind of of of boring use case I would say that that is going to stay after the AI bubble, I think.

And you have plenty of those cases. Summarizing a PDF, for example.

Jonathan: That's actually a very good way to put that. It's the boring use cases that are probably going to stick around. That is very apt. One of the other things that I find so fascinating, this goes back to the security angle, that I find so fascinating about AI is that in some cases it just approaches problems differently than a human would.

And so, you know, if you were to tell it you know, This is maybe a little bit, a little bit more advanced of a, of an example than what we can do right now with AI, but it's, it's close. It's not too far off. So you could say, look at the Firefox source code or, you know, maybe one particular, a function even.

Okay. So if you've got a AI that can pull something from the internet, which some of them can now, you could say, well, look at this function, find me a security vulnerability here, and then write, you know, a, a text input that would trigger it. All right, so this gets back to the idea of doing fuzzing from LLM.

Sometimes, the sorts of vulnerabilities, and even if it's a hallucination, right, even if it doesn't exist, sometimes the sorts of things that an LLM will find is very different from what a human would find, which in and of itself is interesting. And so that may not be quite boring, but I think one of the things we're also going to see is the times where that's useful.

And again, security research is one of those times. It's very, very useful that an LLM thinks, heavy air quotes around that, thinks, it thinks about things differently than the human would. And in some cases, it's an advantage.

Sylvestre: Yeah. And sometimes it is connecting dots that a human would not connect. So for example, it's not exactly fuzzing, but One of the, so we have a partnership with Ubisoft, you know, the video game company, Assassin's Creed and so on.

And one of the experiments that we have been doing with them is using an LLM at video phase. So you send a patch, you say, I'm a Firefox developer. And and I, here is a patch and please tell me how you can improve it. And you have the the LLM, who which is going to tell you, okay, you, you should refactor that one or that code may be duplicated by this one and so on.

So you can do some very cool stuff. And that as a human, you wouldn't think, or a very good reviewer would find it, but it's not always easy to identify what kind of improvement you could do in a patch when you are a reviewer. So I'll ping the reviewer. Not replacing it.

Jonathan: Yeah. Yeah. And so along that same line, something else that comes to mind is you were talking about the alt text for images.

I could see it being useful. And, and I say this sometimes people use the alt text to hide jokes, like XKCD really comes to mind. The alt text for XKCD is. Hilarious, but not actually usually very helpful to understand what the picture is. And so there's this, there's this use case that comes to mind that it's like, even if there is an alt text for an image, you might also want the AI generated one because it might, well, for one thing, it's going to be more useful than the joke that's hidden in the alt text.

But it also might turn out to be helpful in giving additional context as to what an image is. And so there, again, it kind of, it kind of taps into that idea of, well, it, an AI might approach this a little bit differently than a human would, and that can be useful. It's fascinating stuff there.

David: Yeah.

Because as a web developer, I've even seen cases where, you know, you're trying to pass your W3C checker. So you get alt equals image.

Brian: Yeah, exactly. Yeah, yeah, the analysis, we should make sure on some of the analysis, because the analysis I've seen is like empty alt text. We should, we should also make sure to have alt equals image in that list.

Yeah,

Jonathan: yeah.

Brian: The tricky part, you know what would be, that's such an interesting idea. I'm trying to think the user experience of that is like, Maybe, maybe there's a keyboard shortcut when you're hovering or something like this. I've always found picking a keyboard shortcut in a browser is one of the hardest hardest problems.

Jonathan: Yeah, yeah. All right. So let's, let's talk for a bit about about maybe privacy. I don't think we've, we've dug into this one yet. What are, what are some of the big privacy wins that somebody has using Firefox? And I'm trying to remember there was a there was a project that was being pushed.

I think it was at Firefox to do cookie isolation. Was that a Firefox project? What, where, where's that at as well?

Brian: Yeah. So there's a, there's a bunch of privacy features you get when you use Firefox. We had, you do get cookie isolation. There's a feature called total cookie protection that does the sort of you know, domain specific isolation of cookies and other storage. There's tracking protection features, which you can turn on either to different levels and settings, or you get by default in the private browsing mode that goes further and actually blocks or shims, even tracking scripts.

This again gets into the area of like reaching into content gets complicated area because, you know, some sites might rely on a tracking scripts call back to, to operate the page. And so we have all sorts of kind of interventions to work around that. And then we have fingerprinting protections. So fingerprinting is when.

A shared script pulls sort of identifying information off of ordinary web APIs to try to build a profile of you without relying on storage. And so that's also a, you know, feature set that, that we provide as well as, you know, the extension the extension ecosystem. Sylvester, I'm probably missing some some other things, but that's some of the highlights.

Sylvestre: One of the things that I love, which is technology, very technologically, very interesting. It's DNS over HTTPS. It's one of the cool features. So, you know, DNS is a protocol that it is very easy to spy on and your ISP is often doing it. And with DNS over HTTPS, you, everything is secure. So it's one of the cool things that your ISP cannot see where you are going.

Yeah,

Jonathan: I remember that. I remember that being a strangely controversial feature when it was, when it was first rolled out. And yeah, there was, there was some fights over that about the way, the right way to do it and who do you want to use as the end point and you know, I, I imagine some of that was that ISPs were gonna, gonna lose their way to look into what people were doing.

Sylvestre: Also to block you, right? In some country like mine you can do some blocking at the DNS level,

Jonathan: for example. Yeah, yeah. True. Alright, so I know we are, we're actually getting a little bit towards the end of our time, and I wanted to make sure and get this question, and it's one of the ones that you guys you guys included in the rundown, and that is what's, what's some of the fun stuff?

What's, what's something you're, you know, surprising or that you're particularly proud of? Something unique, a story from the history of Firefox, maybe? Let's start with Sylvester. We'll go from left to right on my radio dial.

Sylvestre: Well what I, what I love at Mozilla is I, I know that it can sound cheesy, but it's the, the expertise of the colleagues and the impact that we have on the industry.

We we We are always the underdog, as we discussed earlier. We are tiny compared to our competitors, but we still have a huge impact on the web. And what I see often and some we cannot communicate is how behind closed doors, sometimes we influence the web standards. So seeing that every day, how Mozilla is impacting the future of the web.

It's it's part of the fun of working on that project. And I see that almost every day. And as an anecdote myself, I as a kid, I the fun stuff is working on a project that I, I have seen Netscape and I think it was Mosaic when I was 13 and working on a project like 30 years later on a project that I've seen when I was a kid and you know, it was back in the day when you had one, one frame for me.

per minute seeing a guy in California surfing. And now you have a HD quality on, on the browser. It's to me, one of the fun stuff working on that project and seeing the web waiting for a while.

Jonathan: Yeah. You know, that's to, to, to take a brief detour. That's actually one of the interesting things I'm sure about this is Chromium is.

a competitor, but they're also very much, I'm sure an ally in some ways in making some of these things work. So they're, they're a, a frenemy, I suppose. It's got to be an interesting relationship.

Sylvestre: Well, we collaborate with our competitors in the standard buddies and and many times we, we collaborate with them to tell them, so look, this one is a privacy concern, but they are not going to sign up on the new feature because we have privacy concern.

With that feature, and we collaborate to make the standard better in those cases with them. So we give feedback, we make improvements, suggestions, and so on.

Jonathan: There, there was a Oh, this has been a long time ago. One of the, one of the browser teams always would like, send a cake. To the other browsers, whenever there were big releases.

Was that, was that Chrome was sending that or was that Firefox? Oh, we all do that together. We all do that. Okay. It's still happening. Yeah. One of the, one of the really fun stories I remember from that. So the browser versioning numbers have changed, right? It used to be the semantic versioning where, you know, when, when we, so for example, in Firefox, when we rolled over from three to four, that was a major change.

And then, you know, eventually got from four to five. So there would be like. 3. 0. 1. buildnumber and 3. 0. 2. buildnumber. And then, you know, it eventually has gotten to the point, I think most of the browsers are on board with this now where the, the, the major number is not nearly as important. And so now, you know, we're up to a hundred and something or 200 and something on the version numbers.

And one of the, one of the things that I found so funny there, there, there was, I think it was the Chrome team, the Chromium team, they would send a cake for The major versions and then when it was it was either edge or or I don't remember Firefox. I'll have to go I'll have to go and google this story instead of sending a cake when the versioning numbers switched It was a cupcake because it's like it's not nearly as big a deal as it was but we still want to be nice about it

Lots of lots of fun little stuff like that in there All right. So let's let's ask that same question. To To Brian, like, is there, is there a fun story or something you're particularly proud of? What's the fun stuff that you want to talk about?

Brian: Yeah, I was just thinking earlier in the conversation, I was, I was joking about the keyboard shortcut thing.

This is sort of a funny, but small thing is I was, when I started Mozilla, I was working on the developer tool. So this is the little toolbox that you use if you're a web developer debug your. Your styles and, and, and stuff on your page. Well, well, on desktop, I mentioned it, it is the a web app, basically the browser.

Jonathan: And

Brian: so we have a thing called the browser toolbox, which is the dev tools, but it's sort of popped out in a second window and it's pointing at the browser itself over sort of a remote protocol. And I had joined the team when we were kind of standing all this infrastructure up and I had to you know, come up with a keyboard shortcut for this thing.

And it took me, I was like, okay, well, you know, I think control I that's taken cause that's page info controls shift. I that's taken cause that's dev tools. I ended up landing on control alt shift. I to open the best, the best that I could do. And so but, but I guess, you know, it's been a privilege, I think, to work on the, on the web platform and on a, you know, mainstream production browser that has so many people using it, I think, especially doing it and sort of You know, I came in as a, as a web developer and getting to sort of build that platform and use it also within the browser and sort of co develop features.

You know, migrating stuff in place, some of the really old stuff, migrating it to modern standards and just sort of. I don't know. I just say the, the sort of grind, like every day, make a little bit of improvement, make it a bit faster, make it a bit better, a bit, a bit cleaner. It just sort of watching that compound, even over the time that I've been at Mozilla into the, the, the.

Quality of the product it is today is just really it's just really satisfying and to work with, with great people who are smarter than me in different areas and know there's, there's stuff so deeply, it's just been really a great a great. Yeah.

David: I have one final question as we get to wrapping this up and I promise it's not a gotcha question, but it is a personal one, and I have this platform. So several years ago, Firefox removed PWA support, or specifically the single site browser, where you can make a, Web page act like an app and that is something as a full stack developer I actually use and that's the one thing that I can't implement in firefox Is there any hope of getting that back?

Brian: Let me so the single site browser, yeah, there was like a prototype of doing that to be honest, I don't remember there was just some issues basically with it from like a like it's quite hard to You know to manage kind of the context of this is, do you carry the cookies over? Do you not? So on and so forth.

However, I will say we have started to look at at something like this again. I'll have to find you the details sort of afterwards, but I think we found, we think we found an affordance that we're maybe more happy with. And so I'll have to pull it up. I don't have it handy, but I think it's an area that that, that would be, Worth looking into I'd say it's probably hard in different platforms to do in different ways, but appreciate the The the feature request on that I think sylvester.

I don't know if you have any context on that one

Sylvestre: but we I recommend that you look into connect. media. org. It's a platform where we are receiving feedback from the user and maybe that one is already there and We'd be happy to discuss internally about those kind of things and we use that a lot like We we are not in our in an ivory tower We look at what people are writing on ICONews.

Sometimes we are crying and some, we look at Reddit and we look at connect. media. org and discourse. We, we read a lot what people have been telling us and writing about us.

Jonathan: That's gotta, it's gotta be a, it's gotta be a trip. Right on being on such a huge project that like everybody in the world knows about Like i'm i'm involved with a big project and we have you know We measure our users in the thousands Maybe a million users if we're feeling real, you know, if we're feeling real bold We'll say we might have a million users you guys are just you At least a probably a billion people recognize the name of the project right like it's just insane That's that's got to be a trip to be that big of a project.

Sylvestre: Oh, yeah. It's fascinating. It's also funny to see how people can perceive the company. And and sometimes reading some of the comments and, you know, we don't, sometimes we take the time to answer some of the comments, except when they are very snarky. And knowing how it works behind a closed door, or even behind open doors, because most of the stuff that we do is public, is is quite interesting.

Yeah. Yeah. It's fascinating at times. Yeah. So that Brian has some anecdotes also. Maybe we shortcut.

Brian: You know, I, I'm trying to, you know, I think trying to change and remove features can be really challenging in software that people this won't be a surprise to the audience, I think, but in software that's sort of widely used and people learn to rely on quirks or bugs, or just ways things work removing or changing features just gets really, you Complicated, I think, and so that's something that we, I don't know that there's like a perfect solution for that.

We try our best with sort of experimental infrastructure and communicating and things like that. But you're definitely end up sort of building on functionality. I remember removing. Some really kind of obscure air console UI. Once we built the same functionality into DevTools and just getting like roasted on on Bugzilla or some other, some other forum.

And I was just thinking these are the first time I was pretty new. And, and I just thought, Oh, I just like made a huge mistake on this. You know, I have to go and undo this. And there, and somebody was like, No, no, it's okay. Like this, this does you know, we have the functionality, we can take bug reports and we can improve any, any gaps that are missing, but it, you know, it can be a bit of a challenge.

Jonathan: Yeah. I kind of wanted to ask a follow up on that. And it's like, how do you keep your sanity? If you, if you pay attention to places like, like Reddit and you know, all of the other, all of the other places where people give feedback or talk about your project, how, how do you handle that? It's got to be a challenge sometimes, right?

Sylvestre: I can start. I am a Debian developer, so I have a good training with that. I have been contributing to Debian for like 20 years, so I have a tough skin now. So I learned, I, I'm, I'm I'm always taking feedback with a grain of salt. I know that some people wouldn't talk to me face to face the same way or in the cold.

So I'm always I have a tough skin, but, but what Brian said is we are very careful with newcomers. I had many conversations with engineers who have been reading Reddit or Hacker News, and the feedback that we receive sometimes can be very harsh, so we always work with with with the younger engineers who are not exposed to that, or who don't have a very good experience, and we are telling them, don't worry, that's fine.

If if you make a mistake, What is cool with code is that it's very easy to weave out. We can make changes. We'll ship every a major release every two, every four weeks, a dot release every two weeks. So it's not a big deal, like, at the end. Yeah. And we have plenty of CI. Yeah. So CI is very good to cast those errors.

Jonathan: Yeah, I guess, I guess with that, with dealing with feedback, maybe the, maybe the secret is to realize that the, the people that are talking to you on the internet, they're, they're not talking to you as people, they see you as just, you know, Mozilla, the, the company. And so you can't take any of that feedback personally.

Brian: Yeah. And I mean, I think it's often, you know, people are doing that because they care about the project and they care about their, their app. And so there's, there's some sort of kernel of you know, As I said, it's, it's sort of, it's a good thing to work on something that people care about. I think you do have to learn certain tactics to have the humor and thick skin and stuff to sort of navigate some of that.

But yeah, for the most part, if you look at like connected stuff, it's really funny. People are, people are you know, contributing, like it's for people, especially if somebody is not going to show up and sort of write a patch, but they have, they really care if they want this feature to work or that add on to work and they're sort of contributing.

It's a way to contribute to the project.

Jonathan: Yeah. Very good. All right. Normally at this time, I would ask you guys what we didn't cover that you wanted to, but I, I sort of feel like we have just barely scratched the surface. In fact, pretty much as soon as this call is done, I'm going to start negotiations, see if I can have you guys back because I feel like there's legitimately a lot that we did not get to, and I would like to, because it's, it's a dare and it's super interesting.

So instead I will ask you both this, and that is what what's your favorite text editor and scripting language? And again, we'll start with Sylvester.

Sylvestre: Well I am, I'm biased, I'm bash because of the coroutines, you know why? So that's why I have been contributing to the Rust implementation. And as a deleter, it really depends.

I think VS code is getting better. Better and better. I on my 20 years ago, I wouldn't have said that I was going to use a Microsoft editor, but here I am. Thank you,

Brian: Brian. Yeah, for me, I think for scripting language, it's gotta be JavaScript. That's what I've, that's my sort of go to language. I've been really.

I'm pretty excited lately with some of the new runtimes and technologies. I use the Dino runtime has sort of a really good CLI and system utilities and stuff. So if I need to turn through some files or glue some stuff together, that's my. That's my go to. And then editor. Yeah, I'll use Vim if I'm in the, if I'm in the shell, but VSCode usually for for editing code.

Jonathan: Yeah, understandable. And it's amazing how many people have basically come to that point where it's like, VSCode is just so good. It's what we're using.

Brian: I've got to, I've got to ask what about you guys on your on your answers to those questions? Maybe your audience already knows these, but

Jonathan: They might.

So, my scripting language of choice it depends very much on what I'm working on. The last few times I've had to do shell scripting, I've actually used something called Amber, which compiles down to bash code. We actually had the author on a few weeks ago in Floss Weekly. And then for text editor, my default in the command line is actually nano.

And I think that's because I actually got started on Microsoft products way back in the day and so, like, my first programming was in QBasic. And Nano is sort of familiar. It looks very much like the old Microsoft editor in the command line. So it's just where I feel at home. So those are my two.

David, what would you say?

David: I've actually never answered this question. Python is my scripting language. If I'm on the command line, remote is a server or something. It's a Vim. And my IDE of choice is JetBrains.

Jonathan: That's kind of unusual, though, for a Python developer to be a JetBrains user, isn't it?

High charm! I mean, I know it exists, I know it does, but it's just an odd combination. Alright, well, Sylvester and Brian, both, thank you so much for being here, it was an absolute pleasure. And, like I said, I feel like we very much just scratched the surface of all the things we talk about with Firefox and Mozilla.

All right. Thanks for having us. Yeah. Yeah. Glad to have you both. Okay. What what do you think,

David: David? Oh, awesome. I completely agree. And if I can swing it amongst all the wonderful co host at Floss Weekly has, I would love to come back as well. And I promise no more gotcha questions. That wasn't too bad of a gotcha question.

We've done worse on this show, but yeah, I mean, As a full stack developer having variety in the ecosystem is important. Gecko's always been a little bit of an underdog, maybe a large underdog at times. But just their support for open source for the web ecosystem, everything is just. Second to none in my personal humble opinion and just getting to talk about it is awesome.

Jonathan: Yeah, yeah, absolutely. I am, I'm actually super excited for Firefox on the, on mobile now on Android. I did not know that they had extension support there and that, that is legitimately really cool. I am really excited about that. I think, I think Firefox just became my mobile browser of choice. Which that's, that's saying something, but I think they have a convert.

So I will definitely be giving that a try. So next week on the show, we've actually got something super interesting that I am excited about, and that is the open source AI definition. We're going to have Simon and someone else from from the From OSI here. We're going to talk about that and get into all of that, which that is that is very much the cutting edge of what's going on right now with open source and AI and very much looking forward to that conversation.

Is there anything, David, that you want to plug before we let folks go?

David: Not specifically I would encourage everyone to check out the Twit Network if you don't already. We, I get to hang out with the Untitled Linux Show, the ULS guys from time to time. And there's lots of great news and tech stuff over there.

So I would say that.

Jonathan: Yeah, thank you so much for being here. I appreciate it. All right, and as far as plugs that I have, well of course there's Hackaday. We appreciate them being the home of Floss Weekly. Some fun, some fun stuff going on in the Floss Weekly world as well. Looks like we're gonna be able to get the, the old Twitter slash X account back.

And so you can follow there. That is X. com that feels weird to say x. com slash Floss Weekly. And hopefully by the time you hear this, that will be back back in our hands. Appreciate Twit working to get that back to us as well. And then, yeah, you can follow my work at Hackaday. You can follow me over at still at Twit, the Untitled Linux show.

I appreciate all those that do. And just want to say thank you to our listeners and our watchers. We appreciate it to those that get us live and on the download, and we will see you next week on Floss Weekly.

This week Jonathan and David chat with Sylvestre and Brian about Firefox! What's up in the browser world, what's coming, and what's the new feature for Firefox on mobile that has Jonathan so excited? Listen to find out!

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week we talked with Lars Wickman about Elixir, a modern take on Erlang, and NURVs, a really clever way to run it with Linux on embedded devices. You don't want to miss it, so stay tuned. This is Floss Weekly, episode 811, recorded Tuesday, November 26th. NURVs, real embedded Linux. It's time for Floss Weekly.

It's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and we've got something real fun this week. I've got Lars Wickman with me, and we're going to talk about Elixir and nerves and running things on embedded devices and the Raspberry Pi. It's going to be a lot of fun.

It is just me alone in the hot seat this week, and so I'm, I'm just going to go ahead and bring Lars right on. And first off, welcome to the show, sir. Thank you very much. Yes, I appreciate you kind of stepping in, not exactly last minute, but a little late and we didn't have anybody. And so we, we avoided doing yet another round table discussion or, Oh, God forbid, even worse me doing a monologue for an entire hour.

That probably wouldn't go well. So thank you for being here. Oh, my pleasure. So let's see, where do we, where do we start? You, you are, you're responsible for something called NERVS, which I don't know what that is, I know a little bit, but I don't know what that is. And I

Lars: refuse to take responsibility, but I can't tell you who is responsible.

Okay, well, you're,

Jonathan: you're involved with something called NERVS, which is based on something called Elixir, which is maybe based on something called Erlang. And I, I sort of know what Erlang is. And so maybe let's, let's start with what, what is this elixir thing and why should people care about it?

Lars: Yeah.

So starting with Erlang is not a bad idea. So 40 years ago, the Swedish forests of Stockholm, no, I'm not going to do full story time, but Erlang was created. in Sweden by the telecom company Ericsson, who people might know from like Sony Ericsson and a few Ericsson phones. But they're a massive company that does a lot of telecom equipment, 3G, 4G, 5G, all the Gs.

No longer 2G, I think, I don't think anyone does that anymore. But back in the 80s, they were developing telecom switches. And they wanted to experiment with whether they could develop a language for Developing resilient, reliable, performant telecom equipment because it's, it was a really hard problem at the time the computers were not that fast and they needed to have sort of massive concurrency.

So many phone calls at a time. Yeah. One phone call having an oopsie should not bring down a thousand other phone calls. So a sort of error error handling and resilience. They also ship these sort of resilient redundant boxes that were physically connected. It's not like, Oh, they were network.

No, no. They, they shared a backplane essentially. So they needed to actually run it as clustered devices.

Jonathan: Yeah, so back in the 80s, telecom equipment was basically big iron, right? That's kind of the era still that we're talking about now.

Lars: It's the era of, okay, we're going to put this under, we're going to put this in the ground before we put a parking garage on it.

Jonathan: Yes, I've been in those rooms a time or two, yes. Yeah.

Lars: They also did not want to have to remove the parking garage or even go to the parking garage to update the software. Yeah. So Erlang has supported concurrency distributed computing like fault isolation or reliability, consistently low latency is kind of important for, for telecom and hot code updates since somewhere around the 80s.

It's almost 40 years old now. Wow. I just. recently interviewed on Beam Radio one of the developers that have been maintaining the virtual machine for Erlang. And he's just been doing it for like 27 years now. He didn't make it. He's just the guy who maintained

Jonathan: it. He's a real noob in the field at that point.

Lars: Yeah. So, so Erlang has been around for a long time. Yeah. It's what Like RabbitMQ was built on Erlang. WhatsApp was built on Erlang famously. So they had like 10 years of uptime before they actually had to reboot all their servers for the first time. They did hot code updates. They did all the fancy things and they had very few developers, but a very performant, massive system.

And then they sold for a lot of money apparently, but Erlang has always been sort of a bit of an open secret. It's like only deep nerd wizards. known to use it or, and sort of fought through the documentation to use it. It has not been, it has not been focused on adoption. Not in, in the way that modern languages are.

And at some point Ruby showed up like Ruby, Ruby on Rails. And in that community, Was a guy called Joseph Lim. Well called it's his name. And he, he built up a decent following in Ruby. He did a lot of cool libraries, a lot of useful stuff for the community, very active. And then he got a little bit sick of Ruby being very slow and sort of not great operationally and started exploring other things.

options. And he settled on Erlang, and then he built Elixir on top of the Erlang virtual machine. So it is a separate language, compiles to the same thing.

Jonathan: Ah, okay. And Elixir

Lars: is a modern ecosystem with modern tooling, good build tools, good dependency management, a nice web framework, database layers, all the niceties built out and built up over the last 10 years.

So it's new in the sense that Just 10 years old.

Jonathan: Yeah. And so the original Erlang is, it sort of reminds me of Java and in that it's got, it's based on a virtual machine and it's got some of these, you know, really impressive features under the hood. And so would that make a Elixir kind of like the equivalent to I know there's, I guess Kotlin is one of them, right?

Languages that are compiled to the same kind of bytecode that runs on that existing VM.

Lars: I would say it's kind of the a closure or similar? Well, Erlang and Elixir are very equivalent languages. Okay. With Java and the JVM, if you look at like, I guess Kotlin is a fair comparison in that Kotlin is not a very different language from language from Java, aside from trying to modernize and make things nicer.

Erlang is a functional programming language, so it's functional programming, but it's the functional programming because they wanted the massive concurrency and like, so they, they do, they spawn lightweight processes and communicate via message passing, a bunch of that stuff. And then it really helps if your data is immutable.

So. They chose, they ended up doing functional programming as a consequence of what they were trying to solve for. So Elixir is also a functional programming language. Where I would say the JVM is a fairly general purpose virtual machine as far as I know. I don't spend a lot of time in, with it. I mean, object oriented programming is a very general, Right.

Paradigm. Right. It's not specific in any real sense, but the Erlang virtual machine has, has these primitives of like spawning processes and sending messages. Those are things inside of the machine code, well, the virtual machine's machine code. So it's like, it's a fairly high level of abstraction for, for a system language like that.

And Erlang was built. To build systems. It's like, yeah, you have an OS, but you want to find out if you have disk space. Well, you can just ask the airline system. You, you can stop and start parts of the system. You can inspect the running system at runtime. You can of course, update the system at runtime.

That's the whole hot code updates part. And it had preemptive multitasking from very early on. So it's, it's very much a language and a tool for building systems.

Jonathan: Yeah. And so is Elixir kind of focused on that same thing? And then I guess along with that Elixir Erlang, do they have object oriented stuff added onto them or are they strictly functional?

Lars: They, they are. The least strict, strictly functional language you can run into. So they are high level dynamic languages. So it's not static typing and it's not like Haskell or Elm. And some people are very upset about that. Some people are very happy about that. I come from like PHP and on into Python and then I went to Elixir.

So high level and dynamic has been my tool for the longest time. Sure. So didn't bother me. The FP part was kind of weird to get used to, but it also clarifies a lot of stuff. You don't get this very messy thing where you have classes and objects that interact in weird ways. It's like, no, no, the data you have in front of you is all the data you can deal with and have to deal with.

They're, they're not pure in the sense of the purest functional programming languages, because you have, you're allowed to do a bunch of side effects. You can do logging, you can do IO, you can. Do a bunch of things and you don't have to involve any monads at all. So,

Jonathan: okay. So what, what are people primarily using Elixir for these days?

What's the, what's kind of the problem set somebody today would have that you would say, Hey, Elixir would be great for that.

Lars: A lot of people pick it up to do something like messaging related, like chat or real timey stuff, because it is very good at. sort of soft real time. But it is also just generally good.

So Erlang was built to run services. Services turned out to be like the entire world. Yeah. It's like, okay. Yeah. Define services. But Elixir comparing Elixir to Erlang. Elixir has much more clearly built out like the web toolkit. So the Phoenix web framework, very nice to work with for building APIs and WebSockety stuff and all of that.

And then they built Phoenix live view, which is like, honestly, the best pitch for it is you probably don't have to write very much JavaScript.

Jonathan: I can see that being a winning pitch for, for some people at least.

Lars: Yeah. So it is a server side driven Interactive you, it's a interactive UI framework, but you drive your components and your templates and all that, and keep the state on the server. And you just ship optimized diffs over to, to update the browser side.

So there's, there's a whole thing for like. Very small teams can build fairly ambitious web applications using, using that tool set, but it also plugs together with all the other cool stuff like Erlang comes with clustering out of the box. So if you shove like Elixir onto fly. io that has this. Good private networking across geographic regions.

Like all your machines can just talk to each other and you can do RPC calls across the, across the cluster transparently, you can pass messages around the cluster transparently. It really brings a whole paradigm with it and often cuts out a lot of tools that you end up needing. So if you've done Python web development, for example, that there's like a counter That is just like, what's your time to Redis?

Like, how soon will you introduce Redis to cache something or do a janky queue of some sort, or will you bring in Celery for the queue? Celery has always scared me. It's so strange. But. But I don't need it in Elixir generally. And there are good like queuing libraries and all that tooling if you need it.

But for some very simple queuing needs, it's like, it's almost like you have a small microservice architecture inside of your application. Because you can do multiple things at once and they generally don't impact each other very negatively.

Jonathan: How, how big is the sort of the community, the user base for Elixir these days?

Is it still fairly niche?

Lars: It's very hard to say because, well, ElixirConf EU polled like, I think 800 people, total 650 or 700 in person this year in the EU. So it's not tiny but it's not massive. It's not KubeCon, like, but like, I've worked with Elixir professionally full time for six years now, I think self employed and I keep finding work.

I'm a fairly loud person in the Elixir ecosystem, which helps. Yeah, but yes that

Jonathan: does help a lot. What what is the this was this was not on the the list of topics that We were going to talk about ahead of time, but i'm very curious what's like the security status of it? Like is there are there bug bounties?

Are there people that are? Sort of looking specifically at the, the security of the system, finding vulnerabilities and fixing them. What, what does that look like?

Lars: So for Erlang I don't think there are any bounty programs, not that I've heard, but Erlang is maintained by Ericsson. It still is. Okay. Yes, and it is heavily used by WhatsApp.

Okay. And it is heavily used by Cisco. And also backs a lot of big game company tools. So you'll find Blizzard uses Erlang. League of Legends has used it. It's a bunch of Elixir discord runs on Elixir. So there are large companies that are using it and they take their security reasonably seriously.

Then there's the Erlang ecosystem foundation, which is kind of the umbrella foundation for, for the whole ecosystem. And they do have a security working group. So when things come up, like right now, there's a bunch of conversation around supply chain. Because EU legislation is going to force, force a lot of people's hands in that regard.

So there's a bit of that. And I think the EF recently turned into its own. So if you file a CVE for Elixir for, for Erlang, it's Ericsson but for, for Elixir and a few other things, it should now end up in, in the EF. So. The, the sort of underpinnings for the security work is there. I think it's still fairly early if you compare to the big ones, but Somebody is thinking about it,

Jonathan: at least.

Yeah, I wouldn't say

Lars: I know what the security story of Node. js is either frankly.

Jonathan: Oh yeah, they're, they're, they're working on it.

Lars: That doesn't sound good. Not reassuring at all. So, okay.

Jonathan: That's actually an interesting question because let's see. So does, does Elixir kind of have its own library repository?

So like when you, when you get it, when you start talking about the, and this is what I mean by that. When you start talking about the things, the, the the dependency tree for something like node js, you get into this, this. problem space where just anybody can throw a library up. And so one of the security problems that they have is people put malicious libraries on the you know, on that repository.

Is that a problem that exists in the Elixir space?

Lars: I have not seen or heard people note that there are like malicious libraries coming on to hex but hex. pm very nice repository has a lot of good things in it. Of course, like we can have the same problem there. There's nothing stopping people from publishing.

We don't have the culture of. Sort of proliferating packages in the way the node ecosystem is i've i've run into a package and gone. Huh. This is seven years without updates. I wonder if it still works Cause it seems like it might solve my problem and it just works. Like the, the Elixir language was announced to be generally done.

What was that? Like Elixir 1. 7 or something? It's a ton of versions back. It was probably like three or four years ago and it was, it was, Just met with a standing ovations because it's not a cultural or ecosystem that wants a ton of churn. There are libraries that are sort of leading edge, for example, live you changes fairly frequently because that's still heavily under development, but the language itself.

Changes very little and they have no plans for a 2. 0 because they don't see the need interesting.

Jonathan: So there's no You guys don't have the problem of left pad where you have a whole bunch of you know Libraries, that's just one line of javascript No, that's

Lars: fairly unusual. I think the the general cultural consensus is you can vendor that It's like come on.

Yeah, you just you just add that that's a snippet not a library But of course like there are small libraries I'm sure, but I don't know that they get a lot of use.

Jonathan: Yeah, yeah, that's fair. Okay, so that seems like a reasonable overview of Elixir. We could spend more time on it, I'm sure. But we have other things to talk about.

What, what is what's Nerves?

Lars: Yeah. So when I got into Elixir, I got in, like, I poked around with the web framework and thought, this is cool. But then I saw a talk by Frank Hunleth, who created NERVS. Okay. And he showed putting together some Elixir application and. Compiling it and then burning the firmware to an SD card in record time, because apparently it was like 16 megs instead of two gigs.

And then starting up the device and poking away at like the IAX prompt and running web apps or whatever he was doing, I don't recall the details. And then when he wanted to change something, he just recompiled it and then did. Mix upload and mixes the general tool runner for Elixir mix upload. And it would just shove the new firmware onto the device over that network and do an AB partition switch.

And that compare that to like working with a Raspberry Pi on Raspberry Pi OS, and you'll. either be installing a bunch of stuff inside of Raspberry Pi OS that you will never ever reproduce or you keep needing to re flash Raspberry Pi OS and that takes time. Yes. Yeah, I think the, the base image for nerves has been bloated to 30 megs now, but it's still seconds to burn to an SD card.

Jonathan: And so that's, that's what, the Linux kernel, enough of a minimal system to get you booted, and then the Elixir virtual machine?

Lars: Yeah, so that's fundamentally it. The Erlang runtime system and virtual machine gets started by a tool called Erlinit. Early net runs as process one, and then there, it churns through a few pits before it hits before it hits the Erlang virtual machine, but you don't have anything like system D under nerves because the Erlang the Erlang ecosystem was built to run services and orchestrate workloads.

Interesting. And. This, this is a hearsay, or I'm taking this as second hand information, but I hear that there are challenges running systemd on a read only root file system.

Jonathan: There, I know it's possible but I can imagine that it might not have been designed for that specifically. Sure. Yeah,

Lars: so NERVs by default ships a read only root file system and then a data partition, which means like you can have the really crappy SD cards and they still won't bail on you, or and you won't churn through your eMMC.

So a lot of people take nerves for being like a DIY hobbyist thing. And it's like, Oh, it's a cool thing. You can put nerves on your, and do elixir on your raspberry pie. It was never designed for that. It was designed for industrial and professional use right out the gate. Frank has worked. in embedded in various ways for a long, long time.

And he just found that like the Erlang philosophy of like, Oh, something is failing. Let's find a known good state to re to recover from.

Jonathan: Right.

Lars: Resonated very well with his experiences of trying to make embedded devices behave themselves.

Jonathan: Can, can you run Erlang and Elixir under systemd and, and, you know, more traditional Linux stack?

Yeah, yeah, yeah, yeah.

Lars: I mean, you can, you can put it on anything, essentially. People run it on FreeBSD, people run it on, MacOS, Windows, Linux, all the things. Windows too. That's impressive.

Jonathan: Okay, so it's just when you're using it for sort of this, this embedded or the system is only doing Erlang. That's when you have the, the Er and it start as a, as process one.

Lars: Yeah, yeah, yeah. So that's an ERVs thing. And there's a lot of niceties to that as well. I'm pretty sure, yeah. You can have the, There's a mechanism called the Erlang heart that can monitor your Erlang runtime and restart it if it fails. So it's essentially a watchdog process. And then you can integrate it.

Nervs integrates that with the hardware watchdog. So if either the hard watchdog fails or the Erlang runtime fails for any reason, it knows to restart the device. And this is like stand, kind of standard embedded stuff. If you've done embedded long enough to find out that it's standard. Cause a lot of people just Like, pick up Debian and ship it.

And go, Ha! Embedded device! And that will work until it doesn't.

Jonathan: Right. So, I have to take a moment and say, Isn't it just a crazy wild world where you can put a full Linux Debian install on something and call it an embedded device? Like, we're at the point where that's a thing. That's just, that's nuts.

That is the way it works, but that's just nuts at the same time. Yeah.

Lars: Yeah. Yeah. I mean, two gigs of embedded goodness. Honestly there, there are definitely embedded devices that end up going with like two or four gigs of payload, but that's because they pulled in CUDA. If you, if you buy one of the small Nvidia devices, I would still consider them embedded devices, but they need so much supporting code that it's ludicrous.

Jonathan: Yeah,

Lars: no, but nerves. is the entire concept. Like it's the, it is the system that helps you build your underlying Linux actually I had an interesting conversation with Frank at one point that clarified something for me. He doesn't particularly call NERVs a Linux. It's not like a Linux distribution.

It's not a tool for embedded Linux. He could, he considers. Linux and implementation detail in that regard. And I think if he had his had his way, he would have probably preferred FreeBSD, but Linux gives a lot more sort of support and tooling and like, you get a lot of stuff if you pick Linux. So I think he picked Linux for that reason.

But essentially, NURBS tries to paper over a lot of that. We use the network stack because the network stack is kind of proven. We use a lot of the kernel. facilities because they're useful and good, but You can run a lot of your device from Elixir land, which means you get this message passing. You get like various parts of your system like, Oh, we have this sensor and that needs to inform the backlight.

If you have an ambient light sensor, for example, and it needs to inform some part of the system to tell the backlight to actually bump it up a little bit, because we're in broad daylight, that type of internal communication needs to happen over something.

Jonathan: Yeah.

Lars: And Erlang already has message passing and pub sub and like broadcasting and all that stuff.

So it's very easy to build out this stuff with within an Erlang system. So everything that you can do within The Erlang system is very nice to deal with, but you can pop off like, Oh, we need to start a Python process here. We need to pull in all of like Podman and run a Docker container. Like it's doable.

But the more you can run in Elixir, the more niceties you get, essentially.

Jonathan: Yeah. So, well, this is interesting. Is this, what is, what is nerves primarily targeted at then? Is it for the raspberry pi or did that just kind of come about by accident?

Lars: If you look at the supported systems, it's going to look like it only supports the Raspberry Pi.

Okay. But that's because there are so many Raspberry Pis and nerves, like the base systems we use in nerves are very specific. That's why they can be small. And they, once you put one into production, you can get to know it and know what's in it and make it yours. So it's prefers explicit systems. So that's why we end up having the Raspberry Pi 0, 1, 2 0, 2, W.

Jonathan: I'm very familiar with this. 3a, 4

Lars: and 5. Yeah. And also I think we have kiosk systems that pack in like a whole browser shower thingy. Essentially small compositor and cog, which is a nice embeddable browser. And those are also for the Raspberry Pi, mostly as sort of getting started systems.

But then there's also the BeagleBone Black. We support a bunch of different systems. OSD32, MP1. I, there's some IMX board in there that I don't know off the top of my head. There are also non sort of non officially supported ones. I have a couple of devices from TI's like Sitara processors. AM62 series boards.

And there are contributors that have developed those for, for their clients and made them publicly available. So that's kind of base systems and the fun part is of course If you have an application and you want to put it on a bunch of various Raspberry Pis, if your application doesn't rely on anything too weird, you can just pop it on a variety of them.

Like the, the system part is pre built and separate from your, from your application. You just plop the application down on top.

Jonathan: It's probably the application doesn't necessarily care about even what, what architecture. So, like, one of the things I've run into with the Pi is that most of them are Arm V.

Eight, I think it is. The recent ones are, and some of the old, there's a V

Lars: seven in there somewhere. I think it's the zero in the old ones.

Jonathan: The, no, the zero is actually a V six. So like the, the, oh, yeah, yeah. So that, that tripped me up for a while. I was, I was shipping binaries that weren't working for anybody because we were compiling for armed V seven and those are actually armed V six and that is not the same thing.

Yeah. Yeah. I, I guess with, with the elixir and nerves, you don't, you don't have to care when you're writing an application because all of that gets handled on the, on the OS side.

Lars: Yeah, so like, I know in the embedded space, a lot of people are like, yeah, I'll, I'll write this and see why would I write it in anything else?

But actually I would beg people to consider the fact that there's a very good high level framework for writing C it's called Erlang. So Erlang interrupts very well with C and as a consequence Elixir interrupts very well with C and with SIG there's also good interrupt with Rust and so on as well.

Jonathan: By nature of Rust working so hard to have good interoperability with C I'm sure.

Lars: Yeah. You're catching me at a weird time when I'm very upset at a Rust based library that will not precomp cross compile cleanly. But aside from from the ONNX runtime that I'm having trouble with, generally the Rust libraries have been well behaved. Right, right.

Jonathan: So I, I, I want to know a couple of different things and it's related.

And so I'm curious what, what, like what real work people are doing with nerves. And then I'm also curious, is there a, is there kind of a place where it makes sense to run nerves on x86? And it, it kind of seems like there that you might even do like. VM based microservices, right? Where do you have a virtual machine?

That's just nerves with, you know, whatever elixir application on top of it. I'm curious if people are using that out in the real world.

Lars: I know some people have taken the so we have a generic x86 system in the nurse project that you usually Like you can put it on almost any computer and it will probably boot and work.

But you, if you want a bunch of hardware support, you'll have to adapt it. Sure. But someone took that and hacked on it to get it to run on Vultr, which is a virtual machine cloud provider. So they definitely run nerves based cloud cloud instances. I've been. Curious to experiment with it, but I haven't.

Also like there's a bunch of industrial PC type stuff where you could definitely want to put nerves as a single. So the point of nerves is essentially like it should not really be running your desktop or anything, right? It might, it might work out, but it's not what it's for. It's Single purpose devices is kind of the best explanation, but essentially it's like IOT, IOT gateways, kiosks, that type of thing, where it's like this device has one job.

It might may need to do a lot of different things to do that one job, but it has roughly speaking one job.

Jonathan: Yeah. So, okay. On the other end of that spectrum, then how, how embedded can it get? You know, does this thing run on ESP32s? Does it run on any of the STM devices?

Lars: So, hypothetically, you could get it to run on one or two of those devices just because people have made Linux run on them.

Jonathan: Okay.

Lars: But you shouldn't. Fundamentally. Generally, I would say they have they're too resource constrained. Like the Erlang VM is not terribly, heavy, but like Linux prefers an MMU and a microprocessor, not a microcontroller. And we, NURFS is a thing that runs on top of Linux as it stands.

Jonathan: Has anybody, has anybody tried, and this is nuts, but has anybody tried something crazy like running, running the Elixir, the, so the Erlang VM on like free RTOS?

Lars: So there is a project called Grisp that runs that runs the Erlang VM on Artemis. And then they coupled in the FreeBSD networking stack. It's a cool project, but I haven't done anything with it. It's, it's a wild wild German. I think it's a professor. Professor. But yeah wild German engineering.

Interesting stuff. Per Stritzinger. But then there is also actually a project called AtomVM, which is a re implementation of the The Erlang VM for tiny, tiny embedded devices. So they, they managed to fit it. If the device has 400 K of Ram and they need like two megs of space, I think, for, for a small image, and then they can do a lot more if you have more than that, it's a very impressive project.

They claim it's not production ready. I think they should should experiment more in that regard, but it's very cool. I saw a talk about it in Berlin this year and it's a very cool project. Yeah. Yeah. That's neat. Not. As battle tested and mature as the virtual machine, which has been tuned for like 30 years.

So that's the trade off. I, but I would say like nerves is not particularly heavy, right? So smart is a company I, I work fair bit with him, Frank Oleth, who made nerves. Mm-Hmm. works at Smart Rent and they make like smart thermostats and ho smart home stuff for rental. rental properties. And they are probably the big, as far as we know, they're the biggest user of Nervs.

So it's around a hundred thousand devices is what I've heard. And their most used device is a smart thermostat, LEDs and stuff. It doesn't have a touchscreen and it's a 32 bit single core. All the winter processor clocked down to 325 megahertz. I think it shipped 256 megs of Ram and the U they have like a hundred makes despair.

And I think the overall image is like 30 megs on disk. Yeah.

Jonathan: I am reminded of some work that I've done with OpenWrt, which is very similarly, you know, it's an embedded Linux system. They can get down to very, very small targets. You know, back at one time, they fully supported running on boards with 4 megs of flash.

And, you know, ideally, was it megs of RAM, something like that, but just insanely small targets. And so this, this. Brings to mind an obvious question. Is anybody making routers with NURBS? Is that is that even possible? I know

Lars: some people have definitely explored it. I know a guy who does wifi somewhere somewhere in Europe, Austria, I would hazard a guess, but I don't think he has switched over.

He's still do doing open WRT, I think. And, but we heard. Like, randomly in the Elixir forum, this is, you never find out who's using an embedded framework because, like, it's just secretive industries and not quite as open generally as, as like the web world or whatever. But we found out Southwest Airlines uses NERVs for their in the flight Wi Fi.

Jonathan: Oh, cool.

Lars: On a bunch of their flights. It's like okay had no idea but of course, it's perfectly suited for networking like cisco does a bunch of networking using airline Yeah, that makes sense. It makes sense.

Jonathan: I could I could even imagine honestly I could imagine a mashup project that takes like the open wrt kernel and puts the the nerves runtime So the the whole the whole user space would be nerves and then you'd use all the kernel work The open wrt guys have done and you could probably make a really impressive little embedded device with something like that You

Lars: Yeah, and the way we build our Linux is just Buildroot.

Okay, very similar then. So, very straightforward. I don't,

Jonathan: does OpenWrt use Buildroot? I, it's, it's similar. I think it's sort of their own take on Buildroot, but I'm sure there's some cross pollination that goes on there. So what what, what license is all of this under? So if obviously if people are doing secretive IOT stuff, it's not going to be a viral GPL sort of license, at least.

No, hopefully not. That would be problematic.

Lars: I think most of NURVs is under Apache, but it's like Apache or MIT or some BSD class it's all fairly permissive licenses. So no, no real concerns for doing commercial work. Right,

Jonathan: right, right. Yeah although boy, it's interesting, you know, I'm sure you know this and like a lot of our listeners do too, but the fact that that, that Linksys used Linux and a bunch of GPL stuff is the reason that we have OpenWrt and a lot of this, right, because Linksys built the, built a router and they didn't.

Put Linux on it. And some, some smart user figured it out and said, Hey, by the way, how about you release all the sources that you're supposed to under the GPL. And to their credit, Linksys did it. And then a group of hackers hacked on it and said, well, hey, why don't we just compile this ourselves? And that's where open wrt came from and a lot of this stuff came from that

Lars: and that was how you fixed your blue and black little bug like Linksys router.

I used dd wrt at the time. Yep Yeah, it definitely made it work. Yes. Yes compared to the links firmware

Jonathan: and all people did some crazy stuff with that too Like what what if we desoldered the leds and put a usb port there instead and just bit bang the usb? And oh, it was crazy people did some crazy stuff.

Very impressive. Okay. So now what about what about nerves hub? What is is that sort of a repository for for packages or what's up with nerves hub? You

Lars: Yeah. So nerves hub and here's like disclosure. I'm trying to start a business called nerves cloud hosting nerves hub. Okay. But but I'm also one of the maintainers of nerves hub and I'm in the nerves core team since a little while back.

But yeah, nerves hub is an over the air update service, completely open source project for firmware updates. So you essentially set up your device. We have a, like the lightweight developer offering of like, yeah, you can just set a shared key on your device and can connect, but the good way to do it is of course, with some, some little device certificate chip.

So. There's a good library for the attack 6 0 8, which is a nice little cheap chip that everyone has seen at this point. But you can also do it with a TP except it

Jonathan: doesn't support the 2 5, 5 1 9 curves. Sorry, I, I have been, I have been fighting. There are so many upsetting

Lars: things about that. I have fighting many upsetting exact

Jonathan: problem for days now, but anyway, continue.

I'm sorry.

Lars: Yeah, . But some, something it does do is that it will, do a TLS handshakes with a reasonably private device certificate, which means we can use that for authentication and letting, and then letting the device connect and we can know which device gets which firmware that type of thing.

So fundamentally it's like Nervs Hub is a whole suite of things. It's like, there's some CLI tools for when you built your firmware, if you want to sign it and upload it. You can only use signed firmware because like this, this is not hobbyist grade. It's production grade. It's being used with real devices.

And you actually want to be sure that someone that the right, someone is sending that firmware. Yes. And. All of this is built on top of a small tool that Frank built not in Elixir, and it can be used for, for many other things, but it's called fwup. And one of the cool things it can do is that it can stream writes from like one of these firmware files that it packages.

It can stream the writes essentially like DD would. But straight to the partition you want to update. So NERVs by default ships A and B partitions for switching back and forth and being able to revert a bad firmware. Yeah. Resiliency. It keeps coming

Jonathan: back to resiliency IoT, right? You, when you're doing IoT devices, you never want to have to tell somebody, All right, take the device apart.

Connect to the serial port and then, yeah, just, it's just like, yeah,

Lars: we're going to have to send a technician out to 10, 000 devices across the U S like, no, thanks. I also know a company that does nerves that have power meters across Africa. Getting to one of these devices might take a week. Yeah. Yeah.

Yeah. So important to be able to revert and also important to be able to operate under. Non ideal network conditions. And it's also kind of important to not use too much memory or too much disk space when you're dealing with IoT devices. So what FW up will do by default is it will stream the update.

and only use a fixed amount of memory and no disk while it is fetching, fetching the update. And it will just write them at the pace it can get from the network and like whatever throughput bottlenecks it might have. It will write that to the new partition. When it's done, it will do the switch over, but that means it can sip through a straw if that's what it has, or it can do.

Do it fairly quickly in many cases.

Jonathan: And I assume with, with checksumming and all that built in to make sure that you don't Yeah, yeah, yeah. Flash bad code.

Lars: Yeah. There's a bunch of Blake checksums in there and all that stuff. Yeah, so FW up is kind of the underpinning there. But then of course, Nerf's hub is built with Elixir because Elixir is very good at building scalable, fault tolerant web services.

So You want to run a couple of hundred thousand web sockets. That's not really a problem. And actually there was a blog post very early in the Lexer's history. Excuse me. Where Joseph Lim and Chris McCord, so Elixir's creator and Phoenix creator, dear, my throat is just seizing up. That's fine.

They wanted to see how far they could push the web socket implementation, Phoenix channels. They ran out of big beefy boxes to try it on. But essentially they, I think they scaled it on one machine to 2 million. Oh wow. Active chattering web socket connections. Wow. Without too much issue.

Jonathan: That's impressive.

Lars: And that's kind of the, the scale, like the scalability we're building on top of, and so there's web ui, there's ACL I, there's APIs, and then there's a lot of shipping firmware.

Jonathan: So Nerves hub, is it, is it available to hobbyists if I've only got a couple of devices. Yeah. Does it make sense for me to, to, to get on board too?

Lars: So you could set it up. It's a Docker container. It's very straightforward to set up. Okay. Or you could sign up for Nerves Cloud and use like our hobbyist plan, which is essentially 10 devices. Do whatever you. Because like the real customers in this case is not a hobbyist, it's, it's someone who has a couple of thousand devices.

At least, yeah.

Jonathan: I, I imagine that hobbyist here is also nice to be able to give those potential customers a way to do a test run. Yeah, for sure. Works out pretty well. Cool.

Lars: Yeah, so it's a, it's a clear differentiator. Like, There are many over the air update tools. There are also a lot of assorted, sort of, Embedded frameworks, but most of them don't come with any opinions.

It's like, if you lock, look at Yocto Linux, like what is the opinions of Yocto? It doesn't have any, it's supposed to be general. Buildroot is slightly more opinionated about how to design the whole thing, but it's fundamentally also very open ended. It's just like, here, we will help you get a Linux and some stuff on it.

But nerves is very much like a whole package. It's, yeah, it's actually a good starting point for embedded. I didn't come from embedded, but I'm learning a lot about how to do embedded devices based off of the fact that there's a lot of experience written into the framework and written into the tooling.

Jonathan: Right. So what does it look like if somebody wants to bring something from outside of the Elixir world onto a NERVs device? So the example that you give in our notes here is Python, but you know, there's a, there's even a world outside of Python that, you know, people have stuff that they want to do. So like, can I run a Java virtual machine alongside the Erlang virtual machine on a NERVs device?

You know, can I do Python? Can I do Perl, Flutter, you know, whatever, is there is it possible to run that stuff alongside NURBS?

Lars: Yeah, so you have a bunch of different And some things are easier to run than others. Like, I don't know how much work it would be to get a decent JVM into a Buildroot project.

I've never tried. I don't particularly want to try.

Jonathan: If somebody pays you enough money, I'm sure you'd be glad to, right? I would do it. Yeah. That's the way that works.

Lars: But there are also some blessed paths. Like if you want to do. If you want to integrate Rust, there's a few nice ways to do it. Sure. As I mentioned, C and C interop, pretty straightforward.

Then there's cursed things, like there's a community member called named Coco, and she has implemented an embedded Python, like you run Python as a NIF under Erlang. So essentially you have Python in your Erlang virtual machine. And that's cursed, but it also seems to work pretty well. It's very early days for that particular experiment, but you have a lot of options.

You can also just I mean, if you have Go, Go code, Go makes very nice little binaries. You plop that in just as a file and you can start it easy. But then of course, if you actually want a proper Python. In your system, Buildroot has that, like you can just, you'll have to modify your system usually, is the answer to running more complex things.

Jonathan: It sounds kind of like anything that you can do in Buildroot, you can actually make nerves do as well. Yeah, pretty much. And honestly, again, I have experience with OpenWrt's flavor of Buildroot, but I'm pretty sure it is similar. It's not that difficult to add stuff to Buildroot. You just, you know, you write, you go find somebody else that did something similar, you take their makefile, you modify it a bit, and hey, Bob's your uncle, you got it working.

Yeah, defconfig,

Lars: defconfig, defconfig. Yeah. It's been a wild ride for me since I, like, I've run Linux since I was a teenager, but I've Not really. So for one thing, like actually engaging actively and contributing to open source was something that I started like seven years ago when I got into Elixir because the community was very good and since I've started poking around with nerves, I've had to pick up like kernel configs and def configs that I Just like, I've probably poked some of that when I was doing Slackware and my audio wasn't working, like, but not, not really, I didn't know what I was doing at all.

And now I, I do have a patch for a build root thing where like the WPA supplicant couldn't be configured the way I needed it to. Like that's, that's an interesting journey. It's like get email patches and all of that.

Jonathan: Yeah. I, it's been, it's been years ago, but I when I was messing around with OpenWrt a lot I, I found this edge case where if you were trying to do a compile from scratch, but a multi core compile, the OpenWrt builder would just fall on its face during compile.

And so I've one day, I cleared off some time and I, you know, I troubleshoot it and really dove into it turned out that because, you know, it, it, it builds GCC to be able to build GCC to be able to build the final GCC. Right? It's, it's like cross compiling. That's the way that tends to work. And it would start building GCC and then download the next copy of it and then unzip it over the copy.

It was already downloading. And that was, that was the problem. That seems rude. Well, it, it was broken. It was causing problems. Yeah, the compiler thought it was very rude. But it was one of those deals where it's like, well, once you understood the problem, oh, that was trivial to fix. You could just go in and say, well, don't actually compile GCC with with multiple threads.

And so I've got, I've got that fixed. It's been an open WRT for like a decade now. And it's, it's really, it's really fascinating to me. Like this world of open source that we're in. How easy it is for someone to, you know, if they find a problem like that, to jump in, figure it out, and then, you know, you, you email the patch, and here, you know, here's the fix.

Or, you know, these days, you can just make a PR on GitHub for a lot of these projects. And something that's always really fascinating to me is that it's so it's, it is so accessible for people to jump in and, and make a fix like that, and, and Make the world just a little bit better place, right? I love it.

Yeah, it's beautiful. It is.

Lars: It is. Now you talked about multiple cores. That's kind of one of the sweet parts of running on top of Erlang. So a lot of Embedded devices these days have a few cores to play with. And because Erlang was built to do concurrency and to be able to, like, share nothing, immutability, and also being able to run workloads in a distributed fashion, when multi core came, they were like, if we start more schedulers.

Maybe we can just use all the cores. Of course, it was more complicated than that in truth, but essentially that so by default, when you start Erlang, it will start one scheduler on a thread per core. So like your Erlang program will run your number of cores, number of threads, and then it will distribute the, these like, well, green thread type of, they're called Erlang processes.

across these and like in our most Erlang systems can manage like millions of these if you ask it to but the max parallelism is of course the number of the course you have right but But yeah, so adding more cores is kind of straightforward. So scaling up your, your cloud like, Oh, let's do 32 cores.

And it will actually just use them all. This was not my experience with Python.

Yeah.

Jonathan: Most languages, you've got to work fairly hard to be able to do that. Yeah.

Lars: Yeah, no, it's it's kind of ridiculous in many ways, but the, the probably nicest thing for, for example, in IoT for this is like, Oh, we are downloading a firmware update, but at the same time we're serving the user what they need.

And because it's like preempted multi, multitasking doesn't even show up to the user. Yeah. Because. Nothing can block the the scheduler for very long before it gets kicked to the back of the

Jonathan: line. Cool. All right. So let's say somebody listens and they go, man, Elixir and Nerve sound really good. Where's a good place to go and learn, let's just say Elixir first, so like if I want to dive into the world of Elixir, where's a good starting point?

Lars: So I think the Elixirlang website has a good starting documentation. There's also Elixir School, there's also oh, well, I don't know if exorcism. io is still around. I think they might have folded. They had a very good Elixir track. I have a, I have a good overview, I would say, of Elixir. It's not like, this is how you learn the language.

This is more, it's more, this is how the language works. And this is how the runtime works. But I have a blog series called Unpacking Elixir. And that might give a good idea. I've also given a talk that can, that's kind of a good, slightly longer overview of Elixir, LiveView and all the assorted cool things you can do with it.

But there's a lot of good resources. So Elixir, the language itself, I would start from their website. Then there's the Phoenix web framework. Of course, if you're in the web space that's probably what you want to pick up next. Also good documentation like Elixir and Phoenix. came out the gate with just really good documentation and the documentation tooling for Elixir is very good.

So the whole culture is essentially documented well because we have very nice docs and you like it when the docs are nice. So when you write a library, you better make the docs nice or you will be ashamed. You will feel shame. Yes. Yeah. So I would Just essentially dive in there. And if you want to try nerves, there's a few nice ways to do it.

There's nerves live book, which is a cool sort of interactive thing. You can put on a recipe pie and then open in the web browser and tell it to do things like you can. flip the GPIOs from, from essentially a code notebook or tell it to find your Wi Fi from a code notebook.

Jonathan: So you can, you can write, you can write Elixir code right in the browser then and get it to run kind of live.

Yep. Very cool. Yeah.

Lars: Livebook is also good while you're learning Elixir and you can run it as a desktop app. That's what it's usually Livebook is a bit of a special one. Then there's the Nervs Quickstart project, which is also like just Nervs packages. Bundled up with most of what you would need. And there's some, some decent documentation around for, for all the different parts of Nerve.

So it's just again, and like. There's a good forum, the Elixir forum. We're not typically on Stack Overflow as a community. So there's the Elixir forum and the Erlang forum and they've, they've served us very well. And then there's a Slack and a Discord of course, so. If you start poking around with nerves, dive into any of those and you'll find me right there.

Jonathan: There you go. Goodness, I don't know if I can handle programming in a language that doesn't have all the answers on Stack Overflow.

Lars: But you have all the answers on a real forum.

Jonathan: There you go. So I would be, I would be remiss if I didn't ask on the, on the Raspberry Pi does, do, does nerves have support?

You've mentioned GPIO, but there are a bunch of other sort of interfaces on the Pi. Things like SPI and I2C. You mentioned the ATEC device, right? The little crypto chip, you know, that's over I2C.

Lars: Yeah, so there are libraries for I2C, GPIO, UART, SPI. My first library that I The way I got into nerves, I think we sidetracked from that was that I was trying to get an E Ink display, the Inky from Adafruit, I think running.

from Elixir instead of from Python. And since I'd done Python, I could read the Python code and try to bash my way there. I got pretty far. Then I got a lot of help. The tricky part was like the, the spy stuff and the LUTs and like, oh, a display is not the best part to start, especially not E Ink.

They're kind of complex. Yes, they are. But it worked out. The library works. Cool. And that's it. Like that was GPIO and I2C and SPI, I think, at the same time. Yes. And that was six years ago. And like I have a couple of these like more serious boards. Someone recently ported. Well made a system for this cute little thing.

So this is the M five stack. Mm-Hmm. , which is an adorable little yeah, little device. M five. I don't have an SD card in it right now. Otherwise it would show the lo nerves logo when it booted. But there's a German company that has been working with the Seed Studio re terminal dm. Mm-Hmm. , which is a raspberry pie fundamentally, but it has a lot of nice things on top of it.

So. I've been playing around with that. So this is like Phoenix and live view for listeners. It's an industrial kiosk that you can buy. It's kind of pricey, but it's also IP certified. If you do a proper enclosure, it's like meant for industrial. I also got it working on the clockwork pie you console, which is this adorable little data pad looking thing.

And yeah, people get it running on all sorts of things. So. If you have some hardware that could run Linux some people show up with a, like an ESP 32 and get very disappointed. Yeah, if it can run Linux, generally, you could put nerves on it. It might be a challenge if you're not sort of used to the Linux bits, but generally generally, it's very doable.

Jonathan: Yeah. Yeah. Interesting. It's, it's fun too. You're working with some of the same brands and companies that we are. One of my, one of my side projects is mesh tastic, which is Oh, putting Laura radios on. Some of those things. And I've

Lars: heard of mesh tastic. You're involved in developing it?

Jonathan: Yes. I, I'm, I'm the Linux guy over there, so we now, we now run the firm.

We can run the firmware as a Linux binary, and I, I made, I made that work with real hardware, so that's my claim to fame over in that system. You now have me thinking that, hmm, we ought to see if we can get our stuff put into Buildroot, and I think we may already have a user trying to do that. So, you know, at some point in the future, maybe there'll be an Elixir front end to be able to talk to the Meshtastic binary running on your, you know, little embedded device.

I mean,

Lars: is it just a binary that does most of the radio?

Jonathan: Yes. There's, there's a, there's a couple of there's a couple of libraries that have to be there. Like lib, lib yaml C is one of them. And then there's some kernel stuff that's got to be there, but I can't imagine that would be too difficult.

And it's, it's pretty much all contained in a binary after that.

Lars: Yeah, and like, yeah, the way you do things when they're not in Buildroot is you just add your patches and you add, you make your own Buildroot packages like there's endless ways to get code into your nerf system, because that's necessary if you're doing professional work, it's not like, Oh, no, I'll have to wait.

Eight months for this to be accepted by Buildroot. I refuse to run patches.

Jonathan: Oh yeah, indeed. Indeed. Okay. So is there, is there anything that we did not get to? We, we've been going for, I think over an hour now, which is fine. Is there anything we didn't get to that you wanted to make sure and let folks know about?

Lars: Oh, there's, there's a lot there. I feel like we covered

Jonathan: a lot too, though.

So,

Lars: yeah, I mean, I need to do an honorary mention of like the membrane media framework. This is something I don't see in a lot of ecosystems. There's a company in Poland, like a consultancy that specialized in media streaming, and this, this means that. Instead of trying to make GStreamer do what you want it to do, you actually build your pipeline with Elixir data structures and like the the actor model of Erlang and do a bunch of message passing.

They're of course calling ffmpeg and libmad and whatever you need because That stuff's horribly complex, but, but you can operate all of that from Elixir. And if you've ever done like media streaming and also wanting to tell people what's going on live in a web UI, that's kind of complicated. Yes, it is.

But I very recently, well, a year ago or so, I gave a talk where I was controlling the entire slide. Show with my voice. So that's the elixir machine learning effort is also very interesting, but we, I don't think we have time for it, but membrane was my media pipeline for that. Getting my voice from the browser in processing it, sending out like.

Audio measurements that I could make a waveform from, and then shoving it into a machine learning model and getting everything live into a web view. And this is like, you just stitch these things together and everything's just message passing. Like everything follows the same paradigm. This is. This is kind of the wild thing, whether you're doing a web UI, whether you're doing a sort of a, the internals of an embedded device, whether you're talking to your database, if a thing can tell you that something is happening, like interrupts are lovely in Elixir because.

events that can trigger messages can do such nice things across your system. It's not like you have to poll anything. And that's, that's kind of the ideal. Sometimes the hardware won't let you get away from polling. Thankfully it's not that hard to do polling yet, but yeah, it's a, it's a big ecosystem with very, very particular things going on in it.

I think it's always worth a look. There's a lot of good talks about it as well. There is one particular talk that I think has flipped a lot of people to trying it, which is the soul of Erlang and Elixir by Sasha Yurich. His book Elixir in Action is also a very good starting point for people that are already experienced developers that just want to pick up the language and the concepts.

Sure.

Jonathan: Where can, where should people go to find your work in particular? And the Nervous Hub as well. Plug the URL for that.

Lars: Yeah, so go to underjord. io. So I, hopefully this will be in the show notes because Cause I have no idea what you just said. Underjord you don't, you don't speak Swedish? No.

See,

Jonathan: I read that as underjord.

Lars: Yeah, yeah, you would. The Americans do. It's fine. But yeah, I would search for, if you want to find Nerves, I would search for Nerves and Nerves Elixir, not just Nerves. Yes. It's a common word. You get a lot of neurological stuff. But you can find it. NervesHub, Nerves, it's all out

Jonathan: there.

Yeah, we'll make sure to get it in the show notes. Alright, before I let you go, I've got two final questions that I am required to ask. And that is, What is your favorite scripting language? And I suppose you can, you can talk for a moment about whether Elixir counts as a scripting language, scripting language and text editor.

Lars: Yes, Elixir is my favorite scripting language. It used to be Python, but there was actually a very important, small, small, but important facility introduced in Elixir where you can just write mix install at the top of your script and state which dependencies you wanted to pull.

Jonathan: Mm

Lars: hmm. And having a script that can fetch its own dependencies.

It's pretty cool. It's like, okay, if I have Elixir installed, the script can do the rest. And there's a good example repo of like full of sort of one, one file, do something weird with Elixir. Like start a web server or whatever that, that you can look up, but yeah, I, I generally do my scripting in either bash or elixir depending on sort of how evolved it gets.

Can you do

Jonathan: a, can you do a splat bang elixir to actually run a script?

Lars: Yeah, yeah, yeah. Cool. Userbin and elixir is the usual thing. I was like splat bang. I was like shebang is what I've heard. But I thought you were going to, into Python. And my mind was going to Python because that used to be my scripting language.

It was like splat and double splat and all that. And Python is still a mighty fine scripting language. But actually the dependency management is hell.

Jonathan: Yeah, it has problems. And then text editor.

Lars: So I like Vim or Neo Vim the most but I use Visual Studio Code the most.

Jonathan: That is a very common answer, that exact combination of things.

And yeah, I find myself in a very similar, very similar case. So,

Lars: yeah, so usually Vim mode in VS code is, is how I get the bulk of the work done.

Jonathan: Yep, that makes sense. Alright Lars Wickman. thank you so much for being here. It was absolutely a blast and I got to learn about Elixir and about Nerves, two things I knew very, very little about, and now I feel like I at least have an idea of what they're about.

So, thank you.

Lars: Oh, thank you very much for having me. It's it was a fun time and happy to come back and clarify anything. If you get the weird questions about it.

Jonathan: Yeah. Yeah. All right. And so, yeah, a lot of fun. I do want to plug a couple of things before we let everybody go. Of course, first off, we've got Hackaday.

You can find my security column. Goes live every Friday morning there. And of course, we appreciate Hackaday being the home of Floss Weekly. Over at twit. tv, there's still the untitled Linux show. We record those on Saturday afternoon. And they go live Sunday or Monday, just depends upon when things get finished up.

But you could follow my stuff there and we sure appreciate it. Thank you everyone that watched us both live and on the download and we will see you next week on Floss Weekly.

This week Jonathan and Lars Wikman chat about Elixir and Nerves. That's a modern language that's a take on Erlang, and an embedded Linux approach for running Elixir code on devices.

• https://underjord.io
• https://elixir-lang.org/
• https://nerves-project.org/

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week, Randall Schwartz and Aaron Newcomb join me and we talk about Linux, and some geopolitics, and some open source challenges. It's a great show, you don't want to miss it. This is Floss Weekly, episode 810, recorded November 19th. A rising wallet pays for all boats.

It's time for Floss Weekly, that's the show about free, libre, and open source software. I'm your host, Jonathan Bennett, and today is a little bit different. We do, of course, have a co host. We sort of have two co hosts. We've got Rundle Schwartz. We've got Aaron Newcomb and I'm not sure which one is the guest and which one is the co host.

But it is, it is great to have both of you here today. Yeah. Great to be here.

Randal: It's like the, what is it? The three sort of Sopranos, not three amigos,

Jonathan: three Sopranos,

Randal: three amigos too. But I was thinking, isn't there like. Three opera singers.

Jonathan: Oh, three

Aaron: tenors. Yeah.

Randal: Three tenors. There we go. Not sopranos on the wrong band.

I'm more of a baritone actually.

Yeah, yeah. Me too, actually. So, and as I get older, my karaoke goes lower. So

Jonathan: instead of doing Kermit, the frog, you're going to eventually be You, one of the one of the big animal boy, I don't, I don't even know the Muppets well enough. Animal! Animal! Yeah, that's

Aaron: right. Practice your drums. Yeah, there you go.

Randal: Something like that, yes. I'm going to regret this video already. This is great. Your background

Jonathan: removal tool, I hated that, by the way.

Randal: Oh, oh, I'm sure, I'm sure. I wasn't even looking. I had my eyes half closed. Clothes like animal. That's great. That's great.

Aaron: It's either that, or we're just going to be the, the, the two guys that sit in the back, the two old guys, the two grumpy old guys that sit in the back.

Oh

Randal: yeah. Waldorf and Statler. Oh yeah, absolutely. That's what I feel like more and

Aaron: more these days. I'm

Randal: already doing that. I'm already, I'm already channeling them. Sometimes when people ask me questions, I act like one of those guys. I don't know which one brought both of them.

Jonathan: Yeah, that's a little too real, man.

All right. So, hey, there is, there is some news that I have came across this week and I think it's something that's worth talking about. We'll see where all this leads us. The, the main, the main bit of news though that I thought was interesting is that there is a, there is an Ubuntu, I guess it's a derivative, it's a repackaging of Ubuntu.

It's made specifically for Rockchips, which those are the things in, Well in a lot of here. I've got one here actually you see them in little devices like this And so this is a well, it's a it's it's kind of the equivalent of a Raspberry Pi CM4 Yeah But the the SOC so the system on a chip that runs this is a rock chip and in this case It's the RK3 3566 and we're sort of in this weird world where Rockchip does a little bit of work on like pushing things upstream to the Linux kernel.

In fact, I found out just the other day that there are a whopping, I think it's three kernel maintainers that are at rockchip. com. Right? So, like, it's a little bit better than I thought it was. You have some companies that they'll push hardware out. And it's like, eh, if you want to put it in the kernel, that's fine.

But, if you want to run anything official, here's our kernel from 10 years ago that has all of our patches on top of it. Oh, I hate that. Rockchip is doing a little bit better. But when it comes to actual distros to put on it, one of the real leading ones was the Ubuntu Rockchip distro. And that was run by a guy named Josh.

And a couple of weeks ago he put out an announcement that says, I'm not going to be able to do this anymore. So he's frozen the GitHub, he's frozen the Discussions on it. There's no more releases and yeah, and it's it's it's not super great and one of the things that people have pointed out about this is It's, it's a real problem for all of the people that use this, because RockChip doesn't put out a distro, and a lot of the, you know, like, so, so this is a, what is this one, a Banana, something by BananaPie maybe?

I forget. Or no, this is the Pine64 board. Oh yeah, I've got a

Aaron: Pine64 sitting up here too.

Jonathan: Yeah, so this is one of Pine's boards, and it's, it's, again, it's in that little CM4 format. And a lot of these companies don't put out a distro either, they don't put out images. It's just, oh, go grab the the, the Ubuntu image from from Josh Reich.

And that is now frozen and it's not, it's not great. I'm not sure what direction we go from here.

Randal: So, again, I don't know a lot about Linux, which I've been saying over the decades, I really don't know that much about Linux. Is it that the special Rockchip Linux distribution has like extra drivers to support unique hardware?

I mean, what's missing in the standard Ubuntu distribution? You can't just plop it on there. Yeah.

Jonathan: And that's a, that, that is actually a really good question. I've spent some time over the last few weeks figuring that out. Good question. Because it's not, it's not immediately apparent what the answer to that is.

Maybe. Ah, and I think there's two main things that's, that's different about this. One is it's got U boot built right into it because a lot of these devices you actually flash U Boot. And U Boot then boots the kernel. Well, in some cases, you've got to flash U Boot with something called a device tree that's specific to the device.

And so, you know, you've got that where you've got to build this kind of special image that's got U Boot at the very beginning of it, the right device tree, U Boot, and then your kernel after that. And you've got to have the right things turned on in the kernel, right? Because when you build Linux, there's a whole bunch of modules, there's a whole bunch of compile options, like, do we support this piece of hardware?

Do we support that piece of hardware? Because you have to imagine if you just turn every switch on, you get really, really big kernel binaries. And so one of the other things that this project was doing was, you know, these are the boards we support. Let's make sure all of the different switches are turned on for the images.

And then one more thing I know that it does is there are some like out of tree patches. Where, you know, let's say HDMI for one of these particular boards. Well, that hasn't landed upstream in the kernel, but there is, there are some open source patches floating around. That's like, well, you can patch the kernel with this and this and this, and it makes HDMI work.

So he would, he would integrate some of those things as well. I've got another bore, the, the, this one's by TuringPi, it's the RK1, which has another of these RockChip chips on it. And while, while resear I'm trying to do a review on it for Hackaday, actually, and I kept running into the problem that, like, Trying to boot Linux on it was fundamentally broken in one way or another, unless you were using, you know, this specific Ubuntu image.

And I finally got cobbled together all the patches that you boot needs to be able to then boot like Fedora. So I have one of these now that I can boot Fedora 41 on just the, the regular image and it finally works. But getting to that point, it's just been a pain.

Aaron: I mean, isn't there just, I mean, so I fell away from, from.

Ubuntu, especially on the desktop a long time ago when they stopped supporting Gnome and they came up with their own thing. And I just don't like the, the whole, the whole paradigm of Ubuntu right now, but it is pretty stable. So I use derivatives that are based on Ubuntu or I just run stock Debian, right?

Which Ubuntu is based on. So you've got Debian, then you've got Ubuntu, which is built on top of that. And then other like Linux mint and other popular distributions built on top of Ubuntu. So isn't there, can't you just run Debian? Is the problem just that nobody's, nobody's working on all the special things to get the rock chip and all the peripherals to work correctly?

I mean, it seems like, I mean, this is one of the beauties of Linux actually, right? So if this was like Windows or, or Apple saying, we're not going to support this version of Linux. This Mac book anymore, right? Like that would be bad, right? Cause then you have no recourse, but it's Linux. So one of the things about why should I run Linux when we talk about that a little bit is that, you know, someone else can come along and build it.

Like you have started to test things on. Right. And because it's open source, you can actually come up with something. So I see this as a setback. But I don't see it as the end of the world for Rockchip or, you know, it's a pain in the butt basically for people that want to run the latest and greatest because now they have to run whatever the last version was, go get Debian and build the rest yourself.

Also, the other thing I would say is on these boards, I know, at least for myself, having worked on these boards for the past 20 years or whatever. I very rarely run those boards with a GUI with the desktop anyway, because I want those boards to go do something. I want those boards to go run my lights. I want those boards to go run my 3D printer.

And so very rarely am I running a GUI anyway. So as long as I can get to the command line, as someone actually, as long as someone supports all of the peripherals on that device, I don't really care because I just need to get in there, get to the command line and get something done. That's, that's my take on it.

That's, that's my quick take. Yeah. Not so quick.

Randal: So, so this, that sort of begs the question for me is there not a community around this fork of Linux? Does he not have people that are talking to each other about the distribution that he's no longer maintaining?

Jonathan: I, I know there are people talking because like I am one of the people talking it's just okay.

It's it's early enough that it's unclear what's going to happen here. So one of the, one of the frustrations that Josh had is that he said, I've been trying to get in contact with rock chip for a very long time now. All of the vendors I've talked to have not been able to assist me. I need access to the latest SDKs to be able to fix things.

And then also there was another, there was another Oh he's using GitHub and so each of his artifacts, which is essentially your release binaries have to be under two gigabytes on the free version of GitHub. So like there's, there was beginning to be some need for financial backing for this.

To really make it make sense for people. And so those were a couple of the problems that yes, other people could step forward. And I'm sure, I'm sure somebody will, right. There are some sort of endemic problems that just more volunteers are not necessarily going to fix.

Randal: You would, you would think that Rockchip having a financial interest in pushing these boards out would have an interest in making sure that.

Linux runs on them.

Jonathan: That, that is my take on this. I, I really feel like Rockchip needs to step up. That seems really obvious. It does. It does seem obvious. Yeah. But obviously not all of these companies think about things the exact same way as we do. You know, you, you could imagine that, boy, it would be nice if, like, if we were just describing what we would like the future to look like, it would be great if there was a working group put together that was Rockchip and, you know, Pine, the Pine Store Turing Pi, and every one of these other guys that make one of these boards based on RockChip.

Like, put a working group together, and your thing is, you push things upstream to the kernel, and then build distro images for people to use. Like that would, that would solve the problem. But, you know, there's corporate interests are not necessarily always aligned with the interests of the open source community.

Aaron: And sometimes it's up to the Oh, sorry, go ahead.

Randal: I was just going to say, so explain the hardware relationship again. So Rockchip is a company making some part of this. They make the CPU, the SOC. So I guess I should define And that gets used in people's, people's small devices somehow.

Jonathan: Right, so we should, we should define What an SoC is, it's a system on a chip, which that's actually, that's what CPUs are now.

Like AMD kind of went in this direction and Intel has followed. So everything is an SoC, right? It's a system on a chip. The idea is that that is your, your CPU and what used to be called like the North Bridge and the South Bridge. It's now just all on one piece, one piece of silicon. Or one, one physical chip, I guess, anymore.

It's not necessarily one piece of silicon because they have chiplet designs, which is fun too, but that's not the point. Anyway, so they make if somebody's watching the video, like the, the, the black blob right in the middle of this is the CPU. Rockchip makes that thing. And then you've got the rest of the board around it.

And in this case, you know you've got probably a Samsung memory module on there and maybe a you know, another chip by rock chip. So I can't quite read all of these, but you know, there's one, two, three, four, there's six. I can't read any of

Aaron: them right now.

Jonathan: Yeah. Well, yeah. Yeah. Close one eye and tell me how much you can read.

Hopefully not this one. Right. Oh, I

Aaron: can't

Jonathan: see anything. So, you know, you've got, you've got the people that make the silicon and then an integrator will come along like again, in this case, it's pine, the pine store, pine 64. And they made the, the whole board and then this will slot on a carrier board somewhere.

Well, so to be able to get Linux to boot on this thing and know where all of the ports are for the kernel, like to even know how much, what hardware is there. You've got to have some it's called a device tree. And that's basically just a list of here's the peripherals that are connected. So it tells the kernel what drivers to load and where to find them.

And so somebody has got to maintain that. And that's sort of the piece that's missing is getting that maintainership. So

Randal: it sounds like there should be a cooperative amongst all the people who are Rockchip integrators.

Jonathan: Yes, and I think there is a little bit, like ad hoc, but definitely nothing formal.

Yeah,

Aaron: usually it comes down to the individual manufacturer of the board. Like OrangePi does a pretty good job with their boards. They always have, now it's not as nice as RaspberryPi, I'll get to that in a minute. But OrangePi, at least, you can go to their site for just about any board and you can download their, they will build as part of their process, they will build a distribution that works for, you know, or a bunch of different distributions at the time they release the board that work.

So they'll have their own orange, OrangePi. org. I forget what it's called. Orange Ubuntu based thing. They'll have Debian. They'll have Armbian usually. And, and usually Android as well. So they'll have like those four or five. They might not get updated, but they'll be there. And at least you can pick one to run and maybe you can update it yourself.

So they do a decent job, but the best one out there, of course, that does this is, is Raspberry Pi and Raspberry Pi owns the whole chain. Right. They, they own everything from the Silicon up through the operating system, including all the boards. And, and so they, that's why when someone asks me, Hey, there's this new orange pipe board that has never, you know, they never used Raspberry Pi or anything, but they're like, I'm trying to figure out if I should use orange pie or Raspberry.

It's like, no, just go get the Raspberry Pi because. There's the community there, you know, you're going to get a, a, a semi modern operating system and may not be, you know, last night's release, but it'll probably be last month's release and, you know, you're going to be okay. You're going to get support. You can go to the forums, ask people how to do it.

I, I don't, I haven't even bought the latest Raspberry Pi, but I don't need to because I understand how to do this on other things. But for someone that's just starting out, use the Raspberry Pi. And the reason is exactly this problem. It's exactly this problem.

Jonathan: And you run into, you run into edge cases on the other boards.

Like I've, I have more than once been running one of these not Raspberry Pi development boards, and it's like you install updates and it pulls a new kernel, and guess what? That kernel is from the wrong repository and your board no longer boots. Like that is, that is distressingly common.

Randal: Yeah. Oh geez.

Yeah. Yeah. I don't have these problems with my Mac.

Jonathan: Yeah, well, so, I mean, it's, it, I guess it's, it might be worth kind of talking about, like, why we, Aaron and I, have gone down this rabbit hole, like, what are the things that we want to be able to do with these, with these boards, and the, the two big ones for me, and I've talked about this before, the Raspberry Pi blew my mind, because suddenly I had a little computer running Linux that could talk to the outside world via hardware, So And not just like video on a video camera, it had, it had GPIO.

And so then you can know, you can switch lights on and off. You can wire that into a garage door and open your garage door from the command line terminal. And, and then, you know, the next step on top of that, of course, is you can write a webpage, which calls a command line on the back end. And you can hit a button on a webpage and open your garage door.

The, the thermostat Not while you're driving. Right. You can do it while you're driving. Never. I would, I would never

Aaron: do that while driving. I do it all the time because then the next step from that, the next step after the webpage is you hook it into Google Assistant or Siri or any of those. And then while you're driving, as you're almost home, you say, Open the garage door and you pull in and the garage door's open.

Randal: Yeah. And you just triggered 72 people at home by saying those words. Yeah. Well I didn't say key. He didn't actually say the word. I just said the Oh yeah, I was listening for that. Gotta be careful. Oh wait, I turn my phone over right now. There we go. Alright, now it won't respond. Thank God, .

Jonathan: So like the, the thermostat at my house.

is a Raspberry Pi with a board on the back of it to be able to do it's not actually relays because relays have problems because so it's a it's a solid state relay, but it switches the air conditioner and the heating on at my house. And so I've got a little script that pulls the temperature and switches on whichever one is needed.

And That's been, that's been running now for me for like five or six years, pretty successfully. And so those are the sorts of things, like those are the things that people might want to do with one of these. HTPC, that's another one. Home theater PC. Right? And, and I know people use Raspberry Pi for that a lot because you can do things like you can, you know, emulate your old video games.

You can watch movies, you can have YouTube on it, all kinds of stuff people use it for that you don't run Plex, right? Do you run Plex or Kodi on it? Yeah, absolutely. Cool. And so, you know, there, there's, there's a lot of interesting things there. One of the things I've been doing with them recently is GitHub runners.

So I've got, again, the Turing Pi, I've got it right over there, that's why it's on my mind. It's got, so it's got four slots on it, and I've got four RK1s in it. And I've got two of those set up as remote GitHub runners. And so when we need to be able to build, you know, ARM32 or ARM64 builds on GitHub, it can just, it can farm out to those and do it.

Because, you know, GitHub doesn't yet offer. on their free tier in any ARM builders. So, there there are some places there where it really makes sense.

Aaron: Yeah I always tell people if you're a, I used to be a strong advocate for Linux desktop. Like I used to tell people, oh yeah you gotta ditch Windows and run Linux. Well, not so much anymore, right? I mean, Linux really comes into play, as you've been describing Jonathan on those small boards of course. Linux also comes into play if you're a developer.

Pretty much anything you're going to do in the cloud, right, is going to be Linux based. It doesn't necessarily have to be, but you're almost maybe 90, 95 percent of the time you're going to be distributing your applications out to a Linux based virtual machine, which is running You know, Kubernetes and containers, but the base of that is going to be Linux.

So for developers, I've always said, even when my son was going to school learning software development, I was like, you need to at least have a good understanding of Linux. You may not have to support it, but you have to understand the concept so that when you talk to your DevOps person or your support person and try to figure out why your application isn't working, and they tell you that there's a kernel bug or there's a kernel panic.

Or something like that. You need to at least know what that is. Like you don't have to support it yourself, but some developers do support themselves. So I think like, it's not for grandma. It's, it's not for the person that's never used it before. It's not, I wouldn't even say you could use it for, it's good enough to use for, for your daily driver.

I D I used to at work. I've got two of them here

Jonathan: that are Linux boxes that are my daily drivers.

Aaron: But you know, sometimes, sometimes you're not, and I know plenty of developers that obviously I think probably the number one OS for developers is, is they run on Macintosh, right? So. So, yeah, I think, I think that's one really big area is just to understand, you know, if you have an Android phone, you're, you're running Linux already.

You just don't know it. You know, if you're, if you're using an application on the web, it's running on Linux, probably somewhere in the cloud and you just don't see it. And there's reasons for why, why people choose to run their applications there. Customizable, stable, upgradable supportable, all those kinds of things.

But yeah, for your daily driver, I'm not going to try to convince anyone to try to try to drop their MacBook and go pick up a PC and run, run Linux desktop on it anymore.

Randal: Right, right. Like I'm probably never going to get rid of like a Mac interface unless something really outstandingly better comes along for cheaper.

And that's not going to happen on either axis. The, but the thing is like, I'm running right now sort of because I'm, I'm just a shoemaker's children problem. I want to put time into my cloud servers, but I never seem to get around to it. You know, it's like, eh, that'll, that'll still run tomorrow. It would be fine.

So I've got three free BSD servers in the cloud. And now that I'm doing a lot of stuff with Dart and Flutter. I regret this because FreeBSD is not one of the target platforms either to compile on or to target to build for. And so I can't do anything that I really want to do. Like I want to move my website so that it's a, a Dart application spitting out HTML, which is perfectly reasonable if the, the host box is a Linux box or can at least run Docker.

And even running Docker underneath FreeBSD is. A bit dicey from what I've heard. So so I'm thinking strongly about, let's see, first farming it out. So I'm actually buying separate machines for my web and my mail and all that, rather than having one big monolithic box that I'm doing all of the maintenance on as a free BSD box.

But that would be a break in multiple years of, of history for me. So I've, I have to then learn enough about Linux to be able to have a hosted service of my web and my mail and all that stuff correctly. I could go to. Platform as a service stuff and start doing, you know, like all my mail through Google or something.

I don't quite trust that yet. And I have too many special things I've wired in. So like the way you guys wire up hardware, you should see my proc mail RC and my, and my Pearl script that sits in front of that. And, and it's like all these things. Are very, very heavily customized. And so I have to track the upstreams of all these things I'm using to make sure that my stuff still keeps working at the software level.

So I, I can relate to the hardware problems that you're facing because I've got incompatibilities of this hardware or this software doesn't work with that software.

Aaron: Yeah.

Randal: So it's, it's, it's a similar problem. Luckily I've got things like unfortunately now I have both original stuff on my Mac and brew and Mac ports because they each had their own things.

And my path is about 85 elements long, trying to suck all that in. And what JQ am I invoking today? I don't always know, and they have different features. So it's like, Oh, boy, the path has to be exactly right. And I don't dare move it now that everything's sort of working on my Mac. It's, it's definitely scary times for me.

And I, I also, I just don't want to deal with it anymore. I just want it to work too, which is sort of the anti tinkerer theory.

Aaron: Yeah, but,

Randal: But it means you have to be a bit of a tinkerer to get it sometimes to play well with the stuff that you've also tinkered with. Do

Jonathan: you hope? So, so Randall, a couple of things here.

Yeah. First off,

Randal: sure.

Jonathan: From the little bit of BSD that I've worked with, I don't think you're going to have much of a problem going from an A net B, S, D, open BS, D to, to Linux. So much of it is almost exactly the same. And the , there's, there's better compatibility with Linux. I think in a lot of cases there's more packages available too.

So I, I just, yeah, I don't think it's gonna be a problem for you. I think you can dive into that and not, not really have much issue. Talking about, you know, I,

Randal: I have to, I have to learn the GPL version of all the argument lists, the arguments, because that's, that for some reason, GNU just went off sideways and a lot of things that I use every day.

And it's kind of annoying because I grew up with, I, you know, I used Unix V6, Unix V7 2. 8 So I grew up through real Unix. And I still don't consider Linux to be real Unix that because it doesn't have the same history.

Jonathan: That's fair.

Randal: You know what? Yeah. Is that fair? I think that's fair. Yeah. I

Jonathan: think that's

Randal: fair.

Okay. Yeah.

Jonathan: The same problem. So I have to learn the

Randal: Linux. Yeah. I

Aaron: have the same problem going to b sd.

Jonathan: Yeah. I have the opposite or the opposite. Yeah.

Aaron: The mirror. The mirror problem. Right,

Jonathan: right, right. A a lot of the i's like, why doesn't

Aaron: this work?

Jonathan: A lot of the GNU Corps utilities though, or at least some of them.

Actually support the BSD flags, don't they? I'm pretty sure some of them do.

Randal: They, they've been sort of forced to because POSIX was sort of a compromise in some ways between both worlds.

Aaron: Yeah.

Randal: And so as everybody started moving more towards POSIX compatibility, they ended up having to, the BSD made some concessions and FreeBSD, and Linux made some concessions to be able to add things in to say that they're POSIX compatible.

POSIX compatible if you only use these switches. And so, and so that, that helped quite a bit.

Jonathan: David Ruggles in the chat says, Linux is not real Unix is a feature, not a bug.

You know,

Randal: the thing is though, I remember when Unix fit in 32k. And, and that was considered huge back then. So it's, you know, it, it, it, it's come quite a long way since then. I'm sure there's some parts of the Mac OS I'm running that are under 30 TK, maybe one of the files or something. I don't know, maybe, but, but.

But now they've become these giant monoliths, but there's, but they're also so feature full now. I mean, there's, there's so many features, then they keep adding more features and telling me to upgrade again, and then they're adding more features and I have to learn those new features and then those features aren't compatible with my phone.

So I got to upgrade my phone now. Yes. So

Jonathan: did the BSDs have a package manager? Cause that's kind of a Linux. Oh, lots of them. That's kind of, well, yeah. So that's sort of a thing that was originally, I think originally developed in Linux, like the, the, the yellow dog. So Red Hat's yellow dog package manager was one of, I don't think it was the very first one, but it was one of the first ones and you know,

Aaron: RPM, right.

I think RPM was before the yellow dog. That's the first one I ever used back in 1990. Eight when I was messing around with a red hat, I was like, Oh, I can just use this RPM command to like update my software, get a new program on you. That's pretty cool. Previous D

Randal: from out nearly its very beginning.

It's always had some sort of quartz managing system. That can record dependencies between ports and install dependent ports. That was mostly a source based system. And then they moved to the PKG system later on. And that's kind of where all my systems are right now. They're all. PKG related, which is kind of laughing now that I've got brew, which does some source compiles and some binary downloads and a Mac port still does some source compiles and some binary downloads.

And it's like you know, upgrade time. I never know whether it's going to be four minutes or three hours. I don't know what it's going to compile something or just download a few things. Yeah, so

Jonathan: I, I'm, you know, I've thought about over the years, like, what is it about Linux that really has drawn me to it?

And what is it that I don't like about, you know, those other operating systems? I've got to say the package manager is one of the things that just really, really makes a difference. And in thinking about this, like on Windows boxes, I've had times where, because I, I, I do tech support, I find people's Windows machines that are just in terrible shape.

And one of the things you can do with a package manager, particularly with something like RPM obviously that's what I'm used to, so that's what I'll reference, is every file that is part of your operating system, it lives inside of one of these RPMs, in one of these packages. And so if you have a file that gets corrupted, you can just say, Look at all, basically what you can do is say, look at all the hashes of all the system files and if one of them doesn't match, just reinstall it.

And I, I, I still don't understand why they have not added that to Windows. Now, I know they've got a package manager now that is apparently fairly decent. I don't think it applies to all of the system files though. And that's just a, it's a killer feature for me for Linux.

Randal: And let me, so, so Mac OS is slowly getting better in that third party apps can now create sandboxes and containers for their machines, for their, for their installations, but it didn't always used to be that way.

It used to be that what Apple told you was just make sure it installs in user local somewhere. And it's like, well, that's not a package manager. That's well, there's, it's installed. You want to remove it later? You want to see what's corrupt? You can't. It's

Aaron: yeah.

Randal: And so that's why the third party Mac ports came along and then brew, brew does it a whole lot better, by the way.

I would say Mac ports now seems. So chaotic at this point. I've been slowly looking at my list of things installed with ports and one by one installing, reinstalling them with brew. And then I hope that that's a similar version to the one I was already running so I don't break something. The one I've been dealing with lately that's crazy is YQ.

I don't know if you've ever seen the YAML version of JQ. But it's really, really cool for a lot of different transformations YAML in, transform it the same way JQ does and spit it back out. Except there's two things in the world that are called YQ and they do entirely different things, but they're both related to processing YAML using a JQ like syntax.

One of them actually uses JQ behind the scenes. So it turns everything into JSON and then back, which is weird. And the other. Has a subset of JQ's instructions built in. And so, if I accidentally get the wrong YQ in my path ahead of the other one, because I have them both installed, because they both are handy in different ways, then, ah, it's such a mess.

It's such a mess. One of them is written in Python, by the way, which I think Aaron will probably like. Namespacing,

Jonathan: it's hard sometimes.

Randal: Yeah, yeah, yeah. Hard to think of good names for stuff. All the good names are taken, that's definitely true. All the two letter names are definitely taken. Yeah, that's probably true.

Jonathan: Oh, I did not know about YQ. This is actually really interesting. Oh no, now I distracted you. There you go. Now

Randal: there's two of them. One of them does, like I said, one of them is based on Python and only, and has a built in sort of YAML interpreter. So it has, but it's also then cloned a lot of the JQ operations.

The other one that has YQ. TomlQ and XMLQ, which basically means you can manipulate from one to the other of any of those four things, JSON, YAML, Toml, and XML, and manipulate all of them. That's the cooler one. That's the one that uses JQ behind the scenes and then does in and out mappings for those other things.

Languages, that one is really, really cool as well. So I, again, that's why I have both of them installed. I have the one that does the Python version. I have it as PYQ just to make sure that I can get to it whenever I want to. But yeah as an example, I was using probably going deep off of, I don't know what your, our actual agenda was going to be today, but let me just finish this and we can go back to Linux again, but in in the, in the Flutter world.

You have this tool called pubspec. yaml, which describes all of the things that belong in your release. So it has a list of all the version numbers and everything like that. But what actually gets installed is derived from that, because it also has to have all the sub dependencies and all that.

That gets locked up in a file called pubspec. loc, which is a JSON file. Okay, so now I've got JSON representations of YAML stuff, and I wanted to take the actual versions that got installed and put that in my dependency range in my source pubspec YAML file. So I wrote this About this long single command line that unpacked the, the, the JSON file got to where it picked out the actual dependencies and sub dependencies that were used and spit that out as a nicely formatted YAML file.

And then I dropped that into my PubSpec YAML. That's great. And really cool that I can get two different languages on both ends of that pipeline. So. Yeah, really, look into YQ. Again, there's two versions of it. There's the one that has the multiple languages thing with TomoQ and, and XMLQ, and I've used that too.

I've actually got, oh, XQ I think they call it. It actually goes from XML to and from XML with JQ like operations on it. It's pretty cool.

Jonathan: Yeah, I've actually become quite fond of YAML for config files. When I had to write a schema for doing config files for Project M1, I was like, let's use YAML. It's great.

I like YAML. Now, occasionally The great

Randal: thing about YAML, the great thing about YAML is that JSON is a perfect subset of it.

Aaron: Mm hmm.

Randal: So at any point you're freaking out about indent, you could just go open curly brace, blah, blah, blah, blah, close

Aaron: curly brace.

Randal: And it's, and it, and that can all be indented whatever way you want.

Yeah. You can run it over multiple lines, all that stuff. But what's also cool about it is it's like relaxed JSON. You don't need to quote your keys. You don't need to do it. It's, it's, it's, and you, and you have comments. Yep. So it's like the best JSON parser you can get is a YAML parser.

Jonathan: Yes. Yes. I, I have, I very much enjoyed YAML.

The only thing is every once in a while, somebody will come into it and work on it. It's like, nothing's working. It's like, okay, copy and paste. Okay. YAML is whitespace dependent. You have to actually get, it's like Python, man. You have to actually get your whitespace lined up. That's,

Randal: that's tripped up a few users, but, but again, we'll drop in a JSON mode when that happens and it's fine.

Jonathan: Yeah, I have to look at whether our parser could reason us in this project reason a C library to parse it. I got to look and make sure that our parser can actually handle the JSON curly brackets. That's

Randal: in the spec for YAML. So if it's not able

Jonathan: to parse it, it's not spec compliant. If it's not able to parse it, we need to find a better library.

Is that is that what you're telling me? Yeah, and I know that I

Randal: know that C one that comes out of library does it. Pretty accurately. It's pretty, pretty on spec. So,

Jonathan: yeah, cool. Yeah, cool. All right. Let's see what, what direction, what direction were we going? What direction did we want to go north? Let me, let me, let me throw out,

Aaron: let me throw out one more Linux use case that I think people should be at least cognizant of.

Yeah.

And that is for anybody building systems for other people. Which I used to do. Or if you're just techie enough and you want to have a good backup in case something goes wrong. I always recommend that people have some, some easy form of, you know, Linux mint or something like that on a live USB stick.

Yeah.

And if you're building systems, as I mentioned, I would always ask people, family members or whatever. Hey, building this system for me. Great. Do you want me to use me? I'm going through puberty there. Do you want me to throw Linux on this system as a backup? And you'll never have to deal with it.

It'll automatically boot into Windows. But just in case you ever have a problem, you can at least select the Linux option and get to your files or get to the internet in case windows decides to throw up on you, which mine is doing. If I disconnect right now, it's because I got a blue screen. And that, that has saved my bacon more times than I can count really, because I've had, I've had cases where, for example, my raid.

Running on the motherboard failed and Windows doesn't know anything about it. Right? I mean, it doesn't even know that there's RAID underneath, but I could go into Linux and run. I think it's test disc test discs, analyzes the, the, the bare bones of your hard drive and it can tell where the files are.

And you can even copy those files out onto a different drive. And I've saved like picture, family pictures and all kinds of stuff just by having. Either a thumb, you know, USB thumb drive with Linux on it, or you know, Linux as a, as a separate partition on the drive that I can recover from. So the one more, one more use case where I think people should at least be aware of Linux and also if you're having trouble understanding Linux, you can always go get my book, Linux for Makers, because the whole reason.

The whole reason I wrote that book was because people were coming to the makerspace from either macOS or Windows backgrounds. They wanted to run Raspberry Pi. And they're like, what are all these directories? I don't understand what they mean. You know, And so that's a big part of the book is just explaining what Linux is.

It doesn't go into great detail because it's not meant to, but it's like, here's what you need to know about Linux to make your experience with Raspberry Pi better. And here's what those directories, here's why they came to be called. USR and ETC and all that, you know, and so people just get a basic understanding of it.

And I think once people get past the, the translation between how it works on MacOS, how it works on windows and how it works on Linux, then they can start to at least function a little bit better.

Jonathan: Yeah, you know, the ironic thing is that Mac OS under the hood is so similar to Linux when you actually get on the command line, because they're both, they're both Unix compatible.

And so you get a program like brew installed and you start installing the binaries, you can, you can get your Mac machine set up to the point to where it almost behaves like a, like a Linux box does. Windows, Windows is like a

Randal: Unix box,

Aaron: like a Unix box, technically BSD.

Jonathan: Yeah,

Aaron: yeah.

Jonathan: Well, so no, that's that's the point.

I mean, they're both open step. They're both Unix boxes. So you can make your your your Mac OS install act very much like your Linux install does because they're both Unix.

Randal: Yeah.

Jonathan: Yeah. That's, that's the

Randal: point. No. Stop making me say that. Linux is not Unix. No, it's not. You can make, you can make Unix a lot like Linux.

It can be a lot like it. Yes. They both have LS. Both of the LSs have 65 parameters, but there are different 65 parameters.

Jonathan: Yeah. I suppose that is always going to be a thing.

Aaron: Yeah. But like, even in my book, I, I showed people how to write the Raspberry Pi operating. system to to an SD card, right? And both Linux and Mac OS use DD to write that image out to the SD card.

And it works the same way basically on either one. So there's enough similarities there. Yes. It's not Linux. It's Unix. But There's enough similarities there. I think people, people could get a, that were, were at an introductory level would look at it and say, Oh, this is just like that other thing I was using.

Sure. But

Randal: there's enough similarities. I mean, I I've, I've in practical terms, I've had Linux in the cloud from time to time for some of my clients. And I've, I've, I've sort of. Poked and prodded my way through those. It's not like I can't do anything at all. It's just that every once in a while I'll type a command and it goes, I don't know what this switch is.

And I go, oh man this, oh, oh, this doesn't have GPL or GNU code does not have that, that switch. Okay, I get it now. Yep. I'm used to the BSD switches, not the GNU switches.

Jonathan: Randall, are you familiar with TLDR? You long didn't read the the command line application tldr. Oh, no. I didn't know there was one. There is one I I assume that you can get it with brew but it's definitely a thing on pretty much all the linux distros So you install tldr and then you just it's tldr and then the name of the command that you want to run And what it'll do is it gives you like the minimal version of the man page And so it'll, these are the four most commonly used switches.

And here's three examples that use them. And it's for, for, yeah, it's really nice. Leo actually is the one that got me hooked on this. Back during one of our first ULS episodes, he, he plugged it and I'm like, Oh, that's really great. So

Randal: brew does have TLDR. I'll have to install it and

Jonathan: play with it later.

TLDR. It's really cool. I won't be like,

Randal: I won't be like like Leo used to do where he would install something the moment I say it on the show, that was really weird. Wait, wait, we're still doing the show. They're like, yeah,

Jonathan: that was. That was his thing. I think that's still his thing in some cases. I think it's just because he's been doing it for so long.

He doesn't have to think about doing the show. It just comes naturally. So he could, Oh, okay. I'm going to install this while we're talking. He probably

Randal: has a completely spare PC sitting next to him that he can just immediately swap to if he screws the one up that he's in front of him. Yeah, I think so.

Aaron: Especially these days, right? Without the studio.

Randal: Yeah.

Jonathan: Yeah. Well, I don't know. Gotta be

Aaron: self sufficient.

Jonathan: Maybe, maybe that means that he's running the he's running the show off of his computer. I don't know. That's what I do. So I can't, I can't mess around with this machine in front of me. I can't mess around with it too much.

Cause it's doing the streaming.

Randal: Yeah, me too. Cool.

Jonathan: All right. Let's see what what, what direction do we want to go next? Hey, we got about 15 minutes left. There's another interesting thing that happened with the Linux kernel. And I guess I'm curious on y'all's take on it. I've covered this in other venues, but since we're a very Linux centric show today, it makes sense.

Do you guys follow where the Russian maintainers got kicked out of the kernel?

Randal: Yeah. Weird. That. But then, then you start wondering how many other, Oh, but you should go ahead and tell the story first for our audience. And then I'll, I'll fill in my comment.

Jonathan: Sure. I'll, I'll give you the, and maybe I'll answer some of the questions.

Cause I had, I had questions and I've looked into it too. I've actually, I've actually got questions.

Randal: We got,

Jonathan: I got a hold of a couple of people and, and anyway, so there was a rather cryptic message, you know the, the, The commit history of the, the part of the email, you know, when, when Greg Carl Hartman wrote it, I think he said something like for compliance issues, we're having to change the maintainers file a bit.

You know, it was, it was real cryptic like that. And people got to looking at what it was is just about all of the maintainers. So that's the, the people that are responsible for different parts of the Linux kernel. The maintainers that had like email addresses that ended in dart. ru just got removed from the maintainers list, no longer maintainers.

Geopolitics and politics and all that being what they are, people got a little frustrated by that and let people know, and there was a lot of trolling that went on some of it was hilarious, some of it was not very helpful, and then of course Torvalds was Torvalds. And, and stuck his oar in, in, in a very Torvald sort of way, which is fine.

I have no problem with that. But so what, what it came out to be is that apparently, obviously, again, obviously geopolitics there are Russian companies that are under My mind has completely gone blank. I've forgotten the word. Sanctions. They're under sanctions. They're under sanctions from the U S and other countries around the world.

And so one of the things that those sanctions mean is that like, there are limits to how you can cooperate with them. And when it comes to source code, even, and apparently the Linux foundation got a visit or a phone call from the U S treasury department, and then said, Hey, I see your open source project.

I see you have people that are partially responsible for it, that are on our sanction list, and this cannot be. Yeah. And I think that's, like, fairly reasonable, because they're, they are the U. S. Treasury is responsible for enforcing the laws and the executive orders that we have in the United States, and the Linux Foundation is based in the United States.

And so, it's just sort of an inevitable thing for that to have happened. I think the colonel could have done better with the way that they like announced it and messaged around it. It came out later that apparently the the internal lawyers of the Linux foundation Gave them guidance, but did not tell them how much of that guidance they could make public And so there was that it's like our lawyers have not yet told us that we can make a statement on this Right, and that's that's just one of the terrible things about the way the legal system works, but it is part of how it works There has been, I think at least one of those developers was able to document that.

No, no, I am not actually employed by, you know, by call or whoever it is. And so is back as a maintainer. The thing that the thing that worried me the most about all of this, and this is mainly, I guess, mainly what I'd like to get y'all's opinion on is that there was a statement made by somebody at the Linux Foundation that said essentially We hope that removing these people as maintainers is going to be sufficient to make the treasury department happy and we hope we will not be required to remove their contributions.

And that, yes, I, I have a real problem with that I have a real problem with that because in the United States we have the First Amendment, which says that, you know, you can say whatever you want to, and we have some, some legal opinions that have pretty well established that code is protected speech, and so I, I really am concerned that the U.

S. Treasury Department even made that threat. But I'm real curious what you guys have to think about that. I

Randal: don't think that so much as a First Amendment protection issue as I do recognizing the open source development process, which is that those lines of code clearly had to get reviewed by non Russian infiltrators. No, I won't say that, that's just evil. Non Russian residents. Right, right. I don't want anybody to misinterpret.

I actually, you know, I'm a world traveler. I've been in 68 countries. I do have a much greater sense of recognizing my kindred humanity in every part of the world. So let's just start with that. Okay. But but I think Every piece of code that went into the kernel ultimately was viewed by many people.

So to say you won't have to unwind particular contributions because of their source is a bit confusing to me. It's not understanding the process.

Aaron: Yeah.

Randal: I also am curious about how it was only Perhaps anybody who had a email that ended in RU, how does that prove anything? So it,

Jonathan: it was, there's a little bit more detail on that.

I think, I think all of the dot RU email addresses were taken out. And then there were some that were also things like Gmail. So essentially what it was is it was, it was people that were known to be Russian nationals. I think

Randal: it

Jonathan: was, it was a little bit more intelligent. It was not, you know, it was not just a said script, right?

Let's just set and remove all of the lines that have dot R U, just greedy. For Hawk or

Randal: Pearl. Yeah. Well, okay. Hawk and Pearl, whatever that is.

Jonathan: Yeah. Yeah. Well, that's the fun thing about the world we live in. There's 15 different tools that can do that for you. Anyway it was a little bit more intelligent than that.

One of my buddies that I talk with And we discussed this idea of what would you have to unwind the patches? And he's like, his, his opinion was you would just have to reset the entire code base to whenever that executive order went out. There's just no other way to do it because you have so many patches built on top of each other.

He's like, that's what you would have to do to comply if they actually wanted you to do this.

Aaron: That's BS. That's, that's, well, first of all, that's a hypothetical, right?

Jonathan: Yes, correct. Correct. So that's not,

Aaron: that's not what they're saying, but the way

Jonathan: I will say this, the way that the mailing list message reads the way I read it.

Is there was at least the threat made that, let me rephrase. There is at least the consideration that this could theoretically be possible. Like, so this is not just me suggesting this could be possible. This is a Linux foundation member saying, we hope that we do not have to do this.

Randal: There's a dangerous component to this that I also want to bring up.

And I, I'm sure everybody's thought of this anyway, but let me say it for the few that haven't. Is this also going to apply to every other us based company?

Aaron: And

Randal: does that mean that the treasury department will be coming after the software freedom conservancy and especially these umbrella organizations like, like the SFC to say, your members all have to comply with this ITAR regulation.

And sanctions that we've had on, on particular governments and individuals, and that would be horrendous for the SFC because they've got like 50 projects, a hundred projects, something like that. Right. And, but they're us based and, and so where, where do we draw the line here? I recognize the intent.

But I don't recognize the practicality of it, given the world of open source.

Jonathan: Yeah. So let me, let me tell you what I think the, the, the real intent was, what I think annoyed people in the U. S. government. So obviously we have a, a, we have a war zone where it's a, it's a somewhat hot war right now and Russia's on one side and U.

S. allies are on the other. Like this is just the reality of the situation. And you have Some of these Russian weapons that are literally running the Linux kernel. Like this is, this is a thing. This is a thing that is known to happen. And I think what happened is the, the treasury department, the United States looked at that and went in essence, our developers that are working on the Linux kernel are assisting.

These Russian developers, essentially because the kernel is running on these weapon systems, we essentially have U. S. developers that are helping to maintain the Russian weapon systems. Right? Like, I think that is probably the thought process that went into this.

Randal: And didn't we also have a recent example of where some apparently cleverly hidden code caused a backdoor in some SSL transport mechanism?

Jonathan: Yes, and so that could be this, it could be related to that. That's the XZ I'd be triggered by it. The XZ libraries, in a very clever way, were attempting to backdoor SSH, and that did actually make it out on a few machines. There were some machines in the wild that had it backdoored for a very short amount of time.

Although ironically, The the time zone work on that happens to line up with us east coast. So I have theories on who is behind that Could be cuba right Yes Anyway, so I I guess I guess this this whole story the thing that really comes to mind for me about it is like Open source has sort of avoided having to play the geopolitical game.

And I'm not sure that that's going to be able to continue forever. And that's sort of unfortunate, but maybe also just the reality of the world.

Randal: Along that line some of the new EU requirements for software in general might apply in very chaotic ways to open source.

Jonathan: So we've, we've talked to

Randal: a lot of issues.

Jonathan: We've talked briefly about that with people like Simon Phipps. Simon actually was a big part of lobbying in how those laws do and don't apply. And so there is actually quite a big carve out for open source in the, in the EU software laws. Which, you know, it, it really, it really makes sense. If someone is not making money for a, for a project, trying to force them to, to follow all of these requirements is just, it's, it's a non starter.

Randal: Right. Yeah. Open source starts at a negative 5, 000 a month deficit. Yeah. You can't quite do that. Right. I don't mind doing it for free, but I don't want to pay 5, 000 a month out of my pocket to hire these six guys they need for compliance with the EU regulations. Yep.

Jonathan: Yep. Yep. And so one of the interesting things with that is, so now you can get it, what it, what those laws do from what I understand is they, they put the onus on the companies that are commercializing.

And so I, we've heard some stories now where these companies are going to open source projects and saying, okay, here's the documentation that we need to be able to use your project. And they sort of expect it for free. And the thing I've been telling people is when you get one of those emails, you just respond and say, we will be glad to help you with that.

Here's our hourly rate.

Randal: Yes, absolutely. Yes. You making money. We making money. Yeah. Yes.

Jonathan: That sounds

Randal: great. Yeah.

Jonathan: So hopefully that'll become more and more of a thing and

Randal: maybe it'll be, maybe it'll be a good thing. A rising wallet pays for all boats. I like it. Oh, I like it.

Jonathan: Just

Randal: made that up. That could be the title for today.

Jonathan: It could be, we've had two or three. Yeah, we've had two or three suggested. Okay. Let's see, we are getting close to the end of the hour. Shall I ask each of you what your favorite scripting language and text editor is? We haven't, we haven't done this for a while. I think I, I think I remember Randall's.

But have they, have they changed since the last time I asked you?

Randal: Not in the last three or four weeks, but just to, just to remind everybody, you know, what's funny is I don't think I ever answered that question when I was asking it all those years. Oh yeah, that might be. So although I would, I would say, yeah, I'm also an Emacs user, but now I do most of my work in VS code, particularly because.

For one it's embedded in GitHub, so I can just type dot when I'm looking at a repo on GitHub and I'm in VS code on that project. So cool. Yeah. Right. I didn't realize that until like, like three or four months ago. And I went,

oh, this is, why didn't I know about this for all these years? I've been downloading this stuff, putting in an opening in my own.

Right. Anyway, so I'm learning, I'm using VS code parse because of that. And also because VS code is being used in the IDX project, which is now open to everybody with the Gmail headers, by the way. IDX. google. com is essentially you get your own little Linux backend, you get VS code, you can suck down Git projects, you can run them in any of a number of architectures, you can control how the Linux is configured with somebody called mix or something that's S S M S I S there's like a configuration language for Linux layouts and stuff.

So, but it's great. And it's free idx. google. com. Go check it out. And it also has a templates for starting up, like say a typical angular project or a typical, in our case, dart and flutter project. And it even will put a web view and a Android view. As emulators right in your window. So you can do incredible development with nothing.

Yeah. So VS code is running in that. That's kind of a long way of saying that. And my favorite programming language right now is now Dart because it's part of Flutter. And I'm getting much better at Dart. Dart's really cool. Dart's like Smalltalk that won finally. So that's great.

Aaron: And of course they've integrated or they're trying to integrate Gemini into this as well.

Oh, of

Randal: course. Oh yeah. Of course. Of course.

Aaron: Yeah. Now it all makes sense. Yes.

Jonathan: Aaron, I, I, boy, I don't, it's been a while since I've asked you this. What what's your favorite text editor and scripting language these days?

Aaron: Yeah. Nothing's changed for me, actually. I mean, text editing. Is still, if I have to drop to the command line and edit a file, it's still nano because all of my systems pretty much are Debian based systems and nano is almost always there on, on all the operating systems, all the distributions that I run.

So, so nano is just easy. It's just easy. I used to be VI when I started, cause that was what was there and Emacs was too hard for me to learn. So, you know. So now it's nano and then languages are still the same Python when I need to do something more higher level and C when I have to do something a little lower level, I go back and forth.

I spend a lot of time in, in you know, the Arduino IDE working on ESP 32s and. And that kind of stuff. But I will say I have been trying to learn more of the Pico platform, which has brought me to VS code more and C and learning the back end stack that I need to get that stuff compiled and all that because because Arduino obvious gates, all that stuff.

Right? So you just like, yeah, there's Build it and upload it to my board, you know? And then it's like, Oh, wait for the Pico. I've got to know like what's behind here. So I've been learning that a little bit too. So yeah, VS code is going to become more important in my life than it has in the past, cause now I have a reason to use it.

So

Jonathan: have you worked with the platform? I owe any. Has that been on your radar?

Aaron: Yeah. Yeah, I did. I tried it. It was kind of, it was kind of a similar experience to me. It's like there's so many dependencies. And once you get it all set up and running, it seems to work. But for me, it was between Arduino, which I had been using and understood and could figure out and then going to platform, which was seemed like it was supposed to make it easier.

But for me, it ended up making it more complicated because things didn't work. When I would try to build stuff. And so I was like, I don't, I just, there was no reason to use it. And like, now there's just a lot of stuff in VS code that I'm just like, yeah, I can do my Python work here. I can do my C plus plus work here and I can, there's so many integrations and you know, all the stuff Randall was talking about.

So yeah.

Jonathan: I have, I have a recently. So I've been told that I have to answer this question too, because it's around table and that's fine. I've recently been doing a lot of work because I've been working on the mesh testing project. So I'm doing a lot of platform IO in VS code. And that's pretty interesting.

What are the, one of the cool things about platform IO is that it lets you define a whole bunch of different targets. And so, you know, we can, out of the same code base, we can support ESP 32 devices. We've also got it rigged up to where we can compile for native Linux, which is what I've been working on mainly.

NRF, the NRF 15. I think that 42, whatever the, the, the thing is STM 32 boards, all kinds of stuff you can do out of platform IO. So that's cool.

Aaron: Is it any easier to set up now than that was before? Because I swear I went through it and I spent a couple of days on it and I was like, if I, if I, if I have to spend a couple of days to get this thing set up, it's not worth it.

Jonathan: I think it's one of those deals where it's like, you want to copy a working config from somebody else's project that I find that's true though. So many of those different things that have boilerplate, like goodness, the first time I went to do an Android application, like the, there was just so much boilerplate you had to get in, get in play now, the, the, the.

Android studio made that a lot easier. You can just say generate new. But still, I think that's, I think for any framework, that's just going to be the thing.

Randal: Yes. Right. On LLMs, LLMs made it also simpler because pretty much that's the thing they trained on is so you'd say, give me a configuration for, and it goes, ah, I've seen that somewhere.

Right.

Aaron: Yep.

Randal: Yep.

Aaron: Speaking of LLMs Randall, I ha I hate to say it, but I'm gonna be going through just a test. It's more of an LLLM test than it's anything else, but I'm gonna be taking my last Running Pearl program and using chat GPT to convert it over to Python just to see how well it does,

Randal: does PT Gemini Advanced is smarter

It's been smarter in every test bed.

Aaron: Yep. But that's my last running Perl script. It's a recipe database that I built 25 years ago. Nice. Wow. For my wife. And now you don't cook,

Randal: and now you don't cook. So it's great.

Aaron: Oh no, I cook a lot. And we use the, we still use the recipe database every, I mean, multiple times a week,

it's,

it's crazy.

But at the same time, it's like, I don't remember what I did 30 years ago now. And so a couple of times I've had to go back in and fix it. And I'm like, crap, I don't remember how this works in Perl. It's Perl code.

Jonathan: So, you know, it's pretty much obfuscated already.

Aaron: Yeah. And I didn't want to call you, Randall.

I didn't want to take advantage of our relationship.

Randal: I've done very little Pearl. I have to now go look things up now. Again, I'm at that stage in your own words. Luckily that's online, but yeah, no, but I read the man pages once again. Although I heard that from Larry as well. So Larry told me about I don't know, about five or 10 years into you know, after the camera book had been published, that That he said yeah, I got to look things in the man page occasionally, because I forgot how I did this.

And it's like, you know, it's, it makes sense. I mean, it's, it's stuff. It's, we only have so much room in our heads. And now we got to worry about new things like LLMs and stuff. I,

Jonathan: I have been told that it's when you know that you're sort of an expert in your field when you go to Google a question and the answer, you find it in something you wrote.

Randal: Yeah, oh, that's that's happened plenty of times for me because I have 255 magazine articles online Yeah, and just the other day not about two years ago. I was trying to solve something for somebody For a client and I went I know there's a way to do this I googled and up came one of my columns. These are all online.

So it's like, Oh yeah, of course, that's how I remember now. I remember writing this. I've started, I've

Jonathan: started finding answers to my questions in my own Hackaday articles. That is a, that is a very fun experience. Yeah, definitely.

Randal: Good with being published, yes. Yeah.

Jonathan: Anyway, I don't know if they actually fully answered.

So if I'm in the command line, I like to use nano. And otherwise I use a lot of VS code. And then on the language. Very much like what Aaron had to say. I very much enjoy C these days. But for scripting, here recently it's just been straight bash code for pretty much everything I have to do. The fact that you can just glue commands together with pipes and all that is really powerful.

I have used a little bit of Amber. We interviewed the guys behind this and Amber is a really cool program. It is a compiler for bash script. And it lets you, the things that you wish Bash script would do that it doesn't do, the Amber language has it, and then it compiles down to Bash code, which is really pretty cool.

So if you find, if you find yourself really annoyed with Bash, but you want to use it still, Amber is something to take a look at.

Aaron: I'll have to go back and watch that show.

Randal: Yeah, it was fun. That's, that's similar to the reason that I converted from using Perl to using Dart for my command line stuff is that Dart actually compiles to local binaries, which Perl never knew how to do really well.

So that's really nice. All my apps are running much faster now.

Jonathan: Yeah. Yeah. Let's see. Do I, I assume that you guys have stuff you want to plug. We'll let

Randal: Randall go first. Sure. So I am again, pretty much heads down in the in, in, at Transparent apparently. If they OBS is being really mad at me.

Yes. So is in the, in the dart and flutter world. So I'm spending a lot of time attending virtual meetings, giving like Q and a sessions there. Sometimes they do presentations. My big presentation coming up is if you're a Pearl user or have been in the past there is the annual Pearl advent calendar and on the 3rd of December 13th, I'm doing a live presentation of a talk I gave seven years ago at OSCON, back when I was still going to OSCONs about half my life with Pearl, which is, at that point I was turning 50 and Pearl was turning, So it was like for exactly about half of my life, Pearl had been around.

And I talk about the early, early history of Pearl and how Pearl impacted me personally, impacted my company, how my company and the trainings we did impacted the Pearl world. So there's this incestuous, you know, loop in, in there that I'm able to kind of highlight some behind the scenes, sort of how did that all happen?

I'm doing it live. There's some sort of registration link that I don't have in front of me, but I can probably give it to you for the show notes. You don't have to register to actually be there. It's just going to be like a a zoom call. But I think they want to make sure they know how many people are going to be there to know what size Zoom package to buy for the one day or whatever.

So yeah I can't imagine seeing a hundred faces in the zoom, but apparently they're close to 100 already registered for this. So I it's gonna it'll be fun. It'll i'm not changing the talk at all, even though it's been I gave it seven years ago, mostly because I, I haven't really done anything else with Pearl.

So the talk really doesn't matter, but I'll, I'll speak of it as if it was the historical moment in time seven years ago when I was half my life, I had spent exactly the Pearl. So that's coming out. That's probably the only thing I really want to pitch in the Pearl world. And again, all dart and flutter stuff.

I have a, I have a YouTube channel, just Randall Schwartz, dart and flutter. You'll find it.

Aaron: Aaron. Well, as usual, you can find me at on YouTube, RetroHackShack and RetroHackShackAfterHours. I'm getting there's been a little bit of a delay because of my eye and I actually did a video on the main channel that talking about what's going on there as a little bit of a warning for people as well, so that they can avoid having to wear an eye patch like this all the time.

So I'm not going to go over it here, but if you want to know like what's going on health wise, you can go watch that video. But I finally got back into actually producing some content after a bit of a break. So I just uploaded a video to RetroHackShack After Hours, which will come out tomorrow morning, 6 a.

m. Eastern Time. And it's all about this really cool Vintage piece of electronics that helps me diagnose vintage computer monitors and TVs.

Ooh, wow.

And it's made, made by Extron and it came out in 2002. So it was right at that sweet spot where you had to, you know, there was no HDMI, you know, you had to support everything.

So this box supports Mac monitors. It supports IBM monitors. It supports Sun monitors. It supports SGI monitors. It supports Apple too. Monochrome composite monitors. And I test all that on that video. And so, yeah, if you're the kind of person like me who likes to pick up this old stuff and test it, but you may not have the machine you need to actually.

Hook up to it. Or the machine you have might not be working enough to produce a video signal. So you can test that monitor. It might be something interesting for people to watch.

Jonathan: It'll help you figure out whether the problem is in the machine or in the monitor.

Aaron: A hundred percent.

Jonathan: Yeah,

Aaron: but, but actually what I found out was that the box that I bought is old enough that the problem could be in the box because I had to buy two of them.

The first one didn't work. So I had to buy another one, which is going to help me fix the first one, but when it works, when it works, it's all pretty much in hardware. So when it works, it, it works really well.

Randal: Oh, that's great. All right. Hey, thank you. Thank you. You mean you mean Aaron? You're not still dressed up for September 19th and

Aaron: just didn't stop No, or halloween either.

Okay Halloween for halloween. I had a I had the eye patch on just because it helps me focus more on this eye and I had a a Shirt like this and I had a ghostbusters t shirt because I wear geeky stuff, you know I just that's just what I had on that day and some little six year old girl Oh When they came to the door, she's like, what are you dressed up as for Halloween?

And I said, I'm a ghostbuster pirate, of course, you know, and she had no idea, but the parents were back in the back and they were laughing. It's like, no, this is just how I look

Randal: these

Jonathan: days.

Randal: Know your audience. Yes.

Jonathan: All right. Thank you guys both for being here. Appreciate it. Jumping in at the last minute. And thanks so much.

Thanks, Jonathan. Yeah. Thank you guys. All right. Thanks. As far as my plugs do want to mention, we appreciate Hackaday being the home of Floss Weekly and you can find my security column goes live every Friday morning. There's of course also the Untitled Linux Show. If today was not enough Linux for you and you want to keep up to date with the, the latest news of what's going on there.

That's over at the Twit Network, twit. tv and be glad to have everybody there. We appreciate everyone that joined us, both live and those of you that get it on the download, and we will see you next week on Floss Weekly.

This week Jonathan, Randal, and Aaron chat about Linux, the challenges with using system modules like the Raspberry Pi, challenges with funding development, and more!

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week, David Ruggles joins me and we talk with Frank Delport about Py4j, a Java library for exposing GPIO and SPI, all kinds of stuff on the Raspberry Pi in Java. You don't want to miss it, so stay tuned. This is Floss Weekly, episode 809, recorded Tuesday, November the 12th. Pi4j, stable and boring on the Raspberry Pi.

It's time for Floss Weekly. That's the show about free, libre, and open source software. I'm your host, Jonathan Bennett, and today we've got something a little different, out of the box. Maybe outside of my comfort zone. But first we have a co host. We have Mr. David Ruggles, the David Factor. Welcome, sir.

Good to be here. I appreciate you kind of popping in at the last moment here saying, Oh, you don't have a co host yet. I can, I can do that. I could be that guy. I very much appreciate it. You're very welcome. Because I think, I, I, it's on me. It was my fault. It was a scheduling snafu that I made. But I think I was going to do the show solo if you hadn't popped in.

So, very much appreciate that.

David: It's going to be interesting. I don't remember which one it was. It was one of the recent ones I was on. It was about Mac and I ended up, oh, Brew! Brew I said I'm not a big Mac guy so many times it became a joke and, you know, now it's, and we ended up calling it the Whopper episode.

So, for this one, I'm not a big Java guy, but I don't know of any funny saying about that, so.

Jonathan: I am here to learn. The question may be, are you a Raspberry Pi guy?

David: Not, I own a couple. And I've run some code on one, but I just, It doesn't fit into my day job, and I don't have enough time for that hobby.

But I love the idea of the Raspberry Pis and that whole ecosystem, and I've done a little bit of Arduino stuff, it's kind of that same you know, software meets hardware. Bye

Jonathan: bye. Yeah, that's, that's the thing about the Raspberry Pi, and so, like, before the Raspberry Pi, there was the, the Beagleboard and Beaglebone, and before that even, there was the Arduino, of course.

If you go even further back, you have things like the BASIC stamp, so this idea has been around for a long time. But the idea of, well, let's take our, our computers that we're used to, that we interact with, and let's give them more ways to reach out and interact with the real world. And that's just the, that's the addictive part for me.

Like that you can, you can hang real hardware. You can, you could, you could do things. And so a lot of times that's turning lights on and off. That's actually where I really got started with some of this was, you know, okay, let's put a relay inside a lamp and then switch the relay. With the way that I first started doing it was the the relay was connected to the arduino The arduino could switch the light back and forth and then the arduino just had a super simple script on it That talked to the local computer That you know so that it could tell the arduino to tell the light to go off and on and then of course You know you you make a web page with a button on it so that you can you know Pull it up from anywhere and turn your light on and off and that's just you know That's geek nirvana.

Like that's that you made it when you could do that So today's guest is Frank Delport, and we're talking about Java on the Raspberry Pi, which, you know, is kind of an interesting thing. So I could, I could see this. With Java, you could do very, very quick sort of prototyping of programs. And there's so many libraries out there.

And then, you know, you bring it to the Raspberry Pi, and it, it is kind of this, like, really easy to get started, really quick prototyping platform, perhaps. So I guess I can see the point of it. Are you familiar with this project at all?

David: I looked at it a little bit enough to have a question about the change in versions.

Okay. That's about as far as I got, so I'll wait for our guests to ask that question.

Jonathan: We will definitely get to that. Let's go ahead and bring bring Frank on. Welcome to the show, sir.

Frank: Hello good evening from Belgium.

Jonathan: Huh. Well, good morning from Oklahoma.

Frank: And thanks for inviting me. You Java lovers.

Jonathan: I've, I've explained it before. My experience with Java happened. Well, one with a much earlier version of Java and two, a much earlier place in my programming career. So I am sure that some of the frustrations that I have would That I had would no longer be things because hopefully I know a little bit better what I'm doing now.

And from what I understand, the Java ecosystem has sort of come along and things are a bit nicer to work with as well.

Frank: Yeah, so people who are listening to this podcast and have a bad feeling about Java, if you're feeling dates from Java 8 or around that time, Please ignore those feelings and, and take a look at, at nowadays Java.

The fun thing is the Java from then still runs on the new run times, but the language has evolved a lot. The run times has evolved a lot of you can write much cleaner codes than you could do. A long time ago and yeah, debugging and the frustrations you have possibly are all gone now with better tooling and, but yeah, everything has evolved as we have so much better IDEs for all languages.

We have a visual studio code for people who want to start coding for free with a very good IDE. What's that? Plugins for all languages. So yeah Java has evolved a lot and, and I hope I can inspire some people who are, who have some frustrations to retry some of these and see what, what has changed.

David: In all fairness, my experience actually is even further back.

It was Java 6, I think the last Java that I touched. Yeah. As soon as I get copious amounts of free time, I will revisit Java. I've committed to that.

Frank: Okay. If you, if you're looking for a starting point, I also write blog posts for fuji. io, which is the website for the friends of OpenJDK, which is a community platform.

And on that Side, I have a small tutorial getting started with Java 30 minutes of video with all the basic functionality of Java which could be a good starting point if, if it's really new to you or you just want to see what the current state is.

Jonathan: Yeah, so let's, let's start with this question that I, I threatened that I was going to ask, and that is, why, why, who, who thought it was a good idea to put Java on the Raspberry Pi?

And of course I asked this very tongue in cheek but I'm sure you, I'm sure you get it a lot. Like, why Java on the Pi?

Frank: Well we go back, I think, about five years, In history, and I was doing Java development at that time for a company in Belgium, Televic Rail, who builds display systems for, for trains and communication systems for trains.

And we were also using Javavix. So those are, Javavix is a graphical user tool for Java. So those were my two tools that I was used to use to, to work with, and I wanted to create a touch based touch screen based application for my son for his drum boot. So it was then nine years old. So a little application on a touch screen where he could start some lights in his drum boot and, and some flushing let strips.

And I wanted to create that with Java and Javavix because those are the tools I know. I wanted to use Raspberry Pis because I had a few of them because I was I'm also doing a Coder Dojo, the coding club for kids. So we have some, some of the stuff laying around and, and, and they say, if you want to learn something new, you should focus on one thing.

And for me, it was the electronics, communicating with. electronics hardware from software. So I needed to use the tools I already knew, Java, Java, Vix, and, and, and Raspberry Pis. I've used that a bit. I used Linux before. So the new thing for me was communicating with, with those devices. And that's how I started with some basic experiments, blinking a LED.

The hello world in, in electronics and I found at that time that there was not that much documentation about running Java on a small PC like the Raspberry Pi. We go five years back. So there were not the Raspberry Pi five we have now, which is a full power device. So we had the Raspberry Pi three, I think at that time.

So I, I needed to find out a lot of stuff and experiment, and that's when I started writing blog posts and, and people who are watching this podcast as a video, they can see my book behind me here on the side. So I ended up writing a book which landed me a new job. So it's, for me, it was a personal journey of, of combining software and hardware.

And Java is my language, the language I use the most and love the most. So that's why I thought that Java on Raspberry Pi is actually a good, good idea. And during that journey, I, I bumped on Pi4j, which is a project which is already I think it started in 2012 a Java library to make it easy for Java developers to communicate with the GPIOs of the Raspberry Pi.

So interact with electronic devices and, and, and, and in those, those last years, I got involved in the, into the Py4j project itself. Thank you. And that's why I'm here to share this message and this love for electronics on the Raspberry Pi.

Jonathan: Yeah. And, and so, you know, with the, with the Pi it's kind of been interesting because the, the, the Linux kernel in general, the Raspberry Pi, it seems has sort of dragged some of these things into the kernel, like not necessarily GPIO access from the kernel, but exposing that to user space.

Like from, from what I can tell, that, that subsystem Was created because of the Raspberry Pi and other devices like it. So how much, how much of this worked when you got involved with Py4j a few years ago? How much of this was already there and working and how much, how much stuff did you bring to it?

Frank: Everything worked actually. So, so on the Raspberry Pi, you had SPI, I2C PBM, all those protocols. You could use them on the Raspberry Pi from Java. And how that was done was by combining the Java code with WiringPy, which is a library At that time to bring this GPIO to different languages also to Python.

So that was actually using wiring pi. And and that's the same thing that still have is happening. So from pi for J some C code is used. to interact with this GPIOs through JNI, which is a way to include C headers in Java code and being able to call this. And what happens is, is when I found this project about five years ago, It, the original creator of it, Robert Savage, an American guy, he was working on a new version to bump it to newer Java versions and new architecture and modern Java, let's say.

But he got involved into other projects, his personal, his work projects. So he had something which was nearly finished. But not completely finished. And that's when I joined. I mainly focus on documentation, writing about this, telling about this. And then some other people like Robert a Swiss guy, Robert, we call him in the project.

And then Tom arts, who is also an American guy who, who creates a lot of example projects. They also joined the project. And now Robert, the original creator moved out a bit. And that's what happens in, in a lot of open source projects.

Yeah.

People start it, then get in, yeah, get a new job, get, get other priorities.

And we are lucky with Pi4j that we have this community joining in. And like you said, the kernel support or, or what Raspberry Pi is doing inside the operating system to be able to interact with this, with this GPIOs shifted a lot now with the new Raspberry Pi 5. They have this new chip the RP1 and that chip is responsible for the communication with these GPIOs.

And what happened with Pi4j is we assumed that it wouldn't work as is when the Raspberry Pi 5 launched. So it took some time before I had the first device here. I tested it and didn't work. So we created the ticket. We need to find out why. And, and the problem is pretty clear. You have these, these C class files we include to interact with the GPOs and they were not compatible with this new chip and lucky, luckily this is an open source project.

So at some point someone said, Hey, I have a pull request for you, which brings RPI5 support to Py4j and it needed some fine tuning. Then, then Robert Eich joined again into this, this pull request. He fixed it. Then Tom did some improves. So we had this whole thing. Suddenly we have support for RPI5 without planning it, which was really great.

This is a problem with, with this, with this new boards. Yeah. This suddenly appear I guess the compute five is already. Available for some people. We don't have it. So we don't know if it will be compatible. We hope so. The

Jonathan: response that I've gotten to that is we can neither confirm nor deny.

Frank: Yeah, everyone is waiting for it.

So yeah, it will be there. But but that's yeah, that's how this project evolves. And it's nice to see that that people love it. Because they are Java developers. I want to experiment with this. I talked about this at several conferences and then people are building very fancy stuff with it. We know that there are some companies who use it, but don't talk about it because they are not publicly used.

Please. Telling we actually just buy Raspberry Pi fives and put them in a fancy box. But yeah, those things are happening and that's also the disadvantage of open source. It's there. It's on GitHub. It's a Maven library. It's a Java library. You can get from the Maven repository, so anyone can use it without telling us.

It seems pretty stable because we don't get a lot of issues. Issues we get on GitHub also get solved very easily by the community again, so, which is really great. And this is, yeah. How this project is, is evolving and, yeah, for me, so many years ago when I started doing this, the blinking let's.

Was after five days of trying out to get Java on the Raspberry Pi and then having something doing something Yeah, that was my aha moment. It works on my machine. I can blink let's which is no rocket science at all, but Once you get beyond that point and you know, yeah, I can use this library and I can communicate to the GPOs.

The next thing is, yeah, a button and a relay and, and, and an I square C display and we can control everything. And another fun thing is, is a Swiss university, the FHNW university FACS school and something in, in Switzerland, they use. Pi4j, Raspberry Pi, Java, in their education system. So, so hardware and software engineers work together on projects where they build some kind of, like, for instance, the crazy, most crazy example is, is you know, the, the shortest route solution.

So you have a driver and he has to go to point B and he has to pass through all these points, calculate the shortest route, which is a typical software problem to solve. Now they combine it with the hardware. Student who has to build this on a 64 by 64 let matrix where they put some sheets of paper on top of it with different routes and they have to match and then they turn it into a board game.

You have to compete against the computer and you get a few seconds to indicate which is the shortest route. And then the computer tells you if you're right. So they have this whole combination of software and hardware, and they use Py4j and they contribute back with examples and with documentation changes.

And they built I don't know if you know the CrowPy, which is a little suitcase. With the Raspberry Pi and all electronic components as an experimentation kit. And again, they contributed all these example code back to the project. So it's, it's, it's fun to see if you have a, have a, have an interesting library project on, on, on open source on GitHub and you get some traction.

That it starts living by itself.

Jonathan: Yeah, some interesting stuff there. So, was Pi4J sort of originally made like for education? Was that the scope originally for it? As far as I

Frank: know, Robert just made it because he needed it himself. Also as a Java developer, he wanted to use, and at some point Oracle who, who took over the Java brand.

So they are the owner of the Java brand and, and, and also steering this, the evolutions within Java. At some point they also started something like that. So at some point there was an idea to bring this officially as a part of the, of the core of Java. Oh, interesting. And Rob, and Robert was working on his thing and he thought, yeah, at some point I can stop my project because it will be in Java.

But Oracle has taken the part of, of, of Java. Backend server stuff, databases Java fix, which is another love of me to build user interfaces. Also has bits. Yeah. Went, went on the sidetrack because of that, because they focus on the server side. But Java was originally born on embedded devices. The first Java.

Was something which ran on, on set of boxes on, on refrigerators on, on, on car systems. So that's where Java was born, but it's, yeah, it, it has this promise of, of right ones run everywhere. So yeah, it runs on the Raspberry Pi. Of course it runs everywhere.

Jonathan: Is, is Pi for J limited to running on the official?

I don't know if it's the official Raspberry Pi OS or if you throw a different distro on your Raspberry Pi, like say Fedora, does it run there as well?

Frank: I have tested Fedora, Ubuntu, that's no issue. I should try it again with the Raspberry Pi 5. I'll put Ubuntu there. I'm not sure. It depends on a few of It depends on what you want to use of the GPIOs because if it's just an input or an output or a PBM or, or you want to use I square C, those are different.

Mm-Hmm. providers we have within PI four J. And if you start it, if you start the Java application with the PI four J library, it first starts detecting which version. Of the Raspberry Pi board, am I on? Because Raspberry Pi 5 has different, needs different native code than, than, than other boards.

So it needs some way to define on which board it's running. So we, we should try it on Fido. I, I have no idea.

Jonathan: It, it might not be a problem now. I just, I have memories of back several years ago when I, it was actually Fedora. I just first started working on the Raspberry Pi and actually having access to the GPIO, which was great, except if you tried to use, in this case it was Python, if you tried to use the same Python libraries that worked on the official Raspberry Pi OS, distro, it just didn't work because the way the upstream kernel implemented GPIO support was different than the way that Raspberry Pi was doing it at that time.

So

Frank: yeah, so yeah, probably the same native C code, which is missing somewhere or, or yeah, not compatible with, with how it works. And

Jonathan: I think in that case, it was actually the, the kernel drivers themselves. I think that was that was right as the slash dev slash GPIO chip. It was coming online and making it into the kernel.

Yeah, an interesting time. David, you want to, you said you had a question that you wanted to get in. Let's let, let's let David have some fun.

David: Okay so I was looking at your website a little bit. Well, not yours specifically, but the project's website. And I actually went down into the GitHub repo.

And we hadn't really talked about the community around this yet, but I noticed that you are number four in contributors, so that is always a good sign. That means you've got other people committing code and supporting the project. So, and it looks like you've got about 20 committers. But. The actual question I had was around the versions.

Version 2. And 1, yeah. Yep, Version 2 though, you're starting to, or you did develop a new plugin model. To kind of, I guess, allow, More third party integration and stuff. So what is really your catalyst in moving? Yeah,

Frank: so that's dates back to the history of Robert Savage and the creator of the project.

So version 1 is what he initially started in 2012. It's based on Java 8. And it is a massive, big project. It included implementations for specific devices like that kind of LCD display, that kind of IO expander, like this chip, this number of chip, which made it very difficult to expand, change something in the, in the project.

And also difficult, yeah, to maintain it. So, and, and, and have people contribute some, something back to the project. So that's why at some point Robert together with some other people from, from other projects, he looked back at how did I design this project? How do I maintain it? And is there a better way to do that?

And I was around the time that also Java 11 became available. And for people who are not really aware of the history of Java, there is a big change that happened after Java 9 with how Java is architectured. So there is quite some change between Java 8 and 11 and then what happened since 11. So once you're on Java 11 on modern Java, as you can call it And what he started doing is he started working on a new repository on version two, which was Java 11, based on Java 11, and it all, the implementations for specific devices for specific electronic components got removed.

So the idea of this version two is the basic IO functionality that is needed to interact with components. So that's the core of PI four J really provide methods. to interact with the GPIOs, with all the different protocols, I2C, PBM, all those SPI, all those protocols are there. And by providing and building in a modular approach, it is easier now to extend it.

And that's exactly what happens with GPIOs. With the RPI5, with the Raspberry Pi 5, thanks to this modular approach, someone was able to say, for the Raspberry Pi 5, we need a complete different provider, which is compatible with the Raspberry Pi 5. And I can do that because I can just create a new module in this project, which is compatible with Raspberry Pi 5, and implement this specific functionality.

See? And it was very easy to test that because it didn't break any other code. But because the project changed so much and it was in a different repository, we decided to create a new repository, which is, it was a big question at the time, how should we do this? Should we clean that existing repository and put a new code in place?

But I would also break the whole arc history. So it was a bit, yeah, we had a question there, and at some point, yeah, you have to decide. And we decided to keep them as separate repositories. So version one is still a repository on GitHub, but it's archived, so you can still look at it, but it won't change anymore.

And now we have this version two, and now we are again at the point where we have to decide to move forward from Java 11 to a newer one. So, but we will stay in the, in the current repository. That's for sure. But there is we have an open issue about that. The idea is for instance, to, to bump the version to version three, for instance and then have some, some forks where we can.

If needed, go back to add security fixes, but do some breaking change that we're not compatible anymore with, with all the Java versions. But that's something which is now ongoing. It's a discussion which is now open on the Py4j version two discussion list on GitHub. So people who have some opinions there, they can definitely join and share how they think about this.

And, and that's also what we do with this GitHub discussions. They ask people what they think about. What should be the next step? Like support for cellular communication. There are, there is a very good Java library, which does only this serial communication. And we have some basic implementation in Py4j.

Should we keep it or just tell people use this library for this specific use case? And that's also how we use, use the discussion list a bit. So makes, make, does it make a bit clear what we did with the versioning there?

David: Yeah. Yeah, I think it makes a lot of sense to because I've, I've run into that situation with repositories myself where you don't want to lose all the history, you don't want to lose those commits and the logs and everything else, but it's a big enough change that it doesn't make sense to continue that.

So, yeah, that's really cool. like

Frank: if you now would go from Java 11 to Java 21, which is now the current long term support version, it's also already. One year old. So it makes sense to bump to this newest version. But actually that kind of change is just changing some version numbers. In the config files, it doesn't break or change any code.

What will happen if we bump to Java 21 is that we could dive into the code again and write some cleaner code in some places. Java, the language got some new methods which makes it possible to write cleaner code in some cases, like where you have a lot of switch statements and stuff like that, and text blocks.

I don't know if there are any A lot of text blocks in Py4j, but that kind of code can be optimized and can be written in a cleaner way now. So if we bump to this Java 21, which is just a config change and then in the next version, we could go into the sources and decide, yeah, this code now can be come cleaner and we should clean this up.

And in Java 21, I think we also got a new way to talk with C code. So, and, and there are more changes coming in Java about foreign memory, how you can write into the memory, which is shared with some C code, for instance, which will become a lot easier. Those are the kind of changes which are coming to Java inspired by what's happening in machine learning in the eye.

You want to be able to interact with C classes with C code, which is very good in doing this kind of stuff. And, and, and these foreign memory APIs, which are coming to Java, they are inspired by that goal, but it's something we can use to interact with the GPIOs also. And, and so. If we evolve to newer Java versions, that's also how Py4j can keep evolving.

Jonathan: That's actually interesting. Something I have on my list here I wanted to ask you about is the way that the question was going to go is like this, so it, does Py4j have support for the Raspberry Pi camera inputs? And then I know something that people are doing with those, it's interesting, is like AI acceleration to be able to make sense of what the camera is seeing.

Is that, are those two things that are in Pi4j yet? Are they on the roadmap? They

Frank: are not yet. We have got a question very recently, indeed. Can I access the camera? No, it's not unless someone stands up again and says, yeah, I know how to do this. As a matter of fact, I do. I have this, actually for the people again watching at this podcast, you're looking at me through a Raspberry Pi because I use Raspberry Pis as my camera.

Oh, interesting. I have an ATEM Mini to switch between different cameras. So I have other stuff lying around. But I use Raspberry Pi cameras and I want to create a little Java application that I can run on my desktop that I can zoom in with this camera and stuff like that, but yeah, it doesn't exist in Java.

So I'm a bit stuck. Yeah, of course. I could look into the Python version, but I'm a bit that's

Jonathan: heresy.

Frank: I want to do this with Java. So yeah, no, no, we don't have camera support, but it's probably possible. And there are a lot of fun stuff, things you could do at that time. I'm wondering if we should add it to Pi4j itself, because we're really focusing on those 40 pins.

And, and the camera is something different. I guess there are also already enough of Java libraries who are doing stuff with, with images. So, yeah, I don't know. It's, it's something to think about. Yeah. I'm

Jonathan: trying, I'm trying to remember. If the Raspberry Pi cameras, if they show up as like a V4L2 device instead of Linux.

Frank: I don't know. I only played with this, what is it called? Pi camera? What is this? The, the, the, the, the lip camera I think that you can, because that's what I do with this thing. And so I, I just have a little service that starts the, the camera as a full screen. thing on top of everything else. And I just use the HDMI output.

So that's what I'm using, but yeah, that's, that's very basic coding. It's somewhere hidden on my block somewhere there. You can find how I done this because yeah, that's also what I do if, if I experiment something I'm the kind of person that forgets what he's done one hour ago. So then it's fun.

Yeah. Then I have to write it down and it's fun to find back your own that can also be frustrating. I, when I started experimenting with Java on the Raspberry Pi, I always came up on my own questions or on, on Stack Overflow. So I was, yeah, going back to the same problems over and over again.

Jonathan: So one of the most fun experiences I've had is googling for a question and finding the answer in my own Hackaday article that I had written and forgotten about.

That's, yes, that's always fun.

Frank: That turns, that proves that you're an expert in that topic because you had the problem, you fix it, and then you had it again.

Jonathan: Yeah, I guess. Okay, so you have support for the Raspberry Pi devices. What about, what about some of the other. Single board computers out there, do any of the, do any of the Pi4j libraries and does the, does the code run on, say, the Pine64 or the BananaPi devices or the OrangePi devices?

Frank: That was the original idea. It got a bit abandoned in version one and it got totally abandoned in version two, but again, it's a modular approach architecture. So there is a provider in there, which is called the Raspberry Pi provider. So that means that, that when the application, the library starts, it detects I'm running on the Raspberry Pi.

So I'm loading that one. If you execute your code on a Windows machine, for instance, or a Mac, it will load the mock provider. So you can toggle an IO. But actually nothing is happening because the provider just returns. Okay. So we got the question recently. It is an issue again on, on, I have to look it up for another board.

And again, we have the same message for everyone. If you really need it, it's an open source project. We accept pull requests. We cannot support it with the people we have now in the team. We cannot do it because we are, we are Raspberry Pi people. That's the device we are using. If you have another device and you are, you know how to do this, then please, yes, join us.

And, and, and I'm looking in the issues, but I don't find it back. Immediately, which port it was. But that's exactly what, what we want to achieve. Yeah. Little issue we have is, yeah. How do they call these contributions to an open source project? Drive by contributions. Yes. Someone really needs this as this was a project and then it's gone.

Yes. And if you then need an update, yeah, you're stuck. So. I don't find which port it was, but indeed yet it's, it is a question which we got already a few times. And at this moment we are focusing on the Raspberry Pi. If someone has a good idea on how to implement this, the problem with that board was that for instance, the GPIOs were in different headers or different sections or different addresses.

I don't know. So it was not How it is now implemented in the Py4j library. It would become a bit complicated to edit, but it's codes. Anything can be done at the end, but someone has to do it.

Jonathan: You have to have somebody that cares enough about it to want to go in and make it all you know, generic to use the GPIO chip interfaces.

And then you have to have that person care enough to stick around in the project to answer questions and maintain it.

Frank: Yeah, and maintain. And I think with the modular approach we now have, It's pretty easy to add it and to keep it stable. We don't need to, the way how, how electronics are, are, are programmed or how you interact with them will not change much.

SPI is there, I2C is there, serial communication is there just simple input outputs is there. There's not much which will change in the field of electronics. So it's not that we will need to invent something completely new within the library that will need to be implemented for Raspberry Pi for all the other boards, but still the library keeps evolving.

We have some improvements. There were quite some I2C improvements and SPI lately. So yeah, those changes can go back to those specific implementations for. Raspberry Pi and other Pi types. So yeah, we need people to stick around indeed. And that's a bit, yeah. I think a lot of open source projects have that issue.

If, if people join for a short time and then yeah, that specific topic, and that's for people who say that Java doesn't change fast, that's actually Also the reason, and, and what we tend to say within Java is Java is boring and boring is good.

You, you have

fun in your private time, have fun with, with things which break whenever they want to break.

But within a company, you want to have stable stuff.

If you

start your application tomorrow, it should be fine. They behave exactly like it did yesterday and, and, and that's exactly what they want to achieve and what they achieve with Java. Java evolves and at, at this since many years now, we have a new Java version every six months.

There are a lot of evolutions within those new Java versions, but you can always run your old Java code on the new runtime. They're like, like, if you went from Python two to three, no way, but that's exactly what Java wants to have. I still have

Jonathan: PTSD from that transition.

Frank: Yeah, me, me too. And I didn't do Python development teams.

I only have some, some small tools on, on Python, so and that's what Java wants to be. It wants to be stable and boring.

Jonathan: Yeah.

Frank: But on the other hand, it is evolving and it's evolving pretty fast, even Stability in mind, and if you want to contribute something to OpenJDK, actually Java is OpenJDK, that's the open source project it's not that easy to get something in.

And that's the reason they don't want this drive by contributions from someone who has a very good ID. But then, yeah, who is going to maintain it? And, and Yeah, I think that's something we can learn from the Java, the OpenJDK project, definitely.

Jonathan: So speaking of OpenJDK, and I was thinking about this in terms of the kernel as well as Raspberry Pi, do you guys have lines of communication upstream?

Do you have a guy at Raspberry Pi Foundation or a guy at Java or somebody at the kernel that, you know, you could shoot an email to when something breaks and And, and get some opinions or,

Frank: no, since, since I started writing about this, I'm trying to get into contact with, with the Raspberry Pi Company, foundation, whatever.

I contributed a few articles to Magpie Magazine. Mm-Hmm. . But they're not that big fan of Java. Hmm. I know . But for, for reasons but the same thing like they had in the past, some, some, yeah, problems with migrating to new versions. And so, yeah, maybe that history is also from there. Do I have a line with, with Java?

Yes, I'm a Java champion. That means that I I'm one of those 400 people who are community. Selected by the other Java champions to, to, to, to talk about this. I'm definitely no OpenJDK specialist but I know some people. The fun thing is I work now at Azul, which is one of the OpenJDK distributors.

We built OpenJDK versions and, and runtimes. And, and just one example we have a project crack, which is aiming to have very fast startup of Java applications. And when we first announced this, I think one or two years ago, I immediately got the question as being the Java and Raspberry Pi guy, does this work on, does this work on Raspberry Pi?

And I tried it and it failed. For two reasons, the library itself was not compatible, the Java version. But there was something missing in the kernel of Raspberry Pi. And then I found out that, that if you just stand up and make a, make a, make a ticket on GitHub and clearly explain what's happening and what's missing and what's wrong, you don't have to fix it yourself.

If you just are able to tell, this is what I'm missing and you can do it there. It was just something, some, some something which needed to be enabled in the kernel settings, whatever. It wasn't the next version of the Raspberry Pi operating system. It was fixed together with 6, 000 and something other fixes, because I didn't know that the kernel was that big of a project.

And so that was also the first, very first time in my life, I compiled the kernel myself. Which is apparently something you have to do if you want to be a real developer. Yes,

Jonathan: yes, of course, of course. I'm glad to hear that you as well had that experience interacting with the Raspberry Pi Foundation, the technical support there, because I've had a similar instance where, and in my case it was something that the documentation said you can do with the device tree overlay to do to do fun things with the SPI chip select pins.

But and I raised an issue on their GitHub and like, Hey, your documentation says this should work. And from what I could tell, it doesn't work. And it did not take long. And they got back to me like, You're right. That doesn't work. Here's the problem. We'll have it fixed in the next version of the kernel.

I was like,

Frank: Wow. And that's, that's, you know, Jeff Gehling. I think the Raspberry Pi video creator.

Jonathan: One of, one of many, but yes.

Frank: One of many, but he's really great. Yes, he is. And, and that's exactly his message. They're all. Many boards like the Raspberry Pi, there are way better boards and Raspberry Pi for the same price, but the effort you need to take to get them running compared to the Raspberry Pi, yes, that's miles apart.

I have, I have the same experience. I have a few boards, only a few that I tried out and just finding An operating system and how you put it on the cart and how you put it on the device is already a problem and that's, there are many things you can say about Raspberry Pi company foundation and their ideas and how they are evolving, but the tooling they create and how they, how easy they make it.

To use these devices and what you can do with it. Yeah.

Jonathan: I ironically, I was, I was on discord last night talking with Jeff about this exact topic, because there's, there is another, there is another board that we both have copies of. We. But we both got sent copies of them to review and in fact, my review of it is hopefully coming out soon.

The review, my review of this thing is like six months past due, because putting Linux on it was fundamentally broken. And as far as I could tell, nobody cared about it. So I just, it's like I, I can't install the, in this case it was Fedora. It's like I can't install Fedora on this thing because the device trees were just goobered and nobody seemed to care enough to try to get it fixed or, you know, maybe, maybe these people knew how to fix it, but they didn't care enough to put out a You know, an actual guide where it'll tell you now to their credit.

I did this, the discord I was in just explain the program, the problem that people are willing to help. So like, it's not like it was complete radio silence. Yeah. But

Frank: it's strange. Why do they make this hardware? I guess they want to sell it. So they want people to use it. So why, why don't they? Try a little bit harder.

Jonathan: I mean, so in this particular case, you've got, you've got Rockchip at the top that actually makes the CPU and Rockchip, I learned last night actually that there are four kernel maintainers that are at rockchip. com email addresses. So it's not like they're entirely disconnected, but. The amount of effort that Rockchip puts into making things work is not as great as it could be.

And then you've got these small companies that repackage these chips for, you know, for various boards. And they're small, and they do not have an unlimited budget to be able to make things work in the kernel and in uBoot. And so, and then, you know, they sell their chips and they invest the money into trying to make it work, and the money runs out.

And they get to this point where it's like, we can't do anything more for you. We're sorry. Good, good luck. And which I mean, on one hand, like financially, I understand where you come to that point, but on the other hand, that's, that's not a great experience for your users. And Raspberry Pi is just, I don't know, I guess that's why they're the unicorn.

They've, they've managed to make that work. And they are slowly, but they are pushing things up to the kernel. It's just. The, the raspberry pies are such a pleasure to use .

Frank: And, and definitely when you combine it with Java, . And, and I wanna come back to, to one of your earlier podcasts. I, I've written down the number 8 0 1 mm-Hmm.

You talked with the creator of J Bank. Mm-Hmm. . And, and yes, if you want to build a Java application, a full size Java application, you need tools, you need Maven, Gradle, something to build it. To compile it, and then you have someone. Some, some tool like JBang, which takes all of this out of your hands and, and, and within the Py4j website, I have a getting started examples based on JBang.

So what is JBang? You write a Java file and a set of Java. And then the name of your file, you do jbang, and the name of your file, and it will fetch dependencies so you can use libraries it will compile it, it will make sure that Java is installed on the machine, on the Raspberry Pi in this case, so it will do all this for you, and jbang installation is just one line of code, you have one script you have to execute in the terminal, so once you have jbang, you can execute this Java code.

And, and so I have created a few examples where you don't need Maven or whatever. You just need this one file, this one Java file, and it will blink a let, or it will listen for the, for the button, or it will, what did I create with, with a little LCD display, I think, and, and then not even using Py4j but the serial the serial library that JFost serial, I think to communicate with a little microcontroller to control LED strips.

So I have all these examples are on the py4j. com website because they make it so easy to get started with something even create a Javavix user interface application with just one file. One file, all your code in that one file, which is a bad coding practice for Java developers, having everything in one file, but to get started, it's really great.

So you can have all these, this, yeah, pretty complicated functionality in one file. And that's what I love about, yeah, for instance, JPEG, which is a great tool.

Jonathan: Yeah, I'm going to let David get back to, I'm sure he's got some questions queued up. But I've got to ask you about this. And this is, so I think, I think this may be Jeff actually that first suggested this.

And it, I've just loved the idea ever since the new RP1 chip. So that is all of that GPIO and SPI and I squared C. off of PCI Express. That is how the RP1 talks to. And so, the idea is, well, why don't we just take the RP1 and put it on a PCI Express card, and put it in desktops, and maybe laptops. And I don't know if, I don't know if the Raspberry Pi Foundation, I don't know if, If I don't know if you're willing to do that, if Evan, I don't know if Evan is willing to do that, but that would be really cool.

And I have to imagine that if that happens, you guys are going to be right there saying, Hey, look, you could, you could use Java with this thing. Yes, because

Frank: we already have the implementation of, of what's happening on the RPA one. Exactly. It's just sending the data to it. That's also why I love this Raspberry Pi.

I'm, I'm, I'm from the generation from the Commodore 64. So yes, I'm that old 40, 40 years ago. But at that time I found the book in the library as was 14 years old, I think about how you can interact with electronic components with the Commodore 64. So And with the book came a print board for eight relays and I had to go to a shop to buy all these relays and all the connectors and I had to solder it and I've never done this before and I fried my my, my Commodore 64.

It was making this electronic buzz when I started it and it didn't do anything. And I unplugged it and tried it again and it worked. And from that time on, I was controlling my Lego trains

Jonathan: with

Frank: basic codes. And some of the less on my, on my Commodore 64. And then we had all these fancy PCs. And the only thing you had was a keyboard and a mouse and, and now you have this Raspberry Pi four, which is way more powerful than this Commodore 64 was for a price.

I once calculated the price of a Commodore 64. I think it's, it would now be 1500. Euros or dollars calculated to current price and you have a RedBull pie for 50. So you have so much power in this, this little device and you can control whatever device you want to it being less or less or displays or, or, or yeah, let's trips.

Which is a difficult topic, but even those kind of stuff you can control with, with, with, with Java or whatever code you want. So the idea of, you know, that the GPIO header of the Raspberry Pi was actually an accident.

Jonathan: I did not know that.

Frank: I know, maybe it's not true, but when they were designing the first Raspberry Pis, they had this chip and they connected it.

And the idea was let's build a computer for everyone. Everyone has already a television. So they put an analog TV connection in it, so nobody had to buy a new one. And then they find out, oh, but this chip has some IO pins and we are not using them. Let's just put them on a header and see what we can do with it.

And those first headers only had the IO pins. 20 connections, I think.

Jonathan: Yeah, it was, it was different. It was different. So

Frank: the header evolved also, so now we have these 40 pins, but to me, these pins, those are for me, the real differentiator between any other computer and the Raspberry Pi. Of course, you can also do a lot of other amazing stuff when you just use software, but those pins are really.

Yeah, those are great.

Jonathan: Yeah. I don't remember if it was during the show or before we started, but the, the Raspberry Pi, it's, it lets you reach out and work with the real world and it's, you know, it's, it's similar to your story with the the controlling the model trains with the Commodore 64. For me, it was the first, the very first thing was being able to switch a light on and off, and I still think that's great.

So I now have smart switches that have custom firmware on them so that I can, you know, I can I can push buttons here on my desk and turn my lights down and on, you know, and it's just, I don't know, it's the, it's the best thing. It's just the best thing to be able to have a computer to reach out and actually do things with the real world.

Frank: Yeah, and that's, yeah.

Jonathan: David, you want to jump in? I'm sure you've got you've got some stuff queued.

David: Well, I was actually going to ask about J Bang, but

Jonathan: he beat you to it.

David: I exactly. But so the other Hackaday thing that you tied to was J Releaser. Yeah. So how does Py4J and J Releaser work together?

Frank: So J Releaser is a tool you use within your Java build chain. At some point it jumps in and it can create a release and, and, and create release notes and that kind of stuff. And you also had them in the, in the, in the podcast, indeed. We don't use it for Py4j, the library itself, because we already had a build flow there.

But we use it from, for some other libraries and tools. So it's part of our GitHub actions. I, I have to be honest, I really hate pipelines and, and building stuff. And, and this, for me, this is the boring stuff. I want to build new codes but packaging and distributing it. And like live Java libraries, when you have a new life Java library, you have to to be, to, to make it available to others.

You put it in the, in the Maven repository, which is a public thing. This whole flow. is boring. And once you have it set up, it will run forever. And that's exactly what we do with JReleaser. So we have it in a few of the other repositories in the Py4j website Py4j GitHub projects where we use it to create releases, but it's really amazing tool.

It's, it's again, something which those are the best tools, tools that people need for themselves to automate the boring stuff. And then suddenly they find I other people are also interested in this, and then it becomes better and better and better. Mm-Hmm. . And, and, and that's exactly what happened with Bo Jang and, and j Releaser.

They are such an amazing tools within the Java ecosystem. And yes, I using where they are meant for, to, to automate the boring stuff and make it easy to, to publish a new version of a library in the, in the Maven repository.

Jonathan: So I've gotta ask, what, what license is PI four J under?

Frank: It's yeah, that's a very good question.

One of the free ones what is it again? It's Apache 2.

Jonathan: Okay, that's a, that's a permissive license, I think, right? Like BSD celled?

Frank: Yeah, I'm, I'm not familiar with all these licenses. There's a whole discussion about licenses I once had when I started a new Java project. What is the best license? It depends if you ever want to make money out of it.

It depends. That's, that's the big question from the start, but it was already there. So this project is not meant to make money out of it. Not the library itself. You can hire me. No, that's not the idea. Yeah, the license is there. We're never going to change that because I heard from, I think, one of your earlier podcasts is, is you have to go back to all your contributors and ask for their approval.

Yeah. Ask for their approval. So we're never going to change that, I think. And I think this, this, this license is free enough for the project that we have.

Jonathan: Yeah. Okay. So the Apache two is a, is a permissive license. There are few restrictions on the use of the code. It is not a copy left license. And there's like this whole flow chart of like what you could do with licenses and what kind of code you can include in other projects.

And not everybody gets it right, which is unfortunate. But yeah, for, for a library like this, honestly one of these permissive licenses is really what seems to make the most sense.

Frank: I think so. Yeah. If you just look around, what others are using. That's what I do within certain new projects for myself is how do others do this?

Yeah.

Jonathan: Okay. I think I know the answer to this already, but is there any support for using the Py4j on say the Raspberry Pi Pico or any of the other embedded targets?

Frank: No, because the Pico is not a Linux system. So yes, it works on the Raspberry Pi zero. The small ones, which are also a full Linux system.

The problem you have there is there is not a lot of memory. It's 512 megabytes, but you can I have Java Spring applications running on the Raspberry Pi 0, 2, the 0. 2, which has, I think, the chip of the Raspberry Pi 4 or something similar. So the chip itself I know it's

Jonathan: 64 bit. I know Yeah, yeah. I've learned that one the hard way, that it's a 64 bit and the early ones are not.

Yeah.

Frank: So the, the, the last recipe by zero is actually, again, a very powerful device, although it doesn't have a lot of memory, but it is very powerful. So yes, you can run Java. And then the Pico is, it is a Raspberry Pi, but this is microcontroller. So it's more on Arduino. Then it is a Raspberry Pi, if I may say so yeah there are projects I know that compile Java to microcontroller codes.

Jonathan: I was going to say we have MicroPython, surely there's a MicroJava.

Frank: There are things like that. But I didn't try them. I should write this down again. That's another one for my, my, for my to do list. It would be really funny if we could just. Make one video as an example of yes. We are running Java on the Raspberry Pi Pico and then see what happens at the Raspberry Pi company.

But yeah, I have no idea what the status is of it.

Jonathan: Alright, so what's coming, what are we looking forward to next with Pi4j? What are some big things on the horizon?

Frank: The big thing will be the move to a newer Java version. So that will make it easy for us to implement new support for yeah, maybe Raspberry Pi 6, although I think that will be the same RPI.

So that we don't, that's the idea I think of, of the, of the company to have this chip now as, as the, the central point of communicating with USB network GPOs, all these things. So that will probably not break. And then just. Seeing what appears as issues for the moment, the main changes are bug fixes like better support for SPI, I2C, some things that were discovered by users, and, and we are lucky again that some of these users fixed the problem themselves.

With a pull request. So that's really, really great. And thanks to all those people. And for the rest, what we are trying to do with the website is we want this website, pubvj. com, to be the starting point for Java on Raspberry Pi. Even if you don't use the library, how to install Java, how to install Java on a Raspberry Pi 0 of the first generation.

Which has a 32 bit on V6 processor, which is an issue. I have to try that out again with the latest operating system because I think there's an issue again there, but I got, I got it working for, for many years. So keeping that website up to date having all the information available there for people to get started, we get.

quite some issues being reported, which are actually returning issues of, I didn't start the right way. So that means that something's missing in the documentation. And that's what I try to focus on is, is keeping these websites as good as possible with all the information. Like if you want to use a Java VIX user interface application on the Raspberry Pi, what do I need to install?

How can I run this? So that's what we want. I'm now working on a J bank script. To update the, the, the wallpaper, the desktop background of the Raspberry Pi that you, that you see the IP number and, and, and which version of Java is installed. Just a little helper thing, but yeah, this exists of course in Python, but.

With Py4j, we want to have this in Java, of course, so let's do this with JBang, and I have it running, I just need to clean it up, commit it, and document it, but those kind of stuff is making developer life easier for Java people who want to experiment with this.

Jonathan: Yeah, so I've got to ask, what was the, what was the craziest or most surprising thing that you're aware of?

that somebody has done with Pi4j. What has somebody done that you've heard about? That's really surprised you.

Frank: We have a few of these on the website. If you go to my English featured projects, I didn't say it well. Uh, Robert von Berg. So he has a company which uses Pi4j and Raspberry Pis. So he has this, this cabinets for the pharmacy of a hospital, and you have drawers with all the medicines.

So if you start picking medicines for a patient, the drawer you have to pick open turns green. You pull it open the box with the pills turns green. And if you put your hand in the wrong box, it turns red. So that's a business use case. But then on the website, we have a cocktail pie. So that's from the same guy who contributed the code for the Raspberry Pi 5.

He has a cocktail maker, which is based on the Raspberry Pi. We have a street artist robot. So a robot that some street artists is taken with him on his shows. And there are other things. I've built a few gaming examples where you have really have a joystick. And then a game created with Javavix and NavigGL, there are different things but I do invite people who have created stuff with Py4j, let us know and if you have some videos or, or, or.

Pictures. We are more than happy to add them to this, to this section of the website.

Jonathan: Yeah. All right. David, is there anything you want to get in before we actually wrap?

David: I am reading through these featured projects. This is pretty cool.

The jukebox,

Frank: the jukebox for instance, it's just, and that's, that's what's happening in the, in this maker space. Hey, you, you. Upgrade an existing old radio or a television and make it something new. And Raspberry Pi is a really great device for all those. And in some cases people use Java and Py4j to make something nice of it.

And that's like the Pi jukebox.

David: So I You mentioned your connection with trains and I also have been a model Railroader in the past and a real fan. So it seems like a project for the future is going to be Running a model railroad on a Raspberry Pi using Java.

Frank: That would be really, I think I saw a tweet of someone using a Raspberry Pi Zero A train, a model train with a camera to have a front facing camera and live stream.

So yeah, why not put Java on it and have some control over the lets and then the button on the website to turn them on and off. Yeah,

Jonathan: it's, yeah,

Frank: definitely.

Jonathan: So we actually have a question from, from the YouTube. from the YouTubes. We don't usually get questions from there, but I see it. Ozcan asks for open source projects, are there any rules that you really care about, like to keep the community positive and productive?

Is that, is that something that you guys have run into? Do you have any, any of those community rules?

Frank: We don't have a rule based, we don't have a rule written down somewhere. I, I find the community pretty friendly Java community. Definitely. I don't know other ones, but I only once stopped a discussion on GitHub on an issue where someone says.

I really need this. You have to fix this. And that's not how it works. Sorry. And that's what I told him. It's, this is not how open source works. If you really have an issue, then please hire someone. If you cannot fix it yourself,

Jonathan: I will be glad to fix that for you. Here's my hourly rate.

Frank: Yeah, you could also interpret it like that, but yeah, sorry, this whole Py4j project is a pet project for everyone involved.

We don't make money out of this. I, I get money for writing articles about it for some magazines at one euro cent per letter. So it, it, it, no, I don't earn money out of it. Sad, but true. So we have, we luckily have a bit of support from this, from this university. Because they really need updated docs and that's the only support we have in this project.

And that's with all projects, with all open source projects. Maven. I talked a few times about it. Maven is a built tool in Java to build your application. It's, it's how libraries are distributed. It's how the whole Java system runs on Maven. And there are four main contributors doing this. next to the job.

And that's, that, that's what's a returning problem with open source. You know, this, this, this this, this cartoon with a lot of blocks stacked on top of each other, and then one little block at the bottom. is holding up all the rest. And there's one contributor and it's an illustration of the open sources.

And I'm happy that you looked into the contributors of PI4J and said you have 20 contributors. I don't even know. And those are the people keeping a project alive. And I can only ask to people, if you have an issue, yes, we definitely want to help you. We cannot promise And as long as you stay friendly and, and help us find, if you have a problem and, and that those are the issues which are fixed very rapidly.

Someone says, I see this problem. I tracked it down. I think this part of the code could be wrong because this and this and this, and then someone says, yeah, you're right. And we can fix this. And you have the next version tomorrow. That's, that's how things get fixed. But if you just throw something like, yeah, it doesn't work.

On my machine. Yeah. Sorry. I cannot help you. Not always the most

Jonathan: helpful. Yeah.

Frank: I cannot help you. It's keep in mind that people doing this. Want to help you, but they're really doing this in spare time. Tom arts, who is creating all this example implementation, half of the website is creating has examples of him.

He just has had the surgery and he messaged this afternoon. I'm no. Dragging myself back to my desk to find Raspberry Pi that I can take with me that I can do something for the project. Those are the people working on a project like this. These are the people who really want to get things evolving, but it's in their spare time.

It's yeah. Don't, don't yeah, keep, be polite.

Jonathan: So to, to condense all that down into a single statement, your, your most useful rule is users don't get to abuse the developers.

Frank: Yes. Stay friendly. Stay friendly.

Jonathan: Alright, so I am required to ask two final questions before we let you go. And I know the answer to at least one of them, maybe, maybe it depends upon how you understand the question.

What's your favorite text editor and scripting language? I know what

Frank: you think about the language.

Jonathan: Well, see the, the obvious answer there is going to be Java, but that's a, it's a question of, do you consider Java to be a scripting language, which is kind of a loophole.

Frank: Yeah. And since Java, I don't know which one you had to do two things.

So you write Java code in a Java file, dot Java. Then you had to go through the compiler. Then you get a class file and then you could execute the class file. So you had to do two things. But since Java, I think 11, you can just, just do Java, my file, dot Java, and it will actually do the compilation for you in the back.

And then run it. So as long as you don't have external dependencies, you can just execute a file like that. So all the things I would do in bash or whatever, like going through a list of files and find something. I can do that in Java because that's, that's the code I know. So yes, it's my scripting language.

Sure. And would you bang that? And then, yeah which text editor? I am the guy. I actually did film school, so I'm actually a video editor. So everything I use is graphical. So no, I don't know how to exit film or VI. If I have to, I use nano. And yeah. Intelligent ID is the Java, itd but also Visual Studio Code.

My website is, is in, is in Cohero. So that's something I do with visual studio codes. Mm-Hmm. And those are the tools I use.

Jonathan: Yeah. Yeah. Good stuff. Thank you so much, sir, for being here today was a lot of fun and for me too. Got to learn some about Java on the pie. I've, I've gotta admit, I, I do not have time to do it and so I am very much.

restricting myself to not go in and try to hack together a quick backend to talk to GPIO on all of the different devices. Cause I just, it would, it would very much be a drive by contribution. It would not help you guys very much. So.

Frank: And, but if you make something, send us some pictures for the future projects.

Yeah.

Jonathan: Yeah. There you go. All right. Frank Delport, thank you so much for being here.

Frank: It was a pleasure. Thank you.

Jonathan: All right. So you've already told us that you feel the need to go and make your own little Java based Raspberry Pi powered train set. Whoa,

David: whoa, whoa. You might be reading a little too much into that, but okay.

Jonathan: I mean, I'll have to roll back the tape. I have to feel like that's almost exactly what you said. No, I said

David: it needed to happen, not that I was going to do it.

Jonathan: Ah, I see. I see. So, what do you think? Have we talked you into picking up Java as your next programming language?

David: Maybe? Again, you know, it's that whole copious amounts of free time, you know, that most people experience.

But, I will say that I will not be as excited Automatically negative of job going forward, as I have been to this point.

Jonathan: Oh yeah, indeed. Indeed. So Frank has his evangelism has made a little bit of progress there. Yes, absolutely. Oh, alright. Yeah, no, it's cool. And I love to see people sort of thinking outside of the box.

You know, the box that I would put them in. And You know, if you had asked me a couple of weeks ago, what about Java on the Raspberry Pi, I would have thought it was a bananas idea. And now maybe it makes a little bit more sense. And obviously there are people doing it! Doing cool stuff with it. Love to see it.

Love to see it. Alright anything you want to plug?

David: Ah, the only thing that I would plug is something and I actually plugged it on ULS, so I'll plug Twit, the Twit Network and the Untitled Linux Show, but check out the Zen Browser if you've never heard of it. Ah, yes. It, I've seen several people talk about how Arc, the browser company, they really love its simplicity and features and everything.

But the com, the Arc browser. Company went a different direction and Zen is out there trying to fill that niche and they are based on Firefox and they're open source. So I would say go check out the Zen browser.

Jonathan: All right. Appreciate you being here, man. Thank you for stepping in at the last minute.

David: Glad to do it.

Jonathan: Yeah. All right. So as far as my stuff, of course, I'm going to plug Hackaday. We appreciate Hackaday being the home of Floss Weekly now. We've also got my security column goes live Friday mornings. And then there's some other Hackaday stuff that happens for me from time to time. Just keep an eye out for that.

Do check out the Untitled Linux show over on Twit. And as far as next week, we don't have a guest yet. So if you want to be on the show or know somebody that should, you can let us know. Either tag us on the socials or you can send an email to floss at hackaday dot com and that'll get to me and we can get people scheduled.

So sure, appreciate it. Thank you everyone that was here, caught us live and those on the download. And we will see you next week on Floss Weekly.

This week Jonathan and David chat with Frank Delporte about Pi4J, the friendly Java libraries for the Raspberry Pi, that expose GPIO, SPI, I2C and other IO interfaces.

- https://www.pi4j.com/

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week, Randall joins me and we talk with Daniel Stenberg about Curl. It's an open source success story that lets you download anything and everything from the internet. It's a great show, you don't want to miss it, so stay tuned. This is Floss Weekly, episode 808, Curl. Gotta download em all.

It's time for Floss Weekly, that's the show about free, libre, and open source software. I'm your host, Jonathan Bennett, and we've got something really fun today. We're talking with Daniel Stenberg about Curl. That is gonna be, ooh, I've written, I've written code a couple of times now to use libcurl, which is part of the curl project.

I know somebody else that I'm sure has used curl, maybe written some code around it. And that is the the amazing Randall Schwartz. Welcome back as co host, sir!

Randal: Hey, hi David. Now Jonathan, sorry. And I Dave, what, where the heck did that come from? Oh my God. It's not enough caffeine and I'm just drinking out of my giant Stanley cup that you can't see anyway.

But no, yes. Jonathan, thank you, thank you for having me back on this show, despite my guffaw, this may be my last show then, we'll see. We'll fix it in post, it'll be fine. Okay.

Jonathan: Nobody will ever see this. Yeah,

Randal: right. Except the live stream and the actual show, I know it'll be there somewhere. It'll be in the transcript too, people can search for it.

When did Randall call Jonathan David for some reason?

Jonathan: Ah, goodness. So you've, I am sure you've done stuff with curl. Have you ever used libcurl? Have you written, have you written code with libcurl? I think at one

Randal: point I wrote some Perl stuff that used curl, the libcurl binding there to get some stuff downloaded.

Of course it was because Perl had direct bindings to sockets and stuff. I actually wrote a FTP mirroring software using just direct sockets. It was pretty cool. It's what actually became the basis for the CPAN. So, the CPAN had been mirrored for many, many years by just having pure Perl code do all that.

I guess I could be ensured that live girl was everywhere and everybody had it installed. I imagine we probably could have shifted to that. And I don't know what they're currently doing these days. I am, I'm sort of out of the Pearl world. I'm sort of, there is, you know something emeritus, something, something, you know, but but I actually went full time dart and flutter starting about four years ago.

So in that respect, I've been using curl from the command line a lot because there are tools that seem like they always want to install themselves on your machine using the extremely dangerous. We can talk about this and the show really dangerous, something, something piped into your root shell. Oh, that always bugs me when I see that.

How does, how does this keep becoming the meme for easy installation? But it's, I have used curl multiple times based on that. So so yeah I'm a heavy, I'm a heavy curl command line user. Yes.

Jonathan: Yeah. So let's bring, let's bring Daniel on and let's ask him about it. First off, welcome to the show, Daniel.

Glad to have you here. Hello, hello. Thank you. Good to be here. Yeah, so Hi. Hi, Daniel. Get it right, Randall. Almost David. That's probably where it came from. Almost David. Ah, it was a

Randal: blur.

Jonathan: It was a blur. So we've had Daniel on the show to talk about Curl and some other things, but it's been like, 15 years ago, back before I had anything to do with Floss Weekly, but Randall was the, the host or maybe the co host, even at that point.

I don't, I'm not sure. I think it was with Leo, like episode 53 or something a long time ago. So welcome, welcome back.

Daniel: Yeah, it's been a while. Yeah. Leo and Randall back in 2009. Yeah. So.

Randal: I think that may be the record for the longest gap on this show. Counting all the shows together. Yeah. The radio show, the TV show.

And now this show I'm sure this is like yeah, this is, this has got to be a record.

Jonathan: Yeah. So we've started out with already we have a question from the chat room and I'm going to corrupt this question just a little bit because I'm a troll and I can. And David Ruggles asks, Why is W get better than curl?

Daniel: Yeah, why is it? So I never wanted to do W curl replacement so I never did. So I have really never. Intended it to be a W get replacement. So sort of, for me, it's always been a case of both are open source and you use the right tool for the right job. So if you want to use W get, you use W get, and if you want to do something else, if you want to fiddle with the protocol in a, in a more sort of more flexible way, rather than just doing a download, then curl is probably your better friend.

Jonathan: Is it, is it fair to say then that W get is kind of like. A pocket knife to curl's swiss army knife plus plus maybe

Daniel: Yes, I would say so. I mean wget is a more limited single more narrow use case basically just download this url to file on disk while curl has a lot more use cases and Really a swiss army knife in comparison.

Yeah.

Jonathan: Yeah, but You know, I talked about using libcurl there for a while, and I think we actually, we still do, we, in ZoneMinder, we use libcurl to get video streams, video and audio streams. And that's, that's part of the process there because it, it talks the right protocol. And so it'll just reach out there and it'll grab it and shuffle bits around.

Curl gets used for sort of everything these days. Do you have a feel of like how many curl installs there are in the world? Are there more curls than there are people?

Daniel: There are way many more curls. So, and usually when I talk to people like this people always think about the. command line tool that hence the comparison with wget, right?

But the, the much more used component is libcurl and libcurl is really, really everywhere. And since that is a little component that everyone is using these days. So we, yeah, we estimate that it's used in somewhere around 20 billion installations. So it's way more than humans on the globe and, and really an average human in, in the Western world, like, like us here, we have a lot of libcurl installations everywhere in our households, you know, phones, cars, fridges, printers, TVs, video games.

So basically all of us, we have 10, 15, 25 installations of curl per person.

Jonathan: That's gotta be kind of surreal, right? To just like look around and go, my camera probably has libcurl in it. My monitor probably has libcurl in it. My led ring light might have libcurl in it Like that's just going to be a surreal feeling like my code is in all of these places sprinkled around the entire world

Randal: Well, so here would be the challenge on that.

Is there more sqlite installations than libcurl installations? Because that's also embedded. That's everywhere, too

Daniel: Yeah, and of course, it's impossible to know, and it's impossible to tell. So it's just a matter of religion, I guess, or guessing. How do you know? I mean, they have the same situation as we do, that it's completely open source, someone downloads it and installs it in a billion devices, but they never tell us, right?

So how do we know this? We can just guess and see some traces somewhere, and I don't know. I mean, I say 20 billion installations. It could be 25. It could be 17. I don't know. So it's more of a. How do you actually know? So comparing, so I often say that it's one of the most installed software components in the world, but I cannot say that it's the most, because as you say, SQ light, maybe Lib Z is one of those also.

Mm-Hmm. And, and the TLS libraries. There are other libraries that are tiny and used in a lot of places. Could be those. I don't know. It's, and it really doesn't matter. It's just in a humongous amount of places. And of course it's, it's, pretty mind boggling.

Jonathan: You've got it. You've got it. You've got enough installs.

You have enough installs now to be like a lot of installs by any metric. And beyond that, it's like, Oh, who's counting?

Daniel: Yes, exactly. And I have this graph I showed once when they, when they told me about that, they actually use curl with the, in the Mars helicopter landing. Right. And then suddenly, you know, two planets as well.

So not only those three, It's 20 billion installations. Now it can also save on more than one planet. That's pretty awesome. I wonder if

Randal: anybody's done an SQLite extension that calls libcurl.

Daniel: I would not rule that out because it's been done on a lot of other databases at least. So yeah. Cool.

Jonathan: I'm sure there's plenty of instances of using libcurl to fetch SQLite databases.

Probably. What license is curl under? Is it, is it GPL? Is it one of the more permissive?

Daniel: It's an MIT, but, but actually, but I filled with it a long time ago. So it's actually not exactly an MIT. I was stupid enough. I should say, because it's not recommended. I wouldn't say that anyone should do it, but it's an MIT.

But so it's actually in the SPDX registry, it's actually its own license. It's actually listed as a curl license, but it's basically MIT with just a few words twisted.

Jonathan: Don't do that. Don't do this at home for no good

Daniel: reason at all. Exactly. Don't do that.

Jonathan: Ah, licensing, licensing will get you every time. So it's been, like we said, it's been like 16 years.

What, what's, what's changed? What is and. The highlights only, please.

Daniel: Yeah, what hasn't changed. Right, right. It seems like 2009 is, I mean, we lost our innocence and open source has exploded in every aspect possible since then. So I think pretty much everything has changed. The internet as an as an So the internet changed, open source has changed, software has changed.

And so there's not a lot of things that that's still the same as it was back then. And suddenly, I mean, curl as a project has changed and my own. approach to it and my own situation has changed. So it's everything is different. And I mean, back in that 2009, that was also before we didn't even do a lot of TLS and SSL back then.

So a lot of clear text protocols that was only the days with HTTP one only. And so it was a lot of, a lot of things were much simpler internet transfers and protocols in general have really, they have become much more complicated, much more, they have evolved a lot in both in security and in complexity.

Jonathan: Yeah. So security is something that's real interesting for curl. I've, I've been following your blog and we've gotten some fun. So I, I, I handle the security beat. I'm not the only one, but I handle the security beat at a hack a day. And I've, I've written up your blog posts a few times here in the last couple of months because Curl is doing some interesting things.

You guys are now, you're a, you're a CNA, so a CVE numbering authority for the Curl project. And that sort of came about, from what I understand, because you had some people that were submitting, maybe you could say, low quality CVEs and trying to make more of them than they should have. And then there's also been a couple of interesting stories about We su suspect you, you suspect where AI has been involved with people trying to find CVEs.

I'd love to love to dive into this.

Daniel: Yeah. So, so first, first, the cv C the CVE side of things. So CVEs, I mean, that's, CVS are made like this's, a database of. Basically issue or backtrack numbers, right? So anyone can pretty much request a CVE ID as a number from MITRE, who's the sort of the head organization of CVEs.

And there's really no requirements from the, I mean, they don't require anything pretty much from the reporter. They just, so I want to have a CVE ID. And if you ask for that for any product, you can just get it. It's, I mean, you can do it today for, for free. And they will just give you that CVE with no questions asked, no requirements, nothing.

And then you as a, you imagine that you have found an issue. And then when you get your CVE, you can publish that at some point later on. And, and there's really nothing in the system that puts a break to that. So that, that just happens one day and that's how the system works. And it's completely. stupid and unhinged, I would say.

So there's really no filter in this. Well, there are filters, but, but, but for the majority of people, there's no filter involved, so that just happens. And, and so then suddenly one day someone can just publish a CV about your product and they can claim whatever because there's really no filter. Nothing involved that prevents that.

I mean, MITRE could theoretically have some technical person who would read it and complain, but they don't. And I'm. There's nothing. So the only way is that this numbering, the numbering authorities within the CVE, they have a veto right so they can deny your CVE, but only for the products that they are responsible for.

So there are a lot of those organizations that there are over 400 now. Well, and now curl is one of those. So now we can actually deny your CVE being created. If someone says we found a problem in curl and if they don't have enough details or enough proof of that, we can just say, no, it's not go away. Try again another time.

So basically we did that to put that filter on. So now stop that. So now we cannot get those. Ridiculously stupid CVs and in the particular one that the last one that I blogged about was one One user found a commit message from a few years ago when I mentioned integer overflow in the commit message The person extracted that and and whoa the integer overflow that must be bad So it's sort of extrapolate make a cve Publish score 9.

8 because, you know, the sky is falling. Everything is going to burn clearly because it's an integral flowing curl, but it was super stupid. It was just a delay. That was. sort of miscalculated. So sure it could do completely wrong delay, but it was really completely harmless. And it was a bug, stupid one that I fixed also years ago.

So it was sort of, yeah, it's, it should never have been a CV in the first place. It's impossible to reject because MITRE refuses to reject it because. They argue that it could be a security problem and NVD that sets the score. They also won't refuse, refuse it. So it's, it's in there. So that's, that's the CV part of things.

And this is potentially a really a danger for any open source project or any project at all, because there's, as I said, there's nothing that stops this from happening again and again, and again, really, unless you have a CNA that says this product is our responsibility. And now suddenly we are the judges of if we should deem this or allow this to be a CV or not.

So that, so now we can stop that. But then the, the, but the, the part about AI is that's a sort of a slightly different angle to everything. So, because at the same time we run a bug bounty program in curl. It's actually sponsored by the sort of the upper project called the Internet Bug Bounty, IBB. And they, they have a.

idea of sort of running bug bounties for internet infrastructure, open source projects. So they they've sponsored bug bounties for, I don't know, 20 different projects or something. Anyway, so we're part of that. And that means that anyone who reports a security problem to curl may then get a potentially big reward cash, big bunch of cash if they find a security problem, which I think it's a really great thing because it really makes people.

Put in the extra effort and spend a lot of time and to research and actually find problems. But I mean, that's a good part of it. But of course, when we say, hey, we can give you a lot of money. It also attracts the people who are sure I want the money, but I might not actually understand what I'm doing. So I'm just going to run some tools.

And then, of course, people ask some questions. friendly AI to find problems in curl and use that and report those findings to us. And, and that's one of the, some of those more hilarious reports when, when, well, yeah, hallucinations is a friendly way to say it, but it's just blatant lies, right? Sort of And sometimes that's hard sometimes to, well, it's hard to say immediately that it is wrong because the AIs are also very good at English and they're very good at, you know, producing a lot of texts and, you know, you get a report that is, I don't know, very long, very detailed and how it takes time to assess, I mean, what are we talking about?

Is it true? Is it not? And in most cases when someone has said, actually found a serious problem. It can actually take a lot of time, right? Because it's the details and you have to understand things. What's the possible outcome, blah, blah, blah. So a very good crap report takes a lot of time to dismiss.

Usually, I mean, the really bad ones, they're just, you know, someone ran a scanner, found some completely bogus things. It was a mistake. I didn't understand it. They, those are the ones you can dismiss like in seconds, right? Those are easy. They're not a burden to us. We get those all the time, but the better the crap, the harder it is.

Threw away.

Jonathan: Do you see a, do you see a scenario or a future where AI actually becomes useful for finding real security problems?

Daniel: I'm sure that they can be useful. It's probably already useful in some senses, but I think right now, I think the ones who are using these tools are not the sharpest knives in the drawer.

So, I mean, you can, you should use it as a tool and then sort of understand the output, maybe try some further dig a little deeper, and then maybe you will find something right. Not just throw some code at a tool, see what it says. Thanks. Copy and paste that report back into us without understanding at all what it said or what it meant, or what it po possibly was really, then, then that's not gonna fly for a long time.

Jonathan: Yeah. I, I think there are some companies working on this, but the, the idea there that really makes sense to me. I think there could actually be some, some use for it is an LLM that is, that is running a fuzzing scanner. And so you're actually taking the output from the AI and actually running the code.

And then you've got a test suite that will find something. And that seems like there's some potential there for actually being useful.

Daniel: Right, exactly. That that's, I mean, taking what it finds, actually throwing that back into the code to actually verify that what you found actually finds the problem for real.

That's, that's a really good way to do it. But, and of course I can imagine that you could, you know, Do something with fuzzers in general and maybe use the AI to improve the fuzzer. So sure. I'm sure AI can be used in a lot of good ways here, but I think as most things with AI right now is that you need humans involved to sort of filter a little bit and guide it and extract and, and, you know, back and forth a little bit before you actually just throw it out on someone.

Jonathan: Yeah, absolutely.

Randal: It's, it's, it's obvious that if the if the fuzz is trying to find things that are similar to what has been bro appear to have broken other things, LMS gonna be pretty good at locating those kinds of things. Mm-Hmm. . So, because you know, really what l m's doing is it's saying.

Here's a whole body of existing text. And here's a little bit of a new piece of text. Is there anything kind of in this whole body of text and sequences of things that would sort of extend this a little bit? And so, so it is the kind of thing that, that works that way. It's, it's You know, it's we could do a whole show on AI.

I'm sure actually dozens of shows have been done entirely on AI. I think that's pretty much every

Daniel: show since the last couple of years.

Randal: I was going to wonder how soon before AI comes into this show, actually, I was sort of taking mental bets about, is it going to be right after this question or right after that question?

And really,

Daniel: I mean, You can also get a sense that that's exactly the way you mentioned is that when when we have got a single security report reported and published and everything, and that's, you can already tell that that's exactly what the humans already do. When there's been one security problem, you, you can be sure that people will investigate sort of nearby situations.

Almost that or the same thing in a different part of the code or something. So it's, it's very common that when we have one issue reported, people suddenly start to report similar things in nearby code or nearby situations. And so on. So yeah, I agree that that's exactly, and I think it's actually a pretty good way to do it.

Because then you have proven that you had one of these flaws. Maybe you have another one.

Randal: Yeah, that that definitely once you crack that egg open, you know, it's like, it's you gotta make lots of, lots of eggs from there. I get it. And I, that apparently I don't cook cause it doesn't even make sense.

Oh man. Oh man. Well, I, I'm not sure what it was before the show or whether in the show, we were talking about the use of curl pipe directly into the shell. Do you have any comments about all these ways of installing things that seem to show up that are curl piped into a root shell?

Daniel: Well, I don't do that Well, I don't think I don't take that I don't take responsibility for that I I I would say perhaps that sometimes I think I mean People have downloaded things from the internet for a very long time without verifying anything.

And people tend to sort of, yeah, and if you just, if you install a code from somewhere and you don't check it out, that's roughly the same thing. So as long as you actually trust the site that you do download this from, I would say it's not that hard. that bad. I shouldn't say that. I wouldn't encourage it still.

Randal: How is curl giving enough information to the host computer to tell whether it's being piped into a shell or just going into a file? Because it doesn't, it doesn't

Daniel: at all, but you can. I've seen people do tricks that you can actually guess if it does that or not, depending on how the shell works. So there's some timing differences, potentially.

So I've seen those that have actually done it differently so that you can do some different display in the shell than the download. So you can do really nasty tricks. That's what works most of the time.

Jonathan: By default, when you URL, it'll just spit the outputs to standard out, right?

Daniel: Yes.

Jonathan: Yeah.

Daniel: As long as it doesn't detect binary in the first hundred bytes.

Randal: So what protocols have been added in 19 years? All of them. I mean, right. Well, right.

Daniel: How many protocols

Randal: are you still, I still primarily only do HTTP and FTP, or is there some other protocols that curl now is including.

Daniel: So in 2009. We were still innocent and young, and then and then all, everything broke loose.

So we implemented pretty much every email protocol after that. imap Pop three SMTP, and then we went into RTSP, RTMP and our tm, and of course all the TL s versions of those.

Randal: Mm-hmm, .

Daniel: And then we went further and went into SMB Microsoft to. File transfer things. And then after that took a little longer and then we did MQTT.

We did, and then we did Gopher over TLS. And then we added WebSockets a few years ago.

Randal: Gopher, TLS. I love it. Gopher over TLS. It's the two ends of the internet spectrum of time. It's kind of,

Daniel: yeah. And you know, this, those enthusiasts are still Gopher users. And apparently some of them are even Gopher over TLS users.

No. And those three people, they contacted us about it.

Randal: And they're just shipping files to each other.

Jonathan: That's incredible. So like HTTP 3 and QUIC, have those been added? Stuff like that? Yeah. Yeah. Yeah. The real, the real new shiny stuff.

Daniel: Yes. So an HTTP is of course, one of the primary protocols that for curl and we added HTTP two in.

So we shipped that already when the standard came, the standard came in 2015 and then we added HTTP three support early on. So we had HTTP three already in 2019. I think we started having that. And we've been shipping it to be three cents. And we Nowadays we have, as we do with, with all the, we support a lot of different TLS libraries for TLS, and we support a lot of different QUIC and HTTP 3 libraries for HTTP 3.

The situation is really complicated for HTTP 3 because of the weird Situation for APIs and the components involved to actually do quick HTTP 3. So it's, it's a little bit of a messy state there. So when I say, yeah, we support it, it's still the fact that in most places where you install curl, it won't be enabled because of situations in the surrounding support libraries.

Randal: So you just mentioned IMAP. I didn't realize that I could actually go fetch some email with my curl. Now you can fetch email and you can

Daniel: even upload email with IMAP.

Randal: Wow. Okay. Cause I'm actually trying to diagnose some problems with my current mail server. And I've been trying to like. Type they're all the right you know, Dart commands and stuff to get that to happen.

But it sounds like I might pop curl out. And so that must mean the curl dash dash help command is now what? How long? Super long.

Daniel: We have actually the other day we added the 266. Command line options. Wow. I'm not sure that's a good thing.

Randal: With all those protocols, how do you test all that? Do you have like a, a extensive test suite that's able to simulate both ends of the conversation and curl complaint? Yeah, well. In the middle

Daniel: or whatever?

We, yeah, we have custom servers written for every protocol pretty much. So yes, and so we test against our own test servers and we have unit tests and we have tests. So yeah, we do a lot of tests and of course, Fuzzing and scanning and everything. Wow. So when you, you know, when you have your code in 20 billion installations, you want it to be at least decent.

Randal: Yeah, you definitely want it. You definitely don't want to show up on this top CV list at all. And I mean, now that you can control that, you can say, Oh no, that's not a bug. But, but I mean, you don't have any bugs anymore. Yeah. Right. Gone. It's all, it's all gone. So so a lot of times I'm typing curl and I get like halfway through the command.

Then I go, I don't know what the switch is. And so I've, I think if you just type like dash dash help in the middle of the command line, will it sort of take what's already there to kind of give a hint about what the rest of the band page to show or not?

Daniel: Well, there are a lot of these command line completion scripts.

So

Randal: yeah,

Daniel: if, so for, for a lot of shells, you can just have those completion scripts and then it can complete all those command line options. And since since a while back, I also offer a, so you can do curl dash H and then do the so you can get a help for that particular switch and get a huge chunk of The man page output in the terminal.

Randal: That's what I was looking for. So there is a way to not have to read the 10, 000 line version every time. Exactly. So you can

Daniel: get the help for just a particular option you want to do.

Randal: Oh, cool. Cool. So I would ask for curl help for IMAP or whatever. So to try to understand those protocols. Cool. That way it makes sense.

Jonathan: So David in the chat room he has a metaphor here that he thinks maybe helps us understand. He says, is curl to data transfer what OpenSSH is to encryption?

Daniel: Yeah, a little bit like that. So we've tried to, or I tried to narrow what, what, So, so the question then of course comes what, what is, what is included in cURL and what isn't included in cURL, right?

What, what, what's ever is subject to be supported and what's not supported. But as long as we have a URL format for protocol and it's related to upload download, then I think it's, it's fine. Fair game for Curl to support it. But now I think there aren't that many more protocols we can add support for it.

Jonathan: Somebody will come along. It's an, it's inevitable. Like how often, how often does that happen that you get a support request that somebody goes, this obscure protocol that I care about, it would be great if it was added to Curl, like that's going to happen. A lot? Weekly? Monthly? Yearly?

Daniel: Yeah, well most of the requests are not as a pull request and then most of them will be someone just asking for hey Why don't we support some weird protocol?

Or something strange because in many cases people They found find themselves in a situation when they already have curl right and then they just want this little extra thing as well. So why doesn't curl do that as well? Because that would be really easy for them because then they could just use curl for everything.

But that's not how we work, right? So we cannot just add support for everything just because of that. So that happens regularly. And, and then of course, I mean, it's not always that easy to just add support for another protocol either. So yes, we also get the occasional pull request that just, you know, here's a new protocol someone made and, and It just never actually materializes because the one who actually wants that to happen doesn't really have the energy and effort or energy or time and whatever to actually make it happen all the way.

Jonathan: Yeah. So speaking about these requests, one of the, one of the things you mentioned in the in the show notes when we were prepping beforehand was you occasionally get crazy emails. And I went and I looked at a couple of these and I I too, not emails usually, but I get support requests for various things like that.

And it's, it's, Okay, set the stage for us. What are the crazy emails that you get?

Daniel: Well, I could set the stage by going back to the, so, curl, runs into a lot of places, right? And it is an MIT license and in the curl license, it actually says copyright Daniel Stenberg, blah, blah. And my email address. And that is, I think that is the key here.

And because Not a lot of other licenses have email addresses in them. So, so anyway, so fast forward. So when the person sits there in his car and you know, I want to enter my GPS position in my map and I can't figure out how it works. I need to contact someone and you

Randal: know, and then

Daniel: how does that person find someone to contact?

I, I guess in some cases they find some kind of open source license screen, you know, scroll through this needs to be an email address somewhere. Oh, there's a guy. I'm going to email that guy and send him some questions. I mean, that's my guess how this is happening because I get so many car questions that it's ridiculous because why, why would I get car questions?

Jonathan: Oh, jeez. That's, that's hilarious. Okay, I, I, I had not, I did not realize it was going to take that direction because I, I am used to getting, I am used to, bless them, I'm used to getting dumb questions, or, or really poorly thought out questions, or sometimes even questions where someone's native language is not English.

I'm trying to have a little bit more grace with those. I get, I get dumb questions all the time. But that's, that's a special kind of special.

Daniel: Yeah, but my, my questions, they're beyond, they're not really dumb. They're just completely out of my league, right? They're just, you know, someone asking about their account on some weird service because they found my name in a game they're playing, right?

And I had no idea. I didn't even know that it was a game or that my code existed in the game in the first place. You know, so I'm so far away from. Whatever they are talking about. So that's quite impossible for me to do anything except, you know, giggling and putting it up and say, here's another funny email.

Jonathan: Do you respond to these, these email them back and say, I'm sorry, I can't help you.

Daniel: I used to do that, but, but I realized over time that it was just, it was just. It actually turned, I mean, it was, I gave the opposite effect many times that they just think that I'm ducking the question, avoiding it and being rude by not answering it really.

So, so they, they just get upset with me and sort of, no, no, no, shut up. Help me instead of, you know, avoiding the question when I think it's really fun as this, I have this, Among those emails, one of the, that woman from emailed me about her Instagram account getting hacked, right? Why do you email me about your Instagram account?

I have no idea what you're talking about. And then she sent me a screenshot from Instagram. Look, your name is in there. Cool. So she did not think that was cool, you know,

but I of course had no idea that my name was in that app. And so, and that's kind of, it's just impossible for me to. help them with their question, actually, because I'm not involved in that. I don't know what they're talking about. I don't have any contacts. It's not my thing to answer.

Jonathan: You mean you didn't put a backdoor in Libcurl so that you can just go in and solve all these problems for people?

Randal: You can't tell us that. You can't put them on the spot like that there, Jonathan.

Jonathan: So that actually, that actually raises an interesting, an interesting thought. With, with Libcurl being as big as it is. Have you, have you, have you had any contributors that you think are trying to, to Jian, Jian Tei, John, John T, whatever, the, the, the, the, the random Chinese, yeah, the random Chinese name that tried to get the back, that managed to get the backdoor into XZ.

Have you seen anything that matches that sort of a, that sort of a pattern?

Daniel: No, never. I've never even seen an attempt like that. And that's also the, that also made me more impressed by that attack because it was so brilliantly performed in so many ways. But no, I've never seen that attempt, or maybe I've just been naive enough to not understand or dismissed it early enough so that it never materialized.

Or maybe it hasn't happened yet. And, you know, one of the main trainers I already have. relationship with is actually JR 10, right? So,

Randal: Or maybe it has happened and, Or maybe it has happened. Yeah. There's

Daniel: so many

Randal: dimensions. And we just haven't found it yet.

Jonathan: That's, that's the terrifying one, Randall.

Randal: Yeah.

Sorry. That's what I'm good at. Yes. Yeah.

Daniel: But no, I think, I, I mean, it's so, so, really, really difficult to actually perform that kind of attack. I mean, not even that guy managed that attack, right? So even though it's so excellently well performed, it failed anyway. So I think there are much better ways to invest all that money and time to actually exploit real issues that we have landed anyway.

Yeah. Those are my probably The better way to attack a current situation

Randal: in the last recent times who's contributed most of the The patches is that are you mostly still the primary author or are you have a lot of it delegated out now and you're just Taking poll requests. How does this? How does it keep getting moved forward?

I

Daniel: think this there's a lot of contributors the other day, I think we are over 1300 Authors and commit authors over time But of course, it's a narrow as a much smaller subset of people that are actually contributing a lot I still do a lot, but there are a bunch of other maintainers who do a significant portions of the commits these days So maybe 10, 20 people that are actually contributing frequently.

Randal: How well has the design held up over all these years? Do you think if you started over, would you redesign it in a different architecture?

Daniel: It has held up surprisingly well, which I think is Part of the explanation why we have succeeded so well, because we haven't had to, you know, rip it apart and do it all over and change the APIs and everything.

So I blogged about it the other day when we celebrated our 18th year of not breaking the

Randal: API.

Daniel: So I think that is an explanation why everyone can still keep using libcurl, right? Because they haven't had to change their applications for 18 years. They can just upgrade to the latest libcurl and everything keeps working.

So in that sense, we have actually succeeded. Excellently well in, in sort of separating the app from the internals of the library. But then of course, we have a few things that I sort of have regretted over the years. We should not have done that, but now we're stuck with it because we won't break the ABI.

So we have to support this until the end of time, right?

Jonathan: I mean, I've written C and C or C one of the other code against libcurl. And it was decidedly not terrible. Which is much more than I can say about some libraries that I've worked with.

Daniel: No, I think it actually is pretty good in many aspects. Then it becomes a little bit hard to use sometimes because you have so many options and so many ways to use it.

Could sort of, you can get lost in among all this.

Randal: It's good to hear that you haven't done much architecture change. Cause I know that I get a lot of questions from people who want like the perfect architecture for their Dart application. And it's like, you really don't know until you get a ways into it.

Right. But I think, I think we,

Daniel: I think we have done architectural changes within, I mean, limited, but we have still supported the same external API, but we have refactored the internals several times, but we've, we've sort of with a little bit of skill and a little bit of luck, we managed to do a fairly good API that I abstracted a lot of internals good enough so that we can, could actually remodel things.

So we could introduce things, for example you know, when, when HTTP2 was introduced back in 2015, right? So suddenly, that was really one of the first protocols that suddenly it was not a single request for a single connection, right? Then suddenly you could do multiple requests over the same connection.

That's a completely different paradigm. So, and by, but by pure luck, really, we had an API that really was agnostic to that. So we could just Transition into that world without breaking the API, we could just add an option and suddenly we could do that. And that was, of course, I mean, we couldn't have foreseen that like 15 years before it was just.

Yeah, we just decided on that API and it happened to work that way when we transitioned how we do protocols. I mean, it's not unlikely or I mean in the future, maybe we can come up with something else that will not be possible with existing existing API because some new thing is not possible to do with this old API.

Jonathan: When, when did the dual API come about because in lib curl you've got lib curl easy and lib curl multi, and so easy. The lib curl easy. You can, you could be grabbing files in just literally like four or five lines of C code. It is like, it says on the 10, it's easy, and then if you wanna do something more complicated, you've got the multi what the, what you guys call the multi interface.

What did that come about? Was that from the beginning?

Daniel: No, well, from the beginning I had the idea of making layers of APIs sort of provide something that would be easy to use to just get the file or get the transfer done when you don't need to do anything fancy, sort of quote unquote fancy. And, but but then I wanted to also have a way to do more than one transfer in parallel.

I struggled a bit to, to do that. figure out how to do that in the API. And then I came up with this solution so that we sort of build the same API into the other API in, so to speak, so that you can just build a lot of transfers and make sure that all of those transfers happen in parallel instead of serially.

So it wasn't really, Sort of from the beginning, but we had this sort of the foundation for the beginning and then we sort of worked it out over the years.

Jonathan: Yeah So we've talked about the the maintainer burden and like how you're still very involved with it does does libcurl pay your rent? Is this how you feed your family?

Have you, have you managed to turn libcurl into a career?

Daniel: Yes. So in, since 2019, I do curl full time now. Yeah, that's great. And yes, so that's, this is the only thing I do. So now I have paying customers, paying to get the code for free.

Jonathan: Yay. How do you, how do you make that transition? Like there are, there are a multitude of programs, open source programs in the world that just, they would love to make, to turn that corner.

What, what does that process look like? How did, how did you, how did you get to that point?

Daniel: It's a really tough Well, yeah, that helps. Yeah, but still, I still struggle with sort of that exact step, right? How do you go from not doing it as a work at all until sort of do that only? Or even doing gradually, that might be even harder, right?

So how do we actually do that? Turn that from not having it as a job and having it as a job. And for me, it was I was in a fortunate position because I had friends who worked then at Wolf SSL and they believed in this as a business concept. So I sort of, I just moved my product into their library pretty much and say, now we sell support under this umbrella.

You pay me my salary from day one and we start selling this and they believed in it and we could do it this way. So I think that was a great way for me to. Kickstart the business and also have a get an infrastructure for the support and for contracts and stuff like that already from day one.

Jonathan: Yeah, that's great.

So is that still the case? If somebody needs some sort of support for for curl, do they go through wolf SSL?

Daniel: Yes, they still handle the business. They signed the contracts and I do. I do pretty much all the support. Well, I have a few others that could do. You know, the basic ask the follow up questions, which version have you tried to blah, blah, blah.

But then otherwise it's pretty much me

Jonathan: Does that does that keep you busy? All right Are you just running around like a chicken with your head cut off trying to keep up with that or is it manageable?

Daniel: It is it's quite manageable. It's actually it's actually pretty good Pretty neat situation, I think. For me, all of this has just turned into sort of landing, landing the dream job, really, because now I work full time on my spare time projects.

And most of the time I'm not occupied by projects. Customer support cases. Most of the time I can pretty much decide myself what I want to spend my time on. And that means, you know, working on much code, reviewing, writing new stuff, whatever you want. Whatever you think is the project needs right now.

Randal: Is this do you have a big community?

Do you have like, like live curl meetups and stuff and annual conferences?

Daniel: We have an annual conference. I wouldn't call it big. But we can we can gather maybe 25 people in a room. All right. All right. But it's an awesome conference. Just curl stuff for two days. Yes.

Randal: Wow. That's great. That's great. Do you have like a, you have like mailing lists and like maybe a discord or something?

Daniel: We have a mailing list. We have IRC channel on of course a lot of things on GitHub.

Randal: Yeah.

Daniel: So yeah, we have a pretty active community of people. It's happening a lot of things. So, I mean we're doing a release tomorrow, actually November six. And in this, we have, we have a eight week release cadence.

So we do new releases every eight weeks sort of on the clock pretty much. And tomorrow we ship 260 bug fixes and five changes. So there's a lot of things happening. Even though we've been around for 28 years and everything seems to be the same.

Jonathan: Do you have problems with distros and distros in particular, but I guess this will be true for a lot of things.

Shipping super, super old versions of curl and lip curl.

Daniel: Yes. Period. Of course. I, I, I don't know. I, I, I sometimes I feel like I'm, I'm even more of, I don't know, hurt or affected by this than others, because for some reason people like to get stuck on really, really old versions of curl. And I think sometimes the distros are not the worst offenders here or people using old outdated end of life versions of distros.

But even a lot of these device manufacturers, they, you know, they got that libcurl install 12 years ago and they installed it in a device and it runs there, you know, and now suddenly they want to upgrade So sometimes we see that oh i'm upgrading from blah blah blah and now this doesn't work anymore And you know, and then you can see that.

Okay They have not touched this in a decade or so and things have changed. Yeah

Jonathan: well, I mean, I think I think a big part of that is SDKs honestly is what I blame SDKs for pieces for for hardware are terrible Pretty much across the board. You have ancient kernels that'll get shipped as part of the SDK.

All of the support infrastructure around it is terrible. It'll, a lot of times it'll have weird, you know, custom code bolted onto it that's terrible. And so you have people that, you know, they get, it's like, oh, here's this new chip to build a router out of. And it's got Ethernet built into it. It's got Wi Fi built into it.

Oh, and here's the SDK from 2006. I guess we're shipping the 2. 6 kernel with it then, huh?

Daniel: I agree, but I think it's also sometimes we have had a culture when people have allowed this to happen. So the device manufacturer, they don't want to upgrade because upgrading is a sort of, yeah, it might break, it might be work, you know, it could just stick to the old thing and it seems to work and we can be fine with that and earning sort of just, you know, Earned some more money on these devices and just avoid it.

So I think it's a little bit of that too, because in some cases it's clearly, they could have upgraded a long time ago, but somehow they decided not to.

Jonathan: Yeah. Have, have you guys felt any impact yet? Or do you anticipate any impact from some of the legislation around this that's happening, both the United States and Europe where, you know, they're, they're trying to put some of the onus back onto.

The people writing the software to, to make sure bugs are fixed and insecure. Yeah.

Daniel: There's a lot of talk about that. Yeah. The CRA, for example, in, in, in the EU. And so, yeah, I'm, I'm sort of looking forward to see where that goes and how that could possibly be converted into an opportunity rather than a problem.

Punishment for me or for us, right? So, so for example, I want to be the one who guarantees my functionality. I don't want to have any middleman on top of me saying that I'm going to guarantee functionality on these things. Just because it could actually be a, you know, a business opportunity, right? If a lot of companies say they ship products with curling and they need to guarantee that there's a functionality.

Sure. I want to be the one who guarantees that functionality, because I think I can do that, but I don't know how that will play out.

Jonathan: Yeah. You know, I've, I've, I've heard people tell stories about, you know, Open source developers, and they'll get a request from some business that, you know, they have no relationship with and like, Oh, we've got to have this and this and this to be able to put our S bomb together.

And, you know, the open source guys are complaining about this. And my take has always been, tell them your hourly rate. Yes, I will be glad to help you with this. This is my hourly rate. Let's make it happen. And sometimes open source devs are terrible about that.

Daniel: Yeah. But That's has actually been my response for the last few years, but the, the typical action from the other end tends to be silence

So it's, it's easier said than done well, but, but it, it might change then with legislation. Oh, sure. So it might. Actually go in the right direction going

Jonathan: forward. The golden part of that silence is that's one less support request that you have to deal with. It at least doesn't waste your time having to fiddle with

Daniel: it.

Exactly. So at least I don't bother to actually respond in a very long and elaborate way. I can just say, sure, I can respond if we just get a support contract first.

Jonathan: Yeah, absolutely. In some ways, I'm gonna, I'm gonna wax philosophical here for a moment. I

Daniel: just wanted to insert a little bit about that, that exact thing, because a few years ago I got several questions from NASA about exactly that.

You know, I got three emails from different departments at NASA asking me questions about, you know, who's writing this and where are you from? Is any contribution from one of these five, seven band Chinese companies or something?

Randal: I don't

Daniel: know. You know, I have no ideas from on, from which companies people are working on it, you know, blah, blah, blah.

But so I got a little bit annoyed by all those questions from NASA and I blogged about it, right? Look at all these silly questions from NASA. Ha ha. And then I moved on, of course, because they didn't answer my, I asked them, well, fun. You're using my code. You can tell me about what are you using curve for?

They didn't, they didn't want to answer that. They say we're using it to further blah, blah, blah. It was just some nonsense. Anyway, I forgot about it, but I think it was last year or the year before that a NASA guy appeared on did a presentation at FOSDEM in Brussels. I talked about their use of open source and what do you know they showed my blog post in his presentation.

At least, you know, it was red. You had an impact. Exactly. At least someone got it. I don't know if it actually had an effect, but it actually appeared on the slide anyway.

Jonathan: Yeah. So I was, I was going to point out that I think open source kind of needs to have this growing up moment where we become more willing to Ask businesses to financially support what we're doing.

And I'm, I'm just, I've got to say I am tickled pink. I am delighted that you have made this work and you've, you've actually turned it into a career and it's taken care of you. I think that is, I think that is awesome. Yeah. One of the, one of the things that recently has been in the news, I've done some writing about it and talking about it is and, and, You mentioned this question from NASA, this is what brings it to mind the Colonel has had to do some reshuffling of maintainers because of U.

S. laws and executive orders and, and geopolitics coming in and messing up everybody's open source fun. I don't, I don't want to dive into like the geopolitics of that. That's not what the show is about, but I do want to ask, is that, is that something that's had an impact on on your work as well?

Have you felt that?

Daniel: I have not I, I guess I don't have any prominent contributor from any of those countries. I think, I mean, how would I know, but not that I have detected anyway. So no one has said anything and I have no one has remarked it. So it hasn't happened, but I actually, I have actually had the reverse because I have right now a support customer who actually dropped, he had to, they actually used a Russian tool as a replacement as, I mean, as current alternative.

Hmm, which one, which they can't pay for anymore because the author is actually Russian. So they can't pay him. So actually, in that case, you

Jonathan: picked up a company.

Daniel: So it's weird. I mean, yeah, I didn't select that. It just happened.

Randal: To Jonathan's point, I think also the you may not be subject to the U. S.

restrictions. And so you know, exactly not the U. S.

Daniel: restriction. I could be subject to EU restrictions because we have a lot of Russian similar restrictions here. So I couldn't do business with a lot of Russian businesses, for example. Yeah.

Jonathan: Yeah.

Daniel: Wow.

Jonathan: So I managed to not ask about it and I meant to ask about it.

You've got something new coming that's kind of related to this support contract thing. A like a long term release, an LTS version of curl.

Daniel: Yeah. So, so I've had this actually debated that question back and forth for a long time, because we're in curl, as I mentioned, we all, We don't break the API, right?

So we keep supporting the old API all the time. So it should just be a question of upgrading to the latest. And that's what we always support in the open source project. We support the latest version. But over time, I've just come to realize that companies are still scared of that jump, even though we say that everything is safe.

And that's one of the reasons why right back again to why people get stuck on old versions. It's safe to just get stuck on an old version because then you don't have to risk getting something bad when you upgrade. So now we're finally going to bite that bullet and start offering a proper long term support version so we will get stuck in time.

Randal: Yeah.

Daniel: Or offer a version stuck a little bit in time at least but still patch it with security fixes and Important stability fixes really, and try to support that in a little bit style, the style similar to how distros do it already. And a little bit try to get some of their business back to us.

Jonathan: Yeah.

This is something that, that you've had requests for, I assume, like, you know, there's a market there for doing this.

Daniel: Yes, I, I assume there is one because I've had questions and I think there is so it's a little bit about testing the waters too. And it's not, at least in the beginning, it's, it's not that hard.

I mean, the support job grows over time, right? Because it's going to be harder and harder to backport stuff and maintain. Code the older it gets so it's a little bit of a Experiment here and testing to see how How interested companies and customers actually are in this but I have faith.

Jonathan: Yeah, i'm sure there will be some interest because you would hope at least that some of these people Places that are using these old versions of curl are doing some of this internally Watching for very severe CVEs and doing fixes.

Daniel: Exactly. So, so, so that's what I'm hoping for here. I want to aim this at these companies that are already doing stuff like this, or they want stuff like this. And you know, they're concerned about their product services and they're already having some support businesses involved for, for, I mean, obviously commercial companies are this directed to.

Commercial businesses.

Jonathan: How, how long, how long are you planning to do support? Like how old of a, how old of a kernel of a curl lib curl install. Do you think you'll backport patches to

Daniel: I've said five years now. And it's, you know, right now it's just putting the finger in there and say, how long do I actually need it to be?

Maybe five. I haven't actually no idea really how long it actually needs to be. So I say five now, maybe I need to. Adjust that as we go and I think As also as it gets more work as older as it gets I also think it gets more valuable to the customers the older it gets so I guess it's also that's also maybe an opportunity to Get more customers for the older ones the longer the support term is but I I don't know Yes, that's the easy answer.

Jonathan: Yeah, I used to think that five years was a Almost eternity for technology. And then I started a business and ran on that for a while. And the next thing, you know, I've got servers that have been installed for five or six years and have almost five years of uptime in a couple of instances. And

Daniel: yeah, I have a customer right now.

They offer their customers 13 years of API stability, 13. So they want that stability from me. So I already have that support burden sort of implied. So. I have already, you know, this set up that we don't break the API anyway. So in theory, it should be possible.

Jonathan: Yeah. We'll see. Yeah. It's

Daniel: a, it's an adventure.

We'll see. We'll

Jonathan: have to, we'll have to bring you back in five years and ask how that went.

Daniel: So see if there's any tears or joy.

Jonathan: Yeah, for sure. Randall, is there anything you wanted to get in before we, we start the wrap?

Randal: No, I was trying to think of anything else that we really haven't covered, but we've kind of covered the whole thing.

I, I think I guess the only last question is sort of a historical question. When you started all this, did you figure it would get this far?

Daniel: Oh, absolutely not.

And it is actually kind of ridiculous, right? Because I started with something that was just such a tiny little piece of toy thing, and it now has, I mean, I actually, you know, I've gone back and looked at the first code that I've actually started working with. That was 160 lines. And now it's actually more than, A thousand times more code, right?

So that's actually, it's now approaching 180 K from 160 lines. So it's the, the growth is, it's, it's amazing. No, I had no idea when we started, where we were going, what I wanted, what would happen.

Randal: I can imagine it must be similar to what, how Linus felt about. Minix when he first released it, like who knew that it would grow into like a universal world industry, you know,

Daniel: exactly.

It's just a simple little thing. Sure. We can have some fun with it.

Randal: A little kernel running on this processor. Who cares? It'll

Jonathan: never be big and important like a GNU's kernel. Right. Exactly. Yeah. Alright. There are a couple things you mentioned in your notes here that we did not get to. And I wanna give you just a quick chance to to plug what these are.

You, you mentioned earlier, and I honestly don't know what it is, W Curl. And then I also see, twirl? T R U R L. I'm not sure how one pronounces that. No. It's quite

Daniel: impossible to pronounce. But I, I, so I like to pronounce it True rel, with an extra E there at the end. So True rel is actually a new tool that I, that we started last year.

It's actually a tool for The tr is actually for the tr command as in transpose, translate as for, and for URL. So transpose, translate URLs, basically parse and manipulate URLs on the command line. Pretty much when you write a script somehow and you'll need to, you know, extract the host name from the URL or add something to the path or add a query part to the URL.

That's really tricky when you do that in the shell script because manipulating URLs is really tricky. It's next to impossible to do in a shell script. And well, you can reg exit somehow, or you can mess it up.

Jonathan: I was just thinking it's a replacement for using Perl with reg X.

Daniel: It is. And, and, and that also goes back to this.

There's been several papers over the years, how there's a common security. Problem when you mix and match URL parsers pretty much use one URL parser to do one thing and you pass on the result or you And they get some results

Jonathan: slightly differently Exactly

Daniel: because urls are a messy. Sorry thing that we should talk too much about because i'm ah, it's it's It's terrible in every aspect.

So anyway, so there's just impossible for two parsers to parse a URL the same way. So you, so, so basically, this is a way to, if you're going to use curl anyway, and you want to manipulate and work with URLs, it's a pretty convenient way to just do that.

Jonathan: So true rel uses the exact same parser code that curl uses.

Daniel: Exactly. So it's based on the exact same URL parser. So it's just a small command line tool for manipulating and fiddling with URLs.

Jonathan: And then what is wcurl?

Daniel: So wcurl, back to the, what's the difference with wget? So one of the most questions, so now people are going to giggle and tell me that the only reason, or the reason I use wget is because then you can type wget.

And apparently you can do that with your left hand only on a QWERTY keyboard. That's, that's apparently a benefit. But anyway, and then you can type a URL. So you can type wget and a URL and it will download it. And you know, you don't have to remember any options at all. It'll just do that magically. And with curl, you actually have to remember some option, capital O.

But people don't do that. So, so someone then introduced the wcurl. Program is actually the replacement for W get because you can type W curl and the URL or many URLs actually, and it'll download that URL to file on disk. So basically the W get version of curl.

Jonathan: Yeah, if you wanted to really have fun with it, you could, you could just add a little bit of code in curl that looks for the binary name that it was called with.

And if it's called as w curl just imply the dash capital o and the people could just make their own sim link and get It free.

Daniel: Yes, that's one way to do it That has also been debated Of course

Jonathan: Of course, i'm not the only one that thinks about but

Daniel: w curl is then only just now It's just a shell script anyway, so it's just a shell script that invokes curl with the right the right way Yeah makes sense Is

Jonathan: is there anything that we didn't ask you about that we should have that we that we neglected to cover?

You

Daniel: Not that I can think of. We covered a lot of area here. We did. We did. I think we're good.

Jonathan: Yeah. Okay. So I've got to ask. I don't know if we were doing this. Randall, do you think we were asking the two questions way back then? I

Randal: invented

Jonathan: the two

Randal: questions.

Jonathan: So,

Randal: yes.

Daniel: And we can, if it's the same questions, we can compare with 2009.

Jonathan: Yes.

Daniel: Yes. Yeah.

Jonathan: So, favorite, favorite text editor and shell script?

Daniel: Well, text editor is still Emacs as it was in 2009. I'm pretty sure that's not going to change.

Jonathan: It's amazing how many people are like, yeah, it used to be VI or Emacs back in the day, but I use VS code a lot now. Yeah, we don't like that. No, I used to be, I used to be as good a lot now too.

I,

Daniel: well, I can see Randall is on my side here

Jonathan: in shell script,

Daniel: shell script. Yeah. When it

Jonathan: comes

Daniel: to scripting, I tend to default to Perl actually. I'm one of those old, old people who still do that.

Jonathan: You're two for two, Randall.

Daniel: Yeah, so yeah, exactly. And I'm pretty sure I said the same things in 2009. Pretty much you could imagine that not a lot of things, you know, I'm writing code in C still too, so my life is pretty the same.

Anyway, you know, I type make in terminals. I use Emacs. Yeah. 16 years later here I am doing the same things.

Jonathan: And you just have more cores and more memory and more hard drive space to do it with now.

Daniel: Exactly. So, yeah, that's true. That's true.

Jonathan: That's and a lot

Daniel: more code to handle too.

Jonathan: Yeah. Yeah. Yeah.

Alright, Daniel Sandberg, thank you so much for being here. This has been a blast and we, we sure appreciate it. It's been really good to catch back.

Daniel: It was fun.

Jonathan: Yeah. Alright Randall, what

Randal: what do you think? He may have more disk space, but I bet he has the same percentage free. Oh, it's sad, but true.

I have seen this as a constant over the decades now. It's sad, but true. It's, it's somehow, for some reason, it always ends up being about 80, 85 percent free. Full no matter how many outside discs you bought and how many things you've wired up. It's always 85 percent full They should maybe they just ship them that way.

I don't know. Maybe that's Maybe i'm not actually getting empty discs. I'm only getting discs that have some a bunch of stuff already on them. But yeah But no, no, that's, that's, that's guaranteed. And just to, just to update. Yes, I do use VS code primarily. Now I do hierarchy max occasionally. I'm still happy that there's a little bit of me in every copy of Guru Emacs that goes out.

Cause I wrote the project for Guru Emacs from many, many decades ago. It now seems, and I do have Perl scripts doing backend work for me, but I don't actually, I don't think I've written a line of Perl and probably I don't know, a couple of months, six months, something like that. So it's all been Dart and Flutter because it's my new.

Claim to fame on you operation stuff. So about our guest, let's get back to that. Not just me. So, so it's, it's again, this is and I, and I sort of mentioned it in the show, it's like, it's, it's this fascinating thing where you invent something small and useful, and then you share it with people and they also find it useful and it grows sort of organically on its own.

It wasn't like either Linus or. Or that guy, the other guy Stallman. No, no, no. Our guests, Daniel, Daniel, see it's the D words. I kept thinking David in my head now. So that's really messed up. It's cause I, just before I was on this show, I watched something with Larry David in it and it just, it just messed up my brain.

So, so so these guys, they start out, you know, not knowing That what they're going to do is eventually going to have such a huge impact. And, and I'm, I'm happy that we live in a world where that can happen and that And, and, and yes, to give to give some credit to Stallman, the idea of fighting for free software, which became the fight for open source software, which is less restrictive,

which

became an opportunity for us to do what we're doing and, and for the world to be expanding and building upon each other's work, at least at the, the software.

Engineering level. So

Jonathan: yeah,

Randal: yeah, it's, it's, it's really good. And it's, it sounds like he's had a great time with the project too. He's still with it. So that's that means he's, it's probably going to stick around for a while. And he's probably gonna stick around for a while around it, which is good.

Have being a BFDL, you kind of have to do that, but, but you know, he could have abandoned it a long time ago, or at least having some fun. Making changes to it and fixing things and adding new features. I didn't realize they could do IMAP. Cause like I said, I needed to F I need to diagnose some problems with my IMAP servers.

So I was trying to do it the hard way, watching stuff on, you know, typing the right things with SO cat and things like that, trying to get that working. So cat can kind of get you there, but not the same way as having the whole protocol. Yeah.

Jonathan: I'm, I'm just, I'm so delighted that he's able to make a career out of it and he's, he's not, it's not killing him.

And he's actually able to make some money from it. And I just, I think that's, I think that's great. I wish, I wish more of our foundational tools, we had that sort of success story. Because we don't for all of them. Right? There's, there's some of them where somebody has been, you know, just maintaining it thanklessly and not getting paid for it for 20 years.

And I like these, I like these a lot better.

Randal: Well, like, like when we talked to the NTP guy who had basically been managing single handedly the project for, for years, you know and getting an occasional grant here and there to take care of that, but

Jonathan: yeah,

Randal: yeah. So there's definitely that.

Jonathan: Yeah. And then there's, there's even things that are super important that like nobody's even heard of, like the term info files.

Right? Like how many people across the world do you think know how to write a terminfo file? You could probably count them on one hand. Those things are obscure and ancient and super important.

Randal: And then we have entire projects dedicated to things like TZData, you know, which are critical to everybody. Yep.

And nobody knows it's a handful of guys on a mailing list that That you have to listen to crazy requests from governments to say, can you change the DST time next to next week? We can, and nobody will have it working for six months. It's just the way it is. Crazy stuff. Yep. Yep. All right.

Anything you want to plug Randall? Just that I have been doing a lot of online presence to deal with Dart and Flutter. If you're interested in all that stuff, you'll probably already know how to find me, but I'm on all the major places. Flutter. dev slash community will lead you to the places that I hang out at and I patrol on a regular basis.

I'm also appearing at conferences and doing virtual talks. I am doing a talk coming up in March. The middle of December, if you're a Pearl person, you already know about the Pearl Advent calendar. Well, the Pearl Advent calendar has been going for like 15 years now, I think. So as a special celebration, I've got like December 19th or 18th.

One of those days right in there is going to be me doing a live presentation of My half life with Pearl or half my life with Pearl. I was going to switch the two around to kind of make it funny. Half my life with Pearl where, and I gave this talk about 12 years ago at OSCON, but at that point. I was 50 Pearl was 25 and I was able to talk about the relationship between me and Pearl and my company and the people around me and what is it, what does it meant to me, what does it meant to the world?

Because Pearl at that point had already had a significant impact on creating the. com boom and everything like that. So I have this talk that's about a, about a one hour talk that talks a lot about, about to some people behind the scenes history that you've never heard before. To kind of talk about how that all kept interrelating for those 25 years.

So I'm giving a version of that talk not updated for current because I haven't done much apparel since then, but at least it will be accurate as of the, the talk I gave 12 years ago. So, yeah, yeah,

Jonathan: cool. All right. I appreciate you being here, man.

Randal: Yeah. Thanks for inviting me again.

Jonathan: Yeah. All right.

Let me hit the right button here. That button. Okay. So the things I want to plug, of course, is you can get my security column at Hackaday. It goes live every Friday. And then of course we appreciate Hackaday being the home of Floss Weekly. And the show tapes on Tuesdays and goes live on Wednesdays.

And then the other thing that I'll plug is the Untitled Linux show over at Twit. And that is live every Saturday afternoon, we have a whole lot of fun with that as well, and you should check it out if you can. So to everyone that caught us live, and those that get us on the download, we appreciate it, and we will see you next week on Floss Weekly.

This week Jonathan and Randal chat with Daniel Stenberg about curl! How many curl installs are there?! What's the deal with CVEs? How has curl managed to not break its ABI for 18 years straight? And how did Daniel turn all this into a career instead of just a hobby? Listen to find out!

• https://daniel.haxx.se/

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week Dan joins me and we talk with Josh Bressers about ANCOR and open source security. We talk about the kernel and its many CVEs, the state of the CVE system in general, what an SBOM is and why you should care, and more. You don't want to miss it, so stay tuned. This is Floss Weekly, episode 807, recorded Wednesday, October 29th, bitten by the penguin.

It's time for Floss Weekly. Floss Weekly. That's the show about free Libre and open source software. I'm your host, Jonathan Bennett, and it's going to be a lot of fun today. We're talking about Angkor and security and open source. Surely that's not a thing anybody has to worry about. Uh, that said a little tongue in cheek for those of you that don't realize.

Uh, first off, we've got. A great co host today. The wonderful, the original Linux outlaw, Mr. Uh, Dan Lynch, Dan, welcome to the show. Hey Jonathan, how are you doing? I am, I'm doing great. We were talking just before we started that, uh, this was kind of an early morning for me. So one of two things are going to happen.

Either I'm going to get tired and kind of get a little loopy and punch up happy during the end of the show, or my energy level is going to drop and it's going to be real draggy. So we'll, we'll see. I'm feeling pretty good at the moment. I've. Got the go juice, the coffee to keep me going. So

Dan: that's good stuff.

Yeah. And, uh, I managed to make it on time, even though the hours different here, our clocks have changed in the UK. It's an hour earlier. I once on, uh, on plus weekly with Randall years ago, a long time ago, I turned up an hour late because I didn't realize that the U S clocks changed differently to the UK ones.

Jonathan: I remember that. That was fun. So when we were setting up, when we were setting up for the show, um, In the past week, I made sure and sent both of you guys an email saying, Don't forget, time change is a different weekend in the US and the UK. Yeah, it's, it's no fun to, it's no fun to show up to do the show and not have everybody here.

No, no. Uh, Dan, this is actually a guest that you sort of brought to us, isn't it?

Dan: Yeah, so I was at odd camp recently, which a guest of the show only a few weeks ago now you had Gary on to talk about that with Simon as well and I was at camp and I met my old friend Alan Pope, a poppy as he's known online and he he kind of hooked things up and suggested, you know, the guest for today and I'm really pleased that he did because it's going to be a great show.

Thank you

Jonathan: Yeah, so we've got Josh Bressers, who is the VP of security at ANCOR. And from what I understand, ANCOR does a lot with open source tooling, security tooling around open source, which, I mean, this is sort of a match made in heaven for me. There's a couple of things I really care about, and so I am super interested to hearing about it.

Uh, let's just, let's go ahead and bring him on. Um, let me push the right buttons to make that happen. There we go. Josh, welcome to the show. Awesome. Thank you so much. I'm super excited. Yeah, it's gonna, it's gonna be fun. Now, you are the, you have a VP of security at Anchor. What, let's start with this. What is, what is Anchor?

Like, what is the, the, the, the problem space that you guys as a company are in?

Josh: Yeah, for sure, for sure. So, we have a product, an enterprise product, built on a foundation of open source. We call ourselves, like, Next Generation Software Composition Analysis. Everyone has Tons of open source today in their environment.

And so we have tools that help you scan into your stuff, basically, figure out what the packages are, and then you can do vulnerability scanning. You can do, like, policy management and enforcement, things like that, plug it into your CI system. You know, there's, we, we, the, the, the US federal space runs us a lot.

There's tons of compliance in that universe, and they're running open source absolutely everywhere. It's everywhere. Mm hmm. Mm hmm. And so you've probably heard of things like, you know, like SSDF and STIG like, we have a FedRAMP webinar in a couple of days to talk about all this stuff. And it's just, it's a, it's a strange universe.

And I've been in open source just literally forever at this point. I mean, more than 20 years. And so it's super fun. We've got these two open source projects, SIFT and GRIPE, that get quite, quite a lot of attention. SIFT is an SBOM scanner. You can point it at containers and directories and whatever you have.

It will generate a software bill of materials for. Kind of whatever and then you can take that software bill of materials, and that's the thing that you can then, you know, look at for policy enforcement you can scan it for vulnerabilities. You can just ferreted away for later like that. The famous example is log for J right where we all we are.

Everyone has battle scars from that. And if you had. A catalog of all your open source, you know, do I have log for J means you go to the search box and type log for J and you're like, oh, there it is over here. Let's go look and see what's going on versus saying, do we have log for J? Like, I don't know if we have log for J.

Someone should go find out. Yeah, it, it, so it's, it's been wild. And in fact, it's funny because I started at anchor, I think like a month before the log for J incident happened. And so like it was trial

Jonathan: by fire.

Josh: It was amazing.

Jonathan: Yeah. Yeah. Yeah. So, so SIFT is the S Bomb finder. What is, what is GRIPE then?

Josh: Uh, vulnerabilities.

So you can either point it at the same artifacts, be it directories, containers, whatever, or you can just hand it an S Bomb and the, the, the magic there. Is so you scan something for an S bomb, right? And your software bill of materials theoretically doesn't change because you've got your container or your release or whatever, but the vulnerabilities do because there's new vulnerabilities found every day, every week, whatever.

And so you need to keep re scanning things for vulnerabilities over and over again. And if you have to scan, let's say like, you know, a multi hundred megabyte container that can take, you know, 30 seconds, a minute, whatever, depending upon your hardware. Versus if you've already done the scan and you have like your software bill of materials.

You can usually scan those in milliseconds. And so there's like a huge advantage just hanging onto those documents and their JSON, right? Like maybe a couple of megabytes max.

Jonathan: And so gripe is, is mainly just going out and looking at known CVEs and comparing it to the software and the versions in your, in your S bomb.

Exactly, exactly that, yes. Yeah, it seems like the world kind of woke up to the idea of, boy, we really need to know what software we're running. I think Log4j was really, yeah, like you say, it was the point where everybody said, oh hey, SBOMs might be a good idea.

Josh: Yeah, yeah, for sure. And well, so I think what happened in Log4j is, so I'm, You can imagine I've spoken with many people in corporations and open source, like all over the place.

And I think what happened is a lot of organizations didn't realize how much open source they actually had in the log for J incident happens. And they, you know, send out the minions to go figure out where, what our open source is. And then they're like, holy crap, where did all this stuff come from?

Because, you know, any, any open source nerd. We knew open source had been taking over the world for literally decades, and we knew it was running everything. But I don't think a lot of the, we'll say, business folk had quite figured out what was going on until log4j. And so I feel like that is truly the moment all of the businesses, all of the people were like, holy crap, this is just literally everywhere.

And that's the moment. I knew some people that were like, can we stop using it? And they were just laughed at because like at this point, no, it's completely out of the question. Right.

Jonathan: And unless you want to turn all of your logging off for your product,

Josh: all of your everything. I mean, it's wild because there's all these surveys that keep coming out.

I mean, I think I read one from red hat not too long ago, where they said 80 to 90 percent of all enterprise applications. Our open source underneath and honestly, I have no doubt, no doubt at all that that's the case.

Jonathan: Yeah, yeah, it seems like all the, all the compliance of, you know, you, you mentioned this sort of almost like an alphabet soup of the compliance things.

Some of those I have heard of and some of them I haven't because that's, I'm not in the enterprise or federal space really. Um, but it almost seems like that's just. That's the price that we pay for having made it. We're here and open source is not going away. So it's time to play the game the big boys play.

Josh: Well, okay, so that's a really good point. So there's a famous blog post. It's called, I am not a supplier. A fellow named Tomas de Pierre wrote it back in the log for JDAs. It's been almost, I think it's been two years now since he wrote it, which is like blowing my mind because I feel like he wrote it a week ago.

But. In that he talks about the fact that he was getting these like questionnaires from companies saying, you know, oh, you're my supplier Do you do you have log4j and your stuff and he's like, I'm not your supplier Like I'm some dude who put some software on the internet Like that's not the relationship and I think that's another really wild aspect of this is the open source developers Like they technically don't owe anyone anything, right?

But if I'm an enterprise using open source I'm the one responsible for that because, I mean, if you think about open source, I often say it's more like a natural resource, right? We're like, you're going to harvest lumber from the forest. You don't ask the forest for things, right? Like you're responsible for the things you'd get.

That's a,

Jonathan: that's an interesting analogy. This is something that I know the Europeans are having to sort of wrestle with as they're, they're looking at making some legislation about security and software and wrestling with trying to, trying to fix that in law. That's right.

Josh: And they have two things in Europe that we're keeping our eye on.

And like the compliance universe is there's the CRA, which is, I do not remember what it stands for anymore. I apologize. But that's the one that's talking about, like, you need to track yourself or you need S bombs. You kind of need all that stuff. And then there's another directive called NIST 2, which is more of a, you know, you're an organization, you're a company and you have to follow certain cybersecurity requirements.

And like, Knowing what you're shipping and providing security updates and things like that are part of that. And one of the challenges in the open source world has been the EU directives initially just said like all software and the open source people were like, Whoa, hold on there. Like you can't have this work.

Yeah, right, right. Exactly. And so, I mean, places like the Apache foundation and the eclipse foundation and like the Linux foundation, they really did a nice job of fixing a lot of those problems. So now there are. You know, there, there's sections carved out that say like everyone except open source type things, which is good.

Cause that's what we want.

Jonathan: Yeah. I mean, our, our own Simon Phipps was actually part of that. He would keep, he's in UK, but obviously he would. Several times went to Brussels and was talking to the people making the laws and trying to educate them on you know Here's how this thing called open source works And here's why what you have written this this piece of what you have written in this law just does not reflect reality at all That's right.

That's right. It's some of the you know, I I I Did I read the entire CR? It's the Cyber Resilience Act.

Ah, that's it. Thank

you. At one point, I read either parts of it or the whole thing. I can't remember which for some coverage of it. And like, there's some really good stuff in there.

Yeah. There's actually

some really good stuff in there.

Like, you know, if you are making software, like if you are a company that is making software, if you're putting your name on it, you are required to have a security contact. It's, it's mind blowing how many companies just Don't. It's like, Oh, I found a security vulnerability. How do I tell them? They don't know.

You don't know. Nobody knows. Might as well just drop it on Twitter because there's no better place to do it. You can, you can, you can email their contact at email address, but you'll know, you'll probably never hear back. If you do hear back, what you hear back will be written by Chad GPT in today's world.

I'm experiencing that. Or a lawyer. Well, or a lawyer. Cease and desist. Trying to hack our product. Exactly. Yeah, it's, it's a little, um, so what, uh, You guys, the, the Anchor Company, um, let's turn back to that for just a second. What's sort of the, the space that you're primarily focused in? Because it seems like everybody that is in this space has their own niche.

Um, we, we talked with, uh, we talked with the guys from Workbrew, which that's actually the commercial offering now attached to Homebrew, and they were cool. Like that was, that was a great project to talk about, but they've got a, they've got a very narrow niche, and that is. People running open source stuff from Homebrew on Macs at their workplace.

And they want to be the guys that own that. We've talked with some other people that are all about, you know, uh, JavaScript, you know, the, the, the, the full stack JavaScript, and they want to be able to catch those kinds of patch or watch for CVEs in those packages. Um, and you know, there's some places that kind of seem to get left behind, like the actual.

Linux binaries and packages. There's not as many people working on that, which might be a problem. Uh, but what, what is sort of the niche where Anchor fits in here?

Josh: Yeah. So I would say we're a little to the right. You don't hear the word shift, right? Very often, right? It's always shift left everybody. But the thing we like to do is we work with, we'll say the operation side of DevOps.

Cause we also, it's like, Oh, it's the DevSecOps everybody, but there's still people who are, we'll say a little more operations than dev in every organization. And so what we do is imagine you acquire software from somewhere. It could be internally developed. It could be an external thing, who knows? And you need to deploy it.

And so you scan it when you bring it in and then you know what you have, right? You, you've got your S bomb, you know what it is. And then. There is the vulnerability aspect, which obviously is important these days, because, you know, vulnerabilities are in everything, and then also there's the policy. So the intent being that you have all this software kind of flowing into your organization, and then using the tool Anchor has, Anchor Enterprise, it gets scanned, and then there's policy gates, for example.

So you can say, like, everything looks good, we can just keep flowing it through the system. Or you can say, oh, something is weird. Like, We're going to stop the train here and someone's got to go figure out what's going on and so it's kind of that. And then the other aspect is once you have things like deployed in kubernetes will say again that whole security vulnerability piece.

You can say like I see all of these containers running in my kubernetes cluster. Now, let's reapply the policy continuously, right? That's like you always hear about, you know, continuous compliance and things like that. And this is kind of one of those examples where you can, you can get a report every day, every hour, every whatever.

And you can look at it and be like, okay, all of my stuff is passing our policy. Or you can be like, Oh, holy crap, a bunch of stuff just failed. What's going on? And then the intent is, you know, humans get involved and, um, Kind of figure out what's, what's

Jonathan: up. So is, is Anchor mainly aimed at, uh, the, the container world then?

So can it, could I say, here's my, here's my Linux server that runs my website. I'm going to, you know, point either, uh, either the open source tool or the Anchor enterprise at it and say, I want to catalog every package and every binary that's installed on that Linux server. And I want to know when a CVE hits, is that in scope?

It is not

Josh: for the enterprise product, but with SIFT you could do that, yeah, SIFT can scan just, you can point it at slash and it'll find all the binaries, it'll find the packages, it'll find, I mean, so this is actually one of the really interesting points in this whole universe that doesn't get talked about a lot is there's no such thing as we'll say like just a node application, right?

You've got like the node. js binary that runs it all. You have supporting libraries usually that help with that. To work. You've got your packages on the system. You've got maybe some random binary crap. Someone installed in their direct home directory somewhere. You know, there's like all this random stuff.

And so we like SIFT does a pretty good job of finding a lot of there's still some things that can't identify because like, for example, if you build your own binary, like Maybe we can't catalog that, it just depends how we're detecting it. And so we also have this concept that we're working on. We call it known unknowns, where it's like, I found a thing, but I don't know what it is.

So, maybe someone should go look at that thing and see, like, is it something we care about, or is it something we can be like, Eh, whatever, it's just some random trash someone threw in a

Jonathan: directory. You're almost, that's really interesting, you're almost into the realm of threat detection then, with that sort of a capability.

Interesting.

Josh: A little bit. I mean, I would say there's, there's some strange overlaps in that regard, you know, everything from like observability and you've got threat detection and kind of same. It's one of those. I have all this data. Now, what do I do with it? Right? And in fact, there's a lot of people that take the output of these kind of tools and they just throw them into their Splunk server, their elastic server, whatever.

And then they use it as part of their whole, like, detection process, where they can say, like, Oh, I've got this evidence, then I can follow it around and, and see, like, it started out over here and, you know, that file ended up over there, or whatever.

Jonathan: I could, I could see somebody taking these, these unknown files and just, let's do a mass upload of all of them to, like, Total Virus and see if we get any hits.

Exactly. For sure. Super interesting. So, uh, you're detecting CVEs. We touched on this briefly before the show started, and I think it will be real interesting to get your take on this. So you wear a couple of different hats. You're not just the anchor security guy. You're also the open source security guy.

You've got a couple of podcasts, I think, that cover this. That's right. That's right. I have a podcast called the

Josh: open source security podcast actually today. And we're, I mean, we're like seven years into that one. I mean, we're 400 episodes and change at this point. So yeah, I've been doing it for a while. I love it.

It's a lot of fun.

Jonathan: Yeah. So. You have thoughts, I'm sure, on the, um, the absolute flood of CVEs coming out of the Linux kernel then.

Josh: Yes, I do. In fact, I, so Greg KH was a guest on my show shortly after he started doing this. And I am a massive fan of the work they're doing. I think the Linux kernel is a great example of kind of doing this right.

And it is creating a ton of work and it's a lot of effort on their part and of the people who receive it. So like, if you're on the receiving end of this, I'm I, I, I totally get it. You're overworked. It's ridiculously hard to sort through all this stuff. But at the same time, the reality is these are potential problems.

And that's fundamentally what we're dealing with when we're talking about like vulnerabilities in software. It's like, this is a, a risk that we have to understand. And as Greg, you know, so eloquently puts it, when he talks about this is a Linux kernel is run on everything from like milking machines to literal rocket ships.

And so they can't say, we know exactly how the software is being used. So we can say this bug doesn't matter. They don't know. And so that's okay. But this is where. As the security people on the receiving end, we need better tools that help us categorize and understand what we're getting. And now there's like 7, 000 layers in this.

There's everything from the data in the CVE program is absolute crap and anyone who's ever worked with it will that's not No, one's going to argue with me over that point. Like we need better data. We need better tools. I mean, this is part of what I'm doing is I'm trying to help build better tools to let us deal with all this data and information in a more coherent way.

Like we can't humans have to get as. Far out of this process is possible. We're talking about now this year, we're going to see a little over 40, 000 CVEs across the industry, and that's probably way too low. I have another thing. I think I sent you guys a link to my blog post where I talk about the size of open source.

Like there are 4 million NPM packages, 40, 000 CVEs in 4 million packages. And that's just NPM. Like that feels a little low, you know what I mean? And so it's quite likely that if we. Had good data and we had more research and we had the ability, imagine a world where we had 500, 000, a million CVEs per year, and like all the security people just like, you know, had an accident, but like, that's what we need to think about is like, that's the scale of what we're dealing with.

Like open source is absolutely gigantic. The Linux kernel is absolutely gigantic. And there, there are a lot of vulnerabilities in this stuff, but we have to learn how to deal with it better. Better. That is our new challenge. Now it's not the whole, Oh, I'm going to read every CVE and figure out how it affects me.

Like, no, no, we need tools to be able to say like, this is the CVE. You need to look at today, these other, you know, 7, 000 that came out. Don't matter. Don't worry about those. And that, it's a really hard problem.

Jonathan: Yeah, so I've, I've mused just a little bit that with the kernel, and so you, you touched on it, so what, for those that don't know, what's happened is in the Linux kernel, there's been a push to better report CVEs.

And they have essentially come to the point now to where they consider more or less every bug to be a CVE. And they've got a valid point to this, like, because any bug running in the kernel space, because it's so tightly ingrained with the system, there's a really good chance that it can be used maliciously.

And so, like, that's a fair point. Um, I, I have mused that this might be just a little bit of malicious compliance. I, I, do you think, do you think maybe just a little bit there, there's a little bit of, of, of, uh, would, would that be schoenfreude? Um, So I,

Josh: I, I thought so at first actually, but after talking to Greg, I decided no.

I think like this is like actually legitimately, The right thing to do, which makes me a little sad. Cause the malicious compliance angle like abused me. Exactly. Yes. Uh,

Dan: so with this kind of increased, um, I don't even have a good word verbosity. I'm going to say, which makes me sound clever, but increased verbosity, I suppose, of, of reporting of CVS and stuff.

Is there a danger that people become less. Um, sensitized to them, if you know what I mean, so like the, the, the, the boy that cried wolf or something, do people then start to say, well these guys are always saying there's something going on, and, you know, is, do you think that ever happens, that, that the sheer volume of them now is, is kind of making people think, oh, you know, this is probably nothing?

Josh: I don't think it will because of compliance. I think when you're in any sort of regulated environment, you have auditors that care, and they don't have to do the work, so it's no skin off their back, right? If they, if they have to say, oh, you have to deal with 7, 000 of these things a day, like, whatever. So, I, I think your point could be valid if it wasn't for regulated industry, because yes, you could easily say, like, the Linux kernel is releasing, you know, hundreds of CVEs per week, and actually, I think it's more than hundreds, it's, it's two or three hundred.

Um, but yeah, all these CVEs per week and none of them matter. So I'm just going to completely ignore it. But at the same time, I guess this is actually part of Greg's talking points is if you just follow current, you don't have to care. Right. And I know that's easy to say of, Oh, just upgrade your kernel.

It's not that simple, but if you're trying to follow. More closely than we maybe did in the past, that does make a lot of these problems go away. And I think that's part of his intent, is by just having so many and saying like, just, just follow upstream. Don't try to figure this crap out, because it really doesn't matter as long as you upgrade.

Dan: That kind of goes back to Jonathan's point about malicious compliance a little bit. Is it a good way of forcing people to follow the latest current kernel and say, or more people to kind of go? And, and isn't to get up with.

Jonathan: And is it related to the dropping of all of the long term support kernels?

Because nobody used them anyways.

Josh: Right, right. Well, I mean, that was, that's the business model of the distros, right? Is we're going to support the software for decades. And everything will be fine, except, oh, we can't do it with the kernel now, because it's too big and it's too fast and there's nothing we can do.

So, yeah, I don't, I mean, so dropping the LTS, That's something I watched quite a bit, and I mean, it fundamentally just came down to no one wanted to do the work, right? And I think that's just your classic supply and demand. Like, if there's a demand, then find some money or find some people and make it happen, you know?

And it's also open source, so you can't be like, oh, those darn Linux developers. Like, no, just go do it, right? That's how it works. Yeah.

Dan: Yeah, that's true. That's very true. Okay. So, uh, so josh, I want to back up a little bit and talk cause I, I'm not, I know a little bit about security, but I'm not really up on a lot of this stuff.

So I, for the benefit of some of our listeners, I know we have a very technical crowd who listened to this show, but some of the layman, I suppose. Um, can you tell us exactly what an S bomb is and how it works? So it's a, it's a, it's a, am I right in thinking it's a bill of, uh, what is it now? Well, in fact, why am I trying to answer my own question?

I don't know what I'm doing there. Can you tell us exactly what an S bomb is and why

Josh: it's important? Yeah, I mean, so, it's a software bill of materials, right? The idea is, all of your software has stuff in it. And some of the stuff you wrote, most of the stuff you didn't. And so you have Like you might have NPM packages, you might have Rust crates in there, you might have Go, whatever.

And so basically what a software bill of materials is meant to be is on any system, any application, it catalogs all the stuff in it. So like if you do an NPM install and it installs 100 things, those 100 packages would show up in the software bill of materials. And you can also imagine that you can have like different Kind of pieces of software where you have a, a container image and then you have your software and those, those are two different software bill of materials.

Now you put them together. Now you have a third software bill of materials that is functionally the other two pasted together. And so the idea is just like, it's imagined when you, when you buy something from Ikea. Right. They always have that list of all of the parts that should be in the box. And in fact, I don't think he has ever screwed me over on missing parts, but you know, that's the intent here is if I get a piece of software, what are all the pieces in the box essentially?

And then we can start asking questions about those pieces. And like, one of the really interesting things I like to do is, so if I get an S bomb from somebody, I'll, I can, Look at it and understand it. No, I shouldn't, I should use the word look at loosely because there's usually thousands and thousands of packages of this stuff.

Like I'm, I'm kind of an armchair data nerd. So I just put everything in elastic search and then I do stuff from there, which is not, I know that's not how normal people work and that's okay. That's why we need tooling to help us do this. But anyway, someone gives me an S bomb and says, here's the S bomb for the thing I built.

And then I scan it. And then I say, what's, what do I see? Are they the same? Are there things different? Like, did stuff get added somehow no one knows about? Did stuff get removed? How did that happen? And this is like one of the really cool things you can do, is you can watch your software as it kind of traverses from the left to the right.

And you can be like, why did this file get put here? How did that happen? Or why did we remove these packages over here? I don't understand why, why we did that. And so there's, there's just all this weird stuff you can start doing and asking questions about your software, because, I mean, let's face it for the last, what, 30, 40, 50 years, the vast majority of software has been kind of YOLO.

We're like, we type make and we get something out the other end and we're like, I guess that's it. Like go run that in production.

Jonathan: Yeah.

Josh: It's, it's funny when you think about it, but at the same time, it's like this stuff is like running the world. I mean, I feel like we should know a little more about

Dan: it.

Yeah. Is there a standard format for S bombs? Could I take an S bomb that was generated by SIFT for example, and ingest it into, uh, into another. You know, program another, another, another company's tool or another project's tool? In theory,

Josh: yes. So, Okay. So we are kind of early days still. There are two primary S bomb formats.

There's something called SPDX and Cycle and DX. And They're they basically do the same thing. I mean, I know you can nitpick over Oh, this one does, you know, it records dates a little different or whatever, but what it doesn't matter fundamentally they're the same thing and the intention being that you have interoperability between tools and and whatever and Some of the tools can do this.

Okay, not all of them can because like any good standard nothing makes sense So there's tons of wiggle room And they're working on fixing it, like they're on, I mean, SPDX is I think 2. 6 now, SPD, no, SPDX I think is version 2. 6, and they keep adding changes to this stuff, and they're constantly trying to, I guess, bring the, the formats a little closer together so they work better, like one of the things SIFT does, so, SIFT, if you run SIFT, the default output format is something we call SIFT JSON, And it's funny because the S bomb people like, Oh my goodness, did you make a third format?

I'm like, no, no, we did not make a third format. We are basically just, we just take whatever SIFT has in memory and we dump it out to JSON with the intent. There is more data in that than either CycleNDX or SPDX collect with the intention being you can convert between CycleNDX or SPDX because like if you have an SPDX document you can't today convert it to a CycleNDX document You will lose information because there are those little niggles back and forth So you kind of it's kind of imagine Google Translate where you know, you you can You keep translating something back and forth and you end up with a hilarious output at the end, same sort of problem today where it's being worked on, but I mean, you have to remember too, like these formats are not like, it's not old, right?

We're, we're still very early days here and it's getting better. It's getting better. Thank goodness.

Dan: So you mentioned that the sheer amount of data and so on, is there, I'm going to, I'm sorry to have to invoke this, but I'm going to, um, is there an application for AI in this where you could use something like an LLM or something and train it with some of this data?

I mean, there's people

Josh: looking into that for sure. I, I think it is an excellent question. What can we do with them? Like, I, I haven't seen anything I would say is particularly compelling yet. Just because I feel like this data is kind of boring and There's not a lot of wiggle room. Like if an LLM gives you a, a, a weird version or a weird package name, that's potentially bad, but I know there's, there's interest today in doing things like saying, okay, I've got this S bomb, I've got all these vulnerabilities in it.

You know, what, what should I start working on first? How can we look at this data and maybe unwind it a little bit using something like an LLM to maybe understand the vulnerabilities And your product is like a really good example is so let's say I have a container image and it has a website open to the Internet, right?

That is going to have a very different threat model than if I have an application that is. Downloading a CSV file from a website and then parsing it and uploading it somewhere else, right? And so this is, that's one of the places I've seen some of the LLM efforts look interesting because you can say, like, this is what my application does.

Now, you know, based on this information, look at these vulnerability details and tell me which ones sound like they are going to affect me in a very bad way.

Jonathan: You know, I'm, I'm aware because I cover this stuff for Hackaday that, um, people are using AI in security. Research and it's not necessarily going well.

It's actually, it's actually a real problem for open source projects. They are getting particularly those that have bug bounties. They are getting vulnerability reports that have been generated by AI. And the problem with that, and this is sort of just in general, this is the problem with AI today, Um, they sound very plausible.

And these vulnerability reports, they are like, very well done, well written, very plausible sounding reports, that are totally hallucinated. Yes. Is that something that you're seeing more broadly? Is that something that, uh, I guess you guys are hooked into? To even observe it?

Josh: Not I I'm not affected by that in any way I know some people that have dealt with it and they've they put the hammer down pretty hard because it really really annoyed him a lot However, there is some really cool security research I've seen in the LLM universe where you have Researchers training up models, you know the code models and they're saying like here is a description of a vulnerability I want you to write an exploit for this vulnerability Against this application right here, and that's been reasonably successful, which is a little terrifying sometimes because most vulnerable, like what's the stat?

I think like 3 percent of vulnerabilities are actually exploited, but with some of this technology that could even if it doubled. The workload probably quadruples, right? I don't think we're talking about linear scaling here between the vulnerability rate and the work that the security analysts have to do.

And so that's kind of a weird, scary part. And then there's also people doing research where they're basically saying, I want you to just take this application and find a cross site scripting flaw in it. And that works surprisingly well also, which they are there. We start getting into, you know, the whole like 500, 000 million CVEs per year.

If we have. automated discovery, obviously, you're going to have, you know, a huge influx of vulnerabilities, which we are not at all prepared for in any reasonable way today.

Jonathan: Yeah, boy, you could imagine, uh, uh, you can imagine a scheme where you take a, uh, you take an LLM and you write the prompt that says, you know, find this kind of vulnerability and write an exploit for it.

And then you hook that into on the backend, a fuzzer. That actually tests what it could, what it spit out. And then if you were to feed that output back into the LLM, and I imagine there are people trying to do this right now, but you actually get a positive reinforcement loop that actually works, because it'll tell the, it'll tell the LLM, that one worked, that one didn't, and particularly as you start scaling up the speed in which it can churn through that loop.

Yeah, that's going to find real vulnerabilities. Of course it will. So

Josh: that happened. DARPA had, uh, like basically a security capture the flag event. And there was a, an AI, it was originally done by Carnegie Mellon. It was called mayhem. Um, David Brumley's behind it. He has a company now and he just changed his name.

I can't, it used to be for all secure. I don't remember what they changed it to, but anyway, mayhem was designed to basically look at like a binary. Find problems in the binary. So we're talking about like just operating on straight up binaries. Like we're not even looking at code here. Find problems in binaries, fuzz it, find out how to exploit it, write the exploit, then also write a patch in the binary.

So the other team can't exploit it against you and then going and attacking them with the thing you just found, which is like all automated. It's like, holy crap. This is ridiculous.

Jonathan: Yeah. Well, so that's, I guess that's the next stage of this. When we, when we finally reached security nirvana, you've also got, so you would have to write it, you'd have to write a really good test suite to make sure your program is working correctly.

You run through this process I just described to find the vulnerabilities, and then you also tell the LLM to fix the vulnerabilities, and you feed that back into it as well. And as soon as you fix the vulnerability and still pass your test suite, Might have good code on the other end of it. Hopefully. We can dream.

Yeah, right, right. Okay, so, the many, many, many vulnerabilities found, lots of CVEs. Um, how are the various organizations and agencies that are responsible for Tracking and catalog, cataloging these, you know, you've got NIST, you've got MITRE, you've got the NVD. How, how are they doing with it? Oh my goodness.

So it's, it's

Josh: been a rough year. So, so anyone not in this space might not know this, but in February or March of this year, I forget when it was, NIST runs a database called NVD, which stands for National Vulnerability Database. And what they used to do is. MITRE, which is a government contractor, runs the CVE program, right?

And so people who, like, get CVEs, you get them through MITRE. You can have a company that can assign their own CVEs, but they go through MITRE. Or, like, security researchers can just go to MITRE and ask for a CVE. So those all go into MITRE's data set. And that data is very bad. As anyone who's ever looked at it knows.

So NIST had a program called NVD, and they would take the MITRE data, And they would add, they would enrich it. They would add some version information, some like package information. They would add like severity. They would guess at severity. Their severity scores were often very bad. And they would kind of add this information.

And then they just stopped one day, like no word or warning, no nothing. And, and I, in fact, uh, uh, there's a handful of us security nerds who like, look at this data. And I remember we were like, Do you guys like didn't stop and we're like exchanging notes and being like, holy crap, like they stopped. There's, there's no new data.

What's going on. And then of course we'd email them and crickets. No one knew. And so they just like literally fell off the face of the planet for probably six months almost goodness. And then at the same time you have Sisa. starts a new program, they're calling Vuln enrichment. So now CISA started doing some enrichment.

NVD has sort of come back and they're doing some enrichment, not everything, but some, there's still a giant black hole in the middle where they didn't do anything. And then you also have MITRE trying to ask the CNAs to do a better job of providing nicer data. So now instead of having one, And a half, we'll say organizations providing not good data.

We have three organizations all providing, we'll say incomplete and difficult to use data. And so for, for vulnerability nerds right now, it's like, it's a mess. It's really hard to figure out what to do. It's really hard to understand. I mean, this is a huge challenge we have, because obviously we have gripe at anchor and gripe means vulnerability data, and so we've got, you know, a team of people that are doing their best to try to unwind all this and make sense of it.

And it's. It's not, it's not the most fun, not the most fun at all. We've ever had, but you got to do what you got to do.

Jonathan: So I don't know.

Josh: It just kind of is what it is today.

Jonathan: There's, there's several different directions. I want to go with that first. You mentioned CNA is what's a CNA. They call that a

Josh: CVE numbering authority.

That is where, and this, this is like such a fricking mess too sometimes. So, um, yes, curl, which we've all heard of. So, so the guy behind it, Daniel, um, baggery goes by online. There were all of these CVEs being filed against curl. And he was upset with them because they were kind of garbage and they shouldn't have been filed and he tried to get rid of them, but working with NIST and MITRE is like an impossibility for the most part.

They just, they don't have to listen like their, their, their customer is the U. S. government. So, haha, no. So anyway, CURL has become a CNA. A CVE numbering authority with the sole intent of being able to get rid of these crap CVEs and just not have to file them at all, which is a super heavy handed way to just like match reality.

And, and the thing is like, Daniel's way brighter than a lot of people and like he works on curl full time. So he can pull this off. But like, if you're an open source project that you work on an hour a week.

Jonathan: Yeah,

Josh: there's no way you can do this kind of thing. And so that's one of the complaints I have is just the fact that all of this, all of these organizations, all this data, it's just like such a one way black hole that no one knows how it works.

No one knows what's going on. I mean, I've been complaining forever that this is exactly the sort of thing. That should, like, exist in some sort of open source foundation

Jonathan: and

Josh: have, like, proper bylaws and, you know, proper governance and we understand how everything works and all of that, but I've been screaming into that void for more than 10 years now.

And put it

Jonathan: on

Josh: the

Jonathan: blockchain, right?

Josh: I'm sorry,

Jonathan: I couldn't resist. That will solve everything. Um, you've got a few companies out there that are trying to pick up some of the, like, GitHub really comes to mind. I've been, I've been involved with a couple of CVEs now that, uh, GIF, GitHub has been the CNA for, and they've, they've got a lot of automated tools with that.

I think they do have some human review in there as well. And that's actually been a really good experience. Um, they do.

Josh: So GitHub's doing some very clever things here, actually. Mm-Hmm. So they, they are a CNA, they will assign CVEs. Yeah. But they also have their own data set. They have the GitHub advisory database.

Mm-Hmm. and, no, I'm sorry, . They, they kept yelling at me about this. It's the GitHub vulnerability database holds GitHub advisories. . That's how that works. Okay. It's very important to get this, but they have a team of people that what they do. So actually, I said there were three organizations, there's kind of four, if you consider GitHub, where they have a team of people that look at all of the CVEs that affect the, the ecosystems GitHub cares about.

They focus on things like, like Maven and NPM and Python and Go. And there's a couple of others. I can't, Ruby is one of them. And what they do then is they. Do a really nice job of kind of teasing out what is the ecosystem, what is the package and what version fixes it. And what I love though is if, if you find an error, you just submit a pull request and then they either say, yep, that's it.

Perfect. It'll be like, I don't think this is right. Like. I think this is the thing, and then you can have like an actual discussion in the issue and you can come to a conclusion and everyone can see it because it's, you know, just GitHub and all in the open and their data, quite frankly, is the best vulnerability data that exists today.

But unfortunately, it's just a subset because obviously they do not have infinite resources to do this. So they purposely constrain themselves. But because of that, it is just so good. It's amazing.

Jonathan: Yeah. So I have tried, by the way. I cover vulnerabilities and I've found typos. When in particular comes to mind, I found a typo in a CVE on, I forget whether this is before or after the switchover.

So, you know, maybe it was at MITRE or maybe it was at NIST, you know, things have moved around a little bit over the last couple of years. I found a, a, a, a typo in what version fixed a vulnerability. Like you go to the vendor's website and they say one thing, you go to the CVE and it says something else.

So like, Oh, Hey, I'll get this fixed here. Shoot an email off real quick. Surely they'll get it fixed. I still haven't heard back. I bet it's still wrong.

Josh: Yes. So I've, I used to do this all the time where I would, I would try to get things fixed and I have quite frankly, just given up because what it usually is, is you say, Hey, MITRE.

And yes, you have, like, you can look at the website, you can see the version, right? Like, I have definitive proof this version is wrong.

Jonathan: Yeah.

Josh: And I remember the, the one, like, blazing into my mind when I just threw my hands in the air and gave up is I emailed MITRE, and I'm like, hey, look at this, like, here it is, and they said, oh, no, you have to go to the CNA that filed that, but who's that?

Like, and I, you couldn't tell you now they're putting the CNA details in the CVE data. Thank goodness. But at that time it wasn't there. So I'm like, who's the fricking CNA? And they're like, oh, it's so and so. It's like, so I have to go call, like email this person. And I sent them an email and they never replied.

And I'm like, you know what? This is ridiculous. I'm just, I'm out. So now when I see problems, I just go to GitHub. I make sure it's correct there. And like, I'm happy. And that works.

Jonathan: Yeah. So one of the, one of the other things that has been a big problem for the industry is the, um, the CVSS system, the numbering system, like how, how severe a problem is.

And there's this, what, what, how shall we say it? Um, it's almost a conflict of interest because like the vulnerability researcher is looking for the highest score they can get. Because sometimes they get more payout, right? Like sometimes you get a higher payout because you're working on a bug bounty. So if you find a CVSS 10, you're going to make thousands of dollars.

Whereas if you find a CVSS 5. 5, you'll get a couple of hundred dollars. So like they are, they are, um, there's an incentive. They are incentivized to get a higher. CVSS. And then on the other side of it, you've got, in some cases, the vendors that it's like, they do not want to make the news for having a CVSS of a 10.

0. They want that to be a 5. something. And, oh, getting those fixed when they're wrong has been such a problem over the years, because sometimes they are wildly wrong. Yes.

Josh: Yes, you're exactly correct. And you have researchers will, they'll, they'll mis describe the, the vulnerability. And, and the thing is like NIST has a very limited window into what things should be rated.

And additionally, we're like using CVSS completely wrong. It's not meant to be a, a just like raw way to score this stuff and then rank it. There's, you're supposed to add in other information. Like, how am I using it in my environment? Like, you know what I mean? Yeah. What is, what does it look like? Is it, is it being exploited?

There's actually a system that's newer ish called EPSS, Exploit Prediction Security System. EPSS is really cool. What they do is they, uh, it's part of, um, first. org is where they, they kind of, that's where their home is. And they're collecting like all of this data. And it's not all public data, which, It's like all of this threat data is super expensive and the fact that they're getting threat data for free is amazing.

So I'm not going to harp on that, but they're kind of getting all this threat data. And what the EPSS score is meant to be is it's meant to be like, what are the odds this vulnerability will be exploited in the next 30 days? And so the intent being things with higher odds, those are things you should look at.

And now that's an example where you can like, if it has a high CVSS and a high EPSS, now we're, now we're talking like that's something to look at. But if it has a high CVSS and a low EPSS, it's like, eh, we should probably double check what's going on here. Cause this is weird. And so EPSS, I think has a lot of.

I have a lot of hope for it, and they're on, I think, version 2 of the model, version 3 is coming out very soon, I believe, I might be wrong, version 3 or 4, I can't remember, but they have graphs that show how accurate it's been, and when it started out, it kind of sucked, as all things do. Do when they begin, but they're actually doing a really good job now where their, their prediction models are quite good.

And they update it once a day. This is the same idea of like, you can scan your S bomb for vulnerabilities every day, and it could be different. The EPSS score is going to change every day just because like the threat landscape is constantly changing out there in the universe. And so this is one of those things, like you can look at your EPSS score functionally every day, and you can get a better idea of like, what do I need to work on?

What do I need to worry about? Whereas like, again, CVSS. Once you have a CVSS score, it's kind of set in stone, right? Cause that's just how it works. And that doesn't always make sense. Things change. The universe changes around us and we have to try to adapt to that.

Jonathan: I was, I was pretty excited to see the new CVSS 4.

0, the new revision of it. That's supposed to be a little smarter and, you know, supposed to not give you 10. 0 scores for things that are really, really trivial and not actually that serious. And so every time I go to cover a CVE, I go on, Oh, what's the, what's the CVSS 4. 0 score? Surely, by now, it's been out long enough that somebody's using it.

I have not seen, I have not seen anybody use. The CVS has 4. 0 scoring for anything. So like it, it theoretically exists.

Josh: So the vendors hate it, actually. The secret there is, so for, for we'll say low and moderate things, it definitely bump, it kind of, it brings everything to the middle is what I would say.

The high things are coming down, but the low things are coming up. So the vendors hate it because now things that might've been low or moderate are being bumped into moderate or importance. And like you said, The vendors have an incentive to lower the severity of things. So from their perspective, CVSS4 is an anti feature.

Because it's going to actually create more work for them and their customers.

Dan: Makes sense.

Josh: Yeah, yeah.

Dan: Yeah, so, so Josh, how many people are working on, on SIFT and, and GRIPE? What I'm interested in is the community engagement. Are you getting code in from outside of, of the company? Uh, are you getting pull requests, are you taking pull requests, that sort of thing?

Yeah,

Josh: Yeah, oh for sure. We definitely are like we love pull requests. We love the community. We've got a discourse We've got it's all a github. We've got community meetup every other week I think it is the the open source team with with popy They do like a live stream once a week where they kind of do like some of their bug You know, wrangling to figure out what to, what to do.

I mean, we've got a team of, so I think we have a dedicated team of, I don't know, is it five people, six people? I can't remember exactly what it is, but then obviously you've got more anchor involved as well. And then, yeah, I think we have like hundreds of contributors. I don't remember the number off the top of my head.

I'm, I'm ill prepared for that question, but it's, It's, it, it's pretty impressive, honestly. And there's, we're, we're seeing more and more community involvement, which is a double edged sword though, right? Because you get more bugs, you get more pull requests and it's not like, I always love this. Oh, open source.

We'll give, do our free work for us, but like, it's free, like a puppy. Cause you have to, you know, you have to read the bugs. You have to triage of the bugs. You have to. look at the pull requests and be like, Oh, we'd like to do it this way instead, whatever. And, you know, it's, it's a lot of work, but it's really cool because yeah, we're seeing, we're seeing some pretty cool contributions.

In fact, where there's, um, uh, there's a thing called Bitnami, which is like a container runtime universe. Like there's, they just submitted a, a giant patch to SIFT to properly catalog and index their containers, which is amazing, you know, like that's exactly what we want. We've seen a couple other organizations, you know, Give us, um, we call them catalogers in SIFT, which is where we have the ability to be like, oh, this is Like, um, I think it was just a NET that got added where it's like, this is, you know, a NuGet repository.

So figure out what to do, NuGet cataloger. And, and yeah, like we're, we're pretty pleased with, with how it's going. And, and like, that's the dream, right? Is you start an open source project and ideally you want a community to build up around it, which makes us very excited and very happy.

Dan: Hmm. And I, I know I don't have, I shouldn't have, I shouldn't really have to ask this question on an open source podcast, but I want to anyway.

'cause I think it's always great to get people's opinions. What's the benefits to, uh, Ancor in releasing these things as open source? Because we, we preach it all the time here, but we've, we'd like to. So let you preach it, I suppose. All

Josh: right, all right. So I've got, I spent a decade at Red Hat

Jonathan: and

Josh: I, I spent time at a little Linux startup called Progeny Linux Systems before that, which the, uh, Ian Murdoch, the founder of Debian created a startup and we'll say it wasn't real successful, but, uh, But anyway, so I spent a lot of time at Red Hat.

So like, my career is steeped in open source. And I remember my first job out of college was at a company that wrote life insurance software in COBOL. And this was like right after the dot com crash, so it was dire. I'm like, I will literally take any job. And And I did, but I remember they, like, I'll never forget it.

You know, there were people there that were like, this open source stuff is stupid and it's crap and it's never going to catch on and you're wasting your life on it. And of course being young and, and filled with, you know, optimism, I was like, Oh no, no, it's, it's the future. It's going to win. And like, thank goodness it did.

Cause I'd be, you know, probably broken homeless otherwise, but, but, so the thing is like, Angkor has a bunch of red hat DNA in its blood. Like there's a bunch of red hat folks that are at the company. And so it's. Thank goodness, not something we had to like educate or understand, but fundamentally. So the idea is we've got the, like the, the bottom of the pyramid is open source.

We have sift and gripe and we have a bunch of like this tool called grant. We've got a whole bunch of other like open source libraries that kind of drive all of this stuff. And then, The intent being by making a tool like sift and a tool like gripe open source, you, you get community contributions are awesome and we'd love that, but you also get a certain amount of just kind of engagement in the wider universe, right?

Where you have people running these tools, you have open source projects running these tools. You've got, um, we just had a Allen actually webinar with Google where Google is scanning. Like all of their stuff with SIFT, they're literally scanning, you know, millions and millions of things every day with SIFT.

Like, that's really cool. They have good feedback, obviously, as well. And that's like the power of open source. Right. And then what we did is we took kind of those. Core functions and we built product on top of that because it's easy to say, Oh, you're going to run SIFT across your entire infrastructure.

And if you're Google, you can do that because you have an army of engineers. But if you're a small company working on something or even a medium or even large company working on something, you have to ask, like, is it worth just buying a solution that will do all of the plumbing and all of the monitoring and all of the policy and all that stuff?

Or. Should I hire an army of people that are going to do this as well that will probably cost somewhere between three and ten times as much as just buying a product. And so that's kind of like that very red hat model of there's like that core base functionality and then there's things, there's value added to the top and we've got, you know, we've got support, you've got services, there's, we can help you figure this stuff out because it's weird and hard and, and if there's a problem with your data, like, Open and support issue.

And it's going to come to my team and we're going to tell you what's going on. And often we just fix it because it's like, oh yeah, that data is bad. We'll go fix it. And so that's kind of the beauty of that. And yeah, that's what. It's been a treat being there because I don't have to fight for whether they should or shouldn't be open source or whether this is or isn't a good idea, like everyone totally gets it.

It's awesome.

Dan: That is very cool. Um, so I always like to, to, to ask people how they got involved in, in all of this. How did you get into what you do now? How did you get into open source initially? And how did you get into computing even? How far can we go back with this? You can go pretty far back,

Josh: man. I mean, I've been, I mean, I'm, I'm old.

I've been doing this for a long time. You know, I, I grew up without a computer in the house. Punch cards, was it? No, not quite, not quite that far. Not the punch cards. No, no, I don't. I didn't have access to punch cards. I would have gladly used them if I could have. But no, I mean, the first computer that came into my house was like a, it was a 386 running MS DOS.

Right. So, I mean, we're talking like early nineties, somewhere in there. And I just, it was, I loved computers. I've always loved computers and I'll never forget like my. You'd go to school and you know us old geezers We had like a couple Apple twos at the school and we do things with them and stuff And I remember my my parents came home from a teacher conference one time and they were started yelling at me And I'm like what and they were like you have to stop helping them with the computers because you don't know what you're doing I'm like, what do you mean?

I know what I'm doing. It's like I knew more about the Apple two than the teachers did. Cause like I would read anything I could get my hands on. It talked about computers. Right. And I mean, I remember people laughing when I tell this story, but like, I used to write down basic programs on a notebook paper and I would like, I would run the program in my head.

I didn't have a computer. So I'd be like, what was this program going to do if we run it? And you know, I'd write it out and figure out what the program would do. Which nothing long, of course. course, but like, I, I just always loved it. And so, you know, computing will say for a while, and then this Linux thing you start hearing about, and it's like, wait, I can get free stuff.

Like, well, I mean, we'll say a lot of software was free back then, if you knew the right people, but this is like, you know, free compilers, I can get, you know, a nice. Uh, a Unix terminal that doesn't cost any money and it works and it's awesome. And then I think the real thing that tipped me over the edge was I go to college and they've got sun workstations everywhere, right?

Cause I'm an engineer. It's just sun gear every like the sun, a sun workstation, like 45, 000, right? They were crazy expensive. It was 45, 000 in nineties money. So we're talking a lot of money today and I'll never forget. It was like, there was this. This stuff I wanted to do where there were programs I wanted to run, you had the ability to, you know, do like X windows between systems where you could like run something on the server and you get the X windows.

Window on your system. And I was like, Linux, let's me do all this for nothing. Like this is a no brainer. And so it was really, I mean, I mean, honestly, it was just, I was too poor to buy a sun workstation. That's kind of what really got me into it. If I, if I had a ton of money, I'd be running sun gear at the time, but it was Linux, but then the, you know, the thing is it just, it, it, it gets you, right.

You get that bug of like, oh, I can talk to the people who do this. I can send patches. I can work with them. I can build my own kernel and. You know, completely trash my machine and learn more than I can imagine in the process. And, and it just, I don't know, I caught that bug and it's just, I love the community.

I love how it works. I love everyone helping each other. It's just everything about this universe has been a treat. And like, I, I've built my life and my career around it. And I've, it's been lovely. I, I have absolutely no complaints or regrets and I love open source. It's, it's absolutely amazing. And. Every day I just think like what's the next crazy thing that's going to happen and I'm never disappointed.

We'll say that but yeah, it's fabulous.

Dan: Awesome. So, uh, I just came up when you were saying that you got bitten by the bug. I was thinking bitten by the penguin by a penguin. The penguins don't have teeth though, do they? So I don't know how quite that would work, but

Josh: they have like sharp. Barbs, I saw a picture one time inside of a penguin's mouth.

It's like a Lovecraftian horror.

Jonathan: Oh, that's great. All right. We are, boy, we're starting to run out of time here. Um, is there anything that we didn't ask you about that you want to want to cover and make sure folks know about? Many

Josh: things.

Jonathan: Yeah, I'm sure.

Josh: No, it's, I mean, I think the one thing I would say is, so I, I adore, this universe, like the open source security, all that stuff.

And if like anything I've talked about catches your interest and it's a topic you'd want to discuss or, or learn more about or anything, like reach out. I love talking about this stuff. I will talk your ear off if I can. I mean, I have a podcast just cause I love it. I've actually have two podcasts. We talked about the open source security podcast.

My other one, I call Hacker History, where I just take like old people and I say, tell me your hacker story and that's it. And then that's it. We learn about crazy things from the days of yore. And it's, that was super fun. It's, it's, it's, it's tough to find guests, but I'm always looking for guests. So if anyone wants to tell their story, like hit me up there too.

And yeah, it's just, that's kind of the biggest thing is just, if you want to have a chat ever reach out, I would absolutely love to, cause there's so much going on, there's so much interesting stuff and it's just so much fun to talk about, I love it.

Jonathan: I was going to have you make sure and plug your podcast, but I think you just snuck that in there, didn't you?

I did.

Josh: I did. I'm pretty shameless like that, so.

Jonathan: A skilled, a skilled commentator.

Dan: That's

Jonathan: right.

Dan: In security speak, it was the payload in the, uh. That's right. Yes, exactly. Quickly run off the end of my security knowledge there. I went payload. Yeah. Okay. Show code has been executed. Yeah, that's right.

Jonathan: All right.

So, uh, what's your, what's your favorite then, uh, scripting language and, uh, editor?

Josh: Okay. So I, I was alive for the VI versus Emacs wars and I still run VI all the time. It is. Probably my favorite editor. I think I use VS code, we'll say for anything remotely difficult, but VI still is like, that's where it's at.

It's amazing.

Jonathan: It's amazing. How many people answer VS code to that question either directly or indirectly. It's so good,

Josh: but it works. It does what I need.

Dan: Yeah. It's a good tool.

Josh: And then scripting language. So I, I know many of your guests, because I, I listened to the show. There's always a, is Python a scripting language?

And I, I, I've been thinking about this like since we, you booked me, I'm like, should I say Python? I'm like, I gotta say Python. Cause the thing is, every problem I have at this point. Ends up with a Python script to do it, you know, like I'd love to say shell or like bash or whatever, but it's Python. Like it's funny.

It's it's become like the, you know, Swiss army sledgehammer of programming. I think it little things are big things that it all works. It's it's kind of a replacing Pearl

Dan: for that position.

Josh: Oh, yeah,

Dan: for sure. Somewhere. Randall just exploded. I thought I had a faint explosion in the background. Yeah. Yeah,

Jonathan: that was

Dan: Randall.

Jonathan: We have discovered recently that any language could be a scripting language if you really want it too badly enough. We had somebody doing the Java for scripting, for system scripting. There

Josh: you

Jonathan: go. I remember C

Josh: Shell, right?

Jonathan: Yeah. Yeah. Yeah. It exists. It does. I, man, I am, I am in a group right now that, uh, one of the guys is a huge C sharp fan and I'm sure he, he does.

No, he actually, he does. I'm sure he uses that C

Josh: sharp C shell. There was a shell from long ago that had a C syntax, but it was a shell. I know. Right? Like I remember that too. Yeah. We'll say corn shell and flood us all with horrible memories of each like some pain with your pain.

Jonathan: Oh, that's right. All right, this has been this has been great. I don't want to let you go. It's been so much fun We'll have to head back. I actually wrote a note in our back channel Like next time we need a uh, we need to we need to do a roundtable because a guest flakes on us I'll have to reach out to to this guy and I think our guest from last week.

We also made that note about so Maybe we'll do that, maybe we'll have a running list of guests and co hosts and when we have to scramble we'll just shoot a mass email out. So this is what we're doing, this is where we're doing it. Awesome. Uh, it'll be fun. Alright, thank you man, I appreciate it so much, it's been glad to, it's been really good to talk to you, uh, Josh Bressers everybody, thanks for being here.

Thank you much. Alright. Uh, what do you think, uh,

Dan: what do you think Dan? I thought it was great. What a great discussion. As you said, uh, we, we could, we, we seem to say this on every show, but we could have kept going for a long time now talking about all kinds. I really thought we're going to get into penguins at the end there and whether, you know, with the whole penguins and teeth and all that, I was, I could have gone in that direction.

Um, I was, yeah, I was interested. Something I'll, I'll ask, I'll have to ask Josh when he's talking, I'll check his podcast out, the hacker history one, because I'm really interested in the whole phone freak thing and from the seventies and the sixties and all of that original. Audio hacking of, of phone switches and so on.

So, so I'll ask him if he's got any, had any of those kind of people on Yeah. Uh, yeah. Fascinating guest. Fascinating guest.

Jonathan: Yeah. That, that is, that is really interesting history. And if he hasn't, we'll have to make sure and request some. Um, there are, there are still some people out there that are active. Um, right, right.

That, uh, one guy that I've, I follow on Twitter and he, he. Evan Dorbel is what he goes by and just some great stuff. The cool thing about Evan is that he, he was also like an audio, he was a phone nerd, but also an audio nerd. And so he has recordings of the phone system from way back in the day. And so like, he's got these narrated clips of.

He, this is, this is what you hear when you do this and this is what the equipment on the other end is actually doing. And it's, it's really cool. So we'll have clear all clips from like all the different exchanges and like, here's how this one sounds different from this one. That's why. So Josh, I know he's still listening to us.

Josh, if you haven't had him on yet, see about Evan Doorbell and some of those guys. That'd be a lot of fun. Uh, alright, uh, do you wanna plug anything Dan?

Dan: Um, I want to, yeah, I mean, uh, go to my website, danlynch. org, if you want to find out what I'm up to, and, um, I'm going to do a community style plug, because, um, I run our, I'm one of the people who run our Linux user group in this area, and I think everyone should, you know, uh, the whole Linux user group movement kind of has lost, you know, Momentum in the U.

K. And I'd like to say, you know, if you're interested in Linux, have a look, look dot org dot U. K. Find out where your local log is. Or if you're near the northwest, come to Liverpool. We run Liverpool log. We have stuff on every month. So check that out.

Jonathan: The Linux user group scene has just moved online. It's still, it's still alive and well, it's just It happens in Slack and Discord and a little bit in the IRC and Telegram and

Dan: Signal.

I have been to Linux user group meetings with people all like 10 people sat in a room all hunched over computers like this talking to each other online and I think, why are you here? You could have done this from anywhere, you know, you're not even talking to each other, you know, anyway, that's just my own personal gripe.

Jonathan: Yeah, we, we nerds are, we're a weird breed sometimes. All right. Well, thank you, man, for being here. I have a couple of things that I want to plug, and, uh, of course, the first off is Hackaday. We've got my security column. If, if this is not enough security for you, you can get even more of it, week to week coverage there.

It goes live every Friday, uh, and we also sure appreciate Hackaday being the home for Floss Weekly. Um, there's also the Untitled Linux Show over at Twit. If Linux is more your thing, you can find our weekly musings about that over there. Uh, we do not yet have a guest for this upcoming week. on the 5th, but on November the 12th, we're talking with Frank Delaporte about Pi4j, which is another Java thing, but it's, it's doing, uh, it's doing Java on the Raspberry Pi, and specifically, it's about being able to access all of those peripherals, you know, GPIO and SPI and I2C and UART with, with Java bindings.

And so that, that sort of, that sort of tickles my, my interest, interested bone I've been working with. Some of that stuff myself here recently. Um, and then one other thing that I will let folks know about is, uh, later today in my time, I don't know when all of this will go live, but, uh, I'm going to be interviewing with Brody over.

He's got a, the YouTube channel. Um, is it just Brody Brody talks Linux? I don't, I forget what his channel is actually called. You can search for Brody. You'll find him. One of the other things he does though, is he does tech over T where he interviews folks. And I. commented on something on, on Twitter slash X and he sent me a PM.

He's like, Oh, by the way, I've been meaning to invite you to come to this. So that is happening later today. So watch for that. If that's something that you will find interesting. Uh, we sure appreciate everybody that's here. Those that caught us live, those in the discord, making notes and commenting back and forth and all of y'all that get us on the download.

Sure. Appreciate it. We will see you next week on Floss Weekly.

This week Jonathan and Dan chat with Josh Bressers, VP of Security at Anchore, and host of the Open Source Security and Hacker History podcasts. We talk security, SBOMs, and how Josh almost became a Sun fan instead of a Linux geek.

• https://opensourcesecurity.io/
• https://hackerhistory.com/
• https://infosec.exchange/@joshbressers
• https://anchore.com/

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

This week Jonathan and David chat with James Smith about Manyfold, the self-hosted 3d print digital asset manager that's on the Fediverse!

• https://floppy.org.uk/
• https://manyfold.app
• https://github.com/manyfold3d/manyfold

You can join the conversation in the Hackaday Discord, watch live or get the video version of the show on Youtube, as well as getting the full story and show links from Hackaday. Oh, and follow the official Mastadon account!

Theme music: "Newer Wave" Kevin MacLeod (incompetech.com)

Licensed under Creative Commons: By Attribution 4.0 License

http://creativecommons.org/licenses/by/4.0/

Jonathan: Hey folks, this week David joins me and we talk with James Smith about Manifold, which is the host your own repository for 3D printing. It's just recently been wired up to talk with the rest of the Fediverse and it's got a lot of really neat features going on you don't want to miss it, so stay tuned.

This is Floss Weekly episode 806, recorded October 22nd. Manifold, the dopamine of open source.

It's time for Floss Weekly. That's the show about free Libre and open source hardware, hardware, software, software today, software about hardware today. I'm your host, Jonathan Bennett, the oftentimes tongue tied, and I've got a co host with me, of course. I've got Mr. David Ruggles. Welcome, sir. And we're, we're talking today about, well, about Activity Pub once again, but not Mastodon, not directly Mastodon this week.

Talking about Activity Pub and 3D printing. David, I, I know, let's see, you're on Mastodon already? Yes. But not, you don't have a 3D printer yet?

David: No, I do not.

Jonathan: Okay. But

David: it's something I'm interested in, it's just one of those things that it's down on the priority list and there's never enough time.

Jonathan: We are going to do our best, I think, to talk you into getting on the 3D print bandwagon today.

For this, I'm sorry, and you're welcome. Um, let's, let's go ahead and bring our guest on as well. It's Mr. James Smith, who is part of the Minifold project, which is all about curating and storing 3D printer models, and then sharing some of that metadata on, on Mastodon, on ActivityPub. It's it maps somewhere in there.

It's something like that. James, welcome to the show.

James: Hello. Thank you very much for having me.

Jonathan: It's great to have you here. Yeah. I was, I was saying before the show got started that I was, I was excited about this because I like Macedon. I like activity pub just like on, on its own. The, the idea of it, I think is really great.

And then I am also something of a 3d printing nerd. And you put the two together and it's, it's just, it's just great. I recently picked up, and we're going to try not to make this all about 3D printing, we're going to talk about the technology too, but I recently picked up a a Bamboo Labs Mini, and it lets me do some really fun stuff like this.

You know, I got it with the multi material, and so, you know, you can do a single print that's got a couple of colors in it, and so I've got a, it's actually a Meshtastic radio I printed out a case for, and I've just been going nuts with stuff like that. Because it's such a huge upgrade over the old printer that I had.

And I was, I was telling, I was telling James before the show I, one of the other things I do is I print out some miniatures for tabletop gaming. And I purchased a I purchased a miniature from HeroForge back several years ago. And I, I printed it on my, as has been called the, my first printer. The, the very cheap model that I started with, and then I forgot about it.

And then I got the new bamboo and it's, Oh, I need to go back and try to print that model out again. Oh, where is it? That was like, Two desktops ago. And so I have on the other side of my desk, right over there on the floor, my old, huge desktop is out and running to be able to pull model files and other files off of it now.

And I was telling James before the show, it's like. Why hasn't somebody come up with a repository you host your own yourself that you can put all your models into And that's what many fold is isn't it?

James: That's it. That's it. I mean you've you've described the journey that That made it happen. I think really which is that I've got too many models lying around.

I don't know where any of them are or what they are so I need some way of organizing them And that that became A little side project and that grew into Manifold.

Jonathan: No, so it is, it is Manifold, not Minifold.

James: It's, it's, it's an interesting one. It's kind of between the two because it's, the name comes from like, Manifold being, you know a property of a 3D If it's got no holes and things like that, it's what's known as manifold.

And it's many because there's lots of instances and things like that. So I, I kind of sort of. Drive straight down the middle between the two, but I think, as I said earlier, it's I, I use it written down a lot more than I say it. So I, I don't, whatever. It's fine. There, there,

Jonathan: there is a long tradition of open source projects having names that are very easy to write out and sort of difficult to pronounce with various takes on how to pronounce them.

James: I think of it as a, it's a wide, it's a wide cluster of how to pronounce it. It's fine.

Jonathan: And, and, and manifold has a tie in with activity pub. Is it activity pub or Macedon in particular?

James: No, it's activity pub. So this, this is really, really new. So manifold's been building for three years or so. I've been building it.

Like I say, initially as a side project and and then managed to get some some open source funding for it. But one of the things that I got a funding for was to add in the activity pub integration so that not only can you have your own sort of collection and and organize it and things like that, but also you can connect it up to others.

So yeah, it's pretty new. We properly, properly joined the Fediverse, I suppose, couple of weeks ago. Which is, which is really cool.

Jonathan: Oh, wow. It's that is very new then. So this was not, this was not an original feature. Okay. Okay. So

James: no, no, no, it was always something I thought like, Oh, it'd be cool if I could make it do that, but yeah, it's, it's taken a while to get here.

But but yeah, it does it does do it now. Yeah. So what is the, what does the

Jonathan: integration look like? It's like, what, what what gets posted out to the Fediverse? Is it like, I printed this, I printed this and here's a picture of it. You know, what, what does that look like?

James: So at the minute. So we're taking a very sort of agile approach to to getting features out at the moment.

We've got the sort of the first things that will that will happen in terms of ActivityPub is that if a creator adds a new model, That will get posted as a as a note across on things like Mastodon and so on. So you can subscribe to creators on public manifold instances. You can subscribe to them from Mastodon or other, any other Fediverse software, and you'll get the notes when they release things.

There's a lot more to be added in there. One of the things that the the instances will be able to do is actually directly sort of follow both creators, but individual models and updates collections, things like that to to follow those on other services more directly. So it's not just like, oh yes in my Macedon feed I saw a nice little post.

It's actually like a link. You know, it's like a single archive almost, so that's what we're building towards.

Jonathan: Yeah. Is all of this opt in or are we sort of putting models out there? It's all okay.

James: Oh, no, no. Very opt in. Okay. The everything above Running it as a single user on a private network is, is opt in and controlled by, by various feature flags and things.

It was originally built as a private archive. So I built it as a, a thing to literally just post on a Raspberry Pi in the corner of the room there and be able to catalog a directory full of files in in my browser. But then we've added in multi user capability Lots of other, lots of lots of other bits and pieces.

The, the federation and, and all these things are all optional features. So no, it's, and it will only happen for public content as well. So we've got very fine grained Google doc style permissions on things. So you can say actually, yes, I want to make this item, this creator, this item public. So if you were to look at the, uh, my own.

Manifold instance, which is publicly available on the internet. You can see my stuff that I've published as if it was thingiverse or one of the other model hosting sites, but you can't see my private archive, which is also on there.

Jonathan: Yeah, I was that that was a question that I had. I was going to ask because not every license for STLs.

Lend itself to sharing publicly like that.

James: No, absolutely. I've, like I say, I'm using it to manage my my models that I've published as Creative Commons and things like that. But also it's, it's stuff that I've bought from creators on Patreon and things like that which are, yeah, which are not for sharing, so.

Yeah, handles both of those situations.

Jonathan: Yeah, now is it Is this simply for STLs or are there other file types? Can we, can we load G code in there? Can you do object files?

James: Yeah, it will. Yeah. So a whole load of of model files are supported STL OBJ, lots of sort of older formats 3MF which is finally a decent format for these things.

And a lot of others, but also images last couple of weeks ago, we added a PDF and text files and video, even because some models come with assembly instructions and you want to have those there alongside it. So but yeah, this yeah, all sorts of things. G code, sCAD, FreeCAD. All sorts of stuff.

Jonathan: Now, I've, I've got to ask, this is something that you see some of the other sites will do does, does Minifold render? Like, if you give it an STL, will it give you a preview of what it looks like?

James: Yeah, so the primary, the primary, the thing I did first, where it started was, I want to point, and this is why I did it in the browser, really, it sort of drove the whole thing, I want to point, A server at that folder.

And I want to see the models that are in it and render them. So it uses three JS. If you, if you go to look at a model it, you get a list of all of the. individual files that are there, it renders them all, you can play with them, zoom around, find exactly which one you want, then grab that file to go and take off to printing, which for, for things with lots and lots of parts in is, is very useful.

Jonathan: In just a second here I'm going to give it to David and I'm going to let him ask the questions that I'm not thinking to ask because I am already a 3D printer nerd. First though, I've gotta, I've gotta This is, this is crazy. And you're going to really, really impress me if you answer yes. Is it also a slicer?

Does it slice as well as dice?

James: No. No, sadly not. No, I mean, the slicer is a massive bit of software. And there's some really good ones out there. So that would be reinventing the wheel. People have asked. There is work that people have done in browser based slicers. I look at that as the sort of thing where it'd be really nice to integrate with some of those things.

So it'd be really nice to be able to have things automatically open up in your slicer of choice. Or to be able, if you do have G code in it, to be able to automatically send that over to OctoPrint or something like that. People have asked about things like the The thingiverse customizer where you could actually do some of the editing of models The scope I think for sort of maybe plugins and things that do that.

Yeah, I I tend to be not a fan of things that grow to encompass every possible feature. It's But if I can make plugins work for stuff then Then that would be that would be interesting

Jonathan: I could, I could even see something like editing of OpenSCAD scripts.

James: Yeah, that's exactly, yeah. Exactly what people were suggesting with the customizer approach.

Jonathan: Yeah, no, that would, that would really be cool. You could also imagine a flow where you've got your, your models saved, and then you open one of them. It moves it to your computer, opens it in, Your slicer, your preferred slicer, Cura or whatever, you render it, you slice it, and then it uploads it back to Minifold.

And then from there, you've got a button to press that'll kick it over to your printer. Like, you can imagine this, this workflow that would be really, really cool. But,

James: it's, it's, I, I've recently switched over to OrcaSlicer, and one of the immediate things that, I loved about it was that when you hit the print button and it sends the file to your printer, it's got a, it's got a tab in the UI for the web interface.

If you're running OctoPrint or if I am, I'm running Mainsail,

The Clipper UI online. So right there in the slicer, it's taken me to The web view of my printer, which is amazing. It's, it's integrated that workflow. So now actually what I want is like, what about the first tab there that actually is pulling in a manifold instance or something like that?

So. Yeah, there's lots of possibilities, I think.

Jonathan: Yeah, exciting, exciting ideas for the future.

James: Too many ideas. That is usually the problem. Yes, that is

Jonathan: usually the problem with open source. David, you want to jump in and ask the, ask some questions that I'm just glossing over?

David: Sure, I can ask the dumb questions, or the any man questions.

So, I've got CAD experience, it's mostly 2D stuff. And, I don't have a 3D printer. It's something that I've looked at many times, and it's just down on my priority list. But, so, Some of the terminology and stuff, I'm familiar with some of it, I'm not. First question I have, and maybe this is for both of you, but you said slicer.

I'm assuming that is something that takes a 3D object and creates like, parts of it so that you can print it out where you might not be able to print the entire 3D object because of its complicatedness, so, so you can basically print parts of it and assemble it after the fact. Is that what a slicer is?

James: Nearly. Nearly. The slicer is actually, so all, all 3D printers, I think all of them at the moment anyway work in layers. So if you've got a a filament printer, you've got a, an extruder and it's, it's, it's drawing a path around like, like there's a lot of 2d CAD stuff like laser cutter or anything like that.

In fact, the the language I think is the same as is used for for things like laser cutters. So it's generating this tool path. And if you've if you've got a resin printer, which is often doing like a it works. whereby you have a screen which is curing some UV resin, and you do that layer by layer and it slowly sort of builds up the model.

And the slicer is what turns that 3D model into a series of Basically,

David: okay. So does your slicer maybe resolution or thickness of each slice? Is that dependent on your printer functionality?

James: Yeah. Yeah, that's right. And different printers will go to different different resolutions. And to be honest, the amount of time that you want to spend as well.

It's I can go down to 0. 04 millimeter. That's a very slow print though. It's a very slow print, yeah. I don't do circle

David: back to what you were talking about before about integrating that slicer functionality are slicers something that is printer dependent or is pretty much any slicer able to work with any printer?

James: The good question. The two, I think, I think on a consumer level, there's two big types of printer. You've got the filament type known as FDM fused deposition model. Sorry. My daughter's knocking on the window. That's

Jonathan: fine. Bye.

James: It's better than a barging through the door, which is what I thought she was going to do. Yeah, you've got the the FDM printers and then you've got the resin and they work in two different ways. One is extruding this big wiggly line of plastic and one is building it up almost like a, you know, like a flip book.

Right. So I think in, in, Until fairly recently, you'd probably have a different bit of software for both. There are now some that are combining those two into one bit of software. But there's lots of different Oh, and all the printers will, will have like, Oh yeah, here's the slicer that we recommend and, and things like that.

There's lots of them out there. There are about Five different forks and minor variants of one, one in particular, which is just confusing to me, but it's but yeah, choosing one that, that works well that you like the UI of things like that. They, but they're all The resin printers are a bit different, but all the filament printers tend to work on on G code, which is this language for, like I said, originally for CNC stuff.

So it's that's a, that's a standard. So they're all creating that kind of kind of thing. So they're pretty cross compatible.

David: Okay, so it does seem like there may be the possibility of somebody developing a plugin at some point that would have that slicer functionality. It's possible, yeah.

James: I think Octoprint had one.

I don't know, I've never actually tried it, but Octoprint did have one, I think at one point. Whether it's still there, I don't know. So, it's been done.

David: So we could go a lot of different directions with this. I read through your bio and the website and you've got just an interesting history. So next time we have a round table and we're missing a guest, I recommend that we see if he's available and we just go into some of these other things.

Come

James: and talk about anything, yeah. Yeah, I've done, I've done, you know what we said about too many ideas, right? That's, yeah. That's, that's me. So yeah, I've done too many different things over the years.

David: You combine the creativeness with the open source and the possibilities are endless. And then you get somebody with the mind for it and you want to try every one of those endless possibilities.

James: Yeah, the difficulty is not starting projects, it's getting them, yeah, remotely finished. I mean, that's been really nice about this one is that it's I have, I'm not going to say I've built a community around it because I don't really think I did anything, but there's a community has appeared. There was one day back in May, I think that someone popped up in the support channel that we have on matrix and somebody else answered the question.

And I was like, Oh my God, that's when you know,

Jonathan: you've made it.

James: Oh, how did that happen? This, this has never happened on any project I've ever done before. So Having, having community to, to keep you engaged and actually get a bit of the dopamine hit from delivering things is really nice. Most of what I do has failed over the years.

So

David: it's, it's not failure. It's it's learning what doesn't necessarily work in the community. Absolutely. Oh, so how large is your community now? Do you have any feel for that?

James: Do you know, I, that's, that's a really good question. I'm in a way, I'm really proud to say, I don't know because knowing too much about them is, is kind of, you know, a bit of an antithesis of the like self hosting kind of control your own information world.

So I'm taking that as a strength. No, but I mean, there's, there's lots of a few, a few sort of proxy measures, I suppose. I mean, that's about. I don't know, 60 people in the support chat. Normally we've got a few hundred followers now on in, in the Fediverse. One of the things I don't know is how many there are, how many instances there are.

No idea. I I've made a I built an anonymous tracking thing so that people could report. How that their thing is running and how many of them there are first of all, it's really difficult to get people to click a button that is completely optional and, and not shoved in their face. And secondly, I made it so privacy enhancing or privacy preserving that actually, whenever I change the code, it forgets everything.

And that just starts from zero again. So it's very much a proxy. The, the. One interesting question, which I don't know if anybody listening knows, if there's any way I can work out from the Docker download stats, the Docker pull numbers, what that actually means. Is it, you know, if I've got like 50 polls, is that going to be one machine pulling it every, you know, 10 minutes?

Cause somebody has got an auto update on or is it 50? I don't know, but I do know that we get Docker pulls within moments of each release. And, you know, we'll have a couple of thousand over a week. So, but I don't know what that means, but it's, it's going up and that's a good thing. So

Jonathan: yeah. I'm pretty sure it's accurate.

Docker pulls are actual downloads, not just checks for version.

James: Yeah,

Jonathan: that seems right. David.

David: Yeah, I've done some, I kind of don't want to assume that and it only pulls it in when it's built. So, I mean, you might have some script that's checking automatically for updates and then pulling it down, but it's still going to be a new download.

James: Yeah, I, it's kind of, I don't, I'm not sure whether I trust GitHub's counting because it's on the GitHub container registry. Do I trust that their number is actually pulls or that it is? People have just asked what the latest version is. I don't know. So if anyone knows that, let me know. I'd love to know.

But no, we've got enough community going on that that it's keeping me going and it's, it's building momentum and we get contributions, code contributions, lots of feature requests. People report bugs really quickly, which is great. That's actually a good

David: sign of it really is right there.

James: Yeah.

Absolutely. Yeah. Yeah, there was one I, I accidentally pushed in the the previous version. And I think within about two minutes, somebody had said, Oh, on this, you know, on the settings page over here, when you go and change something, it crashes. Well, I haven't changed anything in the settings page. So you're just using this.

You're not trying stuff out. Wow. So yeah, it's great. And that, that meant that I could fix it straight away and, and get a release out in the half an hour. Yeah. That's awesome. I'm a,

David: I'm a web developer, so I definitely interested in that side of things. And I'm sure I'll have more questions as we go along.

But you mentioned that You've done audits against the web sustainability guidelines. Yeah. So what does that look like for you? What, what,

Jonathan: what is the web sustainability guidelines first? Yeah. I only dabble at doing web development and I'm not sure what that is.

James: I, I think most people are in the same boat as you.

So You've heard of things like the Web Content Accessibility Guidelines,

WCAG,

and things like that, which is all these good things that we should be doing to make sure sites are accessible and so on. The idea of the Web Sustainability Guidelines, it's now a there is a W3C community group that's put these together.

I think some, some people I used to work with are involved in it. I, I used to be more involved in the sort of sustainability climate side of things. So it's always been like an interest. I used to run a meetup in London around, uh, using the web for sustainability stuff. So but what they've come up with is actually a, a kind of a draft spec of, okay, here's, here's how you.

You know, these are the, these are the guidelines for, for building sustainable science. It's yeah, just, yeah, if you look it up, you'll you'll find the thing, but it's. It's still in draft. It's very early days. It's not that a lot of people are using it, but it's something that I was interested in.

I've been interested in for a long time, like say, so I was like, well, when I had the opportunity to spend some time on it as part of this open source funding, I, I said, right, we need to do an accessibility audit. We need to do a security audit and I want to do a sustainability audit as well. And so I did, there isn't a.

process as such. So I actually sort of rolled my own DIY assessment type process, very much just like a red, amber, green type thing, going through each item on the list and going, yeah, that's, that's okay. And this one, Oh no, I hadn't thought about that, that kind of thing. And then hopefully do it again, maybe in the new year, see how things have moved on, but it's been really useful.

Just in terms of even, even that sort of very rough approach was really useful in into pointing at pointing out where, where the inefficiencies were, where the, you know, what was going to be more impactful about this application. So yeah, it's, it's really good. And obviously one huge thing with this is that It's 3d models and they can be really big and that's by far the probably the biggest impact of the thing.

So you know, that's that's on my sustainability, Roadmap now but it's nice to treat that as a as a sort of primary thing I think alongside accessibility and and security and and things like that to to have it in mind as As a goal so actually in the release nodes now we have like a special section which is Sustainability and performance because those two go hand in hand.

So when those get Improved it it points it out to people. So

Jonathan: So this basically boils down to not wasting cpu cycles I mean put it very very broad strokes at its

James: most at its broadest stroke level. Yeah, network transmission all that kind of thing But it goes into a lot of other stuff as well.

There's a huge overlap with accessibility. Build things that people are able to use and they'll be able to use them more efficiently. There's a big section on product design and things like that, you know, build what people actually want. And then you're not building things that are pointlessly using using power where they don't need to.

No no major current trends mentioned.

That

David: was a very suspicious cough.

James: Yeah. Yeah. I don't know. You'd need something, I don't know, something very clever to analyze what that was.

David: So I

James: used to work in AI and this is, this whole thing is just hilarious to me.

Jonathan: Don't get me started.

David: We'll put that, we'll include that in the red table. That's not for today.

Yeah. So, this, not to go too far down into sustainability, but I assume you can also see it even as a UI UX issue where Absolutely. Think about a web page. If they're having to click through five screens to do one thing, that's five different hits. That's additional. Okay.

James: Yep. And it goes, it's, it goes to that sort of level of pointing out all those all those guidelines of, you know, Around those things and those things that I wouldn't have, you know, I'm a Very much a developer technologist.

So, you know my thinking was like, oh, yeah, where are my data? Where is my server running? Is it, you know powered by renewable energy, etc And that's like one one point is that you know, there's so much else And it goes into all these other things. That really take it and you know a whole team Developing, developing a product can, everyone can do something.

It's really good. Really interesting.

Jonathan: I love the fact that when they made these guidelines, they included a lot of just good hygiene sort of stuff like that in there. So it's useful, you know, it's useful for, for everybody, developers. That, you know, may not have a whole lot of concern about the actual sustainability bit, but it's still useful guidelines.

I think that's really neat.

James: Yeah, absolutely. It's a bit like the universal design thing for accessibility, right? You know, good accessible websites help everybody. Not just, you know, people who need it all the time. It's just good design. So,

Jonathan: yeah. All right, I want to ask about funding because you've, you've talked a little bit about getting some money for things.

And, you know, for some open source projects, that's like the holy grail to be able to get paid for stuff. And it's, it's not quite the panacea that people might think it is, but it is kind of nice to be able to get paid for doing open source work. I believe I also saw that you are a fellow open collective member, which that is cool.

That's where, that's where mish tastics. Open source budget is at and the cool thing about open collective is it's like it's all right there. It's completely open So we can go. Yeah, I love

James: it. We yeah, absolutely. We use that for I also admin, the mastodon. me. uk server which we've got a couple of thousand people on and that's all supported by Donations and through open collective and everybody can see exactly what we're doing where the money is going on the server costs.

So it's great. But yeah, the, the funding is, yeah, it's, it's an interesting thing. I've been very lucky to get some funding from it's, it's basically, it's European union funding through the European union have a research program called next generation internet. NGI and they have various funds within that, like NGI zero I'm never quite sure how it breaks down.

Anyway, that's all administered then by a foundation in the Netherlands called NL net. Who put out calls every, every quarter for it's all. Specifically for open source you know, what you're doing has to be open source in order to to benefit from this, but it can be you know, a theme around bringing control of data back into back into users hands.

It could be a theme around security or, or whatever, but yeah, the funneling, funneling that money into open source Developers is is amazing. So I was lucky enough to get allocated, allocated, get to be awarded some money for that, which has let me spend the last few months full time on this, which is incredible.

I've got another application in, so hopefully it will carry on. But in, in the long run, it ideally needs to build up a, a sort of sustainable base from. From users on, on Open Collective, I'm hoping that's where it goes. We've got now quite a few projects that are sustaining themselves that way.

And it's, it's interesting. People, especially in the, in FedEva seem to be, You know, there, there was always this idea that, Oh, people won't pay for stuff on the internet. It's like, in the early days, I wanted to pay for Twitter. I liked it. I was like, I'll, I'll give you some money, but there was no way of doing so.

So they had some other business model that didn't care about what I wanted. And that, that kind of, no, I am willing to pay for something that I think is good. It's it's, yeah, it's, it's changing, I think. And there's a lot of things doing it.

Jonathan: Have you, have you had reach out or worked with any any of the printer manufacturers?

You can tell us about it. I have no secrets mainly

James: because I have no success on this. I did try. I did try once. I was like, Oh, is anybody interested in sponsoring this thing? And obviously I just went straight in the bin. The one thing that I cannot do is marketing. That's just not, it's, yeah, I don't know how or or whatever.

I mean, that, that would be cool. That would be a really good thing. I think it's, you know I think a natural fit for many of them. It's also a bit of a reaction to what many of the printer manufacturers have done. So. The first sort of big model sharing model publishing site was Thingiverse, which was which is run by Ultimaker.

I think so. I think. And it was, you know, dominant for a long time in sort of the early early days of these things. That's now, well, well, it went through a real. Bad patch where like just search didn't work for like a year and things like that. So people were going to other, other platforms things like that.

It was, it was building up sort of, you know, more more other platforms, but it seems that every printer manufacturer is now bringing out their, their own hosting site. We now have printables, which is. Somebody I can't remember which one is it Prusa. Is it? Yes, it is. It's Prusa. Thank you. And there's there's loads of other I came across another one today, which I think had I don't know if it was Bamboo or whatever, but it was like very much Download the bamboo thing here into bamboo slicer.

Okay, this is this is theirs and None of these things work together, right? So You have creators who will be Publishing on Five, six different platforms now, which is just wild to me. And then you have things like, I can't choose how I use Thingiverse, right? I can't say, Oh, Thingiverse is useful. I can pay for it.

No, I can't do that. Instead I have to watch a whenever I download, there's a 30 second ad that pops up. I mean, it's a static image ad I've seen it in side two seconds and decided that I'm not interested, but then I have to wait. So this, this, they're all trying to find ways to monetize these things, I suppose.

And Corey Corey Doctorow's term which I don't know, family friendly show. I, I won't I won't say I'll let you say it. I is, is definitely a thing, right. And, and. I think we've learned over what we've learned of the sort of the consolidation of the big tech and big social media networks is that these walled gardens aren't great.

So actually publish your own stuff, host it yourself. We make it as easy as possible. And then, you know, syndicate it out from there. So that's very much a kind of a a driving force behind Behind manifold as well. So I think that's something pretty manufacturers should be excited about, but it's also like, I want to stop you all building your own hosting sites.

Although feel free to run one and put your name on it and then join the Fediverse, right? It's that's something they can do by all means. Absolutely. That, that actually brings up, I'd have to see that happen.

Jonathan: Yeah, I would too. It brings up an interesting question though. What, what license is manyfold under?

It's MIT.

James: Okay. So very, very permissive. Very permissive. Yeah. I've long. Got over the the idea of trying to, like I say, nothing I do has ever worked. So, you know, I might as well give it away. No, it's, it's MIT, everything that I've, that I've done with it is, you know, there's other things that we've published as part of that part of the development is MIT as well.

I'm not, you know, looking to make a walled garden and get everyone onto one platform and then, you know. close the gates and and start making money off them. It's just not, it's not a thing. So if I can get people running their own stuff and you know, pay the bills alongside somehow, then that's fine.

Jonathan: Yeah. Yeah. That's the ideal. Do you have any really big instances of Manyfold? Is there anybody out there that's trying to run one as sort of a community instance?

James: I know there are some, I know there are some starting up because a couple of people have asked me in the last couple of weeks about getting things up for community groups.

One of the things that I very quickly put on the roadmap and launched last week was OpenID login, so people can integrate it with their existing authentication systems. And Fortunately, the, the way I'd built it made that very easy, which was nice. So we could get that out. It was a complete surprise feature.

I did not have that in the plan at all. I mean, I don't really have a plan, but you know but yeah, it was really nice to be able to say that actually, yeah, I can do that. There we go. That's the, the dopamine of open source, right. But I know of. I, I thought I had a fair few, I thought I had quite a, quite an archive of models.

I know, One instance that's pointing at 17 terabytes, which is insane.

I've had a few sort of bug reports come in from, from the the person running it. It was like, it gets a bit slow.

I'm surprised it hasn't set your house on fire or whatever. You know, it's, it's a bit slow. Yeah, I, I, I've been impressed actually how well it has coped with, with big instances. I, I'd like to see some, I'm just starting to collect public instances now. I've just put a list on the website of known public instances that people are happy to put there.

Currently, it's the two that I run which is the which is my own one and the demo.

There's a demo instance. You can go and actually play around with all of the features without any authentication or anything. But yeah, it'd be, yeah, I'm looking forward to seeing some, some instances come up, but at some point I'd like to run like a flagship instance so that anyone can just sign up for an account and start publishing stuff.

Yeah. That would be really nice. Yeah, budget is probably the thing at the moment. I've, I've, I've mentioned that as a possible thing in, in the sort of next round of funding as I actually like, it would be great to get a flagship instance up.

Mm hmm.

So,

Jonathan: yeah, is, is there a, is there a way to search across the different public instances of this?

So say I'm looking for a specific model of something and I just want to say, go out and, and, you know, look at all of the many fold instances, all of the things that are public. Is there a way to do that yet?

James: No, not yet. That's a general. So, so there's a few things. One, there's, there's one. main, there's a few main sort of 3D model search engines.

So I think Fangs does a has a search across other sites. And there's one called Yegi as well, which, which does that. I'd like to make it so that manifold instances can be indexed by those. If necessary, so that somebody who is running a search indexer, the stuff will pop up. And then there's a big thing generally about federated search and search of the Fediverse because every instance has this problem, right?

Whether you're on if I search on Mastodon, I'm only seeing what my instance knows about. So how does that work? And I know that there's work happening there, which hopefully I can. I can hook into and build upon in some standards compliant way, that would be ideal. So yeah, at the moment, no, but it's definitely.

Yeah.

Jonathan: And it's something you want to do. I think it would be really cool. I'm looking forward to that. So the the thing that we had to talk about with, with Mastodon, I think it comes up here as well. Maybe, hopefully it does. Hopefully it hasn't come up here, but do you, do you, do you guys ever have any problems with people like uploading inappropriate things or even just straight up trying to spam on Miniverse?

So

James: at the moment. No, because I don't know of any other instances. But it will be a thing, right? That will be a thing.

Jonathan: That's when you know, you've made it

James: Yeah, definitely. So I mean one thing I did last actually the the sort of current One of the current themes of development that I'm doing now that we are in the ActivityBub world is building in the required moderation and admin tools.

We've got things like moderator permissions in there already. I added in a sensitive content flag last week. But building those things so that it will respond to Reports from other instances give you a, you know, report log, things like that. So yeah, that's been in the, in the roadmap from when we started thinking about federation stuff.

It's definitely something we need to support. We need to be able to control federation as well. So which instances we do and don't federate with or, but that's all individual choice, right? That's not, I say we, it's not. It's everybody separately. But yeah, definitely want to build in those controls and learn from what's worked elsewhere in the Fediverse and what's but also what people have missed as well.

David: It does seem like Moderation around 3D objects is a little bit different than moderation around text. Yes.

James: Yeah, but you know, we've got comments and descriptions. So, you know, Comments and comments. There's text as well. And images.

Jonathan: Images is actually one of the ones that's a little scary. Because if you, if you tick somebody off, they can, there are, there are literally illegal images that you have to potentially deal with.

Absolutely.

James: Yeah. Yeah. Yeah. So, so knowing, I mean, it's been really handy having been a, an admin of a, of a Mastodon instance and bringing that experience to building this and actually going, yeah, okay, I need to be able to say for this server, don't, just don't talk to it and never ever accept any images.

from anywhere over there. That's definitely something we need. So I mean, I say that aren't mastered on moderation journey has been very smooth over on our server. It's been, it's been a nice, a nice ride, but I know, I know it's not that way for everybody. So, yeah. Yes.

David: To get So I know that this could be a potentially deep topic because you do have a PhD in it, but the explain it like I, like I'm five overview.

And going back to that massive survey you were talking about the size of those 3d models and everything you've mentioned in your blog post that you're actually looking at developing ways of 3d compression.

James: Hmm.

David: What does that look like?

James: So, yeah, this is this is one of the fun bits of of the project that I've I've got funding for.

I haven't done it yet. It's going to be coming on hopefully sometime in a month or so. I've really got to get this done. 3d models are big. They're, and some of them aren't. really big. I mean, some, some creators will create quite efficient measures. Some will just have millions of points in there that you really don't need.

So Yeah, things like, and this speaks to the sustainability thing as well, right? These things are expensive to send over the internet. They're they take a lot of space. So one of the things Manifold will do is it'll detect old inefficient formats. So you mentioned OBJ Wavefront. That's really inefficient.

It's all ASCII. It's just, it's massive and it doesn't need to be. There's ASCII encodings for STL, things like that. And we can convert those to 3MF. Which is a nice it's a, it's, it's a sort of more modern format, but then it's also zipped up. It's a zip file basically. So there's that, but then that still doesn't help you sending it over the network because you've still got to send the whole thing.

One of the you know And the internet might've got fast enough that we don't really worry too much about this these days, but you know that PNGs can be, and GIFs actually back in the day, could be progressive. So you could, they've got all the information you need to display them at a low resolution and then it refines and refines and refines.

So there are ways of doing that with 3D models. And like you say, I did a PhD in this years and years and years ago. 25 years ago, probably approaching I started. And It was when it was, it was all around animation of very very dense meshes. And the fun thing was while I was doing my PhD they invented graphics processors and suddenly what was a very dense mesh wasn't by the time we could just do it all with a, with a nice graphics card.

So. What I did for my PhD wasn't massively relevant, but some of the things that I, there are algorithms that I read about while I was while I was preparing to do sort of my research that are, there's a thing called progressive meshes written created by a guy called Hugh Hoppe, who I probably pronounced his name terribly wrong.

I'm sorry. Which basically you can imagine you have like, imagine you've got a cube, right? Or a very simple shape, right? You're going to get that really quickly. And then instead of loading the whole thing, you just say, all right, that, that corner there, split that into like that, split that on that, on that, on that, on that, on that, on that.

And then the thing refines. And. This is a really nice, nice thing, you get this sort of stream of progressive enhancement of this model, which means you can see what it is very quickly, but you can wait for the full detail if you, if you want to. So that's a It's a really, it's a really nice algorithm for that.

And, and I was looking around to see if anybody had done anything with this particularly, around transmitting them over the internet. But, you know, I mean, I was playing around in VRML back in the 2000s, but 3D over the internet hasn't really been a, you know, a huge thing. It's, it's all, you know, in games and things like that.

And you're just downloading gigabytes in one go and then you've got it. That sort of continuous transmission hasn't really been an issue. So nobody had done anything with this particularly in terms of that as a sort of, as sort of as a, as a transmission format. So I got in touch with Guy and said do you know if anyone's done this?

And, you know, we should put together a, a spec to do it. And so I've done a draft spec for it actually as a GLTF extension, which is a sort of format designed for, uh, for sending over the network. People have done mesh compression in that, but it's not this sort of progressive transmission thing.

So I've written a spec, I've got to write the code, and, and actually get it working. But the idea is that if you're looking at a, a very complex model on some other server, you'll actually get a low resolution version of it instantly, and then it'll refine over time. One fun thing about that is that it provides an interesting, interesting thing.

possible approach for commercial models in that I could show you a low resolution version of the commercial model and you can see that and you can look at it and flip it around and all that kind of thing. But actually if you, you know, when you've paid for it, you get the whole stream. And you get all that fine detail.

So, it's, yeah, it's a really interesting thing. It's, it's, it's nice to be able to flex the the 3D graphics brain for the first time in a good many years. Yeah. Doing this. Yeah, that progressive,

David: Concept that you just mentioned, and not just looking at it, but I could envision where it could be something where you could, like, print it in low resolution to test if it's what you want before you spend the money and get the full resolution product.

James: Yeah, that's true. Yeah. Yeah. Yeah. Yeah. Test size, you know, yeah. Yeah, yeah, that's a good point. Super interesting. That's a good point. I haven't really worked out what the sort of commercial obviously there's lots of commercial creators out there, right? So I need a solution if anybody's going to use this thing for that.

I don't quite know what that is yet. But I'd like it to be possible in some way. I'm not, not interested in like payment processing and things like that. Somebody else can handle that. But like, how do I, how do I make sure I've got a permission model that works for. for sort of sharing of a preview and then and then full versions of things.

It's kind of what I'm thinking about. But there's a lot of a lot of commercial modelers out there who who I think would benefit from being in control of their, you know, seizing their means of distribution, right?

Jonathan: All right, we have a, we have just a minute before it's time to wrap and I want to say something we didn't get to. Yeah, it's gone fast. Something we didn't get to. It has gone

James: fast.

Jonathan: I wanted to ask you about is what was the experience like adding ActivityPub? How, how big of a pain was that? Or is that something that is actually fairly simple?

James: It's got some sharp edges. It's no, it's all right. It's all right. There's, there's some good, you've definitely got to learn it and understand it and, you know, read the specs and things like that in order to to be able to, to do it. I think it's not, the basics aren't super difficult. A friend of mine did a a great blog post on how to build an activity pub server in 20 minutes in PHP like in a single page of PHP, which is, you know, you can do that.

There are interesting bits like signed requests, which is a little bit tricky and difficult to to work out exactly exactly why it's not working. Some of this stuff can be hard to debug because one of the things is actually it's all needs to be on the internet and talking to other systems. So I mean, while I was doing a bunch of the development, the deploy process was just the debug process was so slow because I had to build a version, deploy it up to my machine, then talk to other things on the internet, et cetera.

So it was it can be a bit like that. But one of the One of the things that I've done as part of this is I wanted to, so I built the the site's built in Rails which has been a, something I've worked in for many, many years. It's very nice for programmer speed, which for me is a good thing to optimize for.

It's just me on my own. And one of the things you can do with that is it's, you can plug in engines that do you know, that bring a bunch of features in one go. Like we have one for authentication and we have things like that. And I wanted to make a reusable component for activity pub. Mastodon is also built in Rails.

Their activity pub is kind of, well, it was the, the site was built before they adopted activity pub. So it's a little bit It's not sort of easily extractable. I don't think I was kind of going in thinking, Oh, what if you could pull out the activity pub call from masters? I've definitely read some of the code.

It's been useful to see what I've been doing wrong. Mainly actually, why is this thing not responding when I send it stuff? It's where, where is it sending it through debugging through other people's code to, to work out what you've done wrong is, is interesting. But . Yeah. The, I wanted to produce this reusable engine and I found that somebody had already had this idea.

And so I've been contributing to that project. There is a rails engine now called Fed Rails. Hmm. Which, when I started looking at it, it supported things like following back and forth and things like that. It's that was it had some of the basics down. I've over the last little while.

Implemented the the signed activities, different activity types multiple actor types. So in our system in, in Macedon, everything that you can follow is a person, right? That's simple in manifold. It's a creator, which is a person, but also it's a model and it's a collection. And there's all these different things.

So getting multi actor support in there Lots of things. So I've done a bunch of code for that which is, so actually all the ActivityPub core of Manifold is in that reusable engine. So, and, and I'll be carrying on with that and building more and more into it as we as we go. So anybody can pick that up and reuse it.

Hopefully fairly easily. Yeah.

Jonathan: Manifold is built in, in Rails, Ruby on Rails? Yeah. Yeah, that's right. That's one of those languages and frameworks people have very strong opinions about one way or the other. They do. They do.

James: I, I won't start a flight war.

Jonathan: I like

James: it. And that's

Jonathan: what matters to me. That's what matters.

Yes. Mash was data from the chat. Opinions are available. Yes. Yes. I can help supply some of those if you'd like. Master tailor from the chat wants to know what about that other federated protocol at, which that's the one blue sky is working on. Any plans to support that?

James: Not at present. No. I don't know anything.

I, I haven't looked at it that much, to be honest. I know that BlueSky does it. Is there more than one BlueSky server yet? Is there, is that even a thing? I don't really know. I'm not there. That's what I think. We, we've we, I did bridge the manifold. Fediverse account over to Blue Sky the other day, and that's about as far as I've gone.

I don't know. Should I? He asks the

Jonathan: audience. I will say this. I went looking for, oh, have any other open source projects picked up AT and started using it? And as far as I could tell with my quick search, the answer is no. And if that ever changes, I would think maybe you should think about it. But until then, I don't see the point of it.

James: Yeah, I think others. Other bigger, more important applications will pick it up first. And that's when I'll notice and think, Oh, maybe I should do something like that. But I'm always willing to be corrected or to learn more. Yeah. There you go.

Jonathan: All right. So I want to make sure and ask what, where do you guys need help?

So if somebody said, Oh, I know, I know how to program in Ruby on Rails, or maybe even if somebody doesn't, but they want to say, I want to roll up my sleeves and help with many fold. Where's a good place to jump in and, and start at? So

James: it's, so I've tried to make it accessible. I don't know whether I've managed it.

We have a bunch of issues on GitHub that are labeled good first issues that are fairly self contained things that don't involve you understanding the entire application. But it's not just Rails. There is, there is Ruby code. There is plenty of JavaScript code. If you want to do 3d JavaScript, I've got some jobs for you.

There's lots of stuff I want to, to add into three JS for instance. So there's, there's things there, there is design. I am not a designer. The whole thing is built with bootstrap. Because I'm incapable of doing anything better. So, you know, design UX one things we do have people we do have a little team of translators who've stepped up.

We're actually translated fully into four, five, four languages five in total. English doesn't count. French, German. Spanish and Polish Polish, because somebody popped up and said, I want to translate it into Polish. So I went, all right, sir. And, and he did, and that was the first one that actually got to a hundred percent translated.

So people want it in their language. Please do. I would love to have Russian. I, then I think after that it's Japanese. I don't know, I think, but things that I can't understand when they're wrong is, it's definitely, yeah, on the horizon. Although I mean, I can, I can read Russian. I just don't know what the words say.

Understood. But I did it at school so I can read the alphabet, but that's about it. And so but yeah, there's that but also things like documentation, I mean, open source really suffers from. From documentation issues, I've tried to do some. I've discovered that I would rather do almost anything else than write some bits of documentation.

But also i'm a really bad person to write it. I know how it works and how not to break it so those kinds of things like What haven't I explained? Is is not something I can see very well. So There's loads of yeah loads of opportunities for people to get involved. We're on we have a Chat space on matrix.

And there's all the codes up on GitHub and there's GitHub discussions there and things as well. So yeah, if it seems like something that, you know, I, I'm very much of the opinion that there's so much more to open source than there is just code. So, you know, there's, there's a lot more that that people could could do.

Jonathan: If somebody wants to run it, is the Docker image the easiest way to go?

James: Yeah. Docker image is pretty much the only way to go. Somebody did ask the other day how to run it on bare metal and a few of us just sort of all chorused in going, just don't,

Jonathan: you don't want it. You don't actually

James: want to do that.

I mean, nobody wants to set up a Ruby runtime and, and all these different bits and pieces. No, the whole point is all of the self hosting stuff I've done has the, the by far the easiest way it's been through, through Docker containers. Yeah. Normally the Linux server containers. I don't know if you guys have come across those.

It's a group that put out a, a set of applications as containers that all work in a standardized way. And I was very pleased when manifold got into that so you can run our own. Container. We have a version that has a separate database is the standard one. We also have a solo container, which is just run one container.

Everything's in there. And there's the Linux server one as well. There's instructions on the website for running it on things like Synology, Unraid All these various Various methods, but yes, it's all Docker. I imagine it's not

Jonathan: doing anything too obscure so you can run it rootless with Podman and all of that.

And it just works.

James: Yeah, we did a I don't know about Podman. I assume it will. We did yeah, one of the big changes to, to get the the Linux, everything was stopping it running as root and letting it run on read only file systems and things like that. So yeah, it can do that.

Jonathan: I will I'll have to give this a try with Podman then I'm my, my Linux usage is mostly on sort of the red hat fedora side of the fence.

And so I tend to go with Podman when possible. So I'll give it a try. I'll give it a try. And we'll let you know

James: what happens. That's, that's all the that's all I can ask. Let me know when it breaks. I mean, yeah, it's still under very active development. This is, you know, last week I put out three feature releases and a patch release for a broken feature.

So yeah, things break, but then I fixed them. So I've never lost anyone's data yet in 86 releases. Yeah, there's, there's not been any data loss. I'm hoping to get to a 1. 0 release maybe early next year, something like that. There's a few a few of these sort of the, the, the various features that I've got funding for, but a few other little bits to get in there that I can then say, okay, yeah, that's, that's a baseline.

And Then, then we go somewhere else, then we go from there, I don't know. But I do want to get to that one point, I hate projects that just perpetually hang on. A zero dot something or other. Yeah, yeah, yeah. Yes,

Jonathan: yes. Okay, so, I've got a, is there anything that we didn't ask you about that you really wanted to let folks know about?

James: Oh, I have absolutely no idea. I completely lost track of the conversation. No, I think just, no, come and come and check it out and come and have a look. If it sounds interesting, come and come and see. Well, I'll, I'll help people get it up and running. Cause like I say, it's early days. You still get very personal support.

Actually that's, it's not just me. There are now a lot of people in the support channel who will, who will help out getting it up and running. Yeah. None, none of whom I actually know. It's amazing. So

Jonathan: what what's the best place? So I see, I see many fold app. Yeah. If somebody wants, that

James: is the best place to

Jonathan: start.

Yeah. If somebody wants to donate, where, what's the link for that?

James: There is a donate link on that website. I think if you go to manifold.app/donate, I think that will probably work. Okay. He says nervously. But yeah, if you, you can click through to the main site. It's, there's a donate link downside.

There's also community links which will take you to the various channels. That are available there's a source code link for GitHub up in the, up in the top and yeah, all the documentation and getting started guides and all that sort of thing. Very cool.

Jonathan: Very cool. All right. So I've got to ask before we let you go I'm required to just about contractually.

What's your favorite text editor and scripting language?

James: Ooh. Ooh. See, how do you, how do you define text editor? So the thing I use day to day at the minute is VS code. I've gone through so many over the years. I was definitely an Emacs user for a long time. At the moment I'm using VS code. It's reasonably nice.

It's, it doesn't. break. And that's about all I need it to do. Actually it does break sometimes. It does break sometimes, but you know, yeah, yeah, yeah. Everything breaks sometimes though. So no, that's what I'm using at the minute. And I mean, favorite scripting language has got to be Ruby because it's just it's fun.

I've been using it for so long now that my favorite scripting language is one that I can get stuff done quickly in and not think about the language. And solve the problem. Yep. Yep. That's the way it goes. I'm not averse to any of the others. They're all great. I know many languages. Most of them are lovely.

Jonathan: Most

James: of them.

Jonathan: Not naming any names. Not naming any names.

James: I have mixed feelings about some that I have to work with. And that's where, that's where we shall leave it. Yes, yes, that is fair.

Jonathan: All right, James, we are, boy, we are beyond out of time. But thank you so much. It's been a blast. Talking with you and about Manifold.

And I literally have a terminal pulled up. I'm trying to get it installed on the machine behind me to let you know where the pod man works or not. So as soon as, as soon as we get close, yeah, I'll finish up with that. Thanks for being here. Thank you very much.

James: Yeah.

Jonathan: All right, Mr. David Ruggles. What do you think?

Have we convinced you?

David: Oh, well it's still a question of time. I never have enough time, but now I want to definitely get a 3d printer and get into the ecosystem. But yeah, that was fun. And and I just, it's always fun to talk to somebody that's just got such a wealth of diversified experience.

So I definitely, he, he, Hey somebody, if he's available, we keep him in for the roundtables. Yeah, that'd be fun. It'd just be fun just to pick his brain about so many different things.

Jonathan: Yeah, yeah, that was a lot of fun. And I'm, I'm real fascinated by this idea of what he's doing with 3D models, trying to compress them in different ways.

I think there's, there's definitely some room for that. And I didn't, I didn't get to ask him, but I actually wonder whether some of that is going to be cleaning them up as well. Because it's one of the things you get when someone's used like Blender or what have you. To produce a model, there'll be a lot of internal geometry that just either never gets printed or maybe never should get printed.

And there are some algorithms out there to try to clean those up, and not all of them are amazing. So, definitely some room for someone to work on that. But it's cool. I am literally working. I have a tab pulled up right here. I am working to try to get this installed and running using Podman. It's pulling right now, so if my internet connection craps out, that's what's going on.

Alright, a lot of fun though. David, do you have anything you want to plug?

David: Not specifically. I would encourage everyone to go check out Twit. That's the other place I like to frequent. So everybody else should like to frequent it too, of course.

Jonathan: Of course. Of course. So next week we are talking with Josh Bressers from Anchor.

Which that is about automated software compliance. And apparently they have quite a few open source projects that are connected to that. And so looking forward to looking forward to that, it's going to be next week. And then we've got some other things in the pipeline coming down the way. The, we've got the The J4Pi project coming up soon, which is about and we talked about scripting with Java.

Well, this is about scripting with Java on the Raspberry Pi. And that's, that's actually going to be pretty interesting because they're doing some fun stuff, exposing all of the different peripherals, SPI and GPIO and all that. So actually quite, quite a bit looking forward to that one as well. As far as plugs, we do want to make sure and thank Hackaday for being the home of FlossWeekly now.

And there is of course my security column, goes live Friday mornings there. And then there is, as we mentioned, the Untitled Linux Show over on TwitNetwork. And make sure and check that out as well for even more Linux y goodness from your host to co host today. We appreciate it. Thank you everyone for watching, those that get us live and on the download.

And we will see you next week on FlossWeekly.

Jonathan: Hey folks, this week we talk with Andy Piper about Mastodon. There's a new release, 4. 3, with all kinds of new tools. We get into the philosophy of Mastodon, the cool things you can do with it, and more. You don't want to miss it, so stay tuned. This is Floss Weekly, Episode 805, recorded Wednesday, October the 15th.

Mastodon, bring your own algorithm.

It's time for Floss Weekly. That's the show about free, libre, and open source software. I'm your host, Jonathan Bennett, and we've got something fun today. We're talking with Andy Piper about Mastodon. And of course, it's not just me here. I've got a co host today, Jeff Massey. How are you doing, sir? Oh, I'm doing good.

Doing good. Happy to be here. I appreciate you sort of sort of stepping in at the last moment I've I've had a bit of a health issue that I have been battling through for Well in retrospect over a month now, but past week or so was real interesting So things got a little behind but we're here. We made it and I appreciate Jeff being here

Jeff: Well, I'm just honored to be on the short list to be able to fill in and you know, I, it's, it's sometimes hit and miss based on, you know, my day job, but the Stars Aligned had an, had an opening and said, sure,

Jonathan: be glad to help.

Very cool. Now today we're talking with, like I said, Andy Piper about Mastodon and Jeff, do you have a Mastodon account? I do not. Okay. So our goal today, our goal today is to talk Jeff in to going and getting a Mastodon account somewhere or host it yourself. Like that's even an option. That's one of the beautiful things about Mastodon.

If you really wanted to, you could prop your own account up somewhere.

Jeff: Well, and you know, the interesting thing is I've heard about it quite a bit, but I haven't done a lot of research. I'm not an expert in it, so I'm, I'm really curious to find out, you know, all the. I know the high level stuff, but I, you know, a little under the hood and kind of what's going on and, you know, where's it been, where's it going?

The whole, the whole nine yards.

Jonathan: Yeah. And that's definitely part of something we're going to talk about because Macedon just had a new release. And I, you know, I don't remember what the version number is. It's a 4. 3 Macedon 4. 3 and we've got the man that knows about it and let's go ahead and just bring him on.

That's Mr. Andy Piper, and first off, we sure appreciate you being here. Welcome to the show, sir.

Andy: Hey, Jonathan. Hey, Jeff. I'm, I'm really, really excited and honored to be on a show, which is not only all about free and open source software, which I'm a huge advocate of Floss Weekly but also on Hackaday, which is another one of my favorite sites.

And I'm one of the things I do is I have a small maker studio here with my wife in the UK and you know, I'm on Hackaday. Literally all the time, all the time, every day checking stuff out. So yeah, so that's great to be here. Thanks for inviting me.

Jonathan: Yeah, we, we appreciate you being here. And now I've got to, I've got to ask, maybe I'm jumping a little too far ahead, but what's, what's the stuffy?

What, what is the, is that an elephant?

Andy: This is our Brand new just launched although we've been talking about it for, gosh, months, I think, because we've been through the whole process, but this is our Mastodon plushie. It's available now in the Mastodon store. Depending on your geography, we are working hard to make sure that as many folks can have their own.

Look at this, look, see, see how squishy he is? But I was away this weekend for reasons I was actually at Ogg Camp, which is something you talked about here on Floss Weekly a couple of weeks ago. And I got back to the studio Monday morning and my new my new work supervisor, let's say, Was here waiting to keep an eye on my contributions to the mastodon project Yeah, really excited.

It's it's a lovely plushie and also got to give a shout out to dopper 2 who is our designer so They've done all of the artwork for mastodon. So you can see the load the mastodon here. Not not this This is this is something different the mastodon logo on my shirt here the the mastodon mascot And all of the beautiful artwork we have You Around our website is, is by somebody who, who I know, only by username doper2.

And that, that's obviously the mascot the plushie. So, yeah, super exciting.

Jonathan: Is there also a rock band that goes by the name Mastodon?

Andy: There is indeed a rock band called Mastodon and so there is sometimes a little bit of search confusion depending on what you're looking for. But if you're looking for the Mastodon social network, then or the social networking software, then that is us.

And Mastodon is also a rock band. That's

Jonathan: fun. So you want to search for Mastodon social to get you guys. That's fun. That's the one exactly right. Yeah. If you're on a website that looks like heavy metal and has whales, you're on the wrong place.

Andy: Reaching down over here because I also have got a little 3d printed Mastodon mascot key ring that I've I made myself over here, but these are not available to buy.

But yeah, I have a lot of fun with it. The mascot is really, really cute. Is

Jeff: the the 3d printer file

Andy: out there so

Jeff: people

Andy: could I I So not at the moment because again, you know, it's done by our designer. I don't have permission to share that. So We'll see whether we can do that in the future. We'll we'll figure that out

Jonathan: There'd be there'd be a lot of fun actually to have that available.

Yeah, absolutely

Jeff: So you'd say with the Oh, I was gonna say with the rock band. Is there any naming conflicts? I mean not counting search but any legal You

Andy: Look, I, I, I am not aware of anything like that. I am not a lawyer, so I I can't comment. But I, I'm certainly not aware of any problems around that.

Jonathan: You mean trademarks? Surely open source projects don't have problems with trademarks. Nobody would fight over stuff like that. Let's not go there right now. Andy, I'm so glad you're here because if you weren't here, that's what we were going to talk about. And I was not looking forward to that. Oh, right.

Okay. Okay. Okay. So let's let's talk about Mastodon. You guys just had a new, a new release and what version of 4. 3? What's, what's the big, what's it, what's new in 4. 3? What's the, what's the roadmap? What did it look like? What new stuff did we get? What new fun things can people do?

Andy: So first of all, I want to say that it's taken about a year to get from 4.

2 to 4. 3. There's a ton of moving parts. We've got kind of the core engine of Mastodon. We've got the web front end. We've got our own Android and iOS apps as well. And then there's an API, which means that anybody can build their own third party client apps if they want to for either for you know, Android and iOS too, cause they, they prefer to, to build something for themselves or for other platforms that we don't currently have apps for, which is pretty exciting.

And. When we think about what's new, there's, there's kind of the new stuff that you get as a user, which is really cool. There's also a bunch of new stuff that if you want to run your own Mastodon instance, because you don't have to come to an existing Mastodon instance, you can run your own for you and your friends or your community.

Hackaday. social. Very excited to see that Hackaday have their own Mastodon instance.

Jonathan: Probably needs to be updated still.

Andy: Need to persuade Elliot. I think he may be the one that runs it to just refresh that up to 4. 3. And then also there's some new stuff for developers. So Mastodon 4. 3 for users gets a nice new look and feel.

It's, it's an evolution of the look and feel, but it's really nice and consistent. We've done a lot of cleanup improvements around, around the user experience. Especially around the media support. So you can now do things like drag if you want to. Attach a bunch of images to your post. You can drag and drop those in the composer to reorder them.

We've got also group notifications. So if you're, if your post gets really popular and you suddenly start getting loads and loads of boosts or likes, then those things will be grouped. The other thing that's really useful I find in notifications in 4. 3 is the ability to filter notifications. So you can do things like filtering out, if you want to, brand new accounts or from domains and things like that.

So there's some really nice stuff there. Another cool thing is you can now add a little tag to your blog posts. Fediverse creator, it's an open graph tag that you add to the top of your page. And then that will add, when you share a link to your post, then that will add a little a credit, an author credit under your blog post.

Link card, so that will enable people if people are sharing your blog posts or for writers and journalists, if people are posting their stuff on, on Mastodon, then it gives a little link to their Mastodon profile so you can discover them and go and find out what they've been talking about, which is really cool.

The embedded posts have been updated as well, so they look a lot nicer, so that's kind of a lot of the stuff for users. That's in the, in the default web interface. Incredible work really by by the very small but dedicated engineering team that do most of the work on the core team and then there's a load of stuff for admins and developers as well.