Episode 789 transcript

FLOSS-789

Jonathan: This is Floss Weekly, episode 789, recorded June 26th. You cannot eat the boards. Hey, this week, Doc joins me and we talk with Igor and Ricardo from the Armbian Project. It's all about bringing Debian to every armboard imaginable and trying to get good support for all of them. You don't want to miss it, so stay tuned.

Hey folks, it is time for Floss Weekly. That is the show about flossing. Free Libre and open source software. I'm your host, Jonathan Bennett, and we're going to have a fun time today. We've got the one, the only Doc Searles with us. Hey Doc, how you doing?

Doc: That may be one too many. I don't know, but yeah, I'm fine.

One too many. I'm

Jonathan: good. I'm good. It's good to have you back, back in the the, the, the secondary hot seat, as it were,

Doc: it's still

warm.

Jonathan: It's still warm. There you go. I like that. So today we've got the folks from Arman that is the Debian on arm, and I think some other things, maybe risk five we'll have to ask.

Are you, are you familiar with Armbian Doc or is this UN territory? I am now . You are now.

Doc: I am. I am now. And it's funny, when I first looked at it, you know, it's a, it was a tiny print and I thought. Ambient. That's interesting. Why did, why did they name it after a drug?

No, not

quite. No, no, no, not quite.

Not quite.

Jonathan: Yeah. So it's, it's no, it's, it's going to be a lot of fun. It's going to be really interesting, but of course it is sort of the, one of the kind of premier Linux distros for ARM and the, the list of boards that they support is really impressive. I'm going to have to, put a bug in their ear about adding a board to their list because i'm sure they get this all the time people like hey why don't you support this or that and you know there are reasons we'll ask them about that yeah let's go ahead and not waste any more time let's go ahead and bring them on so we've got igor and we've got ricardo and i want to say welcome to the show guys Hello, thanks for having

Ricardo: us.

Jonathan: Yeah, yeah, it's, it's gonna be, it's gonna be a lot of fun. I'm looking forward to it. So I want to, two questions and I'm not sure which order to put these in because they both kind of depend upon each other. Give us first, I guess, one of you take it, the 30, 000 foot view of what Armbian is and why somebody would reach for it.

And then the other question that sort of goes with this is what role do each of you play in the project? Okay,

Igor: let me try to do some very quick introduction. So RMB is like a base operating system for single board computers. That would be like a definition, but it's a lot more actually. It has a very strong build framework.

Which allows us to build images from, from sources. So, and customization in this process is really extreme. So we provide small IOT, let's say very minimal images, several images and several desktop variants. So, and also we implement all available technologies like video acceleration that kind of stuff. So, and Argonne even it has really let's say extreme diversity at, at the source point, so all the, the, the hardware has some specialties, different bootloaders different way how to put things together, so. At the end for the user, it looks the same. So this, this challenge is really still is but it used to be much bigger in the past because the, the, the development of the basic SDK was really, really poor.

Now it's getting better, but still, it's a challenge to bring like you mentioned, if you want to bring new hardware, new custom hardware, We say this is more like a custom hardware board. It's quite a lot of work. So it's not that simple to to add new, new hardware to Armbian. But once the hardware is in Armbian, it's quite easy to, to change it to some other distro.

So that switch is, like very simple.

Ricardo: Yeah we could say that Armbian, essentially, if you ask the users, it's a Linux distro, right? So a typical user, you ask him, what is Armbian? It's, well, something like Debian or Ubuntu, but it's made for my single board computer. And for this perspective, it's something that he could go online, find an image, right, that he flashes to an SD card.

And actually gets that board booting, which seems simple enough. If you're coming from a Raspberry Pi kind of thing, that's very obvious, right? You go to Raspbian, you get a download, you flash it with the image flasher and you boot it. So it's, Armbian is essentially if you look at the code of what Armbian is, it's an image builder.

So it's a build system that is able to compile all the bits. Necessary for those boards and that's starting from the infamous blob and then the bootloader and then the kernel and then user space on top of this. And so Armian was born mostly as a giant script that did build all these parts and made them work together.

And later it evolved into being a distribution, so it not only builds the the images that you can download and boot, but it also builds and maintains package repositories so that you can then upgrade your packages and maintain them up to date. So it's essentially a Debian Ubuntu derivative. But with very specific parts for those components that are not covered by other OSs. For example Ubuntu has support for a few SBCs, Debian has support for a few SBCs. RBM is building currently for 240 different boards. So it's yeah, there's a whole mishmash. And the big challenge about RBM is managing this diversity and this complexity.

And not let that lose control, and especially once a board is in, or a a SOC family is in, then we try to keep that updated and provide. LTS kernels updated, security updates, that, that kind of thing.

Jonathan: Okay. I, I have, I have a bunch of questions, place I want to go, but first I do want to ask I, I try to give you guys a twofer question.

So let's That didn't work. It didn't work. So let's start with Ricardo. What is your sort of how do you, how do you fit into the project? And then once you're done, we'll go back to Igor. Okay.

Ricardo: Yeah. So right now I'm doing this mostly part-time, right? It's a I'm not really dedicated into this, but a few years ago, I, I dedicated a couple of years, essentially two years to completely or almost completely rewriting the build system.

Mm-Hmm, . So I consider myself the lead developer on the build system. That does not mean that I am a kernel hacker or a boot loader or hacker. So I, I know how to build this thing. And I know a few of those families, so Rock Ship, some Amlogic I, I have an idea about them. But I'm I have no idea about all winner, for example which is a very, very popular family inside.

So I came to Armbian essentially because in my day job in the years before I was building cloud images Debian cloud images and Ubuntu cloud images for running on clouds. Right? And that was essentially the bootstrap and that kind of stuff. And then I found Armbian when I got aboard. And said, well, this is I can shoehorn this to produce cloud images.

But then I made a fork and once I did a fork, then nobody would take my pull request because it then made Armian build cloud images. So I then started doing transforming that huge amount of bash into something extensible. And not just by having scripts, but by really having a framework that allows us to customize different points. In, in, in that build system and then later we made this distributed so we can build the the separate parts so we can build today. We're building about 600 boot loaders. 57 kernels and this can be distributed. So my thing is really, my approach is really looking this from a mile high and trying to understand what is needed for the real experts, real kernel experts inside each of those boards and families so that they can do real good work and have tooling available to them so they can maintain this over time with minimal effort.

Jonathan: Igor, same question.

Igor: Okay my current role is more more like more in management of the project. So project management as a, as a, as a whole business development taking care of people their needs about the internal group. So that kind of tasks uh, but in the, in the development, I'm more involved in, in automation.

So I'm developing and maintaining automation for these systems. I'm involving new people to help me in this automation. Also, I'm still overlooking the build framework maintenance. So, like a main reviewer, kind of, but I'm not developing much in this segment anymore. Before, I was like, I, I'm the, that guy who who made that initial script back a year, years back.

Like it's almost 10 years back now where everything started. And then where also this community started to build around. So we started with a cheap Chinese SBCs based on Orbinar. So. I'm on the other hand more acquainted with Orwin than rock chip and then, or raspberry pie or others.

So Orwin I spent many years and we have quite a, a, a big team around those chips. And also today more around rock chips. So my role is like Founder. Yeah, okay, founder, yes.

Ricardo: Let's read this from the side. Creator and founder, come on.

Igor: Yeah, of course, yes. But I'm still, like, moving it as a full time job now for two years.

It was impossible to, to I would say eventually it became impossible to do this, this kind of work on a side. Yeah. So my, my full time job was start to fall apart, so that's all I'll say.

Doc: So, so, so I'm wondering how many how many single board devices do both of you have laying around that you work on? And where do you keep them? Because you're very, you have nice looking surroundings there. And my guess is you have a workshop or a garage that's full of these things.

Ricardo: If you can see mine here, well, there's about 12 boards going on there.

You can see the Linux heartbeats there. But I have a lot, a lot more this is really gets out of hand very quickly at the beginning, you start getting, Oh, you get a board somewhere. I started with a TV box. We can talk about that later. But then yeah, you get one board and you buy one more than some someone sends you another one.

Then a friend can't get one to work. And then it sends it. Well, I don't know. I have a few dozen.

Igor: Yeah. My, my number is unknown. So I would say yes. Yes. I can actually I have one Big box where it says retro. So I put there single board computers, which, which are already for museum. Let's say I don't deal with them anymore.

But then I have a full rack, full size rack. It's in the other room. Uh, there are plenty of those boards, but they are in in in action. So I'm running a test, test operation there. So all, all, all the code that is changed every day goes to those boards and we see what's happening. So it's a test facility.

So there are around 50 boards currently that are running.

Jonathan: Ah, so I, I, I work some with ARM boards. I do some things, I do, I have some raspberry pies, and I got started way back in the day with the beagle board, I think, like the beagle bone was one of the first ones that I worked with. And, I, I have sort of a an observation, and that is, the boot process on arm boards is terrible.

Is this, have you experienced this too, or is it just me? Yeah, it is. It is.

Ricardo: It is complex. So essentially these boards, you can take them into three big categories, right? So something that boots and it has a proper UFI, ACPI firmware. So based on usually the K2, Tiano core. And those are very, very restricted.

So those are really for big server boards. So all the ED is 128 cores. Those have proper UFI firmware so they can boot generic operating systems. Say you can just get Ubuntu ISO and put it in and it'll boots. Will it work perfectly? You don't know, but it will boot, right? And there are boards, they're essentially based on U boot.

So U boot is this very popular open source project. Most of those boards at the SDK level, so at the silicon designer level have been brought up using U boot. And then you have a, a, a, a very small number of boards that represent a very large amount of users, which is the Raspberry Pi, which has Broadcom specific bootloaders, which is different from all this that I said.

So those are, I might be forgetting some here and there, but those are the main things. So if you're doing the Raspberry Pi and you're doing the Raspberry Pi the way it's intended to do is very easy and it works. If you have the UFI board server boards they call it server ready certified or something like that, then it's just UFI.

You're gonna have troubles with the A CPI tables and other PCI stuff, and then your GPU won't work , but it, it'll boot. But then this large majority of those boards fits into the, the UBO category. And then that's not only U Boot because you need also firmer the blob. Right? So this is mostly closed source provided by the silicon vendors.

Or in some cases fully open source but then that, that gets extra complex. So the main thing about those boards is you really need a way to get into their console. And this is about UARTs, right? Serial consoles. And if you set that up correctly, then well, you can watch it boot. Otherwise, you have to wait and hope that the bootloader works, and it brings up the kernel so that you can get some HDMI display.

And if it doesn't boot, then you're lost. So yeah, it gets intricate. You really need low level access to the board to be able to debug it and understand what's going on.

Jonathan: Yeah, so I've observed that we, we kind of see more of the UEFI. Starting to come to even smaller boards. And one of the interesting things is, is people are doing.

UEFI shims, essentially, where they'll write something that maybe you boot will boot or even for the Raspberry Pi, this exists, you know, it'll boot through the, you know, the proprietary Broadcom system. And then it is a little tiny piece that then gives you UEFI. And so that sort of seems to be the direction that things are moving.

And I've got to say, like, as an end user, UEFI makes things easier. It really does. Oh, for sure. You get grub, you get all sorts of stuff that just starts working the way that you expect Linux to work. For sure. But, I, I'm, it, it fascinates me that you say that it's such a pain to work with. That I, I didn't expect that.

Ricardo: It's not, it's not really a pain to work with. What I mean is usually if you buy an x86 PC you just assume that your BIOS works. Right? Yes. Sometimes, sometimes you need to flash it. Oh, there's a, a support for some new RAM. But usually it just works, right? This is not absolutely, I can't say this is 100 percent true on the ARM side of things.

It is on the very professional server boards where everything's tested and validated. But for the smaller boards at lower costs, they simply isn't there yet. So this ADK2 project is really catching up, especially on the Rockchip platform. So the 35 6x and the 3588 boards have a really good support.

For a DK two, which is yeah, a, a, a complete UFI firmware, which can access all the devices, can boot from the network, can boot from sat, can boot from NVME and this kind of stuff. If you go to Raspberry Pi, yeah, you can put a, a SD card with the UFI firmware on it, so you can, it puts its internal boot loader.

And then chain loads. And then everything works, but it's well, is it booting from the SD and then later loading the OS from a USB. So it's not really built into that. It's completely different if you buy an unfair server, or it's just the firmware is there. It's open source. It really has been tested and they are ironing out all the kinks. To the point that it can boot Windows on ARM, right? So, so it really is, it goes into there. But really, it's much heavier, it's much harder to work with. It's C if I'm not mistaken. So if you're developing something for those boards, you usually don't need the UFI thing. If you're doing some IoT project, U Boot was gonna get your kernel loaded faster in a more consistent way, simpler, right?

So Yeah it does depend a lot on the use case, but most of those boards are still on on Yubu, right? Some we've done in Rvn some experiments, some experimental images using edk2, especially for the rock ships and they work, they work fine. But we're not ready yet to commit. There's a very large investment in Uud, right?

And we are very few guys supporting this. So it's not the time yet to move completely. Even if we could, that would be addressing about 20 or 30 boards out of the 240.

Jonathan: So I'm curious, and maybe this is a better question for Igor, what, what does the process look like to get a board added to r bn? So it, I'm imagining that somebody.

makes a new issue in the GitHub and says, Hey, I've got this cool board. Why don't you guys support it? And you hope your users are nice, but sometimes they're not as nice as you would like. Is there some process where you say, if you send us one, we'll probably be able to support it or give us SSH access into your board and we'll try to support it.

Is there, is there some kind of standardized process you guys have?

Igor: Well, it's, it's a bit difficult because you need to allocate resources. So if you don't have resources. I would say the team the people who are currently working on the project or maintaining some, some hardware is, is already over to the top with, with everything.

So I, I, I ask guys. This it's usually, it's usually hardware dealers that are, uh, sending us some samples. So they are showing, they want to take this board, they want to look on it. It's, it's not coming from users that much, also from users as well. But it's, I, I ask, if there's, if there is no answer if nobody will take it, there is nothing I can do, I would say.

Because I have on the other side people who will do bring up if they can pay the bills with this. So so, but if there is some strange, weird hardware that only one user has, it's like, it's too expensive to, to, to, to start. So we cannot cover this. It's it's really. Difficult. I mean, and you need the hardware and you need some, some support from, from vendor as well.

Okay.

Ricardo: Okay. Igor, once, once Igor has your address, he sends you boards you didn't ask for. I don't do that. No, it's not true. It just happens by accident. Okay. Not really. From a, from a technical perspective. Usually the, the board vendor, right? The, the guys who are doing the PCBs, they bought the SOC from someone, right, from some Silicon vendor.

These guys have an SDK usually very old Linux, very old year boots. They bring it up, they, they send it over and say, Hey good luck with this. So essentially the, the more common nowadays is that some user takes that, boards that into some kind of more mainline ish Format and then submits a pull request.

So it's usually a hobbyist user that is well, knows enough about this stuff to do this. Sometimes it's developers, just random developers in our community find a board at a conference or something like that, and then, then bring it up. But yeah, it's mostly pull request oriented. Has, there has been have been some cases where, well, a vendor came and said, I'll send you five of those boards, will you bring it up? I find that fine, but I also find that I cannot eat the boards. So what I mean, they're awesome boards, but I

Jonathan: cannot eat them. So yeah, Bitcoin on them to pay for food. Have you had vendors actually offer to to cover your time to pay you money to do the bring up?

Igor: Yes, in a few cases, yes. Yeah, that's always handy. Yeah, but it's not that good still.

Jonathan: Sure. So is RMBN running mainline on all boards? Any boards? Is there this massive pile of stinking weird patches that you guys have to apply on top of mainline? How does the kernel side work?

Igor: Well, it's not

Jonathan: depends,

Ricardo: right?

Igor: It's not smelly patches because most of them are ours.

Jonathan: No joke. I, I write code. My code is sometimes smelly. You can, you can admit to it. It's fine. No judgment here.

Igor: I know, I know. No, just yes, of course, it's heavily patched mainline kernel. So, and again, it depends on which family. So, we have all winner, which is Which is massively patched and we have a rub chip, which is a little bit less and then we have several which are just Just a few patches on top of mainline.

So that gives you also the status of mainline kernel by let's say Soc family and also the interest on the other hand So if you have small users you also have a lot of Finds less problems and you have less patches, so yeah, it's, it's, it's connected.

Ricardo: It also depends a lot, essentially, on the age of the board and the interest it generated.

So when it, when it got added very likely, I'm, I'm going to generalize, but yeah, it got added using a vendor kernel which is Not just an old mainline kernel, it's an old kernel that has been heavily hacked upon by the vendor. That means that if I take that same vendor source and try to run it on a standard machine, it won't run because it has been hacked at.

So usually, sometimes, especially new SOCs are added in that way. And then as the work starts on the mainline kernel, then there's a whole bring up that, that is very basic stuff like clocks and pin controllers and this kind of stuff. Once that gets the first lump, right then we can, we can add the mainline kernel support, which has probably can bring up a few CPUs, can bring up the RAM.

It can boot and have a serial console. It won't have internet, it won't have HDMI, it won't have anything. So this is the situation with the latest Rockchips, for example. It's two years ago we could get some decent experience using a vendor kernel that was completely out of date and insecure. Or we could get a serial console on mainline.

Two years have passed and now we can get almost a great experience on mainline. So this has been done mostly by people outside of Armbian, but also by Armbian developers themselves. And Armbian has been this channel to expose a lot of users to this without them having to go pull from a crazy git tree somewhere.

They just pull Armbian and say, build me whatever is the latest that you guys can have about this, right? So for each board, we do have a few, we call them branches. They're not really branches, but essentially we have a version of the thing that's called legacy or vendor, which is essentially the SDK supplied by the vendor, sometimes with some fixes.

Then we have something called current. Current is usually a LTS mainline kernel, so currently would be 6. 6 or 6. 1 in some cases. Yes, 6. 6. And then we have an edge branch, which is where the action happens, right? So this is currently at 610 RC5 or, or something for most boards. So this is really a bleeding edge and, and, and trying to bring the latest and greatest not only from the mainline, but also with patches on top.

So it's, it's like really dual headed serpent

Jonathan: there. Yeah. So I, I have, I have noted that. Vendor kernels are terrible. And you look at some of them sometimes and it's like, Are you guys not ashamed to have pushed this? Like, it's so old and it's so bad. I, I, What I can't figure out is why more vendors don't work with the upstream kernel to try to get their patches to work.

upstreamed. It seems like it would be such, such a smaller maintenance burden on them rather than trying to bring this five year old kernel tree along for every revision that they've got. Just, I, I really don't understand why there isn't more effort from the people actually making the, these things to get their, their code upstreamed.

Ricardo: Yeah, and even you see a few vendors which are not there. They're not upstream first, right? Which is essentially what Google is a bit forcing Qualcomm to do. So you guys want to do on the Chromebook, so you've got to be upstream even before the silicon is ready for consumption. So that's the ideal world when you Some, some big company like Google setting the rules and then those vendors are forced to do it.

But we, we do have a few vendors that are at least trying to get a bit closer or at least trying to work on an LTS kernel and then backport the fixes actually, so that you're not completely out of a support. But it's pretty rare. So this, these guys have this embedded mentality, which you do one kernel, you ship it off into a device, and you never touch it again.

Yeah. So it's, it's really about their mentality is about where they put their money and developments. And of course, you're submitting patches to the Linux mailing list, people are gonna look at it, I'm gonna tell you to change your stuff. Right. And don't send me

Igor: this.

Ricardo: Right. So these vendors want to get out something that they can demo.

And, and show how many FPS or whatever it does. They don't care about making a maintainable code. So the, the, they don't have the financial incentive because yeah, it's saying, oh, we are mainline first is nice, but it doesn't get the demos and the FPSs and whatever. It's difficult

Igor: to control this for them.

So so they have a SOC, they, they got a kernel which SOC vendor is kind of supporting and, uh, they're. Paying for that support and they don't want to spend anything more than that. And so for for some Specific use case for IT for industry. They need one feature or two features to work and I'm totally happy with that so they will not invest into new kernel because All this good enough.

So my TV is running kernel 4. 14 and it will never be updated. And all those devices IoT devices are the same. And we, there is little we can do, but we are pushing, let's say mainline and we are getting, let's say interest from not just from end users, also from industry who are having those weird All winner or rock chip devices, can you maintain a mainline kernel for us?

Because this mainline code, which is up, up there, let's say upstream, is also not something that always works. It it, if nobody is maintaining certain sections, it start to collapse. And this is what industry is afraid of. So it doesn't work. So , yeah.

Doc: So I'm wondering what, what, what, how do you guys support yourselves? What is your business or is a different business for each of you? And is there, do you have a side gig or is there money in this? I'm not, I'm not clear on, on, on the economics of your lives.

Igor: For, okay. Ricardo has a job. I am here full time.

Let's say I invested some, some, some of let's say my savings into this. So the company is providing support, so professional support services. So we have, let's say, a few, few support deals and that keeps us running. Of course, we got donations and that stuff, but that's that doesn't pay the bill. So.

Ricardo: Yeah. And from my side, like I said, I can only eat the boards, right? They try to pay me with boards, but that doesn't really work out. But to be fair, I did get a few years ago more than one job engagement using RMUN as a base and coming from RMUN, right? So someone found a video on YouTube that showed some of the boards that I brought up.

Someone in in California said, well, I can use this for this audio project. This board is really interesting. I got hired, got a good gig doing that for, I dunno, a couple months. So yeah, I do get some gigs out of out, out of this, but yeah, I know really no intention on the, the, the involvement in RB and the build system proper is really a community effort because there's no money there.

Yeah, it's

Igor: can the board.

Jonathan: I'm, I'm curious. You know, we've got some, the UK and the eu, they're starting to pass regulations and pa passing laws to try to make iot things more secure. It does that have an impact on you guys and I guess I guess you could imagine an impact like directly on RMB in but I'm I'm actually hoping that this is going to have kind of a secondary impact on RMB in where maybe it forces more vendors to look at sort of the Development that you guys do is there is there any interplay between RMB in and some of these new regulations?

Igor: No, but we were involved, I would say we tried to be involved because we didn't the project didn't start. But related to security there was some, some I would say foundation gave out some, some funds to, to improve security in, in in the OS level and they provided some, some funds for it. We tried to apply for that, but we didn't. We didn't came through. So But, so it was, the point was to so it was security title was security and there was some European funds, whatever, I don't know to improve, yeah,

Ricardo: NGOs come up with yeah, some funds for those and sometimes we can get some of that but not, not, not much really.

It's, it's really The, the IOT aspect of this is, is the interesting part is because Armbian is a general purpose operating system, right? It's just like Debian or Ubuntu. So, okay. You can have a minimal image and you can have a CLI server image, and you can have a full desktop image, for example, but that's not where the power lies, the power lies in the build system and it's extension framework.

So. This really gets people who are, who get one of those boards and say, well, I want to make a product out of this, right? And then they can customize that image and trim away what they don't need, add what they need on top of this. And then it becomes like that, that company's IOT project. So Armbian's really, doesn't really have too many features in, in the final images that you can go and download for IOT.

But it is really in the development process, in the build process, and when you get a hands on with the build system, where you get the value out of this. And this comes essentially because it makes it very easy for people, well, I can get this Debian Ubuntu going, and now I just need an audio player.

Right. I need an audio player that also starts when it boots. Right. And there you go. You have an appliance and that's an IoT project.

Igor: We got we got I think two, two two projects uh, from, from from industry that they were interested in switching from Yocto to Armbian. That was, so they came to us and they said, can you help us?

We have this Yocto thing this BSP, whatever. And, but our customers so their customers were Saying this SDK is so complex, so if they want to change something in the OS, it's, it's, it's really, it's really painful. But Armbian gives this, provides this really, really simple. So it's really simple to change something, to add something, rebuild image, and flash it to the device.

So this process is really simple. Much, much less complicated than Yocto, for example. Yocto, you really need to learn. You really need to invest some time. I think with Armbian I don't know, Ricardo, how fast do you think a new, a new person can, can, let's say, develop it, of course. Yeah,

Ricardo: depending on what, what, what is your IOT project, right?

Depending on what it is, and is, is your base software packaged already in Debian or Ubuntu? And is your kernel already brought up, and your, your U boot is already working? So it's really just glue codes essentially saying instead of what when you bring up you're going to bring up a generic desktop you're going to bring up something that is like a kiosk mode something for example or something that doesn't even use a display just outputs audio or read sensors or and then there's of course some companies need more of this so they need over the air reprovisioning of those images They need metrics, right?

So they, they, they want to collect telemetry data from this. So yeah, there's it really becomes quite disconnected from the original image. A certain a certain point, but it should, they, they can use, really be simple. So I, I've seen prototypes being built in today. for all your stuff.

So it's really really cool, interesting stuff.

Jonathan: So I'm curious, the name Armbian obviously suggests that you guys are very much ARM focused. I'm curious if you have some support for things like the the Star5, the Civision 5 2, which really is one of the first RISC boards that's actually usable a little bit more than a toy.

You can actually, if you really wanted to, you can compile on the board, which I think is an interesting thing. Um, Is there RISC support in RMBN? Is there NIPS support in RMBN? You know, are there any of these other esoteric sort of

Igor: No, no, we didn't go further. So actually we start with Arm, arm first with arm third two beat, HHF, let's say the, the back in the days.

Then it was upgraded to arm 64, and then Ricardo came to the idea that X 86 would also be good to have. So it was added X 86. I'm running it on my laptops, on my servers, so and of course, risk risk five. We have the support exists. We have, I think, two boards that are officially supported. I don't I don't know if this one is among those, but there are not that many different chips anyway today.

So yeah.

Ricardo: So I was guilty of adding the x86 support to Armbian, which made it AMDivian or whatever. And then, yeah, some, some other folks that got really into Division 5 initially they started sending some patches and then I had to add build system support for this because the two chains are different and there's no TFA, it's SBI.

And there's a lot of smaller things to be done on the build system side. So I integrated those. I really, I actually don't have any RISC V boards running, but I added support for a half a dozen. So, and that really is the case. I'm just trying out the build system and then let those guys build images and see if they work.

I hear they work, but I never saw them work myself. So it's really, there's, I haven't had any focus on the RISC V side of things. They are interesting, future wise, but I haven't seen one that I said, Oh, this one, I want to have it myself. Yes. Yeah. But yeah, I'm yeah.

Igor: Yeah. We are currently working on one eight core what is the banana pie?

F three. Oh yeah.

Ricardo: The F three. Yes. Banana. That's eight core. So it starts making interesting. I personally not interested in things that have much less than eight gigs of ram. So my ideal board today has 16 gigs of ram. So, yeah, unless it hits that sweet spot for me, 8 cores, 16 gigs of RAM, I personally have little interest in them.

I

Igor: don't know how many memory actually this one has. Is it 8 only, or?

Ricardo: I think it goes up to 16, yeah.

Doc: I'm curious about what your users are doing. I mean, what are, what are the typical uses of these boards as they go out there? There must be some sort of, you know main use or maybe it's just all over the place, but there must be some interesting stories about it too.

So, but that's usually the question Jonathan asks.

Ricardo: Oh, there's a law, there's a law, there's users running a CLI application. So they're doing stuff like running home assistant, for example. So Home Assistant itself supports a few boards that they well, sell or, or, or, or support directly in the Home Assistant OS.

But using Armbian, you can get almost all, any of those 240 boards and, and, and run Home Assistant on them. That is very popular. And I mean, I mentioned Home Assistant, but there are others, of course, right? The, the DNS filtering thing by whole. Many, many others. So, so this auxiliary kind of things or run a, a router or a firewall, that, that kind of stuff.

That's very popular. And I think there are users, yeah, for NAS situations. So people getting those boards and then you can hook up a lot of SATA disks or NVMe disks to them and running a NAS solution like Open Media Vault. Based on the Armin kernel, that's very popular. And then on the more recent more powerful boards, they have GPUs and faster cores.

There's a lot of users actually using them as daily drivers as desktops, right? So they're doing we, we built some KDE Neon Plasma 6. images that were very, very popular a few months ago. So those, those get actual 3D acceleration they get using the vendor kernels, they get video acceleration.

So those are really starting to get really good to run as a very cheap desktop, a very power efficient desktop. So it's, yeah, it's, I, there's, of course, a lot of people in the middle, I'm not mentioning here, but it's either people running really headless. CLI applications or desktop stuff, I think.

Doc: I'm also wondering about I'm involved in endless conversations about personal AI and, and having an AI appliance that will, people will need an AI appliance of some sort.

Now that would seem to me, That'll require a specialized graphics chip or something like that in it. But I don't know, maybe in the armed world, I'm not familiar, familiar enough with it. So is, is, is AI on your radar at all in terms of what people are wanting?

Igor: I don't know , but I think yes. It's, it's, it's a hot, it's a hot topic.

There's the reason and some of those boards are coming with chips for acceler ae acceleration. , but on very basic with basic capacity, I would say. So they're not so powerful, but I don't know what they're used, what they can use before I, I'm

Ricardo: doing ai at the day job, right? Using those very, very, very, very expensive.

And dvia cards , so running cards that have 80 gigs of video ram, for example. So for really large language models. There's nothing even remotely similar at the arm. Of course, if you get one of those big Ampere servers, you can plug in an NVIDIA card into it and it will work. So you can get one NVIDIA card running on an arm board.

But if you look at those smaller boards that have GPUs in the, in the SLC. Those are really much different and much less powerful and have shared memories, so they're really not adequate. The interesting thing is that some of the new boards, so the the latest, not the latest, but the second latest generation from Amlogic and the latest ones from Rockchip, they have what they call an NPU, so it's a new a numeric thing, so those are specific for inference.

You cannot train models on them, but you can run and infer against them. So usually very useful for, especially those boards that have multiple connectors for cameras, for example. So you can do object detection in real time. But this is really only starting to show up in the mainline kernel, and some of those are re get really, really interested be interesting because they are similar to the GPU cores in the same SOC.

So the drivers for this are being added by Tomeu Visoso used to be a collaborator guy, and now has been working with Libre Computer for this. They are being added into Mesa. So it's a really fascinating. See how this is going. There's not really a super standard API, either the kernel level or at the user space level.

So OK, maybe tensor flow. Has, has this aspect of but it's really very fragmented right now. Those boards do have six flops NPUs, five flops NPUs. So they're decent enough for, especially for those image processing use cases. That is actually more like machine learning than proper AI, right?

It's no usually I associate AI with the large, large language models. And those can run on the CPU as well. So especially on those machines that have 1632 gigs of RAM, you can infer those models and run them actually on the CPU using stuff like Olama and other open source projects as well.

Jonathan: Yeah, the Sozo's he calls it the, the rocket let's see the, what was the rest of that?

The, the, the, the rocket accelerator. That's it. And yeah, he's got the rock ship one. Yeah. For the latest rock chip. And he's got a demo of it doing Real time object detection at reasonably fast frame rate too. That, that is, that is actually pretty impressive. The fact that that's, that's mainline, like we're not, we're not depending upon a vendor kernel with that.

And it's actually running in Mesa. That, that made me feel really good when I saw that, you know, a little bit of hope for the world.

Ricardo: Yeah. And he did the Vivanti one from Indiana Logics and they call it Vivanti. Which is just Vivanti, I'll come backwards.

Jonathan: Yeah,

Ricardo: and that works pretty great. And that is, the full stack is out.

So you can get both the kernel and the Mesa parts of CL. You can actually run the models. The rocket stuff, I think it's too cutting edge still. It's bleeding edge. So it's just being sent to the kernel mailing list. I don't know how the user space is. It's very new. This also is doing a fantastic job and I hope more people sponsor him to, to keep on doing it.

Doc: I'm wondering I, it feels to me being old and having been around before even the PC was there that, that the, the AI world sounds to me a lot like the mainframe world sounded in 1974 when, when you couldn't, everything had to run on a terminal and, and, and the, the, The idea of personal computing was oxymoronic.

It was like, it made no sense because there was no such thing yet. And, and yet I think the primary use in the long run for AI is going to be ordinary things in users lives, you know, I mean, you know, I, you know, for me, I mean, one of my examples is what are, what are all the property that I have? Where did I leave this thing?

Where was I three weeks ago, Thursday? You know, what route did I take to get to someplace? I mean, I, I know a lot of this data is in the hands of giants that I don't have it, but but a lot of it is laying around here, you know, like just getting, I don't have this problem, but if. If your credit card bill has Amazon on it, it doesn't line up with Amazon's record of what it sent you because it accounts for it differently.

The bottom number is the same. I want an AI to do the algebra on that and do the logic to figure out, oh, wait a minute, that was a business expense, that wasn't. And there's just a lot of stuff in our everyday lives that are, that's out of control that an AI would help with. And, And interestingly, I don't think our desktop computers are made for that.

I mean, I think there may need some kind of network attached, something else that can handle that stuff in a dedicated way, like, and, and also like selectively disclosed data about me to the marketplace on an as needed basis as well. I don't know if you guys have thought about that at all, but, it seems to me that's sort of like the unexplored part of AI at this point. Everybody's sort of like modeling their idea on, gee, you need giant server farms, you need these expensive NVIDIA boards, and the rest of it.

Ricardo: Yeah, but you do need them. That's the point. So those, especially for

Doc: some things, right? Not for everything. Yeah,

Ricardo: especially. Especially if you're training models, right? So if you have this, this set of data, it's labeled manually. You don't, don't ask by whom, but it's labeled and then you need to process this this training of those models, you know, into something that can run, you really need those, especially the, the VRAM is really important.

So this is not the same DDR4 or 5 RAM you have in your in your, in your PC. There's this. Two orders of magnitude faster. So yes, there's, they're expensive, but once those things are trained you can do inference on, on quite commodity hardware, right? So you can do them on the CPUs. Just check out, Oh, llama, for example you can, you can get running with those Lama tree.

Gemma and other models from, from, from the big players, right? Because they're pre trained as it were. Yeah, they're pre, they're pre trained. Exactly.

Doc: Yeah.

Ricardo: So if you're really working in, in this area or training your own models, and then that, that changes the thing that, that's why NVIDIA is so so far ahead.

It really is about the APIs that you use to program against. So CUDA and this kind of stuff. So there's really really at the beginning of this there's no, no effort to standardize any of this yet because its value hasn't even been proved completely yet. So why standardize an API for this?

It's very, very appropriatory stuff right now. So

Jonathan: So, MashedPotato No open source at all. Yeah, MashedPotato in our live chat says, Doc, have you considered Microsoft's recall for this? Exactly. Oh, wow. It, it, so it's, that's funny though, because what you're talking about is sort of the same thing that Microsoft was trying to tap into with recall.

The problem is, the way that Microsoft did it, By taking screenshots? No, there's, there's the security problems and all that, but they tapped into the creep factor. It's creepy that an AI is taking screenshots of everything you're doing. Yeah, somebody dropped the ball on that one.

Ricardo: Yeah, and I, I can go to chat GPT and ask it to write a device tree from a vendor kernel to mainline, and it hallucinates so beautifully.

It's like, it really has no idea what a device tree is. What a kernel driver is it's it's really it's fine for really basic stuff But all it's all it's doing is it doesn't serve arbian at all

Jonathan: It's putting it's putting letters and characters into the same order that sort of resembles the other things Labeled device tree that it's seen in the past and that's not very useful

Doc: It's funny, I gave a a talk here at the university where I'm attached.

And it was actually about the history of open source and the history of, of, of standards, of internet standards. And, and most internet standards are with the IETF now, and the IETF has these RFCs, requests for comment. And, and each standard is kind of attached to an RFC. So I said, make me a table, this is ChatGBT, which has the RFCs on the columns.

And it has standards or uses, I mean down the left. It had about at least half the RFCs wrong. And the dates wrong. You know, and, but the stuff it put in the boxes of what it did was like, that's half right. But I realized, I created an enormous amount of work for myself with that. And I got rid of it, actually.

And I came away thinking, you know hallucinate is the wrong word. Bullshit is the right word, because that's exactly what it is. They're bullshitting, you know, they're making stuff up and they know, it knows to the degree it has knowledge that is, I don't have an answer here, you know, I mean, a human being will say, Transcribed That's a good question in order to buy time and then say I don't have an answer.

These things like, I'm going to give you an answer. It could be anything, right?

Ricardo: If you look at this kind of stuff, so the Raspberry Pi 5 now has a PCI exposed thing. We've we've exposed full size PCI exposed on the stuff like the rock pro 64 since 2018, right? So it must hit critical mass. With the user so they all know the Raspberry has a thing. Yeah, but other stuff had that years ago.

Why didn't anyone commemorate this at all? So this AI thing will also, I think, settle once it reaches a certain point, right? Right now it's all too new and all too impressive. But that main thing that don't assume that what you saw that specific demo work extreme, extremely well for

Speaker 3: will

Ricardo: also work for your use case.

Yeah. Right? So the demos are impressive when you get to actually, everyone wants to deploy AI in their companies. Nobody actually wants to talk to an AI in their personal life.

Doc: Yeah, that's true. And, and An interesting thing with AI too is that if you, if you want a clear answer to something, you don't want a different answer the next time you ask it, right?

You want it to be the same. It is never the same. You know, you want it to draw you a green field with people on it and the people have balloons and you ask it to draw it again, but without the balloons and you get a brown field with completely different people and. Three balloons because you used the word balloon.

I mean, I'm just making this up, but that's I mean, that's, yeah, it is early days, for sure. Yep, yep. Yeah.

Jonathan: Alright, so there is something else I wanted to make sure and get to before we let everybody go, because we've been talking almost an hour, imagine that. So I have, I have gotten, I've gotten excited about different boards multiple times.

I mentioned I started with the BeagleBone I, for a while I spent some time trying to get the what were they called? The Utilite boards, I think was the name of them. This has been, this has been years ago, like 10 years back. But I heard

Igor: about, but I don't recall what is the, the, yeah,

Jonathan: they, they were Well, they, they were Vivante was the GPU on them.

And I saw it.

Igor: IMX, IMX. Yeah, they're IMX

Jonathan: boards. And there's been several times that I've seen boards and it's like, Oh, this is really cool. And then, you know, you'll get a post from somebody like Fedora saying, Hey, we support this now. And it's like, yes a mainline distribution supports it. I'm going to go buy one of these and use it for, you know, what have you, a set top box.

All kinds of, all kinds of cool stuff. Plans that I've had in the past that I want to use one of these little arm boards for because they're tiny and they'll fit anywhere and Almost without fail you buy the board you get it in you follow the instructions from Fedora or Ubuntu or whoever it is and Either it just fails to work completely Or you're stuck forever on a terribly old kernel, or they forgot to mention that they have no HDMI output, or there's always been something that's broken about it.

And I have just sort of given up and just started using Raspberry Pis for everything with the Raspberry Pi OS, because my experience there has been that stuff just works the way that it's supposed to on the Raspberry Pi. Yeah. We,

Igor: we try to, we try to divide Let's say hardware did all that will all that will work almost in almost always.

So we call this like a standard support and everything else is like. Uncharted territory, because if we don't have a maintainer that will take care of this, that when kernel changes, he needs to check if board still works. Otherwise it is possible that you will have Exactly the same experience which you just described.

Speaker 3: Mm-Hmm.

Igor: described that soic will not work. You will need to push it manually, you'll need to hack it together, that it'll boot up eventually, or, or there will be some errors. The kernel will crash or something like that. So we, we actually narrow this all giant 200, 300 boards down to, I don't know, 50 or, or even less, which we keep an eye on, keep an eye on, on it.

And there. You should be pretty, the experience should be pretty solid. So like download, flash, burn, run.

Speaker 6: Yeah,

Ricardo: and the funny thing is if you get any of these, let's call XYZ boards. So if you type into Google XYZ, HDMI doesn't work. I bet the first result will be Armbian with some solution, right? Uh, on the forum or, or, or, or on issue on GitHub or, or this stuff.

Yeah. We have support for boards that have been abandoned completely by the vendor. I'm not going to name them. They don't

Speaker 3: exist. They simply

Ricardo: gave, gave up. They, they are still printing those boards. They are still selling them, but haven't made a single software release for them in years. So we are actually the, the premier supplier of the software running there.

We never got a single cent of the donation from them or any kind of support or engineering support or anything, right? So it's, it's really a community effort at that point. Like Igor said, there are some boards that we are a bit more confident about. That they can run that they can take misuse.

And you're not gonna break them by flashing to the wrong thing that kind of stuff. And then there's unfortunately, a great mask. of those boards that, I gotta say the truth, you want to play it's a great way to learn. It's really nice, you get one of those boards, you go into CloneArmbian, see it building, understand how, what it is doing, what patches are being applied, it's a great way to learn.

So I get this more and more people that are know enough about kernel developments but don't want to know about the bootloader that they can go a little bit into the kernel without having to handle the bootloader by using a framework like Armian, right?

Speaker 3: Yeah.

Ricardo: And so we see this across all the areas, especially in user space.

Oh, Armian doesn't build user space, we use Debian user space mostly, like glibc and everything. down from that but for some of those silicon vendors, they have specifics like video encoders. So then it gets into Chrome and video for Linux and, and, and OBS and and this kind of stuff. So it has everyone who has an interest in, in developing this has really a lot of leeway to, to, to go in and learn.

It's really attracts this, this kind of stuff. If you want to be trouble free. Completely. Then, yeah, I stick with a vendor that only works with one silicon provider for all their boards, if I'm not mistaken. Broadcom and Raspberry Pi, they're the same thing, essentially. So yeah if you like your boards to boot up the GPU first and then activate an ARM core later, yeah, it's, it's great.

But yeah I, I, I hope they move away from this model eventually, right? Let's see, Broadcom is also not a company that's very well liked these days. So let's see how that goes. Yeah.

Jonathan: Okay. So question from the chat room, mashed potatoes, his last question thought he wants to know, how does Armbian differ from Raspberry Pi OS?

Oh, completely,

Ricardo: completely, completely. In the end it's the same, right? So Raspberry Pi gives you an image, which is based on Debian. I think it's currently based on Bookworm. But they rebuild their user space to adapt to that. They insert a lot of utilities, which don't belong in a normal Linux system. So if you look at that boot partition, it's a fact.

32 MS DOS partition with a config. txt and whatever CM line things. So that's all adapting to their proprietary bootloader, which is out of reach for most of us, right? So they don't, we don't have the source for that. So they do adapt that, they do have their own utilities for tuning things. And most importantly, their kernel, the big thing about the Raspberry Pi Foundation is the and the, and the, and the software they produce is that I don't know much, but if you, if you take a well known hat, I don't know, is it a shield or hat, whatever, hat and you take the most famous hat you will get support for that hat directly into that image.

Right, so there's a kernel device tree overlays already baked into that. It's there's already an option that you can set somewhere that enables that thing. That

Igor: experience

Ricardo: can only be curated if you really have a very small set of silicon to support it against. Right, and in the case of a Raspberry Pi, you can do with essentially the same kernel run on all the Raspberry Pi boards.

So in, in, in essence first, I think the Raspberry Pi Foundation has put a lot more engineering efforts, right, into the software than any of the other boards suppliers, by far, right? They, they produce not only goods and well integrated, well tested stuff there, but they also produce documentation and training materials.

And then how to use and the forum is active. So it's, it's really if you don't like diversity, it's, it's ideal there, right? It is for us. One, we have no, no corporate sponsor. We are a community efforts. We have a 200 X more different boards to, to support. So you can imagine that the level of Polish and refinements.

It's not the same. So you can go and get your hats going with any of those boards, because in the end, they're just I squared C, SPI, UARTs or SDIOs or that kind of stuff. So you can make it work, but it's probably not pre made for you and ready to go. So it's a great learning opportunity, but if you really want something that's pre made and ready to go, then it's it might not be ideal for you.

Yeah, but if things just

Igor: work, it's not fun. That's

Ricardo: that's also It's, there's a lot of tinkering, right? Yeah, yeah. To tinker with those things is it's fun sometimes. It's,

Igor: Yeah, it's like going with preset Lego is the same. You have a plan how to put things together and you have an aeroplane running.

But if you have just an open box of Lego set and you need to assemble something, it's more challenging and more fun because you need to really think wider. This is more, if you look on this. diverse ARM hardware. It's, it's, it's weird, weird boards, weird way of booting. But the end, if you add little, if you play a little, you learn a lot.

Jonathan: Yeah, yeah. Okay, so I want to, I want to ask some wrap up questions, because again, we're, we're past the bottom of the hour. It's been a great discussion. So quickly, if, if somebody wants to get started with Armbian, learn more about it, what is the what's the, the best place to go to?

Igor: Fix some board that it's not working well. Let's say that would be probably the easiest way. I encourage people when they come and they said, Oh, this board, which we don't, let's say, support officially, but we have it there on the download page. It's there. We don't know. It builds. Sometimes it works, but it's not official support status, and they complain it's not working.

Try to get it working. So you have tools. We will help you and and sometimes people really put an effort Just recently some guy bring up one really cool hardware Helios 64 it was and it is a NAS. It's like a NAS case And rock chip based hardware, but it was like the company went out of the business.

We couldn't it was a lot of It was immature software support, so we couldn't, let's say, bring it up and pay the bill, so it was, and we said, we cannot continue, so.

Ricardo: But the community, yeah, I would say go to github. com slash army and slash builds. Yes. That's where you want to be that's where we all live.

There's also documentation REPL, there's firmware REPL and other stuff, but find the build REPL get, get building get a Linux machine, try it out. You can build it for x86 if you don't have a board. You can build cloud images, you can build minimal images, you can build on, on ARM for x86. You can build on RISC V forearm.

So it's really interesting to get, get into the build system. So this is what attracts, I think, most people. They, they, there's a learning curve, of course, but it's, it's much easier than if you start with an embedded appliance system, like Yocto or, or, or, or Buildroot or, or others. So it's, it's quite friendly.

We try to be friendly to users who are just. trying this stuff out. And sometimes these users become developers. It's, it's really fun.

Jonathan: Is there a, is there a single board that is kind of the premier supported? Is there one that you would suggest somebody starts with? Somebody says, I want to run Armbian on something Arm.

What's the board you recommend?

Igor: Rockchip latest probably would be because there is a lot of people around. There's

Ricardo: a lot of interest on the Rockchip 3588. So those are the

Igor: but the, but there is still, let's say a lot of things that doesn't work well. Perhaps previous generation would be safer.

Safer start. So previous logic 3, 5, 6, 6 or something like that. Yeah. The 35 60

Ricardo: eights are working excellently excellent. They're great routers. You can get boards that have two five gigabit in, thats two of them. NVME. SPI flash and our boards, they're costing about 60, 50, 70 bucks. So they're, they're pretty cool.

There are four core, a, a 55, so they're not super fast. If you want to go to something, there is more cutting edge than get the, yeah, the rock ship, 35, 80 eights. Those are eight cores. Four of them are fast day 76. I think they go up to 32 gig gigs of Ram. So you've got, there's people doing Homelab clusters, Kubernetes clusters on, on this stuff.

There's a lot of interest into this, so the kernel mailing list is bubbling with, with stuff about this stuff. So it's really cool. It is, of course not gonna be perfect but you're gonna have a lot of fun. And, and they're actually usable application nodes, right? So.

Jonathan: I've got the Turing Pi, the Turing Pi 2 carrier with some Turing RK1, which is based on the 3588.

I've got them here right now, and because there is a bug in the kernel, when you use it with NVMe, I am trying to compile my own kernel. And Partway through the compile process, it is repeatedly crashing the board. So that's what I'm fighting right now. Get some cooling and get some good power. That's the first always stuff.

Yeah, I don't know that it's a cooling or a power issue, but we'll see. I hate

Ricardo: to say that, but I've been proven that some, even if you don't think so, try a different power, try different cooling. And do try Armbian. I'm not sure. I you probably heard of Joshua Hyek, who's doing the Ubuntu images, especially for the RK1.

Jonathan: I have, yes, I have actually, I'm in a Discord chat with Josh talking about some of these problems. He's been great. Yeah, exactly. And,

Ricardo: It's very fun because this is when we, we two years ago, we decided to do a disk kernel for this SoC. In a more disconnected from the build framework. So it has its own REPL and this has attracted not only Armian developers into that, but other distros and other projects are, are also using that kernel.

So that Joshua had his own Ubuntu distribution that was much simpler, let's say, than, than Armian. But then we ended up collaborating on this. And I'm going to mispronounce his name. Xuleng Feng which people know as Amazing Fates. There's a lot of really wizard level guys boogie ice cream.

There, there's some of those guys who are already a legendary level and they're a level of expertise into those things, both in the vendor and the mainline kernel. So. Yeah, it's fun all the way down.

Jonathan: Yep. Excellent. We are, we're absolutely out of time. And so I am just going to jump straight to the end.

I'm going to ask each of you, our, our famous final two questions. We will we'll, we'll start with Igor and then we will go to Ricardo. So Igor, I'm going to ask, what is your favorite text editor and scripting language?

Igor: Scripting language, Bash. The text editor, Joe, Joe.

Jonathan: Hmm.

Igor: Okay. Or n or Nano or Nano.

So Joe or Nano? It depends.

Jonathan: Gotcha. I, I tend to,

Igor: but I, I, I use, I used VI as well, so I'm handy with vi a, a, a real,

Jonathan: a real Renaissance man there. Where it comes to text editors. Ricardo, same question.

Ricardo: I'm forced to answer scripting them bash. Mm-Hmm. Of course. The Arman written in Dash if I could say j, I would

And I do use Intelli J Brains IDs for literally everything.

Jonathan: Oh, okay. I have spent some time inside of well, Android studio is really where I got to use it, but that is, that is a reskin JetBrains. So just the same. Yeah. Same stuff. Yep. Yep. Excellent. All right. Thank you guys so much for being here.

Was a lot of fun and the hour literally flew by and I've, we will have to have you back in a few months and talk, chat more and chat about what has changed. So thank you. Thank you both for being here. Thank you. Thank you for having

Ricardo: us. Yep.

Jonathan: All right. Thank you. Yes. Yes. Doc. What'd you think?

Doc: That was good.

That was really good. And I was gonna say, it's a, it was a very fast hour and 15 minutes actually. Very much so. You know interesting stuff, interesting stuff. You didn't ask about the weirdest use that they had. You sort of did. You kind of did. I sort of did. You stole my thunder. I sort of did. Yeah, I know.

I'm sorry.

Jonathan: We also ran out of time though. We'll get it next time. I'm sure, I'm sure they've got some fun stories. I don't know, last week we asked about that, and the guy's like, Well, the most weird one I really can't talk about on the air, but. Maybe they have some of those stories too.

Doc: Yeah, like the one where the guy got killed, we didn't talk about that one.

Well, no, it wasn't quite

Jonathan: that, but anyway. Alright, Doc, you have anything you want to plug before we let you go?

Doc: Oh, yeah So look up KWAI, KWAAI dot AI, I think it is. It's a, it's a open source personal AI. Yeah, KWAAI. AI. I, I mean, it's, it's this open source effort. It's all voluntary. It's a nonprofit KWAI itself is a nonprofit.

It's based on a South African word. And they have some hackathons coming up. I am the Chief Intention officer of that effort. Oh, fun. They named me that because I wrote this book called The Intention Economy, which in part it inspired it. It's a voluntary position. I mean, nobody's getting paid right now on this thing, but it's but it's fun.

And I, I, I really love to see people get interested in it. Yeah. So open source, personal ai.

Jonathan: Yeah. You gotta, you gotta hook us up for an interview. I, I tried to reach out to, we do. Do that. I tried to reach out on one of the social media networks, the, the, the, the one Microsoft owns, which ? Well, there's, I know the business one, I forget what it's called.

The Business one, which I, I, the busiest one there, there is. Anyway, so I, I tried to reach out and haven't heard back, but if I've got the, ill, I'll, I'll, I'll

Doc: do the reaching and I may do it in 10 minutes when I'm on a call. , there you go. With them. You know, I, I'd like to get their top tech guy on there.

Oh yeah. to talk about it. Yeah, that'd be great. And, and it may be early in some ways because it's so such a work in progress right now. But it would be, we, we should do it, but we'll do it. Make it happen.

Jonathan: All right. Very good. So coming up for the show, we have next week is Amber, actually, which it's, it's interesting that both of our guests today said that Bash was their favorite scripting language.

Amber is a sort of a more structured scripting language that then compiles down to Bash code. It's really fascinating stuff. I'm looking forward to that. And so that is that is next week. And starting Starting in July, so starting next week, just so you know, we're going to be doing the recordings on Tuesday.

We're going to give that a try for a month, and if it works well, we're going to move to Tuesdays, because that's going to make, that's going to take what I have, about 6 hours to get done, and give me 30 hours to get it done. And so my stress levels are going to go down if we can record on Tuesdays. It's going to be great.

If you want to find me, my work there is of course everything on Hackaday. We've got the security column goes live on Friday mornings. And we are now bringing the show with some video on YouTube. So you can find the Floss Weekly YouTube channel. Now if you search for that on YouTube, you're going to find two different things.

You're going to find the Untitled Linux Show. Which is my show over on Twit, because we sort of inherited the Floss Weekly YouTube channel from them. And then there's the actual Floss Weekly YouTube channel, which is just this show. So go, go subscribe to both of those. That's what you should do. All right.

Thank you everybody for being here, those that caught us live, and those that'll get us on the download. We sure appreciate it, and be back next week on Tuesday for Floss Weekly.