Episode 775 transcript

FLOSS-775

Jonathan: This is Floss Weekly, episode 775, recorded Wednesday, March 20th. Meshtastic Central.

Hey, this week we're talking with Ben Meadows and AJ McWilkin about Meshtastic. There's been a lot that's changed since the last time we've had them on. There's been a Rust developer that's added to the team. There's been some new privacy features. There's a web flasher that's really easy to use. Oh, and they convinced me to join as one of their developers.

You don't want to miss it. So stay tuned.

Hey everybody. It's time for Floss Weekly. That's a show about free Libre and open source software. I'm your host, Jonathan Bennett. And today we've got Rob Campbell with us. Hey Rob, where did you come from?

Oh,

Rob: well, just been hanging out with you at the Untitled Linux show, but.

Jonathan: Yeah, so today is today was sort of a scramble day we had We had a guest canceled not too very long ago, and all of my co hosts, regular co hosts are still traveling after scale, you know, that big open source conference I'd hoped to bring a couple of them on to talk about it.

And in the emails over the past couple of days, everybody kept saying, we're still on the road. I'm going to be in the air. I have a doctor's appointment that day. It's like, okay, fine. We'll put the call out. Anybody can come. That's how Rob got here. Anybody can come podcast with me. I'm so special. No, Rob, Rob just recently, Rob just recently joined the the cool group because he got the new the new microphone.

It sounds great. So that was a

Rob: requirement.

Jonathan: That's the requirement. That's really the requirement. We like to have good audio. So many people listen to the show. So today Rob, we've got a couple of devs from the Mesh tastic project. And this is not something that you knew a whole lot about and that's sort of on purpose because I We've talked to Mesh tastic on Floss Weekly before and ever since then I have been addicted to it.

and buying devices. And then I started writing code for the project and it eventually got to the point to where they gave me the developer badge and said, you're one of us now. And here, let us start sending you hardware to test with. So full disclosure, I'm sort of on the inside on this one and I have gotten hardware from these guys.

So just. Know that. And Rob is sort of here as the audience proxy. The one to ask, I have no idea what you're talking about, please explain that thing, that developer y thing that you just said.

Rob: Yeah, I've heard you talk about it for some time now, off and on, but I had not gotten around to looking into it until

Jonathan: yesterday.

So our goal for the show today is to convince Rob to go and order some hardware. There we go. All right. So our two guests today are Ben Meadows, Meadows, Meadows. I have to ask him how to pronounce that. And he is a full stack C sharp dot net and view JS web developer, but we love him anyway. And he is sort of the lead dev, the guy on the top of the mesh tastic stack.

And then we also have. Adam McQuilkin, I may call him AJ because his username in discord and GitHub is AJ McQuilkin. And he is a computer science and engineering degree holder from Dartmouth. He holds a patent in USB hardware, HID design. And he is, let's see, a certified wilderness first responder working with SAR port 10.

part time that's Search and Rescue, which is interesting. We may talk about that too. And he is one of our developers, and he's the Rust guy on the project, and that's interesting to dive into as well. We talked last week about Rust, and why everybody should use Rust, and maybe this week we'll talk about what it looks like when a, a young, talented, excited Rust developer wants to join your C project, because that can be interesting.

Let's go ahead and bring the guys on. Ben and AJ, welcome to the show. Great to be here. Thank you so much for having us. Always a pleasure. Yes. Thank you guys for agreeing to be here at the last moment. It was literally yesterday and I went into our shared chat channel and said, guys, we have less than 24 hours till the show.

I need somebody.

AJ: We need anyone bottom of the barrel. Just please come on

Ben: whenever you throw up the bat signal. I'm there. Yep.

Jonathan: Yep. And I appreciate it. So let's start with Ben and let's, because we're on sort of a different platform now being in Hackaday let's start real quick with that 30, 000 foot view, what is MeshTastic and why would somebody be interested in it?

Ben: Yeah. So the kind of the elevator elevator pitch of MeshTastic is. It runs on a largely like Arduino code base on little low power devices that have LoRa radios. And the idea is that you or at least the typical path for folks is that they go and download the Meshtastic app and they pair to these devices and send text messages and position, location, information, telemetry messages, any, any kind of small things that you can fit into small packets.

Meshtastic is capable of sending that over the LoRa protocol. And we have our own kind of spin on, on LoRa. We use LoRa peer to peer protocol to send those messages over the mesh network formed. By by these little devices. So that's kind of the elevator pitch. I like to say is off grid texting

Jonathan: so let's let's get out of the way real quick a couple of questions that we see all the time And first off is this lorawan?

And second is this a replacement for the internet? No,

Ben: and no You know the the lorawan question almost I would say comes up daily in our discord people are like Hey, how do I get this on to lorawan and and there's really You know, the, the, the first part of that, of those you know, Laura peer to peer and Laura Wann, they both share the same CHIRP protocol of Laura, but Laura Wann is very much like a more structured multi channel based communication, whereas Meshtastic using peer to peer is very ad hoc large packets.

They're fundamentally incompatible. There's, There's some shared hardware, which makes it really interesting. And I think that probably adds some confusion to the matter. But even the, the large nature of, of the lure of peer to peer packet size we're still talking about 256 byte packets.

So these are, these are very small payloads that we're sending. This is not a, not a replacement for the internet, unfortunately dashing some people's hopes and dreams.

Jonathan: You know, we barely have

AJ: enough space to send the IP header, not much less any video streaming, audio streaming, anything like that. So, unfortunately, yeah, keep having to

Jonathan: say no to that one.

There is a project where you can you can tunnel IP through Meshtastic, but it comes with a whole bunch of, there's a whole bunch of asterisks on that one. You know, like a bunch of caveats. Look, you've got to be doing this on the fastest settings. Please don't do this on the public channel. And You're on your own.

Good luck.

Ben: Yeah, I think, I think there's some interesting cases there for like SSH sessions over, over the highest bandwidth channel, but it's. It's not what I would call a great experience for the most part.

Jonathan: Yeah, so we do have a question and this this question actually came in even before we started the show.

It's David Ruggles, another, another friend of ours. He says, I'm excited to hear from Eshtastic today. I want to start playing around with this. My first question is how do we get started? He says he's played in both the Raspberry Pi and the Arduino ecosystems. He normally starts with a breadboard and wires.

And he's seen the Rack Meshtastic Starter Kit referenced in the docs, but he says it looks like it needs soldering to get started. Is that true, and is it the best place to start?

AJ: And do you want to take that one? I have some of the hardware, I can be like the showcase if you want.

Ben: Yeah, the Rack Meshtastic Starter Kit is a really great Great one to get started with.

Yeah, I just got one right there. So there shouldn't necessarily be any soldering required unless you want to add a screen, but you don't have to have a screen within Meshtastic. Plenty of people run, run nodes headless and just keep everything in the app. The screen is more for diagnostic usage unless you're trying to use the device as a, as a standalone.

A standalone MeshTastic device, which requires a lot more buttons and more, more involved stuff. But no, you shouldn't need, you just need a battery if you want to run it portably. And that's, that's a great one that I would recommend getting started with. And we have a lot of documents in our MeshTastic.

org that kind of have some instructional stuff in terms of getting started and recommended hardware. But that one, that one's the number one for me, the, the rack.

Jonathan: Yeah, and then what about Raspberry Pi? Is there any option for using that?

Ben: Yes, I'm glad you asked,

Jonathan: Jonathan. This is a very inside, very inside baseball question.

Go

AJ: ahead. Yeah, I was going to say, I was going to divert to, usually I divert to you, actually, for that one. You are the one who knows

Ben: about it. One of our promising new developers, Jonathan Bennett, has has, has worked very hard on Linux native support for, for Meshtastic. So you know, we have Raspberry Pi, I think, is the most compelling target because it's, it kind of aligned with the whole.

low power portable operation. Not that the Raspberry Pi is, is equivalent to some of the Arduino stuff in terms of power usage. It's obviously because it's a full blown Linux SBC, it's going to be a little bit more power hungry, but it, you know, there's, there's a, there's a lot of opportunity there in the, in the future for like base stations and stuff, but the key, the key with the the Raspberry Pi and the, and the Linux native in general.

Is making sure that your Laura radio that you're adding on to that has SPI bus exposed. Yes. One of the things that we see almost all like every day is some folks coming in to the discord asking about

Jonathan: that one particular wave share

Ben: hat. Yeah. It's always, it's always that wave, the one wave share board that has you are, and, and we just can't support.

Those, those types of of hardware. We have to use SPI to talk directly to the LoRa radio because we have such a a tight communication with, with the LoRa radio using a library called RadioLib that's kind of underpins all of our transport level stuff. So that's, that's kind of the, the The basics of the Raspberry Pi is you need, you need header, your your Raspberry Pi with headers and your, your Laura radio hat.

And you can obviously breadboard your own if you can find a compatible bare Laura radio. But I always recommend that folks start with the. WaveShare SX 1262 hat that has SPI. Yes.

Jonathan: Yeah. There's, there's some fun stuff going on there. What's that, AJ? Yeah, I was

AJ: going to say, like, so I'm, I usually recommend, like, it depends on where you are.

Like, I think technically, like, it sounds like if you're doing Raspberry Pi and you're ready for soldering, then I would agree with Ben. I think you'd probably want to go the route of going maybe a Raspberry Pi or the rack starter kit. But I guess for non technical people, like I started this project and I wasn't.

Like super technical about all of this. I started with one of the TVMs, right? These are like pre made boards. You can get these, you stick a battery in and you can immediately get started and go. So I guess I'd caveat that with like, if your goal for this project is to just chat with people, then. The, the racks can sometimes be a bit of a pain depending on how much setup you want to do, but if you want something to just get started in going, we have Etsy shops.

We have, these are available on wherever Amazon, Alibaba, all of that. So I guess that's like a small caveat to what Bennett said.

Jonathan: Yeah. And it kind of depends upon what your budget is because you can, you can buy the rack for not very much. You can, you can buy one of the T beams for not very much, or you can, you go to the Etsy shop and buy something like the messenger.

Which that's you could barely see it. It's the little red thing back there with all the buttons on it. And that is It's a pretty impressive device but it costs like I forget exactly how much if you have him build it for you It's like 200 I think and you know when you when you first see that price compared to everything else you go Oh my goodness, that's ridiculous How much that is until you go to build one yourself and you realize One, how many parts go into it, and two, how fiddly it is to put it together.

And it's like, okay, that's, yeah, that's a reasonable price, isn't it?

Ben: Yeah, there's not a whole lot of turnkey devices out there. And so you, you know, right, right now, I think we're at a, we're at a point where there's a lot of, you know, kind of cottage industry makers like, like Tony Troffo that, that put out stuff to, to help people get started.

And then there's a few, a few manufacturers, like the TECO device from the Lego is, is a quote unquote turnkey device. It's got a screen, it's got a GPS it's got a injection molded case.

Jonathan: Yeah. So David asks again, can Meshtastic be used for IOT communication or is it overkill for something like that?

He says, he's looked at LoRan directly and it seems to have a high barrier to entry maybe Meshtastic is either better documented or easier to get into. I would say IoT is one of the things that is very interesting for, for Meshtastic. You want to expand on that?

Ben: I would say it depends on what kind of IOT stuff you're talking about.

Like telemetry is we have a lot of native support for that, particularly for we have a list of sensors on our, on our website that are, that are supported. We, when we launched 2. 0 of MeshTastic, we decided to cut a lot of the kind of fiddly analog based sensors. And. Kind of draw a line in the sand to say it's we've got to have I squared C sensors There's tons of stuff on the on the market that at a fruit and Rack wireless with their wisp block system for their for their starter kit support that can gather like temperature Relative humidity barometric pressure all those sorts of key key device met our environmental metrics and We really wanted to make it easy for folks to just plug and play.

So the, the thing about the I, I squared C device sensors is that they get auto detected on startup. So if you plug it in, if you plug in a supported sensor, it should just connect out of the box and you turn on the environmental telemetry module and it'll start sending those metrics out over the mesh and you can, you can do all kinds of stuff with them at that point.

AJ: Yeah, in comparison to LoRaWAN, I think they're, I mean, like Ben said earlier, they're the same technology underlying. So I guess it depends, like LoRaWAN is dedicated for this and there's a lot of professional support around it. So if you're gonna like deploy something that's like, that needs to be bulletproof and you want like a company to blame for it, if it goes wrong, then I don't know if we're the right way to go, but one of the cool things that I always like to accent when I hear this kind of question is that LoRaWAN is like, we are a mesh.

And so your range in theory can be as far as you want. And so if you need to deploy like kind of a variable, a variable network in like a very mountainous terrain, let's say, and or a very flexible situation, then oftentimes the mesh will serve you very well for that kind of thing. So I guess that's one of the advantages of our platform over lower WAN that I'd highlight too.

Sure.

Rob: So AJ, how did you get started in the project?

AJ: Oh, man. So how did you want to go? I guess. So I, I found MeshTastic. So, okay, so this goes into the start point. So I worked search and rescue in college part time. And so my, the team I was working on was having communication troubles. And so like, I went down a bunch of rabbit holes of like, how can I solve this?

And eventually I found Meshtastic through that. And then I did a Capstone project. I realized that there was not really a way to manage networks at scale. And so my Capstone project in college was to build what's now become the Meshtastic network management client, essentially to figure out a way to manage.

Networks at scale, or at least that's the, the North star of that project is to manage that network at scale. And eventually I, I bugged Garth and Ben enough and just kept poking them. And then here we are a couple, what is it? I think a year and a half later now I think I got started in like early 23 or early late 22, maybe.

Yeah, I don't know. We can, I guess, I don't know how deep you want to go into that side of it, but there's a lot of lore there.

Rob: So what's the status on the management project?

AJ: Oh man. So currently it's mostly works. No. So I think it's the management client is always in a state of churn because it's like there's, I think people, it's been cool to see how excited people are about the management clients.

And I think there's been a lot of. Different ways. I think people want to go with the client. And so right now like right now this week, like working on a, a refactor to make actually Jonathan, you, you know, this too, like to make the connections more reliable and to make sure that it doesn't randomly like timeout serial connections, that kind of thing.

I've been a

Jonathan: guinea pig for that.

AJ: You have, I've just been pinning, pinging him randomly. I was like, Hey, I just pushed a change. Can you fix this? Or can you test this? And then you're like, no, it's still times out. I'm like, okay, I'll go back. Go back and work on it, but yeah, like right now, I think my, my personal goal for it is to get it to a point at which it's completely reliable and it, it just works as a standard client and then to start accenting more of those like network based features, such as like graph analysis or.

Even remote administration so I guess I view those at this point more as like North Stars than actually practically implemented stuff.

Rob: How long do you expect it to take before you get it

Ben: to,

Rob: That kind of status or how

AJ: complete is it? How complete is it? Well, I guess, what do you, what do you want to do with it?

I guess is the, you know, the threshold. Yeah, I mean, I, I don't, I don't know. I think it's, I always hesitate to put like an actual timeline on it just because, you like the unfortunate reality for me is that I'm, I'm doing this as like as a part time thing and I like have a full time software job.

So it's kind of like at the mercy of that, that side of my life. I don't know, I guess it's, it depends on yeah, I guess what you're looking for. Like, I think in terms of it being bulletproof, like I'd love to get there by B3. Like one by the time, whenever best Chestick releases B3, I'd love to have the network management client ship as a fully.

Tested fully reliable client, but I guess I kind of hesitate to put more of like a strict timeline than that.

Ben: One of the things that I think is also worth highlighting about, about kind of the state of flux within the network management client is it's actually some of the features that relies upon or fairly new into the firmware so the, the neighbor info module that we, that we added to kind of support some of the.

Visualization of the, the mesh topology is still pretty new within the firmware. So that we've kind of had to go back and retrofit the firmware with some of these, these features that would even make the things that, that AJ is wanting to do with the, with the management client possible. So we, we're trying to remove some key features.

bottlenecks on our side to make sure that he's got what he needs on, on his end. And that's kind of a, a common theme I would say with Meshtastic is like, everything goes through the firmware. So you have to, You have to have a lot of collaboration and interaction between the client developers and the firmware guys.

Rob: I understand there are other clients. What is the, what's the status of those other clients? It's still a lot of work to do on those.

AJ: I guess, Ben, you want to take that? I can speak only probably to the web and Python

Ben: clients at this point. Yeah. I would say that the most complete experiences within MeshTastic are the, the Android and the iOS app, because those.

Those are kind of, I think, where most folks use MeshTastic is for like the text messaging and the mapping features. So those are pretty feature complete.

Jonathan: They also, they also each have dedicated developers, which I think is one of the things that makes a big difference. And,

Ben: and MeshTastic originally shipped with the Android app.

So that one's kind of, kind of been brought along the longest. And then Garth later on Came along and developed the iOS app. There's also some, some clients in terms of like the Python CLI client for doing some of the, a little bit more advanced features for, for configuring nodes, for instance And that one, we're actually, that's another one that got shipped with the original version of MeshTastic and has not really had a prod, a platform owner maintainer.

I've just kind of been helping it limp along lately. So we, we actually put patches to it, right? Yeah, we actually put out a I put out a GitHub issue the other day to just, Kind of call for like, Hey, somebody want this,

AJ: right? And that's the unfortunate like a lot of like I forget to mention this, but a lot of mechanics clients are like primarily single developer things where.

Like someone takes ownership of it. Like the management client has been kind of my like pet project for a while. And then like each, each client usually has like one person who's kind of that primary. And I think one thing one thing we've been chatting about is like because I'd love contributors to the management client, but I think like one problem that's having is it's, there's a huge barrier to entry on the code base, just cause like fundamentally the stuff we're doing is like somewhat complicated and we're working in like a weird desktop stack, that kind of thing.

But it's kind of like an open question on the project right now of like, how do we first of all, get people excited about actually working on the clients. Cause we were very lucky to have a huge contributor base to the actual firmware which is phenomenal. Like we always love, love that community.

But I guess one thing we're, we're considering now is like, how do we make our clients accessible? How do we get people to feel ownership over the clients? And specifically the. The, the, the, the very top of mind one, like Ben mentioned was the Python is the Python client where we haven't really had that person for a while or

Jonathan: for someone like I'll, I'll jump in real quick.

Sorry, I'll, I'll be one of the guests on this one for just a second. There's also the, the web client that is great. But it's, it's behind a little bit too. There's some things that just, you can't, you can't set there or that don't work there yet. And I don't know that there's anybody that really has made that their project either.

We, we have somebody that recently added, so a developer came and essentially said, well, why doesn't the web client work on the Linux native stuff? It's hard. And nobody has spent the time on it yet. And he's like, Let me see what I can do and came back, you know, a few weeks later with a big patch that made it work And it needed a little tuning up, but it works.

It works really well. I think there's there's one or two little little crash Issues that can occasionally cause a crash. I'm still trying to figure those out But though the wider web client like the actual JavaScript stuff of it needs some love And I'm not sure who's gonna jump in and and do that.

Maybe it's going to be me. I don't know. Hopefully not.

Ben: Maybe we need to put an issue out there on GitHub for that one too. Yeah, we probably should. There's also the there's also some unofficial clients kind of in the community too. I think there's some, some Go code that is capable of communicating with devices.

And then I've got a C sharp based CLI slash library and. It's multi platform, so.

Oh, there's a,

Jonathan: yeah, there's other stuff too. Like I've seen on over on on Reddit, somebody put together a a cross platform client like a really simple one. Well, no, I was going to talk about the LISP one first. Oh, I haven't seen the LISP one. I think he said it was, I'll see if I can find it.

Yeah. He basically, he said, there's no, there's no simplified client yet. So I wrote one and I contacted him. I'm like, dude, this looks really good. Would you be interested in coming on the discord and making this a little bit more official? And he's like, well, maybe, but it's in LISP. I doubt anybody would really want to do anything with it.

Okay, fine. And then there's also another one that's written in, in Flutter and Dart. Randall, when you go to listen to this later, you. Somebody, somebody's using your language. But and that one looks pretty interesting too, but those are, you know, strictly unofficial at this point, but I think, you know, it's, it, for, for one thing, it's pretty exciting to see somebody doing this, like an unofficial client, somebody cares enough to write it, but also it's pretty interesting for the project as a whole, because I think there probably is room to have a unified client that looks the same on Android and iOS and the desktop.

And that, There's some benefit to that too. So interesting stuff. Well,

Ben: and worth mentioning, you know, AJ's net management client started out as, as a unofficial client, but we brought it into the organization. So there is a path for that. Like if, if we see you know, promise there in terms of. of that being a useful thing to pull into the ecosystem.

It doesn't make sense for every project. Like, we probably wouldn't pull in that Flutter app, for instance, because we have our own official apps on iOS and Android, but there's definitely room for, if we identify other things in the community that look like they're kind of aligned with the project as a whole, we might pull those in in the future.

Yeah,

AJ: I guess something I'll, I guess I'll add to that, like from my perspective, like the fact that we're seeing a lot of interest in like a simplified client, I think is interesting to me just because that like from a design side kind of implies that we're too complicated. Right. And so there's this kind of like this issue of what persona retargeting in our actual client applications.

Like, we're kind of assuming with a lot of our configuration with a lot of our the, the, the, the number of knobs that we expose to people. Right. Seems like it's overwhelming. And I guess that that kind of begs the question of like, do, well, there's been discussions of like, do we need all these knobs? Do we, whatever, whatever, but also like, yeah, how do we design this for someone who's both new and someone who's pretty experienced with the project?

But I guess that's, that's kind of something for some visibility that we're discussing internally as well.

Rob: Speaking of someone who's new, someone like me who hasn't gotten into it yet. I could see all kinds of use cases like the sensors and stuff, but mentioning the iOS, Android, the ability

Ben: to text.

Off grid, how, how does the, for example, the iOS

Rob: app interface with the, the Laura?

AJ: Yeah. So essentially I'm speaking a little bit out of my depth on the firmware. I'm mostly client, but essentially the way that works is for example, like the TV, right. It has two, it has two radios in it. Technically one radio would be the lower radio that actually does the long range networking.

But it also has a Bluetooth radio in it. And so essentially the way the mobile clients primarily work, and I think the Android might also support serial. I don't quote me on that, but essentially they'll connect to Bluetooth and then that message will go into the radio, the radio will do whatever they need to do with the messages.

And then if it should, if it decides that's the right way to go, it will then send that out to the mesh, which is only over Laura. I don't know if you have anything to add to that then, but that's like, I think that's the highest level explanation.

Ben: Yeah. Yeah. Bluetooth is typically how we connect to devices from the mobile side of things.

The Android does. as AJ mentioned, it supports serial and Wi Fi connection for, for some of the like ESP 32 targets to have a Wi Fi radio, but Bluetooth is generally the Avenue.

Jonathan: One of the, one of the Outstanding to do issues that I'm aware of is adding Bluetooth support to the native client because you know, people will have their like a pi zero with a with a radio and a screen on it.

It's like this thing is great But how do I talk to it? So

AJ: What

Jonathan: do I do with this thing, you know, it's like, oh well the bluetooth it's not it's not there yet It doesn't work. It's it's on it's on the list. Eventually. We'll eventually get to it

AJ: Well, that's the fun thing is there's a lot on that list That's how these projects work.

Unfortunately, there's, there's always a list and whoever takes it off that list will be the person who, who sees it first, I guess.

Ben: I've

Rob: read on the site, it mentioned a range. I think it was 158 miles. And I don't remember the kilometers. And I believe those were more like. Theoretical maximums are the maximum that somebody has done.

What what's a realistic range? Well,

AJ: one of the cool things about Laura is it's it's essentially line of sight. Right. So the reason that our range is that distance is because that's the furthest people have been able to get line of sight.

Jonathan: There was a balloon test, wasn't it? Oh, I think there was a

AJ: balloon test.

There was also someone who did a shot from Calgary down into Montana. Which is that one blew my, that, that one blew me away when I saw that. It was like, you went, not only went cross border, but you went like significantly cross border. So I guess that's my understanding is that it's primarily just how far you can see.

Cause we don't, we don't diffract that much, but if you can see someone just because of the nature of how Laura works, you can get significantly signal specifically below the noise floor. And so yeah, that's, that's kind of one of the cool things about us is if you can see it, you can likely

Ben: talk to that person.

Yeah, and worth pointing out for that particular one that you mentioned from, from Calgary into Montana, that was just with there, there was no directional antennas or anything that was just little, yeah, just a little omni directional antennas. And that, that one, I believe was 254 kilometers. So really, really impressive.

I mean, it's actually hard past a certain point to overcome the curvature of the earth. So you really have to get like two spots with pretty significant elevation to be able to do something like that. But I always say, you know, a few miles is kind of the, the, the range that most folks would get. Getting one node up highs is really the, the game changer for, for extending range.

If you're I think if you're operating. In a mesh where you have a bunch of people with handheld nodes talking to each other you're, you're going to be fairly disappointed because it's going to be similar to like two way radio ranges. But if you, if you get one node up high, you know, on a drone or set something up on a hill or in a tree it makes a huge difference.

Jonathan: Yeah, so one thing to add real quick there, I don't think we've made extremely clear, is it meshes, which means, you know, if Radio 1 sends a signal out, Radio 2 up in the tree sees it, even if it doesn't, this is important, even if it doesn't have the encryption key to be able to decrypt it, because Meshtastic is all encrypted, It will still repeat that and send it back out.

And so radio three can be, you know, theoretically even over the curvature of the earth. But if you've got one in the middle, that's up high enough, you can bounce off of that and get even further. So, you know, if, if you want to spend the time to put your nodes exactly where they need to be, you can really have a lot of range on this.

So there, there's, there's options. It's flexible.

AJ: So with that, there's been a lot of, I'm sorry. Go for it.

Rob: I was just gonna gonna say, so with that meshing yeah. How many, how many notes can you mesh together? What's the scale we could really

AJ: look at? ? This is the eternal question. We don't, we don't have hard numbers.

I mean, we have like theoretical numbers and we have like, we have the levels at which networks start failing, but that's like, my understanding is that somewhat still somewhat of an open question of like exactly how many nodes we can support. Yeah. That depends a lot on like your hop. You know, you're so we have, we have a parameter like compliment, right?

Where it's the number of bounces, essentially a packet can make before the client just stops or a given node will just not repeat that packet anymore. And so it depends a lot on that, but I don't know, Ben, what's the number? Like a hundred something is I think the most we've had like

Ben: sustainably.

Yeah, there's, there's the theoretical numbers are, I think a little bit over 300. Yeah. Is, is the theoretical and we've definitely had some, some meshes that approach that size. We, a thing that we've actually been running into a lot more lately is larger meshes as we've introduced features like our MQTT.

So that's, that's a a message broker that you can enable a connection to to basically do internet backhaul to your, to your mesh. So you can kind of glue different, different mesh networks together. And that has created some absolutely massive regional meshes. The U. S. one, I don't know how many there are.

nodes are in that, but you, you pretty quickly, yeah, you pretty quickly run into limitations of what, what the little these little Arduino boards can, can handle in terms of, of RAM usage. So we've actually had to make some changes like rolling the oldest node off of, off of the internal database to make room for new nodes coming in.

And so as you get kind of closer to these theoretical limits, we have this. This sort of rolling ring buffer of nodes coming coming through the device so it's it's definitely presented a lot of technical challenges, but we don't have like a We always say that that maximum number of nodes in a mesh is kind of a fuzzy number because it depends a lot on on your configuration what hardware you're working with and How chatty everything is.

It's it's kind of a sliding scale.

Jonathan: Yeah. One, one thing to add in there with that, with my developer hat still on it depends upon how you have your nodes set up. So if you've got somebody that is sending a location packet every 10 seconds, you're not going to be able to get as many nodes on the network because you're saturating the airtimes.

Whereas if you have all of those nodes set up, you know, kind of carefully to behave well. On your mesh, you can get a lot more, you know, you have a lot more room, a lot more headroom to grow. So it kind of depends, as you said, it depends on a lot of things.

Rob: I was looking up when I was looking into this, I was seeing some different discussions about how long it takes, say you're just sending a regular, you know, Text message to somebody.

How long it takes to travel. I assume if you're just doing one hop, it's fairly quickly, but if you have multiple meshes, does that slow it down, or what are we looking at there?

Jonathan: Yeah, Ben, do you want to take that one?

Ben: Yeah, it's, you know, for, for point to point delivery, it's usually less than a second in, in most cases on the default, but then you obviously get some additional time to rebroadcast, and we actually have some some weighted rebroadcast delays that that make sure that there is not a as much of a contention window so that nodes don't sort of rebroadcast over each other and, and and cause packet loss.

So it could take a few seconds to, for, for that delivery to happen as you increase the number of, of hops involved to, to actually deliver that message if you've got, If you don't have direct line of sight to that particular node you're communicating with.

Jonathan: Because all of, all of your devices that can talk to each other, they're going to be on the same frequency.

And on that frequency, in a, in a given region, You can basically only have one of them talking on the airwaves at the second. It's a, it's a shared ether the way the old school ethernet used to work. And there's not a, there's not a clear path to go, to go from a shared ether to the way ethernet works now.

There's been a few ideas thrown around about how to do this in some very complicated ways. But for now, it's all, it's all shared airwaves. So, you know, only one can talk at a time. And that does limit how quickly things can propagate. Cause every, every, all the devices sort of have to take their turn.

All right. I'm going to take my developer hat off and I'm going to put my my web, my podcast hat back on. And I'm going to ask some questions again. We had another one from the chat room. And that is. Is the system store and forward or is it real time communication only and basically when an app connects could it see historical messages or only messages sent while it's connected?

And there's really, he didn't mean to, but he's asking two very different questions here. Let's talk first about what happens when you connect an app to a radio that's been running for a while.

Ben: Yeah, there's we have a concept of a we call it the two phone queue and it's basically a prioritized queue of messages that have not been delivered yet to the phone.

So those, those queue up as you receive text messages over the mesh. But there's not really a whole lot of storage there because again, we're talking small Arduino devices with not a whole lot of, of memory. So the, you know, we always say like there could be a few messages stored on the device and then once you connect, those will get delivered to, to the phone clients.

But it, I wouldn't count on, on guaranteed delivery for those if you're not connected. And, and the other question about the store and forward versus real time it's in, in most cases it's real time, but we do have. a new store and forward Module that you can enable on the firmware and that is specifically for router And router client role node.

So that's a configuration that you have to enable for a particular node and it's also limited right now to the ESP32 devices that have a external PSRAM module. And the reason for that is, We don't have a whole lot of memory to work with, so keeping those kind of a, a ledger of those messages that have been observed over the mesh takes up quite a bit of memory.

So, so we have to, we have to use external RAM basically to, to store those. For now, we've, we've talked, and I think there's an issue on GitHub to talk about the potential of maybe storing those on the file system instead. Even like SD card, flash memory. And I think Jonathan, you, you know, the, the Linux native, right.

That should be, that should absolutely be, be something we, we have plenty of resources there. Yeah. So but those are, those are, you know, features that you can unlock, but they're not necessarily native. To, to the, to the mesh if you're setting up things on the defaults.

Jonathan: Yeah, one of the, one of the tricky things about this is, you know, there's a, there's a bunch of schemes that you could dream up and add to it, but we're so bandwidth restricted.

You want everything to be simple and basically have no no control message overhead or as little as possible. So, you know, you could, you could imagine some scheme where. You know, you store a whole bunch of messages and then each client sends back and asks for them one at a time, and that way you would know that you hadn't missed any, but it just, it overwhelms the mesh because the bandwidth on this to be able to get that kind of range.

The bandwidth is so low. I've been, I've been noodling for a while on a sort of an inspired by git sort of scheme to do this. But again, you run into issues where you just don't have the bandwidth to send all of the control messages you need to, to make sure that you don't miss anything in the middle.

And it's just, it's a complicated problem. And the store and forward module, I think is really one of the nice solutions to try to figure that out. So, let's chat for a minute, and AJ, I'm asking this sort of tongue in cheek, I think you'll understand what I'm getting at here, but I'm going to ask Ben, and then I'll ask AJ, sort of the same question, maybe the opposite side of the coin.

Because we had, we had a Rust developer on last week we had Wolverson on last week, he's one of the guys that's written books about Rust, and AJ, He really kind of made the case that Rust is a really cool language and we should all try it out, which it is definitely on my to do list. But I'm curious, what does it look like when you've got an established C or C or any other language really, but C in this case, an established project and an excited young developer comes in and says, Hey, not only should you have written this in Rust, I want to rewrite some of this in Rust.

And your C developers. Kind of go, surely Rust can't be that hard, right? And then you had the experience I did where you go and you look at some Rust code and it's like, I have no idea what this is doing. So I'm going to ask Ben first, how do you successfully navigate this when you have the young, excited Rust developer come to your project?

Ben: I think with any, with anything like this, you know, you give people an opportunity to like, hey, what can we do with, with this new technology? Can you show us some, Some examples of how we, how we might integrate it. And I'll, I'll just go ahead and say, I'm not a big C, C, C plus plus guy. I, I you know, I hadn't really touched C or C plus plus since college.

But until I got back into kind of Arduino ecosystem and, and started playing with mesh tastic. So I'm not super opinionated about. about like idiomatic C So there was, there was never any, any any presumption on my part of, about whether we should do things with one or the other. So I'm probably the wrong person to ask on that front, but I know there, there's definitely some there's some opinionated you know ideas about how things should be structured with like idiomatic C versus, versus Rust.

I personally like what I've seen of Rust with the limited amount that I've played with it. I think I've done a tiny bit of, a tiny bit of code in the, in the

AJ: You built the flasher, right? You built like a prototype flasher. Yeah. Yeah, a little

Jonathan: bit. That was cool. Yeah. I want to talk about the flasher here in a minute, but I think it's an interesting topic to chase down.

So, AJ, let's, let's ask sort of the opposite side of the coin to you. How does a young, excited Rust programmer come to a project that is full of C code and Not get yelled at and told to get off my lawn You know, how do you how do you approach that in a in a way that's that's likely to succeed

AJ: Well, so I guess it depends on what you mean by succeed and I think this is kind of the journey i've had to go through right of like I think

Jonathan: No, right.

It's like I think like you have to change your

AJ: definition of success, right? because because fundamentally like what what makes meshtastic cool isn't the language it's written in it's the community that's around it, right, right And so I think one thing that I've been thinking a lot about is you want to kind of push the virtues of Rust in this case specifically without alienating these very important members of our community.

These people are very valuable to our community. So I think, I think the client has been a good way to go because the management client, the infrastructure is written in Rust. And I think. That's been a good way to kind of get a sense of what the community's kind of taste for this language is, is because there's, if there were to be a migration in however long.

Fundamentally, there's like two steps to that. One is people would have to learn the language and two people would then have to rewrite the firmware in that new language that they've learned. And so I guess I'm trying like kind of doing this incrementally kind of feels out that first step of like, is this even a language people are interested in learning?

And if not, then I guess I don't, I don't want to be the person who breaks the community by pushing rust at all costs, frankly, right? I don't know. The community is worth more than the specific language it's written. And I guess it's kind of my current my current stance on this. But I don't know, there definitely has been a bit of the, I know Such as another one of the people who's like very pro Rust does that have you worked with him at all?

Like he's, he's the, okay. Yeah. So he's the, he originally wrote the web client and so he's, he's also been pretty pro Rust. And we've been like echo chambering each other a fair bit, but at this point, I think it's I'd be partially waiting for the embedded Rust ecosystem to mature a little bit as well.

Is one of the things is. There's a cool framework called embassy, and I think it's like 95 percent of the way to the point at which we'd be able to push like a prototype, but I don't think it's, I think we'd want to wait until it's like closer to a hundred. So I guess, yeah, the ecosystem, you don't want to push people into an ecosystem one that's like not completely ready and two, yeah, that kind of like.

You almost have to like dip people's toes into it. I think before you actually like throw the bucket over their head of like, here's the new language, here's the new framework. Here's just write this now kind of thing. Yeah.

Jonathan: Yeah, that's perfect. That that's very much sort of the flavor I was, I was thinking of.

So let's, we can talk about that more, but we are, we are getting really close to out of time actually. And I know there's at least one more new thing. And I think this is, this is one of Ben's things. The, the web flasher is fairly new and actually really pretty cool. I, I used it for the first time just the other day to test something out and it was like, Oh, okay, this is, this is really neat.

So. Give us the give us the rundown on that, Ben.

Ben: Yeah, we had a we had kind of an unofficial community web flasher that that Thomas for, for those of you on, on discord and, and GitHub, he had actually hosted that I think like in his basement for the longest time. And it was really neat because it utilized ESP homes.

I think it's called ESP web tools, which actually uses the some Chrome chromium based browsers ability to use web serial. to actually flash ESP 32 devices. And it, it wasn't a permanent solution. It worked, it worked pretty well for what it was, but we wanted to bring a, a similar solution to to the official mesh tastic projects.

And also incorporate some of our other platforms like we have Pico, the RP 2040 support, and we also have NRF 52 from Nordic boards as well, which is what Rack Wireless runs on. And those, those flash in fundamentally different ways. So we wanted to kind of provide a little bit more of an interactive approach to to flashing those boards and kind of walk through walk people through the steps Because a lot of it has been confusing and unfortunately,

Jonathan: you have to you have to hold this button while you plug it in Okay, that's probably long enough now.

Let go of the button look for a drive that showed up. Yes

Ben: And telling people to go through the, and use the device install scripts in the, in the zip the firmwares is never fun. You know, people look at you like you have, you have five eyes, but but we we, so we brought the the new web flasher on online and it uses the official expressive port of ESP.

tool to JS and it works, works pretty well. We've also got like a serial monitor that I added recently. So folks can kind of debug stuff without having to download like the Python client or a, or a serial terminal application like putty or, or TO. So yeah, it's, it does a lot of things.

Jonathan: All this catering to windows peoples.

Come on.

Well, that's that's fun though. So what else is new? Is there anything that's new that we that we didn't ask about or didn't cover? I know we got to do some set theory, set math in our brains here. Yeah,

AJ: right. I'm like running through all the stuff we've been working on. I

Ben: would say just the growth of the MQTT stuff is really, that's been blowing me away.

The project in general has gotten quite a bit more popular since the last time we, we we did the podcast. But the MQTT has absolutely grown. And I think a lot of that is due to the feature that we added called MQTT client proxying, and so the elevator pitch for that feature is it uses your phone's internet connection to to actually connect to the an MQTT broker.

And most people just use the public one that we expose at MQTT. meshtastic. org and it allows, allows that internet gateway functionality, but not through a necessarily through a direct connection. So it, it can kind of proxy the traffic between the MQTT broker. And your device via Bluetooth or serial or any other any other way to

Jonathan: connect.

Yeah, and that has made possible some a couple of really cool online services projects, with with live maps That's one of the coolest things that's happened recently, I think so there used to be There used to be a map of meshtastic nodes, but the problem was that it was user reported, and it was all manually entered.

And so there was a lot of stale data in there, and there was a lot of things that, you know, nodes that would show either nodes that would show up that weren't actually there, or a whole bunch of nodes that were there that didn't show up on the map. And so a couple of people stepped forward and started scraping that MQTT feed and putting it, putting a live map together with that information.

And those are really cool. There's, there's a couple of them I know of. And I can't remember the name of both of them, so I won't mention both of them. Won't mention any exact names, but you can, you can Google for it and find it pretty quickly. And it looks like a semi official one. Do, am I supposed to mention this AJ?

Mostly for

AJ: your reference, but there's discussions internally about This is functionality that a lot of people want and how can we How can we kind of enable that in an easy way? I

Jonathan: had not seen this yet. So there may be, there may be an official MQTT based map in the works. We'll keep that one unofficial for the time being.

That's fun though. Yeah,

Ben: there's been a lot of community ones and I think it's kind of brought to light that we might need to have some official you know, curated mapping for, for the MQTT, just because it's, it's been so popular. And, and, you know, that it obviously exposes its own set of, of issues that we've had to kind of take, take a more discerning view of like, well, do we want, do we want this public server being used to, serve locations of folks, you know, so we're having those types of discussions.

Yeah. I would say a lot more just because it's such, such a big and popular project and we want to keep, keep everything Safe for folks and

Jonathan: yeah, I I I think everybody shouldn't everybody should know this If you have your positions broadcasted and you connect to the MQTT server You are sending out your positions over the internet And people can find them if they want to Keep that in mind.

Yes, keep that in mind. That's actually one of the, one of the main reasons why I did the position precision feature. And, and that's the idea. That's something relatively new in the firmware where you can set a value and essentially say, Hey, I want to shave this many bits. bits off the end of my latitude and longitude.

And so, you know, you can essentially, you set this and then it, it blurs your position essentially. And it, what it does is it intentionally sends out an incorrect position, but it also specifies, I am, you know, this is the size of the circle. And the position that I am reporting is right in the middle of this circle, but my actual position is just somewhere within it.

And so you can set that to be as precise as a couple of doors away, or you can set it to be as imprecise as this is probably the city that I'm in. That's new. And I think that, you know, once, once everybody knows about it, I think that's going to be kind of a big deal for these maps and things like that.

And

Ben: the iOS app has some really beautiful visualizations for that. You get some nice, Giant circles around your around your location.

Jonathan: Yeah, he had to talk me out of the way that I wanted to do it at first He had to talk me out of like that's a bad idea. That's a bad idea. That's okay. Fine. It's a bad idea We'll do it some other way

AJ: Love to hear what the idea was at some point.

I'm curious. I didn't follow this

Jonathan: it was a instead of a Value assigned to the channel. It was a set of eight values that was just assigned into the rest of the settings. And then as you, if channels moved around, whatever client moved, the channel would also have to move those values around. I was like, it should be easy.

Right. And he was like, This is not a thing that's going

AJ: to be easy. Well that means the client developers have to remember how to do it,

Jonathan: right? Exactly, exactly. He was like, this is going to get out of sync. It's just, it's going to be a disaster. Okay, fine, we'll come up with a different way. And actually, the way we came up with to do it, I really like, because it gives us now a place to put Per channel settings that don't, don't belong in the QR code.

We're, we're getting into inside baseball here. This is, this is where Rob was supposed to jump in and say, I have no idea what you guys are talking about. This is meaningless.

Ben: It's easy to do on this project. It's so

Jonathan: deep. I know why Rob wasn't jumping in. He's muted. Unmute yourself, Rob. I

Rob: was listening.

I was listening very trying

Jonathan: to figure out what are they talking about? Oh, All right, well, hey guys it is it is 12 30 central time at least we have filled an hour with this. It has been a lot of fun Thank you each for being here. Is there anything that you want to plug that we didn't mention? Ben

Ben: now just visit us at mashedastic.

org join the official discord Yeah, it's it's it's a huge community and there's a lot of helpful people there. Yes

Jonathan: The disc the discord is where things are happening. That is for sure. Yes Aj anything you want to get in right before we let you guys go? Discord

AJ: and come check out the management client.

Come tell me where it's broken

Jonathan: I think I think there's probably going to be a release of that like a Either a three run or a dot four release before too long and hopefully hopefully fewer things will be broken there

AJ: That's the dream. Always the dream.

Jonathan: All right. Thank you guys. Thank you so much for being here.

Thank you so much. All right, Rob, have we convinced, have we convinced you? Are you going to go order some hardware?

Rob: I mean, this talk really has got me excited about the stuff. I just haven't quite figured out what my personal practical use case would be, but I mean, that's never stopped me before on spending money.

Jonathan: Yes, yes, absolutely. Oh, you've talked me into spending money on a few things. I've got a pine watch somewhere around here. That seems like the coolest thing.

Rob: Yeah, me too. And I'd never touched mine after I took it out of

Jonathan: the box. Yup. So, there, actually, there is a there is a Meshtastic device, it's from Liligo, a couple of them, that are in the form of wristwatches.

So, I've got, I've got one here that I tried for the longest time to add Meshtastic support to, and it just doesn't have enough RAM to do it, not quite. Although, I might revisit that at some point. But, you know, one of the, one of the things that I find really interesting about Meshtastic is if you've got a GPS in your device, then you can set it to broadcast your position.

Well, you can, you could do that on a, you know, a private channel. And so then you can have like a home base where you can keep track of those devices. And so I've got one in my van. And so when somebody goes on a trip in the van, I can pull it up and say, Oh, okay. They're. at the post office. Okay. Now they're at the library and now they're on the way home.

And I find that really useful. And I also love that I can do it without having to send my data up to Google or to, to Apple or whoever. And then the other thing that I keep in mind is one of these days, cause I live in Oklahoma, you know, tornado alley, one of these days, the big one tornado wise is going to hit my town and It is not uncommon for when that happens that communications just totally go down.

And so the other thought that I have in the back of my mind is it would be nice to have this set up to where when that happens and the power is off and the phone lines are down and the internet is out, I can still get a hold of people and do some coordination. You know, who needs help, who needs a pickup, that sort of thing.

And so those are sort of my dual use cases that I keep in mind for Meshtastic. So when you get it

Rob: working on that watch, I could get my kids some new watches. And then when the power and everything goes out, I can find them wherever they're

Jonathan: at. Yeah. Yeah. And that is, that is exactly the one of the other things that I keep in mind.

Lily Go has been teasing. A, a watch, the, the more powerful version of this with a GPS built into it. And I think when that actually comes out, it's going to be a really interesting project, but it's, it's not quite done yet. It's not fully baked. I think they're having trouble squeezing the size of battery they want into it, as well as the full GPS that they want into it.

So fun times. We'll get there eventually. All right, Rob, thank you for being here, man. Anything you want to plug?

Ben: You

Rob: know, come check me out on the Untitled Linux Show with Jonathan. And also I always plug my website, robertpcampbell. com and then you can find links to connect with me. Yeah,

Jonathan: I love it. I love when folks plug their website, because if we ever get mad at Rob and kick him off the show, you'll still be able to find him.

Alright, next week we have, scheduled at least, Simon Kelly, who is one of the developers behind DNSmasq. DNSmasq is one of those projects that you, I am sure, use. Because it probably runs in your router, but you probably, a decent chance you've never heard of it. It's, it's one of those projects that makes the internet work, and it's going to be super fascinating to talk to Simon about it.

And maybe find out from him how many of these new cyber security laws are going to make his life difficult as an open source developer of this thing that is everywhere. So that is next week, and you don't want to miss it. Thank you to everyone that was live in the chat room, took some questions from there.

And thank you to everyone listening on the download. We sure appreciate it. Make sure to share the show. Let your folks know, let your friends know about it and we will see you next week on Floss Weekly.