All right. What's up, what's up, what's up, Freedom Tech. Welcome to Freedom Tech weekend.
My name is Marks. I am here every Friday at 1530 UTC. Sometimes I'm right on time. Sometimes I'm
a little late, a few minutes late, just as I continue to get things figured out here.
I'm also running a company. I run OpenSecret, which I happen to be wearing an OpenSecret shirt
today. Uh, just a coincidence, but, um, open secret, we make maple AI, which is actually
going to be the focus of today is maple AI. Um, I have not wanted to do an actual episode on maple
just because maple is part of every show as a tool. It's on screen being used to help run the
show every day. I don't need these headphones on anymore. Let's take these off here. Pardon me.
I'm just chatting a little bit anyways right now as people join the stream.
Okay, and checking. It's working. All right, great. So yeah, I have one. I've been avoiding
doing an episode specifically about Maple because I am a Freedom Tech developer as well. Me and my
co-founder, we build the most private cloud AI that you can find out there on the market right
now. And we are trying to enable individuals, businesses, activists, journalists, organizations
around the world that are trying to fight against oppression in their countries. We're trying to
enable and build these tools that people can use to have some semblance of privacy and sovereignty
as they start to embrace AI and use AI to help make them more productive. So that's what we're
going to actually talk about today, but we're going to talk about some other AI tools as well.
It's not just going to be Maple. I want to show off. There are a couple of local AI tools that
I like to use. There are so many other ones that you can just keep playing with them, like just
try out new tools and see how they go. But I'll show you a couple just to help get you going down
the path if you aren't already going down that path on your own. All right. We are working live
on YouTube. That's great. We have a few people already in here. Welcome. I'm going to open up
the chat as well. Feel free to drop in the chat. If you have any AI tools that you like using,
I would love to know about them and we can even show them off here a little bit as well.
And then if you also have just like concerns about privacy with AI or any thoughts on the topic,
drop them in, drop questions, drop concerns, and we can have a chat about them. So that is great.
Welcome everybody. I like to give just a few minutes for people to join and kind of get into
get into the room if you will. I'm going to make sure that this is live on X.
So let me go over here and just click on that. I got the little live circle around my avatar.
That's great. Sweet. It is streaming and then I'm going to go retweet this.
Normally I don't have Maple retweet my Freedom Tech Weekend stuff but since we are talking about
maple today. I'm going to retweet it somehow. Let's see. Come in here.
here we go quote okay come join to learn about privacy
let's see chat gpt data retention problems
and how you can use private AI tools like Maple
to protect yourself.
Sweet.
okay you are the faithful you are here right now already in the stream so welcome we've got 11
people now welcome everybody um just quick recap drop things in the chat we're going to be talking
about the chat gpt data retention problems and kind of break down the story and go through all
the drama that has been breaking this week it's actually been brewing for a few weeks
really for a long time, but really coming out in the last few weeks. And then we're going to look
at some solutions. What are some local AI solutions that we can use? And then what are
some hosted solutions that we can use? So I know that a lot of people who watch this are also
freedom tech fanatics. So I'm sure that you've already figured out some AI tools that work for
you. So drop them in the chat and we will check them out and talk about them. I'm going to make
sure that I have zap stream open so I can see the zap stream chat. It does not come into my tool
like everything else does. So I just don't want, I don't want to miss any of the chats over there.
Great. Okay. I got zap.stream set up. Let me move this over here and we are going to start sharing
screen and jump right in. Okay. So hit the screen share here. We're going to share my home server
screen that I've got right there. Last week I kept getting booted off. Hopefully it doesn't
happen this time. I don't like this window setup. That is dumb. So let's go like this,
like that, and then change the size. Perfect. Okay. Looking good. Like I should.
went and took a walk around the neighborhood. Okay. So here we are. Let's go in. What is all
this drama about? So we have ChatGPT, right? A lot of people use them. They are the biggest,
most well-known AI company out there. There are others that compete with them. Claude being
probably the second biggest one and then Grok, right? So in my mind, those are the big three.
ChatGPT, Claude and Grok. Each one of them are hosting the servers. They're hosting your data.
They're hosting the models. So everything that you do and say goes through their servers
and is processed by them. All three of those companies train their own models.
And so they need data to train off of, oh, I guess I, there's meta also, right? Facebook,
let's not leave them out because they are huge and massive as well. But they're going down a
slightly different path of the open source route, but they also gather data and host data. They have
all of the Facebook and Instagram and WhatsApp data. So they have lots of stuff that they train
on as well. And Mark Zuckerberg has been very open about this. I was in the room when they had
LamaCon back in April and got invited out there because we build on top of Lama as one of our
options. And so I was in the room when Mark basically told us all that we have all this
data from Facebook, Instagram, and WhatsApp, and we're not going to give it to anybody else.
And we're just going to train the heck out of it. Um, so that's what you're doing when you
use those platforms, your training models, all of these platforms, all four of these,
you're training, helping them to train their models. Um, and yeah, I'll get to that in a
little bit. So there's really a couple of ways to look at, look at companies, right. Um,
is that you have the big four that are training these big beefy models and grabbing as much data
as they can. And then you have all these other AI companies that are building on top of those tools.
A lot of people call those wrappers, right? We call these wrappers in the space where you're
basically wrapping a user interface around these big services and these services expose APIs so
that your app doesn't have to train its own models. You don't have to run your own models.
You can just call through to chat GPT or Claude and use their stuff. So a big example of this is
cursor, right? Maybe you've heard of cursor. It's one of the big vibe, vibe coding tools. If you're
not familiar with vibe coding, it basically means using AI to write code. And so a lot of people
download cursor and they run it on their computer but cursor actually calls through to the clod
api and uses that then you either pay clod for a premium pro subscription in order to keep using
it because you hit a cap on your free tier or you can switch cursor into a local mode where it uses
a local ai that is not as powerful right so that's one example of a wrapper okay there's a lot of
of these. So what's, uh, what's the big, um, the drama here? Uh, we'll get to that in just one
second. Sorry. I want to show you one more thing. And that is, let's see, can I get to my channel?
Hey, I got on the first try. Sweet. And look, we're streaming. So I just spoke last week at
the Oslo freedom forum, basically on this topic. I didn't realize, I know that some of these
articles were published, but I didn't realize at the time that all of this was going to explode
that that happened with chat GPT and their data retention. I already knew that this stuff was
going on and the writings on the wall, like their incentive is to capture as much data as you can.
I recommend people go check this out. I normally don't like shill my own speeches and talks and
stuff, but, um, gave a talk a lot about, um, freedom of speech, but then freedom of thought
and how we have fought for centuries over freedom of speech. And that, that battle is still going on
that we have not won the freedom of speech, but I mean, we, we obviously have a lot of freedoms.
I'm speaking to you today. I don't give a shit about which country you're in. Like, I don't,
I don't care. You can come from any country you want to, you can speak any language you want to.
AI can help translate this for you, whatever. But, um, I I'm speaking to you and I hope that you can
consume this content. I hope that you have a voice where you can speak. The new battle that I see is
over freedom of thought. And that is, we have all these AI products now that are gathering our data
and then are helping us to think, right? And so I could just type into, you know, to Maple and ask
it to do something, right? Help me come up with a business plan for a lemonade stand. Okay.
This is going to help me think, right? Now, Maple is not recording this. It's all encrypted. It's
private and we'll get to that. Right. But this is what I mean. So we're, we're asking it to help us
think we still have to read this. We still have to think critically ourselves, but with these other
platforms, Chad GPT, Claude, Grock, they are turning on these long-term memory things where
they will start to remember stuff for us forever intentionally. And then as we talk to the AI,
we are going to become more dependent on it to kind of be a digital brain for us. So we have our
meat space, our physical brain, and then we'll have this digital brain. And what's kind of this
more crazy, futuristic, dystopian thought is that they are the ones hosting our brain for us,
hosting our long-term memory. And they actually have the ability to alter that memory in any way
that they want to. Let me jump over spoilers. I'm just giving you spoilers all over the place here.
So let's jump over to censorship, right? There are multiple layers where they can censor us.
And I'm focusing right now on memory where they could go in and they could delete stuff from our
memory. Like if we are, if we're just having a thought about something that's very controversial,
as we do, every human has thoughts that are really dark, scary, negative, bad. We would never share
with anybody because that's how our brain processes the real world. We take input and we,
um, we have to like game play out some of these scenarios to understand what is right and what is
wrong. And, um, you know, 99.9% of these thoughts, these negative, or these kind of like immoral
thoughts we have, we don't act upon right now. Some people act upon more of them than others.
and those people usually end up in prison or worse.
But us normal, happy, contributing members of society,
we still have these thoughts.
So as we start to chat with AI,
we need to be in a safe, trusted space
where we can debate about these things
and really talk to an AI and really push it on it.
And these things are gonna end up in our long-term memory.
And as they end up in our long-term memory,
are these platforms going to censor us?
Are they going to remove these things from our memory or even worse?
Are they going to add stuff into our memory that we aren't aware of?
We have zero insight into this.
We don't know what is being done to that.
And so as we depend on this thinking, oh yes, this is our digital memory for us.
And then we start asking questions.
We're going to expect things that we actually thought about to be in there.
And if they're not in there or if they've been altered, that is going to change the outcome
of our AI chats and basically effectively change our thought process. And we start to lose our
freedom of thought. Okay. So that is the backdrop here. I just wanted to set a backdrop for you
on this entire conversation and where I see this. I call this censorship, subconscious censorship,
because they are going to be, they want us to be dependent on them as a digital brain.
and then they are going to be the keepers of what our brain has inside of it.
So just want that to kind of sink in for a second,
because that is where this whole thing starts to come into play.
So I would recommend you go check that out.
I think my talk will actually be published on the Oslo Freedom Forum channel as well.
They're starting to chop those up, but I went ahead and chopped it up myself
because I wanted to get it out there as well.
So yesterday, this account on X, they've got 66,000 followers, posted a screenshot of a court order and an article about a court order for OpenAI.
So OpenAI has been embroiled in a lawsuit here with the New York Times.
New York Times is suing them for copyright infringement, saying that OpenAI is hoovering up, taking up all of the data, all the written works of the world, including New York Times, all of their copyrighted work.
Think what you will about New York Times.
I have my opinions about the New York Times and the validity and accuracy of a lot of their content.
But whatever.
That's not exactly here or there.
They are taking all of this.
And then when you start to chat with AI right Like when I just had this in here is this considered copyrighted content Am I reproducing the New York times
So if I were to read a page from the New York times, a digital page,
nobody reads the paper anymore.
I kind of miss it, but nobody does.
If I were to, um, take an article from the New York times and then, you know,
say, Hey, write me, no, not using AI.
If I just took it and just like typed it out word for word, but
changed a few words here and there and then published it, I would be running afoul of copyright.
Okay. And I don't know the exact threshold. I'm not a lawyer, but I don't know the exact threshold
of how much I need to alter in order to be fair use, that kind of stuff. But there is some point
where I have to do that. And maybe I should learn that. Let's ask how much of a piece of content
do I need to alter in order to fall under the fair use copyright protection? Let's ask Maple,
because I don't actually know the answer to that. But I would be running into problems.
And so where it comes with AI is if I ask AI, write me an article in the voice of the, you know,
that I was, say I was publishing to the New York Times and this, this author, this journalist,
write me an article like that, right? It starts to look very similar to the New York Times,
but I am not actually copying the New York Times. So where are these lines? All right, let's look.
Fair use is a doctrine that permits limited use of copyright material without obtaining permission
from the copyright holder. To determine whether your use of copyright material falls under fair
use courts, consider the four factors, following four factors, purpose and character of the use.
Is your use for commercial or non-commercial purposes? Is it transformative, meaning you're
using the material in a new or innovative way, or is it merely reproductive? Number two, the nature
of the copyrighted work. Is the original work a factual or fictional piece? Is it published or
unpublished? Three, amount and substantiality of the portion used. How much of the original work
are you using and is the portion use significant to the original work? Number four, effect of the
use on the market for the original work. Does your use harm or help the market for the original work?
So pretty vague. Let's see if it gets into, let's see, short answer.
how much can I use of the original?
I think it's in there, but let's just ask this instead.
There's no specific percentage, but a rough guideline is one to two sentences or 10 to 20%
of the original work for images, 10 to 20% for music, five to 10 seconds for video, 10 to 30
seconds of a clip. Okay. So those are just kind of general guidelines. So New York times is
complaining that chat GPT is effectively using much more than that. And that you can reproduce
the New York times. If you wanted to, we could create our own website. I'm not going to encourage
you to do this, but you could create a website that spoofs the New York times regenerates all
their articles, but maybe you feel like they aren't factual enough. So you're going to make
them more factual who cares whatever that's the that's the gist of this lawsuit so a court order
came down saying open ai now has to start retaining all of this user data so that it can be examined
and looked at and i this is where i'm not 100 clear but i think it's because they want to
effectively audit later on for lawsuit purposes to see if copyright infringement is is being
breached right um so this caused a lot of drama um
this article is from six days ago deleted or not court order and open ai silence leave users in the
dark about how chat gpt user data is handled and then ars technica wrote an article two days ago
openai slams court order to save all chat gpt logs included including deleted chats right so a lot of
uh really long articles here but the the gist of it is that there are three things which article had
it um sorry one of them had just like a nice bullet point list and i'm trying to remember which one it
was let me search the page
Okay. Well, what it came down to the three that I've seen discussed the most are deleted chats.
So you as a user go in and delete a chat that you don't want to have stored anymore.
Those now have to be retained. Temporary chats. This is, this is like private window on your
browser, right? Incognito mode on your browser. But with AI, you say, don't remember this chat.
I just want to talk about it right now. And as soon as I close this chat, you can forget about it.
okay so that's the other one they have to retain those now and then the third one is api usage
this is massive the api usage part because there are all these apps out there that use the chat
gpt you know remember at the very beginning of the stream we talked about the wrappers
right not the little waynes little waynes right here no i'm not talking about that kind of wrapper
i'm talking about these uh wrapping paper right these apps out there that are ai apps that just
call through to open AI, Claude, I don't think Grok has an API, um, meta, right?
Some of these other ones, these hosted ones, the hosted deep seek meta and deep
seek are different because they have open source versions.
We are using the open source versions of, um, both llama, which is meta and deep
seek, but, uh, they have hosted ones as well.
And when you use the hosted ones, you are giving them your data.
Just remember that.
Okay.
So these rappers use these APIs and a lot of these tools, um, I've met some of them
when I was out at LamaCon, I spoke with a founder who is doing a, um, it's something
that's kind of adjacent to the medical community with therapy.
And he, he's like, oh, it's cool.
We, we have a zero data retention policy with open AI ZDR.
Those are all the rage right now with AI is zero data retention policies.
So he felt like he was safe because they're not keeping his data.
I'm sorry, but it's all bullshit.
It's just company agreements saying we will not use your data, wink, wink, but it can change
on a whim right here.
This court has ordered them.
Now we can debate whether or not the court should even be allowed to do this.
I don't know that they necessarily should.
I still want to learn more about this. This seems like potentially a big overreach for the courts
to impinge, impinge, to infringe on user privacy. They're just going in now and reaching into user
data and forcing a company to do something that it doesn't want to do, right? So there's a whole
another debate here to be had on whether they can do this or not. However, they are doing it. So
let's just talk about this for the moment. Um, they, uh, let me back up. I lost my train of
thought, but, um, they have to keep all of our data now. Yeah. So the zero data retention policies
can go out the window with a court order. Right. And even that you still don't know if they're
actually not retaining your data. They're just signing an agreement saying we will not retain
your information. And so if you have a therapy app that talks about very sensitive things
and has the names and personal private information of underage children who are dealing with some
really sad situations that you're putting into here, you hope that they're not retaining it.
You hope that they are not, that it's not going to get leaked out in a data breach at some point.
You hope that that information is not being trained. Models are not being trained on top
of that information and that in a future instance, when somebody asks ChatGPT about
a sensitive topic, somehow the name of that child and their very specific situation
is used in an AI chat. You hope that it's not. It shouldn't, but there's nothing really
preventing it from happening, right? So the more that you give this very personal information to
these models, the more that you are joining this public library of data that the entire world can
access. And you have no control over whether or not the AI uses the real names or the fake names,
uses real life situations, or says, these are hypothetical situations that just happen to be
the exact same situation that you typed into ChatGPT six months ago, three years ago. Who knows,
right? I hope you're starting to see the big data privacy problem that we have on our hands here
with all of this. And this was already a problem before this court order. The fact that
that open AI, Claude, Grok, and Meta have all this data means that they can do what they will with it.
And hackers can get in and steal the data. And then there's a middle ground of not even hackers,
but rogue employees, employees who have been bought off to sell this data, to grab this data,
to expose this data, right? I don't know if you follow the big news last month, but Coinbase
had a large breach of user data. What happened here is they leaked a bunch of
of government documents for people.
So when you sign up for Coinbase,
if you are wanting to buy Bitcoin or cryptocurrencies,
like fart coin that they just added, which is so dumb.
But let's say that you are using this platform
to buy Bitcoin and cryptocurrencies.
You have to upload your government ID.
So that could be a driver's license.
That could be a passport or some other government ID.
And they say that they're going to keep it safe.
And I believe them.
They do their best to keep these under lock and key as much as possible.
But due to government regulations, they are required to collect this and hold on to this data, right?
That's another debate.
Is this even something the government should be requiring of them?
I don't think it should be.
I 100% disagree with these laws that require all of these companies to collect our private information in the name of anti-money laundering.
I think it's a fake made up law and rule that should go away.
But right now we have to live within these rules.
So Coinbase is collecting this information.
They had employees in their company that were working with outside third parties to slowly
find ways into the different parts of the company so that they could gain elevated access
and eventually steal a bunch of these government documents and transaction records and a whole
bunch of information, right? You can go read up on this. This is not a Bitcoin cryptocurrency show,
but it's a very real thing. So anytime you give data over to these companies, it is sitting there
as a treasure trove for internal employees or hackers on the outside, or both to collude together
to make this information public.
Okay, what does Chachapiti store?
This is an article from January this year,
just kind of, it looks like it's probably something
they do at the beginning of the year
to rehash what's being stored on the servers.
I think this article is really great.
This website is nightfall.ai.
It could very well be generated, a generated article.
I don't know, but it seems helpful.
You do need to go verify everything, right?
You've always had to verify everything on the internet,
but now that things are generated with AI, you need to verify them even more. But they collect
all sorts of things about you. I'm going to come down here. Why do they do it? They fine-tune their
models. They have human review. Humans are reading your data. Okay. Despite these practices,
OpenAI explicitly states it does not use data for marketing or advertising campaigns or selling
information to third parties without consent. Without consent, right? Well, you most likely
gave consent in that really long end user license agreement they didn't read and that you just
checked that box because you tried to sign in without checking it. And then it was like, hold
on. And a little red thing popped up that said, you need to check this box in order to continue.
You did it and you continued. I don't know that for a fact, but this is usually where they hide
that consent is just in the fine print. Okay. What stays and what gets deleted? Deletion timelines,
user initiated deletion. By the way, this is, like I said, January, this is before the court order.
They delete a chat from the interface 30 days later. So if you delete a chat in the interface,
it's going to hang around for 30 days, presumably so you can like undo it if you want to, but also
so that they can train on it if they want to. Account termination, full account deletion erases
profile data but may retain anonymized interactions for model training unless opted out so if you
didn't go into the settings i don't remember but i'm sure that they're on by default where you're
going to donate your data to be trained excuse me then it's going to make it into their training
practices and then operator exceptions data from openai's operator agent including screenshots and
browsing activity persists for 90 days post-deletion for abuse monitoring. I didn't read this bullet
point earlier. Okay, what is that? Operator is this thing where you can tell it to control your
web browser. I'm smiling right now. If you're on the audio, you can't see me smiling. But like,
this is some dark dystopian stuff that they are building that is really cool and is going to
continue to happen. I think these are powerful tools that we need to have privacy underpinning
them, not the way that they're being done right now on Claude and on ChatGPT. It is taking
screenshots of your screen and your browsing activity. And then even after you delete it,
it going to stay on their servers for 90 days nine zero And they say it for abuse monitoring right That usually means child stuff right It could be other things Maybe it means abuse of their system I don know But a lot of these
privacy sacrifices, right? These compromises we make for our privacy are in the name of things
like terrorism or child abuse. You name it. They pick some really scary things to throw out there
and say, we need you to sacrifice. We need you to turn over your data. We need you to
open up your doors. You're not a criminal, are you? You're not hiding and harboring terrorists
and child abusers. So you have nothing to hide. So just go ahead and give us all your stuff,
right? No, that's not the way this works. We are selective about our privacy. We selectively reveal
what we think is needed at that moment with the interaction, with the transaction that we're
happening with someone. Okay. The relationship we have with somebody, the business transaction we
have with somebody, we need to be in power of revealing that. It is not let us collect everything.
And then even after you tell us to delete it, we're going to hang onto it for 90 more days
in the name of something scary, but really they can do all sorts of other stuff, right? That's
just the reason, but then they can train on it. They can sell it. They can give it to the government.
they can do all sorts of things. Okay. Europe has the GDPR stuff, and there's some shortfalls with
that. Insufficient anonymization. Let's see. 2024 EU audit found that 63% of ChatGPT user data
contained personally identifiable information, PII. If you don't know this term, learn this, PII.
Okay. You're going to, you're going to start seeing it everywhere now that you've heard it,
but it basically means really personal information that is about you. Um, this is huge. Okay. So 63%
of it contained PII with only 22% of users aware that opt out settings existed, right? Most users,
when they open up a piece of software, when they download an app, when they sign up for something
online, they go with the default settings. Okay. The, the software developer and the vibe coder,
whoever is behind building that piece of software, they put their opinions in there. They set the
defaults and a majority of the users are just going to go with the defaults and never change
them. Even if the settings are right in their face, they trust that the, that the developer
knows what to do. And, uh, with reason, right? The developer has put a lot of effort into this,
trying to maximize user experience because they want users to be happy with the product and find
value in the product. And so they set the defaults to be the quote best thing for the users. Well,
there are other defaults that are set that maybe are not the best for the users, but are the best
for the company. And that are things that those are things like data retention. So I'm sorry,
this is going long. Hell, we've already been going, we're over 30 minutes. Dang. Okay.
I haven't even gotten into Maple and local AI. Might have to make this a two-part thing, or
I'm going to keep going as long as I can, but we'll just keep on going as long as we have people.
We've got 51 people here. Welcome. Welcome to the stream. We are going over the ChatGPT data
privacy problems that exist, and then we were going to talk about some AI things that you can do
for yourself to have privacy with AI. Okay, so there are all sorts of opt-out things. They bring
up temporary chats that, um, that won't record it. Activate in temporary chat mode prevents
interactions from appearing in history and delete some after 30 days, though they may still train
models. If the opt out is inactive. I didn't even know this part. I thought temporary chat legit
meant that it wasn't saved on the servers. It was saved for 30 days even before this court order.
Okay. So sorry, but if you ever had a temporary chat where you were like, I'm incognito now,
I'm going to say all of these really weird, crazy things that I don't want anybody to ever know about
me. Sorry, those were already being saved for 30 days and they were training the models off of it
unless you opted out of that. Okay. Now the court order is having them retain them indefinitely
beyond the 30 days. Enterprise users can have end-to-end encryption for all interactions. So
if you are a large corporation and want to pay a ton of money to open AI, then you can encrypt
your stuff end to end. I don't know the specifics about that, but there is still some trust involved
that open AI is how they're running the servers, but this is likely pretty good. You just have to
pay a lot of money for it. And then you can also set custom retention policies. If you are a rich
person, right? A wealthy company. There was something else in here that I read through
earlier that I don't remember what I wanted to call out, but it had to do with enterprise users.
Free, let me search for the word free. That might be my keyword to go off of. Okay. Yes.
Indefinite storage for non-enterprise users. A pivotal 2024 policy change removed the ability
for free and plus users to disable chat history, meaning, and then this is all bolded, meaning all
prompts and interactions are retained indefinitely unless manually deleted enterprise and team
subscribers retain this opt-out capability with data purged after 30 days so sorry if you're a
free user or even just a pro user your shit is on their servers and it was being retained
indefinitely already before the courts ordered them to retain even more information okay i hope
with sinking in. There are more articles that talk about this. Yesterday was a very busy day
on X and on Noster for me because this was going all over the place. This guy right here is the
co-founder and CEO of Hugging Face, which is one of the biggest platforms for sharing AI models.
He was talking about, you know, who's working on AI encryption and privacy.
It's a very important topic in my opinion. I would love it if people jumped into this thread
on X and posted TriMaple AI because we're working on it. I'd love for him to know more about it.
Obviously, I mentioned it and we got a few others in there on their own who did it and asked them to.
Really grateful for that. But he's not the only one. There are multiple big accounts on X asking
this question now. What are we doing about data privacy? I'm sorry, but like, why was nobody
talking about this before? Like this, to me, this seemed obvious, which is why we started this
company. Um, and so I don't want to say I told you so, but like, come on. Okay. What can we do
about all this enough about the drama? You can go read up more on it. What can we do about this?
We're running low. I don't want to run a long time. We have all the time that we want, but,
um, I try to keep these to half hour because I post this on the podcast stream and then we have
a YouTube playlist and then obviously it's on X and zap.stream. So I want to keep these relatively
the same so that they're digestible. So I'll quickly go into some of the things that we can do.
What can we do? One of the local LLMs that I like to use, local LLM things, is called LM Studio.
Okay, so you have options. You can run all of your, these are the two options really, right?
You can run all of your AI locally, right?
And what that means is you run it on your laptop.
You run it on your phone, right?
I've got a phone.
Or you get like a little home server.
There is a server called Start9.
We've brought them up on previous episodes, right?
You can get something like Start9, Umbral,
or you can get like an actual legit server
that has like a really nice GPU inside of it.
That's really more what you need, but it's more expensive.
and you can just have a computer running in your home that's plugged in all the time, 24-7 running.
And then you set up the internet connection so that when you're away from the home,
your phone can talk to that home server and do your AI for you.
This is the most private AI that you can get.
Local, running on your own box.
You could even make it so that you can have the internet turned off while you're using the AI
and know that your stuff is not going anywhere.
You can see exactly what's being stored.
Okay, this is awesome.
and it used to be difficult to do this.
It's getting easier every day because more tools are coming out.
LM Studio is one of those really easy to use tools.
Okay, what it lets you do is you download it,
and it's available on like all the major platforms, I believe.
Let me get to the download page somewhere.
Downloads.
Download LM Studio.
As you can see, they have Mac, Windows, Linux.
different processors, different versions, that kind of stuff. Right. And then they're also open
source, I believe. Trying to remember. Yes. Get up. There it is. Okay. So they're open source as
well. So you can see the code behind this, what's powering this, right? So a very powerful tool,
but what makes it so awesome is then it's just point and click to install these models.
So they have all these models in their catalog and you can just be like, oh, DeepSeek R1. I heard
everybody talking about this because it supposedly crashed the stock market for a few days,
which was so dumb. But because having better AI out there just means we're going to be able to do
more stuff with it, not less. So I thought that was silly. But you can just, when you're in the
LM Studio app, you can just like click on these and install them. And it tells you roughly how
much RAM, how much memory you need on your computer to be able to power them, that kind of stuff.
Okay. Really easy point and click installation. So LM Studio is great. I recommend people
download that and just try it out and play with it, right? What are the trade-offs? Well, you get
super strong privacy, but you have an underpowered computer or your laptop does not have a high-end
NVIDIA GPU inside of it that is going to run as fast as ChatGPG or Maple. You also, you have to
kind of maintain it yourself. So you have to make sure you're updating the models or if you're going
to do a home server, you have to make sure that it's always turned on, that it has security patches
applied to it, that you have an internet connection to it when you're away from the house, poking
through your own personal firewall, that kind of stuff. So there's cost and work involved with that.
But you get great privacy. Okay. So we wanted to come up with something different
that took the power of cloud compute, right? Let me sign out.
actually no I'll just do a private window try maple.ai we want to take the privacy of local
but bring it to the cloud because nvidia has these massive gpus called the h100s and they're
always innovating and coming out with bigger stronger ones right that can run these big models
with local ai with something like lm studio I don't know if you notice this but if I go to the
model catalog. They have these numbers next to them, right? 3.8B, 14.7B. These are the number
of parameters that can be run in a model. And the more parameters, the more RAM your device has to
have just to be able to load the model. But then also it has to process it, the speed at which it
can process it. So your devices can run these sizes. Well, Llama 3.3 that we run is 70 billion
parameters, right? That doesn't run very well on an iPhone or on a Linux laptop,
unless you install massive amounts of RAM. So in order to run the strongest, biggest models that
are the fastest and the most accurate models, you need to have cloud compute level stuff.
And those are H100s and anything better that comes along.
so we set out to say what can we build that combines the two okay and that that's where
maple was born maple uses something called secure enclaves or confidential computing
to do all this and so what you get is um you get your phone here and it makes an encrypted
connection over to our servers which is normal right but then it uses secure enclaves
Secure enclaves are the things that are on your phone that secure your face, your thumbprint for ID.
It secures your wallet when you use like Google Pay or Apple Pay, that kind of stuff.
The secure enclave is making that all secure and safe.
So these exist on cloud servers now.
And so you can use secure enclaves in the cloud.
AWS has a version.
Microsoft Azure has a version.
NVIDIA has one on their GPUs as well.
And so they put these on there.
And what it does is it creates this safe box where you can put code.
Everybody can see the code that's running in the box,
but they can't see the data that passes through it, right?
So if you're in the kitchen making food and you're using a recipe,
everybody can see the recipe that you're using,
but they can't see what actual ingredients,
what kind of flour you might have used or what kind of,
if you put sugar in it, what kind of sugar you're using
or what kind of eggs?
Were they eggs that were out from free range chickens
in your own property or were they bought at a store?
They don't know.
They just know the recipe that you use.
That's what secure enclaves are.
So you know the data that's being used,
but you don't, sorry, you know the code that is being run,
but you don't know the data.
And this is big when, you know, we just a second ago,
we talked about ChatGPT
and how they have these agreements with enterprises
and they say the zero data retention policy,
They say, we're not going to keep your data.
We're not going to store it, right?
We promise.
Well, you don't actually have a way to look at what code is running on the servers.
You don't know if they actually inserted a thing because one of the governments came
and knocked on their door and said, hey, there's this business over here.
We want to have access to what they're doing.
So will you please just, we know you have a zero data retention policy.
If it ever comes up in courts, you know, you can fight it.
You've got a lot of money or maybe we'll help you fight it, whatever.
please just like insert a line of code that makes a copy of all of their data and puts it in a secret
database that'll never be seen by anybody other than us that can happen um because you can't
verify what's running on the servers so confidential computing gives you the ability to see what's
running on those servers and that is through a process called attestation right big long a word
but um you know live secure enclave verification this is on our website before we even sign in and you can come see that our servers are verified So I click on this this is my proof of security This gives me like a fingerprint of the code
the build of code that's running on the server.
And then you can actually look at all the servers
that are in between me and Maple, right?
Their AWS has their own servers.
So if you have been around app development
and internet security for any kind of time,
This looks familiar to you.
This is very similar to HTTPS,
to secure socket layer TLS kind of stuff, right?
So this little lock icon that every browser has now,
we've all been conditioned to see this lock icon.
If I click on this, I get to see the certificates
so that I know that this domain, trymapl.ai,
going over HTTPS is secure.
It's secure because some company up on the stack of trust
signed it and said, yes, this domain was registered by this person and they are the ones running it.
They put the certificate on their servers. We agree that this is who they say they are. So that
way some other company isn't pretending to be try maple.ai and spoofing and, but they don't have the
certificate, right? Well, secure enclaves are this next evolution, right? We started with HTTP. I'm
just going to highlight it here on the screen, right? The letters HTTP. This was the first version
of the internet that was not secure. Everything just kind of went over the wire. And if you were
able to clamp into that wire, metaphorically, you could see all the traffic going across.
Well, then we started to log into bank accounts and buy things on eBay and buy plane tickets and
other things. So then we said, let's make this secure. So we added an S on the end and called
it HTTPS. And we used encryption to encrypt everything as it goes across the wire. So now
if you tap into that wire and try to listen, it's just a bunch of garbled, a bunch of garbled mess,
right? It's encrypted. Uh, we think that we are on the cusp of a new upgrade and that is HTTP SE.
That's what I'm calling it. Uh, HTTP over secure enclaves. And these are pieces of software that
are running in different enclaves. So we have maple running an enclave. We have open secret
backend running in an enclave and watch. Okay. Sorry. Maple, I guess technical maple's running
in your browser. It's a JavaScript app running in your browser that talks to the enclave
of open secret in the backend, but you can have more enclaves. Other apps are running and their
API is running an enclave and you can start to verify, not only verify that your stuff is
encrypted in transit, but now you can verify what software is running on that server and that they
are protecting your data, encrypting your data in the way that they say they are. So a promise that
we make here with Maple is we make the software commitment to you that you have a private key
that is going to privately encrypt all of your chats. So let me go to our GitHub.
I know it's on here somewhere. Oh, there it is. Okay.
so we're the company open secret we make maple and so you can come on here and you can say okay
open secret this is the server software that is powering it and if you are a developer you can
actually go in and look at our rust code or you can just simply feed this code to ai and say hey
please help me understand this code base what are they what are they doing to you treat my public or
to treat my data my user data are they saving and recording my user data are they doing it any way
that is unencrypted or are they staying true to their encryption promises, right? So our encryption
promise to our users is that when they sign up, right, when they go here on trymable.ai
and they click log in and sign up, they get these options. It used to be, and it still is with a lot
of privacy tools are kind of hard to use because they require you to generate your own private key
and it ends up being this long string of text or like a group of 12 words or something that you now
have to manage and maintain on your own. And then if you somehow lose that, right, let's say you lose
your encryption key, all of your data is now rendered unreadable. And that's just how encryption
works. If you're not familiar with like private public key encryption, it's effectively a one-way
street. You have to have that private key that you encrypt and it goes one way. And the only way to
decrypt that is with the private key. If you don't have the private key, mathematically, you cannot
unscramble all the stuff until we have new computers. In theory, quantum computing might
be able to break that encryption algorithm. We don't know. But for now, the whole world operates
under this model that that encryption has not been broken. I will say it's possible that
governments, defense departments maybe have already broken that encryption, but their incentive is
actually to keep that quiet because they want to, if you've broken that encryption, you're just going
to want to watch all of the world's traffic go by. So you are not incentivized to let everybody know
that you've done that. So for now, we're operating on this model that this encryption is solid and
working well. Okay. With that said, because we're using secure enclaves, we can take that scary user
experience away from people and we can just give them something that they know and use already.
You can sign in with email or you can sign in with some kind of OAuth provider, right?
Eventually we want to allow other options like maybe log in with Noster or maybe you could log
in with a private key. So if you do want to have an extra layer of security, maybe you generate
your own private key and log in with that. These are things that we've kind of tossed around,
right? Pass key, other things. For now, you just do this. And so you can just, boom, log in with
one of them or sign up with one of them. And it's very easy. It looks like we have login here when
it should say sign up. Um, or maybe not, I don't know. But what happens is once you do that,
you get signed into Maple, like I'm signed in over here, right. And you get dumped into this page
and a private key has been generated for you. So that long complicated key is generated for you
in the enclave and then stored encrypted in our backend. So if I were to log into our database,
right now, all I see are a bunch of encrypted rows that I cannot read. And I'll never be able
to read unless I have your private key, right? Or unless I, uh, unless I push a malicious version
of our code to the servers, and then you log in with your private key. Um, and that, that version
of the code saves your private key down, but that version of the code is right here on GitHub.
So if I were to push something new, it's going to show up on GitHub and your Maple client
knows which version of the code it's talking to. And if I push something that the Maple client is
not expecting, this will not connect. This green verified badge does not show up. It turns red with
big red X. And when you try to do anything with Maple, it's going to say, sorry, I'm not going to
let you do this because I am not talking to the correct software on the server. Something's been
tampered with. You need to go check that out, right? So that's where this system of trust
becomes an open system of trust,
which is why we call our company OpenSecret
because all of your secrets are protected by the fact
that we are running open source code
that runs in enclaves that can be verified by anybody.
Now, down here in the instructions on GitHub,
we have instructions for you
to how to make your own build of the code.
And then you can get that checksum
that you can verify yourself.
So let's go back here, right?
This fingerprint, we explain to you in the instructions
how to generate that fingerprint.
and then you can compare the two yourself.
The software is making that comparison for you already,
but don't trust verify, just do it yourself.
And you can make sure that you are operating
under the privacy model that we told you
we're gonna provide, right?
So you can keep an eye on our GitHub
and look at changes that come over the different commits.
And then you can also watch the Maple thing as well
because we make changes there.
So we develop in the open.
And people are able to see what's in there and see that we are not inserting back doors.
We're not inserting things that log your stuff, right?
And I think this is the way forward for cloud computing.
Start9 has the opinion that the way forward is no cloud computing, but home servers.
And I really, that resonates with me.
And I think that's important.
I think that's great.
I run my own home servers.
I know a lot of people run their own home servers.
Let's do more of that.
and let's build software that can be truly decentralized and can interact in a very easy way
with other people using the trust and the privacy that we set on our own servers.
We're not in that world yet, but they're building it. There is going to be a long transition period
between right now where we are on these cloud systems like ChatGPT that capture all of our data,
right this is the reality we live in right now and to get to home servers where everybody's
running their own private ai is going to be a lot of work and effort to get there let's keep
building it okay this is not in place of that but i think that there's a huge middle area for this
transition where we need to bring people better privacy we need to protect people we need to
build a safer world right now um we can't wait for that because people are getting screwed over
right now. And so that's where Maple comes in. And then even in that world where we all have
our own private servers, if secure enclaves in the long run get adopted heavily, then we might
have this dual system where you have your own home server and secure enclaves in the cloud.
The cloud is always going to have more powerful hardware than you have at home because
businesses can afford bigger, beefier hardware and can share the cost and spread the cost over
lots of users. So we might end up in this really cool world where you have a home server and you
have cloud software that's running on secure enclaves that talk to each other. And I think
that's an even cooler future where we combine the two. So I want to keep marching toward that
direction. We're going to do this with Maple, start nine, and others are going to do it with
home servers. And let's work together. And let's, let's build this brighter future that leaves
behind just plain HTTPS leaves behind this trust where you can't see what's running on the servers.
Okay. We are coming up on an hour. I'm going to call it here. Okay. We got up to, we have 77 people
on the live stream. Welcome everybody. I really appreciate you tuning in. I've been doing this
every week now for a little over two months. We're showing off different freedom tech tools.
We're trying to give you something, one thing that you can do, one app you can play with this weekend
to try and decouple yourself from the captured technology systems that are out there,
the things like the chat GPT, right? So this weekend, my challenge to you is to play with
one of these tools. Go sign up for Maple, right? You can get a free account. Let me go back. Okay.
You can get a free account and it's super easy. Is this hard right here? Sign up with email,
sign in with one of these. There are some privacy people that don't want to do that. That's fine.
I just want you to know that over half of our users use a privacy email.
We don't know who they are because it's just some temporary generated thing that is an
alias that forwards to some other email somewhere, or it truly is a burner email account that
they use, right?
I don't recommend the burner ones necessarily unless you know how to operate with it because
in the future, if you need to reset your password, you might run into some problems, but maybe
that's fine for you.
Maybe you just want to use this as a temporary account and get a new one.
Everybody has their own risk model that they operate under, but this couldn't be easier.
So just go to try maple.ai.
That's the website or go into the Apple app store and download maple AI.
Leave us five-star review while you're there.
We would appreciate it, but download it and sign in this weekend and play around with a
version of AI in the cloud that is similar to chat GPT, but is 100 is, is completely
private and confidential and gives you protection of your privacy. We do not have as many features
as ChatGPT. That is readily apparent as soon as you log in, right? But we're adding them constantly.
Every day we're adding new things. We just added this last week. This lets you put in your own
character, your own system prompt for your chat, right? We are adding DeepSeek. We are adding
document upload. We are going to add long-term memory, excuse me, long-term memory storage in
the future that is done correctly with privacy. We're going to add a web search. So you can do
things like deep research where it goes on searches web for you with privacy. We want to bring all of
these really important, um, productive and valuable tools, but do it in a way where you don't have to
sacrifice your privacy and give things up. Okay. So that would be my recommendation. Go to Maple,
but also play around with the local stuff. LM studio is great. One that I wanted to show off
was whisper. Um, I'm on a Mac and so I use one called Mac whisper. This is how I do. Oh, I'm in
private. This is how I do my transcripts from my weekly podcast that I do right. Mac whisper
is when I'm done recording, I take my recording and I dump it into Mac whisper and it generates
the transcript for me. It knows how to differentiate between different speakers. So if I have a guest
on. It'll differentiate between those. And it's all local. You can have it go out to ChatGPT if
you want to, but I don't. I just run it locally. It takes about 10 minutes to transcribe my episode.
So I just walk away and do other stuff while it's doing that. And then I come back and it's done,
right? Grab a local AI tool, sign up for try maple.ai or download the app in the app store.
That's your challenge for this weekend. All right. Looks like the stream stayed running this whole
time, which is dope. The screen share kept working the whole time. It doesn't always do that.
I'm going to stop sharing the screen here now. Thanks everybody for joining today on this episode
of Freedom Tech Weekend. We are here every Friday, 1530 UTC, and we're talking about a different tool
that you can play with this weekend to decouple your life from captured proprietary systems that
hang on to your data and use your data against your wishes. I would just like to say, hope you
have a great weekend. And if you have any recommendations for tools that you want to look
at, or if you say I'm using this thing and I depend on every single day, I don't know how to
break away from it. Shoot me a message on Twitter on X. My username is marks underscore F TW for the
win. Or I'm also on Noster marks at primal.net. Hit me up on one of those or leave a comment on
the YouTube video on this X post as well. And let me know what you want us to look at in the future.
and we will do that. All right. Have a great weekend. Peace.