Hey, we are live. Welcome to Freedom Tech Weekend. My name is Marks. I am your host
week in and week out. It's just me and the chat. What's up, chat? So go ahead and toss
in any questions that you have. Today we are talking about open source LLMs, how you can
run your own AI at home on your own laptop. You can probably do it on your phone too.
Today we're not going to talk about the phone, but I'm going to show you some tools, and then you can do your own exploration from there and maybe even get it going on your phone.
But this is Freedom Tech Weekend.
What is this?
This is something that we do every week where we show you one thing that you can play with this weekend when you have free time.
One open source tool or Freedom Tech tool that you can use to start decoupling your life a little bit more from closed technology.
ways that you can have more control over your data, more control of your workflow,
just more freedom and independence in the way that you use computers. Because we,
over the last 20 years or so, we have coupled our technology with other people's servers,
other people's computers. We call it the cloud. And there's a lot of good reasons for this.
We were able to scale. We were able to do so many more things, communicate so much better
over the internet. But this has come with some trade-offs. And the biggest of that is our privacy
and access to our data. You see data leaks all the time. You see your data being sold to
advertisers, to third parties, to governments. And so we need to look at these trade-offs and
let's see, is it worth sharing our data with third parties if that's what we're getting in return?
And then you can make your own decisions. There are still plenty of cloud-based things that I use
every day and I make that trade off. But there are plenty where I have decided to take back a
little bit and do it on my own or use other tools that help me do it in a more private way.
If you don't know who I am, again, my name is Marks and I run a software startup,
an AI startup called Maple AI. Maple lets you run some of the most powerful, I think we have
the most powerful open source models right now, and they're fully encrypted. So you get to run
confidential AI in the cloud, but end-to-end encrypted with your own private encryption key
that is generated in a secure enclave on the server. Today, I'm going to show you how to run
local AI. So you might say that this is a competitor to what we offer, but really it's
all part of the same package, right? We want people running local AI, and then we want people
using Maple when they need something more powerful, because your laptop is simply just not as
as the GPUs that we have in the cloud.
So we're going to show you kind of the whole package and help you out.
All right, we've got people joining.
Welcome, everybody.
I see people hopping in the stream.
I'm going to do a quick audio check, make sure that things are working because I have
been known to have problems with that in the past, but it looks like we're good.
We're streaming on YouTube and I can hear it.
Awesome.
Okay, well, I am going to just kind of keep an eye on that because I'm running it on my
laptop today.
My home server, ironically, is not as powerful as my laptop. I need to get a better home server,
but priorities, money, all that stuff. So for now, we're going to run on my laptop and I'm going to
show you what we got. Today we're hoping to cover AI that does text chat. So you can simply just
chat with AI back and forth on your local setup. We're going to look at image generation, possibly
video generation. I'll at least show you kind of some of the tools, but I don't know if we'll
actually do any video gen and then we have audio using whisper which is an open source audio llm
sorry open source audio model and then i'll show you how you can do your own coding locally with
with uh coding models we won't do a lot of it on this stream just because there's only so much time
so let's get into it i'm going to start sharing the screen we've got over 100 people in here
Welcome, everybody. Thanks. If you want to spread the word, just hit that like button. I cringe at
saying that, but whatever. The algorithms need it. So hit that like button. Go and subscribe to this
channel. I'm coming to you over the TFTC channel on YouTube. Truth for the commoner. We align very
closely on our views of freedom tech and trying to get more people taking advantage of these
awesome tools that are out there, using them, provide feedback to the developers, suggest ideas,
And then maybe use some of these tools.
Maybe today you learn what tools you can use to write your own programs, write your own software.
Because you're not a software engineer, but you want to build stuff.
So grab these tools, start building stuff, and you too can make your own freedom technology.
So that's what we're going to try and help people figure out how to do in their lives.
Okay.
We're five minutes in.
Let's start sharing the screen and let's get into all of these local models.
so let me just do this
we're going to share the entire screen
because I'm going to be jumping between apps
and so it'll just be too much to try and
try and just do one window right
okay
screen
there we go
okay looks like it's up
again please feel free to share
anything in the chat, any comments, questions you have. And I'll try to watch for them. I'm not,
I don't have ZapStream open. I guess I could quickly try it out real quick here,
but it's in this browser. Let's see. Looking like ZapStream is working today.
I'm not going to be able to watch the chat just because, well, maybe I can. Let's try.
Let's do a new window. I'll drag it off over here and then I'll mute it. Okay. I'm going to try
and watch the Zap stream chat if anybody hops in there. If not, it's all good. It's all good, man.
Okay. I think I got it. All right. Why run your own local AI, right? You got chat GPT.
They just came out with new models. GPT-5. It's all good. It's super powerful. All that stuff.
You've got Claude code and you can run that in cursor. Why do you want to run your own local
stuff? One of the biggest reasons is freedom and privacy. If you are running something on your
computer and you have the internet turned off, you've got the best privacy. Full stop.
That's where you start. Then you start to look at tradeoffs from there. Maybe your computer is not
as powerful as a cloud server. So what do you do? Do you buy a cloud server and put it in your home?
That is an option. It might be expensive. It might require you to do a lot of tinkering and
updating yourself. Maybe you have some power requirements where you have to bring extra,
you know, get an electrician or do some electrical work to get more power to that GPU because the
cloud-based GPUs require much, much beefier hardware or power requirements. And they make
make a lot of noise. Those fans are very loud. I have been there in the data centers with these
NVIDIA chips that are running and they're super loud. So then you look at other trade-offs like,
can I use a cloud AI? And if you're going to use something like Grok or ChatGPT,
you are giving them all your data. Even if you use the incognito mode and say this is a private
chat, what they do is they actually just hide that from the UI after you're done chatting.
Like it disappears from the UI. But if you look at their retention policies for data retention,
they retain those chats for at least 30 days. Most of them do. ChatGPT has recently had a lawsuit
where they're being ordered to retain them for longer. And those chats also make their way into
the models that get trained. And so, they're really they're retained indefinitely because
they become part of that corpus. So we have a middle ground. This will be my quick pitch here
of Maple AI. We provide a chat GPT experience, but you're fully encrypted end-to-end. So your device
uses a private encryption key using our secure servers, the secure enclaves, and then runs the
AI in the cloud. It's also secure enclave with the GPU. These are all open source models. None
then let's say that they are proxying to OpenAI.
Well, what you're depending on then
is that they are just kind of mixing your chat
with everybody else's chats,
and so you get lost in the crowd.
But the moment that you put any personal information
in that chat, you put your name in there,
you put something that is identifiable
for your location or something,
then OpenAI now knows about you.
So you're only safe, you know, in the crowd until you self-identify somehow.
And it's easier than you expect because you just get in the flow of chatting and you out yourself in some way.
So with Maple, you are in your own private vault.
All of your data is in a vault with a secure enclave.
You're chatting with open source models.
We have built the closest thing to installing your own cloud server in your home.
closest thing as possible that we can. But the tradeoff is still that you are trusting our
secure enclaves and it's running in the cloud, but then this is open source. So you can go look at
our GitHub. You can see all the code that's running on the server. You can build the code
yourself. You can get that checksum and compare it against what we are running the secure enclaves,
and you can verify that we are not logging your stuff. Okay. So let's get into it.
Some of my favorite tools. Real quick, somebody, when I posted this, shared this blog post here,
Average Gary did. I mentioned that you can build your own home server.
This looks like a great option. If you really want to go and get a huge beefy thing going,
you can build your own cluster at home with multiple computers, wire them together so they
can run massive open source models. This one's using the framework, uh, framework base. And,
um, I wanted to get down to the cost. Where was it down here? So there's, they spent about $8,000
on that. And then you can get like an M4 pro mini cluster spending $20,000. But this is an
interesting thing if you want to do. For the most part, they're still slower than the cloud ones,
but it's all about trade-offs, right? And this is a very new blog post. It's just from yesterday.
Okay, let's go into some of my favorite tools, and let's look at this question here.
Is there a way to request your chat data from OpenAI if you realize you don't want them having
all of your data? So there are a couple things you can do with OpenAI. OpenAI data delete.
I'm not going to log into my chat GPT right now on the stream, but go look at here and you can see how you can submit to have all of your data deleted, which is kind of the final step, right?
But prior to that, you can actually go into OpenAI and you can look at there's like a memory feature where you can see what it knows about you.
And it'll show you a box and say, here's all the things that I know about this person as you've chatted over the months.
then their AI says, oh, this thing that they said seems really important. That is their
date of birth or their weight or it's their name or the city they live in and starts to remember
these things, right? So, you can go in there and you can actually delete stuff from that if you
want to. That only affects your chats. And that's also only the thing that they show to you. We have
no guarantee if you delete it out of there, if it actually gets deleted out of their system.
It's very likely and very possible that they have kind of like some secondary
view of you that they don't share with you and that they are constantly compiling.
So, really the best way is to go in and go into the data controls and submit to have your data
deleted completely. But that's like a full score stir if you're deleting everything from ChatGPT.
But you can do that and then start over with a new one. I use ChatGPT, but I'm very careful.
I don't chat about anything personal or private. I simply use it for specific business things
that I'm doing. I use Maple for everything else. I also use Maple for business things.
We're doing a fundraise right now. We've been doing a lot of strategy talk. And I've been
using Maple. I've been using DeepSeek, this big one, 671B. I've been using the new OpenAI
open source that came out this week to do it. So it's very powerful. I only use ChatGPT for things
that we simply just don't have in Maple like image generation if I need powerful image gen online.
So hope that helps. Right on. Okay. So let's look at some of the tools. The one that I use the most
for running my local AI is called LM Studio. It's super easy. And it's available on every platform.
You can get it on Mac, Windows, Linux. So I use the Mac version obviously because I'm on a Mac.
But let me show you what that is. So this is LM Studio. Let's see.
How do I get fresh?
I don't know what's going on here.
All right.
Let me get a new one open here.
I don't know what's going on.
Clear all messages.
Sure.
There we go.
Okay, let's start fresh.
So, hello.
I don't have a model loaded.
Okay.
So, I tried typing to it and nothing happened.
So I go into my model loader and what do we got here? Well, let's go look at my models
So these are the models that I have downloaded. I recently downloaded the open AI one
That's what we're gonna look at today. Mostly. It's the 20 B and
Then if you look over at maple here, we have open AI
120 B what's the difference there?
so this is
effectively the number of parameters the the size if you will but the amount of stuff you can do and the amount of stuff the
of brain power that it has to run your your query dive into this more i'm doing like really high
level on that but dive into that more if you want a more technical understanding and i'm happy to
chat online about it if you want to want to go deeper but um so the smaller one basically the
smaller this is 20 billion parameters this means that it will run on a smaller device like your
computer because the more parameters you have the more ram you need the more memory you need on your
computer right so you can see this is a 12 gigabyte model and it has to load this entire
thing into ram and i'm gonna do something a little dicey here let's just open up let's see
so here is my i move the processes off the screen but here's my my memory usage on my computer i've
got 36 gigs of ram on this laptop currently using 30 gigabytes um and then i've got some swap swap
is just the hard drive that you have. So if it runs out of RAM, then it uses the hard drive as
RAM. And on newer computers now that we have these solid state drives, it's really close in speed to
RAM itself. So it's much better than the old days when you had these platters that were spinning and
you had to page out to the hard drive. So it's a little bit better. So you might think I'm not
going to be able to run this, but what it'll do is Mac OS will boot some other things out of memory
that it needs and then it'll start paging out to my SSD and give me more RAM that way.
So again, I'm running the smaller one locally. I don't have enough processing power on this
laptop to run the 120B and so that's why it's nice to have a cloud option available as well.
And how do I get these? Well, let me show you. Super easy. LLM Studio just lets you download
stuff so i go to search and you're like okay i want quinn coder a powerful 30 billion moe coding
model from alibaba quinn joining us larger 480b counterpart right so this one would not run on
your laptop unless you had the most amazing thing in the world you probably need a home server for
that or or group a bunch of laptops together and then it's got these different formats you can
look these up mlx is the one that i'm running because that is what uh is the uh the metal
Apple's metal stuff, right? Here we go. MLX, a new AI ML framework for Apple silicon by Apple.
So it supports that, which is great. I can just hit download and it'll start downloading it.
I got enough space. Let's see. I've got 79 gigs available. Let's just hit download
while we're going through other stuff. Okay. So this will show you what it does. So it just
gives you a really nice easy thing. Where do people typically get models from? Let's look at
that. So we have this website. The most popular place is called Hugging Face. Hugging Face is
this community where people will post open source models and it's kind of like a GitHub for models
if you will. So obviously when OpenAI dropped their two open source models they very quickly
were showing up on
a Hugging
Face and you can see how many downloads they've had. The hearts that
they got. Here is Quinn image, which we're going to look at today as well. So these are the biggest
ones, most popular, but you can browse millions of models on here. Hugging face can be super
overwhelming for a newcomer, right? You come in and you're like, okay, I don't know what all these
things are. And maybe there's like a whole bunch of different versions because people will take
this version of open AI and then they will do some transfer learning on it. They will train it in
their own way. They'll do something and they'll repost it. So now you don't know which one to get.
So you need to kind of get versed on Hugging Face if you really want to come in here. It's
great. It's a great place. I recommend people check it out more. But if you're a more casual
AI user and you just want to use this stuff, that's where these other tools come in. And that's
where LM Studio is great for text-based chat. You just come in and they kind of do some of the
curation for you, right? So you can come in here and see here's all the different ones that they
have. All right. So I'd recommend starting here. If you want to get more technical, go get more
technical on Hugging Face. The next one is Comfy, which is for ImageGen. And this is a great place
to run things like Quinn Image or other image generation models. We're going to look at that
here in a bit. And then you have Whisper, which is an audio model, also from OpenAI. There are
other ones, but this one has been pretty easy to use. And the way that I like to use Whisper for
doing audio on my computer is I use a tool called Mac Whisper. I don't go deep into this very often
with people because Mac Whisper is closed source. So it's not like full Freedom Tech because it is
closed source. It'll also let you use the actual API to talk directly to OpenAI or Anthropic or
others if you want to. I just run it locally. I don't talk back to the cloud servers.
But that's how I do dictation and other things is I use Mac Whisper. But it uses the open source
model. So you can go look on GitHub at the Whisper model. You can see how it's built,
that kind of stuff. And then I thought Comfey did video as well because I did, you know,
put in the thumbnail here that we're going to look at video. So, yeah. Yeah. So, you can do
video on Comfey as well. And then obviously the last one is code. We're downloading
quen coder right now but i wanted to show you
i lost it um quen coder
was that here oh yes i already have it open okay
so here's quen code you can go on github and look at it they've got lots of stars
but you can download this and run your own ai coding tool right on your laptop if you want to
using Quinn. Quinn code 3 is pretty powerful. A lot of people are using it. For building Maple,
we leverage Claude code a lot. We're trying out the new chat GPT stuff they came out with
yesterday. We use cursor with Claude a lot as well as Claude code. Those are super powerful.
We're also looking at Quinn. It would be great to do some more stuff locally for building it.
but we build an open source tool. So, in some ways, it's already going to be slurped up by
these LLMs in the future. So, we make that trade off. All right. Let's go back and look at LM
Studio because I would love to actually load a model and show you how it works.
Okay. So, we're here. Let's go back to our chat. Let's just drag that down there. Okay.
Okay. Select a model up here. Here's where I come to select my model. I'm going to do the open
source of chat GPT load model. Okay. So it's loading. It's pretty quick. It still takes a
while though. If we were to look at my memory footprint again, let's see what happens here.
That says load in the model. So it did free up some memory and then it's now loading and we see
we're hitting we're starting to hit memory pressure as we put this model in okay so now
we're starting to page out swap uses over three gigabytes it was just barely above one earlier
our real memory use is 32 oh sorry physical yeah 32 excuse me
okay the model is loaded now so let's say hello
okay so it's thinking what does it mean by thinking okay hello how can i assist you today
so this is reasoning effort lm studio lets you look at does the model reason what does reasoning
mean well reasoning is where the model like is introspective and looks at your thing and says
okay here's what they said let me build a prompt for myself based off what they said and then i
will run the query. Whereas the earlier versions of ChatGPT simply just ran your query against the
LLM. So if you do low, you're going to get almost no thinking at all. How can I assist you today?
What things are you good at? So I turned off. I made reasoning really low. And so it doesn't
think for any seconds now. It just spits things out to me. So this model is good at writing and
editing, drafting emails, that kind of stuff, reports. It's good at research and summaries,
coding help, learning tutorials.
So here's all the stuff that it's good at.
And it was pretty fast, right?
It gave us 32 tokens per second.
I had this one was 357 tokens total.
And the time to first token was 0.62 seconds.
These are all metrics that people track a lot
when they're running LLMs.
When you run this in the cloud, this is all so much faster.
But this one's pretty quick for being an open source model
that you're running locally.
Now, if I wanna turn on reasoning more,
build me a business plan for a lemonade stand.
Whoops, I misspelled that, but it's okay if you misspell
because it knew what I was saying anyways.
See, it already, like you can see what it's saying.
Oh, come on, I want to view this.
There we go.
Okay, the user says,
build me a business plan for lemonade stand.
So it corrected my spelling.
And then, you know, it says,
they likely want a comprehensive business plan.
So you can see like what it's doing,
what it's thinking through,
and then it's building its own prompt
based off of what I asked it.
So it's still thinking.
Okay, now it's giving me my business plan,
spitting it out here.
If I was doing this in Maple, it'd be much faster.
In fact, we could do a side-by-side comparison
if we want to see how fast the bigger model is
to the smaller model.
My download is complete.
All right, so I've got QuenCoder 3,
I currently have reasoning turned off for this version.
So maybe I'll do this without any reasoning.
Let's do that.
I wanna do like a true comparison.
Clear.
Okay, let's start fresh.
Put the reasoning left for it down to low.
Correct my spelling mistake.
Okay, so we're gonna do a side-by-side speed test
between GPT-OSS, which stands for open source software,
120B versus the 20B, which I'm running locally on my laptop.
This one's running on Maple on powerful Nvidia GPUs.
Okay.
There we go, all right, it's ready to go.
So I'm just gonna hit return and go.
Okay.
So they're both spitting out pretty similar as far as content goes.
Obviously Maple is ahead.
I didn't do this before the stream.
I was kind of worried that it wasn't going to be faster, but in theory it should be faster.
So this is on the operations plan still on number five.
We're on number seven over here on the left.
Okay.
done. This one's still running. Now in Maple, we don't show you your like tokens per second or
anything. But it feels very much, it's much faster. So I can be getting more done here.
I can already be chatting with it more. How about for a trading card stand?
Okay, this is done now. So we took, we got around 30 tokens per second, which is pretty good.
We didn't do any reasoning. Let's see. This was 1400 tokens and time to first token was
half a second, roughly. Pretty good. Okay. So, that's local AI. Again, you can change the model
you're running. Let's switch to CoinCoder. Let's just do that really quick while we're here.
Context length. Okay. So, what's context length? This is how many, basically how much text you
want to pass in. For lack of a better way to explain that. Just the amount of text you want
to pass in. Setting a high value for context link can significantly impact memory usage.
And that's because whatever you pass into it, it has to load it all into RAM
for the LLM to process it. Let's hit load. Okay. So, for example, let's scroll back on
Maple while this is loading. All of this stuff here is now context. Like, everything you see,
even the first question I asked it, and if I continue to ask it follow-up questions,
everything you see on here becomes context within the LLM. So, the longer that I chat,
I need some water. The longer that I chat, the more context it's getting, which is helpful,
but also the more expensive it gets, the more computing power you need, the more memory you need.
It just, it makes everything bigger. Okay. Is this loaded? I think it's loaded. Okay. Let's
Oh, there's the button. When I'm on screen, I kind of forget stuff sometimes.
All
right. We're up to over 500 people watching. Welcome, everybody.
It's a wonderful Friday. This is Freedom Tech Weekend. If you're just joining us, I am not
Marty. I'm not Odell. I am Marks. I run, I'm a co-founder of Maple AI, secure encrypted AI in
the cloud. You can go to try maple.ai. Today, we're talking about how to run your own local AI.
So we're running things locally with LM Studio.
We've got Comfy UI for doing image generation.
We have Mac Whisper for doing audio.
Comfy UI also does video.
And now we are about to try out coding.
So let's do something simple.
Let's say my favorite,
because I often have to download YouTube videos for clips.
My own stuff.
I'm not doing anything bad.
But let's say write me, I don't know, I don't want to, what do I want to say here?
How about we want to convert, we want to convert audio.
So let's say I recorded some audio in M4A, but I want to convert it to MP3.
Okay, write me a JavaScript that will convert M4A audio file to MP3 audio file.
Let's see what it does with this. I don't actually know what it'll do here.
Okay, so it's loading CoinCoder. Here's a JavaScript solution. Okay, so it's going to
use FFmpeg, which is understandable. And then here's my JavaScript file.
What is it using? How am I? It should give me... Okay, so here it's telling me how to install
FFmpeg. This is awesome. It's like your own little mini GitHub. It gives you installation
steps and everything as you work through the different systems you might be on. And then
it's going to have you use Node. So I'll have to install Node on my computer if I don't have it.
I already have it. It tells you how to run it. So that's pretty cool. I'm not going to dive deep
into this right now, but you can just grab LM Studio, download CoinCoder 3 through the UI,
and start writing your own stuff. Super basic way to start. Okay. So whatever job you're working,
If you're working full time and you're like, man, I could really use some little tool to help me do
this thing. Maybe you get a lot of data from somewhere at work and you're like are always
copying and pasting and doing manual conversion or you're doing manual analysis on it, throw it in
here. Download Elum Studio. Throw it in here. You can attach documents, right? I can come in here
and I can attach stuff. Toss it in there. Have it write a script to a JavaScript or some program.
you can probably have it write an app. Can this write iPhone apps? Make it a macOS native
application instead. I don't know what this will do. Okay, so it's making an electron app.
That's fine. So it's not going to use Swift. I probably could do Swift, but it's not going to
here, probably because I started with JavaScript, and so it's just taking that. But you could build
your own little app that runs on your computer, and then you like drag and drop files onto it and
stuff. Okay. But it's spitting it out. Now, if you're going to be coding full time for your job,
then you would want to get like an actual developer environment, whether that's a command
line interface or an IDE integrated development environment. But this is a great way to get
started for people who just want like one-off things that they're doing to help make the job
better. So you can really kind of show up and show off at your job by using a tool like this
to build little conversion scripts for yourself or utilities for yourself that are highly specific
to your job role. Like you're not going to be able to go online and find some tool that
understands exactly what you do day in and day out and the kind of data you're dealing with.
You're always going to be trying to figure out how do I take this tool and shoehorn it into my work,
right? Well, with this, you can build the tool specifically to your requirements.
You can even have, let's open up a new chat here,
something we do all the time.
Build me a technical spec specification
that I can hand to a coding LLM for this feature.
Okay, so we're gonna say convert audio
from any format into mp3.
Accept audio files that I give you.
Handle multiple files at once.
And last one, we will say, well, let's just do that.
Okay.
So then it builds you a specification
about all these edge cases that I might not have considered. Like, hey, we need to have all these
different bit rates. And it's probably going to go into the different formats. Yeah, here's all
the different formats I might need to worry about. I can read through this and say, okay,
don't do that thing, but yes, do this thing. And then I can add more. I can refine this, right?
And then when I'm done, it's like, all right, I got this whole thing. Now I can come in here,
build a new version of this JavaScript file using the following spec.
I should probably start a new prompt instead of building on the old one, but that's all right.
I might want to start clean and not have the context from before. All right, so it's processing
it. It's taking a little while. Okay, it's having to read this giant prompt I just gave it.
And now it's implemented for me.
Okay, so what we're doing here is we're showing you how to be a product manager, a project manager, like a technical program manager, whatever these PM roles are, right, that you maybe have on your team.
And then we're also showing you how to be your own mini software engineer, a data analyst, like whatever role you might have at your desk job.
If you don't have a desk job, if you are working with your hands, like there are other ways you can use this too to get your job done.
So many things you can do here.
Back in the day, I worked with some guys who would go in and repair homes who had had flood damage or some other problem where insurance was going to give them money to do repairs on their house.
They would come in and assess it and they would have to like bid out really quickly.
And there were multiple people, right?
The homeowner had some leak in their home, a pipe burst, and their floors were ruined.
Maybe a lot of their walls were damaged.
They're calling a bunch of people, bringing them in.
These people all assess the situation, and they need to get back to them as quick as they can with a bid that they can honor, that they're not going to lose money on.
So they want to do the highest price, but also the lowest price, as close as they can get to being efficient.
Well, I helped someone build a little software tool that they could run on their iPad and they could show up in the house.
This was a long time ago before other people had done this.
And they would just go on their iPad and they would tap this, that, here's the damage, here's the size, all that stuff.
And it would give them a quote right there on the spot.
And that was really beneficial to them.
It gave them a huge edge up in their business.
And so they could bring their contractors in and work on jobs, get a lot more jobs.
They could use a tool like this now.
They wouldn't need me necessarily.
They could build their own specification, right?
They could talk to, you and I talked to this
for just a few seconds.
You could spend an entire week, two weeks if you want to,
really homing and refining some of these prompts.
And then you could go and dump it in here
and have it build the app, go back and forth,
iterate on it a ton and build this really kick-ass app
that you can use.
So just trying to give ideas
of how you can do these things on your own.
Okay, but again, it's kind of slow,
Depending on your hardware, if you want to buy your own big beefy server and run it at home, you can.
Or you can use cloud tools as well.
Let's move on to ImageGen.
This is one I have not done very much, and so I'm actually really interested and excited to try it out.
Before we got on the stream, let's see.
What would you use to go from a query to a tabulated data graphic chart of the data model returned?
So let's see.
You know, it's interesting.
So one thing you can do is you're going from a query to tabulated data to graphic chart.
So if you're using spreadsheets, for example, then you can actually have the AI print out
CSV, comma separated values.
So let's do a new one.
Let me show you what I mean.
Actually in Maple, I have an example of this.
So let's open up my history.
So Austin weather data.
I was doing an example one time where I downloaded
the weather in just a text file
and then I uploaded it to Maple
'cause Maple can do documents images.
Our document stuff is paused right now.
We had some issues with the reliability,
but we're bringing it back up soon.
But so I had this document that I put in here
and then I asked it, like, give me some trends.
I could just come in here and say,
give me a CSV output
of the data that I can put into a spreadsheet.
And so now it's thinking
and it's gonna spit that out for me.
Jeff G, I'm a coder, but love AI, changed my life.
Totally.
I've been a software engineer for what, 20 years now?
On and off, I've done other roles,
but I write software as well.
And AI has totally changed it because it used to be
that you were kind of stumbling through,
figuring things out.
You knew the concepts,
but every language was slightly different.
You had to learn that syntax.
You had to learn an API.
You had to understand all the ins and outs and intricacies.
Now you can just say, hey, like build me this thing.
And then you can look at the code.
You can tinker and tweak it.
It's awesome.
It's really phenomenal.
All right, so here we go.
So now I just, I could copy this and save it.
you know, I open up my text editor. I don't want to do that right now because I don't know what's
in there, but I could open my text editor, save this as a.csv file, and then you can open this
up in any spreadsheet program, Excel, whatever. And then from there, you can make charts, that
kind of stuff from that data. So I hope that's helpful with what you're looking for. Okay,
image generation. Let's try it out. So I just clicked on the first workflow that they had,
and they already have this thing in here.
This is a little more technical
than what we were seeing with LM Studio.
It's kind of showing you the model that you're using,
the prompts that you have.
You can tweak all sorts of stuff.
We're not going to do all this,
but it gets very technical.
So beautiful scenery, nature, glass, bottle, landscape.
Let's just hit go and see what happens.
Where's run?
Run is somewhere.
oh it's underneath there we go okay y'all can see it but i couldn't because
my streaming software was covering it
okay so it's running did it run
oh okay there we go so it ran it um let's do another prompt so i got that little glass thing
let's do another one um
I wanted to watch it go. Browse templates. Image generation. New fresh one. Let's do a guy sitting there at home chatting with a robot.
okay so it's showing you as it goes through it's doing like the sample stuff
so we're at 57 and 15 25 i don't know what the two different percentages are but
it's doing stuff here let's go look at so i've got a lot of memory going on i should close lm
studio which i'm going to do actually let's free up some ram all right so we got this image here
let's zoom in I mean it's decent right it's not as good as chat GPT I could
tweak things right because chat GPT has done a lot of tweaking on these so I can
up the number of steps I wanted to take so how many times is it gonna go over
different things you can do there I did download the new Quinn Quinn image how
How do I load that?
Let's see really quick.
There's gotta be a way to load that.
Load checkpoint.
Can I just drag it?
No.
Well, if we can't figure it out on screen, that's all right.
I wanted to though.
Can we do video?
Let's try video.
Okay.
Let's do missing models.
So I can download a model.
Let's do this really small one
just 'cause I know it'll download quickly.
Again, a lot of these tools make it so much easier.
If you're just joining the stream,
you can hop into Hugging Face
and you can see all the models that are available.
This is the most popular place for people to go do models,
but it can be very overwhelming because there are so many.
There's almost 2 million models.
So we're gonna use these tools
that kind of curate some of them for us.
All right, so now we've got it.
Hoof, look at how busy this is.
This is way more complicated,
but it's really powerful too.
So let's just hit run with the default prompt
that it gave us.
Prompt execution failed.
Value not in your list.
Okay.
Tutorial, well, there's stuff I could read through here,
but I don't wanna do it right now.
Just see if there's anything quickly jumping out to me.
Looks like the model's loaded there.
I probably need more models.
Another question.
Hey, don't worry.
Chat all you want to.
I'll ignore the questions if you're being too chatty,
but you're good.
The question is, can Comfy UI load fine-tune models?
Let's see.
I think ComfUI can load any model that you want to.
Right?
So they have all these templates here for doing different kinds of things.
But when you go in here, these are all your models that you have.
And you can get any model and throw it in here.
Text encoders, diffusion models, Quinn image is a diffusion model.
That's why it's in there.
So all you do is you get your model and you're going to stick it in one of these folders.
So this is Comfy UI's folder for their models and you put it in here.
It should be able to handle it.
I can't guarantee it, but you can also go look on their website, Comfy UI.
Let's ask Maple.
Can Comfy UI load fine-tuned models?
I'm just going to ask your question directly.
And let's change it over to GPT.
Yes.
Stable diffusion checkpoints.
It can.
All right.
So here's more details for you.
You can read through all that if you want to.
But the short answer is yes, if we trust AI.
Obviously, we have to verify ourselves.
But it looks like promising that you can do it.
So lots of info there.
All right.
Last one to show you is Whisper.
This one might be a little difficult.
I'm going to quit comfy to free up some RAM.
This one might be slightly difficult only because I'm using audio here on the stream.
So I don't know how well this will work, but let's, you can do lots of things in here.
So you can record a meeting that you're on, right?
A lot of people love to have AI join a video call.
It drives me crazy when it's some, this third-party cloud service, because now I know that all
my words are being recorded on some random server that could be susceptible to a data breach or sold
to advertisers or shared with who knows who. So, I'll just be aware of that, right? But you can
record it locally. Why use a third-party server when you can just download an app like MacWhisper,
although verify that this app is not transmitting stuff. You can use tools like Little Snitch and
verify that it's not sending data out to somebody else. I would love to find something like Mac
Whisper that is open source, but this has just been like the easiest thing to use so far. So
that's what I'm using. But yeah, so I can record a meeting. Another thing that I use it for is when
I'm done recording this show is I just drag the audio file into here and it creates a transcript
for me. So that's a, that's a good one. There is this transcribed podcast thing, but what it does
is, let me show you a previous one. Do I have any saved? I usually don't save them.
So, okay. Well, here it's taking one of my previous shows, the MP3s, and it's going to
load it. So, you can see actually what it does here. We're not going to wait for this whole
transcription, but you'll just get the gist as it starts. So, it's currently loading the model
into memory, and now it's got it loaded, and it is starting to transcribe. It's probably going to
take a while because I'm live on the stream. But you can see it's starting to show if there were
multiple speakers, I can go in here and I can add multiple people to this. So Marks is one of the
speakers. Return. Let's say that we had Sally as a speaker, right? So then I can go through and
start. I could like assign them. It would actually detect. So it can detect two different voices
then it will make its best guess and just boom it'll be marks Sally marks
Sally and if for some reason I got those flipped then I just simply change the
names here or I drag them and it'll reverse them all so it's pretty awesome
I love that aspect of it let's delete these here's my transcript as it builds
it here are the segments and then when I'm ready to export this I just let me
back here. I usually do it this way. Export. I'll export it as an SRT file. That's the one I tend
to use. But there's all sorts of options here that you can go with. So pretty sweet. So I
export the SRT and then I just stick that into my RSS feed along with my podcast.
Let's cancel this.
Yes. And so that's also another great comment here from Docstacks. Transcripts from
whisper to summary from LLM into timestamps for your YouTube or X description is a huge time
saver. Totally agree. In fact, I have a massive prompt that I wrote for Freedom Tech Weekend
where I say, you know, here is the transcript of my episode. I want you to output all of these
things, chapters, a show description, give me some tags that I should put into YouTube for this.
I have it generate an image prompt. It doesn't generate the image itself, but I say like
create a prompt that I can give to an image gen AI for making a nice YouTube thumbnail for this,
all sorts of things. So I've got like this eight step process that instead of me doing it myself,
I just take the transcript, paste it in there, and it builds all that for me. And I can do that
all in Maple or locally. It's awesome. One other thing you can do with this that is really helpful
is you can have it do dictation. So like this little screenshot here, this will explain it all.
Most computers do dictation built in like Mac and Windows and stuff, but you're using the like the built in stuff which might go back to their servers.
So if you want to keep a local, you can set up dictation here and then any text box on your screen.
You can just hit a button on your keyboard and the microphone will start and you can start talking to your computer and then it will dictate it for you and process it 100% locally here on your computer using Mac Whisper or some other tool.
I think one is called super whisper if I remember correctly. This is really powerful, especially
when it comes to writing prompts itself. When I'm talking to AI, I can bang it on the keyboard.
Some people prefer that. Sometimes I do. But a lot of times I just hit the microphone button
and dictate to AI and I can speak more like a human because AIs are trained to think like a
human. So when you talk to it in a more familiar way, a more conversational way, then you tend to
to get better results. So dictation is awesome. You can set it up to any key that you want. You
can set up a custom one if you want to. And then what was the thing? Let's try this. I don't know
if it's going to work. Hello, testing, testing, testing. Yeah, it's not working right now because
my microphone is going into the stream. And so it can't do it here. But if I was not streaming
right now with my microphone, then what I just said would show up here in this box,
Just as the test.
And then you can use it anywhere on your operating system.
It'll go into any, it goes into the main level of the operating system.
So cool stuff.
All right.
I think we need to wrap it up here.
We've been going for almost an hour.
We have over 700 people.
Welcome everybody.
Hope you enjoyed today's show.
But we went into a lot today.
Gave you a really good primer in how to do your own AI.
Just real quick.
Like LM Studio is great.
Comfy UI, I just tried out for the first time today for doing image gen and video gen.
And then you've got Whisper for doing audio stuff.
Hugging Face is a great place in general to find models.
Then you have Quen Code, which we put into LM Studio and wrote some stuff for ourselves.
If you want to delete your stuff out of ChatGPT, we can go to OpenAI and request deletion.
And then there are all sorts of blog posts, just like this one that was shared,
for building your own home server if you want something more beefy to chain together multiple
computers because if we want to run these large models for example maple maple runs the new gpt
os 120 oss 120b we also run deep seek the largest deep seek model out there 671b it's massive it
requires huge computing power huge memory to run it you're not gonna be able to do this on your
laptop. So if you want to do bigger things, you can chain them together. All right, let me end
the screen share and let's do a proper send off. So thank you everybody for joining today. Really
appreciate you being here on Freedom Tech Weekend. I feel like the last couple were kind of short and
kind of lame, to be honest. I was traveling and there was a lot going on, especially when you're
doing a startup. Startup, traveling, family, life gets busy, right? No excuses. Let's just, let's do
do the right thing. So I wanted to bring you a much more substantive show today. I hope that I
gave you some things that you can get excited about and try out this weekend. Try out any of
the tools that we did. If you have other ones like ping me, ping me on socials. I'm on X,
I'm on Noster, I'm on YouTube. Hit me up and let me know which ones you like using.
And let's keep the conversation going. And then because I would love to learn from you as well.
And then also get your own private encrypted cloud AI.
Go to try maple.ai, get your free account, upgrade to pro when you need to do more.
And we will talk to you next week on freedom tech weekend.
Hope you all have a great weekend.
Later.
Thank you.