Soon the NVIDIA Jetson platform can power your LLMs for Home Assistant!
Excited to have Ollama running onboard Jetson! It's super easy to get started with, and you can download & run practically any LLM in just a couple basic commands. This came out of the Jetson AI Lab Research Group, which has seen a flurry of activity - thanks to everyone for all your contributions! 😊 Forum post with the details: https://lnkd.in/gcUXhc3a The next community meeting is next week, and there has been lots of progress on Home Assistant and Piper TTS to discuss too! https://lnkd.in/giGHBebQ
Ollama is awesome. Should also be able to run the 8b version. How much GPU RAM does the new Nano max out at?
Oh, no way. I JUST got an Orin NX module for my cluster, this might be really fun to play with.
Senior Software Engineer, Platform/Infrastructure
1dIt's 16GB. It can do llama2 fast or llama3 slowly. I made the computer from Wargames as a chatbot and I'm gonna wire it up to my telnet BBS.