Wednesday, July 16, 2025

Hello, Gemma! - Part 1: The Build

Hello, Gemma3n!

Jetson + MatFormer + PLE Caching + Audio Input

Bringing Google latest open-source model, Gemma3n, to NVIDIA Jetson Orin to enable on-device, live audio chat!


Google Gemma 3n is nothing short of incredible! In addition to incredible multi-modal performance in a tiny, efficient package, they also managed to add multi-language audio input!!

 

Design Considerations: 

  • Python + transformers + torch
    • Audio input is so new, we'll have to leverage the latest tranformers package from HuggingFace to leverage it.
  • jetson-containers
    • To make things easy, I'll build a container with the latest transformers for the Jetson using jetson-containers
  • Piper for efficient on device text-to-speech 

 …video coming when I get a chance shrink it…

Full build and details on GitHub: GregariousEngineering/hello-gemma 

Up Next!

  • Wake word and query completion detection
  • Internet access
  • Remote LLMs 


at
Labels: , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

BrainBit Flappy Bird! No Coding Edition

Gemini 3 Pro + Antigravity Google released a multi-model that excels at agentic coding, and a new IDE to go with it! Naturally, I need to te...

  • "Real" Spinach & Arugula - The Plan
    "Real" Spinach & Arugula Part 1 - The Setup The Problem We're spoiled with dark, rich, fully savoyed spinaches and spicy,...
  • (no title)
    Upcoming Posts:  29er Bike Tires!  iPhone Privacy Settings!  Ultra-High Performance (UHP) Tires for Colorado!  STOP USING PESTICIDES!...
  • iPhone Privacy Disaster (Mostly) Mitigated
    Not So Hidden Tracking ID(s): For years I've been periodically resetting the evil tracking identifier, included in every request fro...
  • Hackles Feedback Hypothesis
    Hackles Feedback Hypothesis “The  facial feedback hypothesis , rooted in the conjectures of  Charles Darwin  and  William James , is that on...
  • Blog Moving Eventually...!
    This blog will eventually move to a less intrusive host (than the world's largest advertiser). I apologize for my laziness...
  • Apple Privacy Feature?!
    How to quickly disable Face ID (until unlocked) Hold Volume Up + Side Button until Shutdown/SOS screen appears, then hit Cancel. “Your passc...
  • Hello, Gemma! - Part 1: The Build
    Hello, Gemma3n! Jetson + MatFormer + PLE Caching + Audio Input Bringing Google latest open-source model, Gemma3n, to NVIDIA Jetson Orin to e...
  • Llama Data Science 🦙🦙🦙
    🦙🦙🦙 Llama Data Science🦙🦙🦙    I created a simple Python command line tool that leverages Ollama Client and llm-sandbox to test how well...