Hello, Gemma! - Part 1: The Build

Hello, Gemma3n!

Jetson + MatFormer + PLE Caching + Audio Input

Bringing Google latest open-source model, Gemma3n, to NVIDIA Jetson Orin to enable on-device, live audio chat!


Google Gemma 3n is nothing short of incredible! In addition to incredible multi-modal performance in a tiny, efficient package, they also managed to add multi-language audio input!!

 

Design Considerations: 

  • Python + transformers + torch
    • Audio input is so new, we'll have to leverage the latest tranformers package from HuggingFace to leverage it.
  • jetson-containers
    • To make things easy, I'll build a container with the latest transformers for the Jetson using jetson-containers
  • Piper for efficient on device text-to-speech 

 …video coming when I get a chance shrink it…

Full build and details on GitHub: GregariousEngineering/hello-gemma 

Up Next!

  • Wake word and query completion detection
  • Internet access
  • Remote LLMs 


Comments

Post a Comment

Popular posts from this blog

Turn Off Those Super Annoying Autoplay Ads in FireTV

Image
Bad Decisions: In moves seen aimed at ensure I finally switch devices, in addition to recently adding ads to Prime, Amazon decided to start autoplaying ads as soon as the device turns on. Buried Settings: Fortunately, this can be disable under Settings > Preferences > Featured Content. While in Preferences, I stopped in Privacy Settings to turn Off the slew of tracking On by default. Repeated Pain Sadly these settings are per device, not account, so you’ll have to make them on every device.
Read more

Hackles Feedback Hypothesis

Image
Hackles Feedback Hypothesis “The  facial feedback hypothesis , rooted in the conjectures of  Charles Darwin  and  William James , is that one's facial expression directly affects their  emotional  experience. Specifically, physiological activation of the facial regions associated with certain emotions holds a direct effect on the elicitation of such emotional states, and the lack of or inhibition of facial activation will result in the suppression (or absence altogether) of corresponding emotional states.” https://en.m.wikipedia.org/wiki/Facial_feedback_hypothesis I ask, does smoothing or raising a dog’s hackles affect their emotional experience??
Read more

Change these “Off-Facebook” Facebook Privacy Settings

Image
Change these “Off-Facebook” Facebook (and Instagram) Privacy Settings New Settings! OK, perhaps not new. It’s been many years since I checked any social media regularly and I should really just delete my accounts, but like many, I leave them collecting dust… But then, over the weekend I received a slew of recovery tokens for my Facebook account, which prompted me to head over to Settings to check activity and enable MFA. While there I noticed the new settings, buried deep as you’d expect. I am genuinely ecstatic about these settings. Their defaults leave nearly everyone open to widespread invasion of privacy, but they still have the settings available. I suspect we have the ever increasing barrage of new consumer privacy oriented laws from cities and states (e.g. California Consumer Privacy Act) to country unions (GDPR). Apply these in addition to standard privacy settings, as described well in Consumer Reports’ How to Use Facebook Privacy Settings . They also apply to connected Instag...
Read more