At Imagine Innovators, we just announced our Developer Plan, offering expanded access to Edge Impulse’s powerful tools, completely free! Get GPU training, longer runtimes, more projects, and production-ready licensing, all at zero cost. Details: www.edgeimpulse.com/blog/introdu...

have moved mixing out of the rack to the model1. such a great performance focussed mixer! #improv #techno

It's about time I updated my recommended #FPGA links. Send me a suggestion this #FPGAFriday. 😊 projectf.io/recommended-...

i wish this was always an option...

still "finding" ( i.e. stepping on ) screws from the great M3 drawer spill of Sept '24

I'll take "n^2 - n cursed images" for $1000, Alec

Great interview with @jascha.sohldickstein.com about diffusion models! This is the first in a series: similar interviews with Yang Song and yours truly will follow soon. (One of these is not like the others -- both of them basically invented the field, and I occasionally write a blog post 🥲)

Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n

"In what part of the world do you currently live?" lol

keep coming back to "Scalable MatMul-free Language Modeling" arxiv.org/abs/2406.02528 full of fun ideas! am slowly mapping my FPGA audio rate cached casual dilated convolutions network to operate using their mat mul free GLU version of self attention. slowly getting there!

ooffff! the "DeepSeek-V3 Technical Report" is info dense! arxiv.org/abs/2412.19437

still my fav "at doom's gate" cover! ( i'm probably 1K of those 2M views ) www.youtube.com/watch?v=TZG1...

this looks like it's going to be a good read! www.amazon.com/AI-Robotics-...

went camping in an area with lots of eucalyptus and was surrounded by koalas! so cute!

first logic gate waveshaper results tonight! this is network represented purely by combinational logic ( see arxiv.org/abs/2210.08277 ) i.e. just look up tables; no mults or adds. trained in jax ( 3 nested levels of vmap! ) & inference in amaranth. inference speed should be ~10s nanoseconds.

my largest category of splurge spend for last month was "bandcamp" :D #winning

and here i am trying to debug why param_set((2,3,16), seed=1) vs param_set((2,3,16), seed=2) isn't working... ::/

by a lot of measures this was a completely irrational purchase, but, you know what i like it! it's ok to be irrational some times.

started learning "Amaranth HDL" this morning. my prior FPGA neural net stuff was verilog but switching to amaranth since the match the gateware on a device i'm targetting. as someone with a python background it's 1e6 times nicer than verilog!!

this board looks really fun! "a small, low cost board with the Raspberry Pi Pico processor (RP2040) and a Lattice Semiconductor iCE40UP5K FPGA" pico-ice.tinyvision.ai

"Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms". interesting approach using 2nd order info to avoid the exploding/vanishing gradients that come from non differentiable relaxations.. www.youtube.com/watch?v=7aFP...

this is super cool!! i'll definitely be porting my fpga wavenet ( matpalm.com/blog/wavenet... ) in the new year

still the best tech speech ever. "whatever path you're on, i advocate for long, hard work." so, so true. knowledge & skills do not just pop into our heads, we have to push and shove and jam it in ourselves. www.youtube.com/watch?v=YOZn...

i always have this weird compulsion to move back to seattle. no idea why, given how bad the weather was lol

Today STMicroelectronics @ST_World announced the incredible new STM32N6, a powerhouse microcontroller with built-in neural processing that achieves 600x performance over current high-end STM32 MCUs — and offers native Edge Impulse support. Press release: www.businesswire.com/news/home/20...

love the arpeggiator on the oxi one, so playable and fun! #improv #eurorack #techno www.youtube.com/watch?v=mbpF...

my 2025 goal is to play a hardware only improv techno set in a full club have played a few times in smaller settings around friends but hearing one of my tracks recently on a proper club PA has really inspired me! dithernet.bandcamp.com/album/chapte...

i love seeing people ( myself included ) rediscover old things. the recent flood of interest in AI which, for a lot of people, is 100% LLMs, is the best case i've seen of it! it really helps me solidify my only internal concepts to see someone else explain a decades old idea through new terminology.

"the new iris dataset" i'd almost forgotten about this!

the dog always waits. the gate in spring is shaded. she will run like hell. #haiku

i can't repost this hard enough...

keras3 vmapped jax models to tflite via orbax matpalm.com/blog/keras3_...

sadly none of it "just works" with a keras3 jax model ( exported as stateless_function ) #sigh no doubt i've just "simply" got my `jax_module.methods[constants.DEFAULT_METHOD_KEY]` wrong or some other magic arg combo, but not sure i can be bothered today...

stuck a jax laptop sticker up in a nightclub bathroom friday night. hoping some spaced out dev has a 3am WTF moment...

#showYourSetup #eurorack

(this week) jax to tflite is done with orbax ( nightly build only) ai.google.dev/edge/litert/... ( and that's orbax-export, not orbax-checkpoint, since it's been split ) not sure what'll be next week, we'll see! #canYouKeepUp

whenever i see someone talk about a small 1,000,000,000 param LLM i think about the 20,000 param conv nets we've got deployed on microcontrollers. TBH am glad to not be in the LLM space right now

Edge AI is maturing, and we’re getting pretty good at sensory perception. What comes next? In this post I explore what we’ll need to bring real intelligence to the field. I’d love to hear what you think 👇🏽 open.substack.com/pub/dansitu/...

have a number of old blog posts that are still useful but have code that no longer works... i know i should update them, just hard to be motivated to do so :/ #lazy

oh! just had an idea! in my fpga waveshaper ( matpalm.com/blog/wavenet... ) the limiting factor for speed was the ADAC which ran at 196kHz ( 5 micro second inference ) ...🧵

updated my yolz blog post ( you only look zero shot times ) matpalm.com/blog/yolz/ still scrappy and hard to read i think but my time boxed hacking allocation has run up. onto something else!

it's time to celebrate when you reach 6D inputs tensors !

sometimes a jax.vmap can cause a MASSIVE blow out in jit time... i wonder what tools are available to understand why? ( it's not really profiling and jaxprs get pretty complex pretty quick... )

The latest edition of Make Magazine is out and includes an article by yours truly 🤓 I talk about the collaboration project with @Odd_Jayy and give you steps to create your own local #LLM chatbot on a #RaspberryPi. Check it out: www.makershed.com/collections/... #maker #ai #electronics

it's _so_ important to write throwaway sampling debug tools. here's one for my zero shot object detector with top row being best performers and bottom row being worse. you never quite know what little quirk you'll pick up reviewing these...

after a couple of months of crappy weather it's about time to get jumping again!

latest small drone setup. super simple but fun.

the AQT lib github.com/google/aqt/ is one of the few libs that does full quantised training ( not just quantised _aware_ training ) but it's implemented via a substitution of the dot_product? isn't jax all about fn transforms? why can't this "just" be a rewrite with tracing? i.e. `quantise(fn)`