There's literally nothing you can't compute with matrix multiplications and point-wise nonlinearities: they are Turing complete, as we've known since the 90s.
There's lots of problems with LLMs, but none of them come from matrix multiplication. Why do some people keep saying that?
add a skeleton here at some point
18 days ago