Is basic image understanding solved in today’s SOTA VLMs? Not quite. We present VisualOverload, a VQA benchmark testing simple vision skills (like counting & OCR) in dense scenes. Even the best model (o3) only scores 19.8% on our hardest split.

Here’s what I’ve been working on for the past year. This is SkyTour, a 3D exterior tour utilizing Gaussian Splat. The UX is in the modeling of the “flight path.” I led the prototyping team that built the first POC. I was the sole designer and researcher on the project, one of the 1st inventors.

🚀🚀🚀Announcing our $13M funding round to build the next generation of AI: 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 that can generate entire 3D environments anchored in space & time. 🚀🚀🚀 Interested? Join our world-class team: 🌍 spaitial.ai youtu.be/FiGX82RUz8U

📺 Now available: Watch the recording of Aaron Hertzmann's talk, "Can Computers Create Art?" www.youtube.com/watch?v=40CB... @uoftartsci.bsky.social

Here's a recording of my 3DV keynote from a couple weeks ago. If you're already familiar with my research, I recommend skipping to ~22 minutes in where I get to the fun stuff (whether or not 3D has been bitter-lesson'ed by video generation models) www.youtube.com/watch?v=hFlF...

A thread of thoughts on radiance fields, from my keynote at 3DV: Radiance fields have had 3 distinct generations. First was NeRF: just posenc and a tiny MLP. This was slow to train but worked really well, and it was unusually compressed --- The NeRF was smaller than the images.

Here's Bolt3D: fast feed-forward 3D generation from one or many input images. Diffusion means that generated scenes contain lots of interesting structure in unobserved regions. ~6 seconds to generate, renders in real time. Project page: szymanowiczs.github.io/bolt3d Arxiv: arxiv.org/abs/2503.14445

I made this handy cheat sheet for the jargon that 6DOF math maps to for cameras and vehicles. Worth learning if you, like me, are worried about embarrassing yourself in front of a cinematographer or naval admiral.

Can someone point me to a video of renderings of some 3DGS-like algorithm *as it is being optimized*? I want to see all those little Gaussians wobbling around.

Next week is the one year anniversary of this paper showing that videos generated from Sora are nearly 3D-consistent. I'm surprised we never saw any follow-up papers in this line evaluating other videos models this way, it would be helpful to track these metrics over time. arxiv.org/abs/2402.17403

Veo 2 now has a public price point: $0.50 per second. Very important number to keep in mind when considering the future of generative and non-generative media. Taken from cloud.google.com/vertex-ai/ge...

To quote @jonbarron.bsky.social.

I just pushed a new paper to arXiv. I realized that a lot of my previous work on robust losses and nerf-y things was dancing around something simpler: a slight tweak to the classic Box-Cox power transform that makes it much more useful and stable. It's this f(x, λ) here:

My pitch for an "LLM-native" alternative to citation count/h-index etc: 1) Train an LLM on a new paper and record the average loss during training. 2) Evaluate the retrained LLM on benchmarks. Your Google Scholar records avg_train_loss * benchmark_delta, and you go write your next paper.

Great interview with @jascha.sohldickstein.com about diffusion models! This is the first in a series: similar interviews with Yang Song and yours truly will follow soon. (One of these is not like the others -- both of them basically invented the field, and I occasionally write a blog post 🥲)

With the CVPR 2025 rebuttal deadline over, it’s the perfect time to submit a demo application for CVPR 2025. Demos can be any application of computer vision in the real world and are a great way to show off your work to other #computervision enthusiasts. Submit here docs.google.com/forms/d/e/1F...

fun test for image and video generation systems: add "the camera is upside down" to the prompt (especially for shots of people) and then vertically mirror the output. Even the best models struggle, with upside-down teeth and blinks, and gravity tugging everything up. Here's Veo 2.

I fed the "spinning dancer" illusion (a silhouette of a spinning figure that can be seen as rotating clockwise or counter-clockwise, left) into Runway Gen-3 (right). It resolved the ambiguity by having the dancer face the camera and oscillate, which is kinda clever.

`A 1960s NASA scientist with a white button down shirt and black heavy rimmed glasses, with a giant thick alien umbilical cord coming out of the back of his body. The cord is holding him up in space, and he is levitating around his office. Wide angle, full body shot.` #Veo2

"A fun children's educational program where Mr. See-thru teaches kids about how the gastrointestinal system works using his semitransparent abdomen." #Veo2 I'm surprised by how well Veo 2 understands human anatomy, and amused by the things that it doesn't yet understand.

I have been having fun turning the Mario Brothers into a 1940s industrial film using Veo 2.

In case it's helpful to others writing papers or doing comparisons, I'm hosting raw mp4s for the #Veo2 results I've been posting, with no Bluesky-induced compression: drive.google.com/drive/folder.... Here's `A kraken emerging from the ocean at the beach on coney island`, which I forgot to post.

`An advertisement for a shoe made of broccoli romanesco, rotating on a turntable` #Veo2

Going to put "Towards" in my title, you can't stop me.

Happy Hanukkah! "A man using an upside-down Hanukkah menorah as a jetpack, blasting off" #Veo2

"A colorized 1950s TV commercial. Left: a dad struggling to pick up a heavy box. Right: that dad wearing a full-body mechanical exoskeleton, lifting the box with ease." #Veo2

"A giraffe riding four skateboards, one for each foot" #Veo2

Happy Hanukkah! "A row of industrial robot arms grabbing latkes from a conveyor belt and dunking them into a row of bowls of sour cream" #Veo2

It's been a month and this website still isn't nearly as good as the other one.

"Tilt-shift footage of Woodstock 1969" #Veo2

"A banana in a bodybuilding competition, striking different poses with its floppy little peel-arms" #Veo2

"A person putting on a hat that is a giant strawberry, and then the person grabs a chunk of it off their head and eats it" #Veo2

"A woman using a hairdryer, except the hairdryer is shooting a fountain of rainbow sprinkles onto her head and hair" #Veo2

"A giant millipede who just escaped from the zoo and is wreaking havoc in downtown manhattan" #Veo2

"Aerial footage of someone flying a wingsuit through Times Square in Manhattan, in 1974" #Veo2

"A canopy bed galloping across the savanna" #Veo2

I tried to render what an o3 prompt for ARC-AGI might look like from an LLM's point of view. A hard way to ingest a grid of integers!

okay I've had it with people calling a collection of 3D Gaussians "a splat". It's ungrammatical! The proper term for a group of Gaussians is "a carl", named of course in honor of Carl Friedrich Gauss. A murder of crows, a pod of whales, a carl of Gaussians. Got it? Sheesh.

Every time someone calls a collection of 3D Gaussians "a splat" a radiance field angel loses its wings.

Image: "A pig sitting in front of a slot machine" Motion: "The arm of the slot machine moves down, causing gold coins to come bursting out of the bottom of the slot machine, showering the pig with coins" #Veo2

Has anyone figured out what genesis-world.readthedocs.io is or does yet? I still don't get it.

`A tracking shot following a futuristic personal submarine blasting through an cybernetic underwater utopia of glass domes and tubes and sparkling lights at high speed.` #Veo2

`A hamster klezmer band` #Veo2

`An elderly man in a suit bursting out of an egg` #Veo2

Is there an LLM service that improves the HTML of a website to make it render better across different resolutions and platforms? I want an LLM agent that tidies some HTML, renders a page into virtual mobile and desktop images, looks at the renderings, and repeats.

Spotted this in the wild. I know it kinda doesn't make sense, but CVPR 2025 submissions can't be publicly stated as such on arxiv or on your personal website. If you wrote "in submission to CVPR 2025" somewhere on the open internet, you should delete it ASAP.

If you ask Grok 2 to generate an image of yourself, it appears to look at your Twitter profile picture, summarize it into a text description, and then generate an image from that text. It will then tell you the image was just a guess based on generalities about your tweets.

The popular Max/MSP UI for procedural audio generation is an interesting glimpse into one potential future for image/video generation interfaces like ComfyUI. Having a time-axis is gonna make things a lot more complicated. cycling74.com/products/max

I'm doing another AI Art Gallery with @cvprconference.bsky.social in Nashville this year 😍🤖🎨 Submissions for artworks using / about computer vision are open until 9th March 2025 More details: bit.ly/CVPRAIArt25 #CVPR2025 #creativeAI #AIart

Is this really the correct citation for the Gaussian distribution? Why is the citation count so low?