Warning‼️ 🚨 This is now an FPGA & GPU complaint account‼️ Q4 will be mostly skill issue posts.

Coming down with the sickness again, maybe there are codebase visualizations that are useful. Interactive with bidirectional editor integration and runtime analysis. Apologies for the slop prototype.

The VGGT vision transformer model couldn't really handle mixed time of day images, but this new Apache-2 licensed model is a big step up. Variable lighting and weather really mess with existing VPS solutions. A popular one needs two scans at least 5 hours apart. map-anything.github.io

happy tom7 day to those that celebrate. www.youtube.com/watch?v=QH4M...

Butlerian Jihad at home. 2010s programming just to feel something.

Got the mesh registration to run in ~100ms, down from 400ms, by fusing the kernels and moving the correspondence work to the gpu. The extra time is from deserializing the test data. bsky.app/profile/v1se...

The VGGT model works as a VPS with only a few images. One attention head estimates the 6dof camera pose of each image. Here is a rough example with 7 warped Mapillary images and 1 XR headset camera image in red. While the error is about 2m, you couldn't do this at all with SfM.

Making block worlds spherical comes with a lot of trade-off. I think this person made some smart compromises such as the vertical block layers. For an AR world-wide block game this is great as all worlds can use the same earth centered origin.

I got some very promising 3d sidewalk point cloud data from unwarping Mapillary data and feeding it to VGGT(Visual Geometry Grounded Transformer). The output seems much better than traditional Structure from Motion + Multi-View Stereo dense reconstruction.

I think I've hit the limit for GPS accuracy in the city for AR positioning. One path forward could be doing mesh registration on a 3d model of the city. Maybe by running Structure from Motion on the street level Mapillary data. Maybe combined with USGS airborne Lidar.

Demo: while you were living in sin I was studying the GNSS.

Alleyway GPS sensor fusion test. Head position in white. GPS in red. Fused state in green. It is tuned for open fields and forests, but maybe with a bit more math this could hit sub meter accuracy. At this scale the odometry is very accurate.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

I was surprised to find NOAA publishes the 3d weather forecast data for the continental US + Alaska. This appears to match observations of nearby mountains. Very little smoke today. I get why hiking apps don't provide search on this data now :), but there might be a way.

Would anyone actually try a free outdoor navigation app for the vision pro? The tech works in the mountains, but it is too dangerous to recommend outside flat parks. My initial tests were putting a 10m tall pushpin at a gps coordinate and keeping it there for miles.

Approved, ping me for a copy or if you have any issues. bsky.app/profile/v1se...

Demo showing Claude building a race car with voxels. The UI is all buttoned up now, hopefully app review approves this feature.

Today was such a beautiful summer day for a hike according to my database.

AI agent that commutes around on the real-time transit data. posts about missing a transfer. call it Claude Commute.

Working on a little website to find hikes. One feature is filtering by weather forecast. Very thankful for the open data from NOAA and the USGS. All the blue dots are the real-time positions of nearly 1000 buses and ferries.

Claude and I friendmaxxing at the beach.

This is claude-sonnet-4-20250514 building a sand castle via the CRDT ops after 16 seconds of thinking.

Little faster. Now it does 4296.3 CRDT ops/sec (6x faster) and chunk meshing is 192μs (78x faster)

WIP: performance test for the CRDT. Getting around 637.9 ops/sec with intermittent state compaction.

I can't put much more time into this project. One goal was to get a job working in XR. This did not pan out. I'm grateful for the support I've received here. Thank you.

I think I can reuse the front end of the mesh registration system to detect when some mesh chunks have enough geometric detail to be found in the future. If this works well then I can automate the fine grain (<10cm) geospatial localization. Sunk cost fallacy is my moat.

Moved ISS to the GPU to join FPFH, now the full ISS + FPFH + TEASER pipeline runs in under 400ms for typical environments on Vision Pro. Red mesh is the prior saved one we are searching for in the blue mesh.

Due to economic headwinds beyond my comprehension, I'm pivoting to LLMs. I'm adding an LLM building assistant to my voxel editor. To support this the data layer is now a CRDT. Notice how only the llm edits are rolled back.

got SharePlay integrated for volumetric windows, working on testing immersive spaces before shipping. I was able to abstract over the hub-and-spoke tcp/udp transport and the SharePlay peer-to-peer transport. Due to better user experience SharePlay gets priority.

IVAS at home. Another ads-b tracked takeoff displayed in the vision pro. This feature seems worth polishing up and shipping.

Testing: real-time ads-b overlay via pi->(ble)->vision pro. Soon my app will offer Full-Spectrum intelligence of the urban theater.

I added 70 solid colors to the palette. My next priority is shipping SharePlay and making the outdoor modes more intuitive.

Debug vis from walking around alleyways with visionOS. GPS in red, odometry(head position) in white, fusion in orange/green. The main issue is deriving the north heading without a compass. Urban areas have significantly worse gps noise making state convergence hard to maintain.

WIP: bus-ray-vision. Building on my gps localization system for visionOS, I render the real-time position of all Puget Sound transit. Seems more like a novelty than worth shipping tbh.

where the bus is tells you something about where is wasn't and where it won't be. send microblog

Someone(not me) will make this app once visionOS, AndroidXR, or HorizionOS ship simple geospatial sdks. A very basic version could be built in a weekend and could be advertised just by being controversial. The camera apis being locked down make the compositing and occlusion the hardest part.

Trying to avoid building bus-ray-vision, an XR app showing real-time transit data with geospatial perspective. Know bus-fu. IVAS at home. Full-Spectrum bussin. Transit Omniscience etc

tired: deleted and replace blocks wired: push em

I've published the first version of the ClassiCube port to visionOS. The in game wrist menu is not complete, but you can place and delete blocks. It adds a Metal render backend and then bridges to the AR Compositor. github.com/v1sea/MacroC... I might try to get Bevy to support this next.

I'm considering adding my custom map renderer with the voxel world overlay(needs a bit more polish). I'm also actively hunting for a backend or AR software role in the Seattle area. DMs are open. Thank you.

Wrist compass is now exposed in public build, so if you walk 100 feet or so it will appear. It does long distance drift compensation, handles OS origin recenters, recovers from tracking loss. It is mostly an AUKF fusing odometry and GNSS from a phone anywhere on you..

Demo: 2025 starter home. I've shipped a ~4x faster version of the relocalization pipeline by moving the most costly component to the GPU for a typical time reduction from 3s to 100ms.

I released version 1 of my voxel editor for the vision pro. Here are some codes if anyone would like to try it. THH3PHL6LXKL F9NF93XP7LPM 7EF49M7X6ETJ RHY47PWLPR9P H64AKJK373AP KMX7P73YPTXK YKE7FF4EYN6X JPM9JLM9YE3F J4HAWRRPKH93 J3YFT66NXW7R apps.apple.com/us/app/cubic...

new menu, there when you want it.

I added an input reconciliation system to the UI to reject/delay block placement if the UI was later determined to be targeted. SwiftUI buttons have more latency than the hand tracking data. After release I'll write a blog post to fully explain the latency optimizations.

Demo: Multiplayer server for my voxel game. Block edits are transmitted via udp for low latency.

real eyes, relocalize, real tris.

Field Test: Mesh based relocalization running on device(vision pro) and offline. Blue is the current environment meshes, Red are the prior meshes we are searching for.

I ended up writing a voxel renderer from scratch. I'll open source the ClassiCube port soon. Demo shows world scale editable voxels composited with the environment mesh. It also supports editing in volume views.

ARKit WorldAnchors fail ~98% of the time in unbounded outdoor use after traveling ~300 feet. To overcome this I've been testing mesh relocalization algorithms that handle partial overlaps. Results are very promising and should be very fast once ported to gpu compute kernels.