I wrote my first rust code and polars extension - pl_series_hash. It runs xx_hash over an entire series to get a single hash u64. Works on most all Polars data types including nested structs, it's fast. WIll be very useful for caching summary stats. github.com/paddymul/pl_...

Looking at the data with Buckaroo - My talk from PyData Boston is now up on youtube. www.youtube.com/watch?v=Htah...

After a bunch of work with prot, I got my custom find-grep script plumbed into emacs. It searches a python project for matches in preferred files first (not site-packages, node_modules, py, js, tx extensions preferred). Then it runs a much more comprehensive search of those excluded directories

Life pro tip. Next time you have to assemble IKEA furniture, buy yourself a 4mm hex t-handle. and a 4mm wera (or wiha) allen key. So much easier. www.kleintools.com/catalog/t-ha...

UGH, I thought I was so slick. I had a nice function that used polars to convert a csv to parquet with enum columns for sparsely populated columns. I thought it saved about 30% on a 700 meg parquet file (10G csv). Then I dug some more and found out that I was encoding all possible sparse value

Some content is on Youtube and few other places (heavy equipment, dirt bikes...). But I realized generally I have been searching youtube for more topics because I trust the results more than SEO spam... And this is after I switched to ddg. Obviously there are shills on yt, but beats listicles.

There is some good stuff on facebook. Fun post from the mainframers group about an IBM Sytem 370 card for the IBM PS/2. www.facebook.com/share/p/17tx...

On the toughest most annoying problems I need to try to put in a half hour to an hour a day, and let ideas percolate. But actually do that work, it's much more effective than hours of grinding. I'm thinking about devops stuff especially

You hear those stories about how garbage collection on lisp machines in the 80s didn't really work and engineers just restarted the machine once a day. Then I realized I do the same thing with firefox/chrome. Need to upgrade to a 32+GB laptop

I have started heavily typing my python code. I’m late to the party on this one, I know, but here are some thoughts. I was prompted by finally getting eglot mode working with basedpyright in eMacs.

Hit a snag with the multi-index refactoring. The short of it is that dataframe serialization is really tricky and relying on assumption and default behavors will bite you, always

And I now have pandas MultiIndex columns rendering nested locally.

Bahhhh. Traitlets doesn't like typing. I'm trying to add types to my project, and I have written most of them, but my project heavily uses traits, and they don't get along with python typing

I'm kneedeep in adding proper multi index-column support to Buckaroo. MultiIndexes are kind of a corner case of pandas, super powerful but often forgotten. Many tables don't display them properly because they are tricky. They don't serialize to JSON natively. #python #pandas #datascience #pydata

So, apparently SRAM NX 1x12 isn't a good fit on a 20 inch wheel. The RD hits the tire in the 3rd largest gear. #cargobike #bike

I have been playing with beartype. It's a library for fast runtime type verification in python. The maintainer is one of the most entertaining writers I have seen. Period. His response to my feature request/(trying to understand his library) has brought so much joy. github.com/beartype/bea...

#Cargobike wheel build complete. Excited to try this tomorrow

I made a new Buckaroo general overview video. youtu.be/t-wk24F1G3s (6m50s). It demonstrates the main features of the Buckaroo table UI. #python #jupyter

Buckaroo Autocleaning released. Dataframe columns are inspected and cleaning operations are suggested. Different strategies can be interactively cycled through. All of this is extensible. Because it uses heuristics and not LLMs, everything is local and fast. www.youtube.com/watch?v=A-GK...

Just had my first emacs coaching session with prot. I have used emacs for years and I'm at a local optimum. So excited to have someone to work through and intentionally get better at the program I live in. protesilaos.com Do any other programmers have coaches?

There is cad vs cad esports. If you are a UI/UX designer, pay attention to how efficient experienced cad users are. Even better, This videos shows a competition measuring the mass of a complex design. Open to any program (Solidworks, NX, Catia) www.youtube.com/live/C1CqIcf...

I've been writing a #Clojure compiler in JavaScript, and I just benchmarked it with my NES audio emulator and it clocked at over 60fps which means it's fast enough to run in real time. Compiler source: codeberg.org/bobbicodes/n... Emulator: codeberg.org/bobbicodes/n...

Autocleaning for buckaroo is really shaping up. This is the JLisp code editor showing operations that were added by an auto-cleaning strategy in purple. Multiple strategies are available and they can be cycled through quickly to try different approaches. #buildinpublic

There is a whole genre of youtube videos explaining how the @mcmastercarr.bsky.social website is so fast. Love the company, love the attention to non react framework of the day JS stuff. www.youtube.com/watch?v=-Ln-...

Playing with lisp, building mini languages and frameworks is so much fun. It brings a sense of wonder.

I'm having fun improving my little lisp interpreter. Writing a couple of macros. Added gensym and symbol-value. Gensym was easy, symbol-value and getting it to work in define was much more complex.

I made a styling gallery for Buckaroo on @marimo.io There are a variety of configurations displayed with narrative explanation, a live widget, and the exact configuration code. Play with it in the browser thanks to the Marimo WASM version. marimo.io/p/@paddy-mul...

Cleaning up my home directory. Pruning .git's saves a bunch of space find . -type d | grep \.git$ | xargs -I{} sh -c 'cd {}/..; time git gc --prune=all'

Just stumbled onto VTable from bytedance. It looks like a pretty serious JS Table offering. Possibly a competitor to ag-grid. github.com/VisActor/VTa...

Marimo is working hard on making notebooks better. The team has a vision for a better notebook environment and they are executing on it, in the open.

Buckaroo now works on marimo. marimo.io/p/@paddy-mul... . Sometimes there are benefits to supporting extra platforms. I found a bunch of performance problems because of marimo, that now benefit all notebook environments.

I have been working on a dataframe diffing tool built on top of buckaroo and datacompy. Here is a preview. Source here github.com/paddymul/buc...

I saw the office for www.motorvalleyaccelerator.com/en/index.htm... While walking through Modena. Interesting to see startups around cars/racing in Italy

To give an example of what I was talking about in this #datavis #typography thread: this figure uses at least seven (just counted!) different weights of Source Sans Variable to both create visual hierarchy/emphasis and to maintain even visual texture

@chittimarco.bsky.social can you recommend any urbanism/transit tours in Florence, Milan, or Genoa?

Ahh, so frustrated. trying to watch the race from italy first #f1tv acted like the pre-reace show was in progress, with no option for the race. Tried logging out and back in, found out race is geo restricted (even with a VPN). Poor UI, and hard to help yourself w/goog when you don't want spoilers

I think this may be the new hardest problem in computer science - this is very comprehensive, I added some of my own commentary here simonwillison.net/2024/Nov/23/...

I mostly plan to post about python, data analysis and what I’m building with Buckaroo for jupyter. Currently though, I’m on vacation in northern Italy. Excited to figure out bsky.

Just found this cargo bike list. go.bsky.app/33Xj6wT

Did the Ducati Factory tour today, pictures prohibited 😔. Stood on the floor feet from workers on the 2 engine assembly lines. New line has 36 stations, 5 minutes each, 2 rest stations. Crank/cam machined in house, all else sourced. Amazing to see fixturing and lean manufacturing in person.

Just trying to keep track of starter packs I have added. bsky.app/starter-pack...

I found this Apollo book from the 70s wit what looks like computer generated charts. I'm curious what charting software existed back then. #dataviz

@mkennedy.codes What kind of bike are your riding in your profile picture? Does it have an engine?

I have to update the error messaging, because the next release will add backwards compatability with notebook 6.5.4 thanks to anywidget.

I'm getting into bluesky through starter-packs, it's a great concept. I wish the app kept track of which people I'm following came from which pack. I don't even remember all of the starter packs I have visited

I started working on Buckaroo just before @manzt.sh made AnyWidget. Between that and IPYReact from @maartenbreddels.bsky.social I probably would have saved 4-8 weeks of JS Packaging debugging and errors.

Thanks @lernerpython.com Excited to see more of the python community here! go.bsky.app/M5WjB4s

@brettkollmann.bsky.social Happy to see you here. Have you seen a football starter pack for bluesky?

I just got to the bottom of a very complex frontend rendering problem with #svg recharts, and #jupyter. It looked like a browser bug, but happened in both #chrome and #firefox. SVG elements with patterns would initially render normally, but subsequent charts would have no pattern, only an outline

Dev question, who has the best #github release notes and changelog. I'm looking to spice up buckaroo releases and announcements. #devrel #python #typescript