Shayon Mukherjee
@shayonj.bsky.social
π€ 524
π₯ 244
π 39
Everything, everywhere - always in progress πββοΈπ¨π½βπ³π»
For funsies
www.shayon.dev/post/2025/27...
loading . . .
An MVCC-like columnar table on S3 with constant-time deletes
A thought experiment in building Parquet-like columnar table on S3 with row-level deletes using conditional writes, tombstone files, and a single-object transaction pointer.
https://www.shayon.dev/post/2025/277/an-mvcc-like-columnar-table-on-s3-with-constant-time-deletes/?t=1
7 days ago
0
0
0
βοΈ
15 days ago
0
0
0
Bypass PostgreSQL catalog overhead with direct partition hash calculations
www.shayon.dev/post/2025/22...
loading . . .
Bypass PostgreSQL catalog overhead with direct partition hash calculations
Eliminating PostgreSQL catalog traversal overhead with local partition calculations for up to 20x faster hash partition queries.
https://www.shayon.dev/post/2025/221/bypass-postgresql-catalog-overhead-with-direct-partition-hash-calculations/
2 months ago
0
2
0
This is what progress looks like
3 months ago
1
0
0
"The paradox of artificial general intelligence might be that the more we chase it, the more we appreciate what we already have: human intelligence, in all its messy, creative, contextual glory. And perhaps...."
www.shayon.dev/post/2025/17...
loading . . .
Is AGI paradoxical?
If AI learns from human intelligence, can it ever truly transcend its origins? Is AGI a technical milestone or philosophical mirage?
https://www.shayon.dev/post/2025/172/is-agi-paradoxical/
4 months ago
0
2
0
Good morning HN!
add a skeleton here at some point
4 months ago
0
3
0
Fresh off the press - some raw thoughts on "Premature Closure" in the world of AI.
www.shayon.dev/post/2025/16...
loading . . .
Pitfalls of premature closure with LLM assisted coding
When LLM models generates clean, professional-looking code, it's tempting to stop exploring alternatives. But therein lies the risks that comes with premature closure. So what is premature closure?
https://www.shayon.dev/post/2025/164/pitfalls-of-premature-closure-with-llm-assisted-coding/
4 months ago
0
2
1
reposted by
Shayon Mukherjee
Tines Engineering
4 months ago
loading . . .
Why We Moved Our ECS Services from Fargate to EC2 | Tines
At Tines, we migrated our ECS services from Fargate to EC2 to improve performance, reliability, and flexibility. Hereβs what we learned.
https://www.tines.com/blog/migrating-ecs-fargate-ec2/
0
0
2
reposted by
Shayon Mukherjee
Tines Engineering
5 months ago
Engineering teams know all too well how hard on-call can be. At
@tineseng.bsky.social
, one tool is critical to how we make it easier: Tines. Izabela from our team shares how we do it:
www.tines.com/blog/tines-a...
loading . . .
How our engineering team improves the on-call experience with Tines | Tines
At Tines, we have two types of on-call: daytime and out-of-hours. In this blog, we explain the daytime on-call shift, its pros and cons, and how we make it easier by using Tines.
https://www.tines.com/blog/tines-at-tines-engineering-on-call-experience/
0
4
2
As much as I love shipping features, systems and performance wins just hit different! π
5 months ago
0
4
1
Finding about PostgreSQL vacuums, heap truncation and standbys the hard way π
github.com/postgres/pos...
loading . . .
https://github.com/postgres/postgres/blob/092e72a93023fc9cea9cf673ab577ccd8537fb69/src/backend/access/heap/vacuumlazy.c#L3169...L3188
5 months ago
0
2
0
This was a fun one
www.shayon.dev/post/2025/11...
loading . . .
A PostgreSQL planner gotcha with CTEs DELETE and LIMIT
How a seemingly straightforward DELETE query using a CTE and LIMIT returned more rows than expected due to query planner optimization.
https://www.shayon.dev/post/2025/119/a-postgresql-planner-gotcha-with-ctes-delete-and-limit/
6 months ago
0
0
0
github.com/shayonj/pg_h...
Usually moving to partitioned tables come with a cost. Turns out calculating which partitioned table you can send your pointed queries (without querying database) makes those queries _extremely_ fast. More to come soon.
loading . . .
GitHub - shayonj/pg_hash_func: Determine the target partition index for an integer key according to PostgreSQL's default hash strategy, without querying the database
Determine the target partition index for an integer key according to PostgreSQL's default hash strategy, without querying the database - shayonj/pg_hash_func
https://github.com/shayonj/pg_hash_func
6 months ago
0
2
0
Reverse engineering PostgreSQL's hashfn.c (
github.com/postgres/pos...
) in Ruby to speed up point queries on partitioned tables (as one does π)
loading . . .
https://github.com/postgres/pos..
6 months ago
0
2
0
reposted by
Shayon Mukherjee
Dan McKinley
7 months ago
I have some thoughts about
boringtechnology.club
for you, as the work turns ten years old this weekend.
loading . . .
Choose Boring Technology
How to be old, for young people.
https://boringtechnology.club
2
31
9
reposted by
Shayon Mukherjee
Craig
7 months ago
Great article from
@shayonj.bsky.social
about async vs. sync commit in Postgres. I've talked about this with folks for year that it's not all or nothing you can toggle per session/transaction/etc. Though can't recall many folks writing it up, so thanks!
www.shayon.dev/post/2025/75...
loading . . .
Selective asynchronous commits in PostgreSQL - balancing durability and performance
Safely leverage PostgreSQL's asynchronous commit for significant performance gains
https://www.shayon.dev/post/2025/75/selective-asynchronous-commits-in-postgresql-balancing-durability-and-performance/
0
5
1
Wrote a bit about an interesting find lately
www.shayon.dev/post/2025/75...
loading . . .
Selective asynchronous commits in PostgreSQL - balancing durability and performance
Safely leverage PostgreSQL's asynchronous commit for significant performance gains
https://www.shayon.dev/post/2025/75/selective-asynchronous-commits-in-postgresql-balancing-durability-and-performance/
7 months ago
0
3
2
Are folks running PostgreSQL DBs with synchronous_commit on or off? πΆοΈ
7 months ago
1
0
0
"While these tools boost productivity, theyβre not a replacement for critical thinking. Taking the time to understand why something works (or breaks) and building strong mental models isnβt just busy work..."
www.shayon.dev/post/2025/53...
loading . . .
Challenging AI generated code from first principles
While these tools boost productivity, they're not a replacement for critical thinking. Taking the time to understand why something works (or breaks) and building strong mental models isn't just busy w...
https://www.shayon.dev/post/2025/53/challenging-ai-generated-code-from-first-principles/
8 months ago
0
0
0
Some more ramblings from working on PostgreSQL (Had its brief moment on HN)
www.shayon.dev/post/2025/40...
loading . . .
Scaling with PostgreSQL without boiling the ocean
Practical scaling strategies for application developers who don't have a dedicated database team
https://www.shayon.dev/post/2025/40/scaling-with-postgresql-without-boiling-the-ocean/
8 months ago
1
6
2
Counting is a hard problem.
10 months ago
0
1
0
Planning a wedding with your parents is a group project that no one prepared you for.
11 months ago
0
0
0
Post run, half baked thought: In memory store, flush to S3 every N interval. Reader replicas straight from S3 (?) New S3 primitives (conditional writes, and append only) for strong consistency and other good semantics. Costs ? Unknown πΈ
11 months ago
0
0
0
Rollercoaster of emotions next few days.
11 months ago
0
3
0
Very heartwarming to see other companies finding pg_easy_replicate (born out of @TinesEng) useful for large scale PostgreSQL migrations π«Άπ½
github.com/shayonj/pg_e...
loading . . .
Cross Cloud Migration Validated Β· Issue #187 Β· shayonj/pg_easy_replicate
Just wanted to put this out there - cross cloud migration is validated. It is definitely possible to migrate from AWS >> Azure or vice versa, the only minor preparation step is that you need to ens...
https://github.com/shayonj/pg_easy_replicate/issues/187
11 months ago
0
2
0
Just realized with pg_flo you can use the same DB as both source & destination. This lets you create shadow tables & remap data flows within a single DB using the routing feature. Useful for data migrations when you need time to verify data integrity!
www.pgflo.io
loading . . .
pg_flo - Stream, transform, and route PostgreSQL data in real-time
pg_flo is a powerful PostgreSQL data replication tool that enables real-time streaming, table routing, data transformation, and secure data masking between databases.
https://www.pgflo.io/
11 months ago
0
0
0
Happy to see pg_flo in top 5 on Hacker News today. Looks like all positive feedback. Excited to continue hacking on it.
11 months ago
1
2
0
Wrote a few words about using
#PostgreSQL
page location (CTID) for large data cleanups as a workaround
www.shayon.dev/post/2024/30...
loading . . .
Using CTID Based Pagination for Data Cleanups in PostgreSQL
When dealing with very large PostgreSQL tables (we're talking 15TB+), sometimes routine maintenance like archiving very old data can become surprisingly challenging
https://www.shayon.dev/post/2024/303/using-ctid-based-pagination-for-data-cleanups-in-postgresql/
12 months ago
0
1
0
Just shipped the v1 of new table routing feature in pg_flo. This way you can easily route PostgreSQL changes to different tables & column names in the target database. Lots more soon. In the mean time, a small and simple site:
www.pgflo.io
loading . . .
pg_flo - Stream, transform, and route PostgreSQL data in real-time
pg_flo is a powerful PostgreSQL data replication tool that enables real-time streaming, table routing, data transformation, and secure data masking between databases.
https://www.pgflo.io/
12 months ago
0
1
0
reposted by
Shayon Mukherjee
v
12 months ago
Reminder: SQLite is 35% faster than the filesystem while also using 20% less disk space read more here:
www.sqlite.org/faste...
2
12
5
Noodling on a tool to sync (CDC)between Postgres DBs. Define rules to anonymize, filter, or transform data before it reaches the destination. Considering a cloud version as well to keep it zero-touch-ops. Just setup & go. Early days, open to feedback. Design is flexible
github.com/shayonj/pg_flo
loading . . .
GitHub - shayonj/pg_flo: Easiest way to move and transform data between PostgreSQL databases
Easiest way to move and transform data between PostgreSQL databases - shayonj/pg_flo
https://github.com/shayonj/pg_flo
12 months ago
1
3
0
Not a yolo day, today.
12 months ago
0
2
0
reposted by
Shayon Mukherjee
Lawrence Jones
12 months ago
Ok so turns out this is possible! There's a Chrome extension called Sky Follower Bridge that does the matching for you. This worked with zero drama.
add a skeleton here at some point
1
2
1
reposted by
Shayon Mukherjee
Craig
12 months ago
Giving this starter pack thing a try, if you want a little Postgres in your feed (and who doesn't) here's some folks work a follow -
go.bsky.app/Acp7hmk
add a skeleton here at some point
2
40
19
you reached the end!!
feeds!
log in