Using Codex to create a hybrid of CMU's omnomdombomb.github.io/ai_evaluatio... and alfredhw.github.io/airubric/ + my own twist. (1)

So far people are trying to argue with me are not even touching my actual point. Even the cost argument is odd because if you concede llms can be useful just not worth the cost, cost drop as tech becomes more efficent, so what then?

Librarians who insist LLMs can *never* be useful for *anyone* & anyone claiming so is deluded are really comparable to flat earthers. Even skeptics like Gary Marcus & Margaret Marshall - one of the authors of thestochastic parrot paper is on record for saying LLMs can be extremely useful. (1)

Wow its 2026 and people are still throwing around cringe terms like "rockstar librarian" unironically & self identify as one. Lol.

It is trendy nowdays in librarianship to critique efforts to improve efficiency for our users, claiming that's not the point. The last I checked- save the time of the user is a law of librarianship. not "push our views/values" on our users. I acknowledge efficency is not everything but

Spare me from librarians running around saying "trust is our super power" and its "our core strength". Typically accompanied by the hubris of thinking because they are a librarian, their words should automatically be given extra weight even if they have no expertise. Vocation awe much?

I have always said among all the subfields of LIS, information literacy is the area which eludes me the most and makes me feel a sense of confusion on what the core really is. I think I'm starting to slowly realise why (1)

An advice i can give to librarians starting out is to focus on learning from first principles. Actually understand how things work as deeply as you can. It can be difficult & gains will not be obvious at first but in the long run are more adaptable & capable of seeing implications of changes(1)

It really feels to me all the academic ai search tools are inching closer and closer to reinvent reference managers though all but one is saying so. This might be a good timing given new capabilities due to LLMs & current leaders Zotero, Endnote, Mendely seems stagnant (1)

What Are Libraries Actually Doing About the Sustainable Development Goals? katinamagazine.org/content/arti... - honestly this whole libraries "doing sdgs" has all the hallmarks of libraries "fighting fake news". Trendy but really not doing much

[Blogged] AI academic search needs better frameworks for understanding and evaluation. These three librarian projects are a start open.substack.com/pub/aarontay...

New COUNTER standards for handling "AI agents". Also "So Phase two will focus on third-party and off-site tools, and we’ve had great feedback and engagement from scholarly AI developers like Scite and Consensus..."

Post covid a button seemed to have been pressed & I reached a level where I started getting more consistently asked to keynote. I have never considered myself a good keynote speaker. (1)

Hot take iwith chatgpt/claude becoming connected to academic index via MCP, products that are basically wrappers around openalex, semantic scholar will not be worth it unless they can show real value. This is particularly so if the target audience are freshman / lower level undergraduates.why?(1)

Went back to try some of the ai search tools that i find have horrible retrieval but are still subscribed by many libraries & they seem to have improved - no longer complete jokes but still not good enough for serious work.

[Read] Performance of Large Language Models in Automated Medical Literature Screening: A Systematic Review and Meta-analysis - cover 18 studies (4 full text screening) using LLMs for screening from 2023-2025. www.medrxiv.org/content/10.6... . (1)

Another camp. library technologist running the discovery layers etc. Eg IGELU. I am known to this camp & invited to keynote at their annual meeting.There are really smart guys here but due to nature of catalog/discovery layer they tend to align with IL peeps in their pov supporting the novice user

I suspect I test AI search tools quite differently from many academic librarians. My first "gate" is retrieval effectivess. If it is bad, I call it a day without testing further. This seems common sense but I think many librarians have other equally important or even more pressing concerns(1)

I have been blogging a ton of my thoughts on how "AI" has been changing academic search and it's nice to see how others are building stuff based on my work (with citations). I came across 3 examples in the last week that essentially vibe coded a web app to make the content more digestible (1)

Reading all these "studies" on ai search tools and I'm getting really frustrated. A lot of them seem to be making the same errors (1)

[Blogged] From Fixed Search Workflows to Agentic Academic Search: Promise, Progress, and the Invisible Menu Problem - First-gen AI search: fixed pipelines Now: agents that pick their own tools. However, now it is harder to know what is possible.... aarontay.substack.com/p/from-fixed...

Learning query languages are always challenging. The business/finances ones are always tricky... and diff for diff platforms, add the need for domain knowledge eg reporting standards, jargon - finance librarianship is very tricky. I never got very good at querying wrds, bloomers etc

Just posted some thing new on substack and got a notification someone contributed to my ko-fi.com/aarontay a record amount. A hour later, yet another contribution at the same amount? Also started doing this year, probably should have started in 2009 (when blog started) ha.

The pros and cons of managing a librarian team of specialists vs generalists. Obviously your leadership style needs to be different

Working (gradually) on #SystematicReview write-up, and feeling glad I have all the searches for early on (where I tested words and proximities) saved in EBSCO. And that I went through the rigamarole of putting them in the right place to transfer to the new platform.

" Mentorship Index (M-index) measures a scientist's contribution to mentoring junior scientists. The M-index counts the number of publications for which the scientist served as last author where the first author (typically the mentee who led the work) was relatively new to science" (1)

What If Claude or ChatGPT Could Search Academic Databases For You & Then Do Something Useful With the Results? My blog is where I write to think so its overly in the weeds. But when I write for my institution users, I try to keep it simple & results are better library.smu.edu.sg/topics-insig...

Months ago I read this horrible review of a tool that spends most of it complaining the tool gives horrible result ranking but ends justifying that poor relevancy ranking is good cos learning needs friction & I just went this person doesn't know the diff between "good" and "bad" friction. But (1)

why did elsevier change the name from Scopus Ai to the generic "AI Discovery". Scopus is such a strong brand. yes they moved from Sciencedirect AI to Leapspace but at least that made sense since it goes beyond sciencedirect but is "AI discovery" (former Scopus AI) going to search beyond Scopus?

@lisalibrarian.bsky.social co-chair of United2Act, shares an update on phase 2 of this collective action against paper mills. United2Act is a project from COPE supporting coordinated, practical responses to the challenge. www.researchinformation.info/analysis-opi... @researchinfo.bsky.social

looking at OpenAlex closely again...

I think a lot of people are surprised that a lot of agentic stuff is just LLMs running deterministic tools or running code to do heavy lifting. Someone even wondered what's the point when he can do it himself But I think this misses the point by a lot. (1)

I made a prediction in late 2022/early 2023 that in 3 years time, most academic databases would have RAG functionality. "Most" might be inaccurate but a lot do. I didn't expect using LLM to generate boolean would be that popular.. (1)

What Happens When AI Can Use the Library liblablexicon.wordpress.com/2026/04/02/w... A taste of the future ..

I did a whole thread on my recent content since I moved to substack that I was pretty happy with. bsky.app/profile/aaro... - these are the ones that after posting i think aren't very good or even very wrong in a serious way (my stuff is always a bit wrong but some are off in not so important ways)

[Watched] Research and Application Webinar: The new AI-powered tools for scholarly literature discovery." youtu.be/qEjW5E-LzHw?... - great talk by Steve kramer Business librarian at UNCG university libraries. I could nitpick some things but overall great

I notice, I always get asked to peer review things I am not that qualified, but the same journal would not send me things I think I would be perfect to peer review. It's kinda fustrating but maybe things I feel I have the most expertise in, there are also many librarians who want to review

I said it many times of the various sub fields of LIS , I understand information literacy the least. When I first started, seniors would tell me they were doing IL but it was just teaching mechanic skills eg nested boolesn, rote learning of database for diff areas, harping about apa citation

Because I use Claude to edit and critique my posts, I realise Claude seems most impressed by this. Even 3 months down the road, I would ask some random thing & it would link it to this all the time as if I invented some great concept. :) . Thinking I should publish in LIS journal ?

As usual, I see evidence synthesis people coming in to argue with me, even when its obvious I was talking about normal academic search/narrative reviews. A earlier draft had a big notification saying this is not about ES but it was too clunky. (1)

I moved to substack in June 2025, here are some of the posts at aarontay.substack.com that I am proudest of (1)

Scott Cunningham substack is interesting

Its hard to believe while my blog musings about librarianship was always focused on discovery, I did post on a wide variety of topics too eg about being a librarian, about open access, about bibliometrics & scholarly comm. But for the last 4 years, its like almost 100% impact of Ai on search. (1)

[Read] I Can’t Get No Satis-Searching: Reassessing Discovery Layers in Academic Libraries share.google/k2QQXQq3izcZ... - I decided to look & see what recent LIS literature says about Discovery layers and found this 2024 paper. - Spoiler alert - a bit disappointing

Well Alison is a information literacy researcher and agrees i think with my hot take on librarians rushing to teach things they ill prepared to teach. I must be mostly right?

Interestingly if you look at all the issues early library discovery layers almost all but one issue was related to fact the designers didnt realise despite the name, majority of searches was for known item. Issues like inability to filter book reviews, local collections being buried by Article etc

This blog post is connected to a few older posts I've done. There's obviously 2014's " Why Nested Boolean search statements may not work as well as they did" explaining the prob in modern context post 2010 context (1) musingsaboutlibrarianship.blogspot.com/2014/07/why-...

Frankly librarians need to remember the law is "save the time of the user" not "ensure tools give us the opportunity to teach things" :)

I first started as a academic librarian in the late 2000s. Here's a look back at how I experienced changes in search technology. Your experience might be different if you were in medical/law etc (1)

Stating the obvious. Chatgpt/claude etc are really good at search now. I was trying to recall a old blog post i wrote and struggled looking through past posts but "ai search" found it trivally.If you are still denying their effectiveness you are pretty much close to a flat earther now