"Librarians already have all the skills we need for AI/search/the future. We just need confidence." Can we stop with BS like this? It's just embarrassing.

[Blogged] What a year of testing & thinking about AI academic search taught me aarontay.substack.com/p/what-a-yea...

Slight complement to my horseless carriage argument. I argued LLM generating boolean is not useful except in rare cases where you can get zero hits, this isn't the case in most article search. But this is common when trying to find the right dataset/non-article database e.g. econ/finance databases!

Trying out a modified version of the Primo MCP server for us based on og version here github.com/geheharidas/...

For discussion. I know some of the leaders, conferences might be a bit odd (8 is very loose) & you could quibble Knowledge organisation is also a mechanism affecting ranking eg KG and graphrag etc but traditionally they haven't really been that successful in impacting the search im familar with?

Hot take. If your objection to AI is based on just hallucinated ghost references and even with RAG you can't guarantee no ghost references you clearly dont understand the tech. By blending code based validation with LLM you CAN with 100% certainty ensure anything cited was retrieved by the search.

I hesitating to write about or even hesitate to release my skill that formulate and pilots boolesn search strategy in Pubmed by searching in Pubmed and Browse MeSH because I know the evidence synthesis librarians are sharpening their knives ha....

Prob once you grasp the power of agentic flows. You see some fair complex procedure, you think hmm I could make this a skill and start trying figure out the flow, the connectors...

Why the Gates Foundation Abandoned Article Processing Charges (and What They’re Doing Instead) katinamagazine.org/content/arti... - I hear there has been a swing away from supporting the APC model....(1)

Being thinking about agents lately. One side would say you can't trust the agent cos technical objection a,b,c that it's doesn't do. Then you say it could do that take logs, write reasoning trace then other sides retreats to well someone needs to take responsibility & it has to be human.

Some reflections on attending FORCE 2026. (1)

still one thing i've noticed about Scholar labs vs other deep search/research tools is it's not as precise as Say ai2 paperfinder etc... It has a much broader interpretion of what is relevant than find other equalvant tools

Scholar Labs update: Search 10x faster, 3x deeper scholar.googleblog.com (gemini 3.5 flash?)- lightning fast now! Still limited to max 50 relevant, but can evaluate past 300 results (ive seen as high as 500+).

Publishing in 2026: How we got into such trouble and how we get out of it By Professor Virginia (Ginny) Barbour, Co-Chair of the Declaration on Research Assessment (DORA) #force2026

#FORCE2026 is officially open! “To Go Far, Go Together” sets the tone as we move from “Publish or Perish” to Purpose & People in research assessment reform. Great to see DORA Co-Chair Ginny with SMU President Lily Kong, after strong opening keynote on the need for reforming research assessment.

Starting the day, @ianmulvany.bsky.social’s keynote on AI’s effects on the Scholarly Communication community and how we can and must address the changes #FORCE2026

Aster Zhao from HKUST now presenting a comparison between human, AI-assisted and AI-dominant systematic review production. Funny she used an Elsevier paper to benchmark as it was useful using work in the most closed access publisher as it likely wouldn’t have been crawled #FORCE2026

Huajin Wang presenting on Developing an assessment framework to support critical evaluation of AI-powered academic search engines at #force2026

Quality of presentations & speakers at #FORCE2026 are really something There are more specialized Scholarly Comm venues but in general, this is the one to go if you want to hear leading thinkers talk about everything from Research Assessment, Open Science, Scholarly publishing, repositories etc

Petr Knoth of CORE on managing AI bot access to open scholarly infrastructures #force2026

At opening keynote of FORCE2026 by Lily Kong, President of Singapore Management University (SMU) - "From 'Publish or Perish' to Purpose and People: Going Together on Research Assessment Reform"

At FORCE2026 pre workshop Pre-conference Workshop 4: Global perspectives on Open Research Information

Hmm. Saying Boolean give better results than "Natural Language search" and citing a "2004 Springer" paper in 2026 really shows insane bias. Natural language search in 2004 and 2026 might as well be 2 different things....

[Read] Ranked (In)direct Citation Searching in Systematic Reviews: A methodological case study www.medrxiv.org/content/10.6... - author of paper is also author of Local Citation Network (1)

[Read] A practical guide to evaluating sensitivity of literature search strings for systematic reviews using relative recall - www.cambridge.org/core/journal... (1)

[Read] A practical guide to evaluating sensitivity of literature search strings for systematic reviews using relative recall bsky.app/profile/aaro...

I have a blog post i really want to write. But im procrastinating because I know its going to be hard to write plus I have other responsibilities...

When I read a study with title is ai chatbot better than Google; I don't expect a study that "analyses" what sources each ai cited, number of sources, years, paywall or not etc. I keep seeing such "studies" that just describe such things with zero insight into why. As for better than "Google part"?

At breakout on impact on Gen ai on Research Assessment

Panel.

At #FORCE2026 Satellite Event Shared Principles, Regional Realities: Advancing Research Assessment Reform in the Asia-Pacific in Singapore , Singapore Management University

[Blogged] Hot take: Stop calling poor search rankings necessary friction for learning aarontay.substack.com/p/hot-take-s...

Hmm now tempted to try to create a seperate claude/Codex skill based on the "objective approach". My current one asks optionally for seeds & mines + compare doc freq in gold standard vs global (essentially high df in later is penalised)but is basically using the conceptual approach

Sigh now I am down the rabbit school reading about objective vs Conceptual approaches to search strategies link.springer.com/content/pdf/... & relative recall of included studies from multiple systematic reviews pmc.ncbi.nlm.nih.gov/articles/PMC... , www.cambridge.org/core/journal...

Had a look at 2010's Altmetrics Manifesto - funny, it says something a bit different from what i thought it was saying... zenodo.org/records/1268...

People were talking about agents as far back as 2023/2024, but it all seemed theoretical to me, beyond my reach. But Claude Code (skills + connectors) seem to crack it. Even without advanced use of agents to orchestrate subagents, just LLM+tool+reasoning+context from expert = you can go very far (1)

I guess for a acad librarian, I can be considered a early adopter of tech. But it's all relative. I think I'm roughly 4-6 months behind the really bleeding edge. That's roughly the amount of time I really started playing with agent skills, Claude code vs their emergence.

This!!

I find myself agreeing more and more with @mikecaulfield.bsky.social that people really need to get away from thinking LLM/"AI" is everything. In fact, more things can be explained by understanding how the *search* works (which may or may not be even that related to LLMs).

While the concerns put here are not unreasonable, they are just speculation. Do such AI rag or deep research tools lead to more bias towards the scholarly rich than say Google Scholar? What we need is empirical evidence on retrieval. If something is not retrieved it cant be summarised! (1)

This is a great study and (one of?) the first of its kind! A note of caution from a human contributor though: given the slow academic publishing process, the findings should primarily be interpreted as a snapshot of AI capabilities (and the capabilities of researchers using AI) in 2024. (1/4)

Nice article. That said I think Studies on full text vs title abstract search Studies surprisingly didnt always show full text side winning all the time. I suspect this is due to limitations of old tech and with proper modern retrieval it should be a no brainer to have full text indexed

I love blogging and conducting workshops. Because the questions I get, often help me understand my area even more...

Thinking of doing an update to doi.org/10.1146/kati...

[Hot take] AI Academic Search and the Missing Middle of Literature Discovery aarontay.substack.com/p/ai-academi... - Is overly focusing on Undergraduate information literacy and evidence synthesis making us underestimate AI academic search tools?

Weird hypothesis. If you have a paid version of ChatGPT or Claude hooking up a academic specfific mcp like Wiley’s/scite/Consensus/SciSpace rarely gets you better results and sometimes even worse compared to using it build in web search tool

I have like almost half a dozen fully completed drafts of what i call my "hot takes series" for my blog. Trying to muster up the courage to press publish ha

From what I'm hearing many more "ai academic search" are looking to do similar things, providing Cochrane-like reviews+ PRISMA flow charts. I dont object to that but I think there is value also in being focused on supporting narrative reviews which are most literature reviews. Still i understand.(1)

Elicit goes deeper into supporting systematic reviews...

Another set of results playing with academic AI search tools to see how reproduible their searches are by running the same query 5 times in the same day, grabbing their results in csv/ris and doing a comparison, quite fascinating the differences. You cant see the header but they are respectively (1)