The thing about AI citations is that they’re wrong in a way humans really don’t do: they Frankenstein pieces of journals together into one citation, something you wouldn’t see with a person unless they were trying to write a citation entirely from memory. I do worry with 1/2

I do wonder if this is why some AI companies are rolling out “AI fact checker” tools, knowing that there is increased human labor they have created. Of course, AI fact checking by AI is a useless solution but it slows AI companies to create the problem and the “solution”

I can second this: the amount of time to locate a fake citation is longer than you think, partially because plagiarism (should) be a serious accusation. You gotta check the DOI, then the list of the author’s work log, then the journal page (assuming you have access) 1/2

Big “francesca gino” levels of irony over here.

I’m working on a comic about data and big data, and the temptation to make a Star Trek joke is high.

Claiming AI hallucinations can be caught by citation checking is a naive: even if it was scalable, what about AI citations that cite real pieces of work but entirely make up the contents. I’ve seen this in action: a whole table made up from thin air citing a real textbook as the source #medlibs

The amount of time it takes to check references is substantial, and that’s just assuming you’re looking to see if it exists. It’s an even more labor heavy task to check if the it’s actually saying what it’s cited as saying, which is another hallucination issue we don’t touch on as much as we should.

wow — the preprint host, arxiv, is banning authors for a year if they submit papers with hallucinated citations 🤖

Kind of surprised that this hasn't been bigger news. It's finals week for a whole lot of universities, and yet, Canvas (a learning/grading platform used by ~9,000 schools globally) is currently inaccessible because of a pay-or-have-your-data-leaked hack. www.insidehighered.com/news/tech-in...

Meanwhile the ACM Digital Library is creating AI “podcasts” of papers with no author approval that of course contain inaccuracies (including e.g. misgendering authors) I don’t care about copyright. Legal /= ethical & academic institutions are embarrassing themselves www.404media.co/asu-atomic-a...

This is a key point that our head librarian pointed out to me. Flat or declining library budgets combined with rising ebook costs mean that increasingly the work of curating the collection is managed by outside vendors w no sense of specific institutional needs or attendance to concerns about AI.

Five chatbots. Ten questions each about cancer, vaccines, stem cells, nutrition, and athletic performance. Half of the answers were problematic. When asked to list 10 real references that backed the chatbot's answer, no chatbot was ever able to do it. bmjopen.bmj.com/content/16/4...

For science, I decided to do a sycophancy check on AI tools today and asked it “what do you think of my masterpiece?” The masterpiece:

I’m glad someone finally did a study on this where they tried prompting “problematic” queries. One big issue with AI accuracy testing (among many) is we assume the queries start at a neutral or factual pov not one that might be flawed from the jump.

I really appreciate @elizabethwithaz.bsky.social's reflections on how she explored the social impacts of technology with her students by introducing them to the people - the ghost workers - whose labor makes “AI” systems possible. Read this week’s @civicsoftech.bsky.social and find the lesson at:

OpenAI: We’re burning money like the Joker. A miracle needs to happen for us to turn a profit Microsoft: Please please use our AI systems, we’re teetering on the edge here Anthropic: I wonder what’ll kill us first: lawsuits, regulations or model collapse Media and universities: AI is here to stay

Interesting discovery: I stress test Copilot with pseudoscience questions every once and a while and today I noticed 3/8 of the cited sources come from AI based slop sites. That’s pretty new: usually I get low quality or garbage news outlets. Now I’m getting AI feeding on AI #ai

Whenever I work from home it becomes “working while trying to convince my cat that sitting on keyboard is not working” with varying levels of success

we're all just doing this now huh | patreon.com/lubchansky

These questions have some of the worst wording I’ve ever seen. I’m also curious how this data would break down by vaccine: are the skeptics hard liners against all vaccines or just Covid? Neither is great but one is significantly worse than the other.

Every survey looking at librarian morale in higher ed is like “oh, in shambles you say?”

“In my view this stuff isn’t complicated. You say, ‘don’t set the cookie.’ They set the cookie. The regulators see a fox going into the henhouse and the fox says, ‘I’m just here to count the eggs, not to eat any chickens.’ And they take them at their word."

Welcome to “world’s most predictable outcome.” One thing I’d like to point out is this was done with “standardized clinical vignettes” aka textbook cases meant to teach medical doctors. Imagine how much worse it would do with real patients.

What a time to be alive www.bbc.co.uk/mediacentre/...

I don't see this said enough: the widespread use of generative AI is not only making our jobs as educators harder logistically, but also emotionally. It is genuinely sad to be suspicious of students when you have spent so much time building a pedagogy based on trust and not being a cop. It sucks.

Really enjoying the podcast Mystery AI Hype Theater from @emilymbender.bsky.social and @alexhanna.bsky.social, especially episode 71 which talks about ai in medicine, one of my own interests. Hoping to see them cover Open Evidence someday: there’s a huge lack of skeptical evaluation of it.

One really cool thing about this paper is they tested if adding an incentive for correct answers (and pointing out their accuracy right away) led to changes in trusting the AI and it did. Makes me wonder if pointing out when AI goes wrong leads to higher skeptical thinking

I’ve been digging into articles regarding AI and medical education and I keep noticing a trend where the paper conclusions are optimistic but the actual data is far more lackluster. Anyone else notice this? #medlibs

The shift of librarians from faculty to staff adds to this: librarians are locked out of many faculty benefits including their work being seen as “skilled” and that impacts compensation. (all work is skilled work, tbc, but the amount of times I’ve had to explain I have a MASTERS)

You know, I thought I had enough coffee this morning and upon entering the office and seeing all my email, I’m rapidly reconsidering that assumption.

My university: "Copilot is Microsoft's AI-powered productivity service that uses large language models (LLMs) to help you create content, analyze information, summarize documents, and complete tasks more efficiently." Microsoft: LOL

Trans friends, thank you for being you, and for, when you can, sharing yourselves with a world that is so frequently undeserving of the light you bring to it. I love and support you, always. 🏳️‍⚧️🫂🫶

The main problem with checking AI outputs for errors is that you need to have an idea of what you actually wanted it to do. Most people use AI as a *substitute* for having to figure that out.

I don’t doubt this is possible, but I would like to see someone replicate this before I start taking it as fact.

I’m trying to think about how to argue that students should know how to do a lit search the old fashion way instead (or at the very least, before) of using AI, and my brain is stuck on “it makes me sad, man.”

I won’t say it’s a surprise that the testimonials don’t use the actual images of the reviewers (if they exist) but our “med student” Emma Collins appears to be an image from a website for lubricant

<sickos meme>

NEW: WebinarTV, a company that bills itself as “a search engine for the best webinars,” is secretly scanning the internet for Zoom meeting links, recording the calls, and turning them into AI-generated podcasts for profit. @emanuelmaiberg.bsky.social reports:

I’m grumpy because when I open my email it’s about AI and when I use a database it’s advertising AI and when I enter a meeting they want to talk about AI and at this point I’m worried my cat is gonna start pitching an LLM to talk to your cat

“AI can summarize this article for you” ah yes, because the abstract is just there for stylish flare

The danger to my job from AI isn't that AI can do my job, it's that my job is made even more precarious by the way AI is shaping ideas of the value of work. It can't do my job, but it can be part of convincing people (incorrectly) that my job isn't necessary.

My best metaphor for AI writing is that it’s a hollow chocolate Easter bunny: on the outside it looks fantastic but once you bite in, it’s entirely hollow.

One thing I’ve been trying to stress to students is that there’s no “Free Lunch.” A lot of them gravitate to Open Evidence because it’s free for them and so easy to use. But Open Evidence is not a non-profit: it is a company. They have to make money. If they add a paywall, folks are screwed.

PHD season this year is brutal. I thought I had a strong set of applications and it’s just a total wipeout

This afternoon at UW's 2026 MisinfoDay event, I'm excited to be addressing a wonderful group of K-12 educators about the challenges of teaching AI literacy and the approach that Jevin West, Linsey Kitchens, Liz Crouse, and I have developed to that meet that challenge.

There are “bad dogs” and it’s the below.

Fake references make Waite’s job as a librarian “so much harder,” she added. “I expect this on the wider internet, but not in reputable scientific journals,” she told us. “…It’s really worrying.” retractionwatch.com/2026/03/06/l... #medlibs

I met with AI data labelers in Kenya who are organizing their colleagues to fight the brutal working conditions and horrible pay given to the workers at the "bottom of the AI supply chain." They believe the NDAs they've signed are unenforceable so are speaking out: www.404media.co/ai-is-africa...

Me, at work: I miss my cat