As excited as I was last winter about teaching my Learning at Purdue in the AI Era class,, I am apprehensive and unsettled about repeating it now. Even as the core principles around the course remain similar, advances like Agentic AI reduce the academic ground to quicksand. Uncertainty, Ahoy!!!

AI sycophancy is so curious. I've been studying whether a bot's degree of agreeability modifies student desire to use them. I think we would do well to integrate explorations of gender bias more intentionally. papers.ssrn.com/sol3/papers....

The equity-gap in student use of free vs. paid LLMs makes sense to most of us. This preprint provides some empiricism behind it. arxiv.org/pdf/2511.23157 This will only get worse as access to ultra-premium models increases.

After multiple months using Copilot pro on trial (normally $30/month per user) I would say its ability to search all your Microsoft stuff at once is the killer app. Its chatbot inference and output abilities are as bad as an airline phone bot that makes you say all your questions.

This is an awesome small-scale look at how sycophantic LLMs lead learners astray in problem-solving. Next, I'd love to look at what type of sycophancy actually attracts students if given a choice of bots. arxiv.org/abs/2510.03667

Sycophancy in bots is an inimical part of AI in Teaching and Learning. When the bot wants to tell you that you are right, high dependence almost certainly means you will inculcate incorrect knowledge. Love papers like this who explore sycophancy in the discipline arxiv.org/abs/2510.04721

Skeptical of most 3rd party IT company analyses of other software, but this SCALE AI paper outlines many of the limits of creating and evaluating "AI as Tutor" studies. Too many edge cases. arxiv.org/abs/2510.02663

Lots of guesses about student use of AI are turning out as expected: Students use them to solve/complete homework; they ignore LLM-posted questions guiding them to think deeper, single-prompts dominate interactions with little dynamic engagement. arxiv.org/abs/2509.08862

Want more studies on small-level feedback mechanisms around disciplinary understandings. No need to have an LLM measure "everything". Start small, start with a need area. arxiv.org/abs/2508.14823

Purdue's AI Academy finished with 70+ instructors creating projects, plans, tools or critical approaches around and in response to AI. I was particularly enthused when multiple participants said "I thought I was gonna learn the tech, but I learned about learning"

Today on the podcast: Study Hall! @leaton01.bsky.social @michellemillerphd.bsky.social and @thedavenelson.bsky.social and I discuss three recent studies exploring the intersection of AI and teaching. Cognitive offloading, chatbot sycophancy, & more! intentionalteaching.buzzsprout.com/2069949/epis...

Democratizing prompt for LLMs: Read and review new terms of service for X company. Compare and contrast with previous versions. What should I be aware of? What might any consumer be wary of or concerned about?

I love this work. Biggest takeaways - AI Agents "often fail at effectively guiding students toward mastery" and "students prioritize scores over feedback, leading to off-task behavior that can hinder growth." Authenticity and relatedness are needed for these efforts. More humans in the process.

As models become tuned to specialized academic content, the gap between novice and expert ability to critically evaluate outputs will grow. This paper demonstrates the change in student error recognition from GPT4 to o3. More longitudinal studies like this please. arxiv.org/abs/2507.20995

It was relatively easy to keep up with the frontier AI models and a few open source clones. Agents are like electrical appliance manufacturers in the early twentieth century. What is actually useful? What is a niche product? arxiv.org/abs/2503.11733

I've been focusing a lot on recently on trust and perceptions of trust + sycophancy + bot capability. Appreciate this preliminary approach for gauging Uni-owned bots vs Chat. arxiv.org/abs/2505.10490.

Increasingly convinced that simulation practice in clinical settings is one of the biggest "killer app" prospects for AI in education. Generate at scale, add nuance + complexity easily, personalize, etc. iovs.arvojournals.org/article.aspx...

I ran a small experiment with two syllabus chatbots - a Ted Lasso overly sycophantic version and a caustic Severus Snape version. Students really didn't like Snape's recalcitrance, but also hated Lasso's bland fawning style. We need more work on sycophancy and its effects arxiv.org/abs/2311.09410

He shared at The Other Place, but I have to say here that @hamel.bsky.social 's YouTube chapter generator code is a godsend to academics who want to parse through hour-long video talks, especially when they concern AI and many are saying the same things. Link in comments

This is solid math education work. "LLMs have mastered a superficial solution process but do not make sense of word problems." That superficiality and illusion of explanatory depth is one of the biggest dangers to student users who might think 'seems legit'. arxiv.org/abs/2506.24006

We've got decent work on sycophancy. This paper gets at the heart of the student use peril. Overconfidence in the tool AND yourself will redirect the output towards your conclusion. Makes it easier to dismiss the friction of error, and easier to outsource your thinking work. arxiv.org/abs/2506.10297

Purdue professors in the late 60s were apparently either Mary Poppins or Sam Kinison

This has been poked at in lots of places, so I won't, but to me it demonstrates the pitfalls of AI educational research when computer scientists omit learning scientists from their work, and vice versa. arxiv.org/abs/2506.08872

AI education is way too focused on LLMs and DNNs these days, excluding a lot useful topics in AI that are non-ML (e.g. search, planning, knowledge, etc.). That's one of the motivations behind this series I started, www.informationga.in/blog/ai-expl..., which is a first principles treatment of AI.

Beyond LLM metrics or any gauge of effectiveness, the faculty I have worked with vibe first with the voice, syntax and prosody trends of each bot. DeepSeek spurred smiles and curiosity among many colleagues. Big hat tip to: @natolambert.bsky.social www.interconnects.ai/p/latest-ope...

Would like more studies like this that quantify the impact of active AI use on exams. While this is only part of the puzzle, I think, modifying learning outcomes beyond disciplinary recall benefits from detailed study of how gameable current assessments are. www.sciencedirect.com/science/arti...

What I really appreciate about this study is the confirmation that those who value human dialogue in learning are more tuned to artificial efforts, and those who don't think of feedback largely in terms of clarity, not content. link.springer.com/chapter/10.1...

In 25 years of teaching, I have never once felt sad nor a sense of loss when a class ended. The students in my AI course have activated both feelings in me. I will truly miss their energy, insights and critical thought about learning in the AI era.

Link to the paper now available: dl.acm.org/doi/10.1145/...

I’m pleased to announce the publication of the first of a three-issue special volume from Thresholds in Education about generative AI’s impact on teaching and learning coedited with Stephen Monroe. We have amazing essays from educators. Please give them a read! open.substack.com/pub/marcwatk...

Students value 24/7 availability, GenAI as + to academic performance, and think faculty can identify AI much more than faculty do. Fac think students will ask AI more questions, and think a fully online degree is worth doing more than students do. www.sciencedirect.com/science/arti...

Significant Kudos to @anthropic.com for publishing these studies. Hard to know w/o classification methods being shared, but it seems: 1) Anthr and I have wildly diff't definitions of "create & improve educational content" 2) Too many students use 1-shot prompting www.anthropic.com/news/anthrop...

The problem with most machine-based random number generators is that they’re not TRULY random, so if you need genuine randomness it is sometimes necessary to link your code to an external random process like a physical noise source or the current rate of US tariffs on a given country.

"Deep Research tried to summarize all this...and ended up with a kitchen sink approach." @dereklowe.bsky.social expertly lays out the limits of DeepResearch output from generalist LLMs. www.science.org/content/blog...

How can AI support rather than replace student writing and thinking? How can we encourage confidence and skepticism of AI outputs? Our approach, developed at UC Davis, combines peer review with AI feedback, emphasizing reflection and agency. 1/3

With levity lately being necessary for decent sleep and productivity, I offer that Llama is the only major LLM model to take seriously my request for help with the coming giant ant invasion and takeover. poe.com/s/n1GxOFniiB...

Love article from @robinwelsch.bsky.social et al on overestimation of ability with AI use. Increasing evidence that AI will exacerbate what @emollick.bsky.social highlighted: The illusion of explanatory depth. We would do well to layer metacognition into all our teaching. arxiv.org/abs/2409.16708

Data point #3,781 - Algorithms compress solutions towards the center. For fields like nutrition and health, they will always diminish the outlier elements, which is precisely where expertise is needed doi.org/10.1177/0260...

When we outsource our feedback of written student work to an LLM we complicitly and fully endorse the application of societal hierachies to that work. It's baked in. Add grades and things get real messy. Great work from @mariekheath.bsky.social and Melissa Warr. journals.sagepub.com/doi/full/10....

I appreciate the simplicity in this construction management approach to evaluating LLM output in learning activities: 1) Clearly define problem, reqs, constraints 2) Modularize into smaller problems 3) Define metrics for successful generation. doi.org/10.3390/buil...

No correlation between # of “active learning” techniques used in class and student achievement. Relying on an algorithm for data and analysis minimizes impacts of your effective outlier teaching techniques. Earnest effort, but misguided.

@becksup.bsky.social @chronicle.com

How much access does OpenAI's ChatGPT Edu plan offer users? Thanks to CSU Chino's resource page, we have a clue and it isn't very much. 10 uses of GPT-4o reset every 5 hours. Unlimited use of GPT-4o mini. Nothing about o1 or o3. Is this equity?

Is 90% accuracy good enough? For fields where data was previously inaccessible or inexorably shaped by investigator bias, maybe. Algorithmic literacy opens a host of discussions with students about trust, data and conclusions. www.biorxiv.org/content/10.1...

Not overly surprising, but helpful to have Anthropic's report on economic uses of Claude. More supplementary to cognitive tasks than automation (57-43%) & high usage correlates with bachelor's degree or higher. assets.anthropic.com/m/2e23255f1e...

The Office of Science and Technology Policy is requesting input on an AI policy. They are seeking "concrete AI policy actions needed to address topics raised." www.federalregister.gov/documents/20...

One of my students "broke" the Poe version of DeepSeek today. They have given approval to share. poe.com/s/8Bga7NbZsK...

One of my students introduced me to an excellent word game that illustrates how the word vectors behind LLMs work. Guess a word and you find out how "close" you are to the target.

Biggest conclusion from Part 2 of US Copyright office report on AI? "The Office concludes that, given current generally available technology, prompts alone do not provide sufficient human control to make users of an AI system the authors of the output" www.copyright.gov/ai/Copyright...

I’ve engaged with DeepSeek a bit this week and discovered something that I’ve never seen in another model: real-time, retroactive editing of output. DeepSeek changed an output after it was displayed to me. I can’t tell if this was before it finished or after it had posted. More below