Today we had a successful 8th IEEE Int’l Workshop on Next Level of Test Automation (#NEXTA2025). Highlights included: ✅ KEYNOTE by Serge Demeyer ✅ TALKS - Influence of Pure & Unit-Like Tests on SBFL - Witness Test Program Generation - Adaptive Testing for LLM-Based Applications ✅ DISCUSSION!

SEER Lab’s Jeremy Bradbury & Riddhi More are in Naples, Italy 🇮🇹 attending the 18th IEEE Int’l Conference on Software Testing, Verification and Validation (#ICST2025) where they are presenting two research papers. Dr. Bradbury is also a member of the faculty panel at the ICST Doctoral Symposium.

Looking for evidence of data leakage in the #HumanEval code generation #LLM benchmark? Check out our analysis of a subset of HumanEval tasks vs comparable tasks on #ChatGPT, #Claude and #Llama. @riddhimore.bsky.social

Yesterday Riddhi More presented our paper “Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models” at the 1st Int’l Workshop on Fairness in Software Systems (#FAIRNESS2025) in Montreal, Canada. 🇨🇦 Paper preprint, slides & code/data available at: bit.ly/4brEBJm

MSc student Riddhi More & Prof. Jeremy Bradbury are in Montreal, PQ this week attending the IEEE Int’l Conference on Software Analysis, Evolution and Reengineering (#SANER2025) and the 1st Int’l Workshop on Fairness in Software Systems (#FAIRNESS2025).

Today Dr. Michael Miljanovic presented our latest paper at #SIGCSE2025. The paper titled “How Effective and Efficient Are Student-Written Software Tests?”, is co-authored by Amanda Showler, Michael Miljanovic, and Jeremy Bradbury. 📄: www.sqrlab.ca/publications...

SEER Lab’s Stacey Koornneef and Prof. Michael Miljanovic are in Pittsburgh, PA this week attending the SIGCSE Technical Symposium on Computer Science Education (#SIGCSE2025).

Thank you to the organizers of the #AI Meets Software Quality Int’l Colloquium for inviting Riddhi More & I to speak on Monday. If you're interested in #LLMs & #flakytests the preprints of the papers presented are available at bit.ly/40OvYUa (FlakyXbert) & bit.ly/4jILzNT (augmentation bias).

Tomorrow Prof. Jeremy Bradbury & MSc student Riddhi More will be presenting "FlakyXbert: A Few-Shot Learning Framework for Detecting and Classifying Flaky Tests" at the AI Meets Software Quality International Colloquium (bit.ly/42DAZ4M) #AI #LLM #testing #flakytests #dataaugmentation #bias

Our paper "Addressing #DataLeakage in #HumanEval Using Combinatorial Test Design" has been accepted for publication at the Int’l Conference on Software Testing, Verification & Validation (#ICST2025), Short Papers, Vision & Emerging Results. co-author: Riddhi More arxiv.org/abs/2412.01526 #benchmark

Are you working on #softwaretesting & passionate about test #automation and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation #NEXTA2025, co-located w/ #ICST2025 Submission Date: Jan 10, 2025 [UPDATED!] bit.ly/nexta2025

I'm happy to share our paper "Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models," has been accepted to the 1st International Workshop on Fairness in Software Systems (FAIRNESS 2025), co-located with #SANER2025! #ontariotech @riddhimore.bsky.social

Happy holidays from the SEER Lab! ❄️

I’m excited to share that our paper "An Analysis of #LLM Fine-Tuning & Few-Shot Learning for Flaky Test Detection & Classification" has been accepted for publication at the Int’l Conf on Software Testing, Verification & Validation (#ICST2025). This paper is first-authored by @riddhimore.bsky.social

I had a great time today attending the #OntarioTech Cakes & #AI Winter Show-And-Tell event. A great opportunity to engage with researchers from across the university and establish interdisciplinary collaborations. Thanks to @petelewis.bsky.social for organizing! 🍰

Interested in joining the Software Engineering & Education Research Lab? Find out more at bit.ly/3ARpCe8 #OntarioTech 🇨🇦

Last night I participated in a graduate student panel and a poster session at @ontariotechu.bsky.social! It was great to be able to network with everyone and see the research coming from other graduate students!

I'm happy to share that w/ Pooria Madani, Khalil El-Khatib & Natalija Vlajic we have been awarded a $382,000 grant from the National Cybersecurity Consortium to study "Transformative Adversaries: Leveraging Generative Pretrained Transformers for the Dev. of Next-Gen. Metamorphic Malware Engines"

Did you know that the Software Engineering & Education Research Lab is located on Ontario Tech University’s state-of-the-art campus in Oshawa, ON, Canada? 🇨🇦 [photo credit: @ontariotechu.bsky.social]

I was happy to have Prof. Pooria Madani give a guest lecture today in my Applications of LLMs to Software Engineering graduate course. He presented “Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats” — arxiv.org/abs/2410.22293 #OntarioTech @pooriamadani.bsky.social

New paper alert! 📃 ⏰ How Effective and Efficient Are Student-Written Software Tests? - Amanda Showler, Michael Miljanovic, Jeremy Bradbury Proc. of the Technical Symposium on Computer Science Education (SIGCSE 2025) [to appear] Preprint: bit.ly/sigcse2025te... #softwaretesting #CSEd #OntarioTech

Are you working on #softwaretesting & passionate about test #automation, test generation, and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation - #NEXTA2025, co-located with #ICST2025 Submission Date: Jan 3, 2025 More Info: bit.ly/nexta2025

I’m happy to now be on 🦋! 👋🏻 I’m a CS professor at Ontario Tech University in Oshawa, Canada. 🇨🇦 At #OntarioTech, I lead the Software Engineering & Education Lab (www.seerlab.ca) where we research applications of #AI, #NLP, #LLMs to #softwareengineering, #softwaretesting & #CSEd.