ICCL is hiring an Office & Operations Manger! The office is the heart of our organisation and ensuring it runs smoothly is an essential role. We are looking for someone experienced in operations, finance and governance. If that sounds like you, then apply before the deadline of 5pm on Sunday 31 May

New from @mbrauh.bsky.social !!

"We must develop and adopt rigorous AI evaluation methods to have a clear-eyed view of such a consequential technology." @mbrauh.bsky.social

"In measurement science, deliberately choosing what background concept/construct is of interest, systematising it into a specific criterion & operationalising it via design of eval methodology are each distinct & important steps that, when skipped over,can undermine validity of any conclusion drawn"

...Both industry reports and academic papers typically study base models, yet a recent paper found that the ChatGPT models available via the official, public API had substantially different performance than that observed through interaction with the chatbot web interface."

"This is a challenge especially for academic work, given the lack of direct access to deployed AI systems, though industry model cards do not always assess deployed systems, either. ...

"AI is not the first field to grapple with questions of measuring complex, abstract, real-world phenomena. A growing body of work highlights the validity gaps of current evaluation practices, drawing on measurement science developed in the social sciences."

on the use of "LLM-as-a-judge" in evaluation "Even for concepts where LLMs do have this function, what are the implications of basing the field’s sensemaking about its own progress on a small number of models developed behind closed doors?"

"AI developers complement benchmarks w human evaluation, shorthand for outsourcing highly structured microtasks to crowd workers. Though the results are far less visible & less widely reported than benchmarks, a sprawling global industry of human annotators produce both training & evaluation data"

"This result in models which have SOTA capabilities on benchmarks in company press releases, while being brittle & narrowly useful in practice. A culture of leaderboards, which combine scores of numerous evaluations into a single ranking, only increases opacity of what a model is truly useful for"

"The limitations of benchmarking extend beyond technical challenges. By design, benchmarks are proxies for more complex, real-world tasks.[...] In practice, few AI benchmarks justify why they are a good proxy for the intended real-world task and even fewer validate if they are."

"From early 2010s, the AI research community has focused on specific benchmarks to track technical breakthroughs. However, benchmarks saturate: the models become so good at the benchmark that it no longer captures improvements. Succeeding on the benchmark no longer reflects how the model performs."

"This opacity of both models and evaluation methods stifles the field, already dogged by limitations in AI evaluation practices, and limits progress towards a science of AI evaluation." writes @aial.ie's @mbrauh.bsky.social www.adalovelaceinstitute.org/blog/account...

📢 In the context of the ongoing trilogue negotiations on the #AI Omnibus proposal, together with 32 other organisations and individuals, we contributed to an open letter to EU institutions, expressing our concerns about current proposals that could weaken the scope and effectiveness of the AI Act.

The 🇺🇸+🇪🇺 oil&gas lobbies claim the EU needs to weaken its methane regulations or face gas supply shortages. But energy consultancy Rystad shows that to be not true. Oil&gas distorting reality to further its own commercial interests? @paulinesophieh.bsky.social thinks so. Via @politico.eu

We are delighted to welcome @abeba.bsky.social of @aial.ie to @dcuioe.bsky.social for the Irish Learning Technology Association's 25th annual EdTech conference, June 3- 4, 2026. #Edtech26 #edusky We greatly look forward to her keynote talk examining #genAI tools in education! ilta.ie/edtech-2026/...

Researchers at the @aial.ie @tcddublin.bsky.social & ADAPT have released new findings addressing the lack of visibility into AI training data despite it being an obligation under the AI Act. Learn more: www.adaptcentre.ie/news-and-eve... @researchireland.ie

We think the existing consumer laws already show how several of these issues are unfair in the Unfair Commercial Terms/Practices directives, but also that the situation requires further concrete action and changes through our 12 recommendations. 18/

Issue 7: The lack of provider's responsibilities is also at odd given that they decide how the models are developed, what data is used in training, and how the model is configured. An example of this is the use of Grok to produce CSAM where X reiterated only users are responsible for outputs. 12/

Issue 5: The manner in which users are assigned full responsibility and liability over inputs and outputs is at odds with the actual ability of users to control only their prompts but not the underlying AI model or the system prompts which are solely controlled by the provider. 10/

Issue 4: GenAI services come with "no warranty", "no assurance", which raises the question on how they are being marketed – and if it is misleading to consumers. If true, this also has impacts on use of consumer protection towards AI Act enforcement based on the marketed 'purposes'. 9/

Issue 3: Terms mentioned service may change without notice, which is ambiguous on changes to UI/UX but also the quality, underlying model, and safety mechanisms. Consumers are given no information, no controls, and no ability to decide, while providers retain complete control. 8/

New paper from @aial.ie! @harshp.com, Dick Blankvoort, Adel Shaaban, @sashamtl.bsky.social & me We analysed 6 GenAI ToS--finding missing info, major power imbalances & user obligations that are impossible to meet without violating the terms arxiv.org/abs/2603.18964 & aial.ie/research/ter... 1/

Independent AI researchers face rising risks - from harassment to legal intimidation. A new resource by the Coalition for Independent Technology Research & @aial.ie shares tools on safety, wellbeing, and collective care. Read more ⬇️

at @aial.ie, we've been navigating emotionally challenging & politically contentious research. i initially set out to assemble internal resource but following conversations w @thecoalition.bsky.social, it developed into a blogpost worth sharing w others in similar situations aial.ie/blog/support...

🚫 If you or your teen see harmful or illegal content online, be sure to report it to the platform in question. And remember, this applies even if the content is AI-generated. Find more tips and resources around online safety at www.cnam.ie/general-publ... #AIAware #OnlineSafety

i conclude with a caution: while cognitive bias and algorithmic bias share some historical roots, uncritically applying methods from human cognition studies to model evaluation can be misleading, shortsighted, and unhelpful 12/

Black women scholars, alongside key real-world incidents, played a crucial role in bringing the concept of algorithmic bias into the mainstream 3/

fresh off the press from yours truly: oecs.mit.edu/pub/b61joemo... I offer an overview of algorithmic bias. I trace its historical roots, examine canonical scholarship and notable real-world incidents, and explore how algorithmic bias emerged as a field of study 1/

the lawsuit: knightcolumbia.org/documents/hp...

The Coalition for Independent Technology Research (CITR) has filed a lawsuit against the Trump administration’s censorship of noncitizen academics and independent researchers under its immigration policy: www.npr.org/2026/03/09/n...

The team (Dick Blankvoort, @harshp.com, & Maximilian Gahntz) will be presenting this work at FAccT 🎉 in June, and you can access the preprint and analysis here: aial.ie/research/gpa... If you have feedback and/or are interested to collaborate with them, please reach out to them. end/

This work is timely and is already being covered by media: www.techpolicy.press/how-big-ai-d... 15/

Despite their declared assurances or signing of codes of conduct/practice, no big provider has provided a summary. Only 4 providers have explicitly done so – and all are small orgs or open source developers. This sinks arguments against the obligation being burdensome or excessive. 2/

New paper from team @aial.ie! aial.ie/research/gpa... EU's AI Act Article 53(1)(d) is an obligation for GPAI model providers to publicly provide a 'summary' on their model’s training data. The team assessed published summaries along 6 dimensions & found that all big providers failed on all 6. 1/

As Director of @aial.ie at ADAPT & @tcddublin.bsky.social, Abeba and her team have since produced influential research on #surveillance practices, secured #EU funding to develop audit frameworks, and contributed to both national and #global #AI policy discussions. More about the lab here: aial.ie

Dr Abeba Birhane @abeba.bsky.social @aial.ie, ADAPT & @tcddublin.bsky.social, features in @researchireland.ie's inaugural strategy: Curiosity, Capability, Competitiveness – Charting Ireland’s Research and Innovation Future 2026–2030. 📌 Read the strategy here: www.researchireland.ie/news/researc...

today we had a class of young students (12-14 yr old) came to visit the @aial.ie on campus. they were so keen to learn about AI and most importantly I was blown away by the type of questions they asked us: water consumption of data centers, how openai makes money, why RAM prices keep going up, ...

well, this was quick www.irishtimes.com/politics/202...

I appeared as an expert witness before the Joint Committee on AI at the Houses of Oireachtas (parliament of Ireland) to discuss "AI: truth and democracy" this morning. You can read my opening statement here: www.oireachtas.ie/en/publicati...

During her visit, @abeba.bsky.social will be taking part in our Technomoral Conversations event exploring AI narratives and counter-narratives – a collaboration w/ @braiduk.bsky.social & @edfuturesinstitute.bsky.social 🗓️ 11 Feb 18.00-19.30 📍 Edinburgh Futures Institute & online 🎟️ edin.ac/3MZEm0a

This February, we look forward to hosting @abeba.bsky.social for one week as our Distinguished Visiting Scholar! Dr Birhane founded and leads @aial.ie and is assistant professor of AI @tcddublin.bsky.social. She researches AI accountability with a focus on audits of AI models and training datasets.

if you’re passionate about AI accountability research and enjoy working in a vibrant lab with a multi-disciplinary team but not interested in doing traditional academic work, this position might be for you

I’m looking for my right-hand person to come help me run the @aial.ie - Job Title: Lab Coordinator, AI Accountability Lab (0.8 FTE) - Pay Scale: (€58,999 - €69,325 per annum pro-rata) - Closing Date: 11-Feb-2026 12:00 Apply here: my.corehr.com/pls/trrecrui... Main Responsibilities👇🏾

please apply and share with your network

I am assembling resources for @aial.ie to mitigate/reduce risks (due to our research) from potential: 1)retaliation, defamation lawsuit etc for work on politically charged topics 2)emotional harm from dealing with sensitive issues (CSAM, hate, etc) know of any helpful resources? pls share/repost

also, i'm looking for a phd student to join our team at the @aial.ie bsky.app/profile/abeb...

are you disgruntled by the current safety evaluation landscape? curious about what conceptual clarity, methodological soundness and rigour in AI evaluation might look like? if so, consider coming to dublin and doing a phd with me apply here: aial.ie/hiring/phd-a...

i'm hiring a postdoc to map the age-verification space and do some audits if you’ve worked in this area, pls apply: aial.ie/hiring/postd... pls share widely

Are you passionate about exploring what a conceptually cogent, methodologically sound, and well-founded AI evaluation and safety research might look like? Come do a PhD with us. Closing Date: 10 February 2026 Apply here aial.ie/hiring/phd-a...