The AI Evaluation Substack

2025 October "AI Evaluation" Digest

"Beware; for I am fearless, and therefore powerful.”

23 mins ago • 

AI Evaluation

Is the Definition of AGI a Percentage?

Zachary Tidler, Marko Tešić, Lorenzo Pacchiardi, John Burden, Lexin Zhou, Manuel Cebrián, Fernando Martínez-Plumed, Jose Hernandez-Orallo

50 mins ago • 

Lorenzo Pacchiardi

, 

Lexin Zhou

, 

Manuel Cebrian

, 

Fernando Martínez-Plumed

, 

Jose H. Orallo

, 

Zack Tidler

, and 

Marko Tesic

3

September 2025

2025 September "AI Evaluation" Digest

What could possibly go wrong?

Sep 26 • 

AI Evaluation

9

August 2025

2025 August "AI Evaluation" Digest

Between a rock and a hard place

Aug 29 • 

AI Evaluation

9

July 2025

2025 July "AI Evaluation" Digest

Long live OpenML!

Jul 25 • 

AI Evaluation

11

3

June 2025

2025 June "AI Evaluation" Digest

Illusion is all you need

Jun 27 • 

AI Evaluation

11

1

May 2025

2025 May "AI Evaluation" Digest

Ethical standards in AI evaluation

May 30 • 

AI Evaluation

13

April 2025

2025 April "AI Evaluation" Digest

En attendant Turing: a Tragicomedy in Two Acts

Apr 25 • 

AI Evaluation

10

March 2025

2025 March "AI Evaluation" Digest

Overhauling Difficulty in Item Response Theory.

Mar 28 • 

AI Evaluation

9

February 2025

2025 February "AI Evaluation" Digest

It’s high time to change the paradigm.

Feb 28 • 

AI Evaluation

6

January 2025

2025 January "AI Evaluation" Digest

Distil, baby, distil!

Jan 31 • 

AI Evaluation

11

December 2024

2024 December "AI Evaluation" Digest

Think before you act!

Dec 27, 2024 • 

AI Evaluation

10