“New CSAIL research highlights how LLMs excel in familiar scenarios but struggle in novel ones, questioning their true reasoning abilities versus reliance on memorization” https://news.mit.edu/2024/reasoning-skills-large-language-models-often-overestimated-0711

Reasoning skills of large language models are often overestimated | MIT News | Massachusetts Institute of Technology
news.mit.edu