"... the situation with compressed sensing can be compared with the current situation with modern large language models and related AI tools. Here, the field is almost entirely dominated by empirical research, often from industry rather than from academics. As such, there is a lot less clarity on what the key ingredients are to make a given AI technology suitable for a given use case; there are spectacular successes that cannot be replicated, next to seemingly promising uses that hit an unexpected wall (or, conversely, unlikely applications for which AI tools are far more effective than anticipated). With the notable exception of the mathematics of optimization and numerical linear algebra which are both somewhat mature, most of the theoretical mathematical framework needed to explain the strengths and weaknesses of AI is still in its infancy. (Though I would say here that the main bottleneck is not exactly lack of funding in basic research in the foundations of AI, but more that the mathematics itself is not understood to anywhere near the level we would like.)..."
I wonder how much large language models are assisting current PhD candidates seeking a doctorate in large language models,
"... the situation with compressed sensing can be compared with the current situation with modern large language models and related AI tools. Here, the field is almost entirely dominated by empirical research, often from industry rather than from academics. As such, there is a lot less clarity on what the key ingredients are to make a given AI technology suitable for a given use case; there are spectacular successes that cannot be replicated, next to seemingly promising uses that hit an unexpected wall (or, conversely, unlikely applications for which AI tools are far more effective than anticipated). With the notable exception of the mathematics of optimization and numerical linear algebra which are both somewhat mature, most of the theoretical mathematical framework needed to explain the strengths and weaknesses of AI is still in its infancy. (Though I would say here that the main bottleneck is not exactly lack of funding in basic research in the foundations of AI, but more that the mathematics itself is not understood to anywhere near the level we would like.)..."