This really resonates with me. I'm early in my PhD and I'm researching a niche form of data compression and IC design. I don't use any AI at all in my research, I do it the super old fashioned way, I read papers cover to cover and sections of textbooks to familiarise myself with the field.
I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.
But it does make you wonder, why bother? Probably a frontier model could one shot my algorithm in a day or less. It's incredibly depressing. At least I'm not forced to use it now, but I fear I will have no choice after I join academia or industry in the future.
I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not.
As to why bother; our knowledge will keep expanding even with new tools, don’t believe either the hype or the predictions of doom.
>> I’m no big AI cheerleader, but why don’t you try these models to see what they offer - they might suggest some things you haven’t thought of, or save you some time in your research.
One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of their feet by those same companies.
Another reason is to avoid inadvertently plagiarising the work of other mathematicians. Any mathematical insight that comes out of an LLM is the result of training on the entire bibliography of mathematical research, but those insights are spat out of the models without attribution. If you use AI in your matematical research you are only using the work of others without even knowing who they are and what they contributed.
And yet another reason is to avoid polluting your mind with the ideas that come out of the AI. Maybe you get a hint that pushes you to one direction, when you would go into an entirely other direction without that hint. And then maybe that becomes a habit and you can't find new directions without asking the all-knowing oracle.
tl;dr: opsec, integrity and independence are the reasons to not use LLMs in your research. I don't.
Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest.
Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.
Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.
Thanks. Regarding plagiarism, I agree that it is possible to check; I don't know how easy it is. But I do note that the mathematicians who complained that their work was plagiarised by AI did not seem to realise that they, themselves, were using a plagiarism machine and were instead quite comfortable admitting that, yeah, we worked our Euler result out together with Sol, Claude and the gang.
To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work.
Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.
I think plagiarism machine is a stretch personally in this domain. In art or writing I could see it persuasively argued (see attempts to generate famous books or imitate illustrators).
Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work.
Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.
Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?
Yes absurd fantasies like it did that without human help and guidance. OpenAI found out there was a solution and humans attempted to brute force generating all possible solutions while threatening the mathematicians involved (generous interpretation), or stole some ideas and took shortcuts to know where to look and brute forced it so they could claim credit.
Neither looks good for OpenAI or those who support them.
Advanced studies give you more general skills than just the subject you study. You become more generally intelligent and appreciate the world differently. This is a very good feeling and valuable on a personal level.
I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.
But it does make you wonder, why bother? Probably a frontier model could one shot my algorithm in a day or less. It's incredibly depressing. At least I'm not forced to use it now, but I fear I will have no choice after I join academia or industry in the future.