While the reasoning capabilities of LLMs are steadily improving, they still lack the depth and nuance of human reasoning. Additionally, LLMs fall short if the data available for a topic on the internet is biased, for example, when certain viewpoints are under-represented online.
LLMs also suffer from hallucinations, occasionally generating inaccurate or misleading outputs in subtle ways that are difficult to detect.
Despite these limitations, LLMs might be able to generate a valuable set of initial arguments. Humans can use this output as a starting point to craft their own arguments, or study the retrieved data to craft better, more nuanced questions.