There are a lot of rumors that any code fed to an online LLM is used to further train the LLM.
Generative AI is all the rage, and as artificial intelligence continues to advance, there are now A.I. systems that can analyze your code and provide suggestions for improving it.
Code generation can help developers in the same way that code auto-completion, refactoring hints and linting have helped developers in the past 5-10 years!
LLMs are cool because they can accurately predict the next token, but they’re far from beating the good ol’ human mind. Sometimes they miss the bullseye, by a lot.
On the surface, this seems like a fantastic tool – who wouldn’t want an expert AI system reviewing their code and giving tips? However, there are some important considerations on whether you should actually use these code-analyzing AIs.
My main concern is that AI is not human, and lacks the nuance and context needed to provide high-quality advice on complex code. In the future and with tools like Github Copilot, it will be possible for the AI to have more control of your codebase and give better advice. Today GitHub Copilot only knows about the contents of your current file but is otherwise blind to the rest of your codebase.
It cannot reason about the intent or requirements behind design decisions.
Because these LLMs are so new and often suffer from hallucinations, it seems pretty obvious that developers don’t yet feel comfortable blindly taking advice from an AI.
However, some benefits of using LLMs through an IDE’s extension can be for explaining code to developers, especially when there’s a lot of code: you can get an explanation by simply highlight a code fragment and clicking a button. This can be quite helpful for developers who are trying to learn a new codebase, or who are simply reading code quickly.
On Hackernews, there have been examples of people developing the same piece of code in different languages or using different frameworks, all thanks to the power of AI.
But the two things that I believe an AI with a good code-trained LLM can help at are:
Code improvements: For example, write a lot of documentation, improve the readability of code, add types to TypeScript etc.
Git workflows: Probably my favorite thing, is writing git commit messages from the changes
Overall there are clear benefits, not deep ones, in utilizing AI for code analysis, but also risks if solely relying on it for improving code.
The best approach is to use AI as another tool to improve developers’ jobs, because AI code assistance and analysis can significantly boost productivity and code quality.
