What is Meta's CodeCompose?

Transformer LLMs are changing our relationship with code.

Photo by Markus Spiske on Unsplash

In recent times I’ve been trying to bring software engineers news on the various A.I. capabilities and LLMs upgrades for coders.

So while GitHub Copilot thrives along with ChatGPT, what Google, Amazon and Meta will build I think will keep improving the field. The coding tool, called CodeCompose, isn’t available publicly — at least not yet. But Meta says its teams use it internally to get code suggestions for Python and other languages as they type in IDEs like VS Code.

What is It?

CodeCompose is based on the InCoder LLM that merges generative capabilities with bi-directionality. They have scaled up CodeCompose to serve tens of thousands of developers at Meta, across 10+ programming languages and several coding surfaces.

In the paper Meta discusses the unique challenges in terms of user experience and metrics that arise when deploying such tools in large-scale industrial settings. Curiously they also present metrics from their large-scale deployment of CodeCompose that shows its impact on Meta's internal code authoring experience over a 15-day time window, where 4.5 million suggestions were made by CodeCompose.

Code-Completion LLMs are Getting Better Fast

If you have tried ChatGPT or GitHub Copilot you will understand how these LLMs trained on code work and are getting better. The folks at Meta have also put numbers on it.

Quantitative metrics reveal that (i) CodeCompose has an acceptance rate of 22% across several languages, and (ii) 8% of the code typed by users of CodeCompose is through accepting code suggestions from CodeCompose.

Qualitative feedback indicates an overwhelming 91.5% positive reception for CodeCompose. In addition to assisting with code authoring, CodeCompose is also introducing other positive side effects such as encouraging developers to generate more in-code documentation, helping them with the discovery of new APIs, etc.

• 8% of Code suggested is Accepted

• 92% favorable positive reception within Meta to CodeCompose

On Google’s side, Paid Colab users will also have access to autocomplete suggestions, which can save time and reduce errors when coding. A Codey Bot.

Developers will be able to use the new Codey AI assistant based on Google’s new PaLM 2.

More on CodeCompose

CodeCompose is based on the InCoder LLM and has been trained on 10+ programming languages at Meta. As such, it inherits the property of LLMs to be multi-lingual.

The largest of several CodeCompose models Meta trained has 6.7 billion parameters, a little over half the number of parameters in the model on which Copilot is based.

For all intents and purposes like the Coding tools of other BigTech companies, Meta’s code-generating AI model is a significant step forward in aiding developers and enhancing their productivity.

It’s not clear what will happen with the copyright though. TechCrunch notes that Microsoft, GitHub and OpenAI are being sued (Verge) in a class action lawsuit that accuses them of violating copyright law by allowing Copilot to regurgitate sections of licensed code without providing credit.

While Microsoft Copilot and ChatGPT seem to be the ones prefered by developers so far, I wonder if Google or Meta’s tools will be able to catch up in their efficacy. Amazon has of course CodeWhisperer.

The competitive arena of LLMs in Code is getting mighty interesting in 2023 it turns out.