Meta has launched Code Llama, a state-of-the-art large language model (LLM) designed to generate code, marking a significant step in the realm of AI-assisted coding. With the release of Code Llama, Meta aims to enhance developer workflows, making them more efficient and reducing repetitive tasks. The tool is expected to lower the barrier for newcomers to coding and serve as a valuable asset for experienced developers.
Code Llama will compete with existing AI coding tools, such as OpenAI's Codex and GitHub Copilot. Meta built the tool on top of Llama 2 and offers it in three variations: the foundational code model, a Python-specialized version, and an instruct version fine-tuned for understanding natural language instructions.
In its official announcement, Meta describes a suite of AI-driven tools that will help coders automate and complete tasks with the Code Llama model:
- Code Specialization: Meta's says that they designed Code Llama as a “code-specialized” version of Llama 2. It can generate code, complete code, create developer notes, documentation, and even assist in debugging. The model supports multiple programming languages, including Python, C++, Java, PHP, Typescript (Javascript), C#, and Bash.
- Model Variations: Meta has released three sizes of Code Llama, with 7B, 13B, and 34B parameters. Each model underwent training with 500 billion tokens of code and code-related data. The 7B and 13B models cater to tasks requiring low latency, such as real-time code completion, while the 34B model provides the best results for coding assistance.
- Extended Context: All models can handle up to 100,000 tokens of context, allowing users to provide more context from their codebase to make the generated code more relevant. This feature proves especially beneficial for debugging in larger codebases.
- Fine-tuned Variations: Meta introduced two additional variations of Code Llama: Code Llama – Python and Code Llama – Instruct. They designed the Python version specifically for Python code, while the Instruct version aims to better understand natural language instructions.
In benchmark testing, Code Llama outperformed other publicly available LLMs on code tasks. Specifically, the 34B model scored 53.7% on HumanEval and 56.2% on Mostly Basic Python Programming (MBPP), matching the performance of ChatGPT. Meta also emphasized the importance of building AI models responsibly and highlighted their efforts to ensure Code Llama's safety. As part of their safety measures, they evaluated Code Llama's risk of generating malicious code and found it provided safer responses compared to other models.
Open Approach and Licensing
Meta adopts an open approach to AI, releasing Code Llama under the same community license as Llama 2. They believe that an open approach fosters innovation and safety in AI development. The company encourages the community to evaluate the capabilities of Code Llama, identify potential issues, and address vulnerabilities.
Some companies claim to be “open sourcing” their AI models, Meta among them, but a recent study shows that this is not really true. The study, conducted by AI researchers from Radboud University in Nijmegen, Netherlands, reveals that some of the most powerful generative AI LLMs are not open-source at all, because the code that was used to train them is not shared with the public.
The study criticizes OpenAI and Meta for being the most secretive LLM producers and says that this is harmful for the AI community. It calls for more transparency and openness from companies, so that others can learn from their work and improve it.
The study examines several LLMs and rates them on how open they are, based on different criteria. The lowest-rated LLM is OpenAI's ChatGPT, which only meets two of the criteria: it has a document and a paper that describe its features and limitations, but they are very vague and unclear. LLaMA 2, produced by Meta, is slightly better than ChatGPT, but still very closed. It fails to meet most of the criteria for openness.
Growing Options in the AI Coding Market
Major tech companies are putting plenty of research and development into AI coding tools. In fact, in recent months we have seen a surge in new AI models for coding, including:
- One of the most prominent tools is GitHub Copilot, a joint project between Microsoft and OpenAI that uses GPT-4 to generate code suggestions based on users' inputs. GitHub Copilot is powered by OpenAI Codex, a generative pre-trained language model created by OpenAI. Earlier this year, GitHub showed its future vision with GitHub Copilot X, which integrates with OpenAI's GPT-4.
- Another tool that got an equity investment from Microsoft is Builder.ai, an AI software firm that offers Natasha AI product manager through Microsoft Teams. Builder.ai and GitHub Copilot are two different types of AI-powered tools for software development. Builder.ai is a no-code platform that lets users make apps by picking from various templates and features, without writing any code.
- Google also partnered with Replit to offer Ghostwriter, an AI tool that helps developers write code. The partnership also gives Replit developers access to Google Cloud and vice versa. Moreover, Google brought code generation and debugging to its Bard AI chatbot. Users can write their coding questions or requests in natural language, and Bard will generate multiple drafts of possible responses for them to pick from.
- Amazon launched CodeWhisperer, a free AI tool that competes with GitHub Copilot. It works with Python, JavaScript, and Java languages and integrates with popular IDEs like PyCharm and Visual Studio Code. It helps users write code faster and easier. CodeWhisperer is integrated with AWS services and tools, such as Lambda, CloudFormation, and Amplify.
- In May, Meta introduced CodeCompose, an AI-powered tool that offers code suggestions for various languages including Python, as developers type in Integrated Development Environments (IDEs) like VS Code. The tool can utilize its understanding of the surrounding code to provide enhanced suggestions.
- Chinese company Baidu is also competing in this space with its own coding AI. Comate is compatible with mainstream Integrated Development Environment (IDE) frameworks and supports more than 30 programming languages, with a strong emphasis on C/C++, Python, and Java.
- OverflowAI is an AI coding solution from Stack Overflow that was launched last month. OverflowAI is a web-based tool that allows users to input natural language queries and get code snippets generated by a deep learning model trained on millions of Stack Overflow posts and other sources. The platform supports various programming languages, such as Python, C#, Java, and SQL, and can handle tasks such as data analysis, web development, machine learning, and more.
Last Updated on October 10, 2023 3:45 pm CEST by Markus Kasanmascheff
