In a significant stride towards advancing artificial intelligence (AI) capabilities in code generation, Meta has released Code Llama 70B. This latest iteration, available for download here, maintains the open license tradition established by its predecessors, including Llama 2 and prior Code Llama models. The move is part of Meta’s commitment to supporting both research endeavors and commercial innovation.
A noteworthy addition to the Code Llama suite is the specialized version, CodeLlama-70B-Instruct, finely tuned for instruction-based tasks. This model impressively achieves a score of 67.8 on HumanEval, solidifying its position as one of the highest-performing open models currently accessible.
Code Llama 70B introduces three free versions tailored for various applications: foundational code (CodeLlama-70B), Python specialization (CodeLlama-70B-Python), and a model fine-tuned for natural language instructions (Code Llama-70B-Instruct).
Mark Zuckerberg, Meta’s chief, expressed enthusiasm about the release, stating, “We’re open sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be important for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here and looking forward to including these advances in Llama 3 and future models as well.”
Meta initially introduced its code generation model, Code Llama, in August. This model leverages both code and natural language prompts to generate code seamlessly. Similar to its predecessor Llama 2, Code Llama is open source and available for commercial use.
Built upon the foundation of Llama 2, Code Llama has undergone fine-tuning with specialized code-related datasets. Meta’s announcement encompasses four distinct versions of Code Llama—Code Llama, Code Llama Instruct, Code Llama Python, and Unnatural Code Llama—each varying in capacities with 7B, 13B, and 34B parameters. The release of Code Llama 70B signifies a significant leap in Meta’s ongoing efforts to advance AI capabilities and their practical applications in coding scenarios.
