Llama

What is Llama?

Llama is an open-source large language model created by Meta. It aims to provide individuals, researchers, and businesses with access to powerful AI models, tools, and resources. The company believes in democratizing access to these technologies and enabling innovation.

The latest version, Llama 2, is now available for research and commercial use. When users download Llama 2, they receive the model code, model weights, a user guide, a guide for responsible use, a license, an acceptable use policy, and a model card.

Llama 2 was pre-trained on publicly available online data sources. The fine-tuned model, Llama Chat, was trained using publicly available instruction datasets and over 1 million human annotations. These models are trained on 2 trillion tokens and have double the context length of Llama 1. Llama Chat models have additionally been trained on over 1 million new human annotations.

Llama Chat uses reinforcement learning from human feedback to ensure safety and helpfulness. The pretraining of Llama 2 involves using publicly available online data. An initial version of Llama Chat is created through fine-tuning. It is then refined using Reinforcement Learning from Human Feedback (RLHF). This includes rejection sampling and proximal policy optimization (PPO).

The company also offers Code Llama, a state-of-the-art large language model for coding. It can generate code and natural language about code from both code and natural language prompts.

Meta has announced Purple Llama, an umbrella project featuring open trust and safety tools and evaluations. The goal is to level the playing field for developers and advance safety in the AI ecosystem.