Elon Musk’s synthetic intelligence (AI) enterprise, xAI, made its massive language mannequin (LLM) Grok-1 accessible in open supply on March 17. Last week, the billionaire revealed on his social media platform X (previously often called Twitter) that the AI chatbot will likely be open-sourced. Now, it’s accessible for researchers and builders. Notably, the xAI builders acknowledged that solely the pre-trained LLM has been launched in the public area, which implies whereas the weights and community structure might be accessed to construct on prime of it, Grok doesn’t include any coaching knowledge.
Announcing the open launch, xAI acknowledged in a weblog put up, “We are releasing the base model weights and network architecture of Grok-1, our large language model. Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.” The AI agency additionally famous that the LLM is being offered in open supply below the Apache 2.0 licence. Interested people can entry the AI mannequin from GitHub.
As per a report by VentureBeat, the Apache 2.0 licence contains business utilization, in addition to modification and redistribution. This means builders can construct on prime of the LLM, fine-tune it for particular purposes, and promote it. However, the mannequin can’t be trademarked and builders should point out any modifications made to the unique code.
But whereas a business licence is being offered, the model of Grok-1 that has been made accessible is from October 2023, earlier than the mannequin was educated utilizing knowledge from X and it offers a novel character to the Grok chatbot. This means any researchers or builders intending to make use of the mannequin should procure the knowledge themselves.
xAI acknowledged that the open-sourced Grok-1 is a 314 billion parameter Mixture-of-Experts (MoE) mannequin. The parameter dimension is far bigger than different LLMs in the public area, comparable to Meta LLaMa 2 or the Mistral 8x7B. The bigger the parameter dimension, the greater the context window of the AI. This permits it to offer extra contextually correct and thorough responses.
Grok was unveiled on November 3, 2023, as a chatbot designed to compete with the main gamers in the section. It was made accessible to a restricted set of customers who bought the X Premium+ subscription. Musk additionally stated at the time that Grok will get a standalone app.
