Databricks has released a cheap-to-build LLM called Dolly that shows impressive instruction-following capabilities similar to ChatGPT. By taking an existing open source 6 billion parameter model from EleutherAI and modifying it slightly with data from Alpaca, Dolly demonstrates text generation, brainstorming, and open Q&A. The model is two years old and has only 6 billion parameters, but it performs well due to its focused corpuses of instruction-following training data. Databricks believes that the technology behind Dolly represents an exciting opportunity for companies to build their own instruction-following models. Dolly was evaluated based on the InstructGPT paper and was found to exhibit many of the same qualitative capabilities as ChatGPT.
It is demonstrated that a dated off-the-shelf open source large language model (LLM) can be given ChatGPT-like instruction following ability by training it in just 30 minutes on one machine with high-quality training data. Surprisingly, the ability to follow instructions does not require the latest or largest models, as evidenced by our 6 billion parameter model, compared to GPT-3’s 175 billion. The code for our model (Dolly) is open-sourced and we provide instructions on how it can be recreated on Databricks. It is believed that models like Dolly will help democratize LLMs, transforming them from something few companies can afford into a commodity that every company can possess and customize to enhance their products.
Read full article about Dolly: https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html
Databricks is a cloud-based platform founded in 2013 by the creators of Apache Spark, Delta Lake, and MLflow. It combines the best of data warehouses and data lakes to provide an open and unified platform for data and AI. With over 7,000 organizations using Databricks, it enables data engineering, collaborative data science, machine learning, and business analytics. Databricks is headquartered in San Francisco and has offices globally, with partnerships with Microsoft, Amazon, Tableau, and others, and is focused on simplifying and democratizing data and AI to help solve challenging problems.