Calling OpenAI, Musk's Grok was launched

Article source: Yiouwang

Author: Fan Tan

is worthy of you Musk, and he is really in the next big game of chess.

Image source: Generated by Unbounded AI

Just this past weekend, Musk’s xAI suddenly made a move to unveil its own large model Grok before the OpenAI developer conference. Not only is the time in a hurry, Grok also doesn’t use Python, Musk led more than a dozen people to burst their livers for two months, and finally came up with a decent work.

Experience Links:

Musk’s Grok has added another fire to this year’s destined AIGC and large-scale model tracks, and things are getting more and more fun.

01 About Grok

Unlike other common GPT-like models, Grok can get the latest knowledge from X’s tweets in real time, such as Musk’s interview session last week. Massive, real-time, and unique X tweets are undoubtedly Grok’s wonderful moat. And Grok also has a lot of personality,Unlike the common one-board and one-glance AI assistants, the words spoken are somewhat prickly and humorous, and of course it can also be said to be yin and yang. For example, Grok said to Sam Bankman-Fried, who was recently tried in New York, the famous explosive head SBF in the cryptocurrency world: “Oh, my dear humanity, I have great news to tell you here!” Our friend Sam Bankman-Fried, the former crypto tycoon, was found guilty in his fraud trial. Can you believe it? ”

In addition to this regular conversation, Grok has other features, such as multiple conversations being output at the same time, and asking other questions while writing code. After an unsatisfactory answer is regenerated, you can expand the timeline to navigate directly to a different version of the answer, or even use the built-in markdown editor to manually modify the AI’s answer and continue the conversation.

According to the xAI team, Grok aims to mimic The Hitchhiker’s Guide to the Galaxy, answering questions with “a little bit of intelligence” and a “little bit of rebellion.” The developers also have a special tip: if you don’t like humor, don’t use Grok! Strictly speaking, Grok can answer almost any question a human asks, and even if it doesn’t ask, it can ask some suggestive questions. **Speaking of which, the biggest highlight is that based on the massive amount of information in X, Grok can provide real-time information in the real world. **

Coincidentally, Grok is a product with the same name, and the current version is Grok-1, and its prototype Grok-0 began training after xAI was announced. At the moment it is confirmed that the performance of the Grok-0 is close to that of the Llama 2-70B, but the number of parameters is about 33B, which is less than half of it. Over the past two months, xAI has enhanced Grok’s inference and coding capabilities by upgrading to the current version of Grok-1.

The test results show that Grok-1 not only significantly improves its predecessor in math, code, and multidisciplinary knowledge assessments, but even outperforms GPT-3.5. After all, it is Musk’s own hands-on work, and Grok’s unusual path and quality are expected. You can’t find the familiar PyTorch or Tensorflow in the Grok system, and there is not even a Python component, but instead choose the Rust programming language and the deep learning framework rookie JAX.

As for the reason behind it, it is not difficult to understand, xAI believes that the large model training process is like a speeding train, and if one of the sections derails, it will inevitably lead to a series of catastrophic consequences. To this end, xAI has built a dedicated distributed system to ensure that in the event of a failure, the type is immediately identified and automatically handled. In the process, xAI found the Rust language to be ideal for building scalable, reliable, and maintainable infrastructure.

02 Musk and xAI

In recent years, Musk’s reputation has become more and more in China, but many people don’t know, **In addition to Tesla, SpaceX, Neuralink, etc., Musk also has several companies that are not very well-known in China, such as Zip2, X.com, PayPal, all three were founded at the end of the last century. Zip2 develops online city guides for newspaper and magazine publishers, including maps, navigation and yellow pages; X.com is mainly engaged in online financial services and email payment business, and is one of the first online banks to cooperate with federal insurance in the United States; PayPal is the first company in the world to research e-wallets and mobile payments. Looking at these three companies together, did you find anything interesting? Domestic Alibaba has also gone through such a development route.

After entering the new century, Musk also opened several companies: SolarCity, Hyperloop, and The Boring Company, which are not well-known in China. Among themSolarCity is mainly engaged in household photovoltaic power generation business (solar is French “sun”); Hyperloop, which literally translates to “hyperloop train”, is a high-speed passenger and freight transportation system based on vacuum tubes, which is faster than airplanes, and is still in the infrastructure construction stage, and Musk even wants to use it for transportation on Mars in the future; The Boring Company is mainly engaged in urban infrastructure, especially the construction of railway tunnels, which can be used for the research and development of hyperloop in the future, and infrastructure facilities such as tunnels can remain universal. Boring has the meaning of “burrowing” and “boring”, so The Boring Company is often translated as “drilling company” or “boring company” in China.

At this point, I believe many people can have a deeper understanding of Musk, whose ideas are sometimes called “technological idealism”, which has long existed in Silicon Valley. For example, Bill Gates hopes that computers can enter thousands of households, Larry Page hopes that Internet information sources should be diversified, and Steve Jobs hopes that electronic products can also be works of art… There are many ideas like this, and it can be said that this is one of the important reasons why Silicon Valley has been where it is today.

**When OpenAI was founded in 2015, the starting point of Musk and other founders was to bring together the world’s top AI talents and develop the most cutting-edge AI technology under the premise of non-profit. **This development model is more aimed at semi-academic and semi-corporate research entities like Bell Labs that can generate huge social value, and the direct competitor is DeepMind, which was acquired by Google at the same time. Therefore, OpenAI was originally born for AI technology, rather than deliberately choosing to study the underlying technology. In addition, OpenAI has had a series of adventures and good fortunes from the beginning, and it can be said that Musk’s appeal alone has gathered top billionaires, scholars and a large amount of public opinion. But the reality of OpenAI’s predicament at the time proved that the ideas of Musk and other founders were too idealistic, and it could almost be said that they were completely unrealistic. Moreover, in 2018, Tesla also began to gradually enter the AI field, in order to avoid conflicts with OpenAI in the future, coupled with disagreement with OpenAI’s management and its attitude towards AI security, Musk and other founders have left OpenAI. Later, with Sam Altman as CEO, OpenAI transformed into a limited for-profit entity, exploring a new balance between commercialization and non-profit organizations by limiting the profit ceiling and revenue types.

In April this year, Musk revealed to the Financial Times that he wanted to set up a new AI company, saying that he was “concerned about the current training systems of major AI companies”, and was worried that these products would fall into the trap of “political correctness”, so he wanted to establish a brand or company such as “TruthGPT”. In July, xAI was officially announced, working closely with Tesla, with one of the goals of building a large model that can perform advanced logical reasoning, and strive to surpass other competitors of the same type on the market.

At this point, Musk still hasn’t given up, and still wants to compete with OpenAI and ChatGPT, of course, it can also be said that Musk wants to use xAI to prove that even if he leaves OpenAI, he can still make a difference in the field of AI. In addition, after the debut of GPT-4 in March this year, OpenAI also stated that it would not open source for the first time, which caused a ridicule from the outside world that “OpenAI is not open”. Although OpenAI’s true thoughts are unknown to the outside world, and it is understandable, this can actually be regarded as the epitome of Musk’s contradiction with it.

As mentioned earlier, Musk is a staunch “technical idealist”, and one of the original intentions of participating in the establishment of OpenAI was to crack the secrets of AI and promote it to the world. Therefore, in Musk’s view, GPT’s black box system is obviously contrary to its philosophy, although GPT-4 can indeed become popular all over the world and cause a new wave of the times. **I believe that some people will say that Musk is hypocritical, he can’t eat grapes and says that grapes are sour, looking at OpenAI’s eyes that are full of money, only Musk himself knows best.

03 The future of Grok

A few months ago, Musk announced the founding of xAI to the world by saying on Twitter, which had not yet changed its name, that “the goal of xAI is to understand the true nature of the universe.” A few months later, the technomaniac team came up with the first results. Even Musk himself couldn’t resist advertising it on X in advance: “In some ways, it’s the best at the moment.” ”**

It’s worth noting why xAI named its product Grok? Some say it was inspired by a similar concept in American science fiction writer Robert Heinlein’s Stranger in a Strange Land. In the book, Heinlein created a set of Martian languages, in which Grok literally means the modal word “drink”, but also has a deep meaning, symbolizing deep meanings such as understanding, empathy, and human connection, rather than just superficial or intellectual assimilation.

Therefore, xAI chose to name its large model Grok, which also expresses Musk’s consistent vision for AI in terms of understanding and communication, hoping that AI can go beyond mechanical information processing, truly understand human emotions and needs, and achieve deeper and more intuitive human-computer interaction and understanding. In the modern science fiction community, the term is often used to denote an in-depth understanding of complex issues. This statement was also endorsed by Musk himself.

How do you use xAI, which looks good and powerful? At present, there are two ways, 1. Log in to the official website with an account; 2. Upgrade to Premium+ subscribers for X. Musk said that once the early testing is over, Grok will be available to all Premium+ subscribers for $16/month, compared to the competitor’s ChatGPT Plus subscription price of $20/month. At this juncture, Grok’s servers have gone down, which is evident in its popularity.

Interestingly, Musk was also a big fan of the “AI apocalypse” before, and signed an open letter with a group of AI and technology bigwigs, calling for a moratorium on the development of AI more advanced than GPT-4 for at least half a year. And now, while these bigwigs are passionately debating the importance and necessity of AI regulation, Musk has quietly trained Grok, and even released a big move on the eve of the OpenAI developer conference. It also confirms the sentence: “Watching a good brother make money is more uncomfortable than counting the money he loses.” Of course, as someone who has always been able to achieve his own boasts, Musk’s self-justification has always been the industry benchmark: “We believe that AI has great potential to contribute important scientific and economic value to society, so we will strive to put in place reliable safeguards against catastrophic malicious use.” We will do our best to ensure that AI remains a force for good. ”

Even Musk released such words: “In the future, Tesla’s computing power will be used for the inference of large models.” Even with robot-driven taxis, cars still only run 1/3 of the time a week, and the rest of the time computing power is used for distributed inference operations such as SETI. In this way, Tesla will have the most large model inference computing power on the planet! "It’s worthy of you Musk, and you’re really in the next big game of chess.

As the old saying goes, “it’s better to get in the shade with your back to a tree”, and while Grok can use X to access search tools and real-time information, like all competitors on the market, Grok still can’t avoid the common problem of large models: hallucinations. In this regard, xAI believes that the most important way to solve the limitations of the current system is to achieve reliable reasoning. Specifically, there are the following:

**1. Scalable oversight possibilities with tool-aided assistance. **It is also difficult for Grok to provide consistent and accurate feedback, especially when dealing with long code or inferences of complex inference. This is where AI can assist in scalable oversight by finding references from different sources, verifying intermediate steps using external tools, seeking human feedback, and more.

**2. Integrated form verification to ensure safety, reliability and grounding. xAI expects to develop AI’s reasoning skills with guaranteed accuracy and explainability, so that systems can be evaluated without human feedback or real-world interaction. The most immediate goal of adopting this method is to ensure the accuracy of the code, especially to verify the security of the AI in form.

3. Long context comprehension and retrieval. A large model that can effectively discover useful knowledge in a given environment is at the heart of producing truly intelligent systems, and xAI is working on how to enable AI to discover and retrieve information when needed.

4. Adversarial robustness. Many examples show that vulnerabilities in AI systems, whether during training or use, can cause them to make serious mistakes. These vulnerabilities are also long-standing weaknesses of large deep learning models. xAI is committed to improving the robustness of large models, reward models, and monitoring systems.

5. Multimodal functionality. **Grok is not currently equipped with visual and auditory functions, and xAI will work to develop its multimodal capabilities for wider applications.

Decades of career have shown more than once that xAI and now Grok are at best part of Musk’s technological ambitions, and his vision is by no means limited to the current hot AIGC and large model technology. Whether it is Neuralink, which studies brain-computer interfaces and implantable microchips, Optimus, which is a humanoid robot, Dojo, a supercomputer that simulates the human brain, and now Grok, Musk’s recent projects can all point to the same goal: artificial general intelligence, or AGI, that is, to create an intelligent system that can perform various tasks like humans, with human-like intelligence, autonomous decision-making and learning capabilities, and can perform a variety of tasks. and have the ability to understand and reason about the world and solve a variety of complex problems.

Of course, the realization of this ultimate goal is by no means easy, and the inherent characteristics of AI technology determine that it must have results in order to be more easily and directly perceived by the outside world. But Musk is not an ordinary person, but a person who can always realize the boast of the bull, and has now set off on this challenging road.