The Evolution of Machine Intelligence
The evolution of Machine Intelligence (or Artificial intelligence, AI) is one of the main trends shaping the future of humanity. It is a critical piece in addressing the aging population, stagnating productivity rates, and falling efficiency of energy production. It will also be crucial to our expansion beyond Earth. It may turn out to be the most important technology we have ever developed and has a promise of wonderful achievements but also a grave danger for humankind.
Below is our framework on how to think about AI. What it is. What’s the current state and how it may evolve in the future.
What is AI
The term AI has been around for a while and is an umbrella, that includes many computer science fields pursuing a goal of making machines to perform tasks commonly associated with intelligent beings. The definition is so broad, however, that we don’t find it practical to use. Instead, we look at machine capabilities as a spectrum — moving from simple, narrow, and straightforward tasks to more complex and fuzzy.
The further we move along this spectrum the more resources the machine needs (compute power, storage, bandwidth), the more sophisticated should be the hardware and software.
It looks like the hardware is the main driver of the advances. As an example, the concept of the Artificial Neural Network (ANN) has been around since at least the 1960-s, however, the adoption of the technology was hindered by insufficient computational power.
With advances in semiconductor manufacturing, more sophisticated hardware has been built, and new software has been developed to utilize the available power.
The first software has been written with simple imperative languages, like assembler (or before that, punched cards) because that was the most complex instructions the hardware was able to handle. Then came procedural languages, like C, a superstructure on top of assembler, dramatically simplifying creation of programs. The next huge step forward was object-oriented programming (OOP), the technique that needed much more powerful hardware, as well as more sophisticated software (the run-time environment) to run new programs. As of now, we are in the golden edge of Artificial Neural Networks (ANN). Each step above enabled machines to solve more and more complex tasks.
But there is a step-change from OOP and predecessors to ANN. For the former, you “code” the system actions, precisely describing what the system should do. The output will also be precise, and its quality will be defined by the code you wrote and will not change until you change the code.
ANN however is an empty vessel. When set up it can do nothing. You train it to perform the desired action by feeding in data and giving feedback — whether you like the output or not. The output is changing as the ANN strives to get more favorable feedback. This approach is called Machine Learning — the ability to learn without explicitly being programmed.
The benefit of the approach is that you don’t have to figure out the precise algorithm of how a specific action should be performed by the system. The system has to figure it out itself based on the received feedback. The trade-off is that the output is not precise as well as probabilistic.
These shortcomings are the price of the ANNs ability to perform actions that are way too complex to be hard-coded, including those where human experts do not even know the correct algorithm (e.g. there is no known algorithm to reliably differentiate a picture of a dog from a picture of a cat).
The actions we automate with ANN should have some tolerance for imprecision/errors, but it turns out most of the real-world complex actions do. In production, ANNs are complemented by either human-in-the-loop and/or traditional software filtering the critical errors out.
Below is a summary of approaches:
To learn more about what we mean by the “system capabilities’ exploration” , take a look at this note on improving a model results by including “Let’s think step by step” text in the instructions, or this one on the image generation model’s “secret language.”
Note, that in practice the most implementations of ANN-powered systems include different types of ANNs as well as relevant traditional software modules, to compensate for the ANN shortcomings.
When talking about AI aplications below we mean the combined systems.
AI adoption started from tasks with a narrower scope and higher tolerance for mistakes, like recommenders (showing “people also buy” widget, or choosing the next post in the social network feed), dictation, OCR, basic image recognition. Though such tasks used to be performed by hard-coded algorithms, the ANN approach demonstrates much better results.
The adoption is now expanding along the following three vectors.
Boosting current software capabilities in every field.
Just like in the examples above ANN techniques are replacing hard-coded algorithms in every industry from healthcare to military, improving data analysis, simulations, data transformation, and more. It includes industrial machinery and infrastructure, where the next generation chips and firmware will incorporate ANNs.
Augmenting knowledge workers/co-creation.
Early results show that ANN techniques can be used to enable co-creation where humans direct the machine and edit the output. There are early examples in co-writing, co-coding, co-composing, co-designing, co-science, and so on.
Augmenting physical labor.
The physical world is a challenging environment. While more narrow tasks, like keeping the car in a lane or picking a product from a box seem solvable, it is less clear if the current hardware and AI techniques will be enough to fully solve autonomous vehicles, drones, and more versatile robots.
That said, even solving for a subset of tasks above will have a significant impact on the economy. The installation of industrial robots already creates an equivalent of additional 1.3 million workers annually, and this number grew 3x over the last ten years . While most of the robots installed today are traditional ones, AI penetration is growing, especially in the segment of collaboration robots. In services, the adoption of robots is just emerging driven by fullfillment, cleaning, food services, and security.
The following figure lists some of the AI use-cases based on their maturity. All of them are still maturing, as the field is relatively young, but some are further ahead.
Based on our estimates, we believe the overall global market for the AI systems to be on the order of $10T broken down into the four areas below:
- Boosting current capabilities: ~$1T
- Physical labor: ~ $5T (ground transportation ~ $1T, physical labor in mining, construction, manufacturing, wholesale and retail trade, transportation and warehousing ~ $4T).
- Knowledge workers: ~$4T.
- AI infrastructure: hardware, tools, and services to build/run AI systems (~$1T).
Depending on how powerful the technology will turn out to be, we may see a market of a few trillion dollars over the next decade.
The AI system stack
Below are the main components of the stack.
- Talent. The engineers and researchers. But also the headhunters, outsourcing shops, freelance marketplaces, and other services that help connect these experts with the work.
- Hardware. The TPUs, GPU, and CPU are currently the main sources of computing power for ANNs, while novel technologies, like neuromorphic chips, are being developed. Initially, the hardware was configured and set up in-house, though currently developers increasingly leverage capacity available in the public cloud.
- Architectures. Since ANNs are not programmed but trained to perform specific actions, the job of the engineer is to come up with the most efficient architecture of the different types of ANNs (e.g. CNNs, transformers) and symbolic structures (e.g. tree search) based on the task at hand. The building blocks and reference architectures are developed by the open-source community, academia, and businesses, lead by the big technology companies (Google, Meta, Microsoft, Amazon, and others).
- Data. Can be collected and labeled in-house and/or purchased from 3rd parties. The data can also be artificially generated to cover cases that are not well represented in the existing datasets. While there are open source datasets, the data availability became one of the most important bottlenecks for the open-source/academia researchers (the other one is access to powerful hardware).
- Tooling. Software that helps build and manage AI applications. Tools for data management, labeling, deploying, analytics, etc. Also, services that collect or label data for clients.
- Model-as-a-Service (MaaS). While some app developers will design and train the ANNs from scratch, many will use MaaS, where the pre-trained ANN (model) is available and ready to be used with limited fine-tuning. The model does all the processing heavy lifting, while the app collects the data and presents the output in the form (or a set of actions) needed by the customer. It is especially relevant when the model is very large (e.g. GPT-3 or PaLM) and it would be impractical to deploy it separately for each app/customer. Leveraging MaaS also dramatically reduces time-to-market. The most advanced systems leverage multiple MaaS offerings from different vendors to achieve the best results.
- Application. Performs desired by customer actions, delivers end-user value.
- Integration. Implementation of application in the customer-specific environment, integration with other systems, including other apps. Fine-tuning pre-trained ANNs and MaaS using customer-specific data (e.g. adjusting the general language translation service to the customer-specific terminology), ongoing work with the customer data, and AI system monitoring to make sure it performs as planned.
- Embodiment. Sensors, manipulators, locomotion, etc. for the systems interacting with the physical world (EVs, drones, robots).
Known challenges of AI systems and areas of research
While modern AI systems demonstrate quite impressive results in tasks ranging from image recognition to playing Go and even explaining jokes, these are still the early days, and there are multiple areas for improvement, including abilities to better generalize, learn from fewer samples, process multiple types of data, and better energy efficiency. Many of them are interconnected and will likely require innovation at the levels of systems architecture, scale, and hardware architecture to achieve breakthroughs.
We expect these multi-year trends to shape the evolution of AI systems.
- Bigger models and more powerful hardware enabling more advanced actions performed by AI systems.
- Multimodality — the ability to process and relate information from multiple modalities, like text, audio, visual, etc. Models like Dall-E, Imagen, and NUWA are steps in this direction.
- Embedding AI in every device powered by a chip.
- Power law distribution of vendors in hardware (chips), MaaS, and edge autonomous software (powering drones and robots). A handful of companies will control most of the market.
- Maturation of DataOps/MLOps allowing for AI applications at scale in enterprise.
- Embedding AI into all modern software already used by businesses and consumers.
- Explosion of applications built on top of MaaS, where developers benefit from continuous improvements of MaaS and leverage multiple vendors for the optimal results.
- Transition from modular to end-to-end architectures.
- Expansion of real-time AI.
- Co-creation in copywriting, animation, music, computer games, and other creative tasks.
- Declining costs of industrial and service robots, including Robots-as-a-Service / Pay-as-you-use models and growing supply of low-cost robots.
- Growing robot density in the industry (current world average is 126 robots/10,000 employees, while in South Korea the number is 932). Gradual penetration of services.
- China leadership in AI. Scientists from the Chinese Academy of Science, Peking, and Tsinghua Universities are already competitive with the oldest and best universities in the world: Oxford, Cambridge, Harvard, Stanford. Overall, CCP considers AI a critical area for China’s global leadership. China is also by far the largest destination for industrial robots (48% of global installs in 2020).
To dive deeper here is a good overview of the AI evolution in 2021 by Nathan Benaich and Ian Hogarth.