Digital Economy Dispatch #159 -- AI and the Prediction Revolution

Digital Economy Dispatch #159 -- AI and the Prediction Revolution
26th November 2023

It’s tough making predictions…especially about the future!
Yogi Berra

There seems to be no stopping the AI bus. Every day it seems as though a more advanced AI capability is announced, an upgraded AI system is released, or an unexplored application of AI is brought into focus. Alongside this avalanche of news is an increasingly heated debate about AI’s impact on society. The “AI doomers” claim that we are presiding over the beginning of the end, while the “AI boomers” tell us the future has never looked so bright.

As with many of such trends, the reality is that these debates can be more confusing than illuminating. They frequently create a great deal of heat, but rarely provide much light. So much so, that it is easy to lose your way and forget what’s going on beneath the surface. Regardless of your position, this discussion really matters. We all need to find a way to cut through the noise to hold on to fundamental changes that define the shift that is underway.

What is it that defines this new era of AI? While many focus on the technological advances, just as important is to recognize the new economics of AI. Both are essential to AI’s success. Together they release an unstoppable force that is democratizing the power of making predictions.

In a world increasingly dominated by more and more data, AI acts as a powerful lens for interpreting and harnessing its potential. And lowering the cost of AI is the key. In essence, AI is a prediction technology. It excels at recognizing patterns in past data and applying those patterns to anticipate future events. When this is in the hands of everyone, the possibilities are endless. That’s it. It is as simple and powerful as that.

Self-Driving Cars: A Paradigm Shift in Transportation

While that may sound dry and meaningless, just stop for a minute and think about what it means when we have highly accurate, widely available, low-cost predictive power embedded into everything we do. This predictive capability can now be used to underpin a huge range of AI applications, from self-driving cars to personalized medicine. Let me elaborate.

The advent of self-driving cars and other autonomous vehicles epitomizes the transformative power of AI's predictive capabilities. These vehicles are equipped with a suite of sensors, including cameras, lidar, and radar, that continuously gather data about their surroundings. This real-time data is fed into AI algorithms that analyze the environment, predict the behaviour of other road users, and make informed decisions about the vehicle's trajectory.

With a constant supply of high-fidelity, accurate data, the core of self-driving car technology lies in its ability to predict and react to complex road scenarios. AI algorithms can detect and track objects, anticipate potential collisions, and determine the safest course of action. This predictive power enables self-driving cars to navigate roads with remarkable precision and safety, surpassing the capabilities of human drivers.

In this way, the driving experience is transformed. Yet, more importantly, the implications of self-driving cars extend far beyond an individual getting from point A to point B. They hold the potential to revolutionize urban mobility, reducing traffic congestion, improving road safety, and expanding transportation options for those with limited mobility. AI-powered self-driving cars can optimize traffic flow, reducing the time spent commuting and improving overall efficiency. Additionally, their ability to react faster and more accurately than human drivers can significantly reduce accidents, saving lives and minimizing property damage.

As a result, it is the concept of transportation that is being transformed. Self-driving cars represent a paradigm shift in transportation, demonstrating the transformative power of AI's predictive capabilities. As AI technology continues to advance, self-driving cars are poised to become a ubiquitous reality, shaping the future of mobility and redefining our relationship with transportation.

When AI Technology and Economics Collide

All this is only possible because we have dramatically changed both the technology and economics of prediction. Advances in digital technology have brought together the confluence of faster processors, vast data stores, and reliable communications infrastructure with more intelligent algorithms for analyzing data. Bringing all this technical capability online via easy-to-use tools through the cloud puts this in everyone’s hands. What this has meant is that in many situations we have been able to turn problem solving from reasonable guesses about what will happen next into robust predictions on which we can rely.

These are not new ideas. They were first described in detail by Ajay Agrawal, Joshua Gans, and Avi Goldfarb in their 2018 book "Prediction Machines: The Simple Economics of Artificial Intelligence". In that seminal book the authors argued that understanding this fundamental concept unlocks the true potential of AI. By recognizing AI as a prediction engine, we can effectively anticipate its impact on industries and society as a whole and rethink AI’s impact on business and society.

Their perspective is based on a very simple definition:

Prediction is the process of filling in missing information. Prediction takes information you have, often called “data,” and uses it to generate information you don’t have.

Central to their argument is the idea that AI's primary function is not to replicate human intelligence, but rather to create predictions based on existing input. AI algorithms are fed vast amounts of data, allowing them to recognize patterns and generate probable outcomes. This shift in understanding helps us move beyond the hype surrounding AI and identify its concrete applications in various domains.

One key example they present is the field of medicine. Traditionally, diagnosis relied heavily on a doctor's experience and knowledge. However, AI-powered prediction engines can analyze massive sets of medical data, including images, lab results, and medical records, to predict the probability of a certain diagnosis or the effectiveness of a specific treatment plan. This not only improves accuracy but also speeds up the diagnosis process, reducing suffering, saving costs, and potentially saving lives.

The Power of Prediction

These important observations from Agrawal, Gans, and Goldfarb have huge implications. AI predictions can revolutionize business strategies in many domains. By analyzing customer behaviour and market trends, AI can anticipate future demands and assist in developing targeted marketing campaigns, optimizing supply chains, and proactively addressing potential issues.

As we have seen, AI's predictive capabilities are being harnessed across a wide range of industries, transforming forecasting processes and driving innovation. Here are a few examples of how AI is being used for forecasting:

1. Demand Forecasting: E-commerce companies rely on sophisticated AI models to forecast product demand. These models analyze historical sales data, customer behaviour, seasonality, and external factors to accurately predict future demand. This allows businesses to optimize inventory levels, minimize stockouts, and reduce excess inventory costs, leading to improved customer satisfaction and operational efficiency.

2. Energy Consumption Prediction: Utility companies employ AI to forecast energy consumption patterns. AI models consider historical consumption data, weather conditions, and economic indicators to predict future energy demand. This helps utilities optimize resource allocation, reduce power outages, and plan for infrastructure investments.

3. Financial Market Forecasting: AI algorithms are being used to analyze vast amounts of financial data, including stock prices, trading volumes, and economic indicators, to predict future market trends and identify potential investment opportunities. While not foolproof, AI-powered financial forecasting is becoming increasingly sophisticated and influential in investment decisions.

4. Weather Forecasting: Meteorological agencies use AI to analyze complex weather data, including satellite imagery, radar observations, and atmospheric models, to predict weather patterns and generate accurate forecasts. AI algorithms can identify subtle patterns and correlations in weather data that traditional forecasting methods may miss, leading to more precise predictions.

5. Healthcare Risk Prediction: AI is being used in healthcare to analyze patient data, including medical history, lab results, and lifestyle factors, to predict the likelihood of developing certain diseases or experiencing adverse health events. This information can be used for preventive care, personalized treatment plans, and resource allocation in healthcare systems.

These examples demonstrate the diverse applications of AI in forecasting, highlighting its ability to extract insights from vast amounts of data and provide valuable predictions that inform decision-making across industries. As AI technology continues to evolve, its role in forecasting is likely to expand, leading to even more innovative and impactful applications.

Into the Deep Blue Sea

Recasting AI as a prediction engine helps us to gain a deeper understanding of the role and impact that AI can play in every area of our lives. However, we also must keep in mind that AI's predictive power is not without limitations. Bias in data collection can lead to flawed predictions, perpetuating existing inequalities. Additionally, overreliance on AI predictions may blind us to unforeseen circumstances and limit our ability to respond to change dynamically.

Therefore, as we integrate AI into our businesses and lives, it's crucial to keep the following lessons in mind:

• Focus on AI as a prediction engine, not a human replacement: Instead of seeking to create human-like AI, concentrate on leveraging its predictive power for specific tasks.

• Mind the data: Ensure your datasets are comprehensive and unbiased to generate accurate and reliable predictions.

• Human-AI collaboration is key: AI predictions are best used as a tool to support and enhance human decision-making, not as a replacement for it.

By understanding AI as a prediction technology and keeping these lessons in mind, we can unlock its true potential for a data-driven future while mitigating its inherent risks. This will create a future in which AI functions not as human replacement, but as a powerful partner in a collaborative human-machine relationship.