DeepSeek AI: 3 Frugal Innovation Lessons

In 2014, I gave a TED talk on frugal innovation in Rio de Janeiro, Brazil.

I referred to frugal innovation as the art of “creative problem-solving in the face of extreme limits”.

In my talk, I introduced the 3 core principles of frugal innovation.

I am delighted to see DeepSeek AI perfectly practicing the 3 frugal innovation principles:

1) KEEP IT SIMPLE

Rather than build big, bloated AI models, DeepSeek relies on *small* models that it efficiently trains.

“Using efficient knowledge transfer techniques, DeepSeek researchers successfully compressed capabilities into models as small as 1.5 billion parameters” reports TechTarget.

2) DO NOT REINVENT THE WHEEL (Reuse what you already have)

When, in October 2023, the US import export restrictions to China on advanced AI chips, DeepSeek leveraged the less advanced chips it can get its hands on.

“DeepSeek’s model was apparently trained by using 2,000 second rate Nvidia chips versus some 15,000 first class chips for models such as Meta’s Llama” according to The Economist.

As Pat Gelsinger, Intel’s former CEO, points out: “Engineering is about constraints. Export laws limited the available resources, so Chinese engineers needed to get creative — and they did. ”

3) THINK & ACT HORIZONTALLY — BY SCALING OUT DEVELOPMENT

By adopting an open-source approach, DeepSeek has “scaled out” (decentralized) the development of its AI technology by allowing millions of developers worldwide to modify, adapt, and improve its tech faster, better, and cheaper than big tech firms that use a “centralized, top-down” approach to develop and maintain large, proprietary AI systems.

The “decentralized” nature of DeepSeek tech development is an irony when you consider the fact that China is run by the communist party which is “centralized” !

— -

I find it funny that when you Google “Creative problem-solving in the face of extreme limits”, Gemini generates the following content:

“Creative problem-solving under extreme limits” refers to the process of finding innovative solutions to challenges when faced with severe constraints, such as limited resources, time pressure, or harsh environmental conditions, often requiring unconventional thinking and adapting existing tools or methods in new ways to achieve a desired outcome.

Key aspects of creative problem-solving under extreme limits:

Frugal innovation:
Utilizing minimal resources to create effective solutions, often seen in developing countries where access to technology or materials is limited.

Resourcefulness:
Making the most of available materials, skills, and knowledge to overcome limitations.

Collaboration and community support:
Leveraging collective knowledge and expertise to find creative solutions, especially in situations where individual capabilities might be restricted