17 Oct Curiosity, now a survival necessity
In July, Yimian CEO Tony Ren unveiled four words of our core values. Curiosity was one of them.
It drives a newborn to explore the world. It drives us to discover business insights through data at Yimian. It’s becoming more critical to humans in the age of AI.
Today, curiosity is not exclusive to humans anymore. Machines are getting it too.
New developments in machine learning are making machines curious, and turning them from problem solvers to explorers.
It’s called Curiosity-Driven Learning, a new front in Deep Reinforcement Learning.
Reinforcement Learning is based on the maximization of rewards to the machine. These rewards are generally pre-defined externally by humans. It works fine in a simple and known environment. However, the machine will feel confused in a complex or unknown environment. It is not incentivized to explore and understand the unknown…unless it is driven by its internal curiosity “at heart”.
How to give a machine a curious heart?
Curiosity-driven learning rewards the machine for both exploring, and understanding, the unknown. This includes two parts: a curiosity function to capture prediction surprises, and two neural networks to learn the new environment.
Therefore, the machine curiosity is both stimulated and satisfied, both on its own.
To stimulate, the curiosity function is defined as the error between the machine-predicted outcome and the actual outcome, as a result of the machine action and the environmental states. The more surprising the outcome, the more exciting for the machine. This is similar to a newborn baby, who is more interested in kicking that new balloon than grabbing that familiar stuff animal.
To satisfy, the two neutral networks work hard on adjusting model parameters to improve its prediction accuracy. The machine is rewarded by minimizing the prediction error, therefore “satisfying” the curiosity. The baby gets a kick out of learning how the balloon responds to kicks.
Ok, a machine gets a curious heart. That’s cool. Wait a minute, is it good news or bad news to humans?
It’s a bad news to those who are not curious. Solving defined problems is becoming a commodity skill. That’s why AI is replacing so many human jobs of repeated tasks and routine work. If one cannot define one’s own rewards, cannot get excited by uncertainties, and cannot discover new territories, his or her job security is at high risk.
It’s a good news to those who are curious. Curiosity-driven machines are their assistants, who help curious people automate some iterations in exploration. Meanwhile, curious people give curious machines the higher-level guidance in structuring complex natural and social environments into abstract models. At least for now, humans are still superior in imagination, intuition, and emotion.
How to stay curious as a human? We can take hints from the algorithm:
#1. Get excited, not frustrated, by the unexpected. e.g. Enjoy the journey off the planned route after taking that wrong exit. e.g. Ask why the data shows an intriguing glitch last month.
#2. Intentionally steer toward the unknown. e.g. Do not always go to your favorite restaurants. Try that new Vietnamese place. e.g. Scrutinize data from an unfamiliar industry.
#3. Document experience into reusable knowledge just like “building a model”. e.g. Post tips on how to find the best diving sites in Malaysia in your WeChat group of divers. e.g. Answer a question on Zhihu.com or Quora.com.
#4. Define your own rewards. e.g. How about learning one surprising fact about airplanes every day?
For humans surrounded by intelligent machines, curiosity is not an option, is a necessity.
Cover photo: CEO Tony Ren unveils “Curiosity” as one of four core values at Yimian. Photograph by Lan.
For examples on how curiosity helped Yimian to uncover consumer insights, contact Rong Zhang at LinkedIn.