Japanese supermarket chain uses AI to monitor staff smiles and speech

A Japanese supermarket chain is using an artificial intelligence (AI) system to assess and evaluate staff smiles and speech tones. This is in an effort to standardize employee behavior, but it has also reignited the debate on workplace ethics.

The Japanese retail brand called AEON, recently claimed it has become the world’s first business to incorporate a smile-gauging AI system. The system has been deployed across its 240 outlets across the country, according to a report by South China Morning Post. 

Also read: SoftBank will use AI to soften the tones of angry customers on calls

However, it is not the first attempt at toning employees’ or customers’ expressions in Japan. Previously, SoftBank also deployed an AI system dubbed “emotion-canceling” technology to tone down angry customer voices in call center calls. The bank also claimed the technology would help maintain good customer relationships while minimizing employee stress levels.

Japanese supermarket chain will use the AI system for employee training

The system incorporates more than 450 factors, such as a person’s voice volume, tone of greetings, and facial expressions. Some game features are also part of the AI system for staff training to polish their attitude by challenging their scores generated by the software.

The supermarket giant disclosed it ran the system on trial basis before full deployment. AEON said the system was tried in eight stores with 3,400 workers. The company noted, staff attitude improved by 1.6 times in three months after the deployment.

AEON also said its goal was to leverage the software to “standardize staff members’ smiles and satisfy customers to the maximum.”

Workers consider standardizing human behavior with AI unethical 

There are concerns over the policy of using such AI-based  systems as they may increase workplace harassment accidents by the side of customers. This is already considered a serious issue in Japan.

Locally known as kasu-hara, abusive language, and repetitive complaints are considered customer harassment. UA Zensen, the largest Japanese trade union that represents workers in various sectors, surveyed 30,000 employees from service and other sectors. Nearly half of the workers surveyed said they experienced customer harassment to some degree.

One of the participants said, 

“When the service industry workers are forced to smile according to a standard, it looks to me like another form of customer harassment,”

Another respondent said smiles should be beautiful and natural and should not be regarded as a commodity. A third one said that using systems to standardize human attitude seems cold and silly because he said people are different and they adopt different ways to express their affection.

Some observers are also comparing the strategy to McDonald’s Japanese arm’s strategy called “Smile zero yen,,” according to the report. The fast food chain’s menu mentions “Smile” for “0 yen,” since the 1980s, to remind its service staff to greet customers in a pleasant way.

Also read: A Japanese startup will translate manga into English with AI

The idea has become somewhat questionable in recent years because it is now considered an extra burden on the country’s lowest-paid workers. More companies are striving to maintain service quality without sacrificing staff morale, especially after Japan’s Ministry of Health, Labour, and Welfare published guidelines against customer harassment in 2022.

Recently, another initiative was taken with appreciation when a supermarket in the Japanese city of Fukuoka prefecture introduced extra slow counters for checkouts. Customers can spend 20 minutes completing their purchase without feeling rushed or stressed. The measure was taken to accommodate disabled and senior customers. Surprisingly, sales increased by 10% despite fewer customers using the extra-slow counters, according to a TV report.