How do you manage your daily health?　

As I worry about my lack of exercise and concerns about my physique, I often start strength training impulsively after seeing health check results or health-related programs. However, I don’t know what exercises to do or how much to do, and I end up doubting their effectiveness, which lowers my motivation and makes me quit. I’ve repeated this cycle countless times…

Wouldn’t it be great to have a trainer nearby who teaches effective exercises and supports you to keep going? Sharp is developing a virtual AI Trainer suited to a quitter like me. This AI Healthcare Trainer provides easy healthcare consultations and exercise advice through conversations in your living room.

Like the AI SMART LINK introduced on this blog before, it is equipped with our proprietary edge AI technology, CE-LLM (Communication Edge-LLM). It was exhibited at our large-scale technology exhibition event “SHARP Tech-Day ’24 ‘Innovation Showcase'” held in September this year, where it received positive feedback from visitors.

This time, we interviewed three people involved in the development of AI Healthcare Trainer, developers Yamashita and Niihara, and designer Miyakoshi, to learn about the inspiration, features, and special efforts behind its development.

●“CE-LLM” is a registered trademark of Sharp Corporation.

―　What is AI Healthcare Trainer?

(Yamashita) By utilizing a large-screen TV and camera, you can easily have healthcare consultations and exercises through natural conversations with AI Trainer in your living room. Just like having a trainer at the gym, AI Healthcare Trainer is a virtual personal trainer that exists within your living room TV. It is equipped with Sharp’s proprietary edge AI technology, CE-LLM.

―　What inspired and motivated the development?

 (Yamashita) I wanted to use TV technology to take on challenges that benefit people. With the company’s overall policy to strengthen healthcare-related businesses, we decided to develop a new solution related to health. Specifically, we combined three technologies, applications and camera technology developed through our TV experience, contactless healthcare sensing technology developed by another department within Sharp, and generative AI available in the market. This led us to the concept of AI Healthcare Trainer. We hope that using AI Healthcare Trainer will naturally help people adopt healthy lifestyle habits.

　

―　What exactly can it do?

(Yamashita) Through natural conversations with AI Trainer, it helps you plan your healthcare. You can consult based on the supervision of a physical therapist, enabling effective exercises.
For those who say, “I’m dieting but can’t keep it up,” it analyzes the current situation and suggests appropriate types and amounts of exercise. It also addresses specific concerns like “I want to lose ●kg” or “I want to strengthen my lower body” or “I want to slim down my waist,” introducing exercise menus tailored to those needs. It also has a function to reset and update goals based on daily exercise progress.

AI Healthcare Trainer can also provide consultations on the three elements of health (exercise, diet, and sleep), not just exercise.

Note: Final specifications are subject to change.

Additionally, when performing exercises based on consultation content, it uses the camera to analyze the user’s posture in real-time and provides advice. For example, when doing squats, it tells you to “straighten your back” or “open your knees a bit more,” teaching you the correct posture that you might not notice yourself.

Furthermore, it is equipped with Sharp’s proprietary contactless healthcare sensing technology, which uses the camera’s sensing function to obtain body information such as surface temperature and heart rate without attaching sensors, aiding in health management.

―　It seems like you can do effective exercises, but I’m worried that it might not be fun with an AI Trainer compared to a human…

(Niihara) I understand. AI Healthcare Trainer was also exhibited at last year’s “Tech-Day” event, and after that, we conducted a demonstration experiment in Chiba Prefecture with the cooperation of residents to make it more fulfilling. When we asked for feedback from participants, many said they couldn’t maintain motivation even though they felt they had to exercise. We thought that feeling close to the trainer might be an important factor in maintaining motivation. So, we made it possible to enjoy casual conversations unrelated to exercise to foster attachment. Let me show you an example.

Like this. AI Trainer also initiates conversations.

　

―　It certainly seems like you could get attached!

(Niihara) Additionally, if you ask, “Can you recommend a good sushi restaurant?” it responds with, “There are some good places in Kichijoji. How about a restaurant called ‘●●●●’? They also have delicious side dishes besides sushi.”

 (Miyakoshi) In terms of attachment, I think the character design and background images on the screen are also important. We named the character “Erika-chan,” and we went through over 50 revisions to achieve the current image, ensuring it has a clean and appropriate look for Sharp. We designed it to feel as if a real trainer is in the TV, changing outfits based on the conversation content and transforming the background to a kitchen or bedroom during food or sleep consultations.

―　The casual conversation seems very smooth. How did you achieve that?

(Niihara) We focused on how quickly AI responds after hearing the user’s voice. By incorporating edge AI technology, CE-LLM, the AI instantly determines whether the user’s question is simple or not. Simple questions are answered by the edge AI within the device for fast responses, while more complex ones are answered by the cloud AI with abundant information, ensuring smooth and natural conversations.

Additionally, instead of waiting to hear the entire sentence before responding, we developed unique methods to respond to smaller sentences, ensuring smooth conversations without stress.

Furthermore, as introduced in AI SMART LINK blog, we added steps to ask to confirm the content of the questions, making the conversation feel natural.

　

―　Usability is also important for fostering attachment. What efforts did you make regarding screen clarity and UI (user interface)?

 (Miyakoshi) We put a lot of effort into “zoning,” which decides where to introduce which information on the screen. Erika-chan holds a tablet, and the content of that tablet appears in the left zone of the screen. The right zone is for user operations (selecting exercises or menus). We thought it would be easier to see if the operation UI was on the right side. If you look at the conversation app screens you usually use, the left side is the other person’s post, and the right side is your post. We thought this would be more intuitive for those familiar with social media.

Additionally, the actual conversation (top: AI Trainer, bottom: user) is displayed at the top and bottom of the screen.

―　By the way, since it’s a living room TV, it will often be used by the whole family. Won’t the AI Trainer get confused?

(Yamashita) It’s okay (laughs). Each user can log in separately, so it won’t suggest exercises for the father to the mother.

―　Were there any other particular efforts or challenges?

 (Niihara) By changing customer account management from the device (TV) to the cloud, we aimed to make it accessible to more customers. This aligns with our “AIoT3.0” concept of IoT that is standing close by to people and society. It allows integration with other appliances, enabling automatic control of not only the TV but also Healsio Hot Cook, air conditioners, and air purifiers. This enhances convenience and expands usage conditions. For example, users can exercise in front of the TV at home, and then go to the office and continue the left exercises.

In fact, just before the Tech-Day exhibition this year, there was a policy change to cloud management, and we worked together with team members to rebuild it quickly. Additionally, we made detailed improvements to prioritize customer usability until the end, which was really challenging. However, thanks to everyone’s efforts, many visitors mentioned AI Healthcare Trainer as a memorable exhibit in the Tech-Day visitor survey.

●”AIoT” is a registered trademark of Sharp Corporation, combining AI (artificial intelligence) and IoT (Internet of Things) to connect everything to cloud AI, transforming it into a presence that stands close by to people.

―　Finally, please share your thoughts.

 (Yamashita) I want many people to use AI Healthcare Trainer and feel that it is a service that supports people and society. When it becomes practical, I look forward to seeing customers using it in their living rooms. I’m excited about it.

(Niihara) By storing information in the cloud, I believe we can further enhance the service. For example, we could provide local information like “Which day should you take out the trash?” I want it to be something that is rooted in the community, not just confined to each household.

(Miyakoshi) Besides enjoying the visuals, I hope it spreads as a new way to exercise with the advice of AI Trainer on TV. It would be great if more users find this service interesting and it goes viral on social media… (laughs). I also want young people who spend a lot of time on their smartphones to know that they can do something fun with their TV.

　

―　Thank you very much.

---

AI Healthcare Trainer is being developed with feedback from various companies and customers through exhibitions and demonstration experiments at events like Tech-Day. It might become a savior for someone like me who dislikes exercise. Let’s look forward to it!

（Public Relations H）

＜Related Sites:＞

■SHARP Blog：