Q: Please briefly introduce yourself, the Samsung R&D Institute United Kingdom, and the work that goes on there. What projects are you working on?

Samsung R&D Institute United Kingdom (SRUK) was established in 1996 as the first European R&D center outside Korea. As a European Center of Excellence based in the West of London, we can leverage London’s cultural, financial, and innovative features. At SRUK, we are more than a business; we are an R&D global community, hiring and developing the best talent from around the world with over 30 nationalities represented in our office. We are at the forefront of technology with new areas of focus in Artificial Intelligence (AI), Data Intelligence (DI), and Communication Research, along with our already established expertise in our specialist areas of Graphics, Security, Internet of Things, and Telecoms.

I am Mete Ozay, a Principal AI Specialist and the Head of the AI Advanced Research Team (ART). As a Principal AI Specialist, I have been collaborating with different teams from various departments in SRUK and also with other overseas research centers. Within the SRUK, we have been developing innovative solutions to AI challenges for Samsung products aiming to improve the customer and mobile user experience. We have in-depth knowledge and extensive experience with developing on-device and personalized AI systems for various AI tasks including but not limited to 2D/3D computer vision, computer graphics, natural language understanding (NLU), digital security, audio & speech processing and recognition. We have been commercializing our AI systems in several flagship Samsung products, such as Galaxy and Fold/Flip devices.

My team has been developing solutions to fundamental AI problems addressed in the aforementioned tasks and systems, such as compression of hyperscale AI models, optimizing compression, and lifelong learning accuracy trade-offs using continuously generated labeled and unlabeled data at different scales. We have been researching and developing task and product-agnostic methods and systems in physical and virtual spaces, which can support multiple R&D teams in different product-driven projects.

In one of our recent R&D projects, we have been developing AI systems entangling multiple AI models to design and develop new products. In the project, we have been posing novel and peculiar research problems produced by the products and developing state-of-the-art solutions to the problems. The project targets providing users with integrated AI systems that perform multiple AI tasks in harmony and learn from users, enhancing their experience in both physical and augmented reality environments.

Q: Please tell me about the importance of your research field or technology.

Samsung Research provides unique opportunities for AI researchers and engineers to work on cutting-edge technology and scientific research. In the Samsung ecosystem, we can access a wide range of products at different scales, such as earbuds, watches, appliances, phones, PCs, and TVs. Each product deploys multiple AI systems enabling interaction with users at multiple touchpoints, such as camera & display and microphone & speaker. Our researchers and engineers can define novel research problems elucidating the challenges of these products, develop innovative solutions to the problems, and deploy the solutions on real-world systems observing the impact and recognition of their solutions in the users’ everyday lives.

In the AI habitat of Samsung products, we redefine Mobile eXperience (MX) and Customer eXperience (CX) of users from the perspective of AI R&D. For this purpose, we disentangle AI models of the user-product symbiosis. Inspired by the hypothesis that a user is his/her connectome, we identify customers by their experience with AI systems. By MX, we refer to the users’ experience with an AI system on a mobile product such as Bixby on a Samsung phone. By CX, we consider the users’ experience with the brand through AI services distributed among multiple Samsung products in the Samsung ecosystem.

As AI researchers and engineers, we have been characterizing user experience with different devices for particular tasks employed in individual and collective AI services and products. In the Advanced Research Team, we work on developing mathematically correct, scientifically examined and verified, and well-engineered AI systems. For this purpose, we have been studying and advancing mathematical and computational models to understand user-device interactions better and produce personalized solutions for users. These exclusive solutions enable users to own their unique AI services in discrete products. This form of user-AI system symbiosis enables AI models to evolve and learn from the users’ experience with the product. For instance, our AI systems can learn new languages as users learn and speak them on their mobile devices. Our AI models can also recognize users’ new friends based on their personal photos considering their privacy and security.

AI ownership of users scales up to multiple devices residing in the Samsung ecosystem. AI residency of models can make AI systems deployed on users’ individual devices more intelligent as the models travel in the ecosystem. Additionally, the models owned by different users can learn from each other as the models interact digitally, even if users do not interact physically. For instance, we have been developing AI systems and personal AI models that can learn to recognize Korean speech in an ecosystem in Korea while the personal models can teach German to the other host models in the ecosystem.

Q: Can you tell us about the main achievement and rewarding moment in your research or the episode?

Since I joined Samsung Research in 2020, I have been involved in various research projects. As a Principal AI Specialist, I have had the chance to work with various excellent teams of brilliant researchers and engineers in almost all the fields of AI. I have also contributed to commercialization of the solutions in products.

One of my initial achievements was developing fundamental AI methods for deploying and training AI models on edge and mobile devices. As open-source machine learning libraries traditionally do not support on-device training of AI models (especially deep learning models) for edge devices, we have had the challenge of developing novel computationally efficient algorithms and libraries. To solve this unique problem, I have collaborated with several teams of researchers and software engineers, such as the NPU Lab for Samsung Research, who provided us with the required software library named NNTrainer (https://github.com/nnstreamer/nntrainer).

As I work on these different R&D projects, I have recognized the challenges of employing AI methods and models in Samsung products to solve practical problems from different scientific, engineering, and business perspectives. To improve MX and CX of users with the Samsung ecosystem, I have researched and developed novel modular and distributed AI systems providing personalized solutions to users.

We have been developing on-device AI methods to deploy powerful state-of-the-art AI models on mobile and edge devices under various computational constraints with limited data resources. Our proposed model personalization systems improve deployed models using continuously generated limited labeled and unlabeled user data in lifelong learning settings on the distributed edge and mobile devices, achieving state-of-the-art accuracy in computer vision, automatic speech recognition, and natural language understanding tasks. Our deployed models can learn from users’ experiences with Samsung products and their interaction in distributed systems using federated learning algorithms. Our solutions can be deployed on edge devices and large-scale systems supporting thousands of users. We have published our proposed methods, systems and results in several AI patents, top conferences, and journals such as CVPR, ICML, ICLR, Interspeech, and IEEE Internet of Things Journal.

Q: What is your vision for the future, and what goal would you want to achieve?

We are at the dawn of a new age for AI. In most areas, AI systems are poised to replace predictable and routine tasks supporting users as their assistants in daily life. In the next phase, these systems will reinforce users to upgrade their skills and focus on creative aspects to solve practical tasks and achieve scientific breakthroughs such as designing new AI models and materials. To achieve this goal, we have been developing and employing immersive technologies such as on-device personalized AI on micro-scale and large-scale distributed devices in physical and virtual spaces, i.e., metaverses. These technologies have been motivating the exploration of new AI paradigms, such as artificial general intelligence (AGI) and super intelligence (ASI).

I believe Samsung Research has an outstanding and exceptional advantage in shaping and leading this new AI era. I envision that our solutions and systems will ultimately improve AI-user symbiosis to produce results and achievements that exceed what either can achieve alone in physical, virtual, and mixed reality environments.