Technologies of Samsung Devices Created in Ukraine: Interview with Samsung R&D Institute Ukraine Representatives

The Ukrainian mobile device market is saturated with various brands, ranging from mass-market to premium class, and constantly updated with new offerings.

An united team of Samsung R&D Institute Ukraine (SRUKR) and Samsung Electronics Ukraine Company (SEUC) initiated a special campaign to promote Samsung technologies and attract more Ukrainian users.

In particular, Researchers from SRUKR sat down with one of the top five popular Ukrainian information technology (IT) media outlets, Gagadget, to showcase locally developed and commercialized features and their development process.

When was Samsung R&D Institute Ukraine established?
What projects are you working on?

Yuriy Beznoschenko
Head of R&D Operation
Samsung R&D Institute Ukraine

Samsung R&D Institute Ukraine was established in 2009, focusing on developing cutting-edge cybersecurity technologies to protect information, artificial intelligence (AI) to improve human interaction with devices, virtual and augmented reality (VR/AR), and new types of multimedia content. With over 500 employees in R&D, we work on several AI-related projects, particularly for computer vision, handwriting recognition, high-quality graphics, and enriched reality and AR. We are also working on innovations in information security and user data privacy. More importantly, we participate in all stages of the technology life cycle, from ideation to market commercialization.

Why does Samsung have its own app development for smartphones?

Samsung develops its own applications and services for Android smartphones for several reasons. First, it aims to differentiate its products and offer unique features and user experiences, such as, for example, enhanced smartphone camera functions like Nightography mode, as well as handwriting input functions and information recognition using the S Pen. After all, while developing its products, Samsung can qualitatively adapt applications with its hardware (HW) components, guaranteeing a more consistent user interaction and accelerated performance. Second, it seeks to have the ability to control the interface and functions so that users can offer constant improvements or alternatives to existing programs and services because, as a chip manufacturer, Samsung develops its solutions ahead of competitors. Last but not least, it is a strategic solution for monetization through program sales, service subscriptions, and advertising. Moreover, in regions where access to the Google App Store is limited, it is worth noting that providing alternative solutions is an excellent commercial move that helps promote Samsung products in this market.

How did AR Emojis appear, and what tasks were assigned to them?

Sergii Lytvynenko
Head of Visual Intelligence Team
Samsung R&D Institute Ukraine

The development of AR Emojis has been quite the journey and has evolved significantly since its introduction to the Galaxy S8. One of the main challenges in the development of AR Emojis is their style and design. Initially, the main task was creating the most personalized and realistic avatars. Here, we encountered a classic problem called the “uncanny valley effect,” a negative perception of humanoid avatars that aim to imitate the user. As such, over time, we pivoted toward the stylized, cartoonish avatars we see today, and this is of huge credit to the design team.

Are consumers using these products? Which application is used most often?

Regarding statistics and usage scenarios, the exact statistics are a trade secret, but we can confidently say that the number of scenarios for the practical use of AR Emojis has increased significantly. Here are some examples.

•   AR Emoji Camera & Editor – This feature allows users to create personalized avatars and style them accordingly. Thanks to
     partnerships with well-known content providers, users can also dress the avatars in branded items.

Personalized Avatar Creation (AR Emoji Camera & Editor)

•   AR Emoji Stickers – This application allows users to create unique animated 2D stickers in GIF, which can be used in any
     messenger app such as Viber, Telegram, or WhatsApp. Recently, we have seen the popularity of AR Emojis in this format.

AR Emoji Stickers Creation on Smartphone

•   Lock Screens, Wallpapers, Watch Faces – Today, AR Emojis are available on Samsung phones, smartwatches, and TVs,
     allowing users to create unique screen savers and wallpapers and personalize main device screens.

Personalization of Devices with AR Emojis

Now, let’s delve into practical applications aimed at addressing privacy and inclusivity concerns. Amid the COVID pandemic and the surge in video communication services, AR Emojis were exclusively available in the Google Duo app in select regions, allowing users to don AR Emoji masks, catering to those uncomfortable with turning on cameras during meetings. We anticipate that avatar usage will increase. In addition, in our commitment to inclusivity, we made avatars available on Samsung TVs last year as Sign Language Avatars, making TV usage more accessible for people with certain hearing impairments.

What consumer needs do AR emojis address, and do these needs exist?

Yes, AR Emojis have not gained as much traction as emoticons, but we believe this is only the beginning, and there will be many changes ahead. We are currently open to close cooperation with developers with unique ideas for using avatars. Today, the AR Emoji SDK is available for developers to use AR Emojis in third-party applications. So, join in!

What was Game Launcher created for? What purposes can it serve for gamers aside from recording a screencast? Is this its only helpful function?

Game Launcher was created primarily to “organize” the gaming space on your phone. As of today, its functions have expanded beyond simply aggregating game applications and launching them, which is why Game Launcher has recently been rebranded and given a new name: Samsung Gaming Hub.

Gaming Hub

Phones have significantly supplanted PC gaming and can easily compete with game consoles through their mobility and the amount of time people spend away from home. So, what can Gaming Hub do?

First, Gaming Hub centralizes all gaming applications in a single space for better phone organization. In addition, Gaming Hub allows users to aggregate all notifications from games, which, on the one hand, makes it possible not to get distracted by games in the general list of notifications or vice versa and, on the other, enables them to focus only on games to avoid missing the next tournaments, for instance. Second, the native integration with Discord allows users to enjoy their favorite games easily and conveniently, enable streams, and simultaneously communicate by text and voice within the Discord community. In addition, there is a growing trend and widespread popularity of “one tap” and “time killer” games, commonly enjoyed during commutes, while waiting in lines, etc. For such cases, Gaming Hub enables gameplay without installation. Its collection of Instant Games (which do not require installation) is conveniently accessible on Gaming Hub’s main screen. Third and last, Gaming Hub makes quality curated game selections by category and a diverse range of plug-ins (Game Plugins) that allow users to customize their gaming environment according to their needs. For example:

•   Game Booster Plus – Increases performance and speed in games with photorealistic graphics or allows for gameplay in
    battery-saving mode
•   Game Clock – Reminds users to return to their favorite games at the right time
•   Gif Creator – Allows users to create animated pictures directly from games
•   Priority Mode – Allows users not to be distracted by notifications or calls during a fierce tournament

Game Plugins in Gaming Hub

How does ray tracing affect smartphone power consumption? What are the minimum processor requirements, and when will we see the technology on midrange smartphones?

The rapid evolution of mobile processors and video cards opens new opportunities for the gaming industry on mobile phones. One of the clear examples is ray tracing (RT) technology, which was first introduced for the Android platform in Samsung Galaxy S23. RT technology brings a new level of computer graphics—natural lighting, light reflections, and shadows—thus mimicking physical processes as they happen in the real world. It should be noted that RT technology requires massive computing resources, so its adaptation for mobile devices was difficult. All this became possible through the fruitful and painstaking work of hardwawre(HW) and software (SW) developers of mobile chips, close cooperation with key players in the game industry, such as Epic Games and Activision Blizzard, and collaboration with global game studios.

A Visual Example of Ray Tracing (RT) Technology

RT technology in Galaxy phones appeared for a reason. Over the past decade, Samsung has invested much effort and resources in developing the gaming ecosystem and optimizing computer graphics for Galaxy products. In 2016, Samsung was the first to implement the new open graphics standard Vulkan application programming interface (API). During this time, without exaggeration, we gained invaluable experience and formed one of the top computer graphics teams in the world. Therefore, when the window of opportunity for HW and SW innovation appeared, we were ready to implement RT. Undoubtedly, this became possible thanks to close and fruitful cooperation with global players in the mobile chipset market, manufacturers of graphics engines.

Regarding energy consumption, RT requires additional costs and calculations, but in return, the consumer receives a product of a completely different quality. On the other hand, with the right approach in its use, we have the opportunity to find a rational balance between quality and consumption, so we are actively exploring hybrid approaches in which RT is applied only to certain parts of 3D content that have the greatest visual effects for the end user. As for the spread of the technology to midrange smartphones, it is only a matter of time. For objective (contractual) reasons, we cannot tell you the specific names of the companies and games we are in partnership with, but you will see everything for yourself very soon. There’s not much time left to wait!

Doesn’t the closure of the Galaxy Note line mean that the pen has become unnecessary for mass users today?

Olga Radyvonenko
Head of Creative Input Intelligence Lab
Samsung R&D Institute Ukraine

The changes in the model range aimed to reinforce the status of the premium technology, driven by advanced AI solutions ensuring swift and natural handwritten input, text recognition, and other content, such as diagrams, geometric figures, etc. The technology we created for the S Pen has long been known to Samsung users worldwide. It supports text handwriting recognition in more than 100 languages and provides natural input that is as close as possible to drawing on paper. Meanwhile, Samsung Notes, a note-taking app our team is also involved in, has over a billion installs on Google Play Store. For tablets, the Tab S series with the S Pen is the most sought-after Samsung model range in its segment.

Text Input Using S Pen

Handwriting recognition technologies have become really powerful, but has anyone seen a person regularly write something on a smartphone screen?

In terms of interface, the keyboard takes up a lot of space on the screen and is primarily suited for short messages. The pen interface, in contrast, frees up almost half of the screen, facilitating language switching without unnecessary clicks and enabling illustrations/diagrams. This ensures more freedom in writing without the need to adapt to keyboard limitations. Moreover, applications consider the screen size and feature aids that make typing on smaller screens comfortable (the same easy writing pad that can move the enlarged input window by itself). Very often, not everything can be entered from the keyboard, for example, illustrations, sketches, notes on top of photos and pictures, graphics, and mathematical formulas (especially if you need to write them down quickly). Don’t forget about niche things, such as the use of tablets by artists, where pen input is generally indispensable, and having the ability to use a pen not only for tablets but also for smartphones allows users to take the device with them and draw anywhere.

What challenges do smartwatch software developers face?

Illia Fedorin
Senior Engineer, Project Leader
Samsung R&D Institute Ukraine

In a world where every step can be turned into statistics and every heartbeat counts, smartwatches have become the new doctors on call. The wrist is a place for a watch and a platform for health and activity. When we leave the house, we take little helpers who do not just count the time but help us move in the right direction to better health. However, considering these limitless possibilities, it’s crucial to remember that technology is not a panacea. It will not replace a professional medical diagnosis but can be useful in practicing a more conscious attitude to one’s health and well-being. In the process of creating an ideal wearable device capable of monitoring health indicators and predicting pathological conditions, developers faced many technical and ethical difficulties. Solving these challenges required innovation in data encryption, improved sensor technology, and finding a balance between personalization and privacy.

•   Privacy protection:This is paramount in biometric wearable devices. In a world where watches and wristbands can track
     everything from heart rate to sleep quality, there is a significant risk of personal health data being leaked. As such,
     developers face the challenge of ensuring that sensitive information remains secure. Encryption and authentication systems
     are akin to walls of a safe that safeguard your health from prying eyes.
•   Measurement accuracy:It is the key to personal health. To accurately monitor each individual’s health, sensors in wearable
     devices must consider several factors, from skin color to physical activity. Intelligent algorithms and machine learning
     analyze and adapt these changing data to provide accurate health insights.
•   Standardization and regulation: This ensures uniformity for reliability. For wearable devices to be used internationally, they
     must meet strict standards and be approved by regulatory bodies such as the Food
     and Drug Administration (FDA (U.S. Food and Drug Administration) and the European Quality Certificate (CE). This ensures
     that the devices that measure health conditions are safe and work correctly when used.

How do algorithms for determining pulse and blood oxygen levels work?

Embedded photoplethysmographic sensors use light-emitting diodes (LEDs) to project light through the skin and into blood vessels. This lets users record blood flow and changes with each heartbeat. Photoplethysmography (PPG) measures volumetric changes in blood vessels using light reflected or absorbed by the bloodstream. As light passes through the skin and blood vessels, the amount of light absorbed and reflected varies depending on the amount of blood in the vessels. These variable indicators of the light signal are then read by a photodetector and analyzed by algorithms to calculate the heart rate. Photoplethysmographic sensors usually use green LEDs that emit photons at frequencies optimal for penetrating the skin barrier and interacting with the bloodstream. The algorithms in these devices are carefully calibrated to remove noise and interpret data into accurate heart rate metrics.

Measurement of Heart Rate Using a Photoplethysmography (PPG) Sensor

To analyze blood oxygen levels (SpO2), the watch uses pulse oximetry sensors that emit red and infrared light. These wavelengths are not chosen by chance; they have different absorption levels (absorption) depending on the degree of oxygen saturation in the blood. Using the difference in light absorption of the two types of hemoglobin, modern watches accurately calculate the percentage of oxygen in the blood, making for a portable device.

Text Input Using S Pen

ECG Sensor (Left) and Optical Heart Rate Sensor (Right)

While smartwatches cannot replace professional medical equipment, their contribution to health monitoring and fitness training is undeniable. These engineering developments exemplify the philosophy of continuous improvement and technology integration into everyday life, providing a more detailed insight into our body’s state and allowing us to feel more in control of our health.

What are the future prospects of new sensors in watches?

Next-generation sensors are poised to revolutionize personal health and well-being through the continuous monitoring and analysis of biometric data. They pave the way for personalized medicine, preventive diagnostics, and early detection of health problems. Biosensors of the new generation are a revolution in personal health. Scientists are developing the latest biosensors that can surpass today’s methods of measuring medical indicators. Imagine sensors that can analyze glucose levels without painful injections or monitor hydration and stress through sweat. This opens the door to a new era of preventive medicine and personal health. On the other hand, personalized medicine explores a future tailored to you. Future smartwatches will be much more intelligent, and they will be able to monitor the current health status and predict pathological conditions, allowing them to prevent diseases. This is a real breakthrough in personal medicine. Integration with healthcare is a huge help to doctors. The new generation of wearable devices will be connected directly to medical systems, providing doctors with important real-time patient condition data. This opens up opportunities for more accurate and effective treatment. We also foresee intelligent adaptation through the use of AI in wearable technology. AI will become the foundation for the development of wearable technology, learning from a vast amount of data and adjusting to the individual needs of users. This means your watch will be able to predict and respond to health based on your condition.

Undoubtedly, wearable technology holds the key to transforming how we approach health and fitness. Modern gadgets we wear on our wrists are exceptional in their capabilities, and this is only the beginning. Let’s imagine smartwatches that don’t just monitor our heart rates or the number of steps taken but perform complex analyses of our physical condition, warn of changes in blood sugar levels, or analyze our genetic code to provide recommendations on improving our health. Each new day brings news about breakthroughs in measurement accuracy, batteries that can last longer without recharging, and programs that learn to understand our health and habits better. This is not fiction. This scientific and technological progress is already happening. Modern technologies open up exciting opportunities for us. Can we foresee that one day, our watch could recommend the amount of water to drink and which nutritional supplements are best to consume based on our activities and current state of health, or even connect directly with the doctor, transmitting important medical data in real time? These are no longer just possibilities but real prospects that developers and scientists worldwide are working on. Seeing these innovations, we can’t help but get excited about how they will change our lives. With each step forward in this area, we make having a healthy lifestyle more accessible and enjoyable for everyone wearing these small but brilliant devices.

What was the contribution of SRUKR to the evolution of smart health gadgets?

The division’s work spans scientific articles, projects, patents, and commercially available solutions that contribute to better health. Various innovations are paving the way for a healthier future, from smartwatches that monitor heart rates during intense exercise to systems that help people understand and manage their sleep.

The unit’s technological progress in this area is based on using neural networks for real-time heart rate monitoring and developing algorithms that classify physiological parameters through PPG and accelerometry. A significant innovation is smart alarm clocks based on analyzing and predicting sleep phases and systems for assessing respiratory functions and heart rates, integrated into modern smartwatches.

The division’s commercial solutions include algorithms for classifying sleep phases and monitoring heart rate during intense physical activity implemented in consumer smart devices. Such innovations allow users not only to receive valuable information about their own physical condition but also to manage training and rest effectively.

The division’s research also covers the application of radar signals integrated into the latest models of premium smartphones to monitor sleep, heart rate, and breathing, indicating the great potential of radar technology to complement optical methods. However, the main emphasis remains on expanding functionality and popularizing smartwatches as indispensable assistants in maintaining a healthy lifestyle.

Furthermore, the patented portfolio includes innovations in continuously monitoring physiological indicators and methods using radar systems. These achievements highlight the company’s significant role in developing technologies for health and wellness.

These developments significantly impact the technology community and end users, who now have access to advanced self-monitoring tools and personalized information about their health.

What is Samsung doing to protect consumers’ confidential information?

Stanislav Pedan
Senior Engineer of Privacy Technology
Samsung R&D Institute Ukraine

Ensuring the security of the data of our devices’ users is a priority in our developments. A prime example is the Samsung Knox platform. However, even the most robust security systems can be circumvented if the phone’s owner voluntarily or unintentionally sends private data outside the mobile device. To address this, we have developed a technology that analyzes the image when you try to send it and warns of possible consequences if it contains sensitive data, such as details of credit cards, driver’s licenses, passports, etc. This technology is distributed as part of the Samsung’s Smart Suggestions services, available starting with One UI 5.0.What sets our solution apart is that its system can detect private images locally (on the phone itself) without accessing real private data. For end users, this means that it can immediately notify the potential disclosure of information and, at the same time, will not share or store this data.