Blog(2)

• Feature Kernel Distillation

  We study the significance of Feature Learning in Neural Networks (NNs) for Knowledge Distillation (KD), a popular technique to improve an NN model’s generalisation performance using a teacher NN model. We propose a principled framework Feature Kernel Distillation (FKD), which performs distillation directly in the feature space of NNs and is therefore able to transfer knowledge across different datasets.

• Zero-Cost Neural Architecture Search

  Neural architecture search (NAS) is quickly becoming the standard methodology for designing deep neural networks (DNNs). NAS replaces human-designed DNNs by automatically searching for a DNN topology and size that maximize performance.

Focus Areas(0)

Research Achievements(16)

• An Information Fusion Approach to Learning With Instance-Dependent Label Noise

  AuthorLi Li,Rui Chen,Soo-Hyun Choi,Xia Hu

  PublishedInternational Conference on Learning Representation (ICLR)

  Date2022-04-25

• Language model compression with weighted low-rank factorization

  AuthorNatural Language Understanding and On Device AI /SRA

  PublishedInternational Conference on Learning Representation (ICLR)

  Date2022-04-25

• Differentiable Expectation-Maximization for Set Representation Learning

  AuthorMinyoung Kim

  PublishedInternational Conference on Learning Representation (ICLR)

  Date2022-04-25

News(0)

Others(1)

• Samsung AI Center - Moscow

  About Us Papers Play Samsung AI Center – Moscow is the first institution in Russia focused directly on artificial intelligence (AI), where Russia’s strengths and capabilities in fundamental sciences, especially mathematics and physics, are brought together with Samsung global R&D strategy. Samsung AI Center - Moscow is led by Dr. Anton Konushin who is world-known AI Computer Vision expert and associate professor at Moscow State University and Higher School of Econo...