Blog(2)

• [CVPR 2023 Series #5] MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation

  Video Object Segmentation (VOS) is an important problem in computer vision and it has a lot of interesting applications like video editing, surveillance, autonomous driving, and augmented reality.

• Feature Kernel Distillation

  We study the significance of Feature Learning in Neural Networks (NNs) for Knowledge Distillation (KD), a popular technique to improve an NN model’s generalisation performance using a teacher NN model. We propose a principled framework Feature Kernel Distillation (FKD), which performs distillation directly in the feature space of NNs and is therefore able to transfer knowledge across different datasets.

Research Areas(0)

Publications(21)

• You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation

  AuthorMehdi Noroozi, Isma Hadji, Brais Martinez, Adrian Bulat, Georgios Tzimiropoulos

  PublishedEuropean Conference on Computer Vision (ECCV)

  Date2024-09-30

• Exploring compressibility of transformer based text-to-music (TTM) models

  AuthorPablo Peso Parada, Md Asif Jalal, Jisi Zhang, Karthikeyan Saravanan

  PublishedAnnual Conference of the International Speech Communication Association (INTERSPEECH)

  Date2024-09-01

• Recurrent Early Exits for Federated Learning with Heterogeneous Clients

  AuthorRoyson Lee, Shell Xu Hu, Da Li, Timothy Hospedales

  PublishedInternational Conference on Machine Learning (ICML)

  Date2024-07-23

View More

News(3)

• [Blog] Pretrained Bidirectional Distillation for Machine Translation

  Initializing parameters by a pretrained masked language model (LM) [1] is a knowledge transfer method widely applied to natural language processing tasks. Following its success, pretrained neural machine translation (NMT) models have attracted more and more research interest [2,3,4,5].

• [Blog] [CVPR 2023 Series #5] MobileVOS: Real-Time Video Object Segmentation Contrastive Learning Meets Knowledge Distillation

  Video Object Segmentation (VOS) is an important problem in computer vision and it has a lot of interesting applications like video editing, surveillance, autonomous driving, and augmented reality. Basically, VOS is when you try to find and follow objects across multiple frames in a video.

• [Blog] Feature Kernel Distillation

  We study the significance of Feature Learning in Neural Networks (NNs) for Knowledge Distillation (KD), a popular technique to improve an NN model’s generalisation performance using a teacher NN model. We propose a principled framework Feature Kernel Distillation (FKD), which performs distillation directly in the feature space of NNs and is therefore able to transfer knowledge across different datasets.

Others(0)