Introduction

Instance-level image retrieval aims to identify exact object or scene matches from large image collections, unlike object-level retrieval which targets broader categories. While deep models have significantly improved retrieval performance, deploying them in practical settings requires balancing accuracy with computational efficiency.

In real-world visual search scenarios, queries must be processed efficiently on-device, while large-scale image collections can be indexed offline. Using a single heavy model for both query and database images is computationally expensive, whereas lightweight models alone often degrade accuracy. This motivates asymmetric retrieval, where database descriptors are computed using a powerful model offline, and queries are processed using a lightweight model.

A key challenge in this setup is aligning the representations produced by the two models. Knowledge distillation (KD) is commonly used to transfer knowledge from teacher to student, but existing approaches primarily focus on aligning individual representations and often overlook the relational structure of the embedding space.

Re-ranking further improves retrieval by incorporating fine-grained matching, but existing methods based on geometric verification or transformer-based models are computationally expensive. In asymmetric settings, re-ranking is additionally challenged by discrepancies between teacher and student representations.

This blog post explains how we tackle the above-mentioned problems using the following novel contributions:

Background and Related Work

Global vs Local Representations

Recent retrieval methods such as CV-Net and AMES leverage both global and local features to achieve high accuracy, but incur significant computational and memory overhead due to complex matching or transformer-based designs. In contrast, Super-Global relies purely on global descriptors, offering efficiency but lacking fine-grained matching capability.

Asymmetric Retrieval and Knowledge Distillation.

To address efficiency constraints, asymmetric retrieval uses lightweight models for queries and stronger models for database images. Prior distillation approaches, including asymmetric metric learning and contrastive methods, focus on aligning individual representations but do not explicitly preserve relational structure in the embedding space.

Re-ranking Strategies.

Re-ranking improves retrieval through local feature matching. Traditional approaches rely on geometric verification methods such as RANSAC and DELG. More recent methods, including CV-Net, R2Former, and AMES, model richer correspondences but remain computationally expensive, limiting their practicality in resource-constrained settings.

Methodology

Our approach ADORE, addresses asymmetric retrieval through two components: knowledge distillation for aligning query and database representations and efficient asymmetric re-ranking.

Figure 1. Illustration of the ADORE framework: (a) asymmetric training with distillation losses, (b) Initial retrieval using global descriptors, and (c) asymmetric re-ranking with local similarity, DBA, and QE.

Aligning Query and Database Representations

We use a large teacher model f(.) to encode database images and train a lightweight student model g(.) to generate query embeddings in the same space. For training, we use tuples consisting of an anchor, a positive sample, and multiple negative samples, along with three loss functions (Refer to Figure 1).

$ L_{abs}=(1-cosine(f(t),g(t)))^2 $

$ L_{rel}=(cosine(f(t),f(t^\prime))-cosine(g(t),g(t^\prime)))^2 $

The final objective is a weighted combination of the above three losses.

Efficient Asymmetric Re-Ranking

After the initial retrieval using global descriptors, we refine the top K results using an asymmetric re-ranking strategy. This includes the following:

Both DBA and QE ensure that the query and candidate database embeddings are robust to noise and variations in viewpoint, scale and illumination.

The final re-ranking score combines local similarity and global similarity (between refined global descriptors).

Experiments and Results

Experimental Setup

Our framework is trained on SfM120k and GLDv2, with evaluation on ROxford5k and RParis6k benchmarks. The teacher model uses pretrained ResNet-101 (R101)-GeM on SfM120k and R101-CVNet on GLDv2, generating 2048-dimensional global descriptors. The student model (ComRes) starts as R101 and is progressively compressed by removing one residual block every two epochs during training.

Results

Figure 2. Qualitative comparison of retrieval results from Super-Global and ADORE reranking

Table 1. Quantitative comparison (mAP values) of asymmetric retrieval with prior methods

Table 2. Comparison of computational efficiency (time in ms) of asymmetric retrieval methods

Conclusion

We presented ADORE, an asymmetric image retrieval framework that combines hybrid knowledge distillation with efficient re-ranking to address the accuracy–efficiency trade-off in instance-level retrieval. By aligning both individual representations and their relational structure, the student model remains compatible with a high-capacity gallery representation.

The proposed re-ranking further improves retrieval quality through efficient local matching and descriptor refinement, without the high cost of prior methods. Experiments show strong performance across both asymmetric and symmetric settings with significantly improved efficiency.

From an application perspective, ADORE is well-suited for image-to-image search in on-device photo collections, where queries must be processed under strict latency constraints. The design enables heavier models to run in the background to build high-quality gallery representations, while lightweight models handle queries efficiently at runtime. This makes ADORE a practical building block for improving retrieval quality in privacy-preserving, real-world settings.