Boomerang V2: The next generation of multimodal enterprise search

In Moving beyond text conversion: The future of enterprise search, we describe how Vectara’s unified multimodal embedding space enables the retrieval system to search across different data types without requiring the user to choose a modality in advance. Enterprise knowledge lives in charts, diagrams, screenshots, scanned pages, tables, and other visual records. Converting these artifacts into text for your retrieval system works, but it can omit details useful for later queries.

We are closing this gap with a retrieval method that operates on the artifact itself rather than on its description, with Boomerang V2, the successor to Boomerang and our first multimodal embedding model. It maps text, images, and interleaved text-and-image inputs into a shared 1024-dimensional vector space. The model supports an 8,192-token context window, 16 times longer than V1, and produces a single-vector representation that fits naturally into a production vector-search stack. It is also trained with Matryoshka representation learning, allowing the vector to be truncated to 768 or 512 dimensions when storage or search speed is important.

Evaluation and performance benchmarks

Embedding models are evaluated by how well they rank useful data by similarity and find the relevant information in response to queries. We have shared the normalized Discounted Cumulative Gain (nDCG) scores below. It’s a metric that measures how well a system ranks items by relevance. It awards a perfect score of 1.0 to a system that arranges data in a descending order of relevance. In most cases, an nDCG score above 0.80 is considered excellent, indicating that the system places highly relevant options directly in front of the customers.

Text retrieval

Adding images to an embedding model usually lowers text quality, which is the main reason the visual side was kept in a separate model. To address this, we prioritized the quality of text retrieval for Boomerang V2.

We test it on two standard sets. BEIR, the most widely used English language benchmark, includes 14 retrieval datasets covering everything from question answering and fact checking to scientific claims and argument retrieval. MIRACL covers multilingual retrieval in 9 languages: Arabic, Bengali, Chinese, Hindi, Japanese, Korean, Swahili, Telugu, and Thai. In each case, the query and the documents are in the same non-English language.

Boomerang V2 improves on both: BEIR goes up from 45.5 to 47.0, and MIRACL from 48.1 to 53.9. The bigger gain is in multilingual retrieval, which is important for customers whose corpora are not English-only.

Image retrieval

V1 does not have image retrieval. Boomerang V2 adds it. You can enter a question or keywords, and the system will search for and find the right image. It can be a page, a chart, or a photo that best answers your query. Anything in the layout, the chart, or the diagram can be searched directly.

In testing, MMEB-V2 separates images, video, and visual documents into different groups. We report the two that matter for retrieval here. The image group works in every direction: some tasks search for images with a text query, some search for text using an image, and some use an image and a short text together as one query. The visual document group is text-to-page, where the answer might be in a chart, table, slide, or a scanned layout instead of a text paragraph.

Boomerang V2 scores 70.9 on visual documents and 64.0 on images. For reference, the best published results on these tests are 79.2 and 74.8. Now there is no comparison with V1 here. With V1, you would have to add captions to every page first, then search for those captions so the page can only be found if someone wrote the right words for it.

Internal semiconductor benchmark

We wanted to understand how retrieval models work in one of the most technical search settings: semiconductor engineering. Semiconductor documents include specialized terms, equations, plots, circuit diagrams, and device schematics. The information that answers a query might be in a text passage, a figure, or a combination of both. But public retrieval benchmarks rarely cover this kind of technical and visual detail.

So we built a benchmark specifically for this field, covering four areas of semiconductor engineering. Using textbook PDFs not in our training data, we evaluate two retrieval tasks: finding the text passage that answers a query, and finding the figure that answers it.

Queries are written in seven styles because end users search in different ways. Some ask full questions: How does subthreshold swing relate to gate-oxide thickness? Some search for a technical phrase or a few keywords, and some search by symptom: Id-Vd compression at high Vds.

Finding the right text passage improves from 52.0 to 61.5, a bigger gain than V2 posts on either public text benchmark. For figures, it scores 75.8 with a description and 61.2 with just the image.

We ran the same benchmark against four other embedding models, open-weight and proprietary, each using its own documented retrieval settings.

Boomerang V2 leads in the setting that best reflects how enterprise documents are viewed: images with their surrounding descriptions. It scores 75.8 nDCG@5, ahead of Jina v5 omni small (73.7) and Gemini Embedding 2 (66.2). On text-only retrieval, Jina leads by a narrow margin, while Gemini does best when given raw images only. But Boomerang V2 remains competitive across all three cases, avoiding the large trade-offs that the other models make.

Using Boomerang V2

Boomerang V2 is available now. It emits a single 1024-dimensional vector, so it runs on the same index and query path you already use, with no change to your retrieval infrastructure. If storage or latency is more important than the last fraction of a point, reducing the vector to 768 dimensions saves a quarter of the index size and lowers the score by 0.6 points on any benchmark we measured.

Ready to upgrade your enterprise search? Explore Vectara models today to get started with Boomerang V2.