TREC OpenSearch relies on real search engines with real users sharing part of their traffic. We call them sites. These sites connect to our API for sites to share queries and documents. Then when a user issues one of the shared query, in real-time, the search engines ask our API for a ranking. We sample a participant’s rankings and return it under 100ms to the search engine. This engine in turn shows it (interleaved with their own ranking) to the user. Interactions of this user are sent back to our API and made available to participants.
We are grateful to have agreements in place with the following sites:
PS: if you run an academic search engine and want to join us, please get in touch!