Evaluating Elicit

Edited

We encourage you to evaluate how much Elicit helps with your use-case. You could evaluate:

  1. How much time does Elicit save you in screening studies or extracting data?

  2. How accurate is Elicit at extracting data from your studies?

  3. What is Elicit's precision and recall for screening your studies?

You can either use a random subset of data from an ongoing project, or you can use Elicit to replicate part of a systematic review that you've already finished.

Either way, we strongly recommend that when Elicit disagrees with human reviewers, you check who is right rather than assuming that the human review is right -- in our testing, Elicit is often more accurate than humans.

Our support for evaluations

We're generally happy to help with your evaluations of Elicit. In particular, if you're working on an evaluation where you will compare Elicit's performance to human research assistants or to another AI product and then publish the results, we can provide free use of Elicit and guidance about how to get good results. Please reach out at help@elicit.com!