Skip to content
NEWError doctor and Watchtower: failed n8n runs explained in plain words, and emailed to you. See what is new

Library / RAG & Knowledge Bases

RAG & Knowledge Bases

Evaluate RAG Document Relevance with OpenAI and Google Sheets

When a dataset row is fetched or a chat message arrives, a RAG agent answers and OpenAI scores the relevance of the retrieved documents.

Import it free Sign in to open it

What it does

## How it works This template shows how to calculate a workflow evaluation metric: retrieved document relevance (i.e. whether the information retrieved from the vector store is relevant to the question). The workflow takes a question and checks whether the information retrieved to answer it is relevant. You can find more information on workflow evaluation here, and other metric examples here.

The steps

  1. When fetching a dataset row
  2. Evaluating?
  3. When chat message received
  4. Match chat format
  5. Return chat response
  6. Set metrics
  7. Get dataset
  8. Remove Duplicates
  9. Simple Vector Store
  10. Embeddings OpenAI
  11. Default Data Loader
  12. Recursive Character Text Splitter
  13. When clicking ‘Execute workflow’
  14. Calculate doc relevance metric

19 steps in total.

Similar automations

RAG & Knowledge BasesEvaluate RAG Groundedness with OpenAI21 stepsWeb Scraping & ResearchFind Quality YouTube Videos with Relevance Scoring into Sheets16 stepsAI Agents & AssistantsEvaluate AI Agent Relevance with OpenAI and Cosine Similarity17 stepsRAG & Knowledge BasesDocument Chatbot with RAG, OpenAI, Supabase and Cohere Reranker13 steps