Zum Inhalt springen
NEUMarktplatz und dein Team: Verkaufe, was du baust, und bekomm Pläne und Änderungen, die getestet sind, bevor du sie siehst. Neuigkeiten ansehen

Bibliothek / RAG & Wissensdatenbanken

RAG & Wissensdatenbanken

Hilfe-Center-Seiten per manuellem Start mit OpenAI-Embeddings in Pinecone laden

Ruft manuell angegebene Hilfe-URLs ab, teilt den Text in Abschnitte mit stabilen IDs, erstellt Embeddings mit OpenAI und schreibt sie per Upsert nach Pinecone. Beim erneuten Laden werden Einträge überschrieben statt dupliziert.

Was sie bewirkt

## Load help pages into a Pinecone knowledge base with OpenAI embeddings Who's it for Teams running a retrieval-based support or chat workflow who need their help pages in Pinecone, reloadable whenever the content changes. How it works Each public URL in DOC_URLS is fetched on its own and stripped to plain text; a page that fails or comes back empty is listed as skipped. The text is split into overlapping chunks with stable ids, so reloading a page overwrites its chunks rather than duplicating them. Chunks are embedded with OpenAI in batches of 96 and upserted to Pinecone with the text, title and source as metadata. A batch that fails is reported and the rest still load. How to set up 1. Run Run it now first: the sample articles are chunked and shown, nothing is embedded. 2. Create a Pinecone index with 1536 dimensions and cosine metric, and put its host in PINECONE_INDEX_HOST. 3. List your pages in DOC_URLS, attach OpenAI and a Pinecone Header Auth credential (Api-Key), set USE_OPENAI = true. 4. Set TEST_EMAIL or TEST_RUN = false: loading is skipped in preview. Requirements An OpenAI API key, a Pinecone index, and help pages reachable at public URLs. How to customize CHUNK_WORDS and CHUNK_OVERLAP set the chunk size, OPENAI_EMBED_MODEL and EMBED_DIMENSIONS the embedding, and PINECONE_NAMESPACE where the vectors go.

Die Schritte

  1. Run it now
  2. config
  3. document list configured?
  4. list the documents
  5. Fetch the document
  6. read the document text
  7. use the demo knowledge base
  8. chunk the documents
  9. loading on?
  10. STOP: preview - vectors not upserted
  11. build the embedding batches
  12. [cred] OpenAI - embed the chunks
  13. build the Pinecone upsert
  14. batch embedded?

Insgesamt 18 Schritte.

Ähnliche Automatisierungen