Перейти к содержанию
НОВОЕМаркетплейс и ваша команда: продавайте то, что создали, и получайте планы и изменения, проверенные до того, как вы их увидите. Что нового

Библиотека / RAG и базы знаний

RAG и базы знаний

Загрузка страниц справочного центра в Pinecone с эмбеддингами OpenAI при ручном запуске

Вручную получает список справочных URL, разбивает текст на фрагменты со стабильными id, создаёт для них эмбеддинги через OpenAI и записывает в Pinecone с перезаписью, чтобы повторные загрузки не создавали дубликаты.

Что делает

## Load help pages into a Pinecone knowledge base with OpenAI embeddings Who's it for Teams running a retrieval-based support or chat workflow who need their help pages in Pinecone, reloadable whenever the content changes. How it works Each public URL in DOC_URLS is fetched on its own and stripped to plain text; a page that fails or comes back empty is listed as skipped. The text is split into overlapping chunks with stable ids, so reloading a page overwrites its chunks rather than duplicating them. Chunks are embedded with OpenAI in batches of 96 and upserted to Pinecone with the text, title and source as metadata. A batch that fails is reported and the rest still load. How to set up 1. Run Run it now first: the sample articles are chunked and shown, nothing is embedded. 2. Create a Pinecone index with 1536 dimensions and cosine metric, and put its host in PINECONE_INDEX_HOST. 3. List your pages in DOC_URLS, attach OpenAI and a Pinecone Header Auth credential (Api-Key), set USE_OPENAI = true. 4. Set TEST_EMAIL or TEST_RUN = false: loading is skipped in preview. Requirements An OpenAI API key, a Pinecone index, and help pages reachable at public URLs. How to customize CHUNK_WORDS and CHUNK_OVERLAP set the chunk size, OPENAI_EMBED_MODEL and EMBED_DIMENSIONS the embedding, and PINECONE_NAMESPACE where the vectors go.

Шаги

  1. Run it now
  2. config
  3. document list configured?
  4. list the documents
  5. Fetch the document
  6. read the document text
  7. use the demo knowledge base
  8. chunk the documents
  9. loading on?
  10. STOP: preview - vectors not upserted
  11. build the embedding batches
  12. [cred] OpenAI - embed the chunks
  13. build the Pinecone upsert
  14. batch embedded?

Всего шагов: 18.

Похожие автоматизации