Kütüphane / RAG ve Bilgi Bankaları
RAG ve Bilgi Bankaları
Manuel çalıştırmada yardım merkezi sayfalarını OpenAI gömmeleriyle Pinecone'a yükler
Listelenen yardım URL'lerini manuel olarak getirir, metni sabit kimlikli parçalara böler, OpenAI ile gömer ve Pinecone'a upsert eder; böylece yeniden yüklemeler kopya oluşturmak yerine üzerine yazar.
Ne yapar
## Load help pages into a Pinecone knowledge base with OpenAI embeddings Who's it for Teams running a retrieval-based support or chat workflow who need their help pages in Pinecone, reloadable whenever the content changes. How it works Each public URL in DOC_URLS is fetched on its own and stripped to plain text; a page that fails or comes back empty is listed as skipped. The text is split into overlapping chunks with stable ids, so reloading a page overwrites its chunks rather than duplicating them. Chunks are embedded with OpenAI in batches of 96 and upserted to Pinecone with the text, title and source as metadata. A batch that fails is reported and the rest still load. How to set up 1. Run Run it now first: the sample articles are chunked and shown, nothing is embedded. 2. Create a Pinecone index with 1536 dimensions and cosine metric, and put its host in PINECONE_INDEX_HOST. 3. List your pages in DOC_URLS, attach OpenAI and a Pinecone Header Auth credential (Api-Key), set USE_OPENAI = true. 4. Set TEST_EMAIL or TEST_RUN = false: loading is skipped in preview. Requirements An OpenAI API key, a Pinecone index, and help pages reachable at public URLs. How to customize CHUNK_WORDS and CHUNK_OVERLAP set the chunk size, OPENAI_EMBED_MODEL and EMBED_DIMENSIONS the embedding, and PINECONE_NAMESPACE where the vectors go.
Adımlar
- Run it now
- config
- document list configured?
- list the documents
- Fetch the document
- read the document text
- use the demo knowledge base
- chunk the documents
- loading on?
- STOP: preview - vectors not upserted
- build the embedding batches
- [cred] OpenAI - embed the chunks
- build the Pinecone upsert
- batch embedded?
Toplam 18 adım.
Benzer otomasyonlar
RAG ve Bilgi BankalarıOpenAI ve Pinecone RAG ile Belgelerden Soruları Yanıtlama24 adımRAG ve Bilgi BankalarıGoogle Drive belgelerini OpenAI ile Pinecone'a yükleme7 adımRAG ve Bilgi BankalarıPinecone ile GitHub API Spesifikasyonuyla Sohbet Et14 adımRAG ve Bilgi BankalarıTur Güzergahı E-postalarını OpenAI ve Pinecone ile Yanıtlama12 adım