Build a PDF-Powered Chatbot with React Ink and Pinecone

Imagine having a PDF manual and receiving instant, human-like answers through a chat interface. In this tutorial, I'll show you how to build a PDF-powered chatbot that runs directly in your terminal. We'll use Ink, which brings React's familiar component-based approach to the command line, and Pinecone for lightning-fast vector search to power the knowledge retrieval.

Here is our overall code flow:

Prerequisites

1. Bun

2. An OpenAI API key (or compatible endpoint).

3. A Pinecone account + API key, and a created index in Pinecone database

Implementation

Project Structure

markdown
Copy code
src/
  index.tsx  ------------ Entry point – renders Ink app
  components/
    ChatApp.tsx --------- Main chat UI
    MarkdownMessage.tsx - Helper to render Markdown
  services/
    chatService.ts ------ OpenAI API wrapper
    pineconeService.ts -- Pinecone API wrapper
  utils/
    textUtils.ts -------- Simple text utils functions

Step 1: Initial Pinecone Client

In pineconeService.ts :

typescript
Copy code
import { Pinecone } from "@pinecone-database/pinecone";

class PineconeService {
  private client = new Pinecone({
    apiKey: process.env.PINECONE_API_KEY!,
  });
  private index = this.client.index(process.env.PINECONE_INDEX!);

  // methods go here...
}

export default new PineconeService();

We set up the Pinecone client with an API key and choose an index to store our vectors.

Step 2: Initial Chat Service

typescript
Copy code
class ChatService {
  private openai = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
  });
  constructor() {}
  async getChatCompletion(messages: Message[]): Promise<string> {
   // openai.chat.completions
  }

  async getEmbeddings(input: string) {
   // openai.embeddings
  }
}

const chatService = new ChatService();

The chat service provides chat completions function and embedding function.

Step3: Build Termial Chat UI

The ChatApp serves as our straightforward chat interface. It takes pdfPath as a prop when launched, like this:

bash
Copy code
bun start ./path/to/document.pdf

Core ChatApp structure:

typescript
Copy code
const ChatApp: React.FC<{ pdfPath: string }> = ({ pdfPath }) => {
  const [messages, setMessages] = useState<ChatMessage[]>([]);
  const [input, setInput] = useState("");
  const [thinking, setThinking] = useState(false);
  const [fileKey, setFileKey] = useState<string | null>(null);
  const [chatError, setChatError] = useState<string | null>(null);
  const [progress, setProgress] = useState<
    EmbeddingProgress | null | "startChat"
  >(null);
  const handleChatSubmit = async () => {
    // chat function
  }
  const handleFileSubmit = async () => {
    // handle pdf embedding 
  }
  useEffect(() => {
    if (pdfPath) {
      handleFileSubmit(pdfPath);
    }
  }, [pdfPath]);
  return (
    <Box flexDirection="column">
      {/* Welcome title */}
      {/* PDF embedding and uploading progress messages */}
      {/* Chat messages */}
      {/* Chat input and thinking states */}
   </Box>
  )
}

Step4: Split and Prepare PDF Chunks

In `pineconeService.ts`, function `embedding` hold the whole logic.

typescript
Copy code
class PineconeService { 
  ...
  async embedding() {
    // 1. check if file exists
    this.checkFile
    // 2. split and segment the pdf 
    this.getDocumentsFromPdf
    // 3. vectorise and embed individual documents with limited rate
    this.rateLimitedEmbedDocument
    // 4. upload to pinecone 
    this.uploadToPinecone
  }
  ...
}

The core of the getDocumentsFromPdf function lies in the prepareDocument logic, which processes each PDF page. Here's how it works:

typescript
Copy code
import { Document, RecursiveCharacterTextSplitter } from "@pinecone-database/doc-splitter";

class PineconeService {
  // Other methods...

  private async getDocumentsFromPdf(filePath: string) {
    const loader = new PDFLoader(filePath);
    const pages = (await loader.load()) as PDFPage[];
    const documents = await Promise.all(pages.map(this.prepareDocument));
    return documents;
  }

  private prepareDocument = async (page: PDFPage) => {
    let { pageContent, metadata } = page;
    pageContent = pageContent.replace(/\n/g, "");

    // Split the document into chunks
    const splitter = new RecursiveCharacterTextSplitter({
      chunkSize: 300,
      chunkOverlap: 20,
    });

    const docs = await splitter.splitDocuments([
      new Document({
        pageContent,
        metadata: {
          pageNumber: metadata.loc.pageNumber,
          text: this.truncateStringByBytes(pageContent, 36000),
        },
      }),
    ]);

    return docs;
  };

  // Other methods...
}

In this code, prepareDocument removes line breaks and chops each page into 300-char chunks. This ensures the PDF content is ready for further processing or analysis.

Step 5: Embed and upload in Batches

Still in `pineconeService.ts`:

ts
Copy code
class PineconeService{
  ...

  private async rateLimitedEmbedDocument(
    documents: Document[][],
    getEmbeddings: (text: string) => Promise<number[]>
  ) {
    const limiter = new Bottleneck({
      minTime: 50,
    });
    const rateLimitedGetEmbedding = limiter.wrap(async (str: string) => {
      const embeddings = await getEmbeddings(str);
      return embeddings;
    });
    const vectors = (
      await Promise.all(
        documents
          .flat()
          .map((doc) => this.embedDocument(doc, rateLimitedGetEmbedding))
      )
    ).filter((record) => !!record) as PineconeRecord[];
    return vectors;
  }

  
  private async uploadToPinecone(filename: string, vectors: PineconeRecord[]) {
    const fileKey = convertToAscii(filename);
    const namespace = this.index.namespace(fileKey);

    const chunks = this.sliceIntoChunks(vectors, 10);

    await Promise.all(chunks.map(async (chunk) => namespace.upsert(chunk)));
    return fileKey;
  }

  ... 
}

We throttle embeddings with Bottleneck, hash each chunk with MD5, and upsert in groups of 10.

Step 5: Querying Your PDF Chatbot

In ChatApp.tsx, we query the PDF content that matches the user's input. Here's how it works:

typescript
Copy code
const handleChatSubmit = async () => {
  ...
  setMessages(() => [{ role: "user", content: input }]);
  const embeddings = await chatService.getEmbeddings(input);
  const matches = await pineconeService.getMatchesFromEmbeddings(embeddings, fileKey);
  
  // Filter content with a match score above 0.4
  const qualifyingDocs = matches.filter((match) => match.score && match.score > 0.4);
  
  if (!qualifyingDocs.length) {
    setMessages((prev: ChatMessage[]) => [
      ...prev,
      {
        role: "assistant",
        content: "I'm sorry, but I can't answer your question. No relevant information was found. Could you clarify or provide more context?",
      },
    ]);
    return;
  }
  
  const assistantResp = await chatService.getChatCompletion([
    {
      role: "system",
      content: await PromptForPDFResult.format({
        matches: JSON.stringify(qualifyingDocs),
      }),
    },
    { role: "user", content: input },
  ]);
  
  setMessages((prev: ChatMessage[]) => [
    ...prev,
    { role: "assistant", content: assistantResp },
  ]);
  ...
}

This code handles the chat submission, filters relevant PDF content, and generates a response based on the user's input. If no relevant matches are found, it prompts the user for more details.

Final Result

I built a chatbot using a Next.js 14 documentation PDF, and here’s how it turned out:

Here’s the full code demo: https://github.com/mbaxszy7/pdf-chatbot-ink.

Next, I’ll build a PDF chatbot using the Next.js15 App Router and a sleek chat interface!