PDF attribution cookbook

A PDF citation should land on the sentence that supports the answer, not just on the right page. TokenPath attributes text to text. A PDF extractor supplies the other half: a page number and bounding box for every character of that text. When TokenPath returns a source range, the same range picks out the boxes to draw on the page. The complete runnable example lives in the public TokenPath Cookbook.

An answer about Adobe Q2 results beside the source PDF, with $6.62 billion highlighted in both. — The $6.62 billion citation opens its exact supporting text on page 3.

How the mapping works

Keep two values index-aligned while you extract the PDF:

document: the text or Markdown your model reads.
characters: for each character in document, its page and bounding box, or None for characters that are not on the page.

TokenPath’s source range indexes into document, so characters[start:end] gives the boxes to highlight. The page tells your viewer where to scroll, and the boxes tell it what to draw.

1. Extract the PDF with coordinates

The cookbook uses Adobe PDF Extract with getCharBounds: true. Its pdfmd converter builds Markdown and the character map together:

from pdfmd import convert

pdf = convert("report.pdf", out_dir="output/report")

document = pdf.markdown
characters = pdf.chars

assert len(characters) == len(document)

characters[i] is the location of document[i], or None for Markdown syntax and whitespace that the converter inserted. Each location has a zero-based page index and a bounding box in PDF points.

Adobe’s PDF-to-Markdown operation does not return character coordinates. Use PDF Extract with getCharBounds: true, then build the Markdown and its character map in the same pass. The cookbook includes the complete converter.

Use this exact document for both generation and attribution. If you clean up or rewrite the extracted text, update characters in the same step.

2. Generate an answer

Give document to any model as its source. Keep the exact answer string you display.

question = "How did Adobe perform in Q2 FY2026?"
answer = (
    "Adobe reported $6.62 billion in Q2 revenue. Total Adobe ARR reached "
    "$27.10 billion, and AI-first ARR exceeded $500 million."
)

3. Attribute the answer

import os
import requests

response = requests.post(
    "https://api.tokenpath.ai/v1/attributions",
    headers={"Authorization": f"Bearer {os.environ['TOKENPATH_API_KEY']}"},
    json={
        "document": document,
        "question": question,
        "answer": answer,
        # Python string indices count Unicode code points.
        "offset_encoding": "utf-32",
    },
    timeout=(10, 90),
)
response.raise_for_status()
attributions = response.json()["spans"]

utf-32 matters here. Extracted PDFs often contain math symbols outside the Basic Multilingual Plane, and those would shift every highlight under the default utf-16. See Offsets and encoding. Check that each range slices the text you expect:

for item in attributions:
    claim, source = item["answer"], item["source"]
    if source is None:
        continue
    assert answer[claim["start"]:claim["end"]] == claim["text"]
    assert document[source["start"]:source["end"]] == source["text"]

4. Turn source ranges into PDF boxes

def boxes_for_source(source: dict, characters: list) -> dict[int, list]:
    boxes_by_page = {}
    for char in characters[source["start"]:source["end"]]:
        if char is None:
            continue
        boxes_by_page.setdefault(char.page, []).append(char.bbox)
    return boxes_by_page

for item in attributions:
    source = item["source"]
    if source is None:
        continue
    for page_index, boxes in boxes_for_source(source, characters).items():
        print(page_index, source["text"], len(boxes), "boxes")

In the viewer, merge neighboring character boxes on the same line into a few highlight rectangles. When a reader clicks a citation:

Mark the answer range as selected.
Scroll the PDF pane to the first box.
Draw every box over the rendered page.
Show the source text and confidence next to the answer.

One answer can cite evidence on several pages. The page is only where the evidence sits. What gets attributed is the text itself:

The Adobe answer with $27.10 billion selected and the exact matching text highlighted on page 18 of the PDF. — The Total ARR citation jumps from the same answer to its exact source on page 18.

Run the demo

The cookbook downloads Adobe’s public 24-page Q2 FY2026 earnings PDF and checks it against the bundled Adobe Extract character map. It then makes a live TokenPath request and serves the interactive viewer shown above.

git clone https://github.com/tokenpath/tokenpath-cookbook.git
cd tokenpath-cookbook/pdf-attribution

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

export TOKENPATH_API_KEY="YOUR_API_KEY"
python prepare_demo.py
python service.py

Open http://localhost:8000/attribution-viewer/?doc=adobe-earnings and click any linked phrase in the answer.

Use your own PDF

Create an Adobe PDF Services OAuth Server-to-Server credential. Then put its client ID and secret in .env:

cp .env.example .env
# Set PDF_SERVICES_CLIENT_ID and PDF_SERVICES_CLIENT_SECRET.

python run.py path/to/report.pdf

run.py writes output/report/bundle.json. Generate an answer from the bundle’s markdown, then attribute that exact answer:

python attribute.py output/report/bundle.json \
  --question 'What changed this quarter?' \
  --answer 'Revenue reached $42 million.'

python service.py

Open http://localhost:8000/attribution-viewer/?doc=report to inspect the result.

Practical notes

Store the extracted text and its character map together. Never regenerate one without the other.
With retrieval, attribute the PDF sections the model actually read, not the whole file.
For scanned PDFs, use OCR that returns word or character coordinates.
Keep quadrilaterals instead of boxes if your PDFs contain rotated text.
An empty attributions list means there is nothing to link. Render the answer without citations. It is not an error.
Use confidence to rank or style links. It describes how strong the link is, not whether the answer is correct.

Get started

Core concepts

Advanced

Use cases

Cookbooks

Resources

PDF attribution cookbook

How the mapping works

1. Extract the PDF with coordinates

2. Generate an answer

3. Attribute the answer

4. Turn source ranges into PDF boxes

Run the demo

Use your own PDF

Practical notes

​How the mapping works

​1. Extract the PDF with coordinates

​2. Generate an answer

​3. Attribute the answer

​4. Turn source ranges into PDF boxes

​Run the demo

​Use your own PDF

​Practical notes

How the mapping works

1. Extract the PDF with coordinates

2. Generate an answer

3. Attribute the answer

4. Turn source ranges into PDF boxes

Run the demo

Use your own PDF

Practical notes