Ingestion projects break in the same place every time. Someone confirms that PDFs are supported, and six weeks later the team discovers that the PDFs in question are photographs of paper. Format support is a list; extraction is a result. The distance between the two is where a quarter disappears.
Ingestion Formats & Content Shape
What File Types Can Be Ingested, and What Shape
Does Your Source Content Need to Be In?
The list by extension, the pre-processing each format needs, and where the on-device path stops short of the server one.
Built from real buyer questions in our sales meetings
Direct Answer
Ask for the list by file extension, not by category. A production pipeline should take the office formats you already hold — word-processor documents, slide decks, spreadsheets and PDFs — plus scanned images needing text extraction first, plus structured exports such as tables and database dumps, and it should say which it opens natively and which need converting. Iternal states that Blockify ingests Word documents, PDFs, slide decks, emails, images and diagrams on the server and cloud path, and describes the engine as format agnostic so long as content converts to text.
The limit: the device path is much narrower than the pipeline above. The lightweight utility that runs on a laptop handles word-processor documents and PDFs. Iternal states that AirgapAI does not currently work with images or presentation files; that scanned PDFs of old documents need a different technology entirely; that images inside source PDFs are ignored during parsing, so only the text is processed; that PDF ingestion works less well than the other document formats, and local extraction is weaker than the server-side equivalent; and that no work has been done on spreadsheet parsing, with complex workbooks needing restructuring by hand first. Volume work and the wider format list run on the server or cloud path.
Shape decides the result as surely as format does. A parser that opens your file has not told you the content survived. Give a data owner criteria they can check: real text rather than a picture of text, a date on each record, one content type per folder, complex workbooks flattened. Iternal advises against pointing ingestion at a whole file server for that reason.
What goes in, and the condition it arrives in. For more information on how ingested content is split, embedded and searched, visit the retrieval architecture page. For more information on pulling files out of the systems holding them, visit the connectors and refresh page.
The Format List, by Extension
Categories hide the failures: “documents” covers both a born-digital file that parses cleanly and a scanned binder that does not. Read it as a data owner needs it — extension, what has to happen first, which path does the work.
| Extension | What has to happen to it first | Where it runs |
|---|---|---|
| .docx .doc | Native. A pre-processing step cleans the XML inside word-processor files. | Device or server |
| Native. A PDF goes in, markdown comes out; the parser reads text only, so images inside are ignored. | Device or server — weakest device format | |
| .pptx .ppt | Native on the server path; slide content is a shape the engine was built for. | Server or cloud |
| .txt .md .json .yaml .xml | Native. Markdown, JSON and YAML now sit alongside the traditional text formats. | Server or cloud; device coverage not stated |
| .csv | Native. One engine turns a CSV into a structured table a model can read. | Device or server |
| .xlsx .xls | Converted. No spreadsheet parser has been built; Iternal states the format is a mess. | Export to CSV or flatten first |
| .eml .msg | Native on the server path, via an engine that scrubs threads and redacts personal data. | Server or cloud |
| .pst | Unproven. Iternal states these archives have not been worked with before. | Get it in writing first |
| .jpg .png, diagrams | Converted. A prompting workflow extracts meaning beyond raw OCR text, then structures it. | Server or cloud |
| .mp3 .m4a | Converted. Common audio transcribes first; the transcript, not the audio, reaches the data set. | Transcription, then ingestion |
| video files | Not yet an input format, still being built. Transcripts are ingested; the video is not. | Not yet supported |
| .zip | Native. Archives are accepted; executables inside them are not, because an .exe could be malicious. | Server or cloud |
| .jsonl | Not a source format — the finished data set. Data set upload accepts JSONL only. | Output side |
Iternal adds new types on customer feedback — YAML arrived that way — so treat the table as a snapshot.
What Shape Your Source Content Has to Be In
Buyers ask the format question first and the shape question second, and the second decides more outcomes. Hand a data owner these checks and the quality argument happens before ingestion, not after the first bad answer.
- Real text, not a picture of text. Open a sample from the middle of the batch, not the cover page.
- A date on the record. Timestamps help accuracy; without them nothing can tell which version is current.
- One content type per folder. A subfolder per document type lets a whole batch run in one pass. Individual PDFs need no renaming.
- Tags and purpose supplied up front. The engine accepts any number of customer-defined tags and fields, and context can be attached so the blocks specialize toward your goal.
- Markdown for technical manuals. That path emits a distinct technical-manual block rather than a standard one.
- A scoped batch, never the whole file server. The state of the data decides what comes out.
For extraction work with a defined output, Iternal states that five to ten samples plus a written description of what you want is ample to configure the run.
Tables, Spreadsheets and Databases: What Actually Lands
One objection arrives in the same shape from finance, operations and IT alike: it only handles part of our data. Three problems hide inside it.
Tables inside documents survive. The ingestion output is markdown, which looks poor to a human while preserving table structure in a form models consume well. Iternal has invested in structures for model processing of table-based data and will recommend an approach for your document types. Judge the markdown on whether the numbers stayed attached to their headers.
Spreadsheet files are a preparation job. CSV ingests directly; native workbooks do not. Iternal is direct about the device path — local hardware handles spreadsheets and SQL content less well than cloud models, and the unpredictable AirgapAI results Iternal has seen mostly involved spreadsheet content. Flatten the workbook, export to CSV, or run it on the server path.
Live systems are a different question from files. The current AirgapAI release reads files and does not query databases, and Blockify does not integrate directly with ERP or CRM platforms today. An export closes the gap without a downgrade, because a model reads SQL or XML as capably as markdown.
Pin it down: questions for your evaluation
-
Send the supported list by file extension, marking each native or needing a conversion step.Which formats arrive intact, and which cost you a preparation stage.
-
Which of those extensions does the on-device path cover, and which require the server or cloud path?Whether the deployment you are buying covers your material.
-
For our table-heavy and spreadsheet sources, what does the pipeline produce, and what must we restructure?How much manual preparation your team owns before the first run.
-
Run our hardest pages — a skewed scan, a color-coded diagram, a dense table — and show us the markdown.Extraction quality on your own material, not on a clean demonstration file.
When the Records Are Still Paper
Some organizations do not have a file-type problem yet, because they do not have files. Buyers described employee records where nothing is digital, agencies undecided whether to scan in-house or pay someone, and office equipment that was the wrong type for capture. Digitization is the entry condition for everything above.
The economics help. Digitization is scoped at a fixed cost per page: a known unit price against a countable stack, which makes it the low-complexity place to start. Outside scanning firms are built for million-document projects, so small jobs price badly; a scanning service instead rents high-speed scanners and puts operators onsite, and afterwards the originals can be reboxed, shredded or handed back.
One rule saves more than any equipment choice: never print a digital document in order to scan it back in. Files created digitally can be used exactly as saved, so the budget only covers the legacy backlog. Use a real document scanner, not the office multifunction device — curved and skewed scans destroy standard extraction and return gibberish no cleanup step repairs.
One Engine Per Content Shape
A contract and a maintenance manual are both PDFs and almost nothing else about them is alike. Summarizing a contract changes what it says; reordering a manual makes it dangerous. Iternal states that Blockify runs eight ingestion engines tuned to different source content:
- Legal contracts. Keeps the exact, precise wording, because a contract cannot be summarized without altering its meaning.
- Technical manuals. Preserves order and procedure, parses to markdown first, emits its own block type.
- Email. Scrubs threads for the technical knowledge buried in them, indexes the result anonymized, redacts names and dates.
- Human spoken communication. A separate engine, because written and spoken language behave differently.
- Policy and procedure. For material where steps relate to one another and image recognition carries meaning.
- General unstructured documents. The default for everything else.
The design decision worth borrowing: deduplication is deliberately withheld from log, telemetry and legal data, because that material is time-sequenced and must stay immutable. Ask which engine would process each of your record classes.
How PDF and Scan Extraction Fails, and How You Catch It
Extraction fails quietly, which is what makes it expensive. Nothing errors; the answers come back thinner than the source. Buyers described the same failures repeatedly, and each has a test you can run.
- The page that looks fine and parses as a jumble. Test: read the markdown, not the PDF.
- Curved and skewed scans, which return gibberish. Test: put your worst scan through first, not your best.
- Meaning that lives in pictures. Iternal states images inside source PDFs are ignored during parsing. Test: ask a question whose only answer sits in a diagram.
- Documents far larger than anyone scoped. Iternal states the system was not built for PDFs far beyond six hundred pages. Test: time a real one before planning the rollout.
- Proprietary containers that simply do not open. Test: send a sample file for review rather than asking whether the format is supported.
Detection comes down to inspecting the intermediate. Ingestion is visually dull — a PDF goes in, markdown comes out — and that markdown is the last point where a human sees what survived. Budget accordingly: on-device processing runs at roughly a page per minute, and a few hundred-page PDFs took two to three hours through the utility bundled inside AirgapAI.
Answered elsewhere
- How ingested content is split, embedded and searched — see the retrieval architecture page.
- Grounding answers in your documents and proving where each one came from — see the answer-accuracy page.
- Pulling files automatically from shares, buckets and business systems — see the connectors and refresh page.
- Which material belongs in a corpus, and who governs scattered stores — see the governance page.
- How the ingestion engine, the local assistant and a data set relate — see the components overview.
FAQ
FAQ: Ingestion Formats and Content Shape
CSV yes, native workbooks no, live systems no. AirgapAI bundles a lightweight Blockify that supports CSV files, and one engine turns a CSV into a structured table a model can read. Iternal states that no work has been done on spreadsheet parsing and that complex workbooks may need restructuring first. The current release reads files rather than querying databases, so export first.
On the server and cloud path, yes. The ingestion engine performs OCR as it ingests, and Iternal states it uses vision-capable language models rather than classic OCR, extracting text at roughly ten times the fidelity of traditional OCR depending on document quality. The device path is narrower: AirgapAI does not handle scanned PDFs of old documents, which needs a different technology.
Digitization, scoped as its own piece of work. It is priced at a fixed cost per page, which makes it the low-complexity place to start. A scanning service rents high-speed scanners and puts operators onsite; afterwards the originals can be reboxed, shredded or handed back while the pages are indexed into a database you control.
Into the ingestion pipeline on the server and cloud path, yes: Blockify accepts images, photographs and diagrams and runs them through a prompting workflow that extracts meaning beyond raw OCR text. Into the local chat box, not today — Iternal states the current AirgapAI version is text based and does not work with images or presentation files.
A PDF directly is fine on the general path — ingestion is an upload of a PDF that outputs markdown. Technical manuals are the exception: Iternal states that manual ingestion requires markdown because its processing path differs, and that engine emits a distinct technical-manual block. AirgapAI does not support manuals yet.
Test It With Your Own Worst Documents
Every claim here is checkable with one batch of your own material. Take a skewed scan, a color-coded diagram, a dense table and the longest file anyone will admit to, and read the markdown that comes out. What survives is what your people can ask about. What vanishes is the preparation work to budget for.