Knowledge bases rarely fail at launch. They fail quietly, months later, when the source folder has moved on and the data set has not. Buyers described the symptom precisely: a set nobody has touched for half a year reads like a stale backup, and the first wrong answer it produces gets blamed on the AI rather than the calendar. Two questions decide which ending you get, and both surface in the first technical call: what can the pipeline reach on its own, and whose name is on refreshing it?
Connectors & Corpus Refresh
How Do You Connect AI to File Shares and Databases,
and Keep the Index Current?
What an ingestion pipeline reaches on its own, what a person still loads by hand, and the ownership decision that determines whether your knowledge base is still worth trusting a year from now.
Built from real buyer questions in our sales meetings
Direct Answer
Files come in natively; everything else arrives through a landing zone. Blockify ships native support for S3-compatible object storage and SharePoint blob storage, and Iternal states the underlying rule plainly: as long as the files can be accessed and received, the source itself does not matter. AirgapAI pulls documents from any networked file share a customer has, and a share every user can already reach shows up inside the app as a location to pull files from. Scheduled ingestion jobs plus a named data owner for each corpus is the minimum viable answer.
The limit: most of the loop is still manual on today’s build. Iternal is direct about the gaps. There are no direct connectors into enterprise stores such as a wiki or an IT service-management system. Nothing watches a directory and re-processes documents when its contents change. The assistant on the device answers from the data set it already holds rather than fetching from a network location mid-answer, and keeping local models and data sets current is left to the customer. Plan the owner before you plan the schedule.
Three things to settle in writing before you buy. Which sources are natively supported in the build you receive; whether the schedule option matches the refresh cadence your business expects; and what the pipeline does with a document that changes after the set was built. Iternal answers each of those in the targeted questions below.
Scheduled ingestion is the design; a named human on a calendar is what the corpus records. A timer can run overnight forever and still hand you a stale set, because somebody has to decide what belongs in it and what ages out. That decision belongs to a role you assign. For more information on which formats and content shapes the pipeline accepts, visit the file-format page; for more information on how a finished corpus is chunked, embedded and searched, visit the retrieval architecture page.
What Gets Pulled In Automatically, and What Stays Manual
Buyers ask this in their own words: can it attach to a database, can it index files on a local drive, can it hook into the object stores. One boundary sits underneath every variation — configure-once ingestion on one side, an integration your team builds and maintains on the other. The table sorts the common sources across that line.
| Source | How content gets in today | What it costs you |
|---|---|---|
| S3-compatible object storage | Native support in Blockify. You can point it at your own object store rather than the bundled one. | Configure once |
| SharePoint blob storage | Native support in Blockify, which also integrates with major data providers and data lakes. | Configure once |
| Network file shares and mapped drives | Selected directly when a set is built. A share every user can reach appears as a location to pull files from. | Configure once |
| A local drive or a connected OneDrive | The add-my-data button takes dragged files through Blockify and creates a new corpus on that machine. | Per machine, per user |
| A dedicated landing zone | Iternal recommends one location, typically a SharePoint folder or the Drop Zone secure file share, that content is piped into. | Build the feed once |
| Wikis, service-management tools, databases, the open web | No direct route today. Export or script the content into the landing zone instead. | Ongoing, and owned by you |
The bottom row reshapes project plans. Blockify has no direct connectors to other enterprise storage systems such as Confluence or ServiceNow, and the pipeline will not go out to the web to source material, so the content has to be somewhere on your network already. Iternal is candid about the reason: connector breadth costs engineering months per platform. Model Context Protocol is under investigation as the route that would let the software call out to other systems, with no date attached — direction rather than a dependency.
One piece of Iternal guidance is worth banking early. You can point the pipeline at a raw, sprawling share, and Iternal advises against it: cleansing and curating first produces a better corpus, and the gap between a set that answers well and one that answers confidently but wrongly opens here rather than at query time. For more information on proving an answer back to its source, visit the traceable-answers page.
How an Ingestion Job Actually Gets Built
The mechanics are smaller than the anxiety around them. A Blockify job is created by naming it, selecting the files and optionally scheduling the run — three decisions in a user interface, with no code in the common path. Walk the loop once before you design a governance process around it.
- Name the job. The name is the handle your data owner will use for years. Name it for the domain it serves, never for the day it was created.
- Select the files. Point at a folder rather than picking documents one at a time. Iternal recommends a dedicated upload folder per content type, which makes the next refresh a repeat rather than a rebuild.
- Schedule the run. Jobs are scheduled directly from the interface, and AirgapAI carries a schedule option that re-runs local data set creation on an iteration. Iternal recommends running large jobs overnight, when the GPUs would otherwise sit idle. A stalled job is recovered by pausing or cancelling it at the top, then restarting.
- Collect the output. Finished jobs return as data sets you can use in the app or hand on. Blocks export as CSV or TSV, a finished set downloads as a JSON file, and an embedded-block export writes to local storage with the matching embedding model so the package loads cleanly into the assistant.
Scale changes where the work runs rather than what the steps are: larger jobs need server-class models, and the Blockify build bundled inside the local assistant carries fewer controls than the server-side license.
Ownership Is the Control That Keeps a Corpus Trustworthy
One objection surfaces in almost every serious evaluation, and it is the right one: somebody has to own keeping this corpus current and authorized, forever. Buyers laid out the failure modes without prompting. Mislabeled or outdated material dropped into a shared library acts as a poison pill and creates hallucination risk. Stale documents left in the folder before a build waste context. Letting every user process whatever they like produces distilled content nobody has checked, and users end up unsure which candidate source of truth to trust.
The answer is a role assignment rather than a product feature. Iternal names it the same way: the customer owns an ongoing process to clean, authorize and upload documents into the data set, and the model needs an authoritative source list to index as its source of truthfulness. Four responsibilities cover it, and they belong in the statement of work, which is why Iternal engagements carry a RACI appendix.
Responsible
The data owner, one person per corpus
Decides what enters the set, what ages out and when the next run happens. Without this name, the schedule is decoration.
Accountable
The business leader for that domain
Signs that the corpus is the authoritative source list for the questions it answers, and carries the consequence when it is wrong.
Consulted
The librarian or reviewer
Adjudicates the exceptions the pipeline elevates: competing versions of a claim, and the rare unique passage that must survive distillation.
Informed
The people asking the questions
Need to know which set is authoritative for which subject, and when it was last refreshed.
Tooling carries part of the load. Blockify finds the candidate sources of truth, lets you authenticate them and then manages them, distilling competing versions of a document into one maintained master data set, and a hybrid mode processes whole repositories instead of forcing a human review of every file. An autonomous pipeline still needs a librarian back end for low-touch exception handling. Staleness is an ownership gap far more often than a tooling gap: the pipeline that produced a stale set was usually working perfectly; nobody ran it.
Refreshing What Changed: Re-Indexing and Re-Publishing
Can a single updated document join a set that already exists? Yes. Drag the newer documents into that data set to build on it, or create a new one, and the merge is handled intelligently rather than by appending a duplicate. Blockify distills competing versions into one maintained master set: the feedback loop re-indexes each new document against the existing block set, elevates overlapping claims for a person to adjudicate and distills the rest.
Does the set re-index itself when a file changes? No, and the distinction is the one to hold onto: refresh on today’s build is time-driven rather than change-driven. A schedule option re-runs local data set creation on an iteration, and Blockify connects via API so refresh can be scripted at whatever frequency you need. No built-in trigger re-vectorizes documents when a watched directory changes. Iternal describes the current update path as re-ingesting everything or just the new material, says openly that the best approach to re-indexing local sets is still being worked out, and offers professional-services guidance on automating refresh.
Getting a refreshed set back onto the devices. The delivery chain runs back-end ingestion, retrieval database creation, scheduling, content delivery, then packaging onto the offline device. Server-side refresh can be automated and orchestrated out to user machines with Microsoft Intune. One constraint belongs on the architecture diagram now: the device-local implementation cannot centralize a corpus across users, so a set that must be identical on forty laptops is a distribution job you plan for.
Cadence is a business decision. Iternal consults on an organization’s rate of change so sets get refreshed before anything goes stale, and regulated industries need a tighter cadence than the rest. An IT director whose source data moves weekly is a different problem from a policy library that moves twice a year. Four questions turn this into commitments you can hold:
Pin it down: questions for your evaluation
-
Which sources are natively supported in the build we will receive, and which ones require staging files into a landing zone?The line between configure-once ingestion and an integration your team builds, funds and maintains.
-
Does the schedule option re-run the whole job, or process only what changed since the last run?Whether your refresh window and your compute budget survive a corpus that keeps growing.
-
What happens to an existing block when a newer version of its source document is ingested, and which decisions still reach a reviewer?How version drift is resolved, and how much reviewer time the refresh loop consumes.
-
How is a refreshed data set delivered to devices already in the field, and can our device-management tooling carry it?Who performs the re-publish step, and whether the last mile is automated or a person walking to desks.
When the Data Sits Inside a System Somebody Else Controls
A pattern came up again and again, and it has nothing to do with AI. Buyers described case-management platforms that shut off API access, contract data that has to be pulled out of one application before another can process it, and the daily indignity of attaching files one at a time when direct access to the source is what they want. One buyer put it simply: corporations hold enormous amounts of information the AI knows nothing about until somebody brings it there.
Iternal applies one design principle to all of it. An aggregator sits between the AI interface and the underlying source, so extraction happens once and access is granted in a single place. Four routes fill that aggregator, ranked by the effort they impose on your team:
- Export to files. The cheapest route and the most common. Export from the source in a standard format such as Word or plain text and the pipeline takes it. A contract data store built this way stands on its own, so the assistant is pointed at the finished set without touching the platform of record.
- Script the export into a landing zone. A non-AI process queries the source APIs and drops files into a drop zone, which keeps extraction uniform on the application side. An API can also be registered as a data source alongside those file drops, and Blockify can be scripted through its own API to refresh sets on a cadence.
- Hand the extraction over. Iternal runs a secure file share called Drop Zone for source material, and an Iternal team can come on site, take the files and ingest them. Iternal will also build additional sets and refresh existing ones as more documents arrive.
- Reach into the live system. Not available on today’s build. The current version of AirgapAI works with files, does not query databases or other systems and exposes no API, so material is loaded as a point-in-time data set. Agent-driven querying is the stated direction; treat it as upside.
The ranking matters more than the options. Teams that start at route one ship a working corpus in weeks; teams holding out for route four wait on a roadmap. Move the material through the cheapest door available, name the owner, upgrade the plumbing later.
Answered elsewhere
- Which formats, file types and content shapes the pipeline will accept — see the file-format page.
- How a finished corpus is chunked, embedded, searched and sized — see the retrieval architecture page.
- Governing material that is scattered across many stores at once — see the governance page.
- How the pipeline, the local assistant and the data set relate to one another — see the component overview.
- Making live calls into the business systems you already run — see the enterprise integration page.
- Who runs the deployment, and what support looks like once it is live — see the day-two operations page.
FAQ
FAQ: Connectors, Refresh and Corpus Ownership
Both work, by different routes. Blockify ships native support for S3-compatible object storage and SharePoint blob storage, and you can point it at your own object store instead of the bundled one. For an SMB or network share, files are selected directly when the set is built: AirgapAI pulls documents from any networked file share you have. Iternal states the underlying rule simply — as long as the files can be accessed and received, the source does not matter.
Refresh is time-driven rather than change-driven. A schedule option re-runs local data set creation on an iteration, and Blockify connects via API so refresh can be scripted at whatever frequency you need. No built-in trigger re-processes documents when a watched directory changes, so a person or a timer still starts the run. Iternal says openly that the best approach to re-indexing local sets is still being worked out.
Your organization does, and the role belongs in the statement of work rather than in a product datasheet. Iternal names it directly: the customer owns an ongoing process to clean, authorize and upload documents into the data set, and the model needs an authoritative source list to index as its source of truthfulness. That is four responsibilities — a named data owner per corpus, an accountable business leader for the domain, a librarian who adjudicates elevated exceptions, and users who know which set is authoritative. Iternal engagements carry a RACI appendix for exactly this reason.
Yes. Drag the newer documents into that data set to build on it, or create a new set if the subject has moved on. The merge is handled intelligently rather than by appending a duplicate: Blockify distills competing versions into one maintained master set, and the feedback loop re-indexes each new document, elevates overlapping claims for a person to adjudicate and distills the rest. Confirm during evaluation how version metadata is weighted in that clustering step, so a real distinction between two revisions survives.
Not on the current build. AirgapAI works with files, does not query databases or other systems and exposes no API, so material is loaded as a point-in-time data set. Blockify likewise has no direct connectors into enterprise stores such as Confluence or ServiceNow. The working pattern is an aggregator between the AI and the source: extract into a landing zone by exporting files, or by scripting a non-AI process that queries the source APIs. Agent-driven querying is the stated direction, with no date attached.
Rank the routes by effort and take the cheapest one that works. Export to files first — a standard format such as Word or plain text is enough, and a data store built that way stands on its own, so the assistant is pointed at the finished set without touching the platform of record. Next, script a non-AI process that queries the source APIs and drops files into a drop zone. Third, hand it over: Iternal runs a secure file share called Drop Zone, and an Iternal team can come on site, take the files and ingest them. Live reach-in is not available today.
Build One Set, Then Schedule Its Second Run
One week is enough to test the whole refresh loop. Pick one folder that genuinely changes, build a data set from it, write a name next to it in your RACI, schedule the second run, then edit a source document and watch what the pipeline does with the newer version. What refreshes cleanly tells you the tooling fits. What needs a hand tells you where your data owner will spend their Monday.