A friend who runs the IT side of a mid-size hospital told me a story I have not been able to shake. One morning a resident needed last year's comparison scans and the latest imaging report to prep for a tumor board. The files were there — somewhere — spread across the PACS, a department shared drive, and a consultant's laptop. Forty minutes later she had three near-duplicates and none of them confirmed as the final version. She closed the tabs, opened the patient chart, and just went with what she remembered.
Multiply that by every clinician, every day, and you start to understand why "let's just buy an AI" never starts from a clean slate. The model is not the hard part. The documents are.
For a while I assumed the problem was a lack of data. Hospitals are drowning in it — scans, discharge summaries, clinical guidelines, research drafts, governance memos. Surely the fix is to feed the model more. The more I looked, the more I came to the opposite conclusion. The institutions that got somewhere with medical AI were the ones that did subtraction first: smaller files, fewer stray copies, sensitive fields stripped out, knowledge put where people could actually find it.
The hidden tax nobody puts on a slide
There is a knowledge-management comparison I keep coming back to. It found that employees in organizations without a formal knowledge program spend about 8.5 hours a week hunting for information. Give them a program and that drops to 4.6 hours. That is 3.9 hours reclaimed per person, every week, which the report translates to roughly 98 full-time equivalents freed up per 1,000 employees.
Sit with that number for a second. In a hospital, the things people are hunting for are not trivia. They are the current care pathway, the right prescribing rule, the chart template that actually matches this week's protocol, the consult note from the colleague who left last year. A doctor looking for the "latest" version of a document often spends longer than the appointment that needed it. And when a file cannot be found, someone rebuilds it — so now there are twelve versions, and the correct one is buried at the bottom.
This is not laziness. It is what happens when the volume of documents outruns the mechanism for keeping them straight.
The data that nobody touches
Here is a fact that surprised me. Komprise, looking across enterprise NAS storage, found that 60% to 70% of data had not been opened in over 90 days, yet it was still sitting on the most expensive primary storage tier. It just sat there, quietly costing money, never read.
Translate that to a hospital and you get the imaging archive. Once films are digitized, terabytes of DICOM sit almost never opened, while the storage budget climbs every year. A lot of departments are not actually short on storage. They are short on a habit that says: delete what should be deleted, archive what should be archived, and stop pretending every file from 2014 is still load-bearing.
What does this have to do with AI? Everything. Because the tempting move is to sweep all of that cold, duplicated, half-labeled material into the model pipeline. You pay twice when you do — slower retrieval and wasted compute — and you get worse answers, because the model is now fishing in a colder, noisier pond.
More data is not the answer
There is a reflex in this industry to treat every problem as a data problem. More training, more context, more documents. But uncurated piles do something specific and unpleasant to a retrieval system: they slow it down, they dilute the signal, and they make the model confident about the wrong version of a fact. In medicine, a retrieved answer that looks certain but cites the stale protocol is not a quirky bug. It is the thing you were supposed to prevent.
So the people who made progress tended to flip the order. Instead of add data, then hope, they did shrink the files, strip the privacy, tidy the knowledge, then let the model read what is left. The model became usable precisely because there was less noise in front of it.
The wall you cannot argue with
None of this is a purely technical debate, because medical data has a temperament of its own. Patient health information, unpublished trial results, research datasets — a lot of it is bound by rules about where it is allowed to live. For many hospitals, "step outside the intranet" is simply not on the table. And the raw documents are studded with names, ID numbers, record numbers, family histories. Feed them to anything as-is and you have just laid the privacy bare.
That is the part that gets underestimated. The work that actually matters is upstream of the model: catching those sensitive fields before anything is indexed, so the knowledge that goes in is knowledge you are allowed to use. For teams building this, a RAG data engineering base that runs on-premise and masks before indexing is the piece that decides whether the project is even permitted to start. Compliance, in this world, is not a feature you bolt on. It is the entry ticket.
The three quiet moves
The teams that got somewhere did not reach for one silver bullet. They ran three lines together.
First, they shrank the files. Document compression — done locally, so nothing leaves the environment — takes the volume down enough that retrieval and inference stop being the expensive part. The point is not that a PDF gets smaller. It is that the model can finally read the file at all, and you can finally afford to let it.
Second, they structured the knowledge. Rather than dumping raw PDFs into a vector store, they cleaned, chunked, and traced the material first, so a retrieved answer points at something real and version-correct. A model answering from a clean, provenance-tracked index is a different animal from one dredging a folder of scans.
Third, they kept the whole thing inside the building. On-premise deployment means the knowledge and the audit trail both stay where the regulators expect them.
What to actually weigh
If you are evaluating an approach like this, the questions are rarely "which model." They are about fit. A framework I have found useful:
| Dimension to evaluate | What to check |
|---|---|
| Data residency and exit rules | Does any byte leave your network, and what regulations bind that data? |
| Sensitive-field handling | How many categories of personal or health information can it detect and mask before indexing? |
| Format coverage | Which of your real file types — scans, images, Office, OFD, DICOM — are actually supported? |
| Volume reduction | What is the typical size reduction, and does it touch legibility or only footprint? |
| Cost at scale | What does processing and retrieval cost per thousand pages as the corpus grows? |
| Auditability | Are ingestion, masking, and retrieval traced well enough for compliance review? |
| Deployment fit | Can it run on-premise or air-gapped, including on the operating systems you already use? |
I keep the language deliberately relative on purpose. I am not making the call for you. The question that actually decides the project is narrow and blunt: is your data allowed to cross that wall? If the answer is no, most of the popular shortcuts fall away, and the quiet moves above become the moves most worth prioritizing.
The part that stuck with me
My friend's hospital did not fix this with a bigger model. They fixed a slice of it with a Friday afternoon and a naming convention, then a longer project that compressed and de-duplicated the archive, then a retrieval layer that finally knew which version was current. Boring work. Unsexy. But the resident from the opening story now finds her comparison scans in under a minute, and the tumor board starts on time.
Most hospitals are not stuck because they lack data. They are stuck because they have too much of it, too messy, too sensitive, and too hard to find. Shrink it, strip it, straighten it — and the model stops being the project. The tidiness was the project all along. More of these engineering notes live on the blog.
Try it now
To see how a private, on-premise pipeline handles medical documents, visit the UglyPear Data site for the RAG data engineering base, or review SmartSlim for compression. More engineering notes and cases are on the blog.
FAQ
Q: Why does medical AI stall before the model is even involved?
A: Because the bottleneck is usually the documents — unfindable, duplicated, and studded with sensitive fields — not the model. Getting the corpus clean and private is the work that decides whether the project can start.
Q: Does compressing medical files make them unreadable?
A: Compression targets size, not legibility. The published benchmark processed a 443-page OFD in 124 seconds end to end with text and seals intact. The question to ask is whether the compression is lossy enough to hurt reading.
Q: Is on-premise deployment required for medical data?
A: Not always, but anything touching patient privacy or bound by data-exit rules is far safer on-premise. Data stays in the intranet and the compliance pressure drops sharply.
Q: How is a private RAG base different from just feeding files to a model?
A: Feeding directly tends to overflow context, lose the thread, and answer the wrong thing. RAG chunks, masks, and indexes first, then retrieves on demand, so answers are more traceable.
Q: What does "sensitive-field handling" actually mean?
A: Detection of categories like names, ID numbers, and record numbers before indexing, with masking that is either reversible or irreversible depending on policy — configurable at deployment.