Before you read another paragraph, run a quick self-test on the environment you are about to migrate. Answer from memory, without opening a file browser:
- Can you name every file format your organization actually produced last year — not the ones on the plan, the ones sitting on disk?
- Do you know which department holds medical imaging, survey layers, lab datasets, or laser scans, and on which drive they live?
- Is there any system whose files nobody on the IT team knows how to open? Is that written down anywhere?
- Has your migration checklist absorbed any format because a department complained — or only because a template said so?
If you hesitated on more than one of those, you are in the majority, and this article is for you. What follows is a sweep for the long-tail industry formats — DICOM studies, HDF5 datasets, SHP layers, KMZ packages, LAS point clouds — that almost never appear on a migration checklist yet are exactly the files that stall a private or air-gapped move at the finish line. The sweep is deliberately scoped so a small team can run it inside a week.
Why audits go blind in the cold pile
Every migration plan inherits a blind spot from the same place: the pile nobody opens. Komprise's research puts a number on it — 60–70% of enterprise NAS data goes untouched for more than 90 days, yet much of it occupies expensive primary storage. That cold pile is where a decade of one-off project deliverables, exported datasets, and partner packages accumulates, and it is precisely where your rarest formats are hiding. A checklist built from "what we use day to day" surveys only the warm 30–40%.
The human cost is just as measurable. A 2025 knowledge-management survey found that employees at organizations without such a program spend 8.5 hours a week hunting for information; those with one get that down to 4.6 hours. The gap only closes when files actually live where people can reach them. A DICOM archive stranded on a retiring workstation is not in anyone's search index — it is invisible time, leaking out of the schedule every week until someone trips over it during go-live testing.
There is also a deadline effect that audits routinely ignore. Legacy workstations able to open these formats are finite resources. A retired specialist, a failed disk, or an operating system past its security-patch horizon can cut off an entire data category overnight. The sweep below is not bureaucratic hygiene; it is a race against hardware and personnel that are already leaving.
Where the long-tail formats actually hide
In our own client work at UglyPear Data, the same hiding spots come up again and again. Walk these corners of your shared drives and archive servers before you claim coverage:
- Export and staging folders. Systems that "send data out" usually keep a copy in an export directory — imaging modalities dumping DICOM studies before transmission, GIS tools writing SHP layers for consultants, drones offloading LAS scans after a site flight. These folders are nobody's responsibility and therefore nobody's checklist entry.
- Project archive trees. Look for folders named by year, client code, or project number. The final deliverable folder is usually reviewed; the
raw,working, andoldsiblings beside it are not — and that is where the original-format files sit, long after the polished exports moved on. - External delivery packages. Anything a partner sent inbound — a KMZ exchange package from a planning consultancy, a zipped batch of scans from an outsourced digitization vendor — tends to be saved as received and never converted. Search attachment folders and file-transfer drop directories.
- Individual workstation drives. The single most common vault for long-tail formats is one specialist's desktop machine. If your organization has "the person who handles the scanner" or "the one who runs the survey software," their local disk is a primary archive in practice.
- Legacy system leftovers. Retired applications often leave their data behind in proprietary or industry formats on a share that still mounts but nobody maintains. Check anything labeled
migration_temp,backup_20xx, or similar — stale names are a reliable scent.
A practical shortcut: sort your file-server inventory by extension and look at the tail. The mainstream block (PDF, Office, images, video) is expected; everything below a certain frequency threshold is your candidate list. Most teams running this exercise find five to fifteen extensions they cannot immediately explain.
Who to ask — the departments nobody consults
A file-extension list tells you what exists; only people can tell you what matters. The long tail lives with specific roles, and they are rarely in the migration meetings:
- Imaging and clinical staff for DICOM and related medical formats — ask which studies must stay readable, for how long, and who currently opens them.
- Surveying and GIS teams for SHP layers, KMZ packages, and LAS point clouds — ask which datasets are referenced by active planning work versus finished projects.
- Laboratory and research staff for HDF5 and similar scientific containers — ask whether the raw data is reproducible or is the only copy in existence.
- Engineering and design consultants — external parties deliver in formats chosen by their tooling, so ask what they will send over the next two years, not just what they sent last year.
- Records and archive keepers — they usually know about the legacy leftovers everyone else forgot, because retrieval requests still land on their desk.
One question unlocks most of these conversations: "If this workstation stopped booting tomorrow, what files would we lose that we cannot regenerate?" Departments answer that concretely, because they have privately worried about it for years.
What to record when you find something
An audit that ends in a hallway conversation is an audit that gets repeated next quarter. For every long-tail format you surface, log a consistent set of fields — a spreadsheet is enough:
| Field | Why it matters |
|---|---|
| Format and typical extensions | Your matching key against tooling capabilities later |
| Owning department and named data owner | Someone must be able to answer questions during migration |
| Volume and file count | Determines whether it is a pilot or a bulk job |
| Source system and storage location | Tells you what has to be disconnected at cutover |
| Sensitivity level | Patient-identifiable imaging and location data change the handling rules |
| Access frequency | Separates active data from the cold pile |
| Retention obligation | Regulatory and contractual lifespans constrain every downstream decision |
| Current reader and workstation | Single points of failure you are retiring |
The sensitivity row deserves emphasis. Medical imaging carries patient identifiers; geospatial data can reveal protected sites and infrastructure; laser scans of buildings are, in effect, security-relevant models. When these files move into a managed environment, sensitive-information detection should run before anything enters storage — SmartSlim's auto-detection of 13 categories of sensitive information, executed on-device before ingestion, is the pattern we recommend clients adopt for exactly this reason.
What to do once you find them
Discovery is the midpoint of the audit, not the end. The handling order that has worked for our clients:
- Confirm ownership. Every discovered format needs a named person who can define "still must open correctly" — without that, no later step can be validated.
- Sample and verify. Pull a representative handful of real files, run them through the intended processing, and confirm they read back intact. Never bulk-convert on the basis of a spec sheet.
- Pilot compression on your own hardware. Long-tail files are frequently enormous, which is what makes them painful in an air-gapped setting where cheap cloud overflow does not exist. Compression has to happen where the files already are. SmartSlim's recent release extended its on-device compression to exactly these industry categories — medical imaging (DICOM), scientific data (HDF5), geospatial layers (SHP), packaged exchanges (KMZ), and laser-scanning point clouds (LAS) — so a pilot no longer needs a separate tool or a separate procurement track. As a reference point for throughput on dense documents, a 443-page OFD file compresses end to end in 124 seconds; industry formats draw on the same local processing muscle. Current coverage is listed on the UglyPear Data products page.
- Fold the owners into the migration plan. The long tail graduates from footnote to line item once a pilot passes — departments that see their format open correctly after processing become your most motivated testers.
If the archive you uncover is also disorganized — duplicated scans, mystery folders, files nobody can find — pair the compression pass with retrieval tooling. MagiFiler handles conversational natural-language file operations, semantic search by content, and automatic classification and archiving, which turns the newly compressed archive into something people can actually query. More on combining the two in the UglyPear Data blog.
A one-week pace
Teams ask how long this should take. A realistic pace for a mid-sized organization: two days walking the hiding spots and extension tail, two days of department interviews, one day consolidating the log, and a day or two of sampling and verification on the top few formats. An imperfect audit that exists beats a thorough one that stays an intention, because the written version survives staff turnover and gets corrected as the migration proceeds.
Try it now
If the self-test at the top made you uneasy, start this week:
- SmartSlim product page — on-device compression, including the recent DICOM, HDF5, SHP, KMZ, and LAS industry-format coverage.
- UglyPear Data blog — migration field notes and format-handling updates.
- UglyPear Data home — the full private-deployment product line.
FAQ
Q: How much time does a long-tail format audit realistically take?
A: For a mid-sized organization, one focused week: two days on storage walks and extension tails, two on interviews, one consolidating, one or two verifying samples.
Q: A department insists they have no special formats. Should I push back?
A: Verify gently with the extension-tail method — sort the server inventory by file extension and show them the list. Departments genuinely forget formats their systems produce automatically, such as export folders that quietly accumulate imaging or GIS output.
Q: We found a format nobody recognizes. What now?
A: Log it with whatever metadata you can gather, then trace it to its source system or vendor. Do not delete it and do not convert it blind — an unidentified format is, by definition, one you cannot yet validate a round trip for.
Q: Should the audit run before or after we choose migration tooling?
A: Before, at least in outline. Knowing which long-tail formats exist changes your tooling requirements entirely — format coverage is far easier to check against a real list than to retrofit after selection.
Q: Can we just leave the cold archive on a legacy machine and migrate only active data?
A: You can, but you are renting the risk instead of retiring it. The legacy machine remains a single point of failure — one failed disk or one retired specialist away from losing a data category — and the 60–70% of storage sitting untouched keeps costing you either way.