Importing Listings From CSV: Six Checks Before You Publish

Most import guides stop at "map your columns and press Import." That step takes ninety seconds and is not where anything goes wrong. What goes wrong is the second run of the same file, the row whose city is empty, the category slug that does not exist, and the geocoding bill nobody modelled.

This picks up where the data-gathering workflow ends. You have a file: names, URLs, addresses, maybe descriptions, every row verified as genuinely belonging in your niche. It is clean by content. It is still not a catalogue, and the distance between those two states is six specific failures.

Check 1: The file does not have the shape you think it has

CSV has no standard. RFC 4180 is classified Informational — it documents what programs already did in 2005 rather than defining what they must do. It says fields containing commas, line breaks or double quotes should be wrapped in double quotes, that an embedded double quote is escaped by doubling it, and that each line should carry the same number of fields. "Should," not "must," and every exporter interprets that differently.

Three consequences you will hit on a real file:

Row counts disagree. A business description containing a quoted line break is one record to a compliant parser and three rows to a naive split('\n'). If your importer reports 4,812 rows and your spreadsheet says 4,796, one of them is wrong and it matters which.

Field counts drift. An address with an unescaped comma silently shifts every column after it by one. The row imports without error and the postcode lands in the country field.

Encoding is invisible until it renders. A file exported as UTF-8 with a byte-order mark gives your first header a prefix you cannot see, so name never matches name and the whole column maps to nothing. Latin-1 exports turn every accented business name into mojibake that looks fine in the terminal and wrong on the page.

The check: before mapping a single column, parse the file with a real CSV library, assert that every record has exactly the header's field count, and print the first and last three records in full. Any row that fails the field-count assertion goes into a rejects file, not into a "best effort" import. A rejects file you read is worth more than an import that reports 100% success.

Check 2: Running the import twice must not double the catalogue

This is the one that costs a weekend. The first run works. Someone re-runs it because forty rows failed, and now you have 4,772 duplicates and 40 fixes mixed together in a table with no way to tell them apart.

Idempotency needs a natural key — some column combination that identifies the same real-world business across runs. INSERT ... ON CONFLICT (natural_key) DO UPDATE then makes re-running safe by construction.

Choosing that key is where directories differ from ordinary datasets. An id column from the source export is only stable if the source is stable. A website domain is a good key for software directories and a poor one for local businesses, where a franchise's twelve locations share one domain. For local data, (google_place_id) is the strongest available key — and it is also the one field Google explicitly permits you to keep, which the next-but-one check covers.

Whatever you pick, know this about the database underneath it:

By default, two null values are not considered equal in this comparison. That means even in the presence of a unique constraint it is possible to store duplicate rows that contain a null value in at least one of the constrained columns.

That is the PostgreSQL documentation on unique constraints. A unique index on (name, city) does exactly nothing for the 300 rows where city is empty — they all insert, every run, forever. Postgres 15 and later let you say UNIQUE NULLS NOT DISTINCT to change that, but the better fix is upstream: make the natural-key columns NOT NULL, and route rows that cannot supply them to the rejects file.

Check 3: Slugs and categories fail quietly

Two columns cause more silent damage than the rest combined, because neither one throws.

Slugs. They are your URLs, and two businesses called "City Dental" in different towns collide. Whatever your collision strategy — append the city, append a counter — decide it before the first import, because the alternative is renaming slugs in production. Directory URLs are the asset; a rename breaks every backlink and every internal link pointing at that listing, and the only honest repair is a 301 for each one.

Categories. In DirectoryLaunch, a listing's categories live as a JSONB array of category slugs on the row itself rather than in a join table (Categories). That is fast and flexible, and it means a category slug that does not exist produces no foreign-key error. The listing imports. It is simply in no category, invisible to category browsing, and you will notice in three weeks when a filter looks empty.

The check: before import, take the distinct set of category values in your file, diff it against the slugs that actually exist, and print the difference. It is a five-line script and it catches a class of bug that has no error message.

Check 4: Geocoding is a bill, and its size is knowable in advance

Addresses in a CSV are text. Maps, distance sorting and city pages need coordinates, so somewhere between the file and the catalogue every row gets geocoded — and each request is billed individually.

Google's price list puts the Geocoding SKU at a 10,000-request free cap per month, $5.00 per 1,000 from there to 100,000, and $4.00 per 1,000 from 100,001 to 500,000 (checked 5 September 2026; the page itself is dated 1 September 2026). That makes the arithmetic for a one-off import simple:

The last row is two tiers: 90,000 events at $5.00 is $450.00, the remaining 150,000 at $4.00 is $600.00. Note that tiers are calculated on the whole billing account's monthly usage across projects, so an import does not get its own fresh free cap if your live site is already geocoding. The 3,000 queries-per-minute figure is Google's stated rate limit for geocoding, counted as the sum of client-side and server-side queries. Budget from the list price rather than any promotional tier, and cap your daily quota in the Cloud console before the first run — a retry loop that re-geocodes the whole file is a four-figure mistake that takes twenty minutes to make.

The same page makes a point worth repeating, because it argues against the instinct to geocode everything in the browser: server-side geocoding is what you want "when you get a dataset that comes independently of user input, for instance if you have a fixed, finite, and known set of addresses that need geocoding." A CSV import is that case exactly.

Check 5: What you may keep, and what you must credit

This is the check that separates a catalogue you can operate from one you will quietly have to gut.

If your rows came from Google, the caching rules are not symmetric across products. The Places API policy is blunt: you must not pre-fetch, cache or store Places API content beyond the allowed exceptions — and the place ID is the exception, which "you can therefore store place ID values indefinitely." Names, opening hours, ratings and reviews are not in that carve-out. That is why an importer built on Places has to keep refreshing rather than treating the first pull as a permanent record, and why our own Places importer ships a daily refresh job instead of a one-time load.

Geocoding is treated differently. The Maps Platform Service Specific Terms permit customers to "indefinitely cache latitude (lat), longitude (lng), formatted_address, and the structured address values from the Geocoding API" — with a condition attached, "solely to support the direct, End User facing functionality of the Customer Application that initiated the request." Coordinates for showing your own listings on your own map: fine. A coordinates dataset you then sell or syndicate: not what that clause says.

Attribution is a build requirement, not a footer decoration. Displaying Places content obliges you to show Google Maps attribution, to credit the author of every photo and review, to give users a link to the source review on Google Maps, and to state how reviews are ordered. If you show Google's AI-powered summaries you must display the disclosure text the API returns, unmodified. Those are rows in your schema — an author name, an avatar URL, a googleMapsUri, a flagContentUri — and if the importer drops them because they were not in the CSV, you cannot add the attribution later without re-fetching.

Check 6: Imported is not the same as publishable

Five thousand rows landing in a table is not five thousand pages worth publishing. Two constraints bite immediately.

The first is thin pages. A city-plus-category page holding two listings is a page about nothing, and at scale a few thousand of them are the pattern Google's spam policies describe when they talk about content generated primarily to manipulate rankings rather than to help people. We handle it with a threshold rather than a judgement call: minListingsToIndex: 3 means combinations below the bar still render but emit noindex, so a thin page exists for the visitor who lands on it and stays out of the index (Local SEO). Note the limit of that fix — noindex keeps thin pages out of the index, it does not stop them being crawled, which is a separate problem at scale.

The second is descriptions. An import from a scrape leaves the description column empty, and an empty catalogue reads as broken. Our admin ships an AI enrichment pass that scrapes each row's website, writes a description and extracts selling points, returning an enhanced CSV to feed back into the importer. It is genuinely the fastest way to fill that column — and it is also the fastest way to publish five thousand pages of near-identical machine text. The honest framing is the one in our own docs: AI generation is a tool against the blank field in a form, not a page factory. Run it in test mode on a handful of rows, read the output, and if the descriptions are interchangeable across listings, the problem is the prompt, not the volume.

The one that bites on the way out: exporting

Every importer eventually grows an export button, and that is where a directory hands its own users a loaded file. OWASP's CSV Injection entry explains why: when a spreadsheet opens a CSV, "any cells starting with = will be interpreted by the software as a formula." A listing whose submitted business name begins with = becomes an executable cell in the file your admin opens on their laptop.

The mitigation is genuinely awkward, and OWASP says so: Excel may strip quotes and escapes on save-and-reopen, so previously escaped formulas can become active again, and "there is no universal CSV sanitization strategy that is safe for all spreadsheet applications and all downstream consumers." The dangerous set is wider than = alone — it includes +, -, @, tab, carriage return and line feed, plus full-width variants that behave as formula starters in some locales. Prefixing such cells with a tab inside the quoted field is the mitigation OWASP describes as reliable in Excel, with the caveat that the tab stays in the data and will confuse anything that re-imports the file programmatically.

The practical answer for a directory: sanitise on export, and if the file is for machines rather than humans, export JSON or XLSX and skip the problem entirely.

When a CSV import is the wrong tool

It is worth saying where this whole pipeline is overkill or simply wrong.

Under a few hundred listings, type them. The import machinery costs a day of engineering and pays back on volume. At 150 listings, a human entering rows produces better data and catches nonsense the parser cannot.

If your source is a live API, do not round-trip through CSV. Exporting Places results to a file and importing them back adds a stale snapshot in the middle and drops the fields the attribution rules require. Import from the API and keep the refresh path intact.

If listings are transactional, this is not your bottleneck. When users pay each other on your platform, catalogue seeding is the easy part next to payouts and disputes, and a marketplace product like Sharetribe is the more honest starting point — a point we made in more detail when auditing five live marketplaces by how they take money.

If nobody on the team writes code, a CSV pipeline will not save you. Column mapping, rejects files and natural keys are engineering work whichever tool wraps them. A WordPress directory plugin with a hosted importer asks less of you, and that is a real advantage.

What to do tomorrow morning

Before you touch the import screen, do these four things in order.

1. Parse and assert. Run your file through a CSV library, assert the field count on every record, and write a rejects file. Fix the file, not the importer.
2. Name the natural key out loud. Write down which columns identify the same business across runs, make them NOT NULL, and add the unique constraint. Then import the same file twice and check the row count did not move.
3. Diff your category slugs and dry-run your slug generator. Two scripts, ten minutes, and they catch the failures that never raise an error.
4. Model the geocoding bill and set the cap. Rows minus 10,000, divided by 1,000, times $5.00. Then set the daily quota limit in the Cloud console to roughly that number before the first run.

Then import 100 rows. Look at them on the live site — the card, the detail page, the map pin, the category page. Only then run the other 4,700.