ChargeDisha’s map is only useful if it’s more complete than any single source, so it ingests from six: Open Charge Map plus the operator feeds for Tata Power, Ather, ChargeZone, IOCL and BPCL. That immediately creates the classic problem — the same physical charger shows up in three feeds with three names, three coordinate readings, and zero shared identifiers.
The textbook answer is entity resolution: candidate matching, similarity scoring, maybe a model. I didn’t do any of that. For this domain there’s a much cheaper observation: two records are the same station if they’re the same operator at the same place. Both of those can be normalized into a key.
Deterministic IDs
Every incoming record gets an ID computed from its content:
station_id = f(round(lat, 4), round(lng, 4), normalize(operator))Rounding coordinates to four decimal places snaps them to a grid of roughly 11 meters. GPS readings for the same physical station differ across feeds, but rarely by that much — and two distinct stations of the same operator within 11m of each other basically don’t exist. The operator name goes through a normalization table that maps the spelling chaos (“TATA Power EZ Charge”, “Tata Power”, “tatapower”) onto one canonical slug.
Same station in three feeds → same key three times → one document. Deduplication stops being a matching problem and becomes a property of the primary key.
Why deterministic beats clever
The real win isn’t the dedup, it’s idempotency. Ingestion runs repeatedly, and any run can crash halfway. Because IDs are content-derived, a re-run just overwrites the same documents — no duplicate detection pass, no cleanup jobs, no “did this batch already commit?” bookkeeping. Writes go through Firestore batch operations (which cap out around 400 writes per batch, so the pipeline chunks accordingly), and a failed batch can simply be retried whole.
There’s a subtle failure mode worth naming: rounding creates hard grid boundaries, so two readings of one station can land in adjacent cells if the true position sits near a boundary. In practice the operator feeds are consistent enough that this is rare, and the cost of a rare duplicate pin is far lower than the cost of a fuzzy-matching pipeline I’d have to babysit.
The takeaway
When you control the key function, you can often turn a data-quality problem into a data-modeling decision. I’d reach for real entity resolution if I were merging, say, restaurant listings with user-typed addresses. For infrastructure with fixed coordinates and a small closed set of operators, round + normalize does 99% of the job with 1% of the machinery — and the remaining 1% gets fixed by users tapping “report a problem” on the map.