Daylite to Pipedrive CRM Migration
The audit-first, resumable, rollback-capable migration that unlocked everything downstream.
PJL ran on Daylite, a macOS desktop CRM with no useful automation surface and no webhooks. Nothing else in this portfolio was possible until the roughly 10,900 contacts, companies, opportunities, notes and tasks were in Pipedrive with their relationships intact. This is the ETL that did it, plus the calendar migration into Microsoft 365 that went with it.
Approach
The migration was designed like someone who has done one before. The first step is an audit that connects to Daylite, counts every entity, discovers custom fields and flags contacts with no organisation or empty required fields. Only then are the field map and pipeline-stage map written, as YAML rather than code, so iterations do not touch Python.
The migrator runs organisations, then people, then deals, then notes, then activities, each depending on the IDs from the prior step. The ID map is persisted after every single record, so an interrupted run resumes where it stopped. A dry-run mode validates without writing, and a single-step flag re-runs one phase.
The rollback tool
Writing ten thousand records into a production CRM with no way to undo it is how you destroy a client's data. The cleaner deletes everything in the ID map in reverse dependency order and resets the map, with its own dry run. It is driven by the same map that drove creation, which is the correct design.
The pivot
The logs record the API path failing at scale on deal after deal with a date validation error from Pipedrive's v2 API, each wrapped in retry back-off. Three days later a FastAPI web UI appeared that exports Daylite data into Pipedrive-shaped import spreadsheets with background jobs, per-job log buffers and a download endpoint. The pragmatic response was to stop fighting the API and hand Pipedrive's own bulk importer a correctly shaped file. That judgement call is the most useful thing in the project.
Rate limits and honesty
Daylite allows five thousand requests an hour and Pipedrive roughly a hundred per ten seconds. Both are documented and handled with exponential back-off. The project guide also has a section titled "What needs iteration" that admits which assumptions about the Daylite API were unverified, which is exactly why the audit runs first.
The calendar half
A companion script parsed the Daylite calendar export, normalised all-day dates and timezone-aware datetimes into Microsoft Graph's shape, authenticated with app-only credentials, and wrote events through the Graph batch endpoint twenty at a time.