Open source tool to sync data from external APIs to your local DB
Hey IndieHackers,
I built a lot of native integrations for my last (SaaS) product and kept running into the same problem over and over again:
Syncing data from an external (REST) API endpoint to my local database
- Syncing in CRM contacts/deals/accounts/comments
- Syncing in payment transactions
- Syncing in tweets, mentions & replies
- Syncing in orders & transactions from e-commerce systems
Looked easy at first but then things kept breaking every week and distracting me from building actually valuable things... 😭
I felt like there had to be a better way
So I decided to scratch my own itch and started Nango: It's an open-source tool that syncs data from any REST API endpoint to my DB for me with a single line of code:
Nango.sync('https://any.rest.api/any/endpoint');
So far Nango handles:
- Initial sync & pagination
- Incremental syncs (you pick the refresh-period)
- Deduplication, upserts, change detection
- Rate-limits & retries
- Copying all the data to a local postgres DB
You can find all the details & a quickstart (sync data in <3mins) here: https://github.com/NangoHQ/nango
Would love to hear your thoughts & feedback :)
I've been experimenting with Nango the past week or so and it's looking very promising.
It solves a similar problem to Zapier, n8n, Make/Integromat, but with one key difference which isn't immediately obvious on their GitHub:
Zapier, n8n, etc. are designed for getting your data from third party services into your own database.
Nango is designed for getting YOUR CUSTOMERS data from third party services into your database. Nango (or rather its sister project Pizzly) let's YOUR CUSTOMERS log into each of THEIR third party services and pull THEIR data down to YOUR database using a single API.
This is huge for Indie devs who need to integrate third party apps on behalf of their users.
Thanks for the comment and the feedback Paul!
Will see that we can make the Readme clearer in this regard 👍