Data import tools

I almost always use a custom tool that pumps data in via OData. This works well in most cases, but can be slow when you're dealing with extremely large quantities of data. My tool uses a local SQL database that I'll load all the data into (or use the customer's SQL database itself). It uses views to transform the data locally and alias all columns to match the columns in Creatio, then the tool just loads the records in as based on the views since the views transform the data to look identical to the Creatio tables. It stores all original Id values and created Creatio Id values in a table so it can run continuously doing both inserts and updates as needed. I've been happy with this method, but it's not ideal for all scenarios. It does thread out several requests at once, but ultimately it's still doing a record by record load via the API.

I've used Starfish ETL a few times, which is doing the same (moving in one record at a time via the API). It's easy to use, but the price can become unreasonable since they're charging per-record. 

I've done a few initial loads by getting a backup of the database from support, then bulk load all the data into the database locally, then providing a backup back to support to put back into the cloud. This worked great for large sets of data since you can move in millions of rows in minutes, but it does create a lot of extra work/steps so I don't necessarily like to go this route.

I'd love to hear what others use as well.

Ryan

Hi,

I've created this one: https://marketplace.creatio.com/app/data-management-scheduler

But as Ryan said, it's per line, not bulk (at least for now). So it's more fitted ffor regular file injection.

Hi,

I would to add that DSM is used by several customer now who are very happy. It works both on SQLServer and PostGres and Export and Import can be schedule using Processes.

So, do not hesitate to try it.

Stéphane.