I am receiving an error trying to Insert to the AccountFile object. I am able to read from AccountFile just fine and see the file's binary in the 'Data' column. I see that when writing to the 'Data' column, the Terrasoft components automatically converts the data to base64-encoded data. Below is my request payload (with the base64 removed):
{ "QueryKind" : 0, "ColumnValues" : { "Items" : { "Name" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 1, "Value" : "MWOLJGrsPrHCT.pdf", "ShouldSkipConvertion" : false } }, "CreatedOn" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 7, "Value" : "\"2011-11-03T14:57:07\"", "ShouldSkipConvertion" : false } }, "ModifiedOn" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 7, "Value" : "\"2011-11-03T14:57:07\"", "ShouldSkipConvertion" : false } }, "Notes" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 1, "Value" : "MWOLJGrsPrHCT", "ShouldSkipConvertion" : false } }, "Data" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 13, "Value" : "BIG CHUNK OF BASE64 ENCODED DATA", "ShouldSkipConvertion" : false } }, "Type" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 10, "Value" : "529bc2f8-0ee0-df11-971b-001d60e938c6", "ShouldSkipConvertion" : false } }, "Account" : { "ExpressionType" : 2, "Parameter" : { "DataValueType" : 10, "Value" : "c844dc2f-8acd-4dd8-a9e0-fe1a5a943188", "ShouldSkipConvertion" : false } } } }, "RootSchemaName" : "AccountFile", "OperationType" : 0 }
The response I receive back does not give much inidication of what went wrong:
{ "success" : false, "responseStatus" : { "ErrorCode" : "NullReferenceException", "Message" : "Object reference not set to an instance of an object.", "Errors" : [] }, "rowsAffected" : -1, "nextPrcElReady" : false }
Can I get some pointers on how to upload files using DataService?
Like
Dear Aron,
Unfortunately, we do not have any examples of how to upload files using dataservice.
We only have examples of how to do that via O'Data: https://community.bpmonline.com/questions/how-upload-attachments-odata
Maybe this will help you to achieve your goal.
Best regards,
Lily
Lily Johnson,
Thanks for getting back. I am familiar with using the OData approach and have used it extensively. However I am in the process of porting our code over to the DataService because it is a much faster API. Could someone on your team please provide an example? This is my last hurdle.
Can someone in support please point me in the right direction for the DataService? There's nothing in the documentation which covers handling files.
Unfortunately DataService doesn't support transferring fileStreams. Please consider using OData for transferring files. Additionally, please check how our services that upload files work. You can use them them too. For example, you can investigate how the "attachments and notes" detail uploads files and try to use the same service or write your own service according to your needs.
Eugene Podkovka,
That is unfortunate. I guess on that portion of my code I'll have to fall back on OData unless I can figure something else out. Thanks.
I recommend to use Data service if you consider it faster. But when you need to transfer a file just send a separate OData request.
Actually, it's very strange that you found DataService working faster. Did you you use the same authorization cookie and sessionId when you tested OData?
Eugene Podkovka,
Yes I've found it to be much faster - 2 or even 3 times faster in most cases - for both reading and writing. Yes I use the exact same cookie & session id/authorization routine for both API's. We've done many benchmark tests because we use it primarily for performing data migrations from other CRM systems. Why do you find that surprising?
DataService should work faster, but it strange that the difference is so big. Anyway, the advice is the same. Please use DataService for the integration and send an additional request via OData when you need to upload a file.
Let me explain my current situation. We are migrating a customer from a legacy CRM system and we have to migrate 130,000 files related to Accounts Contacts & Opportunities. We are doing this over OData and it is projected to take nearly 10 days to process. The overhead from OData not only slows this down but could put additional stress on your servers. As we will be focusing on doing many more of these types of migrations, I would urge BPM to see if there's a better way to handle large data sets such as a bulk mode API. I won't name names, but a competitor of yours has a very nice bulk API which allows uploading of attachments in a zip format. The zip's are then processed on the server which cuts down immensely on time & bandwidth usage.
I recommend taking the bpm'online application database and restoring it near the legacy CRM database. Then transfer the files via SQL scripts. Then send us the database and we'll restore the bpm'online application with the backup.
Eugene Podkovka,
Thanks that sounds like a great plan - however the client couldn't wait to go live. So they will be in the system for the next week+ while the attachments trickle in. Could I take this approach with a live system, send you the database and you could just move the attachments table over? I realise this is a bit more work.
Aron Hoekstra,
Please take our database and create the scripts that we need to apply. Then give us the database and the scripts. We’ll deploy your backup near the application dB and execute the scripts.