Power Apps Exchange

Please login or click SIGN UP FOR FREE to create your PowerAppsUG account to join this user group.
 View Only
  • 1.  Using patch on high volume of records

    Posted Jan 17, 2022 03:18 PM
    I have a payroll time entry app that has become a victim of its own success. So far all aspects of it are doing well, but I'm starting to see the limits when it comes to the last step in the process which is when the Power App must do two things, prepare a JSON for sending to Power Automate to post into the ERP system for payroll, and then marking those records as "posted" in Dataverse.

    That first step is actually being processed in a more or less timely manner and I'm not too worried about it, however the second step is killing me. Initially it was working fine, but as the record count gets higher and higher this is taking an extremely long time. We are hitting about 3,000 individual records a week now.

    All the records are in a table in Dataverse. Once they are sent over to the ERP I need to update three columns, the status, the user doing the update, and a timedate stamp. I do that with this:

    UpdateIf(colTimecardsToPost,true,{ExportStatus:"Posted",RecordPostedUser:varMyUserRecord,RecordPostedDate:Now()});

    This works, no issue. The colTimecardsToPost collection contains all the records that need to be posted (I do this in a loop due to the 2k limit).

    Next I'm doing these two things to update the Dataverse source (Timecards). The columns here are all the columns marked as required on the table. Not sure if this is a contributor and I could remove the required status as I'm handling all that in the app, just was how the original table got set up.

    ClearCollect(colUpdateTimecards,ShowColumns(colTimecardsToPost,"cr480_bulkupdatestatus","cr480_recordid","cr480_exportstatus","cr480_pay_type","cr480_timecardsid","cr480_costcenterrecordid","cr480_company_code","cr480_work_date","cr480_RecordApprovedUser","cr480_recordapproveddate"));

    Patch(Timecards,colUpdateTimecards);

    This eventually works, but obviously I don't have any kind of counter or feedback I can give the user on how far along it is and it is taking 15+ minutes to complete. The client would like to extend this timecard app to several new sites which will quadruple the overall weekly record count so this is going to become a much bigger issue.

    All the research I have done is that this is the most efficient method, but it is not going to scale to the client's requirements. Just trying to get feedback that if a) this is just what it takes to work with that record count and nothing more can be done, or b) I'm doing something wrong or could do something better to speed it up. Welcome any feedback!

    ------------------------------
    Trae Reinart
    Sr Consultant
    Houston TX
    ------------------------------


  • 2.  RE: Using patch on high volume of records

    Posted Jan 19, 2022 09:06 AM
    Hi Trae,

    Just curious about the Patch() of the colUpdateTimcards collection. I have found using Collect() directly to the destination when trying to Post a collection runs much quicker. I'm assuming Collect() only runs one request as opposed to looping and Patching each record.

    If you try Collect(Timecards,colUpdateTimecards); You may find a performance improvement. Let me know how it goes.

    Anthony.

    ------------------------------
    Anthony Scanlon
    ------------------------------



  • 3.  RE: Using patch on high volume of records

    Posted Jan 19, 2022 01:18 PM
    Hi Anthony-

    Thanks for the suggestion, I actually have a place where that might be helpful, however in this circumstance I'm not adding to the list, I'm just updating three fields of existing records in the list, thus the need for path. Essentially I'm marking these timecards as "Exported" so they are filtered out of the list of pending time entries.


    Cheers,
    Trae

    ------------------------------
    Trae Reinart
    Sr Consultant
    Houston TX
    ------------------------------