Enforce Batches (whole run, not single jobs) to wait for each other











up vote
3
down vote

favorite
2












A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.



If the batch work on the same record we sometime see locking or currupt data.



Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?










share|improve this question
























  • We can run next batch from first batch finish method. So it will execute in sequence.
    – SFDC Learner
    2 hours ago












  • Batch chaining only works when the first batch knows the second batch. This is not the case.
    – Robert Sösemann
    2 hours ago






  • 3




    I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
    – Aidan
    2 hours ago






  • 1




    Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
    – Pranay Jaiswal
    2 hours ago








  • 1




    @PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
    – Robert Sösemann
    2 hours ago















up vote
3
down vote

favorite
2












A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.



If the batch work on the same record we sometime see locking or currupt data.



Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?










share|improve this question
























  • We can run next batch from first batch finish method. So it will execute in sequence.
    – SFDC Learner
    2 hours ago












  • Batch chaining only works when the first batch knows the second batch. This is not the case.
    – Robert Sösemann
    2 hours ago






  • 3




    I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
    – Aidan
    2 hours ago






  • 1




    Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
    – Pranay Jaiswal
    2 hours ago








  • 1




    @PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
    – Robert Sösemann
    2 hours ago













up vote
3
down vote

favorite
2









up vote
3
down vote

favorite
2






2





A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.



If the batch work on the same record we sometime see locking or currupt data.



Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?










share|improve this question















A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.



If the batch work on the same record we sometime see locking or currupt data.



Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?







apex batch asynchronous apex-flex-queue






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago

























asked 3 hours ago









Robert Sösemann

12.6k1074209




12.6k1074209












  • We can run next batch from first batch finish method. So it will execute in sequence.
    – SFDC Learner
    2 hours ago












  • Batch chaining only works when the first batch knows the second batch. This is not the case.
    – Robert Sösemann
    2 hours ago






  • 3




    I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
    – Aidan
    2 hours ago






  • 1




    Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
    – Pranay Jaiswal
    2 hours ago








  • 1




    @PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
    – Robert Sösemann
    2 hours ago


















  • We can run next batch from first batch finish method. So it will execute in sequence.
    – SFDC Learner
    2 hours ago












  • Batch chaining only works when the first batch knows the second batch. This is not the case.
    – Robert Sösemann
    2 hours ago






  • 3




    I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
    – Aidan
    2 hours ago






  • 1




    Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
    – Pranay Jaiswal
    2 hours ago








  • 1




    @PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
    – Robert Sösemann
    2 hours ago
















We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago






We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago














Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago




Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago




3




3




I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago




I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago




1




1




Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago






Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago






1




1




@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago




@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago










2 Answers
2






active

oldest

votes

















up vote
2
down vote













Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.






share|improve this answer




























    up vote
    1
    down vote













    If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.



    Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.



    public static void addBatchIfNotRunning(Id apexBatchClassId){

    ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
    boolean isNeedToInitiateNewBatch = true;
    if(!asyncJobList.isEmpty()){
    for(ApexAsyncJob as : asyncJobList){
    if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
    System.abortJob(as.Id);

    }else{
    //Batch is already queued or in Holding stage so it will pick up new records
    isNeedToInitiateNewBatch = false;
    }

    }
    }
    if(isNeedToInitiateNewBatch){
    //Initiate Batch
    }

    }





    share|improve this answer





















      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "459"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f242299%2fenforce-batches-whole-run-not-single-jobs-to-wait-for-each-other%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      2
      down vote













      Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.






      share|improve this answer

























        up vote
        2
        down vote













        Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.






        share|improve this answer























          up vote
          2
          down vote










          up vote
          2
          down vote









          Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.






          share|improve this answer












          Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered 2 hours ago









          sfdcfox

          244k10185418




          244k10185418
























              up vote
              1
              down vote













              If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.



              Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.



              public static void addBatchIfNotRunning(Id apexBatchClassId){

              ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
              boolean isNeedToInitiateNewBatch = true;
              if(!asyncJobList.isEmpty()){
              for(ApexAsyncJob as : asyncJobList){
              if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
              System.abortJob(as.Id);

              }else{
              //Batch is already queued or in Holding stage so it will pick up new records
              isNeedToInitiateNewBatch = false;
              }

              }
              }
              if(isNeedToInitiateNewBatch){
              //Initiate Batch
              }

              }





              share|improve this answer

























                up vote
                1
                down vote













                If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.



                Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.



                public static void addBatchIfNotRunning(Id apexBatchClassId){

                ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
                boolean isNeedToInitiateNewBatch = true;
                if(!asyncJobList.isEmpty()){
                for(ApexAsyncJob as : asyncJobList){
                if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
                System.abortJob(as.Id);

                }else{
                //Batch is already queued or in Holding stage so it will pick up new records
                isNeedToInitiateNewBatch = false;
                }

                }
                }
                if(isNeedToInitiateNewBatch){
                //Initiate Batch
                }

                }





                share|improve this answer























                  up vote
                  1
                  down vote










                  up vote
                  1
                  down vote









                  If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.



                  Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.



                  public static void addBatchIfNotRunning(Id apexBatchClassId){

                  ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
                  boolean isNeedToInitiateNewBatch = true;
                  if(!asyncJobList.isEmpty()){
                  for(ApexAsyncJob as : asyncJobList){
                  if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
                  System.abortJob(as.Id);

                  }else{
                  //Batch is already queued or in Holding stage so it will pick up new records
                  isNeedToInitiateNewBatch = false;
                  }

                  }
                  }
                  if(isNeedToInitiateNewBatch){
                  //Initiate Batch
                  }

                  }





                  share|improve this answer












                  If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.



                  Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.



                  public static void addBatchIfNotRunning(Id apexBatchClassId){

                  ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
                  boolean isNeedToInitiateNewBatch = true;
                  if(!asyncJobList.isEmpty()){
                  for(ApexAsyncJob as : asyncJobList){
                  if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
                  System.abortJob(as.Id);

                  }else{
                  //Batch is already queued or in Holding stage so it will pick up new records
                  isNeedToInitiateNewBatch = false;
                  }

                  }
                  }
                  if(isNeedToInitiateNewBatch){
                  //Initiate Batch
                  }

                  }






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered 1 hour ago









                  Pranay Jaiswal

                  12.6k32251




                  12.6k32251






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Salesforce Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.





                      Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                      Please pay close attention to the following guidance:


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f242299%2fenforce-batches-whole-run-not-single-jobs-to-wait-for-each-other%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Trompette piccolo

                      Slow SSRS Report in dynamic grouping and multiple parameters

                      Simon Yates (cyclisme)