Enforce Batches (whole run, not single jobs) to wait for each other
up vote
3
down vote
favorite
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
|
show 1 more comment
up vote
3
down vote
favorite
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
3
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago
|
show 1 more comment
up vote
3
down vote
favorite
up vote
3
down vote
favorite
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
apex batch asynchronous apex-flex-queue
edited 1 hour ago
asked 3 hours ago
Robert Sösemann
12.6k1074209
12.6k1074209
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
3
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago
|
show 1 more comment
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
3
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
3
3
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
1
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
1
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago
|
show 1 more comment
2 Answers
2
active
oldest
votes
up vote
2
down vote
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
add a comment |
up vote
1
down vote
If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId){
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty()){
for(ApexAsyncJob as : asyncJobList){
if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
}else{
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
}
}
}
if(isNeedToInitiateNewBatch){
//Initiate Batch
}
}
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
add a comment |
up vote
2
down vote
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
add a comment |
up vote
2
down vote
up vote
2
down vote
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered 2 hours ago
sfdcfox
244k10185418
244k10185418
add a comment |
add a comment |
up vote
1
down vote
If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId){
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty()){
for(ApexAsyncJob as : asyncJobList){
if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
}else{
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
}
}
}
if(isNeedToInitiateNewBatch){
//Initiate Batch
}
}
add a comment |
up vote
1
down vote
If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId){
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty()){
for(ApexAsyncJob as : asyncJobList){
if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
}else{
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
}
}
}
if(isNeedToInitiateNewBatch){
//Initiate Batch
}
}
add a comment |
up vote
1
down vote
up vote
1
down vote
If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId){
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty()){
for(ApexAsyncJob as : asyncJobList){
if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
}else{
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
}
}
}
if(isNeedToInitiateNewBatch){
//Initiate Batch
}
}
If the batch querry is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we dont need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId){
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty()){
for(ApexAsyncJob as : asyncJobList){
if(as.Status == ''Processing' || as.Status == 'Preparing'){ //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
}else{
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
}
}
}
if(isNeedToInitiateNewBatch){
//Initiate Batch
}
}
answered 1 hour ago
Pranay Jaiswal
12.6k32251
12.6k32251
add a comment |
add a comment |
Thanks for contributing an answer to Salesforce Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f242299%2fenforce-batches-whole-run-not-single-jobs-to-wait-for-each-other%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
2 hours ago
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
2 hours ago
3
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
2 hours ago
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
2 hours ago
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
2 hours ago