Introduction to Jobrunr

In a recent project we have successfully replaced some of our usage of ActiveMQ queues with instances of Jobrunr. Lets take a look at what Jobrunr is and why it is attractive.

Using asynchronous processing is a common theme in distributed systems - maybe the client doesn’t need to know whether something was a success or some resource intensive processing takes a while to finish.

Of course, you can use ActiveMQ to push a job for later processing to a queue, but this incurs a cost. ActiveMQ needs to be setup and more importantly it needs to be maintained. Perhaps your organization already has ActiveMQ setup, in which case this may not be a problem for you - or you may want to simplify the external dependencies of your application.

However a big advantage of using Jobrunr is, that it only requires a Storage-Backend - and it quiet likely supports the backend you already use.

Example code

The Jobrunr documentation gives both a quick introduction as well as more in-depth knowledge. However, another great resource to get going with Jobrunr is the java-mag-example.

This example project uses the Spring Boot integration - but please be aware that Jobrunr also has Quarkus and Micronaut integration. And you can also always use Jobrunr with just plain java.

The following code snippet describes a very simple job taking just a string as input.

@Job(name = "The sample job with variable %0", retries = 2)
public void executeSampleJob(String input) throws InterruptedException {
    logger.info("The sample job has begun. The variable you passed is {}", input);
    try {
        Thread.sleep(1500);
    } catch (InterruptedException e) {
        logger.error("Error while executing sample job", e);
        throw e;
    } finally {
        logger.info("Sample job has finished...");
    }
}

From GitHub Jobrunr java mag example - SampleJobService.java.

This specific job can then be scheduled or enqueued using the JobScheduler like so:

final JobId enqueuedJobId = jobScheduler.enqueue(
    () -> sampleService.executeSampleJob("Hello " + name)
);

or scheduled like this

final JobId scheduledJobId = jobScheduler.schedule(
    now().plus(Duration.parse(when)),
    () -> sampleService.executeSampleJob("Hello " + name)
);

Both snippets from GitHub Jobrunr example - JobController.java.

Example setup

This already looked quiet easy, but how difficult is it to setup. Of course for AMQ we need to setup a JMS or AMQP connection which has many knobs that can be tweaked.

For spring boot the setup can be done exclusively using configuration. Adding a single dependency to your project

<dependency>
    <groupId>org.jobrunr</groupId>
    <artifactId>jobrunr-spring-boot-3-starter</artifactId>
    <version>${jobrunr.version}</version>
</dependency>

and a few configuration variable to point Jobrunr at the database it should use:

org.jobrunr.background-job-server.enabled=true
org.jobrunr.dashboard.enabled=true

Jobrunr will pick up your default spring data-source and use that:

spring.datasource.url=jdbc:h2:mem:jobrunr;DB_CLOSE_DELAY=-1
spring.datasource.driverClassName=org.h2.Driver

Alternatively you can setup your own data-source and give it to Jobrunr manually - see this other example.

Dashboard

Jobrunr includes a dashboard, which will let you see the status of ongoing job processing.

Jobrunr dashboard
Jobrunr dashboard

By default the dashboard is available on port 8000 of you application, but this is configurable.

By default succeeded jobs will be deleted after a timeout while failed jobs will be automatically (and configurable) retried. After a default of 10 retries failing jobs will be put into the failed state (and never deleted) - awaiting either manual deletion or re-queuing.

Architecture

Jobrunr will start one BackgroundJobServer instance per application plus a configurable number of worker threads, that will pick jobs from the queue.

Jobrunr architecture
Jobrunr architecture

If you start multiple instances of your application, as is common today, Jobrunr will choose a master among the available BackgroundJobServers - making it extremely simple to scale up job processing.

Support and enterprise edition

Jobrunr is available as open source under the LGPL or alternatively there is a Pro version available including support by the original developer of Jobrunr.

This was a quick intro to Jobrunr for those that didn’t know about it before. Let me know if you need help in deciding if Jobrunr is the right tool for your problem or if you need help in rolling our Jobrunr.

Thilo Bangert

Senior Systems Consultant at Redpill Linpro

Thilo is a systems engineer working with integration and helping customers apply DevOps methodologies. He is hailing from the Copenhagen office and can be contacted by mail using thilo at redpill-linpro dot com.

Ansible-runner

The command ansible-runner is part of the Ansible automation platform. If you have got installed Ansible, then you probably have already installed ansible-runner as well.

But what do you use it for? Well, if you run AWX or the Ansible Automation platform package somewhere in your environment, ansible-runner is part of the magic in the background and running your code. It is also a python library that can connect your code directly to Ansible ... [continue reading]

Portable Java shell scripts with Java 21

Published on February 21, 2024

Git worktrees

Published on January 31, 2024