New version 0.13.0!

v0.13.0

Community

Releases

v0.13.0

🚨 Breaking changes

  • The context property of the Task singleton, which was accessible during task execution, has been removed due to its redundancy with other properties.

  • In workers,

    • the method registerService has been replaced by 2 methods registerServiceExecutorand registerServiceTagEngine.
    • the method registerWorkflow has been replaced by 3 methods registerWorkflowExecutor, registerWorkflowTagEngine, and registerWorkflowStateEngine.
  • The following libraries are no longer exposed by Infinitic. If you were using them, you must now add them to the dependencies of your project:

    • org.jetbrains.kotlinx:kotlinx-serialization-json
    • com.jayway.jsonpath:json-path
    • com.sksamuel.hoplite:hoplite-core

🚀 New features

  • CloudEvents (beta): Infinitic now expose its events in CloudEvents json format. This allow users to build their own dashboards, logs, or even add hooks to some specific events.

    Examples of events exposed are:

    • for methods of workflows: startMethod, cancelMethod, methodCanceled, methodcompleted, methodFailed, methodTimedOut
    • for tasks within workflows: taskDispatched, taskCompleted, taskFailed,taskCanceled, taskTimedOut
    • for workflows within workflows: remoteMethodDispatched,remoteMethodCompleted,remoteMethodFailed,remoteMethodCanceled,remoteMethodTimedOut.
    • for the workflow executor itself (also called WorkflowTask) executorDispatched, executorCompleted, executorFailed

    Each event is accompanied by relevant data, such as the error details for a taskFailed event or the arguments for a remoteMethodDispatched event.

    Note1: The events are generated by reading the actual events from Infinitic's topics, rather than being produced by interceptors in the workers. This approach is deliberately chosen to prevent any potential slowdown in the processing of workflows or the introduction of errors. Additionally, it facilitates the possibility of regenerating events if necessary. Most of the time, event generation is expected to be nearly real-time, as the processing of these events is typically less resource-intensive than the execution of tasks or workflows.

    Note2: the format of these events differs from Infinitic's internal format. This is intentional, as Infinitic may utilize internal data that is not pertinent for public API exposure. Additionally, while those events' format should be eventually stable, Infinitic's internal format may undergo changes to meet evolving needs without affecting backward compatibility with existing messages.

    This feature is currently in beta and may be refined based on user feedback.

  • Delegated Tasks: In certain cases, tasks cannot be processed directly by a worker, and instead, the task invokes another system for processing, typically through an HTTP call. If the external system can process the task synchronously and return the output (or report a failure), the process works smoothly. However, if the external system cannot provide a synchronous response, the situation becomes ambiguous, leaving Infinitic without a clear indication of whether the task has been completed or failed, nor an ability to retrieve the result. Starting with version 0.13.0, Infinitic introduces a "delegated task" feature. This feature, enabled through an annotation on the task, informs Infinitic that the method's completion does not signify the task's completion and that it should await asynchronous notification of the task's outcome. To support this functionality, a new completeDelegatedTask method has been added to the InfiniticClient.

  • InfiniticWorker now offers new methods that allow for the programmatic registration of services and workflows, bypassing the need for configuration files. While initially used for internal testing, this feature can also be beneficial in scenarios where using configuration files is impractical.

🔬 Improvements

  • Infinitic has been updated to use UUID version 7. These are sortable UUIDs that include a timestamp, which is expected to enhance performance when used as primary keys in databases.
  • Idempotency: In scenarios where hardware or network issues occur, there's a possibility that the same tasks may be processed multiple times. Ultimately, it falls upon the user to ensure tasks are designed to be idempotent as required. Starting from version 0.13.0, the taskId can be reliably used as an idempotent key. This is because Infinitic will generate the same value for taskId, even if the task creation process is executed repeatedly.
  • Performance Improvement: Prior to version 0.13.0, initiating a workflow involved sending a message to the workflow engine, which would then create an entry to store its state in the database. Following this, it would send another message to commence the workflow execution in order to identify and dispatch the first task. This task information would be relayed back to the engine for dispatch. The drawback of this approach was evident during surges in workflow initiation (for example, 1 million starts), where Infinitic had to sequentially store 1 million state entries before beginning to process the first task. This could significantly delay the start of task processing in practical scenarios. Since the release of version 0.13.0, the execution process has been optimized. Now, the first task is processed immediately upon dispatch by all available workers, substantially reducing the "time to first execution."
  • Worker Graceful Shutdown: Infinitic is designed to ensure no messages are lost and that workflow executions continue under any circumstances. However, prior to version 0.13.0, shutting down a worker could result in a significant number of duplicated messages or actions. This was because the worker could close while still sending multiple messages. With the introduction of version 0.13.0, workers now attempt to complete any ongoing executions before shutting down, with a default grace period of 30 seconds. This duration can be adjusted using the new shutdownGracePeriodInSeconds setting in the worker configuration.
  • Worker Quicker Start: Upon startup, a worker verifies the existence of the tenant, namespace, and necessary topics for the services and workflows it utilizes, creating them if necessary. Previously, this setup was performed sequentially. Now, it is executed in parallel, significantly reducing startup time, especially in scenarios where a worker is responsible for managing a large number of tasks or workflows.

🪲 Bug Fixes

  • Fix false warning about topics being partitioned
  • Fixed the behavior of getSecondsBeforeRetry, which defines the task retry strategy. When the value is less than or equal to 0, retries will now occur immediately. Previously, no retry would be attempted in this scenario.
  • If the methodId is not specified when using CompleteTimers client method, all timers of the workflow will now be completed. Previously, only the timers on the main method were completed in the absence of a specified methodId.

v0.12.3

🚀 New features

  • A new configuration option maxPoolSize has been introduced to the MySQL storage configuration. This option allows you to specify the maximum number of connections in the connection pool.
  • The tagEngine setting can now be configured under serviceDefault.
  • The tagEngine and workflowEngine settings can now be configured under workflowDefault.

🚨 Breaking changes

  • The entries service and workflow in the worker configuration, which were used to establish default values for services and workflows, have been renamed. The updated names are serviceDefault and workflowDefault, respectively.

🔬 Improvements

  • Not being able to check tenant / namespace does not trigger an error anymore.

v0.12.2

🔬 Improvements

  • Restore config files as data classes instead of interfaces
  • Restore fromConfig method for clients and workers
  • Improve worker logging

v0.12.1

🚀 New features

  • WithTimeout interfaces on Workflows interfaces can now be used to define global timeouts

🪲 Bug Fixes

  • Fix Json Serialization of WorkflowTask parameters and return value for previous versions
  • Fix GetIds on client for inMemory implementation

🔬 Improvements

  • WorkflowTask parameters and return value are now serialized using avro with schema fingerprint - this will improve future backward compatibilities

v0.12.0

🚀 New features

  • @TimeOut annotations on Services and Workflows interfaces can now be used to define global timeouts (including message transportation and retries) increasing workflow reliability - fix #74 & #198
  • Pulsar tenant, namespace and topics are created on-the-fly when needed. ("client-response" topics are not created systematically anymore)
  • We now check that services and workflows implementation defined in configuration are actually an implementation of the provided name - fix #200
  • A subscription is now automatically created to DLQ to avoid losing the messages
  • The workflow engine is now aware of messages sent to Dead Letter Queue

🔬 Improvements

  • Refactor of Transport (Pulsar and InMemory implementation)
  • End-to-end tests are now done directly on Pulsar (when Docker is available)
  • Additional backward compatibility tests
  • Improved logging
  • Throwable are not caught anymore anywhere
  • Use io.github.oshai:kotlin-logging-jvm for Logging
  • Fixed https://github.com/infiniticio/infinitic/security/dependabot/33

v0.11.7

🔬 Improvements

  • Bump to gradle 8.4 and use jvm toolchain 17
  • Update to kotlin 1.9.20
  • Update CI to jvm 17 and separate build, test and lint
  • Bump several libraries
  • Plugin replace ktfmt with Spotless (with ktfmt) for better integration
  • use testContainers for testing Redis

v0.11.6

🪲 Bug Fixes

🚀 New features

With the help of @cyrilStern, workflows' state can now be stored in a compressed format

🔬 Improvements

Bump dependencies version:

  • com.ncorti.ktfmt.gradle from 0.11.0 to 0.12.0
  • mysql:mysql-connector-java from 8.0.32 to 8.0.33
  • org.testcontainers:mysql from 1.17.6 to 1.18.3

v0.11.5

🔬 Improvements

Bump dependencies version:

  • kotlinx-coroutines from 1.6.4 to 1.7.1
  • caffeine from 3.1.3 to 3.1.6
  • kotlinx-serialization-json from 1.5.0-RC to 1.5.1
  • json-path from 2.7.0 to 2.8.0
  • jackson from 2.14.2 to 2.15.2
  • kotest from 5.5.5 to 5.6.2
  • mockk from 1.13.4 to 1.13.5
  • avro4k from 1.6.0 to 1.7.0
  • hoplite from 2.7.1 to 2.7.4
  • pulsar from 2.11.0 to 2.11.1
  • kweb from 1.3.7 to 1.4.0
  • slf4j from 2.0.6 to 2.0.7
  • kotlin-logging from 3.0.0 to 3.0.5

@Enach: Improve MySQL table structure with an additional index on KeySet

v0.11.4

🪲 Bug Fixes

  • fixed a bug occurring for tasks longer than 30 seconds

🔬 Improvements

  • upgrade of dependencies

v0.11.3

🪲 Bug Fixes

  • fix bug in which a state was not deleted after completion of an async child-workflow or method

🔬 Improvements

  • use Ktfmt instead of Ktlint

v0.11.2

🚀 New features

  • new consumer property in Pulsar configuration to define default properties for Pulsar consumers
  • new producer property in Pulsar configuration to define default properties for Pulsar producers

v0.11.1

🚀 New features

  • Workflow Versioning

v0.11.0

🚀 New features

  • new @Timeout annotation
  • new @Retry annotation
  • new @CheckMode annotation
  • new WithTimeout interface
  • new WithRetry interface
  • new timeoutInSeconds parameter in worker's configuration file
  • new retry parameter in worker's configuration file
  • new checkMode in worker's configuration file

🚨 Breaking changes

  • TaskOptions removed
  • WorkflowOptions removed
  • Services do not need anymore to extend abstract class Task
  • Task context replaced by Task static properties

v0.10.0

🚀 New features

  • new completeTimers function in clients
  • Services can now be manually registered in workers (fix #168)

🚨 Breaking changes

  • In configuration files, stateStorage is replaced by storage and stateCache is replaced by cache
  • storage and cache configuration are now properties of those keywords (see doc)
  • InfiniticClient and InfiniticWorker can now be used for both Pulsar and InMemory transport, removing the need for factories

🔬 Improvements

  • Topics used for delayed messages now have a long TTL (fix #170)
  • Improved WorkflowUpdatedException logging

v0.9.12

🔬 Improvements

  • Bump version of Kotlin, Gradle and dependencies

v0.9.11

🪲 Bug Fixes

  • fix BytesSchemaVersion collision in KSchemaReader

v0.9.10

🪲 Bug Fixes

  • fix #164

v0.9.9

🔬 Improvements

  • It's now possible to config Redis pool (maxTotal, maxIdle, minIdle). MaxTotal default is unlimited
  • It's now possible to serialize/deserialize an empty object

v0.9.8

🪲 Bug Fixes

  • fix backward compatibility of workflow state storage
  • fix backward compatibility of Pulsar message serialization

v0.9.7

🚀 New features

  • MySQL can now be used to store states (thx to @GauthierHacout)

🪲 Bug Fixes

  • fix #154

🔬 Improvements

  • WorkflowEngine is now idempotent when receiving multiple times the DispatchMethod message
  • WorkflowEngine is now idempotent when receiving multiple times the SendSignal message

v0.9.6

🪲 Bug Fixes

  • This release fixes a bug introduced in 0.9.4 when getting the status of deferred signals

v0.9.5

🚨 Breaking changes

  • use Java version 11

🔬 Improvements

  • Bump libraries dependencies

v0.9.4

🚀 New features

  • if we dispatch a workflow with a tag starting by "uniqueId:", Infinitic will check if another workflow with the same tag exists before dispatching it change the behavior of channel.receive(): each time await() is applied, the workflow awaits a new signal
  • channel.receive(n) will let you receive n signals through await() and throw a OutOfBoundAwaitException at n+1

🚨 Breaking changes

  • await() method applied to a deferred channel.receive() does not return the same value anymore

🔬 Improvements

  • Topics associated to clients and used for worker naming are now non-partitioned

v0.9.3

🪲 Bug Fixes

This release fixes the issue preventing workers to run in v0.9.2

🔬 Improvements

  • Add Infinitic version to schemas

v0.9.2

This version contains a configuration bug in Java compatibility and must not be used

🪲 Bug Fixes

This release fixes a configuration bug in Java compatibility in 0.9.1


v0.9.1

This version contains a configuration bug in Java compatibility and must not be used

🚨 Breaking changes

  • move Deferred and InfiniticClient to io.infinitic.clients

🔬 Improvements

  • add backward compatibility test on WorkflowState
  • add backward compatibility tests to topics schemas

v0.9.0

🚀 New features

  • new retryTasks and retryTasksAsync methods to retry tasks in running workflows
  • messages that can not be handled by the engines are not put in dead letter queues

🪲 Bug Fixes

  • fix a race condition where a client could send workflows with wrong parameters if sent asynchronously
  • JSON deserialization does not fail on additional fields
  • JSON deserialization use default value on missing fields (Kotlin)

🚨 Breaking changes

  • it's not possible anymore to manage tasks individually (tasks that are no within a workflow).
  • the internal data and topics structure has changed. Hopefully, that should be the last breaking change before v1.0

🔬 Improvements

  • wrap any exception in storage into a StorageException
  • refactor and simplify Pulsar implementations into a new infinitic-transport-pulsar module
  • refactor and simplify in-memory implementations into a new infinitic-transport-inmemory module
  • split tag implementations into new infinitic-task-tag and infinitic-workflow-tag modules
  • remove unused metrics-related code
  • remove task-engine: tasks are now sent directly to workers.
  • update clients, workflow engine, and workers to take into account the task engine removal
  • add blockifqueue=true in Pulsar producer

v.0.8.3

🪲 Bug Fixes

  • Fix #139

v0.8.2

🪲 Bug Fixes

  • Fix #137 - key-shared subscription could not guarantee that a given workflow or task is managed by a unique engine, if more than one topic leads to this engine

🚨 Breaking changes

  • runningTimeout in TaskOptions has now been named maxRunDuration and should be a Duration object

🔬 Improvements

  • Upgrade to Kotlin 1.6.10
  • Bump version of 3rd party libs
  • Discarded messages are now logged as warn
  • Default values in client and workflow's newTask and newWorkflow functions are now null

v0.8.1

🪲 Bug Fixes

  • bump kotlin-logging version to 2.1.20 to mitigate Log4shell

🔬 Improvements

  • remove unused directories node and infinitic-rest-api

v0.8.0

🚀 New features

  • new dispatch syntax to start task and workflow. The need for a new syntax was induced by #130 (removing the async function)
  • methods in the client have now an Async version, useful if we do not want to wait for message sending.
  • workflows can now run multiple methods in parallel! A straightforward application is to easily retrieve the properties of a running workflow
  • we now use String instead of UUID for ids. Using UUID was an implementation leak, that could prevent us in the future to let the user choose for an id

🪲 Bug Fixes

  • Fix #130
  • Fix #56

🚨 Breaking changes

  • removing the join() method on deferred (replaced by Async version on method in client)
  • removing the async function in client and workflow, replaced by dispatch
  • updated internal schemas

🔬 Improvements

  • improved default value when using channels in Java
  • refactored error management in workflows, with new exception: WorkerException, FailedTaskException, FailedWorkflowException, UnknowWorkflowException...
  • bump version of plugins and libs

V0.7.4

🚀 New features

  • authentication added to Pulsar configuration file to access to a secured Pulsar
  • join() method to Deferred to wait for the triggering message to be sent to Pulsar
  • join() method to InfiniticClient to wait for all messages to be sent to Pulsar
  • client and worker are now closeable and wait for having sent all messages
  • an in-memory implementation can be used during development by adding transport: inMemory in the configuration file.

🪲 Bug Fixes

  • fix bugs in workflow engines occurring in some edge cases

🚨 Breaking changes

  • in Infinitic configuration file:
    • serviceUrl renamed to brokerServiceUrl to be on par with Pulsar documentation
    • serviceHttpUrl renamed to webServiceurl to be on par with Pulsar documentation removed setupPulsar from PulsarInfiniticAdmin (not needed anymore)
  • changes in internal schemas

🔬 Improvements

  • needed tenant/namespace/topics are now automatically created at launch time by workers.
  • infiniticClient topic is automatically deleted when quitting
  • improved tests reliability

V0.7.3

This release contains a syntax issue and should not be used


V0.7.2

🪲 Bug Fixes

  • [Dashboard] add missing icons in git

🔬 Improvements

  • [Dashboard] make data loading explicit with animated icon

V0.7.1

🪲 Bug Fixes

  • password for Redis is not exposed anymore in logs
  • fixed a bug making SVG disappeared in Dashboard

v0.7.0

🚀 New features

  • New infinity-dashboard module implementing a first version of the infinitic dashboard. - - This version can:
    • display tasks and workflows (based on existing topics)
    • for each task or workflow, display connected workers and real-time stats of all topics used to manage it
  • A @ignore annotation has beed added to label properties you may want to ignore in a workflow's state

🪲 Bug Fixes

  • fix #119
  • fix #120 - logger variables are now ignored from workflow's state -

🚨 Breaking changes

  • some topic names were changed - you need to use a new Pulsar namespace

🔬 Improvements

  • dependencies were upgraded

v0.6.5

🪲 Bug Fixes

  • fix a bug appearing when for some java distribution due to bad detection of proxy classes

🚨 Breaking changes

  • internal schema has changed - you need to use a new Pulsar namespace

🔬 Improvements

  • upgrade dependencies, including Kotlin 1.5

v0.6.4

🚀 New features

  • [client, task, workflow] an @Name annotation allows you to decouple the name of tasks and workflows from their underlying implementation name
  • [task] tags are now accessible from the task context

v0.6.3

🔬 Improvements

  • better expose errors during task or workflow initialization in workers

v0.6.2

🔬 Improvements

  • use new s01.oss.sonatype.org server for publishing
  • infinitic-client module is now as an api-type dependency in infinitic-pulsar module (users do not need anymore to import infinitic-client)
Previous
Spring Integration