Batch Architecture¶

Here we’ll explain some important concepts for the batch feature.

Note

Some of the comments below describe how things are “meant” to work, but the behavior may not be built-in. It’s assumed that a batch handler will implement the desired behavior where applicable, ideally in alignment with comments here.

Data Structure¶

At its core each batch type has (at least) 2 tables: one for the “header” and one for the rows.

Each individual batch is comprised of 1 header, and 0 or more rows. Each row references the appropriate header.

The schema for header and row tables must be defined up-front by the app; there is not (yet) a way to have “dynamic” batch types.

Control Logic¶

As you might expect, a batch handler is responsible for the control logic.

There should be one “designated” handler for each batch type; you can obtain it via get_batch_handler() method on the app handler.

This batch handler can (typically) be used to create, populate, refresh and execute a batch.

(Non-)Effective Fields¶

There isn’t yet any way to formally declare this, but it’s important to understand that in practice, some data within a batch will “not matter” when the batch is executed.

A simple example is a batch used to adjust product pricing. Each row will have e.g. a product UPC and the new/desired price. But probably each row should also have the product description, and even the original/current price, along with the calculated difference (actual and margin) etc.

The extra fields are there mostly for the user’s sake, to give them better understanding of the context or implications of the batch.

Batch/Row Status¶

Each row in a batch will have a “status” which is meant to give the user even more clues as to the sanity of the batch and if/when it is safe to execute.

Technically the batch itself also has a “status” - for the same reasons, although it is often less useful compared to the row-level status.

Each batch/row has 2 status fields:

status_code - integer value indicating “which status”
status_text - optional string further describing “why” etc.

The batch model classes must define the possible status code values, for both the row and batch proper. These are defined as dicts, where keys are the status codes and values are human-friendly labels.

So in practice since each code has a label, the status_text field is not often needed. But it’s there when even more clues need to be conveyed to the user.

The batch handler should set the row status when it’s first added, and during a refresh. If applicable, it should potentially update the batch status at the same time.

Data Refresh¶

The whole idea behind a batch is that it’s somewhat asynchronous: it is first created, and then “iterations” may happen over time, before it is finally executed.

This can sometimes cause the batch to go “stale” compared to live data. For instance with the product pricing batch example, let’s say a new batch is made on Monday but not executed until Friday. Meanwhile on Wednesday someone manually changed prices for some of the same products, not knowing a batch was pending.

The “solution” here is to “refresh” the batch before executing it. Refreshing a batch will basically re-fetch related data from live systems, re-calculate other fields and the status for each row etc.

Note that some data in the batch originally came from a “source” (e.g. CSV file, or user input) and can never be refreshed per se. Such fields should be considered immutable within the batch, but ideally everything else should be part of the refresh.

There are also scenarios where the user working with a batch sees some rows with “bad” status, which triggers them to make some manual changes in the live system to rectify the situation. They would need to then refresh the batch so it reflects their changes, and presumably gives those rows a good status.

Finally on a related note, when a row is “modified” by the user, it should immediately be refreshed. But this should not trigger a full batch refresh.

Executed Batches¶

Worth mentioning here, that when a batch is executed it becomes frozen in time, essentially. No further edits or refresh can happen for it. The only thing left which can be “done” is to delete the batch. But until then it remains viewable, and user can download related files for it if applicable etc.

Actually this describes the WuttaWeb app behavior, but any other UI should do the same. The batch handler probably will not have logic to protect against modifying an executed batch, as it’s expected the UI will protect against it.

Also important to note that the batch data should (usually) not be changed as part of the execution itself. The whole point of executing a batch is to apply its data to the live system, so as a rule that data should remain “as-is” from the point of execution.