Dry-run mode

What this accomplishes

Test workflows against fixture data without touching live systems.

Dry-run mode is the test mode for workflows. A workflow in dry-run mode runs exactly as it would in production, except that no side effects are produced. The user can see the output of every step; the user can see the log of every step; the user can iterate quickly on the workflow without touching live systems.

This page is the dry-run mode. The blog post on dry-run is at dry-run and replay; the run inspector (which the dry-run output lands in) is in run inspector and replay.

Current availability

This page documents behaviour verified in Miton development builds. Miton is still pre-release and is not publicly downloadable; invited beta users will receive a signed desktop build with automatic updates.

Before you start

• A Miton development build, an active project and Automation open.
• A connected model plus permission for every file, network or scheduled action in the workflow.

How dry-run mode works

The user opens a workflow in the Automation. The user clicks the “Dry-run” toggle in the toolbar. The toggle is a switch: on or off. The switch is per workflow; the user can have some workflows in dry-run mode and others in live mode.

When the workflow is in dry-run mode, every node type respects the mode:

• HTTP nodes make the request, but the response is discarded. The user sees the request and the response in the log.
• Email nodes generate the email, but the email is not sent.
• Database nodes generate the write, but the write is not committed.
• File nodes generate the file write, but the write is not committed to disk.
• Notification nodes generate the notification, but the notification is not sent.
• Model nodes make the model call. The model cost is incurred (the user pays for the model tokens, even in dry-run mode).

The dry-run mode is observable. The user sees a “dry-run” badge on every step in the run log. The user can see exactly which side effects were generated and discarded.

The dry-run mode is not silent. The user sees the generated side effects in the log; the user can verify that the side effects are what the user intended. The user can also see the warnings: “this step would have sent an email to customer@example.com”, “this step would have written 3 rows to the database”.

The fixture data

The fixture data is the input the dry-run uses. The user can provide a fixture for a trigger type (schedule, webhook, chat command) by clicking “Test with fixture” in the workflow surface. The fixture is a JSON object that matches the trigger’s input schema.

The fixture is per workflow. The user can also have a global fixture that applies to every dry-run across every workflow; the global fixture is in Settings.

The user can also use a previous run as a fixture. The user clicks “Use as fixture” on a run in the run history; the run’s trigger payload becomes the fixture for the next dry-run. The user can iterate on the workflow against a real run’s input without re-running the upstream steps.

The per-step replay

The per-step replay is a tighter form of the dry-run. The user clicks “Replay” on a step in the run history; the user provides an input (the original input, or a new input); the step runs with the input. The user can see the step’s output, log, duration, cost.

The per-step replay is useful for debugging. A user whose workflow failed at step 5 can replay step 5 in isolation, with the same input, to see the step’s output. The user can adjust the input, replay again, and iterate quickly. The user does not have to re-run steps 1-4.

The per-step replay is a kind of dry-run. The step runs in dry-run mode by default (no side effects). The user can toggle “live replay” to make the replay produce side effects; the toggle is a one-click affordance, not a configuration.

The development cycle

The dry-run mode fits into a three-step development cycle for a workflow:

1. Build — the user builds the workflow in the visual editor. The user adds nodes, configures properties, draws connections.
2. Test — the user runs the workflow in dry-run mode with a fixture. The user iterates on the workflow until the output is what the user intended.
3. Ship — the user toggles dry-run off. The workflow runs in live mode from the next trigger. The user monitors the run history to verify the workflow is working as expected.

The cycle is fast. The user can iterate on the workflow in dry-run mode without touching live systems. The user can also replay a step in isolation to debug a failure without re-running upstream steps.

The cycle is observable. The user can see the workflow’s run history, the workflow’s log, the workflow’s cost. The user can see whether the workflow is in dry-run mode or live mode. The user can toggle dry-run mode at any time.

The rate limiting

The dry-run mode is rate-limited. A user cannot run a dry-run 100 times in a minute; the dry-run is rate-limited at 10 runs per minute per workflow. The rate limit prevents the user from accidentally incurring a large model cost on a misconfigured workflow.

The per-step replay is also rate-limited. A user cannot replay the same step 100 times in a minute; the replay is rate-limited at 10 replays per minute per step.

The rate limit is configurable. The user can configure a higher rate limit in Settings under “Workflow → Rate limit”; the user can also disable the rate limit (the user accepts the cost of the dry-runs).

What the dry-run mode does not cover

The dry-run mode does not cover:

• Integration testing — the dry-run mode tests the workflow’s logic, not the integration’s behaviour. A user who needs to test the integration should use a staging environment.
• Load testing — the dry-run mode runs the workflow once (or as many times as the user triggers it). The user can use a third-party tool for load testing.
• Real-time monitoring — the dry-run mode runs the workflow on demand. The user cannot monitor the workflow in real time. The user can use a third-party tool for real-time monitoring.

The dry-run mode focuses on the per-run testing. The less common cases are covered by the user’s existing tools.

What success looks like

The dry-run mode is the right shape for testing workflows. The user can iterate on the workflow without touching live systems. The user can see the generated side effects in the log; the user can verify the side effects are what the user intended.

Use the dry-run mode for every workflow before shipping. The dry-run mode catches configuration errors, missing fields, wrong endpoints, and other common mistakes. The dry-run mode does not catch integration behaviour (the user has to test the integration separately).

Use the per-step replay to debug a failure. Click “Replay” on the failed step; iterate on the input and the environment; promote the corrected step to the canonical workflow when the replay succeeds. The replay is rate-limited at 10 replays per minute per step; the rate limit prevents the user from accidentally incurring a large model cost.

Common failures and recovery

• Inspect the failed run and the exact node output before replaying it.
• Confirm credentials, permissions and test fixtures belong to the active project.
• Use dry-run or step replay before starting the complete workflow again.

Related next action

Open the Automation overview to connect this task to the rest of the workflow.