Migrations

xmatix migrate compares the configuration checked into your repository with the live tenant (migrate plan) and then pushes the repository's files to that tenant (migrate apply). It is the last step of a change that started with a snapshot export or a pull, was edited and reviewed in git, and now has to reach the tenant. The CI recipes build pipeline jobs on top of it.

xmatix migrate plan                    # writes xmatix/.snapshot/migrate-plan.json
xmatix migrate plan --keep-server-snapshot
jq '.counts' xmatix/.snapshot/migrate-plan.json
xmatix migrate apply --dry-run
xmatix migrate apply

Overview

The two commands are independent:

• migrate plan reads only. It exports the tenant into a separate folder, compares that export file by file with your workspace, and writes a manifest that lists every file as added, modified, removed or unchanged. It changes nothing on the tenant.
• migrate apply writes. It runs snapshot import, which pushes every artifact file in the workspace's canonical folders. It doesn't read the plan, so it pushes unchanged files too.

The plan is a review document. You read it, or assert on it in a pipeline, before you decide to apply.

In the current release, migrate plan stops every time with "Server snapshot export failed; aborting plan." The plan runs a full snapshot export, and the export's scripts step always fails (see Snapshots). Instead, compare the repository with the tenant by exporting into a clean checkout and reviewing git diff, as in Snapshots, Step 2. migrate apply is not affected.

When to use it

• Before a promotion, to see what would reach the tenant and to spot changes someone made in the product that your repository doesn't have.
• As a drift check, to find out whether the tenant still matches the commit you last applied. Leave out the files that change on every export (see Common problems).
• To push a reviewed change from a workspace to the tenant that workspace points at.

Don't use migrate apply to copy scripts or workflows into a different tenant. See Can I promote from one tenant to another?

How xMatix handles it

What migrate plan does

1. It creates a separate workspace and copies your .xmatix/config.json into it, so the export reaches the same tenant with the same sign-in. By default this workspace is a new xmatix-snapshot-<id> folder in the operating system's temp folder. With --keep-server-snapshot it is xmatix/.snapshot/server/ in your workspace, and any copy kept by an earlier plan is deleted first.
2. It runs xmatix snapshot export --scope <scope> in that workspace. The export's normal output is hidden; only its errors are shown. If the export fails, the plan prints "Server snapshot export failed; aborting plan." and exits 1. The plan passes no --skip, so the export always includes the scripts step, which fails in the current release. The plan therefore never gets past this step.
3. It compares six folders in both trees: xmatix/metadata, xmatix/automation/scripts, xmatix/automation/rules, xmatix/automation/workflows, xmatix/analytics/dashboards and xmatix/analytics/reports. Every file in them counts, including each script's .meta.json. Each file is identified by its path and hashed with SHA-256.
4. It writes the manifest, prints a table of counts and exits 0.

The plan compares exact file contents, not meaning. Two files that hold the same configuration still show as modified if any byte differs, such as the export time in the metadata file.

The temp-folder export is not deleted after the plan. Clear old xmatix-snapshot-* folders from the temp folder now and then.

What migrate apply does

migrate apply runs xmatix snapshot import in the current folder and returns its exit code. It passes --skip and --dry-run through, and nothing else. The import pushes files in this order, continuing past failures:

1. every *.json file under xmatix/metadata/, with metadata push;
2. every *.csx file under xmatix/automation/scripts/, with scripts push;
3. every *.json file under xmatix/automation/rules/, with rules push;
4. every *.json file under xmatix/automation/workflows/, with workflows push;
5. every *.json file under xmatix/analytics/dashboards/ and xmatix/analytics/reports/, with dashboards push and reports push.

What each push does with an added or modified file differs by kind:

Nothing on the server is deleted for a removed file. The exception is metadata, where a full-mode file prunes.

Prerequisites

• The CLI installed and a workspace created with xmatix init.
• xmatix login completed for the tenant in .xmatix/config.json. Check it with xmatix scripts list. Don't use xmatix whoami or xmatix doctor for this, because both fail in the current release (see CLI authentication).
• A committed xmatix/ tree, normally written by xmatix snapshot export --skip dashboards,reports from the same tenant.
• xmatix/.snapshot/ in the repository's .gitignore. The plan and the kept server copy are written there, and xmatix init doesn't ignore that folder.
• The access that each kind of push needs (see Permissions).

Procedure

Step 1 — Run the plan

From anywhere inside the workspace, run:

xmatix migrate plan --keep-server-snapshot

In the current release this command stops with "Server snapshot export failed; aborting plan." (see Overview). Use the export-and-diff comparison in Snapshots instead, then continue with Step 3. The rest of this step and Step 2 describe what the plan reports when its export succeeds.

The CLI exports the tenant into xmatix/.snapshot/server/, then prints a table with four rows: added (local only — push will create), modified (will overwrite server), removed (server only — push will NOT delete) and unchanged. The last line names the plan file. The "push will create" wording is not true for scripts, as the table above shows.

Use --out <path> to write the plan somewhere else, and --scope <scope> to export a different metadata scope. Keep the default scope, TenantOwned, unless your repository was exported with another one.

Step 2 — Read the plan

List the files that differ:

jq -r '.entries[] | select(.Status != "unchanged") | "\(.Status)  \(.RelPath)"' xmatix/.snapshot/migrate-plan.json

Expect xmatix/metadata/tenant-owned.json and every script's .meta.json to be modified on every run, because each export stamps them with the time. For any other modified or removed file, open both copies and compare them:

diff xmatix/automation/rules/Order/CreditCheck.json \
     xmatix/.snapshot/server/xmatix/automation/rules/Order/CreditCheck.json

To compare the metadata without the export time, compare only its tables:

diff <(jq -S .tables xmatix/metadata/tenant-owned.json) \
     <(jq -S .tables xmatix/.snapshot/server/xmatix/metadata/tenant-owned.json)

Decide what to do with each difference:

• A modified or removed file that you didn't change in the repository was changed in the product. Export it into the repository (xmatix snapshot export --skip scripts,dashboards,reports, then xmatix scripts pull --all), review it and commit it before you apply. Otherwise the apply overwrites that change.
• A removed file that you deleted on purpose stays on the tenant. Delete it in the product if it should go.
• An added script can't be pushed. Create the script in the product, pull it, and copy your code into the pulled file.

Step 3 — Prepare the metadata file

The metadata file that snapshot export and metadata pull write is a full pack. Pushing a full pack deletes metadata the file doesn't contain, including the tenant's standard metadata. Before an apply, do one of the following:

• set the file's mode to delta, for example with jq '.mode = "delta"', or
• leave metadata out with --skip metadata and push it yourself as a delta pack.

See What push does.

Step 4 — Preview the apply

xmatix migrate apply --skip dashboards,reports --dry-run

For every file, the CLI prints a would run: line with the push command it would run, grouped by folder. Folders that are missing or empty are reported as skipped. Nothing is sent to the tenant. Check that the paths belong to the workspace you expect.

Step 5 — Apply

Run the apply in a terminal, so that you can answer the confirmations:

xmatix migrate apply --skip dashboards,reports

• metadata push shows the file and its size and asks Continue? for each metadata file.
• rules push asks before it replaces the actions of a rule that already has actions on the server.
• Each push prints its own result. The run ends with Snapshot import complete., or Snapshot import finished with N failure(s). and exit code 1.

In a pipeline nobody can answer the confirmations, so push each kind with its own flags instead. See CI recipes.

Step 6 — Finish the promotion

1. Activate the new script versions. The apply saves them without making them active. Activate each one in the product, or push it again with xmatix scripts push <file> --activate, which saves one more version and makes it active.
2. If the apply created any workflows, run xmatix workflows pull and commit the result. Push doesn't write the new definition id back into your file, so another apply would create the workflow again.
3. Run xmatix migrate plan again. Scripts, rules and workflows that you pushed now show as unchanged, apart from the files described in Step 2.

Fields

The plan file, xmatix/.snapshot/migrate-plan.json:

Options:

Exit codes:

Business rules

• The plan never changes the tenant or your workspace's artifact files. It writes only the plan file and, with --keep-server-snapshot, the server copy.
• The plan always compares all six folders. It has no --skip, so the analytics folders are always part of it.
• The metadata file's name depends on the scope: TenantOwned writes tenant-owned.json and TenantFull writes tenant-full.json. A plan run with a different scope from the one your repository was exported with shows your file as added and the server's file as removed.
• Apply pushes every file in the canonical folders, not only the ones the plan flagged, and it pushes every *.json under xmatix/metadata/. A second metadata file in that folder, such as a tenant-full.json, is pushed as well.
• A full-mode metadata file deletes every row that an earlier pack delivered to the tenant and that the file doesn't contain. A delta-mode file only adds and updates.
• Apply never deletes a script, rule, workflow, dashboard or report on the server.
• Scripts are matched by the script id in their .meta.json, rules by entity name and rule name, and workflows by definition id.
• A rule file with no actions list leaves the rule's actions on the server unchanged.
• A script push is skipped when the file matches the version recorded in its .meta.json. It is refused with exit code 2 when someone changed the script in the product after the last pull.
• Apply runs every push even after a failure, and reports one exit code at the end.

Example

A developer changes the credit-check rule on orders and promotes it to the tenant the repository was exported from.

1. They edit xmatix/automation/rules/Order/CreditCheck.json to raise the limit in the rule's condition. The change is reviewed and merged.
2. Because migrate plan can't run in the current release, they check for product changes on a clean checkout of the last applied commit: xmatix snapshot export --skip scripts,dashboards,reports, then xmatix scripts pull --all. git diff with the generatedAtUtc and lastPulledUtc patterns shows nothing, and git status shows no new files, so nobody changed the tenant since the last apply. They discard the export with git checkout -- xmatix/.
3. The metadata hasn't changed either, so they leave metadata out of the apply.
4. They run xmatix migrate apply --skip metadata,dashboards,reports --dry-run and check the would run: lines.
5. They run xmatix migrate apply --skip metadata,dashboards,reports. The five scripts are skipped as unchanged. The rule is updated, and they confirm the replacement of its two actions. The run ends with Snapshot import complete.
6. A second export into a clean checkout of the merged commit shows no change to CreditCheck.json.

Training

Practice exercise

In a sandbox workspace with an exported, committed xmatix/ tree:

1. Run xmatix migrate plan. Expected result in the current release: "Server snapshot export failed; aborting plan." and exit code 1.
2. In the product, change the description of a test business rule. Run xmatix snapshot export --skip scripts,dashboards,reports and git diff -- xmatix/automation/rules/. Expected result: the diff shows the new description in that rule's file.
3. Commit the change. Export again and run the same git diff. Expected result: no output.
4. Change the rule's description in the workspace file. Run xmatix migrate apply --skip metadata,scripts,workflows,dashboards,reports, and answer the confirmation if one appears. Expected result: the product shows your description, and a new plan shows the file as unchanged.

Quick reference

• migrate plan stops at its export in the current release. Compare by exporting into a clean checkout instead.
• When it runs, migrate plan reads only, and exits 0 whatever it finds.
• The metadata file and every .meta.json show as modified on every run.
• --keep-server-snapshot puts the tenant's copy under xmatix/.snapshot/server/ for diff.
• migrate apply is snapshot import. It pushes every file, not only the flagged ones.
• Set the metadata file's mode to delta, or skip metadata. Never apply a full pack.
• Add --skip dashboards,reports until analytics can round-trip.
• Activate pushed scripts afterwards. New scripts must be created in the product first.
• Nothing on the server is deleted for a removed file.

Permissions

The CLI acts with the permissions of the account that signed in with xmatix login. The server records that account as the author of every change.

• The metadata part of migrate plan and migrate apply needs the setup.metadata.packages.manage capability, the same one that opens Setup → Packages. Without it, the metadata pull or push fails with a 403, and so does the plan.
• Scripts, rules and workflows need the same access that their own screens need in the product.

See the security model for how capabilities are granted.

Configuration

• Target tenant. The nearest .xmatix/config.json above the current folder decides the tenant, for both commands. Keep one workspace per tenant.
• Metadata scope. --scope on migrate plan must match the scope your repository was exported with. The default, TenantOwned, holds the tenant's own metadata and the rows earlier pushes recorded, but not edits to standard rows (see Which rows each scope includes).
• Metadata mode. The mode property in the metadata file decides whether a push prunes. Set it to delta for promotions.
• .gitignore. Add xmatix/.snapshot/.
• Kinds. Use --skip on migrate apply to leave out kinds you don't manage in the repository, and always dashboards,reports in the current release.

Common problems

"No xMatix workspace found. Run xmatix init in your repository root first."

The command was run outside a workspace. Change to a folder inside the workspace that holds .xmatix/.

"Server snapshot export failed; aborting plan."

One of the pulls in the plan's export failed. In the current release this happens on every run: the export runs scripts pull without --all, which always fails, and the plan can't skip it. The plan hides the export's normal output, so the message itself is all you see. Compare the repository with the tenant by exporting into a clean checkout instead, as described in Snapshots. An expired sign-in or a missing permission, such as a 403 on the metadata pull, gives the same message.

The plan always shows the metadata file and the .meta.json files as modified

Every export stamps the metadata file with generatedAtUtc, and every script pull stamps lastPulledUtc in each .meta.json. Their hashes therefore never match. Compare metadata with jq -S .tables, and leave .meta.json files out of assertions, as the CI recipes drift gate does.

Every analytics file is listed as added

Dashboards and reports can't be pulled in the current release, so the tenant's export has no analytics files. See Dashboards and reports in the CLI.

My metadata file is added and a different one is removed

The plan ran with a different --scope from the one your repository was exported with, so the export wrote a file with a different name. Run the plan with the matching scope.

"No sibling .meta.json next to …" during apply

The script exists only in your workspace. Push updates scripts that were pulled; it doesn't create them. Create the script in the product, run xmatix scripts pull, and copy your code into the pulled .csx file.

"Server has a newer current version than your local copy."

Someone changed the script in the product after your last pull, and the push stops rather than overwrite that change. Pull the script, merge the two changes, commit and apply again. Use xmatix scripts push <file> --force only when you mean to discard the change made in the product.

Pushed script changes don't take effect

migrate apply saves a new version of each changed script without making it active. Activate it in the product, or push with --activate.

Each apply creates another copy of a workflow

The workflow's definition id in the file doesn't exist on the tenant, so each push creates a new workflow and the file keeps the old id. After the first apply, run xmatix workflows pull and commit the file, so later applies update that workflow. Delete the extra copies in the product.

Standard metadata disappeared after an apply

The metadata file was pushed in full mode, which deletes every row that an earlier pack delivered and that the file doesn't contain. Always set mode to delta or skip metadata. See What push does.

The apply stops at the metadata or rules step in a pipeline

metadata push, and rules push for a rule that already has actions, ask for confirmation. migrate apply doesn't pass --yes, and nobody can answer in a pipeline. Push those kinds yourself, as in CI recipes.

"Entity '…' not found on server." for a rule

The rule belongs to an entity that the tenant doesn't have. Push the metadata that creates the entity first, or leave the rule out.

Common questions

Does apply push only what the plan flagged?

No. migrate apply doesn't read the plan. It pushes every artifact file in the canonical folders, so unchanged files are pushed too. Unchanged scripts are skipped, but rules and workflows are written again with the same content, and a rule's actions are deleted and recreated. Metadata is the exception, because a full-mode file can delete rows. Use --skip to leave out whole kinds, and use the plan to decide whether to apply at all. If you need to push only a few files, run the per-kind push commands on those files instead of migrate apply.

Can I promote from one tenant to another?

Only partly. Point a workspace at the target tenant, copy the files in, then plan and apply. Metadata and rules can move between tenants, because rules are matched by entity and rule name. Rules still have limits: their target fields are stored as ids from the source tenant, and their delay settings are dropped when a rule is created (see Business rules in the CLI). Scripts can't: each .meta.json holds the script's id in the tenant it was pulled from, so the push fails with "Script … no longer exists on the server." Workflows from another tenant are created as new workflows on every apply until you pull them into the target workspace. Treat each tenant's workspace as its own exported copy.

Can apply delete something I removed from the repository?

Not for scripts, rules, workflows, dashboards or reports. A file that you deleted from the repository shows as removed in the plan, and the artifact stays on the tenant until someone deletes it in the product. This is deliberate, so a missing file can never remove configuration. Metadata is different: a full-mode metadata file deletes rows that it doesn't contain. Set mode to delta to stop that.

Why does the plan exit 0 when it finds differences?

The plan is a report, not a gate. It exits 1 only when the export fails. To fail a pipeline on drift, assert on the plan file's entries with jq -e, leaving out the metadata file, the .meta.json files and the analytics folders, and compare metadata with jq -S .tables. In the current release the plan can't get past its export, so the CI recipes drift gate compares an export with the checkout instead.

Related topics

• Snapshots
• Metadata seed-packs
• Scripts in the CLI
• Business rules in the CLI
• Workflows in the CLI
• CI recipes
• Workspace layout
• Developer troubleshooting