Integration testing checks that separately built components — services, databases, and third-party systems — work correctly together, so the test data has to line up across all of them at once. The core requirement is coordination: the same entities must carry consistent, referentially intact values everywhere the test reaches, so an order in one system still matches its customer, invoice, and shipment in the others. That takes de-identification applied consistently across connected databases, subsetting that keeps related records together, and generated data to stand in for the systems that have no usable production source.

What integration testing needs from test data

Integration testing verifies that independently built components — one service calling another, an application reading from a database, your code invoking a third-party API — behave correctly when they run together. A unit test can get by on a handful of hand-made values because it exercises one component in isolation. An integration test can't: the moment the test path crosses a boundary between systems, the data on both sides of that boundary has to agree.

Picture a single order moving through three systems that were built and are maintained separately: an orders database that records the purchase, a billing service that issues the invoice, and a shipping service that schedules the delivery. For the test to mean anything, all three have to be describing the same order, the same customer, and the same amounts. If the customer attached to the invoice doesn't match the customer on the order, or the shipment points at an order ID the orders database has never heard of, the test fails on a data problem that has nothing to do with the behavior you were trying to verify — and the team burns an afternoon chasing a defect that was never in the code.

That is the requirement integration testing places on test data, and it differs from what any single-system test asks for. The data has to be coherent across every system the test touches, simultaneously. Meeting it is the central problem of test data management for integrated systems, and it only gets harder as the systems multiply: across distributed services and microservices, the same order has to reconcile across more and more independently owned data stores at once.

Why masking or subsetting each system on its own breaks integration tests

The trouble starts when each system is prepared for testing on its own. Independent teams mask, subset, and refresh their own databases on their own schedules, and nothing coordinates the result across the boundaries between them. Three failure modes show up again and again:

  • Inconsistent masking. The orders database and the billing service both hold a customer_id, but each is masked in isolation, so the same real customer becomes 8842 in one system and 3391 in the other. Every cross-system join and lookup that relied on that ID now finds nothing.
  • Orphaned foreign keys from naive subsetting. The billing service is cut down to a manageable size by grabbing a slice of invoices, but the orders those invoices reference are never pulled into the orders database's subset. The foreign keys now point at rows that don't exist, and the test hits a broken join the moment it follows one.
  • Unsynchronized refreshes. One system is refreshed from production on Monday and another on Thursday. For the window in between, the two describe different states of the world, and an entity present in one is missing or altered in the other.

Underneath all three is the same pair of properties that has to survive whatever you do to the data. The first is referential integrity: every foreign key still resolves to a row that exists, so the relationships between records stay intact. The second is consistency: a given real value transforms to the same replacement value everywhere it appears, so the customer on the invoice is still recognizably the customer on the order. Independent, per-system preparation preserves neither across a boundary — which is why the fix has to operate across systems, through consistent, deterministic masking rather than a separate, uncoordinated pass on each database.

Coordinating de-identified data across systems with consistent masking

Coordinating data across systems starts with making transformation deterministic. A deterministic transformation maps each input value to the same output value every time, so a given customer_id de-identifies to the identical replacement in the orders database, the billing service, and the shipping service alike. Do that, and the cross-system joins that broke under independent masking hold together again, because the linking values still match on both sides of every boundary.

Tonic Structural is built around this property. Structural applies consistent transformations across tables and, critically, across connected databases, so the same source value produces the same de-identified value no matter which system stores it. Because the transformation is deterministic rather than random, it is also reproducible: two systems that de-identify from the same source with the same configuration arrive at the same values even when they're refreshed on different days, which closes the gap that unsynchronized, per-system refreshes otherwise leave open. When two systems share an identifier without a declared foreign key between them — a common situation once data lives in separate services — you can define a virtual foreign key inside Structural to describe that undeclared relationship, and the consistency guarantee then extends across it. That is what lets a de-identified order in one database still line up with its invoice and its shipment in the others, instead of fragmenting into three unrelated records.

The same mechanism reaches across databases when you need a coordinated slice rather than a full copy. Structural can generate cohesive subsets across related databases, applying the same consistent transformation as it goes, so the records that land in each system's test copy are the de-identified counterparts of the same real entities. Because Structural transforms production data rather than inventing it, this path keeps production data out of lower environments and helps reduce the sensitive-data footprint your integration environments carry — the coordination and the privacy come from the same pass.

The Tonic Advantage: one consistent transformation, every connected system. The hard part of integration test data isn't masking any single database — it's making the masking agree across all of them. Structural applies the same deterministic transformation across tables and connected databases at once, so a customer, an order, or an account de-identifies to the same value everywhere it appears. The joins that span your services survive the transformation, and the data reaching each environment is coordinated by construction rather than reconciled after the fact.

Building coherent, referentially intact subsets for integration environments

You rarely want a full copy of production in an integration environment — it's slow to move, expensive to store, and carries more sensitive data than any test needs. The alternative is a subset: a slice small enough to be practical and complete enough that every relationship a test follows still resolves. Building smaller, realistic test databases is a discipline in its own right; integration testing adds one demand on top — the slice has to line up across systems, not just within one.

Tonic Structural builds coordinated subsets through data subsetting. You anchor on a table, and its patented subsetter follows the foreign keys outward, pulling in every row the anchored records depend on, with lookup and reference tables copied in full so nothing a query joins to comes up empty — the patented subsetter shrinks petabytes down to gigabytes without breaking foreign keys. For an integration test, though, the subset has to line up across every database at once, which coordinated subsetting handles by combining deterministic filters with the same consistent transformation the systems already share:

  1. Anchor on the entity the test exercises — say, a set of recent orders in the orders database.
  2. Follow the relationships outward from that anchor, so each order brings its customer, its invoice, and its shipment along with it.
  3. Apply the same deterministic filter in each system, so the billing and shipping subsets select the counterparts of exactly those orders rather than an unrelated slice.
  4. Copy lookup tables in full in every database, so shared reference data is identical everywhere.

The result is a set of subsets that agree across systems — isolated, collision-free environments that several teams can run integration and end-to-end suites against in parallel without stepping on one another's data. Working at scale, eBay scaled a multi-petabyte data ecosystem down to manageable subsets, shortening development cycles and fueling automated testing. Standing these environments up is squarely a testing and QA problem, and coordinated subsetting is what makes the data underneath them trustworthy.

Filling the gaps: systems with no production data, and mock APIs

Some systems in an integration test have no production data to derive from. A brand-new service hasn't accumulated any yet, a third-party API isn't yours to copy, and a downstream system the test depends on may not be stood up at all. Masking and subsetting have nothing to work from in these cases, because there's no source to transform — this is the line between synthetic data for testing and masked production data, and it's where generation rather than transformation is the right tool.

Tonic Fabricate fills that gap. Fabricate generates referentially intact data from scratch, working from a description of the schema you need, and it can also model new data on an existing dataset — including the de-identified output Tonic Structural has already produced — so the generated records stay consistent with the rest of the test data instead of forming an island. For the services a test can't call live, Fabricate can stand up mock APIs that respond with realistic, referentially consistent data, letting the integration path run end to end even when a real dependency is missing.

These are two tools for two different gaps, and they work together: Structural coordinates the data you can derive from production, and Fabricate supplies the data you can't. Used side by side, they cover every system in the test path — the ones with a production source and the ones without. And because both can run unattended, the whole coordinated dataset can be regenerated fresh on every run inside your CI/CD pipeline, so each integration suite executes against data that is current, coherent, and safe.

The Tonic Advantage: generate what you can't derive. Not every system in an integration test has a production database behind it. Fabricate generates referentially intact data from scratch for the ones that don't, models it on Structural's de-identified output so it stays consistent with everything else, and stands up mock APIs for dependencies the test can't call live — so the whole path runs, not just the parts that already have a data source.