HW and SW co-verification: finding integration hell before the hardware exists

EXERCISED. Every number on this page was read out of a twin's own traces by run_ims.py, and this page is generated from those run records rather than typed.

The story in five beats

1. Everyone passes. Five teams verify their own part against their own document. The vendor receiver meets its datasheet. The switch meets its latency spec. The processor meets its DMA budget. The software meets its timing against a stub. Every light is green.

2. The status report is green too, and it is not lying. Run the real thing with two targets and the end-to-end latency passes with 9.4 ms of margin, and the network drops nothing. Those are the two numbers anyone would have asked for.

3. The system is shipping wrong data anyway. Three of every five messages carry the PREVIOUS frame's sensor picture, marked complete. Nothing throws. No counter moves. The software reads its report buffer without checking when what is in it arrived, so when Vendor B is late it silently fuses this frame's Vendor A report with last frame's B.

4. So we designed a fix, and the problem moved. Widening the collection window cures the stale data completely. It does not remove the failure, it RELOCATES it: what was a silent data-correctness defect becomes a loud timing defect, and the system blows the deadline at four targets. So we tried the other candidate, the vendor's early-partial mode. Same shape. It also cures the stale data, and the deadline still misses, by 300 us instead of 900. Two fixes, both correct about the bug they were aimed at, and neither one makes the system compliant. They move where the problem lives.

5. Which is what a real programme actually experiences. Fixes rarely delete a problem. They push it somewhere else, and the engineering question stops being "is it fixed" and becomes "where would we rather have it, and how much does moving it cost". A silent wrong-data defect and a 900 us deadline miss are not the same kind of bad: one ships incorrect information believing it is correct, the other announces itself. Choosing between them is a decision, not a bug fix, and it needs numbers.

6. The real finding is the one nobody set out to look for. The freshness defect is a software defect and software fixes it. The four-target deadline is not a software defect and no software fix reaches it. That is an architecture conversation, and the twin surfaced it in an afternoon, before anyone cut metal or committed to a vendor.

Beats 4 and 5 are the part worth dwelling on. A fix that works is a nice result. A fix that is correct, verified, and still insufficient is the result you cannot get from a document, and a fix that trades one failure mode for another is the ordinary experience of every programme that has ever integrated anything. Two hand-designed candidates found the trade and neither found a way out of it, which is exactly the point at which you stop guessing fixes and start searching the design space.

What this demonstrates

Integration hell has a specific shape. Hardware and software are built in separate organisations against separate documents. Each team verifies its own part and each team passes. Vendor parts arrive as timing envelopes rather than behaviour, so nobody can execute them. The software is tested against a stub that behaves like the vendor's typical number. Everything meets for the first time in the lab, late.

The counter-argument is not simulate earlier. It is that a timing envelope is enough to build an executable component, so the interaction can be run before any hardware exists. VSI is what lets a vendor stand-in, an infrastructure model and the real software share one fabric and one clock.

The system

Two receivers from two vendors detect targets. A mission processor runs custom software that fuses their reports. A transmitter sends a combined message on a fixed 50 Hz RF cadence to a communication array. A ruggedized Ethernet switch connects all four.

  TargetScenario                          test fixture, RF not modelled
        |                    |
        v                    v
  ReceiverVendorA      ReceiverVendorB
        |  Ethernet          |  Ethernet
        v                    v
  +-----------------------------------------------+
  |            RuggedizedSwitch                   |
  +-----------------------------------------------+
        ^                    ^
  MissionProcessor      Transmitter
  (hosts the mission              |  RF, not modelled
   software activity)             v
                          CommunicationArray

The whole twin is generated from a Cameo SysML model. The model carries structure, engineering protocols, the vendor timing envelopes and the requirement budgets, and nothing tool-specific: no implementation language and no VSI gateway names, because a systems modeller does not specify either. Those are twin-construction decisions and live in a generator config, from which they are copied into an assumptions file.

One fabric, two languages

The usual objection is whether a scripted stand-in and a compiled model really run together, or whether the compiled half is a diagram. They run together, on one clock.

• C++: MissionProcessor, RuggedizedSwitch
• Python: CommunicationArray, ReceiverVendorA, ReceiverVendorB, TargetScenario, Transmitter

C++ where the component is infrastructure and the point is that it is not a script: the switch's store-and-forward queueing, and the processor with the mission software running on it. Python where the behaviour is a vendor timing envelope or test apparatus, meant to be read and edited in front of an audience.

The conversion was done second on purpose, and it had to earn its place. The two most intricate components in the twin, a queue model and a timed activity interpreter, were re-implemented in another language. The only honest way to claim that is correct is to re-run the identical campaign and require identical numbers, so the results below were compared against the all-Python run field by field: 9 runs, 11 fields each, every verdict, count, latency and stale-message count identical. The comparator was run against a deliberately corrupted copy first, because a comparator that has never failed proves nothing when it passes.

Converting a component's language must not change the physics. Here it did not.

The software takes clock cycles

There is no software block and no interface between software and processor: software that runs on a processor is behaviour allocated to it. The model carries it as the activity Fuse And Transmit owned by MissionProcessor, and the twin runs that activity as a timed interpreter: one action current at a time, each occupying its bound duration in simulated microseconds, advancing only as the fabric clock advances.

Action	Duration	Fabric steps at 100 us
Combine Message	300 us	3.0
Issue Degraded Command	150 us	1.5
Issue Transmit Command	150 us	1.5
Open Collection Window	100 us	1.0
Parse Receiver A Report	1200 us	12.0
Parse Receiver B Report	400 us	4.0

Bound critical path 2300 us, which is 23 fabric steps. The generator refuses to build if any action of real work is bound to zero: software that completes instantly is the thing this demo exists to disprove.

The planted defect

Two defects, and the second is the one that matters.

Loud. Under multiple simultaneous targets, Vendor B's report arrives after the software's collection window closes.

Silent, and the payload. When B is late the software reads its report buffer without checking when what is in it arrived, so it fuses this frame's Vendor A report with the previous frame's Vendor B report and marks the message complete. Nothing throws. No counter moves. A bench test never sees it, because a stub receiver always answers inside the window.

Results

Nine runs. The verdict is never the exit code: the FabricServer exits nonzero on a teardown heap corruption after a complete run, so each run is judged by the fabric stop line plus a trace row per step from every component.

Run	PR-01 latency	PR-03 freshness	PR-04 no drops	PR-05 network	Worst latency	Silently stale
baseline-1t	pass	pass	pass	pass	9600 us	0
baseline-2t	pass	FAIL	FAIL	pass	8600 us	3
baseline-4t	pass	FAIL	FAIL	pass	8600 us	3
fix1-window-1t	pass	pass	pass	pass	9600 us	0
fix1-window-2t	pass	pass	pass	pass	12700 us	0
fix1-window-4t	FAIL	pass	pass	pass	18900 us	0
fix2-earlypartial-1t	pass	pass	pass	pass	9600 us	0
fix2-earlypartial-2t	pass	pass	pass	pass	12500 us	0
fix2-earlypartial-4t	FAIL	pass	pass	pass	18300 us	0

Read the two-target baseline carefully

Latency passes. The network drops nothing. Those are the two numbers a status report would carry, and they are both green. And PR-03 fails 4 of 4, with 3 messages carrying a stale Vendor B report and none of them marked degraded.

The mission processor's own log, and the fixture's ground truth beside it:

[TargetScenario        0 us] frame 0: 2 target(s), truth hash 23
[TargetScenario    20000 us] frame 1: 2 target(s), truth hash 60
[MissionProcessor  28100 us] frame 1: window closed at 8000 us with A=True B=False
[MissionProcessor  28300 us] frame 1: command hash_a=60 hash_b=23 degraded=0 (b from frame 0)

Frame 1 ships hash_b=23, which is frame 0's truth, where 60 was due, and calls itself complete. Every frame lags exactly one behind on Vendor B.

Frame 0 is the tell. It degrades honestly, because there is nothing stale to reach for yet. After that the defect hides itself, so a short bench run that only ever looks at the first frame reports this system healthy.

What the campaign concluded

The freshness defect is fixable in software. The deadline at maximum target load is not, by either candidate fix.

Every fix run clears PR-03 and PR-04 with zero silently stale messages, so both candidates cure the defect the baseline shipped. But 2 of the 6 fix runs miss PR-01:

• fix1-window-4t at 18900 us against a 18000 us budget, over by 900 us.
• fix2-earlypartial-4t at 18300 us against a 18000 us budget, over by 300 us.

Both failures are the four-target case, and the two fixes differ by 600 us on an 18000 us budget, which is noise next to the problem.

Read that as the problem MOVING rather than surviving. Both fixes eliminate the defect they were aimed at; both hand the system a different failure in its place. That is the ordinary experience of integration, and it is why "we fixed it" is a weaker claim than it sounds: the honest version is "we moved it, and here is where to, and here is what that costs".

It also marks the limit of designing fixes by hand. Two candidates, chosen by people who understood the system, both landed on the same trade and neither found a way around it. The parameters that matter are not independent: the collection window, the vendor's emission mode, the parse cost per fragment, the frame cadence and the network's contention all interact, and there are more combinations than anyone will try by intuition. That points at a system decision rather than a patch: pipeline the parse so it overlaps reception, relax the cadence, or negotiate the four-target case with the vendor. The twin can price all three before anyone commits, which is the whole argument. A lab would have reached the same conclusion in week 30, with hardware.

Worth saying plainly, because it is the reason to run the twin rather than reason about it: the design's own arithmetic predicted BOTH fixes would pass at four targets, one of them with 2.6 ms to spare. Both miss. The prediction was optimistic in the same direction twice, and it was written by the people who then built the thing. The measurement is the deliverable.

What the twin does not model

The Ethernet is modelled, not transported. VSI's tcpIp gateway is hub and spoke with at most two ports per socket and cannot carry a five-endpoint switched network, so the switch's serialization, store-and-forward and per-port queueing are computed inside the RuggedizedSwitch component and carried over GenericPayload signals. The queueing physics is real and every parameter comes from the model; the wire protocol is not. Both RF interfaces are marked modelled=0 in the model itself: detections are injected past the receivers' RF front end and the emission is timestamped at the array rather than propagated. The defect found here is a timing and freshness defect, and timing is exactly what this fidelity level does model; a defect that depended on the wire format would need a different twin.

Reproducing it

python cameo2vsi.py IMS_Coverification_Demo.mdzip --out ims
python run_ims.py bootstrap
python run_ims.py run --name baseline-2t --targets 2 --window 8000 --defect 1
python run_ims.py campaign --targets-list 1 2 4

bootstrap is the only slow step. Every scenario after it is a Python behaviour change and a fresh vsiSim run with no rebuild, because the scenario knobs live in ims.scenario.json and the behaviours read it. That is what makes a sweep affordable, and it is the contrast worth drawing: the lab equivalent of one of these rows is a hardware change.