Client Testing
Purpose¶
Standardized client test requirements across all 3 repos (api, websocketmanager, workmanager) and all 3 languages (Python, TypeScript, .NET). Every repo SHALL have the same set of client tests in every language, ensuring SDK parity and preventing regressions.
Workflow¶
How test parity is maintained¶
proto file ──► generator ──► checker ──► CI gate
- Generator (
virtufin-common/scripts/generate-client-tests.py): Reads a.protofile and emits test files in Python, TypeScript, and .NET with test-case ID comments. Run after proto changes. - Checker (
virtufin-common/scripts/check-client-test-parity.sh): Scans all 3 repos for ID comments and reports gaps against this spec. Run locally or in CI. - CI (
virtufin-common/.github/workflows/test-parity-common.yaml): Weekly + on-demand parity gate.
Running the generator¶
# The repos are NOT a monorepo, so the script is fetched
# directly from the virtufin-common Gitea repo at runtime.
cd virtufin-<service>
python3 <(curl -L https://git.haenerconsulting.com/virtufin/virtufin-common/raw/branch/master/scripts/generate-client-tests.py) \
src/Virtufin.<Service>.Protos/proto/<service>.proto \
--prefix <X>
The generator is idempotent. Run it whenever the .proto file changes.
Running the checker¶
cd virtufin # directory containing all service repos
VIRTUFIN_ROOT=. bash virtufin-common/scripts/check-client-test-parity.sh
Exits 0 if all IDs are present, 1 if gaps exist.
Generated vs hand-written tests¶
- Generated (
tests/python/test_generated.py,tests/typescript/generated.test.ts,tests/...Client.Tests/GeneratedTests.cs): Structural coverage from.proto. Do not edit — regenerated on proto changes. - Hand-written (
tests/python/test_*.py,tests/typescript/*.test.ts,tests/...Client.Tests/*Tests.cs): Business logic, patterns, integration. ID-commented for parity.
Both carry test-case IDs. Generated tests provide baseline coverage; hand-written extend it.
Test ID namespaces¶
The spec test IDs (e.g. A11, W18, M30) are only for hand-written tests. They correspond to the rows in the per-service tables below. The generator emits its own test IDs in a separate namespace so that generated structural/stub tests don't collide with the spec IDs:
| Service | Hand-written prefix | Generated prefix |
|---|---|---|
| virtufin-api | A |
GA |
| virtufin-websocketmanager | W |
GW |
| virtufin-workmanager | M |
GM |
The generator defaults --gen-prefix to G + --prefix (so --prefix A → --gen-prefix GA). Override with --gen-prefix <X> if needed.
The parity checker (check-client-test-parity.sh) only validates the hand-written IDs against the spec — generated test files are scanned but their GA* / GW* / GM* IDs don't match the \b(A|W|M)[0-9]+\b pattern, so they're correctly excluded.
virtufin-api Client Test Cases¶
The previously specified A1-A10 and A12-A14 (constructor/package/method/integration tests) are deferred. They described an aspirational set that was never implemented in hand-written code; the generator now provides structural coverage via the GA-prefix test IDs in test_generated.py / generated.test.ts / GeneratedTests.cs (see "Test ID namespaces" above). Only A11 is currently hand-written.
| # | Category | Test Case | Python | TypeScript | .NET |
|---|---|---|---|---|---|
| A11 | Integration | Connectivity check | ✓ | -- | -- |
virtufin-websocketmanager Client Test Cases¶
| # | Category | Test Case | Python | TypeScript | .NET |
|---|---|---|---|---|---|
| W1 | Constructor | Default host/port | ✓ | ✓ | ✓ |
| W2 | Constructor | Custom host/port | ✓ | -- | -- |
| W3 | Constructor | All expected methods present | ✓ | ✓ | ✓ |
| W4 | Package | Proto types importable | ✓ | ✓ | ✓ |
| W5 | Package | gRPC client stub accessible | ✓ | ✓ | ✓ |
| W6 | Package | Transport dependency available | -- | ✓ | ✓ |
| W7 | Package | ConnectRequest schema | -- | ✓ | ✓ |
| W8 | Package | ConnectResponse schema | -- | ✓ | ✓ |
| W9 | Package | ListRequest schema | -- | ✓ | ✓ |
| W10 | Package | ListResponse schema | -- | ✓ | ✓ |
| W11 | Package | DisconnectRequest schema | -- | ✓ | ✓ |
| W12 | Package | SendRequest schema | -- | ✓ | ✓ |
| W13 | Package | SendRequest fields (id, message, timeout) | -- | -- | ✓ |
| W14 | Package | SendRawRequest schema | -- | ✓ | ✓ |
| W15 | Package | StartPublishRequest schema | -- | ✓ | ✓ |
| W16 | Package | StopPublishRequest schema | -- | ✓ | ✓ |
| W17 | Package | WebSocketConnection schema | -- | ✓ | ✓ |
| W18 | Method | Connect delegates to stub | ✓ | ✓ | -- |
| W19 | Method | List delegates to stub | ✓ | ✓ | -- |
| W20 | Method | Disconnect delegates to stub | ✓ | ✓ | -- |
| W21 | Method | Send delegates to stub | ✓ | ✓ | -- |
| W22 | Method | Send with default timeout | ✓ | -- | -- |
| W23 | Method | SendRaw delegates to stub | ✓ | ✓ | -- |
| W24 | Method | StartPublish delegates to stub | ✓ | ✓ | -- |
| W25 | Method | StopPublish delegates to stub | ✓ | ✓ | -- |
| W26 | Method | Close releases channel | ✓ | -- | -- |
| W27 | Error | Connect error propagates | ✓ | ✓ | ✓ |
| W28 | Error | Send error propagates | ✓ | ✓ | -- |
| W29 | Error | List error propagates | ✓ | ✓ | -- |
| W30 | Lifecycle | GrpcChannel construction | -- | -- | ✓ |
| W31 | Lifecycle | Transport creates successfully | -- | ✓ | -- |
| W32 | Lifecycle | Package exports index entry | -- | ✓ | -- |
| W33 | Method | SetTag delegates to stub | ✓ | ✓ | -- |
| W34 | Method | GetTag delegates to stub | ✓ | ✓ | -- |
| W35 | Method | SetTags delegates to stub | ✓ | ✓ | -- |
| W36 | Method | GetTags delegates to stub | ✓ | ✓ | -- |
virtufin-workmanager Client Test Cases¶
| # | Category | Test Case | Python | TypeScript | .NET |
|---|---|---|---|---|---|
| M1 | Constructor | Default host/port | ✓ | ✓ | ✓ |
| M2 | Constructor | Custom host/port | ✓ | -- | -- |
| M3 | Constructor | All expected methods present | ✓ | ✓ | ✓ |
| M4 | Package | Proto types importable | ✓ | ✓ | ✓ |
| M5 | Package | Proto enums accessible | -- | ✓ | ✓ |
| M6 | Package | gRPC client stub accessible | ✓ | ✓ | ✓ |
| M7 | Package | Transport dependency available | -- | ✓ | ✓ |
| M8 | Package | CreateWorkerRequest schema | -- | ✓ | ✓ |
| M9 | Package | ListWorkersRequest schema | -- | ✓ | ✓ |
| M10 | Package | DeleteWorkerRequest schema | -- | ✓ | ✓ |
| M11 | Package | StartWorkerRequest schema | -- | ✓ | ✓ |
| M12 | Package | StopWorkerRequest schema | -- | ✓ | ✓ |
| M13 | Package | RecoverWorkersRequest schema | -- | ✓ | ✓ |
| M14 | Package | LoadCodeRequest has an Id property | -- | ✓ | ✓ |
| M15 | Package | LoadCodeRequest has a CodeSource property | -- | ✓ | ✓ |
| M16 | Package | WorkerInfo schema | -- | ✓ | ✓ |
| M17 | Package | CodeSource schema | -- | ✓ | ✓ |
| M18 | Package | WorkerStatus enum | -- | ✓ | ✓ |
| M19 | Method | CreateWorker with URL | ✓ | ✓ | -- |
| M20 | Method | CreateWorker with content | ✓ | -- | -- |
| M21 | Method | CreateWorker without connect raises | ✓ | -- | -- |
| M22 | Method | ListWorkers delegates to stub | ✓ | ✓ | -- |
| M23 | Method | DeleteWorker delegates to stub | ✓ | ✓ | -- |
| M24 | Method | StartWorker delegates to stub | ✓ | ✓ | -- |
| M25 | Method | StopWorker delegates to stub | ✓ | ✓ | -- |
| M26 | Method | RecoverWorkers delegates to stub | ✓ | ✓ | -- |
| M27 | Method | GetWorkerHistory delegates to stub | ✓ | ✓ | -- |
| M28 | Obsolete | ~~LoadCodeFromContent delegates to stub~~ — LoadCode consolidates both variants behind one RPC (see M3); no separate LoadCodeFromContent stub method exists to delegate to. |
-- | -- | -- |
| M29 | Obsolete | ~~LoadCodeFromUrl delegates to stub~~ — same consolidation as M28. | -- | -- | -- |
| M30 | Error | CreateWorker error propagates | ✓ | ✓ | ✓ |
| M31 | Error | ListWorkers error propagates | -- | ✓ | -- |
| M32 | Error | DeleteWorker error propagates | -- | ✓ | -- |
| M33 | Lifecycle | GrpcChannel construction | -- | -- | ✓ |
| M34 | Lifecycle | Transport creates successfully | -- | ✓ | -- |
| M35 | Lifecycle | Package exports index entry | -- | ✓ | -- |
| M36 | Obsolete | ~~SetEnvironmentVariable delegates to stub~~ — the custom worker environment-variable RPC was removed entirely (process-wide state, unsafe across co-loaded workers); config now flows through the CloudEvent payload instead. | -- | -- | -- |
| M37 | Obsolete | ~~GetEnvironmentVariable delegates to stub~~ — same removal as M36. | -- | -- | -- |
| M38 | Obsolete | ~~SetEnvironmentVariables delegates to stub~~ — same removal as M36. | -- | -- | -- |
| M39 | Obsolete | ~~GetEnvironmentVariables delegates to stub~~ — same removal as M36. | -- | -- | -- |
Rules for Adding Tests¶
- Every test case listed above SHALL exist for every language marked ✓.
- When adding a new test to ANY language, a corresponding row SHALL be added to this spec.
- The CI parity checker SHALL fail if gaps are detected.
- Test naming SHALL follow these conventions so the checker can match them:
| Language | Pattern | Example |
|---|---|---|
| Python | test_<snake_case_name> |
test_connect_delegates_to_stub |
| TypeScript | it("<human readable name>") |
it("connect delegates to gRPC client") |
| .NET | CamelCaseName |
ConnectDelegatesToStub |
Each test case ID (e.g., W18, M19) MUST appear as a comment in ALL language implementations to enable automated matching:
# W18
def test_connect_delegates_to_stub(self, client, mock_stub):
...
// W18
it("connect delegates to gRPC client", async () => {
...
});
// W18
[Fact]
public void ConnectDelegatesToStub()
{
...
}