Broker Test Harness
What the harness does
The ASAP Broker Test Harness stands in for an alarm monitoring company's message broker, so an APIEngine installation can be tested without a real broker and without an emergency call center taking part. It listens for the messages APIEngine sends, answers each one the way you choose, sends broker messages of its own back to APIEngine, and keeps a list of checks that turns green or red as each conversation happens.
ASAP is the arrangement by which an alarm monitoring company and an emergency call center exchange new alarms, address checks, updates and cancellations through a message broker. Everything a test run sends stays between the harness and the APIEngine installation you point it at.
What you need
- A Windows machine that can reach the APIEngine installation under test, and that APIEngine can reach in return.
- The web address APIEngine itself answers on.
- Permission to change APIEngine's
MessageBrokerRestUrlsetting - that setting is what points APIEngine at the harness instead of a real broker. - Connection details for the SBN data server holding the alarm records: server type, host, port, database, user name, password. The harness reads address records from there, and checks that a message it sent changed what it should have changed.
- A server certificate file (.pfx) and its password, if the run is to include HTTPS.
Installing
The download is a zip holding the application, sample messages and an installer pair. Unpack it and run the installer from a PowerShell prompt:
To remove it, run uninstall.ps1 the same way. Either script answers -help without changing anything on the machine. Running the installed application with no arguments opens the window described below.
The window
Five tabs, worked through left to right. The first four are numbered steps; the fifth is the record of what happened.
| Tab | What it is for |
|---|---|
| Connectivity | The SBN data server connection, and the stand-in broker APIEngine will send its messages to. |
| Inbound | Broker messages you send to APIEngine. |
| Responses | The answer the stand-in broker gives to whatever APIEngine sends next. |
| Checklist | What has been tested, and how it went. |
| Log | Every line the harness wrote, as it is written to its own log file. |
Step 1 - Connectivity
SBN data server
Fill in the connection to the data server the alarm records live in: server type, host, port, database, a second database if you have been told to use one, user name, password, character set, and how many seconds to wait before giving up. The path of the APIEngine settings file these details normally come from is shown beside them as a label; nothing on this tab changes that file.
Test connection answers in one sentence. A run works without this connection, but the harness can then only say that APIEngine accepted a message, never that it acted on it - which reads as NOT VERIFIED on the checklist and is never a pass.
Harness listener
- Choose the HTTP port the stand-in broker should listen on.
- Press the play button. The box beside it turns green while the broker is listening and red while it is stopped; the pause button stops it again.
- Copy the address shown and paste it into APIEngine's
MessageBrokerRestUrlsetting, then restart APIEngine so it picks the setting up. The address is given for this machine; from another machine, use this machine's name or address with the same port. - Type the web address APIEngine answers on, and press Test beside it. The harness calls APIEngine and says whether it answered, whether something else answered, or whether nothing could be reached.
Tick HTTPS cert to add a secure port. The secure address is not in use until a certificate is chosen; until then the harness listens on plain HTTP only.
Step 2 - Inbound
This tab sends the messages a real broker would send to APIEngine - the confirmations, rejections and updates that come back from an emergency call center.
- Pick a message. Sixteen are offered, from Address ACCEPT and Alarm REJECT through the call center's CADUPDATE messages and cancellations, and two that are malformed on purpose.
- Fill in what that message needs: the activity id of the alarm or address it is about, and, depending on the message, a map position, the incident number the call center filed it under, or a reason code.
- Check the preview, press Send, and read the answer. The harness reports what APIEngine answered and, where the data server connection is set up, whether the change the message should have made is there.
Further down, Operator message for an open alarm sends the free text a call center operator would type against an alarm that is already open, and the two cancel buttons send a cancellation and its confirmation for that same alarm. Underneath them, the conversation for the chosen alarm is listed oldest first, so both sides of it can be read in one place.
A message the harness sends is judged as a pass only when APIEngine both accepted it and the expected change shows in the data. The two malformed messages are the other way round: being refused is the right answer, and the only pass.
Step 3 - Responses
Pick the answer the stand-in broker gives from the next message onwards, then make APIEngine send something and watch how it copes.
| Answer | What APIEngine meets |
|---|---|
| Accept | The message is checked the way a real broker checks it, then confirmed. |
| Reject with a reason | The message is turned down with the reason code chosen here. |
| Refuse - not authenticated | The sender is not recognised as signed in. |
| Refuse - not recognised | The sender is not allowed to send it. |
| Refuse - broker error | The broker reports a fault of its own. |
| Hold, then answer | The answer is deliberately slow, held for the seconds you set. Most systems give up at 60. |
| Malformed body | The reply is unreadable text rather than a message. |
| Malformed confirmation | The reply looks like a message but is not a valid confirmation. |
| Connection drop | No reply at all - the connection closes mid-conversation. |
The answer can also be set for a fixed number of messages, after which the broker goes back to accepting. Set that count to zero to stay on the chosen answer.
Step 4 - Checklist
Nothing is pressed here. Thirty-six checks sit in five groups, and each one changes state the moment the harness sees that conversation:
- Messages APIEngine sent to the broker - its connection check, a new alarm, an address to check, an update and a cancel request.
- Answers the broker gave, and how APIEngine coped - one per answer on the Responses tab.
- Connections APIEngine used to reach the broker - plain HTTP, HTTPS, and HTTPS with a client certificate.
- Messages the broker sent to APIEngine - one per message the Inbound tab can send.
- Connections used to reach APIEngine - the same three, in the other direction.
Select a check to see everything recorded behind it: what arrived, what was answered, what the data showed, and what APIEngine did next. One line above the list counts how many checks passed, failed, could not be verified and have not run. Save report writes the list and all of its detail to a file for the record.
The Log tab
Every line the harness writes goes to a log file in a log folder beside the program, one file per day. The tab shows the end of that file, newest lines at the bottom, and names the file it is reading. The file is the record - it is there after the harness is closed, and can be sent on when a run needs explaining.
Certificates
Over HTTPS the harness is the server: APIEngine connects to it. The certificate chosen in the HTTPS section is therefore the harness's own server certificate, and the Test button beside it says who it is for and how long it is valid.
A client certificate is the one APIEngine presents when it connects. The harness accepts any client certificate, and accepts a connection with none, so nothing has to be set up on this side for that. Which of the three was used is recorded against the connection checks, so a run can show that APIEngine really did present its certificate.
What a result means
| Result | Meaning |
|---|---|
| PASS | The conversation happened and everything behind the check was right. |
| FAIL | Something behind the check was wrong. The detail says which part. |
| NOT VERIFIED | APIEngine accepted the message, but nothing could confirm it acted on it - usually no data server connection, or a change this data does not record. |
| CHECKING | The harness is still watching for what APIEngine did with the answer. |
| Not run yet | That conversation has not happened. Nothing has failed. |
A check is the worst of the detail behind it, so one bad cell holds the whole check down. A check that is red or unverified stays that way until the conversation is repeated and goes right.
When a check goes red
| What you see | What to do |
|---|---|
| Nothing arrives from APIEngine at all | Check that the broker address was pasted into APIEngine's MessageBrokerRestUrl setting, that APIEngine was restarted afterwards, and that the state box is green. Anything in between the two machines has to allow the port. |
| The Test button beside the APIEngine address cannot reach it | Check the address, that APIEngine is running, and that nothing between the two machines is blocking the port. |
| A message you sent was refused or answered with an error | Read the check's detail: it carries what APIEngine answered. Fix the alarm or address the message names, or pick an activity id that exists. |
| NOT VERIFIED on message after message | Set up the SBN data server connection on the Connectivity tab and press Test connection. Without it the harness cannot look at the data. |
| The HTTPS checks never turn green | Tick HTTPS cert, choose the certificate file and its password, press Test, and point APIEngine at the secure address rather than the plain one. |
| A cancellation check reads FAIL | The system under test may not handle cancellations yet. The check's own detail says so where that is the case. |
The Log tab holds the full sequence behind any of these, and Save report on the Checklist tab exports what was tested, ready to attach to a service request.