Troubleshooting
The broker does not answer
A product installer reports:
The service is running but the broker is not answering. This happens when the broker was restarted a few seconds earlier, or when it is stuck.
- Restart the service: on Windows run
Restart-Service innovative-natsfrom an elevated PowerShell window; on Linux runsudo systemctl restart innovative-nats. - Wait ten seconds, then run the product installer again.
- If it still fails, read the broker log (see Where the logs are below).
The service does not start
The broker writes the reason to its own log, nats-server.log, before it stops (see Where the logs are below). When the log names a file in conf\site.d, correct or remove that file. Then run the innovative-nats installer and choose repair this instance. Repair regenerates the configuration, restarts the service and runs the checks again.
The cluster has only one server
A product installer reports:
The broker log repeats this line every few seconds:
Message storage in a cluster is shared across its servers and stays unavailable until at least one other server has joined. Choose one:
- Add the second server. Install innovative-nats on another server and choose to join the existing cluster. The first server starts accepting work as soon as the second one joins.
- Run this server on its own. Run the innovative-nats installer again on this server and choose
remove this instance from its cluster. Products on this server can use it straight away.
On the first server of a new cluster, the installer's own storage check reads [SKIP] waiting for a second server to join until another server joins. That line is expected.
Joining a cluster fails
Before it changes anything, a joining server checks four things and prints a [PASS] or [FAIL] line for each. It then asks again for only what failed.
| Failed check | Cause and fix |
|---|---|
<server>:<client port> answers | The address is wrong, the server is down, or a firewall blocks the client port. Check the address, and on the existing member confirm the firewall rule for its client port |
the broker password is accepted | The password does not match. On the existing member, run its installer and choose show the broker password |
cluster '<name>' read from <server> | The server answers but no cluster was set up on it. Run its installer, choose start a new cluster with this server as its first member, then give its address again |
cluster port <server>:<cluster port> answers | A firewall blocks the cluster port (default 6222). Open it on the existing member |
After three failures the installer offers [r] try again [s] install this server standalone instead [a] abort.
A port is already in use
The installer stops with exit code 4 when a program other than nats-server is listening on a port it needs. It never moves the broker to another port on its own. Stop that program, or run the installer again and choose a free port at the port question.
The installer cannot use the connection file folder
The connection file folder is C:\ProgramData\Innovative\innovative-nats. The installer checks that it can create, write and lock this folder before it asks its first question, and stops with exit code 1 naming the path and the cause when it cannot. Run the installer from an elevated PowerShell window. If the folder's permissions were changed by hand, reset them from an elevated prompt:
Then run the installer again.
No internet access during a Windows install
The installer downloads the pinned nats-server release from github.com. When the download fails, it names the file and its address. Download that file on another computer, copy it into the payload folder beside install.ps1, and press Enter at the installer's prompt. The file is checked against its expected SHA-256 hash before it is used.
The uninstaller refuses to remove the broker
The uninstaller stops with exit code 21 and names the products that are still registered with the broker. Uninstall those products first. When a product was removed without its uninstaller, add -IgnoreConsumers (Linux --ignore-consumers) to remove the broker anyway.
Installer exit codes
| Code | Meaning |
|---|---|
| 0 | Success, a clean abort, or nothing to do |
| 1 | A package file is missing, or the connection file folder cannot be used. The output names which |
| 2 | Unsupported operating system, or a question was needed and no console was available |
| 3 | Not run with administrator rights |
| 4 | A program other than nats-server is using a port the broker needs |
| 10 | An option was used incorrectly |
| 12 | A check failed after the install or during a verify. The output names the failing check |
| 13 | Taking over an existing broker failed and was rolled back. The previous broker is running again |
| 14 | An unattended cluster join failed |
| 20 | The uninstaller's check after removal failed |
| 21 | The uninstaller refused because products are still registered with the broker |
Where the logs are
| Log | Windows | Linux |
|---|---|---|
| Broker log | <install folder>\logs\nats-server.log (for example D:\Innovative\innovative-nats\logs\nats-server.log) | /var/log/innovative/innovative-nats/nats-server.log |
| Installer run logs | logs\install-<date>-<time>.log and logs\uninstall-<date>-<time>.log beside the installer | logs/ beside install.sh |
The last lines of the broker log name the reason the broker stopped or is waiting. The broker rotates its log at 10 MB and keeps five files.