Debugging Overview

If there are issues that require Heimdall support assistance, please follow the following directions (as appropriate)--please note, most debugging should be done in low-traffic scenarios for best results.

If a packet trace is requested, please follow one of the next two steps as appropriate:

Using tcpdump (on most Unix-like system):

tcpdump -iany -s0 -wcapture.pcap tcp port <proxy port> or tcp port <database port>

If both ports are the same, i.e. 1433 for SQL Server, then only one port is necessary, i.e.:

tcpdump -iany -s0 -wcapture.pcap tcp port <proxy port>

Note: capture.pcap will be the filename saved--it may be necessary to download this using a tool such as scp, winscp (https://winscp.net/eng/download.php), or filezilla (https://filezilla-project.org/) in order to e-mail this file

Heap Dumps

At times, if there is a memory utilization issue, Heimdall support may ask for a live or full heap dump of the server or the proxy. Here are the steps to perform this:

  1. Connect to the heimdall instance command line as root, typically via Putty or SSH
  2. use "ps -ef | egrep "heimdallserver.jar|heimdalldriver.jar" to find the pids of the processes
  3. For a full dump, use the command: "jmap -dump:file=heap-full.hprof "
  4. For a life dump, use the command: "jmap -dump:live,file=heap-live.hprof "
  5. Download and provide the files to heimdall support as appropriate based on size, i.e. via a file transfer site, or e-mail

The difference between a life and full dump is that a live dump only shows objects that are not pending cleanup. Both may be requested, so we can see what memory is being taken by objects yet to be cleaned up.

Stack Trace

If a Heimdall process is getting "stuck" at any point, which could be from a logic deadlock or similar issue, a stack trace can be used to find where in the code the lock is happening. To gather a stack trace, here are the steps:

  1. Connect to the heimdall instance command line as root, typically via Putty or SSH
  2. Use "ps -ef | egrep "heimdallserver.jar|heimdalldriver.jar" to find the pids of the processes
  3. Use "jstack > stack.txt"
  4. Download and provide the file to heimdall support