Request Rules options:
This is list of options for request rules:
These options are saved inside rules configuration file. It contains both request and response rules. The filename depends on the name of the rule list → nameOfRuleList.conf.
These are stored in the following format under rules:
"rules": [
{
"enabled": true,
"type": "K",
"patterns": [
"MultiplexRegexHere"
],
"rowPatterns": [
""
],
"operator": "AND",
"columnNameOperator": "AND",
"intrans": true,
"properties": {
"action": "enable"
},
"notes": "Notes here"
},
]
Each rule might have: * enabled - Boolean to indicate if the rule is active. * type - Action of the rule inside the configuration file, such as Q → Allow. * Patterns - The Java regex for queries to match on. Regex column from GUI. * rowPatterns - Used by response rules. * intrans - Determine if rule matches in transactions or only on auto-commit mode. * properties - Parameters and their values for given rule. * notes - Extra notes that can be left, their presence is marked by message icon next to enabled option. Editable after clicking edit button.
⚠️ Note: You can use SHIFT with mouse scroll to scroll horizontally!
| Key | Type | Description |
|---|---|---|
| allow | Q | Explicity allow a query to bypass the learn and drop rule types |
| async execute | A | Execute insert/update/delete asynchronously, not supported by MySQL proxy |
| cache | C | Use the cache configuration for the VDB to cache matching queries, if possible |
| call | G | Execute (call) a named rule list, must also be included in the VDB config to work |
| debug | Z | Allows to enable and disable debug mode for a connection |
| drop | D | If not allowed with an allow rule, drop the request, and generate SQL Exception |
| extract plan | O | Automatically (periodically) retrieve query plan on queries |
| forward | F | Allows queries to be sent to alternate data sources for execution |
| ignore | I | Ignore rules of the specified type in the current rule-list that occur after this rule |
| learn pattern | P | Generate a firewall learn action to add a rule to a named rule list. A new rule will not be added if one with the same pattern and action type already exists. |
| log | L | Log the SQL observed in the query |
| multiplex | K | Manage multiplex behavior (works only when the multiplex is enabled in VDB config) |
| nocache | B | Disable caching for a query, overrides previous cache rules as well as prevents later cache rules from being evaluated |
| pool | M | Pool and connection multiplexing behavior overrides |
| procedure wrap | 1 | Wrap query into a procedure on proxy side (only for SQLServer). |
| reader eligible | V | Flag that the query is allowed to be processed by a read-only server in a cluster. |
| result | N | Return a (fixed) result |
| retry | E | Retry execution if exception occurs |
| stack trace | S | Generate a Java stack trace in logs on match |
| table cache | W | Apply a TTL to a query based on if the table specified is referenced by the query. Lower TTL values override higher ones |
| tag | X | A no-op action, for use with universal parameters only |
| transform | T | Change the query based on capture group manipulation |
| trigger | R | Trigger another SQL or script to execute as a result of a matching query |
Parameters for rules
General parameters:
| Key | Description |
|---|---|
| delay | Adds X microseconds to each query, to help model the impact of latency between the db client and server, for hybrid cloud impact testing |
| evicttables | Tables that should be flagged as evicted when a query matches the rule |
| exclude | Exclude matching the rule 1/X times, to allow a small sample of queries to match |
| invalidate | Can be specified to override invalidation behavior of writes for matching queries, default is to allow invalidations to occur. |
| log | When specified, behave as if a log rule had matched, used to consolidate rules |
| matchdelta | Specifies the minimum time between matches on a per-proxy basis, overrides matchlimit |
| matchlimit | Specifies the frequency (in X per second) that a rule should match |
| matchthreshold | Only match to unknown queries or ones that are known to have an average runtime longer than X microseconds |
| maxburst | When using matchlimit or ratelimit, specify the number of seconds worth of bursting to allow before limiting the rate |
| notify | If specified as true, flag that the notification configuration should be used to notify on a match |
| notrack | When specified, don't track the performance of this query for statistics records. Useful to "hide" overhead queries that aren't impacted by performance |
| onlyTrans | When specified, only match queries that are in transaction (must have In-Trans checked as well) |
| olderthan | specify the time window in ms to allow content to not be evicted on eviction. Negative prevents caching into the future |
| preparedonly | Match only if a query is a prepared query. This can be used to trigger exceptions to change settings for users to optimize pooling behavior |
| printcapture | When a rule matches, print capture groups of the rule match |
| printmatch | When a rule matches, print the rule that matched |
| printmeta | Print the resultset metadata for results returned |
| printresults | Prints to logs the result of matching queries |
| printtables | If specified, print the table attributes to be used for caching & invalidation |
| printtiming | Prints to logs the execution time of this rule, useful for profiling called rulelists, such as for the SQL Firewall |
| ratelimit | Limits matching queries to X per second. This is tracked on a per-rule basis, so ratelimit and matchlimit can interact on the same rule |
| rowlimit | Set the connection to be limited to X rows per result-set as if the command 'set maxrows=x' had been made |
| sample | Match the rule only 1/X times, often to reduce frequency of logging, but remain proportionate to load |
| serverfilter | A filter (regex) to determine what read-only servers should be used to handle a request. If none match, this is ignored. Only used with forwarding & read/write split |
| stop | If specified on a matching rule, no further rule processing in the current rule list will occur |
| tables | For any matching rule, specifies the tables that should be added to a query's table list. Use "none" to allow caching without tables. Newlines and whitespace are acceptable between table names, but not in table names. |
| update | Specifies if the matching queries should be processed as updates by the cache engine |
| userReset | When executing a query, remove the user's credentials to use the default source user's connection pool and permissions |
| resultQuery | When executing a query, use an alternate query to retrieve the actual result, i.e. from a temp table |
| resultQueryTrans | When using resultQuery, should the initial query and the result query be wrapped together in a transaction? |
| serverFilter | If average runtime will be longer than given 'matchthreshold', then query will be forwarded to data source defined by 'serverfilter'. |
| queryTimeout | Query execution timeout |
| honorComments | The comments in the queries at the regex level are ignored by default. Use this flag to preserve them. Note: Enable it if you want to use the Capture Parameter. |
Rule Specific
For Async Execute:
| Key | Description |
|---|---|
| asyncsize | The maximum length of a query (in characters) that can be made asynchronous |
| batchSize | The maximum batch size of queries to be executed--queries will not be delayed to fill the queue, but this sets the maxiumim size of |
| maxPollInterval | The maximum time (in ms) to poll after an async query is received before a batch is processed, defaults to 0 for no delay the batch. |
| source | The name of the data source to forward the query to for async execute. This source must be specified in the VDB definition. |
| queueName | The per-source named async queue to process the query from. Defaults to the name of the data source the query is sent to. Set to ${table} to allow dynamic replacement of the table name (if only one). |
| holdUntil | When to hold the thread until |
| spoofedResult | The result the DML will return if the result is spoofed (only for immediate holdUntil values). |
| lockTable | Lock tables of async DML while the DML is pending, to serialize access to the table (default to TRUE) |
For Cache
| Key | Description |
|---|---|
| ttl | Specifies the TTL in ms of objects cached, if not evicted due to a write first |
| cachesystem | True if Heimdall should cache system tables, and not just user tables, can break things! |
| unconditional | if true, bypass many internal checks to insure a query is cacheable |
| maxLocalSize | Specify the largest size of a cachable object in local memory, defaults to Integer.MAX_VALUE |
| maxSize | Specify the largest size of a cachable object, defaults to Integer.MAX_VALUE |
| autoRefresh | Specify that queries may be automatically refreshed in cache if invalidated (but not on TTL expiration). Only a limit number of queries are tracked for this purpose, configurable on the vdb properties |
| revalidateOnly | Don't actually serve from cache, but validate that the result in the cache is the same as what the server serves |
| shared | Share a cache object across users. Note, this may bypass security, so should be used with caution. To globally trigger this, use the custom cache key option in the vdb cache settings. |
For Result
| Key | Description |
|---|---|
| responseInt | Return an integer value as a result of a SQL call, i.e. for rows updated |
| file | Return a file containing json result |
For Retry
| Key | Description |
|---|---|
| maxRetries | Maximum number of attempts that will be made if none of them succeeds |
| exception | String that exception should contain |
| retryDelay | Delay between retry attempts |
| query | The sql to execute before attempting to retry |
For Forward
| Key | Description |
|---|---|
| source | The name of the data source to forward the query to. This source must be specified in the VDB configuration. |
| setcatalog | If in forwarding, should we inherit/set the catalog from the parent connection on the forwarded connection. |
| setschema | If in forwarding, should we inherit/set the schema from the parent connection on the forwarded connection. |
| readonly | When forwarding, should the connection be allowed to read-only servers. |
| sticky | Continue forwarding to the designated source in an ongoing basis, and disables multiplexing so the same connection is reused. |
For Call
| Key | Description |
|---|---|
| rule | The rule name to call/execute in place of the call rule itself, if matched. |
For Ignore
| Key | Description |
|---|---|
| action | What type of action to ignore after matching |
For Log
| Key | Description |
|---|---|
| logcolumns | List of comma separated column names/aliases to save in log records, based on the resultset metadata. Use '*' or 'all' for all columns. Note: HIGHLY impacting on performance. |
| slowtime | Response time in microseconds above which logging should be triggered. |
For Extract Plan
| Key | Description |
|---|---|
| threshold | The response time threshold in ms after which the plan is requested, based on query pattern average response times |
| frequency | How many times the query plan can be requested per hour |
| format | What format to use when extracting an execution plan |
| force | By default Extract Plan works only for queries with result set from DB (for example, SELECT queries). To force explaining on any queries set the parameter to true. |
| checkTableMismatch | For SQL Server, check if tables in the explain match what is tracked internally by Heimdall |
For Learn Pattern
| Key | Description |
|---|---|
| rule | The name of the target rule list to add learned rules to. |
| action | Specifies the action of newly added rules, defaults to "allow". |
For Trigger
| Key | Description |
|---|---|
| type | Specifies the type of trigger action to perform. If execute, then it will call the named program or script, and execute on the central manager (always). If SQL, it will execute on the proxy/driver unless in parallel, delay and queueunique is TRUE, then it will process on the central manager. |
| command | The command to execute, either as sql or as a script. Accepts a replacement value of ${tables} to represent the tables in the query. |
| timing | When to execute the command, either before, after, or in parallel to the triggering sql. With before, if an exception is encountered on the initial query, the trigger action is not performed. If an exception is encountered with after, then the exception will be reported to the calling thread. When sql is set to execute in parallel mode, it will execute on the central manager if delayed, on the proxy/driver if not, and exceptions will only be reported in the logs. |
| delay | When running in parallel to the triggering sql, how much time to delay processing (in ms), if any |
| maxDelay | When running in parallel, delayed and queueunique, how often should we run (at least) even if the trigger action keeps occurring, i.e. to insure data inserted is flushed to another db at least X times an hour |
| queueUnique | When performing delayed parallel queries, should the execution queue only maintain unique command executions, to prevent the same command from executing over and over later? |
| source | The name of the data source to execute a SQL query on. This source must be specified in the VDB configuration. |
| onceOnly | Only do this trigger once during the lifetime of the proxy (or a reasonably long period). The call should be able to be triggered multiple times over time, this should be considered an optimization only. |
For Stack Trace
| Key | Description |
|---|---|
| threshold | Response time threshold for stack traces to trigger (in microseconds) |
For Transform
| Key | Description |
|---|---|
| target | When translating a query, the target of the translation, with ${1} representing the first capture group, ${2} the second, etc. |
| recursive | If a transformation should be repeated until no change is made between original and transformed query |
| escapeQuotes | If single quotes in the capture group should be escaped with '' |
For Table Cache
| Key | Description |
|---|---|
| ttl | Specifies the TTL of objects cached, if not evicted due to a write first |
| table | In table cache rules, specifies the name of the table to apply settings for |
For Debug
| Key | Description |
|---|---|
| enabled | If true, enables driver debug logging at that point |
For Pool
| Key | Description |
|---|---|
| maxUserActive | Sets the per-user active pool parameter (idle+busy), and overrides any default set in the source settings. Useful to allow individual users to have more connections than others, and can be set based on AD group, say to allow service accounts more connections than general users (Integer). |
| maxUserIdle | Overrides the per-user allowed idle connection limit (Integer). |
| multiplex | Override the vdb level multiplex setting to either enable or disable multiplexing (Boolean). |
| multiplexSkip | Triggers skipping multiplexing for X number of queries after the current matching one (Integer). |
| reuse | Control if connections can be reused on return by pooling, defaults to true (Boolean). Only impacts the primary (writer) connection, not read/write split connections. |
For Reader Eligible
| Key | Description |
|---|---|
| Lagignore | Ignore the lag detection logic when routing a query. |
| unconditional | If set to true bypass internal checks that would mark query as not viable for reader servers. |
For Multiplex
| Key | Description |
|---|---|
| disable | When the capture parameter from regex is matched, then it is saved in memory and from this point multiplex will be disabled for all queries, unless it will be enabled again. |
| enable | When the capture parameter from regex is matched, then it is removed from the memory (mentioned in the "disable" point) and multiplex will be enabled for next executed query (unless there are any other captured parameters disabled) |