http-json – Rule based JSON parser via http(s)

Project Notes

<h3><a id="Installation_4"></a>Installation</h3>
<h4><a id="Requirements_6"></a>Requirements</h4>
<ul>
<li>Nagios</li>
<li>Python</li>
</ul>
<h3><a id="Nagios_Configuration_11"></a>Nagios Configuration</h3>
<p>Assuming a standard installation of Nagios, the plugin can be executed from the machine that Nagios is running on.</p>
<pre><code class="language-bash">cp check_http_json.py /usr/<span class="hljs-built_in">local</span>/nagios/libexec/plugins/check_http_json.py
chmod +x /usr/<span class="hljs-built_in">local</span>/nagios/libexec/plugins/check_http_json.py
</code></pre>
<p>Add the following service definition to your server config (<code>localhost.cfg</code>):</p>
<pre><code>
define service {
use <span class="hljs-built_in">local</span>-service
host_name localhost
service_description <<span class="hljs-built_in">command</span>_description>
check_<span class="hljs-built_in">command</span> <<span class="hljs-built_in">command</span>_name>
}

</code></pre>
<p>Add the following command definition to your commands config (<code>commands.config</code>):</p>
<pre><code><span class="xml">
define command</span><span class="hljs-expression">{
<span class="hljs-variable">command</span>_<span class="hljs-variable">name</span> <<span class="hljs-variable">command</span>_<span class="hljs-variable">name</span>>
<span class="hljs-variable">command</span>_<span class="hljs-variable">line</span> <span class="hljs-end-block">/usr</span><span class="hljs-end-block">/bin</span><span class="hljs-end-block">/python </span><span class="hljs-end-block">/usr</span><span class="hljs-end-block">/local</span><span class="hljs-end-block">/nagios</span><span class="hljs-end-block">/libexec</span><span class="hljs-end-block">/plugins</span><span class="hljs-end-block">/check</span>_<span class="hljs-variable">http</span>_<span class="hljs-variable">json.py</span> <span class="hljs-variable">-H</span> <<span class="hljs-variable">host</span>>:<<span class="hljs-variable">port</span>> <span class="hljs-variable">-p</span> <<span class="hljs-variable">path</span>> [<span class="hljs-variable">-e</span>|<span class="hljs-variable">-q</span>|<span class="hljs-variable">-w</span>|<span class="hljs-variable">-c</span> <<span class="hljs-variable">rules</span>>] [<span class="hljs-variable">-m</span> <<span class="hljs-variable">metrics</span>>]
}</span><span class="xml">

</span></code></pre>
<p>More info about options in Usage.</p>
<h3><a id="CLI_Usage_46"></a>CLI Usage</h3>
<p>Executing <code>./check_http_json.py -h</code> will yield the following details:</p>
<pre><code>usage: check_http_json.py [-h] [-d] [-s] -H HOST [-P PORT] [-p PATH]
[-t TIMEOUT] [-B AUTH] [-D DATA] [-f SEPARATOR]
[-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]]
[-c [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]]
[-e [KEY_LIST [KEY_LIST ...]]]
[-E [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]]
[-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]]
[-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]]
[-m [METRIC_LIST [METRIC_LIST ...]]]

Nagios plugin which checks json values from a given endpoint against argument
specified rules and determines the status and performance data for that
service

optional arguments:
-h, <span class="hljs-comment">--help show this help message and exit</span>
-d, <span class="hljs-comment">--debug Debug mode.</span>
-s, <span class="hljs-comment">--ssl HTTPS mode.</span>
-H HOST, <span class="hljs-comment">--host HOST Host.</span>
-P PORT, <span class="hljs-comment">--port PORT TCP port</span>
-p PATH, <span class="hljs-comment">--path PATH Path.</span>
-t TIMEOUT, <span class="hljs-comment">--timeout TIMEOUT</span>
Connection timeout (seconds)
-B AUTH, <span class="hljs-comment">--basic-auth AUTH</span>
Basic auth string "username:password"
-D DATA, <span class="hljs-comment">--data DATA The http payload to send as a POST</span>
-f SEPARATOR, <span class="hljs-comment">--field_separator SEPARATOR</span>
Json Field separator, defaults to "." ; <span class="hljs-operator"><span class="hljs-keyword">Select</span> <span class="hljs-keyword">element</span>
<span class="hljs-keyword">in</span> an <span class="hljs-built_in">array</span> <span class="hljs-keyword">with</span> <span class="hljs-string">"("</span> <span class="hljs-string">")"</span>
-w [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]], <span class="hljs-comment">--warning [KEY_THRESHOLD_WARNING [KEY_THRESHOLD_WARNING ...]]</span>
<span class="hljs-keyword">Warning</span> threshold <span class="hljs-keyword">for</span> these <span class="hljs-keyword">values</span>
(key1[><span class="hljs-keyword">alias</span>],WarnRange key2[><span class="hljs-keyword">alias</span>],WarnRange).
WarnRange <span class="hljs-keyword">is</span> <span class="hljs-keyword">in</span> the <span class="hljs-keyword">format</span> [@]<span class="hljs-keyword">start</span>:<span class="hljs-keyword">end</span>, more
information <span class="hljs-keyword">at</span> nagios-plugins.org/doc/guidelines.html.
-<span class="hljs-keyword">c</span> [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]], <span class="hljs-comment">--critical [KEY_THRESHOLD_CRITICAL [KEY_THRESHOLD_CRITICAL ...]]</span>
<span class="hljs-keyword">Critical</span> threshold <span class="hljs-keyword">for</span> these <span class="hljs-keyword">values</span>
(key1[><span class="hljs-keyword">alias</span>],CriticalRange
key2[><span class="hljs-keyword">alias</span>],CriticalRange. CriticalRange <span class="hljs-keyword">is</span> <span class="hljs-keyword">in</span> the
<span class="hljs-keyword">format</span> [@]<span class="hljs-keyword">start</span>:<span class="hljs-keyword">end</span>, more information <span class="hljs-keyword">at</span> nagios-
plugins.org/doc/guidelines.html.
-<span class="hljs-keyword">e</span> [KEY_LIST [KEY_LIST ...]], <span class="hljs-comment">--key_exists [KEY_LIST [KEY_LIST ...]]</span>
Checks existence <span class="hljs-keyword">of</span> these <span class="hljs-keyword">keys</span> <span class="hljs-keyword">to</span> determine <span class="hljs-keyword">status</span>.
<span class="hljs-keyword">Return</span> <span class="hljs-keyword">warning</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">key</span> <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-keyword">present</span>.
-<span class="hljs-keyword">E</span> [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]], <span class="hljs-comment">--key_exists_critical [KEY_LIST_CRITICAL [KEY_LIST_CRITICAL ...]]</span>
Same <span class="hljs-keyword">as</span> -<span class="hljs-keyword">e</span> but <span class="hljs-keyword">return</span> <span class="hljs-keyword">critical</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">key</span> <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-keyword">present</span>.
-q [KEY_VALUE_LIST [KEY_VALUE_LIST ...]], <span class="hljs-comment">--key_equals [KEY_VALUE_LIST [KEY_VALUE_LIST ...]]</span>
Checks equality <span class="hljs-keyword">of</span> these <span class="hljs-keyword">keys</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">values</span>
(<span class="hljs-keyword">key</span>[><span class="hljs-keyword">alias</span>],<span class="hljs-keyword">value</span> key2,value2) <span class="hljs-keyword">to</span> determine <span class="hljs-keyword">status</span>.
Multiple <span class="hljs-keyword">key</span> <span class="hljs-keyword">values</span> can be <span class="hljs-keyword">delimited</span> <span class="hljs-keyword">with</span> colon
(<span class="hljs-keyword">key</span>,value1:value2). <span class="hljs-keyword">Return</span> <span class="hljs-keyword">warning</span> <span class="hljs-keyword">if</span> equality <span class="hljs-keyword">check</span>
fails
-Q [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]], <span class="hljs-comment">--key_equals_critical [KEY_VALUE_LIST_CRITICAL [KEY_VALUE_LIST_CRITICAL ...]]</span>
Same <span class="hljs-keyword">as</span> -q but <span class="hljs-keyword">return</span> <span class="hljs-keyword">critical</span> <span class="hljs-keyword">if</span> equality <span class="hljs-keyword">check</span>
fails.
-<span class="hljs-keyword">m</span> [METRIC_LIST [METRIC_LIST ...]], <span class="hljs-comment">--key_metric [METRIC_LIST [METRIC_LIST ...]]</span>
Gathers the <span class="hljs-keyword">values</span> <span class="hljs-keyword">of</span> these <span class="hljs-keyword">keys</span> (<span class="hljs-keyword">key</span>[><span class="hljs-keyword">alias</span>],UnitOfMe
asure,WarnRange,CriticalRange,<span class="hljs-keyword">Min</span>,<span class="hljs-keyword">Max</span>) <span class="hljs-keyword">for</span> Nagios
<span class="hljs-keyword">performance</span> <span class="hljs-keyword">data</span>. More information about <span class="hljs-keyword">Range</span> <span class="hljs-keyword">format</span>
<span class="hljs-keyword">and</span> units <span class="hljs-keyword">of</span> measure <span class="hljs-keyword">for</span> nagios can be <span class="hljs-keyword">found</span> <span class="hljs-keyword">at</span>
nagios-plugins.org/doc/guidelines.html Additional
formats <span class="hljs-keyword">for</span> this parameter <span class="hljs-keyword">are</span>: (<span class="hljs-keyword">key</span>[><span class="hljs-keyword">alias</span>]),
(<span class="hljs-keyword">key</span>[><span class="hljs-keyword">alias</span>],UnitOfMeasure),
(<span class="hljs-keyword">key</span>[><span class="hljs-keyword">alias</span>],UnitOfMeasure,WarnRange,CriticalRange).
</span></code></pre>
<p>Access a specific JSON field by following this syntax: <code>alpha.beta.gamma(3).theta.omega(0)</code>
Dots are field separators (changeable), parentheses are for entering arrays.</p>
<p>If the root of the JSON data is itself an array like the following:</p>
<pre><code>[
{ "<span class="hljs-attribute">gauges</span>": <span class="hljs-value">{ "<span class="hljs-attribute">jvm.buffers.direct.capacity</span>": <span class="hljs-value">{"<span class="hljs-attribute">value</span>": <span class="hljs-value"><span class="hljs-number">215415</span></span>}</span>}</span>}
]
</code></pre>
<p>The beginning of the key should start with ($index) as in this example:</p>
<pre><code>./check_http_json<span class="hljs-class">.py</span> -H localhost:<span class="hljs-number">8081</span> -<span class="hljs-tag">p</span> metrics --key_exists <span class="hljs-string">"(0)_gauges_jvm.buffers.direct.capacity_value"</span> -f _
</code></pre>
<p>More info about Nagios Range format and Units of Measure can be found at <a href="https://nagios-plugins.org/doc/guidelines.html">https://nagios-plugins.org/doc/guidelines.html</a>.</p>
<h3><a id="Docker_Info_Example_Plugin_135"></a>Docker Info Example Plugin</h3>
<h4><a id="Description_137"></a>Description</h4>
<p>Let’s say we want to use <code>check_http_json.py</code> to read from Docker’s <code>/info</code> HTTP API endpoint with the following parameters:</p>
<h5><a id="Connection_information_141"></a>Connection information</h5>
<ul>
<li>Host = 127.0.0.1:4243</li>
<li>Path = /info</li>
</ul>
<h5><a id="Rules_for_aliveness_146"></a>Rules for “aliveness”</h5>
<ul>
<li>Verify that the key <code>Containers</code> exists in the outputted JSON</li>
<li>Verify that the key <code>IPv4Forwarding</code> has a value of <code>1</code></li>
<li>Verify that the key <code>Debug</code> has a value less than or equal to <code>2</code></li>
<li>Verify that the key <code>Images</code> has a value greater than or equal to <code>1</code></li>
<li>If any of these criteria are not met, report a WARNING to Nagios</li>
</ul>
<h5><a id="Gather_Metrics_154"></a>Gather Metrics</h5>
<ul>
<li>Report value of the key <code>Containers</code> with a MinValue of 0 and a MaxValue of 1000 as performance data</li>
<li>Report value of the key <code>Images</code> as performance data</li>
<li>Report value of the key <code>NEventsListener</code> as performance data</li>
<li>Report value of the key <code>NFd</code> as performance data</li>
<li>Report value of the key <code>NGoroutines</code> as performance data</li>
<li>Report value of the key <code>SwapLimit</code> as performance data</li>
</ul>
<h4><a id="Service_Definition_163"></a>Service Definition</h4>
<p><code>localhost.cfg</code></p>
<pre><code><span class="hljs-title">define</span> service {
<span class="hljs-title">use</span> local-service
host_name localhost
service_description Docker <span class="hljs-built_in">info</span> status checker
check_command check_docker
}
</code></pre>
<h4><a id="Command_Definition_with_Arguments_176"></a>Command Definition with Arguments</h4>
<p><code>commands.cfg</code></p>
<pre><code>define command{
command_name check_docker
command_line /usr/bin/python /usr/local/nagios/libexec/plugins/check_http_json.py -H <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">4243</span> -p info -e Containers -q IPv4Forwarding,<span class="hljs-number">1</span> -w Debug,<span class="hljs-number">2</span>:<span class="hljs-number">2</span> -c Images,<span class="hljs-number">1</span>:<span class="hljs-number">1</span> -m Containers,<span class="hljs-number">0</span>:<span class="hljs-number">250</span>,<span class="hljs-number">0</span>:<span class="hljs-number">500</span>,<span class="hljs-number">0</span>,<span class="hljs-number">1000</span> Images NEventsListener NFd NGoroutines SwapLimit
}
</code></pre>
<h4><a id="Sample_Output_187"></a>Sample Output</h4>
<pre><code><span class="hljs-string">OK:</span> Status OK.|<span class="hljs-string">'Containers'</span>=<span class="hljs-number">1</span>;<span class="hljs-number">0</span>;<span class="hljs-number">1000</span> <span class="hljs-string">'Images'</span>=<span class="hljs-number">11</span>;<span class="hljs-number">0</span>;<span class="hljs-number">0</span> <span class="hljs-string">'NEventsListener'</span>=<span class="hljs-number">3</span>;<span class="hljs-number">0</span>;<span class="hljs-number">0</span> <span class="hljs-string">'NFd'</span>=<span class="hljs-number">10</span>;<span class="hljs-number">0</span>;<span class="hljs-number">0</span> <span class="hljs-string">'NGoroutines'</span>=<span class="hljs-number">14</span>;<span class="hljs-number">0</span>;<span class="hljs-number">0</span> <span class="hljs-string">'SwapLimit'</span>=<span class="hljs-number">1</span>;<span class="hljs-number">0</span>;<span class="hljs-number">0</span>
</code></pre>
<h3><a id="Docker_Container_Monitor_Example_Plugin_193"></a>Docker Container Monitor Example Plugin</h3>
<p><code>check_http_json.py</code> is generic enough to read and evaluate rules on any HTTP endpoint that returns JSON. In this example we’ll get the status of a specific container using it’s ID which camn be found by using the list containers endpoint (<code>curl http://127.0.0.1:4243/containers/json?all=1</code>).</p>
<h5><a id="Connection_information_197"></a>Connection information</h5>
<ul>
<li>Host = 127.0.0.1:4243</li>
<li>Path = /containers/2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615/json</li>
</ul>
<h5><a id="Rules_for_aliveness_202"></a>Rules for “aliveness”</h5>
<ul>
<li>Verify that the key <code>ID</code> exists and is equal to the value <code>2356e8ccb3de8308ccb16cf8f5d157bc85ded5c3d8327b0dfb11818222b6f615</code></li>
<li>Verify that the key <code>State.Running</code> has a value of <code>True</code></li>
</ul>
<h4><a id="Service_Definition_207"></a>Service Definition</h4>
<p><code>localhost.cfg</code></p>
<pre><code>define service {
use local-service
host_name localhost
service_description Docker container liveness<span class="hljs-instruction"> check
</span> check_command check_my_container
}
</code></pre>
<h4><a id="Command_Definition_with_Arguments_220"></a>Command Definition with Arguments</h4>
<p><code>commands.cfg</code></p>
<pre><code><span class="xml">define command</span><span class="hljs-expression">{
<span class="hljs-variable">command</span>_<span class="hljs-variable">name</span> <span class="hljs-variable">check</span>_<span class="hljs-variable">my</span>_<span class="hljs-variable">container</span>
<span class="hljs-variable">command</span>_<span class="hljs-variable">line</span> <span class="hljs-end-block">/usr</span><span class="hljs-end-block">/bin</span><span class="hljs-end-block">/python </span><span class="hljs-end-block">/usr</span><span class="hljs-end-block">/local</span><span class="hljs-end-block">/nagios</span><span class="hljs-end-block">/libexec</span><span class="hljs-end-block">/plugins</span><span class="hljs-end-block">/check</span>_<span class="hljs-variable">http</span>_<span class="hljs-variable">json.py</span> <span class="hljs-variable">-H</span> 127<span class="hljs-variable">.</span>0<span class="hljs-variable">.</span>0<span class="hljs-variable">.</span>1:4243 <span class="hljs-variable">-p</span> <span class="hljs-end-block">/containers</span>/2356<span class="hljs-variable">e</span>8<span class="hljs-variable">ccb</span>3<span class="hljs-variable">de</span>8308<span class="hljs-variable">ccb</span>16<span class="hljs-variable">cf</span>8<span class="hljs-variable">f</span>5<span class="hljs-variable">d</span>157<span class="hljs-variable">bc</span>85<span class="hljs-variable">ded</span>5<span class="hljs-variable">c</span>3<span class="hljs-variable">d</span>8327<span class="hljs-variable">b</span>0<span class="hljs-variable">dfb</span>11818222<span class="hljs-variable">b</span>6<span class="hljs-variable">f</span>615<span class="hljs-end-block">/json -q ID</span>,2356<span class="hljs-variable">e</span>8<span class="hljs-variable">ccb</span>3<span class="hljs-variable">de</span>8308<span class="hljs-variable">ccb</span>16<span class="hljs-variable">cf</span>8<span class="hljs-variable">f</span>5<span class="hljs-variable">d</span>157<span class="hljs-variable">bc</span>85<span class="hljs-variable">ded</span>5<span class="hljs-variable">c</span>3<span class="hljs-variable">d</span>8327<span class="hljs-variable">b</span>0<span class="hljs-variable">dfb</span>11818222<span class="hljs-variable">b</span>6<span class="hljs-variable">f</span>615 <span class="hljs-variable">State.Running</span>,<span class="hljs-variable">True</span>
}</span><span class="xml">
</span></code></pre>
<h4><a id="Sample_Output_231"></a>Sample Output</h4>
<pre><code>WARNING: Status<span class="hljs-instruction"> check </span>failed, reason: Value True for key State.Running did<span class="hljs-instruction"> not </span>match.
</code></pre>
<p>The plugin threw a warning because the Container ID I used on my system has the following State object:</p>
<pre><code> <span class="hljs-string">u'State'</span>: {...
<span class="hljs-string">u'Running'</span>: <span class="hljs-keyword">False</span>,
...
</code></pre>
<p>If I change the command to have the parameter -q parameter <code>State.Running,False</code>, the output becomes:</p>
<pre><code><span class="hljs-attribute">OK</span>: <span class="hljs-string">Status OK.</span>
</code></pre>
<h3><a id="Dropwizard__Fieldnames_Containing__Example_251"></a>Dropwizard / Fieldnames Containing ‘.’ Example</h3>
<p>Simply choose a separator to deal with data such as this:</p>
<pre><code>{ "<span class="hljs-attribute">gauges</span>": <span class="hljs-value">{ "<span class="hljs-attribute">jvm.buffers.direct.capacity</span>": <span class="hljs-value">{"<span class="hljs-attribute">value</span>": <span class="hljs-value"><span class="hljs-number">215415</span></span>}</span>}</span>}
</code></pre>
<p>In this example I’ve chosen <code>_</code> to separate <code>guages</code> from <code>jvm</code> and <code>capacity</code> from <code>value</code>. The CLI invocation then becomes:</p>
<pre><code>./check_http_json<span class="hljs-class">.py</span> -H localhost:<span class="hljs-number">8081</span> -<span class="hljs-tag">p</span> metrics --key_exists gauges_jvm<span class="hljs-class">.buffers</span><span class="hljs-class">.direct</span><span class="hljs-class">.capacity_value</span> -f _
</code></pre>
<h2><a id="License_265"></a>License</h2>
<pre><code>Copyright 2014-2015 Drew Kerrigan.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.</code></pre>