Giters

boblemaire / IoTaWatt

IoTaWatt Open WiFi Electric Energy Monitor

Home Page:https://iotawatt.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Invalid EmonCMS API call

dakaix opened this issue · comments

commented

emoncms_uploader::handle_query_s() is making an invalid call to the EmonCMS API.

The EmonCMS "/input/get" endpoint only accepts GET queries - yet IoTaWatt is sending a POST with no arguments. See https://emoncms.org/site/api#input

EmonCMS (v10.8.5) replies to this invalid query with the HTML of the home page (45K in size).

IoTaWatt Query

POST /input/get?node=IotaWatt HTTP/1.1
host:192.168.1.17:80
content-type:application/x-www-form-urlencoded
Authorization:Bearer EmonCMS_Read/Write_API_Key_Here
Content-Length:0

EmonCMS Response

HTTP/1.1 200 OK
Date: Sat, 19 Feb 2022 19:54:01 GMT
Server: Apache/2.4.6 (CentOS) mpm-itk/2.4.7-04 PHP/7.4.24
X-Powered-By: PHP/7.4.24
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Set-Cookie: EMONCMS_SESSID=n9n3fthsogvsdomv6u6g718cj2; path=/; HttpOnly; SameSite=Strict
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

2000
<!doctype html>
<html class="theme-blue sidebar-dark">
<head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1">
    <title>Emoncms - user login </title>

... Output Truncated ...

After this reply is received the IoTaWatt reboots reporting an Exception had occurred.

Log shown below (IP's changed for privacy), for context this covers the following events:

  • 19:45:41z: IoTaWatt was restarted manually
  • 19:54:00z: 9V voltage reference was connected to IoTaWatt, triggering an attempt to upload to EmonCMS
  • 19:54:01z: IotaWatt issues the POST query to EmonCMS
  • 19:54:02z: IoTaWatt reboots with an Exception
  • 19:54:06z: IoTaWatt attempts same POST query and reboots again, this continues until the 9V voltage reference is removed (breaking the reboot loop)
SD initialized.
2/19/22 19:45:41z Real Time Clock is running. Unix time 1645299941 
2/19/22 19:45:41z Reset reason: Software/System restart
2/19/22 19:45:41z Trace:  9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:2, 1:2, 1:3, 1:3, 1:1[1], 1:2[2], 9:0[2], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[2], 1:2, 9:0, 9:0, 8:2, 8:2, 1:2
2/19/22 19:45:41z ESP8266 ID: 523643, RTC M41T81 (68)
2/19/22 19:45:41z IoTaWatt 5.0, Firmware version 02_07_05
2/19/22 19:45:41z SPIFFS mounted.
2/19/22 19:45:41z Local time zone: +0:00, using DST/BST when in effect.
2/19/22 19:45:41z device name: IotaWatt
2/19/22 19:45:41z HTTP server started
2/19/22 19:54:00z emoncms: Starting, interval:10, url:http://192.168.1.17
2/19/22 19:54:00z WiFi connected. SSID=Wireless, IP=192.168.1.15, channel=11, RSSI -65db
2/19/22 19:54:00z MDNS responder started for hostname IotaWatt
2/19/22 19:54:00z LLMNR responder started for hostname IotaWatt
2/19/22 19:54:00z timeSync: service started.
2/19/22 19:54:01z statService: started.
2/19/22 19:54:01z Updater: service started. Auto-update class is MINOR
2/19/22 19:54:01z dataLog: service started.
2/19/22 19:54:01z dataLog: Last log entry 02/19/22 19:40:40
2/19/22 19:54:01z historyLog: service started.
2/19/22 19:54:01z historyLog: Last log entry 02/19/22 17:55:00

** Restart **

SD initialized.
2/19/22 19:54:02z Real Time Clock is running. Unix time 1645300442 
2/19/22 19:54:02z Reset reason: Exception
2/19/22 19:54:02z Trace:  8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[1], 1:2[2], 9:0[2], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2, 1:3, 1:3, 1:1[2], 1:2, 9:0, 9:0, 8:4, 8:6, 8:8, 8:9, 1:2
2/19/22 19:54:02z ESP8266 ID: 523643, RTC M41T81 (68)
2/19/22 19:54:02z IoTaWatt 5.0, Firmware version 02_07_05
2/19/22 19:54:02z SPIFFS mounted.
2/19/22 19:54:02z Local time zone: +0:00, using DST/BST when in effect.
2/19/22 19:54:02z device name: IotaWatt
2/19/22 19:54:02z HTTP server started
2/19/22 19:54:02z emoncms: Starting, interval:10, url:http://192.168.1.17
2/19/22 19:54:02z timeSync: service started.
2/19/22 19:54:02z statService: started.
2/19/22 19:54:02z dataLog: service started.
2/19/22 19:54:02z dataLog: Last log entry 02/19/22 19:40:40
2/19/22 19:54:02z historyLog: service started.
2/19/22 19:54:02z historyLog: Last log entry 02/19/22 17:57:00
2/19/22 19:54:06z WiFi connected. SSID=Wireless, IP=192.168.1.15, channel=11, RSSI -61db
2/19/22 19:54:06z MDNS responder started for hostname IotaWatt
2/19/22 19:54:06z LLMNR responder started for hostname IotaWatt
2/19/22 19:54:06z Updater: service started. Auto-update class is MINOR

** Restart **

SD initialized.
2/19/22 19:54:07z Real Time Clock is running. Unix time 1645300447 
2/19/22 19:54:07z Reset reason: Exception
2/19/22 19:54:07z Trace:  9:0, 8:4, 8:6, 8:8, 8:9, 1:2, 1:3, 1:3, 1:6[1], 1:6[2], 1:6[3], 1:5[14], 1:6[4], 14:0, 14:4, 14:5, 14:7, 14:8, 1:6[6], 1:1, 1:2[1], 9:0[1], 9:0, 9:1, 8:4, 8:6, 8:8, 8:9, 9:3, 9:5, 9:9, 1:2
2/19/22 19:54:07z ESP8266 ID: 523643, RTC M41T81 (68)
2/19/22 19:54:07z IoTaWatt 5.0, Firmware version 02_07_05
2/19/22 19:54:07z SPIFFS mounted.
2/19/22 19:54:07z Local time zone: +0:00, using DST/BST when in effect.
2/19/22 19:54:07z device name: IotaWatt
2/19/22 19:54:07z HTTP server started
2/19/22 19:54:07z emoncms: Starting, interval:10, url:http://192.168.1.17
2/19/22 19:54:07z timeSync: service started.
2/19/22 19:54:07z statService: started.
2/19/22 19:54:07z dataLog: service started.
2/19/22 19:54:07z dataLog: Last log entry 02/19/22 19:54:05
2/19/22 19:54:07z historyLog: service started.
2/19/22 19:54:07z historyLog: Last log entry 02/19/22 19:45:00
2/19/22 19:54:11z WiFi connected. SSID=Wireless, IP=192.168.1.15 channel=11, RSSI -60db
2/19/22 19:54:11z MDNS responder started for hostname IotaWatt
2/19/22 19:54:11z LLMNR responder started for hostname IotaWatt
2/19/22 19:54:11z Updater: service started. Auto-update class is MINOR

config.txt

{
	"format": 2,
	"timezone": 0,
	"update": "MINOR",
	"device": {
		"name": "IotaWatt",
		"version": 3,
		"channels": "15",
		"burden": [
			0,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20,
			20
		]
	},
	"inputs": [
		{
			"channel": 0,
			"name": "Voltage",
			"type": "VT",
			"model": "Ideal 77DB-06-09(UK)",
			"cal": 20.01
		},
		{
			"channel": 1,
			"name": "Input_1",
			"type": "CT",
			"model": "HWCT-004",
			"phase": 0.1,
			"turns": 1000,
			"cal": 50
		},
		{
			"channel": 2,
			"name": "Input_2",
			"type": "CT",
			"model": "SCT013-000",
			"phase": 2.3,
			"turns": 2000,
			"cal": 100
		},
		null,
		null,
		null,
		null,
		null,
		null,
		null,
		null,
		null,
		null,
		null,
		null
	],
	"outputs": [
		{
			"name": "Voltage",
			"units": "Volts",
			"script": "@0"
		}
	],
	"dstrule": {
		"adj": 60,
		"utc": true,
		"begin": {
			"month": 3,
			"weekday": 1,
			"instance": -1,
			"time": 60
		},
		"end": {
			"month": 10,
			"weekday": 1,
			"instance": -1,
			"time": 60
		}
	},
	"emoncms": {
		"type": "emoncms",
		"revision": 0,
		"node": "IotaWatt",
		"postInterval": 10,
		"bulksend": 1,
		"url": "http://192.168.1.17",
		"apikey": "EMONCMS_READ_WRITE_API_KEY",
		"user": "",
		"pwd": "",
		"outputs": [
			{
				"name": 1,
				"script": "@0",
				"units": ""
			},
			{
				"name": 2,
				"script": "@1",
				"units": ""
			},
			{
				"name": 3,
				"script": "@2",
				"units": ""
			},
			{
				"name": 4,
				"script": "@3",
				"units": ""
			},
			{
				"name": 5,
				"script": "@4",
				"units": ""
			},
			{
				"name": 6,
				"script": "@5",
				"units": ""
			},
			{
				"name": 7,
				"script": "@6",
				"units": ""
			},
			{
				"name": 8,
				"script": "@7",
				"units": ""
			},
			{
				"name": 9,
				"script": "@8",
				"units": ""
			},
			{
				"name": 10,
				"script": "@9",
				"units": ""
			},
			{
				"name": 11,
				"script": "@10",
				"units": ""
			},
			{
				"name": 12,
				"script": "@11",
				"units": ""
			},
			{
				"name": 13,
				"script": "@12",
				"units": ""
			},
			{
				"name": 14,
				"script": "@13",
				"units": ""
			},
			{
				"name": 15,
				"script": "@14",
				"units": ""
			}
		],
		"userid": "",
		"begdate": 1645228800
	}
}
commented

I went through the EmonCMS logs and authentication code, and was able to find the difference between the working test queries I had done and the POST to /input/get.

The root cause was the “HTTP_AUTHENTICATION” header not being passed through from Apache to PHP (“Authorisation: Bearer APIKEY”).

This is disabled by default (as a security measure) but EmonCMS tries to enable it via htaccess. While other parts of that .htaccess were working, the specific Authorization statement wasn’t. I have now instead configured it in the main Apache Virtualhost configuration, and after retesting I can confirm that the authenetication header is being passed through - and both EmonCMS and IoTaWatt are working.

This was an unfortunate combination of factors since other authentication sources (those not using the “Authorization” header) were working, such as the “apikey” variable set via GET or POST. If the goal for IoTaWatt is to try and bulletproof this integration it would be worth considering whether to deprecate use of the authentication method as it isn’t enabled in Apache by default, and as I’ve found the supplied EmonCMS htaccess doesn’t always work. The easiest route would be for IoTaWatt to use the “apikey” variable instead, and would be straightforward to implement.

emoncms.log:

2022-02-19 19:50:52.021|ERROR|index.php|Not Authenticated|input/get
2022-02-19 19:54:01.581|ERROR|index.php|Not Authenticated|input/get
2022-02-19 19:54:06.466|ERROR|index.php|Not Authenticated|input/get
2022-02-19 19:54:11.379|ERROR|index.php|Not Authenticated|input/get
2022-02-19 20:02:44.001|ERROR|index.php|Not Authenticated|input/get
2022-02-19 20:08:35.998|ERROR|index.php|Not Authenticated|input/get

EmonCMS Authentication:

$apikey = false;
$devicekey = false;
if (isset($_GET['apikey'])) {
    $apikey = $_GET['apikey'];
} elseif (isset($_POST['apikey'])) {
    $apikey = $_POST['apikey'];
} elseif (isset($_GET['devicekey'])) {
    $devicekey = $_GET['devicekey'];
} elseif (isset($_POST['devicekey'])) {
    $devicekey = $_POST['devicekey'];
} elseif (isset($_SERVER["HTTP_AUTHORIZATION"])) {
    // Support passing apikey on Authorization header per rfc6750, like example:
    //      GET /resource HTTP/1.1
    //      Host: server.example.com
    //      Authorization: Bearer THE_API_KEY_HERE
    
    if (isset($_SERVER["CONTENT_TYPE"]) && $_SERVER["CONTENT_TYPE"]=="aes128cbc") {
        // If content_type is AES128CBC
    } else {
        $apikey = str_replace('Bearer ', '', $_SERVER["HTTP_AUTHORIZATION"]);
    }
}

EmonCMS Standard .htaccess:

# ------------------------------------------------------------------------------
# Rewrite auth header. Fix Authorization: Bearer
# https://cweiske.de/tagebuch/php-apache-authorization.htm
# ------------------------------------------------------------------------------
<IfModule mod_setenvif.c>
  SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
</IfModule>
commented

Closing as Emoncms/Apache problem.