Feature Request: CPU & Memory Usage (by NR process)

[thejollypop]

Hi Jason,
First of all, excellent work!

I am trying to monitor NR across several platforms, such as Docker, Ubuntu, Debian and MacOS.
I'm still missing two major bottom line insights, isolated to the NR process:

CPU Utilization = 100 - idle_time - steal_time

Memory Utilized = ( (Total - Free Memory) / Total * 100 )

The following are the flows I created to work around it (not 100% accurate).
It would be great if this functionality could be built into future version of node-red-contrib-os
Thanks!

Memory Usage (NR PID)
[ { "id": "8438793e123fe05d", "type": "group", "z": "3ceaa25c31c1e42f", "style": { "stroke": "#999999", "stroke-opacity": "1", "fill": "none", "fill-opacity": "1", "label": true, "label-position": "nw", "color": "#a4a4a4" }, "nodes": [ "c7bf46909b0d5341", "9a4d27b7c61a538b", "3310e69448bd265e", "c9287d1f621f84fb", "54d85711b8c3a0f5", "c3cddae56321d1a5", "7b260e93a04737a2", "e6d84ee8f938af49", "481a4e9c3823ad74", "1a2f5563d6e73a83" ], "x": 254, "y": 179, "w": 622, "h": 142 }, { "id": "c7bf46909b0d5341", "type": "Memory", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "scale": "Gigabyte", "x": 480, "y": 220, "wires": [ [ "54d85711b8c3a0f5" ] ] }, { "id": "9a4d27b7c61a538b", "type": "inject", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "props": [ { "p": "payload" }, { "p": "topic", "vt": "str" } ], "repeat": "", "crontab": "", "once": false, "onceDelay": "0.1", "topic": "", "payloadType": "date", "x": 315, "y": 260, "wires": [ [ "c7bf46909b0d5341", "481a4e9c3823ad74" ] ], "l": false }, { "id": "3310e69448bd265e", "type": "debug", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "active": false, "tosidebar": true, "console": false, "tostatus": false, "complete": "payload", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 795, "y": 220, "wires": [], "l": false }, { "id": "c9287d1f621f84fb", "type": "Memory", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "scale": "Gigabyte", "x": 480, "y": 280, "wires": [ [ "c3cddae56321d1a5" ] ] }, { "id": "54d85711b8c3a0f5", "type": "change", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "payload.memusage", "tot": "msg" }, { "t": "set", "p": "topic", "pt": "msg", "to": "previousmem", "tot": "str" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 595, "y": 220, "wires": [ [ "7b260e93a04737a2" ] ], "l": false }, { "id": "c3cddae56321d1a5", "type": "change", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "payload.memusage", "tot": "msg" }, { "t": "set", "p": "topic", "pt": "msg", "to": "currentmem", "tot": "str" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 595, "y": 280, "wires": [ [ "7b260e93a04737a2" ] ], "l": false }, { "id": "7b260e93a04737a2", "type": "join", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "mode": "custom", "build": "object", "property": "payload", "propertyType": "msg", "key": "topic", "joiner": "\\n", "joinerType": "str", "accumulate": false, "timeout": "", "count": "2", "reduceRight": false, "reduceExp": "", "reduceInit": "", "reduceInitType": "", "reduceFixup": "", "x": 690, "y": 260, "wires": [ [ "3310e69448bd265e", "e6d84ee8f938af49" ] ] }, { "id": "e6d84ee8f938af49", "type": "change", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "$abs(payload.currentmem-payload.previousmem)", "tot": "jsonata" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 775, "y": 280, "wires": [ [ "1a2f5563d6e73a83" ] ], "l": false }, { "id": "481a4e9c3823ad74", "type": "delay", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "pauseType": "delay", "timeout": "3", "timeoutUnits": "seconds", "rate": "1", "nbRateUnits": "1", "rateUnits": "second", "randomFirst": "1", "randomLast": "5", "randomUnits": "seconds", "drop": false, "allowrate": false, "outputs": 1, "x": 375, "y": 280, "wires": [ [ "c9287d1f621f84fb" ] ], "l": false }, { "id": "1a2f5563d6e73a83", "type": "debug", "z": "3ceaa25c31c1e42f", "g": "8438793e123fe05d", "name": "", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "payload", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 815, "y": 280, "wires": [], "l": false } ]

CPU Usage (NR PID)
[{"id":"ac401ac790e1ee2e","type":"group","z":"b96d5cc992392d6e","style":{"stroke":"#999999","stroke-opacity":"1","fill":"none","fill-opacity":"1","label":true,"label-position":"nw","color":"#a4a4a4"},"nodes":["e86e88f88a40efb6","ee71012bf1a80e53","bbe4584c319f7229","3e9e4bcb14727c44","c6a459bb12f666ea","83e79b7f1579119c","15a32dd96a88d924","75832cf54d72f14b","4d197a204dbdc20a","b849936960a52481","78696e2b89db404a"],"x":74,"y":259,"w":762,"h":162},{"id":"e86e88f88a40efb6","type":"cpu","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","msgCore":false,"msgOverall":true,"msgArray":false,"msgTemp":false,"x":410,"y":320,"wires":[["bbe4584c319f7229","3e9e4bcb14727c44"]]},{"id":"ee71012bf1a80e53","type":"inject","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":135,"y":320,"wires":[["e86e88f88a40efb6","b849936960a52481"]],"l":false},{"id":"bbe4584c319f7229","type":"debug","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":615,"y":300,"wires":[],"l":false},{"id":"3e9e4bcb14727c44","type":"change","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"previousmem","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":615,"y":340,"wires":[["83e79b7f1579119c"]],"l":false},{"id":"c6a459bb12f666ea","type":"change","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"currentmem","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":555,"y":380,"wires":[["83e79b7f1579119c"]],"l":false},{"id":"83e79b7f1579119c","type":"join","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":655,"y":380,"wires":[["75832cf54d72f14b","15a32dd96a88d924"]],"l":false},{"id":"15a32dd96a88d924","type":"change","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$abs(payload.currentmem-payload.previousmem)\t","tot":"jsonata"},{"t":"set","p":"topic","pt":"msg","to":"CPU_Usage","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":715,"y":380,"wires":[["4d197a204dbdc20a"]],"l":false},{"id":"75832cf54d72f14b","type":"debug","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":695,"y":340,"wires":[],"l":false},{"id":"4d197a204dbdc20a","type":"debug","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":775,"y":380,"wires":[],"l":false},{"id":"b849936960a52481","type":"delay","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","pauseType":"delay","timeout":"3","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":255,"y":380,"wires":[["78696e2b89db404a"]],"l":false},{"id":"78696e2b89db404a","type":"cpu","z":"b96d5cc992392d6e","g":"ac401ac790e1ee2e","name":"","msgCore":false,"msgOverall":true,"msgArray":false,"msgTemp":false,"x":410,"y":380,"wires":[["c6a459bb12f666ea"]]}]

[Ekristoffe]

Hello,
About Memory Utilized, I see your calculation is:
( ( Total - Free Memory ) / Total * 100 )
Which can be calculated as:
( ( ( Total - Free Memory ) / Total ) * 100 ) [Multiply by 100 is for the whole division]
( ( ( Total / Total ) - ( Free Memory / Total ) ) * 100 ) [Divide all members of the subtraction]
( ( ( 1 ) - ( Free Memory / Total ) ) * 100 ) [Clean the easy part]
( ( ( 1 * 100 ) - ( ( Free Memory / Total ) * 100 ) ) [Multiply all members of the subtraction]
( ( ( 100 ) - ( ( Free Memory / Total ) * 100 ) ) [Clean the easy part]
( 100 - ( Free Memory / Total ) * 100 )
At the end we get the calculation of the pmem available in "memory > payload > memusage"

For the CPU I need to check later what we can do with the data available:
https://nodejs.org/api/os.html

[hobbes1069]

I have a slightly different request but I don't think it's worth submitting a new issue for.

How difficult would it be to report the free memory minus the cache memory? Currently I'm monitoring my RHEL 8 server and it's showing about 92% memory utilization which looks bad, but 3.5GB is being used for cache.

$ vmstat -S M
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0    665      8   3438    0    0     1    11   11    4  2  1 98  0  0

[bakelitephone]

I have a slightly different request but I don't think it's worth submitting a new issue for.

How difficult would it be to report the free memory minus the cache memory? Currently I'm monitoring my RHEL 8 server and it's showing about 92% memory utilization which looks bad, but 3.5GB is being used for cache.

$ vmstat -S M
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0    665      8   3438    0    0     1    11   11    4  2  1 98  0  0

Hi I have the same problem with my Debian boxs, one shows 98.65% memory used because the available buffer/cache memory is not taken into account.
The real memory available is 60% with only used 30% and it is very confusing, please help if you can

[bakelitephone]

FYI, regarding my last comment this is the result of my free -h command:
free -h
total used free shared buff/cache available
Mem: 15Gi 5.1Gi 245Mi 74Mi 10Gi 10Gi
Swap: 8.0Gi 3.0Mi 8.0Gi
As you can see the real free memory is 10Gi NOT 245Mi but the node red memory node says 98.65% used.
It would be fantastic if the node could also expose this "available: memory as part of the object so we could pick it.
This post explains the false memory issue in detail (https://www.linuxatemyram.com)

[hobbes1069]

Poking around this project a bit I don't see very much activity. I'm afraid someone may need to fork it to keep it going or see if any of the current contributors will add someone to the project that's interested in maintenance.

[Ekristoffe]

Hello guys, I am sorry but this is not an option.
Unless there is a solution using only node function and completely multi-platforms (Linux, BSD, Windows).
All the function you hinted only work under Linux and this is not possible unless the node drop other OS support.
This will be more a regression for some users.

Edit: I may have found something which could work.
But this also mean adding a dependency.
The solution would be using this node:
https://www.npmjs.com/package/systeminformation