Netgear DM200 to InfluxDB

This is a python script to webscrape the Netgear DM200 ADSL modem web interface and place data into InfluxDB for graphing in Grafana. Its intended use is on the NBN VDSL2 Network, specifically the DM-200. It will however work with other Netgear modems that use the same UI and others with some modification.

This assumes that Grafana and InfluxDB are already installed and working and that you have a basic understanding of both. This also assumes that the Netgear modem is accessible from within the end users LAN (with or without NAT rules etc).

Just a note, if there is a flap or dropout the Arris NTD will do a soft reboot thus blocking access to the WebUI requiring a factory reset using the reset pin for ~5 seconds. Unfortunately this is just how it is from the NBN overlords.

Click here for some end user setup guides for various consumer routers. Feel free to fork and contribute as desired.

Installation

Clone to machine. I chose to personally run this from the /opt/ directory.

sudo git clone https://github.com/risb0r/Arris-CM8200-to-InfluxDB.git arris_stats

Install python3, python3pip and python3-lxml as required.

sudo apt install python3 python3-pip python3-lxml

Use the pip package manager pip3 to install necessary requirements. Note: Requirements may also have to be installed as root depending on your setup.

cd arris_stats
pip3 install -r requirements.txt

Setup influx with a database

$ influx
> CREATE DATABASE vdsl_modem_stats

Ensure that the database was created

> show databases
name: databases
name
----
vdsl_modem_stats <------

Adjust cm8200_stats.py - Host, Port, Database, Username and Password as neccesary.

# Change settings below to your influxdb - database needs to be created or existing db creates 5 tables - downlink, uplink, fw_ver, uptime, event_log
# Second argument = default value if environment variable is not set.
influxip = os.environ.get("INFLUXDB_HOST", "127.0.0.1")
influxport = int(os.environ.get("INFLUXDB_HOST_PORT", "8086"))
influxdb = os.environ.get("INFLUXDB_DATABASE", "vdsl_modem_stats")
influxid = os.environ.get("INFLUXDB_USERNAME", "admin")
influxpass = os.environ.get("INFLUXDB_PASSWORD", "")

# cm8200b URL - Leave this unless your NTD URL is http://192.168.100.1
ntd_url = os.environ.get("NTD_URL", "http://192.168.104.1")

Usage

Running the webscraper

Standalone (once off)

/usr/bin/python3 /opt/arris_stats/cm8200b_stats.py

As cron

sudo crontab -e

Place the below into crontab. Ctrl + X to exit. This will run the script every 300 seconds.

# m h  dom mon dow   command

*/5 * * * * /usr/bin/python3 /opt/arris_stats/cm8200b_stats.py

Setting up Grafana

Setup the data source as below

Import the .json

If the images are out of wack check the grafana.ini file for the following config change.

$ sudo nano /etc/grafana/grafana.ini

[panels]
# If set to true Grafana will allow script tags in text panels. Not recommended as it enable XSS vulnerabilities.
disable_sanitize_html = true

To Do List

Auto scrape Whirlpool and plug in the CMTS info from the wiki rather than just filling out a text box. Personally, I can't be bothered or care too much for something that will go mostly unchanged.

Thanks

Thanks to Andy Fraley for the initial starting point and grafana json. Thanks to Luckst0r for the current python base code and doing some testing along with those who have made various contributions. There are a few of us lurking on the AussieBroadband Unofficial Discord if ther are any questions or in need of setup assitance.

Thanks to the team at Aussie Broadband for providing dope internet on a government cockup.

CMTS info is a static item and is available on Whirlpool thanks to Roger Ramband for making this data readily available. This data is only relevant to those connected via the NBN in Australia. Can be skipped, removed, deleted, whatever for others.