Skip to content

Commit

Permalink
Merge pull request #232 from netdevopsbr/keepalive-status
Browse files Browse the repository at this point in the history
Closes #202 and #231 - Add keepalive status verification for each Proxbox service (NetBox, Proxmox and FastAPI)
  • Loading branch information
emersonfelipesp authored Jan 22, 2025
2 parents 2906d4c + ded67e3 commit 73f36ce
Show file tree
Hide file tree
Showing 68 changed files with 1,362 additions and 232 deletions.
61 changes: 61 additions & 0 deletions PAST_CONFIG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Configuring plugin in the old way (<=v0.0.5)

## Change Netbox '**[configuration.py](https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/configuration.example.py)**' to add PLUGIN parameters
The plugin's configuration is also located in **/opt/netbox/netbox/netbox/configuration.py**:

Replace the values with your own following the [Configuration Parameters](#2-configuration-parameters) section.

**OBS:** You do not need to configure all the parameters, only the one's different from the default values. It means that if you have some value equal to the one below, you can skip its configuration. For netbox you should ensure the domain/port either targets gunicorn or a true http port that is not redirected to https.

```python
PLUGINS_CONFIG = {
'netbox_proxbox': {
'proxmox': [
{
'domain': 'proxbox.example.com', # May also be IP address
'http_port': 8006,
'user': 'root@pam', # always required
'password': 'Strong@P4ssword', # only required, if you don't want to use token based authentication
'token': {
'name': 'tokenID', # Only type the token name and not the 'user@pam:tokenID' format
'value': '039az154-23b2-4be0-8d20-b66abc8c4686'
},
'ssl': False
},
# The following json is optional and applies only for multi-cluster use
{
'domain': 'proxbox2.example.com', # May also be IP address
'http_port': 8006,
'user': 'root@pam', # always required
'password': 'Strong@P4ssword', # only required, if you don't want to use token based authentication
'token': {
'name': 'tokenID', # Only type the token name and not the 'user@pam:tokenID' format
'value': '039az154-23b2-4be0-8d20-b66abc8c4686'
},
'ssl': False
}
],
'netbox': {
'domain': 'localhost', # Ensure localhost is added to ALLOWED_HOSTS
'http_port': 8001, # Gunicorn port.
'token': '0dd7cddfaee3b38bbffbd2937d44c4a03f9c9d38',
'settings': {
'virtualmachine_role_id' : 0,
'node_role_id' : 0,
'site_id': 0
}
},
'fastapi': {
# Uvicorn Host is (most of the time) the same as Netbox (as both servers run on the same machine)
'uvicorn_host': 'localhost',
'uvicorn_port': 8800, # Default Proxbox FastAPI port
# Although it seems weird, the sudo-user is necessary so that Proxbox automatically starts Proxbox Backend.
# It makes it more "plug-in", without the need to user input manual commands.
'sudo': {
'user': "sudo_enabled_user",
'password': 'Strong@P4ssword',
}
}
}
}
```
138 changes: 17 additions & 121 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,23 +68,17 @@ The following table shows the Netbox and Proxmox versions compatible (tested) wi
- [1.1.2. Using git (development use)](#112-using-git-development-use) - CURRENTLY WORKING
- [1.2. Enable the Plugin](#12-enable-the-plugin)
- [1.3. Configure Plugin](#13-configure-plugin)
- [1.3.1. Change Netbox 'configuration.py' to add PLUGIN parameters](#131-change-netbox-configurationpy-to-add-plugin-parameters)
- [1.3.2. Change Netbox 'settings.py' to include Proxbox Template directory](#132-change-netbox-settingspy-to-include-proxbox-template-directory)
- [1.4. Run Database Migrations](#14-run-database-migrations)
- [1.5. systemd Setup](#15-systemd-setup-proxbox-backend)
- [1.6 Restart WSGI Service](#15-restart-wsgi-service)

[2. Configuration Parameters](#2-configuration-parameters)
[2. Usage](#3-usage)

[3. Usage](#3-usage)
[3. Enable Logs](#4-enable-logs)

[4. Enable Logs](#4-enable-logs)
[4. Roadmap](#6-roadmap)

[5. Contributing](#5-contributing)

[6. Roadmap](#6-roadmap)

[7. Get Help from Community!](#7-get-help-from-community)
[5. Get Help from Community!](#7-get-help-from-community)

---

Expand Down Expand Up @@ -150,84 +144,18 @@ Enable the plugin in **/opt/netbox/netbox/netbox/configuration.py**:
PLUGINS = ['netbox_proxbox']
```

---

### 1.3. Configure Plugin

#### 1.3.1. Change Netbox '**[configuration.py](https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/configuration.example.py)**' to add PLUGIN parameters
The plugin's configuration is also located in **/opt/netbox/netbox/netbox/configuration.py**:

Replace the values with your own following the [Configuration Parameters](#2-configuration-parameters) section.

**OBS:** You do not need to configure all the parameters, only the one's different from the default values. It means that if you have some value equal to the one below, you can skip its configuration. For netbox you should ensure the domain/port either targets gunicorn or a true http port that is not redirected to https.

```python
PLUGINS_CONFIG = {
'netbox_proxbox': {
'proxmox': [
{
'domain': 'proxbox.example.com', # May also be IP address
'http_port': 8006,
'user': 'root@pam', # always required
'password': 'Strong@P4ssword', # only required, if you don't want to use token based authentication
'token': {
'name': 'tokenID', # Only type the token name and not the 'user@pam:tokenID' format
'value': '039az154-23b2-4be0-8d20-b66abc8c4686'
},
'ssl': False
},
# The following json is optional and applies only for multi-cluster use
{
'domain': 'proxbox2.example.com', # May also be IP address
'http_port': 8006,
'user': 'root@pam', # always required
'password': 'Strong@P4ssword', # only required, if you don't want to use token based authentication
'token': {
'name': 'tokenID', # Only type the token name and not the 'user@pam:tokenID' format
'value': '039az154-23b2-4be0-8d20-b66abc8c4686'
},
'ssl': False
}
],
'netbox': {
'domain': 'localhost', # Ensure localhost is added to ALLOWED_HOSTS
'http_port': 8001, # Gunicorn port.
'token': '0dd7cddfaee3b38bbffbd2937d44c4a03f9c9d38',
'settings': {
'virtualmachine_role_id' : 0,
'node_role_id' : 0,
'site_id': 0
}
},
'fastapi': {
# Uvicorn Host is (most of the time) the same as Netbox (as both servers run on the same machine)
'uvicorn_host': 'localhost',
'uvicorn_port': 8800, # Default Proxbox FastAPI port
# Although it seems weird, the sudo-user is necessary so that Proxbox automatically starts Proxbox Backend.
# It makes it more "plug-in", without the need to user input manual commands.
'sudo': {
'user': "sudo_enabled_user",
'password': 'Strong@P4ssword',
}
}
}
}
```


#### 1.3.2. Change Netbox '**[settings.py](https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/settings.py)**' to include Proxbox Template directory

> Probably on the next release of Netbox, it will not be necessary to make the configuration below! As the [Pull Request #8733](https://github.com/netbox-community/netbox/pull/8734) got merged to develop branch
**It is no longer necessary to modify the templates section in `settings.py` and you may revert any changes.**
All plugin configuration is now done using NetBox GUI or its API. You can check the old configuration way [here](./PAST_CONFIG.md).

---

### 1.4. Run Database Migrations

```
(venv) $ cd /opt/netbox/netbox/
(venv) $ python3 manage.py migrate
(venv) $ python3 manage.py migrate netbox_proxbox
(venv) $ python3 manage.py collectstatic --no-input
```

Expand Down Expand Up @@ -261,7 +189,7 @@ The certificates used are from Netbox, considering both applications are on the
If you plan to put Proxbox Backend in another host, I recommend creating another pair of certificates and enabling NGINX in front ot it.

```
/opt/netbox/venv/bin/uvicorn netbox-proxbox.netbox_proxbox.main:app --host 0.0.0.0 --port 8800 --app-dir /opt/netbox/netbox --ssl-keyfile=/etc/ssl/private/netbox.key --ssl-certfile=/etc/ssl/certs/netbox.crt --reload
/opt/netbox/venv/bin/uvicorn netbox-proxbox.proxbox_api.main:app --host 0.0.0.0 --port 8800 --app-dir /opt/netbox/netbox --ssl-keyfile=/etc/ssl/private/netbox.key --ssl-certfile=/etc/ssl/certs/netbox.crt --reload
```

#### (Developer Use Only) Creating self-signed certificates so Proxbox Backend (FastAPI) runs both HTTP and WS (Websocket) via TLS.
Expand Down Expand Up @@ -289,32 +217,7 @@ Restart the WSGI service to load the new plugin:

---

## 2. Configuration Parameters

The following options are available:

* `proxmox`: (List) Proxmox related configuration to use proxmoxer.
* `proxmox.domain`: (String) Domain or IP address of Proxmox.
* `proxmox.http_port`: (Integer) Proxmox HTTP port (default: 8006).
* `proxmox.user`: (String) Proxmox Username.
* `proxmox.password`: (String) Proxmox Password.
* `proxmox.token`: (Dict) Contains Proxmox TokenID (name) and Token Value (value).
* `proxmox.token.name`: (String) Proxmox TokenID.
* `proxmox.token.value`: (String) Proxmox Token Value.
* `proxmox.ssl`: (Bool) Defines the use of SSL (default: False).

* `netbox`: (Dict) Netbox related configuration to use pynetbox.
* `netbox.domain`: (String) Domain or IP address of Netbox. Ensure name or ip is added to `ALLOWED_HOSTS`
* `netbox.http_port`: (Integer) Netbox HTTP PORT (default: 8001). If you are not targeting gunicorn directly make sure the HTTP port is not redirected to HTTPS by your HTTP server.
* `netbox.token`: (String) Netbox Token Value.
* `netbox.settings`: (Dict) Default items of Netbox to be used by Proxbox.
- If not configured, Proxbox will automatically create a basic configuration to make it work.
- The ID of each item can be easily found on the URL of the item you want to use.
* `netbox.settings.virtualmachine_role_id`: (Integer) Role ID to be used by Proxbox when creating Virtual Machines
* `netbox.settings.node_role_id`: (Integer) Role ID to be used by Proxbox when creating Nodes (Devices)
* `netbox.settings.site_id` (Integer) Site ID to be used by Proxbox when creating Nodes (Devices)

## 3. Usage
## 2. Usage

If everything is working correctly, you should see in Netbox's navigation the **Proxmox VM/CT** button in **Plugins** dropdown list.

Expand All @@ -326,7 +229,7 @@ It will redirect you to a new page and you just have to wait until the plugin ru

---

## 4. Enable Logs
## 3. Enable Logs

So that Proxbox plugin logs what is happening to the terminal, copy the following code and paste to `configuration.py` Netbox configuration file:

Expand All @@ -351,24 +254,17 @@ Although the above standard configuration should do the trick to things work.

---

## 5. Contributing
Developing tools for this project based on [ntc-netbox-plugin-onboarding](https://github.com/networktocode/ntc-netbox-plugin-onboarding) repo.

Issues and pull requests are welcomed.

---

## 6. Roadmap
- Start using custom models to optimize the use of the Plugin and stop using 'Custom Fields'
- Automatically remove Nodes on Netbox when removed on Promox (as it already happens with Virtual Machines and Containers)
- Add individual update of VM/CT's and Nodes (currently is only possible to update all at once)
- Add periodic update of the whole environment so that the user does not need to manually click the update button.
- Create virtual machines and containers directly on Netbox, having no need to access Proxmox.
- Add 'Console' button to enable console access to virtual machines
## 4. Roadmap
- [X] Start using custom models to optimize the use of the Plugin and stop using 'Custom Fields'
- [ ] Automatically remove Nodes on Netbox when removed on Promox (as it already happens with Virtual Machines and Containers)
- [ ] Add individual update of VM/CT's and Nodes (currently is only possible to update all at once)
- [ ] Add periodic update of the whole environment so that the user does not need to manually click the update button.
- [ ] Create virtual machines and containers directly on Netbox, having no need to access Proxmox.
- [ ] Add 'Console' button to enable console access to virtual machines

---

## 7. Get Help from Community!
## 5. Get Help from Community!
If you are struggling to get Proxbox working, feel free to contact someone from community (including me) to help you.
Below some of the communities available:
- **[Official - Slack Community (english)](https://netdev.chat/)**
Expand Down
2 changes: 1 addition & 1 deletion contrib/proxbox.service
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Group=netbox
PIDFile=/var/tmp/proxbox.pid
WorkingDirectory=/opt/netbox/netbox/netbox-proxbox

ExecStart=/opt/netbox/venv/bin/uvicorn netbox-proxbox.netbox_proxbox.main:app --host 0.0.0.0 --port 8800 --app-dir /opt/netbox/netbox --ssl-keyfile=/etc/ssl/private/netbox.key --ssl-certfile=/etc/ssl/certs/netbox.crt
ExecStart=/opt/netbox/venv/bin/uvicorn netbox-proxbox.proxbox_api.main:app --host 0.0.0.0 --port 8800 --app-dir /opt/netbox/netbox --ssl-keyfile=/etc/ssl/private/netbox.key --ssl-certfile=/etc/ssl/certs/netbox.crt

Restart=on-failure
RestartSec=30
Expand Down
12 changes: 0 additions & 12 deletions netbox_proxbox/backend/__init__.py

This file was deleted.

3 changes: 3 additions & 0 deletions netbox_proxbox/templates/netbox_proxbox/home.html
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
{% extends 'base/layout.html' %}
{% load static %}

{% block head %}
<script src="https://unpkg.com/[email protected]"></script>
{% endblock %}
{% block content %}
{% include "netbox_proxbox/home/script.html" %}

Expand Down
14 changes: 12 additions & 2 deletions netbox_proxbox/templates/netbox_proxbox/home/fastapi_card.html
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,18 @@ <h2 class="card-header">FastAPI (Backend Service)</h2>
</div>

<!-- Status Badge. Default value is 'Unkown' and is changed based on API response status -->
<div class="d-flex ms-auto " style="margin: 0 40px 0 0">
<span id="fastapi-connection-status" class="proxmox-connection-check badge p-1">Unknown</span>
<div
class="d-flex ms-auto"
style="margin: 0 40px 0 0"
>
<span
id="status-badge-{{ object.pk }}"
class="proxmox-connection-check badge p-1"
hx-get="{% url 'plugins:netbox_proxbox:keepalive_status' 'fastapi' object.pk %}"
hx-trigger="load"
hx-target="#status-badge-{{ object.pk }}"
hx-swap="outerHTML"
>Unknown</span>
</div>

</div>
Expand Down
14 changes: 12 additions & 2 deletions netbox_proxbox/templates/netbox_proxbox/home/netbox_card.html
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,18 @@ <h2 class="card-header">Netbox</h2>
</div>

<!-- Status Badge. Default value is 'Unkown' and is changed based on API response status -->
<div class="d-flex ms-auto " style="margin: 0 40px 0 0">
<span id="netbox-connection-status" class="netbox-connection-check badge p-1">Unknown</span>
<div
class="d-flex ms-auto"
style="margin: 0 40px 0 0"
>
<span
id="status-badge-{{ object.pk }}"
class="proxmox-connection-check badge p-1"
hx-get="{% url 'plugins:netbox_proxbox:keepalive_status' 'netbox' object.pk %}"
hx-trigger="load"
hx-target="#status-badge-{{ object.pk }}"
hx-swap="outerHTML"
>Unknown</span>
</div>

</div>
Expand Down
14 changes: 12 additions & 2 deletions netbox_proxbox/templates/netbox_proxbox/home/proxmox_card.html
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,18 @@ <h2 class="card-header">Proxmox Cluster</h2>
</div>

<!-- Status Badge. Default value is 'Unkown' and is changed based on API response status -->
<div class="d-flex ms-auto " style="margin: 0 40px 0 0">
<span id="proxmox-connection-status-{{ object.ip_address }}" class="proxmox-connection-check badge p-1">Unknown</span>
<div
class="d-flex ms-auto"
style="margin: 0 40px 0 0"
>
<span
id="status-badge-{{ object.pk }}"
class="proxmox-connection-check badge p-1"
hx-get="{% url 'plugins:netbox_proxbox:keepalive_status' 'proxmox' object.pk %}"
hx-trigger="load"
hx-target="#status-badge-{{ object.pk }}"
hx-swap="outerHTML"
>Unknown</span>
</div>

</div>
Expand Down
23 changes: 23 additions & 0 deletions netbox_proxbox/templates/netbox_proxbox/status_badge.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{% extends 'base/layout.html' %}

{% block content %}
{% if status == 'success' %}
<td id="{{ object.ip_address }}">
<span class='badge text-bg-green' title='Proxmox Cluster Mode'>
</strong>Successful!</strong>
</span>
</td>
{% elif status == 'error' %}
<td id="{{ object.ip_address }}">
<span class='badge text-bg-red' title='Proxmox Cluster Mode'>
</strong>Error!</strong>
</span>
</td>
{% else %}
<td id="{{ object.ip_address }}">
<span class='badge text-bg-grey' title='Proxmox Cluster Mode'>
</strong>Undefined</strong>
</span>
</td>
{% endif %}
{% endblock %}
6 changes: 4 additions & 2 deletions netbox_proxbox/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,5 +53,7 @@
path('endpoints/fastapi/<int:pk>/delete/', views.FastAPIEndpointDeleteView.as_view(), name='fastapiendpoint_delete'),
path('endpoints/fastapi/<int:pk>/changelog/', ObjectChangeLogView.as_view(), name='fastapiendpoint_changelog', kwargs={
'model': models.FastAPIEndpoint
})
]
}),

path('keepalive-status/<str:service>/<int:pk>', views.get_service_status, name='keepalive_status'),
]
Loading

0 comments on commit 73f36ce

Please sign in to comment.