Support for netbox 4.x #176

dBitech · 2024-05-11T11:40:36Z

There were changes in netbox 3.7 which were further deprecated in 4.0 Please see Migrating Your Plugin to NetBox v4.0 for porting instructions to make the plugin compatible with 4.x

xXAzazelXx · 2024-07-21T05:55:23Z

Yep it's broken at the moment.
ModuleNotFoundError: No module named 'extras.plugins'

OberMarcLP · 2024-08-22T21:12:53Z

Can I do something to fix it, or must I wait until a new Version of Proxbox is coming out?

Uzoma8Chi · 2024-09-06T12:48:47Z

Why do you not make a new release for 4.x? I already see commit.
de2bb66c78be3cf8d20c93afd2f0e3c5ed6b3d9c\
Since I use docker I do not know how to use non released changes.

fibretel · 2024-09-15T08:53:40Z

Why do you not make a new release for 4.x? I already see commit. [de2bb66](https://github.com/netdevopsbr/netbox-proxbox/commit/de2bb66c78be3cf8d20c93afd2f0e3c5ed6b3d9c) Since I use docker I do not know how to use non released changes.

The github version installs on 4.x, but it still doesn't seem to work on the latest netbox release

psiess · 2024-09-19T15:28:37Z

I'm having the same problem with Netbox 4.1.1. When trying to run python3 manage.py migrate I get the same errors as @Tehniels2000.

Any ETA for a fix for this problem?

emersonfelipesp · 2024-09-25T13:56:29Z

Currently working on support for latest Netbox version.

psiess · 2024-09-25T14:28:03Z

Currently working on support for latest Netbox version.

Excellent. Thank you for the letting us know.

maxdallmair · 2024-10-10T20:48:28Z

Hello, sorry to push this, but is there any ETA?

emersonfelipesp · 2024-10-11T15:29:51Z

Currently working on it, on my test env everything is going fine. You guys can already test it. But I'm improving testing and corner cases, as it's not easy to predict all Proxmox environments.

emersonfelipesp · 2024-10-11T15:32:09Z

I would like to also note that although I love the project and its function, I do it as best-effort and have no return. I'm really glad the amount of people that use it, but I ask for some patience, if possible.

psiess · 2024-10-17T15:58:37Z

Thank you for all your work on this, Emerson. How can I get access to test the update that supports 4.x?

…

________________________________ From: Emerson Felipe ***@***.***> Sent: October 11, 2024 11:30 AM To: netdevopsbr/netbox-proxbox ***@***.***> Cc: Paul Siess ***@***.***>; Comment ***@***.***> Subject: Re: [netdevopsbr/netbox-proxbox] Support for netbox 4.x (Issue #176) Currently working on it, on my test env everything is going fine. You guys can already test it. But I'm improving testing and corner cases, as it's not easy to predict all Proxmox environments. — Reply to this email directly, view it on GitHub<#176 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABWEE446ILQX5JVR2S2RDRDZ27VINAVCNFSM6AAAAABHR3Q4KOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXGY2TIOBQGE>. You are receiving this because you commented.Message ID: ***@***.***>

spacecodeit · 2024-10-18T09:35:51Z

Hello,
I tried to install the current develop branch version but I get the following error:

The complete exception is provided below:

<class 'KeyError'>

'fastapi'

Python version: 3.11.2
NetBox version: 4.1.4
Plugins: 
  netbox_proxbox: 0.0.5

But based on pip list fastapi is installed fastapi 0.115.2.

Do you have any Ideas what I'm doing wrong?

emersonfelipesp · 2024-10-18T15:48:48Z

Hi @spacecodeit could you please the last commits and confirm that fastapi is installed on Netbox Virtual Venv (/opt/netbox/venv)?

Also, now on Proxbox plugin homepage, there is connection status to the services and log messages on the bottom, as you click on Proxmox Full Update button. Send me the output log messages, if possible.

emersonfelipesp · 2024-10-18T15:49:44Z

Hi @psiess, you just need to pull the develop (using git checkout develop) branch and you're fine. I'm currently working on it, but plugin should work.

psiess · 2024-10-18T18:13:23Z

Hi @psiess, you just need to pull the develop (using git checkout develop) branch and you're fine. I'm currently working on it, but plugin should work.

Sorry for the newbie question. How do I pull the develop branch? I cloned the repository and can see the branches

git branch -a

develop
remotes/origin/HEAD -> origin/develop
remotes/origin/auto-update
remotes/origin/backend
remotes/origin/develop
remotes/origin/gh-pages
remotes/origin/main
remotes/origin/optimize-performance
remotes/origin/proxmox-folder-view

...but I can't figure out how to pull the develop branch. I tried git clone develop and git pull develop but neither work.

emersonfelipesp · 2024-10-18T18:33:10Z

I think you already switched to develop.
If not, just issue the git checkout develop command and then git branch to check if it worked.

psiess · 2024-10-18T18:40:23Z

You are right @emersonfelipesp. I was able to install netbox-proxbox, but get an error right at the end of the installation:

Installed /opt/netbox/venv/lib/python3.12/site-packages/fastapi_cli-0.0.5-py3.12.egg
error: starlette 0.41.0 is installed but starlette<0.41.0,>=0.37.2 is required by {'fastapi'}

Can I downgrade starlette without messing up the rest of netbox? Or install a lower version alongside the 0.41.0?

emersonfelipesp · 2024-10-18T18:57:51Z

I'm currently using starlette=0.38.6

Yo can try this version with: pip install -Iv starlette=0.38.6

emersonfelipesp · 2024-10-18T19:04:46Z

@psiess try using the git pull again to get my last commits. I have added/changed:

Added documentation on how to enable Proxbox backend in FastAPI
Updated requirements.txt so that most important packages has a fixed version to install

Let me now if it works!

emersonfelipesp · 2024-10-18T19:14:13Z

I actually had to use starlette==0.40.0 as Github pointed out the previous version had a major security issue.
Just tested 0.40.0 version and apparently looks good.

psiess · 2024-10-18T19:35:25Z

Do I need to re-install after the new pull?

emersonfelipesp · 2024-10-18T20:19:30Z

@psiess I'm not sure, but would be good.

troycarpenter · 2024-10-18T23:11:14Z

I also want to be sure I'm doing this right. I currently go into /opt/netbox/netbox/netbox-proxbox and do a git pull to get the latest. Then, in /opt/netbox I execute ./update.sh with netbox-proxbox in the local_requirements file.

I'm not sure it's working because Netbox doesn't start with the plugin activated. If I remove netbox_proxbox from the configuration.py file that Netbox will start.

Is there a different way I should be doing that?

emersonfelipesp · 2024-10-19T12:33:19Z

Hi @troycarpenter, could you se Netbox logs? I don't get why it didn't work. I'm putting a lot of effort on Docs and Proxbox Plugin GUI so that User knows exactly what is going on. But for now, I'll need some help from you guys to find the bugs.

troycarpenter · 2024-10-19T13:29:13Z

I have tried all the ways I've found online to get Netbox to log to a file, but nothing has worked for me. It's gone as far as creating the log file, but nothing is ever logged there. In configuration.py, I have this:

DEBUG = True
import logging.handlers
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'normal': {
            'format': '%(asctime)s %(name)s %(levelname)s: %(message)s'
        },
    },
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/var/log/netbox/netbox.log',
            'formatter': 'normal',
        },
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'INFO',
        },
        'netbox': {
            'handlers': ['file'],
            'level': 'INFO',
        },
    },
}

That logging section is now a frankenstien based on a few websites.

emersonfelipesp · 2024-11-26T11:54:03Z

Hi @troycarpenter, thank your for your feedback. I'm still coding this part.
As soon as I get Proxbox working on v4, I will then work on robust CI/CD pipelines to avoid this.

troycarpenter · 2024-11-26T16:08:27Z

Thanks. The latest updates with git pull seems to have fixed the error. Now working to get all the processes talking again.

troycarpenter · 2024-11-26T16:19:56Z

Ok, got all the services showing up as "successful" in the Netbox GUI. I hit the Full Update button and got this in the log box:

Getting Cluster Type from Netbox.
 Cluster Type found on Netbox. Checking if it's 'Proxbox' one...
 The Cluster Type found is from 'Proxbox' (because it has the tag). Returning it.
Failed to fetch cluster type for cluster: Proxbox Basic Cluster. As it is a required field, the cluster will not be created.
 Getting Cluster from Netbox.
 Searching Cluster by kwargs {'name': 'pve'}.
 Creating Device pve related with the Virtual Machine(s)
 Getting Site from Netbox.
 Site found on Netbox. Checking if it's 'Proxbox' one...
 The Site found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Device Types from Netbox.
 Device Types found on Netbox. Checking if it's 'Proxbox' one...
 The Device Types found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Manufacturer from Netbox.
 Manufacturer found on Netbox. Checking if it's 'Proxbox' one...
 The Manufacturer found is from 'Proxbox' (because it has the tag). Returning it.
Failed to fetch manufacturer for device type: Proxbox Basic Device Type
 Getting Device Types from Netbox.
 Device Types found on Netbox. Checking if it's 'Proxbox' one...
 The Device Types found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Cluster Type from Netbox.
 Cluster Type found on Netbox. Checking if it's 'Proxbox' one...
 The Cluster Type found is from 'Proxbox' (because it has the tag). Returning it.
Failed to fetch cluster type for cluster: Proxbox Basic Cluster. As it is a required field, the cluster will not be created.
 Getting Cluster from Netbox.
 Cluster found on Netbox. Checking if it's 'Proxbox' one...
 The Cluster found is from 'Proxbox' (because it has the tag). Returning it.
Error trying to create Netbox Device object.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
 Nodes: []
 Getting Site from Netbox.
 Site found on Netbox. Checking if it's 'Proxbox' one...
 The Site found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Device Types from Netbox.
 Device Types found on Netbox. Checking if it's 'Proxbox' one...
 The Device Types found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Device Types from Netbox.
 Device Types found on Netbox. Checking if it's 'Proxbox' one...
 The Device Types found is from 'Proxbox' (because it has the tag). Returning it.
 Getting Cluster Type from Netbox.
 Cluster Type found on Netbox. Checking if it's 'Proxbox' one...
 The Cluster Type found is from 'Proxbox' (because it has the tag). Returning it.
Failed to fetch cluster type for cluster: Proxbox Basic Cluster. As it is a required field, the cluster will not be created.
 Getting Cluster from Netbox.
 Cluster found on Netbox. Checking if it's 'Proxbox' one...
 The Cluster found is from 'Proxbox' (because it has the tag). Returning it.

HOWEVER, I don't exactly know what happened. I don't see any new VMs created (that is, new ones in Proxmox but were not in Netbox). And I can't tell if existing entities were changed in Netbox to match Proxmox.

I do find this error in the log:

Nov 26 16:53:52 netbox uvicorn[136105]: ERROR:    Exception in ASGI application
Nov 26 16:53:52 netbox uvicorn[136105]: Traceback (most recent call last):
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 242, in run_asgi
Nov 26 16:53:52 netbox uvicorn[136105]:     result = await self.app(self.scope, self.asgi_receive, self.asgi_send)  # type: ignore[func-returns-value]
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     return await self.app(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await super().__call__(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await self.middleware_stack(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 152, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await self.app(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 77, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await self.app(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
Nov 26 16:53:52 netbox uvicorn[136105]:     raise exc
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
Nov 26 16:53:52 netbox uvicorn[136105]:     await app(scope, receive, sender)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in __call__
Nov 26 16:53:52 netbox uvicorn[136105]:     await self.middleware_stack(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
Nov 26 16:53:52 netbox uvicorn[136105]:     await route.handle(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 362, in handle
Nov 26 16:53:52 netbox uvicorn[136105]:     await self.app(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 95, in app
Nov 26 16:53:52 netbox uvicorn[136105]:     await wrap_app_handling_exceptions(app, session)(scope, receive, send)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
Nov 26 16:53:52 netbox uvicorn[136105]:     raise exc
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
Nov 26 16:53:52 netbox uvicorn[136105]:     await app(scope, receive, sender)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 93, in app
Nov 26 16:53:52 netbox uvicorn[136105]:     await func(session)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/venv/lib/python3.10/site-packages/fastapi/routing.py", line 383, in app
Nov 26 16:53:52 netbox uvicorn[136105]:     await dependant.call(**solved_result.values)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/main.py", line 175, in websocket_endpoint
Nov 26 16:53:52 netbox uvicorn[136105]:     await get_virtual_machines(nb=nb, pxs=pxs, websocket=websocket)
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/backend/routes/proxbox/clusters/__init__.py", line 520, in get_virtual_machines
Nov 26 16:53:52 netbox uvicorn[136105]:     devices[vm_node] = await Device(nb = nb, websocket = websocket).get(name = vm.get("node"))
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/backend/routes/netbox/generic.py", line 95, in get
Nov 26 16:53:52 netbox uvicorn[136105]:     self.base_dict = await self.get_base_dict()
Nov 26 16:53:52 netbox uvicorn[136105]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/backend/routes/netbox/dcim/devices.py", line 29, in get_base_dict
Nov 26 16:53:52 netbox uvicorn[136105]:     "site": site.id,
Nov 26 16:53:52 netbox uvicorn[136105]: AttributeError: 'NoneType' object has no attribute 'id'
Nov 26 16:53:52 netbox uvicorn[136105]: INFO:     connection closed

emersonfelipesp · 2024-11-26T20:52:59Z

Hi @troycarpenter I will review the logs you provided as soon as possible and I'll back with a solution. Thank you!

emersonfelipesp · 2024-11-27T14:46:27Z

HOWEVER, I don't exactly know what happened. I don't see any new VMs created (that is, new ones in Proxmox but were not in > Netbox). And I can't tell if existing entities were changed in Netbox to match Proxmox.

@troycarpenter trying to solve it, I'm improving a bit the GUI to be more informative other than just logs.

When clicking to Sync Virtual Machines, a new table is created and populated as Proxbox updates Netbox. The image below is the first version, but I will add styling and links (like VM Name will link to the VM on Netbox directly)

troycarpenter · 2024-12-02T18:26:30Z

I tried the improve-gui branch, but the full update button now gives this error:

Dec  2 18:19:07 netbox uvicorn[166683]: ERROR:    Exception in ASGI application
Dec  2 18:19:07 netbox uvicorn[166683]: Traceback (most recent call last):
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 242, in run_asgi
Dec  2 18:19:07 netbox uvicorn[166683]:     result = await self.app(self.scope, self.asgi_receive, self.asgi_send)  # type: ignore[func-returns-value]
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     return await self.app(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await super().__call__(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await self.middleware_stack(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 152, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await self.app(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 77, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await self.app(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
Dec  2 18:19:07 netbox uvicorn[166683]:     raise exc
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
Dec  2 18:19:07 netbox uvicorn[166683]:     await app(scope, receive, sender)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in __call__
Dec  2 18:19:07 netbox uvicorn[166683]:     await self.middleware_stack(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
Dec  2 18:19:07 netbox uvicorn[166683]:     await route.handle(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 362, in handle
Dec  2 18:19:07 netbox uvicorn[166683]:     await self.app(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 95, in app
Dec  2 18:19:07 netbox uvicorn[166683]:     await wrap_app_handling_exceptions(app, session)(scope, receive, send)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
Dec  2 18:19:07 netbox uvicorn[166683]:     raise exc
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
Dec  2 18:19:07 netbox uvicorn[166683]:     await app(scope, receive, sender)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/routing.py", line 93, in app
Dec  2 18:19:07 netbox uvicorn[166683]:     await func(session)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/fastapi/routing.py", line 383, in app
Dec  2 18:19:07 netbox uvicorn[166683]:     await dependant.call(**solved_result.values)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/main.py", line 184, in websocket_endpoint
Dec  2 18:19:07 netbox uvicorn[166683]:     await get_virtual_machines(nb=nb, pxs=pxs, websocket=websocket)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/netbox/netbox-proxbox/netbox_proxbox/backend/routes/proxbox/clusters/__init__.py", line 554, in get_virtual_machines
Dec  2 18:19:07 netbox uvicorn[166683]:     await websocket.send_json({'object': 'cluster', 'type': 'update', 'data': cluster})
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/opt/netbox/venv/lib/python3.10/site-packages/starlette/websockets.py", line 173, in send_json
Dec  2 18:19:07 netbox uvicorn[166683]:     text = json.dumps(data, separators=(",", ":"), ensure_ascii=False)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/usr/lib/python3.10/json/__init__.py", line 238, in dumps
Dec  2 18:19:07 netbox uvicorn[166683]:     **kw).encode(obj)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
Dec  2 18:19:07 netbox uvicorn[166683]:     chunks = self.iterencode(o, _one_shot=True)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
Dec  2 18:19:07 netbox uvicorn[166683]:     return _iterencode(o, 0)
Dec  2 18:19:07 netbox uvicorn[166683]:   File "/usr/lib/python3.10/json/encoder.py", line 179, in default
Dec  2 18:19:07 netbox uvicorn[166683]:     raise TypeError(f'Object of type {o.__class__.__name__} '
Dec  2 18:19:07 netbox uvicorn[166683]: TypeError: Object of type Record is not JSON serializable
Dec  2 18:19:07 netbox uvicorn[166683]: INFO:     connection closed

The screen shows the one VM where it crashed:

And in the log screen on the GUI:

 Searching Cluster by kwargs {'name': 'pve'}.
 Creating Device pve related with the Virtual Machine(s)
 Creating Device object on Netbox.
 Checking if Device exists on Netbox before creating it.
 (1) First attempt: Checking object making EXACT MATCH with the Payload provided...
 (1.5) Checking object using NAME and DEVICE provided by the Payload and also the PROXBOX TAG. If found, return it.
 (1.5.1) Checking duplicate using Device Object as parameter.
 (1.5.1.2) Device Object NOT found using Device ID as parameter. Trying to use Device NAME as parameter.
 Not abe to get Device Object using Device NAME neither Device ID as parameter.
 (2) Checking object using only NAME and SLUG provided by the Payload and also the PROXBOX TAG. If found, return it.
 (2.1) Searching object using GET method
 (3) Checking duplicate object using only NAME and SLUG
 Trying to create Device object on Netbox.
 Error trying to create Device object on Netbox. > The request failed with code 400 Bad Request: {'device_type': ['This field is required.'], 'role': ['This field is required.'], 'site': ['This field is required.']}
Error trying to create Netbox Device object.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to search bridge child interface.
 Creating child interface of a bridge. Bridge interface and child must belong to the same device.
Error trying to create Netbox interface.
Error trying to create Netbox interface.
 Nodes: []
{"object":"virtual_machine","type":"create","data":{"disk":0,"diskread":142561602494,"template":0,"id":"qemu/101","cpu":0.0389353342545002,"netout":1871273994816,"maxcpu":8,"status":"running","maxdisk":322122547200,"mem":29327962112,"name":"flawless","diskwrite":2050943607808,"node":"pve","netin":2667201203987,"vmid":101,"maxmem":34359738368,"type":"qemu","uptime":7791013}}
{"object":"device","type":"create","data":{"name":"pve"}}
 Searching Device by kwargs {'name': 'pve'}.
{"object":"device","type":"update","data":null}
 Searching Cluster by kwargs {'name': 'pve'}.

emersonfelipesp · 2024-12-07T12:05:32Z

@troycarpenter this branch isn't working yet, but I will fix it this weekend. I got no time this past week to work on the plugin, but I think next days will be better.

quiknick · 2024-12-21T05:28:36Z

No matter what I do I can't get past this error. tried v4.1.9 and v4.1.0

{
  "message": "Plugin configuration at PLUGINS_CONFIG (configuration.py) is probably incorrect.",
  "detail": "Could not feed 'PluginConfig' pydantic model with config provided from 'PLUGINS_CONFIG'.",
  "python_exception": "1 validation error for PluginConfig\nproxmox\n  Input should be a valid list [type=list_type, input_value={'domain': 'netbox.domain.n...'}, 'ssl': False}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/list_type"
}

Netbox Proxbox configuration and Proxbox Default config found.
Netbox Proxbox configuration found.
INFO:     Started server process [2499657]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on https://0.0.0.0:8800 (Press CTRL+C to quit)
INFO:     x.x.x.x:35610 - "GET /docs HTTP/1.1" 200 OK
INFO:     x.x.x.x:35610 - "GET /openapi.json HTTP/1.1" 200 OK
INFO:     x.x.x.x:21596 - "GET /netbox/status?list_all=false&plugin_name=netbox_proxbox HTTP/1.1" 400 Bad Request

This could be related to this unresolved pull request fastapi pull request 9928

I also saw this in pydantic Issue 7669

The reason why it doesn't work like it did in V1 is that model validation is now done with the pydantic-core rust package. So instead of calling each submodel's model_validate (or from_orm) recursively in python, only the outermost model's method is called. The control flow then is passed to rust code, which performs validation purely based on the pydantic-core schema of this outermost model. If you want to have hooks back into python code you need to explicitly include them in the pydantic-core schema, either via the get_pydantic_core_schema method, model/field validators, Annotated metadata, or similar pydantic mechanisms. You cannot rely on the python method name resolution to find your custom method since these lookups are never attempted.

Note that this is the intended behaviour and cannot be "fixed", the whole reason why V2 is so much faster and offers other great benefits is that validation logic is performed in rust based on the core schema.

bk-lg · 2025-01-06T05:12:45Z

Hey everybody,
I hope you had great holidays and a good start into the new year.

@emersonfelipesp I don't want to make any pressure into your direction but are you optimistic that there will be a release of a stable new release in the next 1-2 weeks? Is there anything you need to progress further?
I am migrating from racktables to netbox in the moment and need to document a lot of proxmox vms and would love to use this integration instead of wasting hours on manualy copying information.

If you think this will take more time is this no problem. Then I'll do it by hand and will start using this integration sometimes in the future. Thank you for your great work.

emersonfelipesp · 2025-01-07T19:54:48Z

Hello @bk-lg ! Unfortunately this week I will not be able to make any updates to the code, but next week I will launch a new release of Proxbox which is supposed to work correctly with the latest Netbox version. As soon as I publish this new version, I will come back here asking you guys to test it.

I'm currently working on the already found bugs and refactoring the code to use HTMX and Alpine.js, as the current code uses JavaScript vanilla.

Thank you for your interest and I assure you in two weeks you'll a new version!

troycarpenter · 2025-01-07T20:08:14Z

Do you mean 4.1.11 or 4.2.0 (or 4.2.1 which I'm told should be here shortly), or any 4.x release?

bk-lg · 2025-01-08T07:19:03Z

@emersonfelipesp thank you for your response. This works for me perfectly. I'll be glad to help testing the release!

emersonfelipesp · 2025-01-08T12:58:15Z

Do you mean 4.1.11 or 4.2.0 (or 4.2.1 which I'm told should be here shortly), or any 4.x release?

Hi @troycarpenter ! It should work with any Netbox release above v4, but I always develop using the latest one, that is v4.2.0. I plan to be up to date with the latest version so that the plugin does not break as much it did when v4 was launched. But I also think moving to a API-first approach will make plugin break less, because while Netbox or Proxmox does not release any API breaking change version, plugin should work fine.

Maybe I'll use Django Models in the future, subclassing Netbox provided classes, but for now my plugin is simply based on API communication with pynetbox and proxmoxer and extending Netbox layout templates!

I also want to implement more CI/CD tasks and unit tests to prevent failures.

emersonfelipesp · 2025-01-17T21:55:36Z

Hello guys, just giving some updates:

Current work:

I'm working on some parallel issues, but giving special attention to #224 and its branch expose-cfg-via-api which will allow me to:

Eliminate the use of PLUGINS_CONFIG configuration, doing it all via GUI or API, making it way easier to install and use the plugin.
Getting Proxmox and Netbox info via API will permit a better separation of the Plugin (GUI) part of the code and the Backend one. I'm even considering moving all the FastAPI app to a new repository (making some testing on it).
All this will then result in less breaking changes and a easier implementation of unit tests in the future.
Improve security between all the communications, Netbox and Proxbox Backend are already by default using self-signed certificates, but there's a lot of improvements to do. I will use TLS by default between all requests.
Transform the project into a Netbox Plugin indeed, making use of all Netbox offers to plugin development, mainly using Models to implement new Views and bring a lot more of information than just the existing Netbox models permits.

Example of new Proxmox Endpoints CRUD operations directly on GUI:

OBS: The final work will probably not be exactly as the images below (it will be better).

Endpoints List

Add/Edit Proxmox Endpoint

I'm using Netbox latest version v4.2.2 released today to develop the plugin.

Thank you again for all patience and collaboration on this.

Nyak95 · 2025-01-22T08:50:51Z

@emersonfelipesp
Good afternoon, how long do you think it will take for it to work on the new version of Netbox in the docker?

emersonfelipesp · 2025-01-22T16:41:03Z

Hello @Nyak95, I'm finishing a lot of pending PR's, but it is hard to tell you a exactly date. I hope this month (january) the main tasks will be done.

ctark · 2025-01-24T03:00:43Z

Hello @Nyak95, I'm fiishing a lot of peding PR's, but it is hard to tell you a exactly date. I hope this month (january) the main tasks will be done.

Looking forward to it! Thank you for all the hard work :D

emersonfelipesp · 2025-01-24T19:31:28Z

I've just finished moving Proxbox configuration from configuration.py file to GUI and API. The Proxbox API (FastAPI app) is also getting the information only from API now. Maybe I'll create a auto migration from configuration.py to the new way, only if necessary.

There are two pending issues to be done that I think will be enough to plugin start working on Netbox 4.x:

Refactor Javascript Vanilla to Alpine.js/HTMX #215
- This is issue is now the most important one, where I will solve most of the known bugs while I migrate the code to HTMX and Alpine to follow NetBox new stack.
- I'm already using HTMX to get connection status on plugin homepage and it seems to work great. I don't think it will be so difficult to migrate, let us see.
Automatically populate NetBoxEndpoint and FastAPIEndpoint models and starts Proxbox Backend service. #228
- This issue will just make easier to install and use Proxbox, but it is not needed for now, you just have add the information manually and it should be fine.

This comment was marked as duplicate.

Sign in to view

emersonfelipesp added accepted This issue has been accepted for implementation under review Further discussion is needed to determine this issue's scope and/or implementation labels Oct 11, 2024

emersonfelipesp self-assigned this Oct 11, 2024

emersonfelipesp pinned this issue Oct 11, 2024

felbinger mentioned this issue Jan 17, 2025

python312Packages.netbox-proxbox: init at 0.0.5 NixOS/nixpkgs#374518

Draft

13 tasks

Support for netbox 4.x #176

Support for netbox 4.x #176

Comments

dBitech commented May 11, 2024

xXAzazelXx commented Jul 21, 2024 • edited Loading

OberMarcLP commented Aug 22, 2024

This comment was marked as duplicate.

Uzoma8Chi commented Sep 6, 2024

fibretel commented Sep 15, 2024

psiess commented Sep 19, 2024 • edited Loading

emersonfelipesp commented Sep 25, 2024

psiess commented Sep 25, 2024

maxdallmair commented Oct 10, 2024

emersonfelipesp commented Oct 11, 2024

emersonfelipesp commented Oct 11, 2024

psiess commented Oct 17, 2024 via email

spacecodeit commented Oct 18, 2024

emersonfelipesp commented Oct 18, 2024

emersonfelipesp commented Oct 18, 2024 • edited Loading

psiess commented Oct 18, 2024

emersonfelipesp commented Oct 18, 2024

psiess commented Oct 18, 2024

emersonfelipesp commented Oct 18, 2024 • edited Loading

emersonfelipesp commented Oct 18, 2024 • edited Loading

emersonfelipesp commented Oct 18, 2024 • edited Loading

psiess commented Oct 18, 2024

emersonfelipesp commented Oct 18, 2024

troycarpenter commented Oct 18, 2024

emersonfelipesp commented Oct 19, 2024

troycarpenter commented Oct 19, 2024

emersonfelipesp commented Nov 26, 2024

troycarpenter commented Nov 26, 2024

troycarpenter commented Nov 26, 2024 • edited Loading

emersonfelipesp commented Nov 26, 2024

emersonfelipesp commented Nov 27, 2024 • edited Loading

troycarpenter commented Dec 2, 2024

emersonfelipesp commented Dec 7, 2024

quiknick commented Dec 21, 2024 • edited Loading

bk-lg commented Jan 6, 2025

emersonfelipesp commented Jan 7, 2025 • edited Loading

troycarpenter commented Jan 7, 2025

bk-lg commented Jan 8, 2025

emersonfelipesp commented Jan 8, 2025

emersonfelipesp commented Jan 17, 2025

Current work:

Example of new Proxmox Endpoints CRUD operations directly on GUI:

Nyak95 commented Jan 22, 2025

emersonfelipesp commented Jan 22, 2025 • edited Loading

ctark commented Jan 24, 2025

emersonfelipesp commented Jan 24, 2025

xXAzazelXx commented Jul 21, 2024 •

edited

Loading

psiess commented Sep 19, 2024 •

edited

Loading

emersonfelipesp commented Oct 18, 2024 •

edited

Loading

emersonfelipesp commented Oct 18, 2024 •

edited

Loading

emersonfelipesp commented Oct 18, 2024 •

edited

Loading

emersonfelipesp commented Oct 18, 2024 •

edited

Loading

troycarpenter commented Nov 26, 2024 •

edited

Loading

emersonfelipesp commented Nov 27, 2024 •

edited

Loading

quiknick commented Dec 21, 2024 •

edited

Loading

emersonfelipesp commented Jan 7, 2025 •

edited

Loading

emersonfelipesp commented Jan 22, 2025 •

edited

Loading