Push notifications to connected clients

[frankvp11]

Question

I'm looking to make a simple app right now to enable chatting between people.
Ie, there would be a database with users, and various users can agree to chat between each other, similar to Discord.
The details of that are irrelevant, but I'd like to basically be able to send message to specific IP's to update their UI / control the UI of specific users globally, and I was wondering if that was possible with NiceGUI.
To clarify, I want to be able to do the following:
2+ people join the webapp. It shows their usernames (for debugging, I did IPs). Then they should be able to click on the IP which sends a request to the person with that IP. That person can then accept, and they then join a conversation together.
The part that I can't seem to figure out is how to send that request to the person with a specific IP. I'm sure it's possible with websockets, but I can't seem to do that part. The other part that's confusing to me is updating the list of IP's globally. Right now, you'd need to refresh for this functionality, but it does technically work. Help would be appreciated there.

I'm not sure if this is possible to do in NiceGUI given how the backend side of things are setup, but I'd be interested in learning more!
Here is the sample code that I alluded to in the previous text.

from nicegui import ui, Client, app, observables

@ui.refreshable
def clients_cards():
    for client in clients:
            ui.label(client)

clients = observables.ObservableSet(on_change=clients_cards.refresh())


def connection(client: Client):
    clients.add(client.ip)
    


@ui.page("/")
def index():
    ui.label("Connected clients:")
    clients_cards()


app.on_connect(connection)


ui.run()

Let me know if any parts were unclear - I tried my best to explain it

[rodja]

Have you seen our chat app example?

[frankvp11]

I had not yet seen the chat app example.
My one question with this example is the following: suppose three people were to open the app. Is there a way to make it so that the communication is between only two of them / that there's a way for the users to select who they are chatting with?
Also, one of the goals is to extend it to video support. I tried looking into running a backend FastAPI server but I was not able to figure that out either. Any ideas?

[rodja]

Is there a way to make it so that the communication is between only two of them

Yes. It's all python code. You can implement it in that way.

I tried looking into running a backend FastAPI server

You don't need to. NiceGUI runs on top of FastAPI: from nicegui import app.

But for video integration you'll need also some js frontend code (see #2010).

[frankvp11]

Hmm, I've figured out some parts, but I'm stuck on one final one.
Right now, I managed to get it so that if you click on the right users that show up in the list of buttons on the left hand side, and then send a message, not everyone will get it (I still need to improve it, as guessing with Id's is not easy).
When I run it on HTTPS, I can get everyone to properly turn on video. However, the last part I am stuck on. Sharing the video doesn't seem to be working.
Any ideas are welcome :D

#!/usr/bin/env python3
from datetime import datetime
from typing import List, Tuple, Set
from uuid import uuid4

from nicegui import ui, app, Client

messages: List[Tuple[str, str, str, str]] = []
users: Set[str] = set()

messageContacts = {}

@ui.refreshable
def chat_messages(own_id: str) -> None:
    if messages:
        for user_id, avatar, text, stamp in messages:
            if user_id not in messageContacts or user_id not in messageContacts[own_id]:
                continue
            ui.chat_message(text=text, stamp=stamp, avatar=avatar, sent=own_id == user_id)
    else:
        ui.label('No messages yet').classes('mx-auto my-36')
    ui.run_javascript('window.scrollTo(0, document.body.scrollHeight)')

@ui.refreshable
def show_users(client):
    for user in users:
        ui.button(f'User {user}', on_click=lambda user=user, client=client: add_contact(user, client)).classes('text-xs text-gray-500 mx-2')
        video_frame(None)
def add_contact(user, client):
    if user in messageContacts:
        messageContacts[user].append(client.id)
    else:
        messageContacts[user] = [client.id]

@ui.refreshable
def video_frame(frame):
    ui.run_javascript(
        """
        const video = document.querySelector('video');
        const canvas = document.createElement('canvas');
        canvas.width = video.videoWidth;
        canvas.height = video.videoHeight;
        canvas.getContext('2d').drawImage(video, 0, 0);
        const quality = 0.5;
        compressToJpeg(canvas, quality).then(dataUrl => {
            const message = {
                type: 'frame',
                data: dataUrl
            };
            window.parent.postMessage(message, '*');
        });
        """
    )


def request_video():
    ui.run_javascript(
        """
        navigator.mediaDevices.getUserMedia({ video: true })
        .then(stream => {
            const video = document.createElement('video');
            video.srcObject = stream;
            video.play();
            document.body.appendChild(video);
        })
        .catch(err => console.error('Error accessing the camera:', err));

        """
    )


@ui.page('/')
async def main(client: Client):
    show_users(client)
    def send() -> None:
        stamp = datetime.now().strftime('%X')
        messages.append((user_id, avatar, text.value, stamp))
        text.value = ''
        chat_messages.refresh()

    user_id = client.id
    users.add(user_id)

    avatar = f'https://robohash.org/{user_id}?bgset=bg2'

    ui.add_css(r'a:link, a:visited {color: inherit !important; text-decoration: none; font-weight: 500}')


    with ui.footer().classes('bg-white'), ui.column().classes('w-full max-w-3xl mx-auto my-6'):
        with ui.row().classes('w-full no-wrap items-center'):
            with ui.avatar().on('click', lambda: ui.navigate.to(main)):
                ui.image(avatar)
            text = ui.input(placeholder='message').on('keydown.enter', send) \
                .props('rounded outlined input-class=mx-3').classes('flex-grow')


    ui.add_head_html(
        """
        <script>
            function compressToJpeg(canvas, quality) {
                return new Promise((resolve) => {
                    canvas.toBlob((blob) => {
                        const reader = new FileReader();
                        reader.onloadend = () => {
                            resolve(reader.result);
                        };
                        reader.readAsDataURL(blob);
                    }, 'image/jpeg', quality);
                });
            }
        </script>
        """
    )

    request_video()

    await ui.context.client.connected()  # chat_messages(...) uses run_javascript which is only possible after connecting
    with ui.column().classes('w-full max-w-2xl mx-auto items-stretch'):
        show_users.refresh()

    with ui.column().classes('w-full max-w-2xl mx-auto items-stretch'):
        chat_messages(user_id)


    ui.on('frame', lambda f: video_frame(f.args))



if __name__ in {'__main__', '__mp_main__'}:
    ui.run(on_air=True)

[falkoschindler]

Great progress!
For specific problems with video sharing, I'd recommend creating a new discussion with a minimum reproducible code example.