Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when the main app is shutdown, prisma rust is still keeping the connection open in the background #995

Open
vikyw89 opened this issue Aug 6, 2024 · 3 comments

Comments

@vikyw89
Copy link

vikyw89 commented Aug 6, 2024

Bug description

image
and .env changes won't get reflected in the client (example changing url param, connection_limit, etc) which I think is caused by the old prisma rust is still active and using the old .env
image

How to reproduce

There are 2 ways:

  1. try to exhaust all connection pool in the db, and kill the client / port. Then try connecting again with less number of connection pool, it will raise this error
prisma.errors.DataError: Too many database connections opened: ERROR HY000 (1040): Too many connections```

2. run client, then kill server, try to change .env param and run again, client will still use the old .env param
<!--
Steps to reproduce the behavior:
1. Go to '...'
2. Change '....'
3. Run '....'
4. See error
-->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

prisma rust should shutdown when the main python prisma client is shutdown

## Prisma information

<!-- Your Prisma schema, Prisma Client Python queries, ...
Do not include your database credentials when sharing your Prisma schema! -->
latest as of 20240806


```prisma

Environment & setup

  • OS: ubuntu24
  • Database: mysql
  • Python version: 12
  • Prisma version: latest as of today 20240806
prisma client python    : 0.14.0
platform                : debian-openssl-3.0.x
expected engine version : 393aa359c9ad4a4bb28630fb5613f9c281cde053
installed extras        : []
install path            : /home/vikyw/.cache/pypoetry/virtualenvs/benchmark-cb5ao4-8-py3.12/lib/python3.12/site-packages/prisma
binary cache dir        : /home/vikyw/.cache/prisma-python/binaries/5.17.0/393aa359c9ad4a4bb28630fb5613f9c281cde053

@RobertCraigie
Copy link
Owner

@vikyw89 I can't reproduce this with mysql v8, could you share a script to reproduce the issue?

My script
import asyncio

from prisma import Prisma


async def check_mysql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("SHOW STATUS WHERE Variable_name = 'Threads_connected';")
    print(f"Total connections: {total_conn[0]['Value']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("SHOW VARIABLES LIKE 'max_connections';")
    print(f"Max connections allowed: {max_conn[0]['Value']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            ID,
            USER,
            HOST,
            DB,
            COMMAND,
            TIME,
            STATE,
            INFO
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        ORDER BY 
            TIME DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            USER,
            HOST,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            USER, HOST
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['USER']}@{row['HOST']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            STATE,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            STATE
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['STATE'] if row['STATE'] else 'NULL'
        print(f"{state}: {row['connection_count']}")


asyncio.run(check_mysql_connections())

Running this script always outputs the same information for me, I'd expect it to be different if the connections weren't actually being closed.

@vikyw89
Copy link
Author

vikyw89 commented Aug 27, 2024

Screencast.from.2024-08-28.03-32-29.mp4

@vikyw89
Copy link
Author

vikyw89 commented Aug 27, 2024

poetry run test_db.py script

def run():
    import asyncio
    
    asyncio.run(arun())
    
async def arun():

    # await check_mysql_connections()

    await check_postgresql_connections()
    
async def check_mysql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("SHOW STATUS WHERE Variable_name = 'Threads_connected';")
    print(f"Total connections: {total_conn[0]['Value']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("SHOW VARIABLES LIKE 'max_connections';")
    print(f"Max connections allowed: {max_conn[0]['Value']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            ID,
            USER,
            HOST,
            DB,
            COMMAND,
            TIME,
            STATE,
            INFO
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        ORDER BY 
            TIME DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            USER,
            HOST,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            USER, HOST
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['USER']}@{row['HOST']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            STATE,
            COUNT(*) AS connection_count
        FROM 
            INFORMATION_SCHEMA.PROCESSLIST
        GROUP BY 
            STATE
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['STATE'] if row['STATE'] else 'NULL'
        print(f"{state}: {row['connection_count']}")


async def check_postgresql_connections():
    prisma = Prisma()
    await prisma.connect()

    # Get total number of connections
    total_conn = await prisma.query_raw("""
        SELECT COUNT(*)
        FROM pg_stat_activity;
    """)
    print(f"Total connections: {total_conn[0]['count']}")

    # Get max allowed connections
    max_conn = await prisma.query_raw("""
        SELECT setting::int
        FROM pg_settings
        WHERE name = 'max_connections';
    """)
    print(f"Max connections allowed: {max_conn[0]['setting']}")

    # Get detailed information about current connections
    detailed = await prisma.query_raw("""
        SELECT 
            pid,
            usename,
            client_addr,
            datname,
            state,
            query,
            backend_start,
            query_start
        FROM 
            pg_stat_activity
        ORDER BY 
            query_start DESC
        LIMIT 10;
    """)
    print('\nDetailed connection information (latest 10):')
    for row in detailed:
        print(row)

    # Get connection counts by user and host
    user_host_counts = await prisma.query_raw("""
        SELECT 
            usename,
            client_addr,
            COUNT(*) AS connection_count
        FROM 
            pg_stat_activity
        GROUP BY 
            usename, client_addr
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by user and host:')
    for row in user_host_counts:
        print(f"{row['usename']}@{row['client_addr']}: {row['connection_count']}")

    # Get connection counts by state
    state_counts = await prisma.query_raw("""
        SELECT 
            state,
            COUNT(*) AS connection_count
        FROM 
            pg_stat_activity
        GROUP BY 
            state
        ORDER BY 
            connection_count DESC;
    """)
    print('\nConnection counts by state:')
    for row in state_counts:
        state = row['state'] if row['state'] else 'NULL'
        print(f"{state}: {row['connection_count']}")

    await prisma.disconnect()

poetry run start script

def run():
    import subprocess
    import multiprocessing
    
    cpu_count = multiprocessing.cpu_count()
    subprocess.run("poetry run prisma migrate deploy", shell=True)
    subprocess.run(f"poetry run uvicorn server.main:app --host 0.0.0.0 --port 8000 --workers {20}", shell=True)

this is my DATABASE_URL

DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres

this is my prisma init script

prisma = Prisma(auto_register=True, http=HttpConfig(timeout=Timeout(None,pool=None)))

I'm using fastapi server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants