From 28fabee6cbd7cb865f85111d0ffa5fc92ec2ae67 Mon Sep 17 00:00:00 2001 From: Tony Locke Date: Fri, 29 Mar 2024 14:56:01 +0000 Subject: [PATCH] i --- README.md | 534 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 275 insertions(+), 259 deletions(-) diff --git a/README.md b/README.md index 6a9ef54..dcbd07a 100644 --- a/README.md +++ b/README.md @@ -56,12 +56,12 @@ query the table: ``` -Transactions -```````````` +### Transactions Here's how to run groups of SQL statements in a -`transaction `_: +[transaction](https://www.postgresql.org/docs/current/tutorial-transactions.html>): +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -81,9 +81,11 @@ Here's how to run groups of SQL statements in a [3, 'Phineas Finn'] >>> >>> con.close() +``` rolling back a transaction: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -104,19 +106,19 @@ rolling back a transaction: [3, 'Phineas Finn'] >>> >>> con.close() +``` -NB. There is `a longstanding bug `_ -in the PostgreSQL server whereby if a `COMMIT` is issued against a failed -transaction, the transaction is silently rolled back, rather than an error being -returned. pg8000 attempts to detect when this has happened and raise an -`InterfaceError`. +NB. There is [a longstanding bug](https://github.com/tlocke/pg8000/issues/36>) in the +PostgreSQL server whereby if a `COMMIT` is issued against a failed transaction, the +transaction is silently rolled back, rather than an error being returned. pg8000 +attempts to detect when this has happened and raise an `InterfaceError`. -Query Using Functions -````````````````````` +### Query Using Functions Another query, using some PostgreSQL functions: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -125,13 +127,14 @@ Another query, using some PostgreSQL functions: [['2021 AD']] >>> >>> con.close() +``` -Interval Type -````````````` +### Interval Type A query that returns the PostgreSQL interval type: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -143,15 +146,16 @@ A query that returns the PostgreSQL interval type: [[datetime.timedelta(days=12271, seconds=57960)]] >>> >>> con.close() +``` -Point Type -`````````` +### Point Type A round-trip with a -`PostgreSQL point `_ +[PostgreSQL point](https://www.postgresql.org/docs/current/datatype-geometric.html) type: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -160,17 +164,18 @@ type: [[(2.3, 1.0)]] >>> >>> con.close() +``` -Client Encoding -``````````````` +### Client Encoding When communicating with the server, pg8000 uses the character set that the server asks it to use (the client encoding). By default the client encoding is the database's character set (chosen when the database is created), but the client encoding can be -changed in a number of ways (eg. setting ``CLIENT_ENCODING`` in ``postgresql.conf``). +changed in a number of ways (eg. setting `CLIENT_ENCODING` in `postgresql.conf`). Another way of changing the client encoding is by using an SQL command. For example: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -180,15 +185,15 @@ Another way of changing the client encoding is by using an SQL command. For exam [['UTF8']] >>> >>> con.close() +``` +### JSON -JSON -```` - -`JSON `_ always comes back +[JSON](https://www.postgresql.org/docs/current/datatype-json.html) always comes back from the server de-serialized. If the JSON you want to send is a ``dict`` then you can just do: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -198,9 +203,11 @@ just do: [[{'age': 26.003, 'name': 'Apollo 11 Cave', 'zebra': True}]] >>> >>> con.close() +``` JSON can always be sent in serialized form to the server: +```python >>> import json >>> import pg8000.native >>> @@ -212,21 +219,24 @@ JSON can always be sent in serialized form to the server: [[['Apollo 11 Cave', True, 26.003]]] >>> >>> con.close() +``` JSON queries can be have parameters: +```python >>> import pg8000.native >>> >>> with pg8000.native.Connection("postgres", password="cpsnow") as con: ... con.run(""" SELECT CAST('{"a":1, "b":2}' AS jsonb) @> :v """, v={"b": 2}) [[True]] +``` -Retrieve Column Metadata From Results -````````````````````````````````````` +### Retrieve Column Metadata From Results Find the column metadata returned from a query: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -250,17 +260,18 @@ Find the column metadata returned from a query: ['id', 'name'] >>> >>> con.close() +``` -Notices And Notifications -````````````````````````` +### Notices And Notifications -PostgreSQL `notices -`_ are -stored in a deque called ``Connection.notices`` and added using the ``append()`` -method. Similarly there are ``Connection.notifications`` for `notifications -`_. Here's an example: +PostgreSQL [notices] +(https://www.postgresql.org/docs/current/static/plpgsql-errors-and-messages.html) are +stored in a deque called `Connection.notices` and added using the `append()` method. +Similarly there are `Connection.notifications` for [notifications] +(https://www.postgresql.org/docs/current/static/sql-notify.html). Here's an example: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -273,17 +284,19 @@ method. Similarly there are ``Connection.notifications`` for `notifications (..., 'aliens_landed', '') >>> >>> con.close() +``` -Parameter Statuses -`````````````````` +### Parameter Statuses -`Certain parameter values are reported by the server automatically at connection startup or whenever -their values change -`_ and pg8000 -stores the latest values in a dict called ``Connection.parameter_statuses``. Here's an example where -we set the ``aplication_name`` parameter and then read it from the ``parameter_statuses``: +[Certain parameter values are reported by the server automatically at connection startup or whenever +their values change] +(https://www.postgresql.org/docs/current/libpq-status.html#LIBPQ-PQPARAMETERSTATUS>) and +pg8000 stores the latest values in a dict called `Connection.parameter_statuses`. Here's +an example where we set the `aplication_name` parameter and then read it from the +`parameter_statuses`: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection( @@ -293,13 +306,14 @@ we set the ``aplication_name`` parameter and then read it from the ``parameter_s 'AGI' >>> >>> con.close() +``` -LIMIT ALL -````````` +### LIMIT ALL You might think that the following would work, but in fact it fails: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -309,10 +323,12 @@ Traceback (most recent call last): pg8000.exceptions.DatabaseError: ... >>> >>> con.close() +``` -Instead the `docs say `_ that -you can send ``null`` as an alternative to ``ALL``, which does work: +Instead the [docs say](https://www.postgresql.org/docs/current/sql-select.html) that you +can send `null` as an alternative to `ALL`, which does work: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -321,13 +337,14 @@ you can send ``null`` as an alternative to ``ALL``, which does work: [['silo 1']] >>> >>> con.close() +``` -IN and NOT IN -````````````` +### IN and NOT IN You might think that the following would work, but in fact the server doesn't like it: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -337,10 +354,12 @@ Traceback (most recent call last): pg8000.exceptions.DatabaseError: ... >>> >>> con.close() +``` -instead you can write it using the `unnest -`_ function: +instead you can write it using the [unnest] +(https://www.postgresql.org/docs/current/functions-array.html) function: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -350,17 +369,18 @@ instead you can write it using the `unnest ... v=['a', 'b']) [['silo 1']] >>> con.close() +``` -and you can do the same for ``NOT IN``. +and you can do the same for `NOT IN`. -Many SQL Statements Can't Be Parameterized -`````````````````````````````````````````` +### Many SQL Statements Can't Be Parameterized -In PostgreSQL parameters can only be used for `data values, not identifiers -`_. +In PostgreSQL parameters can only be used for [data values, not identifiers] +(https://www.postgresql.org/docs/current/xfunc-sql.html#XFUNC-SQL-FUNCTION-ARGUMENTS). Sometimes this might not work as expected, for example the following fails: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -372,13 +392,15 @@ Traceback (most recent call last): pg8000.exceptions.DatabaseError: ... >>> >>> con.close() +``` It fails because the PostgreSQL server doesn't allow this statement to have any parameters. There are many SQL statements that one might think would have parameters, but don't. For these cases the SQL has to be created manually, being careful to use the -``identifier()`` and ``literal()`` functions to escape the values to avoid `SQL -injection attacks `_: +`identifier()` and `literal()` functions to escape the values to avoid [SQL injection +attacks](https://en.wikipedia.org/wiki/SQL_injection>): +```python >>> from pg8000.native import Connection, identifier, literal >>> >>> con = Connection("postgres", password="cpsnow") @@ -392,15 +414,16 @@ injection attacks `_: (..., 'top_secret', 'Aliens Landed!') >>> >>> con.close() +``` -COPY FROM And TO A Stream -````````````````````````` +### COPY FROM And TO A Stream -The SQL `COPY `_ statement can be -used to copy from and to a file or file-like object. Here' an example using the CSV +The SQL [COPY](https://www.postgresql.org/docs/current/sql-copy.html) statement can be +used to copy from and to a file or file-like object. Here's an example using the CSV format: +```python >>> import pg8000.native >>> from io import StringIO >>> import csv @@ -438,10 +461,12 @@ format: ['3', 'tau'] >>> >>> con.close() +``` It's also possible to COPY FROM an iterable, which is useful if you're creating rows programmatically: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -469,14 +494,15 @@ programmatically: ['3', 'tau'] >>> >>> con.close() +``` -Execute Multiple SQL Statements -``````````````````````````````` +### Execute Multiple SQL Statements -If you want to execute a series of SQL statements (eg. an ``.sql`` file), you can run +If you want to execute a series of SQL statements (eg. an `.sql` file), you can run them as expected: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -487,19 +513,20 @@ them as expected: [[5], ['Erich Fromm']] >>> >>> con.close() +``` The only caveat is that when executing multiple statements you can't have any parameters. -Quoted Identifiers in SQL -````````````````````````` +### Quoted Identifiers in SQL -Say you had a column called ``My Column``. Since it's case sensitive and contains a -space, you'd have to `surround it by double quotes -`_. +Say you had a column called `My Column`. Since it's case sensitive and contains a space, +you'd have to [surround it by double quotes] +(https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIER). But you can't do: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -509,12 +536,14 @@ Traceback (most recent call last): SyntaxError: invalid syntax... >>> >>> con.close() +``` since Python uses double quotes to delimit string literals, so one solution is -to use Python's `triple quotes -`_ to delimit the string +to use Python's [triple quotes] +(https://docs.python.org/3/tutorial/introduction.html#strings) to delimit the string instead: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -523,11 +552,13 @@ instead: [['hello']] >>> >>> con.close() +``` another solution, that's especially useful if the identifier comes from an untrusted -source, is to use the ``identifier()`` function, which correctly quotes and escapes the +source, is to use the `identifier()` function, which correctly quotes and escapes the identifier as needed: +```python >>> from pg8000.native import Connection, identifier >>> >>> con = Connection("postgres", password="cpsnow") @@ -540,12 +571,14 @@ SELECT 'hello' as "My Column" [['hello']] >>> >>> con.close() +``` -this approach guards against `SQL injection attacks -`_. One thing to note if you're using -explicit schemas (eg. ``pg_catalog.pg_language``) is that the schema name and table name +this approach guards against [SQL injection attacks] +(https://en.wikipedia.org/wiki/SQL_injection). One thing to note if you're using +explicit schemas (eg. `pg_catalog.pg_language`) is that the schema name and table name are both separate identifiers. So to escape them you'd do: +```python >>> from pg8000.native import Connection, identifier >>> >>> con = Connection("postgres", password="cpsnow") @@ -561,20 +594,21 @@ SELECT lanname FROM pg_catalog.pg_language WHERE lanname = 'sql' [['sql']] >>> >>> con.close() +``` -Custom adapter from a Python type to a PostgreSQL type -`````````````````````````````````````````````````````` +### Custom adapter from a Python type to a PostgreSQL type pg8000 has a mapping from Python types to PostgreSQL types for when it needs to send SQL parameters to the server. The default mapping that comes with pg8000 is designed to work well in most cases, but you might want to add or replace the default mapping. -A Python ``datetime.timedelta`` object is sent to the server as a PostgreSQL -``interval`` type, which has the ``oid`` 1186. But let's say we wanted to create our -own Python class to be sent as an ``interval`` type. Then we'd have to register an +A Python `datetime.timedelta` object is sent to the server as a PostgreSQL +`interval` type, which has the `oid` 1186. But let's say we wanted to create our +own Python class to be sent as an `interval` type. Then we'd have to register an adapter: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -590,24 +624,25 @@ adapter: [[datetime.timedelta(seconds=7200)]] >>> >>> con.close() +``` -Note that it still came back as a ``datetime.timedelta`` object because we only changed +Note that it still came back as a `datetime.timedelta` object because we only changed the mapping from Python to PostgreSQL. See below for an example of how to change the mapping from PostgreSQL to Python. -Custom adapter from a PostgreSQL type to a Python type -`````````````````````````````````````````````````````` +### Custom adapter from a PostgreSQL type to a Python type pg8000 has a mapping from PostgreSQL types to Python types for when it receives SQL results from the server. The default mapping that comes with pg8000 is designed to work well in most cases, but you might want to add or replace the default mapping. -If pg8000 receives PostgreSQL ``interval`` type, which has the ``oid`` 1186, it converts -it into a Python ``datetime.timedelta`` object. But let's say we wanted to create our -own Python class to be used instead of ``datetime.timedelta``. Then we'd have to +If pg8000 receives PostgreSQL `interval` type, which has the `oid` 1186, it converts +it into a Python `datetime.timedelta` object. But let's say we wanted to create our +own Python class to be used instead of `datetime.timedelta`. Then we'd have to register an adapter: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -623,18 +658,19 @@ register an adapter: [['2 years']] >>> >>> con.close() +``` Note that registering the 'in' adapter only afects the mapping from the PostgreSQL type to the Python type. See above for an example of how to change the mapping from PostgreSQL to Python. -Could Not Determine Data Type Of Parameter -`````````````````````````````````````````` +### Could Not Determine Data Type Of Parameter -Sometimes you'll get the 'could not determine data type of parameter' error message from +Sometimes you'll get the `could not determine data type of parameter` error message from the server: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -644,9 +680,11 @@ Traceback (most recent call last): pg8000.exceptions.DatabaseError: {'S': 'ERROR', 'V': 'ERROR', 'C': '42P18', 'M': 'could not determine data type of parameter $1', 'F': 'postgres.c', 'L': '...', 'R': '...'} >>> >>> con.close() +``` -One way of solving it is to put a ``CAST`` in the SQL: +One way of solving it is to put a `CAST` in the SQL: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -655,9 +693,11 @@ One way of solving it is to put a ``CAST`` in the SQL: [[True]] >>> >>> con.close() +``` Another way is to override the type that pg8000 sends along with each parameter: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -666,15 +706,16 @@ Another way is to override the type that pg8000 sends along with each parameter: [[True]] >>> >>> con.close() +``` -Prepared Statements -``````````````````` +### Prepared Statements -`Prepared statements `_ -can be useful in improving performance when you have a statement that's executed -repeatedly. Here's an example: +[Prepared statements](https://www.postgresql.org/docs/current/sql-prepare.html) can be +useful in improving performance when you have a statement that's executed repeatedly. +Here's an example: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection("postgres", password="cpsnow") @@ -694,13 +735,14 @@ repeatedly. Here's an example: >>> ps.close() >>> >>> con.close() +``` -Use Environment Variables As Connection Defaults -```````````````````````````````````````````````` +### Use Environment Variables As Connection Defaults You might want to use the current user as the database username for example: +```python >>> import pg8000.native >>> import getpass >>> @@ -712,10 +754,12 @@ You might want to use the current user as the database username for example: [['pilau']] >>> >>> connection.close() +``` -or perhaps you may want to use some of the same `environment variables that libpg uses -`_: +or perhaps you may want to use some of the same [environment variables that libpg uses] +(https://www.postgresql.org/docs/current/libpq-envars.html): +```python >>> import pg8000.native >>> from os import environ >>> @@ -732,67 +776,69 @@ or perhaps you may want to use some of the same `environment variables that libp [['Mr Cairo']] >>> >>> connection.close() +``` It might be asked, why doesn't pg8000 have this behaviour built in? The thinking -follows the second aphorism of `The Zen of Python -`_: +follows the second aphorism of [The Zen of Python] +(https://www.python.org/dev/peps/pep-0020/): - Explicit is better than implicit. +> Explicit is better than implicit. So we've taken the approach of only being able to set connection parameters using the -``pg8000.native.Connection()`` constructor. +`pg8000.native.Connection()` constructor. -Connect To PostgreSQL Over SSL -`````````````````````````````` +### Connect To PostgreSQL Over SSL To connect to the server using SSL defaults do:: - import pg8000.native - connection = pg8000.native.Connection('postgres', password="cpsnow", ssl_context=True) - connection.run("SELECT 'The game is afoot!'") +```python +import pg8000.native +connection = pg8000.native.Connection('postgres', password="cpsnow", ssl_context=True) +connection.run("SELECT 'The game is afoot!'") +``` -To connect over SSL with custom settings, set the ``ssl_context`` parameter to an -|ssl.SSLContext|_ object: +To connect over SSL with custom settings, set the `ssl_context` parameter to an +`ssl.SSLContext` object: -:: - - import pg8000.native - import ssl +```python +import pg8000.native +import ssl - ssl_context = ssl.create_default_context() - ssl_context.verify_mode = ssl.CERT_REQUIRED - ssl_context.load_verify_locations('root.pem') - connection = pg8000.native.Connection( - 'postgres', password="cpsnow", ssl_context=ssl_context) +ssl_context = ssl.create_default_context() +ssl_context.verify_mode = ssl.CERT_REQUIRED +ssl_context.load_verify_locations('root.pem') +connection = pg8000.native.Connection( + 'postgres', password="cpsnow", ssl_context=ssl_context) +``` It may be that your PostgreSQL server is behind an SSL proxy server in which case you -can set a pg8000-specific attribute ``ssl.SSLContext.request_ssl = False`` which tells +can set a pg8000-specific attribute `ssl.SSLContext.request_ssl = False` which tells pg8000 to connect using an SSL socket, but not to request SSL from the PostgreSQL server: -:: - - import pg8000.native - import ssl +```python +import pg8000.native +import ssl - ssl_context = ssl.create_default_context() - ssl_context.request_ssl = False - connection = pg8000.native.Connection( - 'postgres', password="cpsnow", ssl_context=ssl_context) +ssl_context = ssl.create_default_context() +ssl_context.request_ssl = False +connection = pg8000.native.Connection( + 'postgres', password="cpsnow", ssl_context=ssl_context) +``` -Server-Side Cursors -``````````````````` +### Server-Side Cursors -You can use the SQL commands `DECLARE -`_, -`FETCH `_, -`MOVE `_ and -`CLOSE `_ to manipulate +You can use the SQL commands [DECLARE] +(https://www.postgresql.org/docs/current/sql-declare.html), +[FETCH](https://www.postgresql.org/docs/current/sql-fetch.html), +[MOVE](https://www.postgresql.org/docs/current/sql-move.html) and +[CLOSE](https://www.postgresql.org/docs/current/sql-close.html) to manipulate server-side cursors. For example: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection('postgres', password="cpsnow") @@ -807,15 +853,16 @@ server-side cursors. For example: >>> con.run("ROLLBACK") >>> >>> con.close() +``` -BLOBs (Binary Large Objects) -```````````````````````````` +### BLOBs (Binary Large Objects) -There's a set of `SQL functions -`_ for manipulating BLOBs. +There's a set of [SQL functions] +(https://www.postgresql.org/docs/current/lo-funcs.html) for manipulating BLOBs. Here's an example: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection('postgres', password="cpsnow") @@ -848,15 +895,16 @@ Here's an example: [[b'all']] >>> >>> con.close() +``` -Replication Protocol -```````````````````` +### Replication Protocol -The PostgreSQL `Replication Protocol -`_ is supported using -the ``replication`` keyword when creating a connection: +The PostgreSQL [Replication Protocol] +(https://www.postgresql.org/docs/current/protocol-replication.html) is supported using +the `replication` keyword when creating a connection: +```python >>> import pg8000.native >>> >>> con = pg8000.native.Connection( @@ -866,20 +914,20 @@ the ``replication`` keyword when creating a connection: [['...', 1, '.../...', 'postgres']] >>> >>> con.close() +``` -DB-API 2 Interactive Examples ------------------------------ +## DB-API 2 Interactive Examples These examples stick to the DB-API 2.0 standard. -Basic Example -````````````` +### Basic Example Import pg8000, connect to the database, create a table, add some rows and then query the table: +```python >>> import pg8000.dbapi >>> >>> conn = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -897,13 +945,14 @@ id = 2, title = Speaker for the Dead >>> conn.commit() >>> >>> conn.close() +``` -Query Using Functions -````````````````````` +### Query Using Functions Another query, using some PostgreSQL functions: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -914,13 +963,14 @@ Another query, using some PostgreSQL functions: ['2021 AD'] >>> >>> con.close() +``` -Interval Type -````````````` +### Interval Type A query that returns the PostgreSQL interval type: +```python >>> import datetime >>> import pg8000.dbapi >>> @@ -933,14 +983,15 @@ A query that returns the PostgreSQL interval type: [datetime.timedelta(days=12271, seconds=57960)] >>> >>> con.close() +``` -Point Type -`````````` +### Point Type -A round-trip with a `PostgreSQL point -`_ type: +A round-trip with a [PostgreSQL point] +(https://www.postgresql.org/docs/current/datatype-geometric.html) type: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -951,14 +1002,15 @@ A round-trip with a `PostgreSQL point [(2.3, 1.0)] >>> >>> con.close() +``` -Numeric Parameter Style -``````````````````````` +### Numeric Parameter Style pg8000 supports all the DB-API parameter styles. Here's an example of using the 'numeric' parameter style: +```python >>> import pg8000.dbapi >>> >>> pg8000.dbapi.paramstyle = "numeric" @@ -971,14 +1023,15 @@ pg8000 supports all the DB-API parameter styles. Here's an example of using the >>> pg8000.dbapi.paramstyle = "format" >>> >>> con.close() +``` -Autocommit -`````````` +### Autocommit Following the DB-API specification, autocommit is off by default. It can be turned on by using the autocommit property of the connection: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -990,17 +1043,18 @@ using the autocommit property of the connection: >>> cur.close() >>> >>> con.close() +``` -Client Encoding -``````````````` +### Client Encoding When communicating with the server, pg8000 uses the character set that the server asks it to use (the client encoding). By default the client encoding is the database's character set (chosen when the database is created), but the client encoding can be -changed in a number of ways (eg. setting ``CLIENT_ENCODING`` in ``postgresql.conf``). +changed in a number of ways (eg. setting `CLIENT_ENCODING` in `postgresql.conf`). Another way of changing the client encoding is by using an SQL command. For example: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -1012,13 +1066,14 @@ Another way of changing the client encoding is by using an SQL command. For exam >>> cur.close() >>> >>> con.close() +``` -JSON -```` +### JSON JSON is sent to the server serialized, and returned de-serialized. Here's an example: +```python >>> import json >>> import pg8000.dbapi >>> @@ -1031,9 +1086,11 @@ JSON is sent to the server serialized, and returned de-serialized. Here's an exa >>> cur.close() >>> >>> con.close() +``` JSON queries can be have parameters: +```python >>> import pg8000.dbapi >>> >>> with pg8000.dbapi.connect("postgres", password="cpsnow") as con: @@ -1042,13 +1099,14 @@ JSON queries can be have parameters: ... for row in cur.fetchall(): ... print(row) [True] +``` -Retrieve Column Names From Results -`````````````````````````````````` +### Retrieve Column Names From Results Use the columns names retrieved from a query: +```python >>> import pg8000 >>> conn = pg8000.dbapi.connect(user="postgres", password="cpsnow") >>> c = conn.cursor() @@ -1064,14 +1122,15 @@ Use the columns names retrieved from a query: >>> assert results == [{'id': 1, 'name': 'Up'}, {'id': 2, 'name': 'Down'}] >>> >>> conn.close() +``` -COPY from and to a file -``````````````````````` +### COPY from and to a file -The SQL `COPY `__ statement can +The SQL [COPY](https://www.postgresql.org/docs/current/sql-copy.html) statement can be used to copy from and to a file or file-like object: +```python >>> from io import StringIO >>> import pg8000.dbapi >>> @@ -1093,18 +1152,19 @@ be used to copy from and to a file or file-like object: '1\telectron\n2\tmuon\n3\ttau\n' >>> >>> con.close() +``` -Server-Side Cursors -``````````````````` +### Server-Side Cursors -You can use the SQL commands `DECLARE -`_, -`FETCH `_, -`MOVE `_ and -`CLOSE `_ to manipulate +You can use the SQL commands [DECLARE] +(https://www.postgresql.org/docs/current/sql-declare.html), +[FETCH](https://www.postgresql.org/docs/current/sql-fetch.html), +[MOVE](https://www.postgresql.org/docs/current/sql-move.html) and +[CLOSE](https://www.postgresql.org/docs/current/sql-close.html) to manipulate server-side cursors. For example: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -1123,15 +1183,16 @@ server-side cursors. For example: >>> cur.execute("ROLLBACK") >>> >>> con.close() +``` -BLOBs (Binary Large Objects) -```````````````````````````` +### BLOBs (Binary Large Objects) -There's a set of `SQL functions -`_ for manipulating BLOBs. +There's a set of [SQL functions] +(https://www.postgresql.org/docs/current/lo-funcs.html) for manipulating BLOBs. Here's an example: +```python >>> import pg8000.dbapi >>> >>> con = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -1166,14 +1227,15 @@ Here's an example: ([b'all'],) >>> >>> con.close() +``` -Parameter Limit -``````````````` +### Parameter Limit The protocol that PostgreSQL uses limits the number of parameters to 6,5535. The following will give an error: +```python >>> import pg8000.dbapi >>> >>> conn = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -1185,10 +1247,12 @@ an error: ... ) Traceback (most recent call last): struct.error: 'H' format requires 0 <= number <= 65535 +``` -One way of working round this problem is to use the `unnest -`_ function: +One way of working round this problem is to use the [unnest] +(https://www.postgresql.org/docs/current/functions-array.html) function: +```python >>> import pg8000.dbapi >>> >>> conn = pg8000.dbapi.connect(user="postgres", password="cpsnow") @@ -1199,10 +1263,10 @@ One way of working round this problem is to use the `unnest ... [[1] * SIZE], ... ) >>> conn.close() +``` -Type Mapping ------------- +## Type Mapping The following table shows the default mapping between Python types and PostgreSQL types, and vice versa. @@ -1211,82 +1275,34 @@ If pg8000 doesn't recognize a type that it receives from PostgreSQL, it will ret as a ``str`` type. This is how pg8000 handles PostgreSQL ``enum`` and XML types. It's possible to change the default mapping using adapters (see the examples). -.. table:: Python to PostgreSQL Type Mapping - - +-----------------------+-----------------+-----------------------------------------+ - | Python Type | PostgreSQL Type | Notes | - +=======================+=================+=========================================+ - | bool | bool | | - +-----------------------+-----------------+-----------------------------------------+ - | int | int4 | | - +-----------------------+-----------------+-----------------------------------------+ - | str | text | | - +-----------------------+-----------------+-----------------------------------------+ - | float | float8 | | - +-----------------------+-----------------+-----------------------------------------+ - | decimal.Decimal | numeric | | - +-----------------------+-----------------+-----------------------------------------+ - | bytes | bytea | | - +-----------------------+-----------------+-----------------------------------------+ - | datetime.datetime | timestamp | +/-infinity PostgreSQL values are | - | (without tzinfo) | without | represented as Python ``str`` values. | - | | timezone | If a ``timestamp`` is too big for | - | | | ``datetime.datetime`` then a ``str`` is | - | | | used. | - +-----------------------+-----------------+-----------------------------------------+ - | datetime.datetime | timestamp with | +/-infinity PostgreSQL values are | - | (with tzinfo) | timezone | represented as Python ``str`` values. | - | | | If a ``timestamptz`` is too big for | - | | | ``datetime.datetime`` then a ``str`` is | - | | | used. | - +-----------------------+-----------------+-----------------------------------------+ - | datetime.date | date | +/-infinity PostgreSQL values are | - | | | represented as Python ``str`` values. | - | | | If a ``date`` is too big for a | - | | | ``datetime.date`` then a ``str`` is | - | | | used. | - +-----------------------+-----------------+-----------------------------------------+ - | datetime.time | time without | | - | | time zone | | - +-----------------------+-----------------+-----------------------------------------+ - | datetime.timedelta | interval | If an ``interval`` is too big for | - | | | ``datetime.timedelta`` then a | - | | | ``PGInterval`` is used. | - +-----------------------+-----------------+-----------------------------------------+ - | None | NULL | | - +-----------------------+-----------------+-----------------------------------------+ - | uuid.UUID | uuid | | - +-----------------------+-----------------+-----------------------------------------+ - | ipaddress.IPv4Address | inet | | - +-----------------------+-----------------+-----------------------------------------+ - | ipaddress.IPv6Address | inet | | - +-----------------------+-----------------+-----------------------------------------+ - | ipaddress.IPv4Network | inet | | - +-----------------------+-----------------+-----------------------------------------+ - | ipaddress.IPv6Network | inet | | - +-----------------------+-----------------+-----------------------------------------+ - | int | xid | | - +-----------------------+-----------------+-----------------------------------------+ - | list of int | INT4[] | | - +-----------------------+-----------------+-----------------------------------------+ - | list of float | FLOAT8[] | | - +-----------------------+-----------------+-----------------------------------------+ - | list of bool | BOOL[] | | - +-----------------------+-----------------+-----------------------------------------+ - | list of str | TEXT[] | | - +-----------------------+-----------------+-----------------------------------------+ - | int | int2vector | Only from PostgreSQL to Python | - +-----------------------+-----------------+-----------------------------------------+ - | JSON | json, jsonb | The Python JSON is provided as a Python | - | | | serialized string. Results returned as | - | | | de-serialized JSON. | - +-----------------------+-----------------+-----------------------------------------+ - | pg8000.Range | \*range | PostgreSQL multirange types are | - | | | represented in Python as a list of | - | | | range types. | - +-----------------------+-----------------+-----------------------------------------+ - | tuple | composite type | Only from Python to PostgreSQL | - +-----------------------+-----------------+-----------------------------------------+ +| Python Type | PostgreSQL Type | Notes | +|-----------------------|-----------------|-----------------------------------------| +| bool | bool | | +| int | int4 | | +| str | text | | +| float | float8 | | +| decimal.Decimal | numeric | | +| bytes | bytea | | +| datetime.datetime (without tzinfo) | timestamp without timezone | +/-infinity PostgreSQL values are represented as Python `str` values. If a `timestamp` is too big for `datetime.datetime` then a `str` is used. | +| datetime.datetime (with tzinfo) | timestamp with timezone | +/-infinity PostgreSQL values are represented as Python `str` values. If a `timestamptz` is too big for `datetime.datetime` then a `str` is used. | +| datetime.date | date | +/-infinity PostgreSQL values are represented as Python `str` values. If a `date` is too big for a `datetime.date` then a `str` is used. | +| datetime.time | time without time zone | | +| datetime.timedelta | interval | If an ``interval`` is too big for `datetime.timedelta` then a `PGInterval` is used. | +| None | NULL | | +| uuid.UUID | uuid | | +| ipaddress.IPv4Address | inet | | +| ipaddress.IPv6Address | inet | | +| ipaddress.IPv4Network | inet | | +| ipaddress.IPv6Network | inet | | +| int | xid | | +| list of int | INT4[] | | +| list of float | FLOAT8[] | | +| list of bool | BOOL[] | | +| list of str | TEXT[] | | +| int | int2vector | Only from PostgreSQL to Python | +| JSON | json, jsonb | The Python JSON is provided as a Python serialized string. Results returned as de-serialized JSON. | +| pg8000.Range | range | PostgreSQL multirange types are | represented in Python as a list of range types. | +| tuple | composite type | Only from Python to PostgreSQL |