parsed.org

Tips by tag: postgresql

Adding a Database and a User by xinu on Dec 18, 2005 01:43 PM

To create a Postgres user sam and a database samdb that is owned by the sam user:

$ su - postgres
$ createuser -P -A -D sam
$ createdb -O sam samdb

(Change the -A, and -D options to affect the new user's abilities.)

Edit the user's password with:

template1=# alter user username_here with password 'password_here';
accountcommandscreatedbcreateuserpermissionspostgresqluser
Booleans In The WHERE Clause by xinu on May 03, 2005 05:21 PM

If you're going to use a boolean condition in your where clause, you're going to want to make sure you use an operator so that the index is used (if present).

For example, this will not use the index:

db=> SELECT id FROM users WHERE active;

This will use the index:

db=> SELECT id FROM users WHERE active = true;
expressionsgotchaindexperformancepostgresqlsql

Create a custom type which describes the structure of the rows returned:

CREATE TYPE func_return_type AS (column_name column_type, ...);

Then create the function:

CREATE OR REPLACE FUNCTION some_func (param_type, ...)
RETURNS SETOF func_return_type AS '
 SELECT columns FROM table WHERE condition;
' LANGUAGE 'SQL';
customfunctionsplpgsqlpostgresqlsqltypes

Use the form ::type to ensure that a value is cast to the proper type:

SELECT * FROM table WHERE column = 5::bigint;

In this case, column is indexed and is a bigint. The index will not be used if the query planner doesn't encounter a value of the index type. Note: this only applies in PostgreSQL 7.x.

gotchaindexperformancepostgresqlsqlsyntaxtypes
Convert Text to Bytea by cygnus on May 02, 2007 04:38 PM

If you need to convert a TEXT column to a BYTEA column, you can use a cast expression to do the work:

ALTER TABLE mytable ALTER COLUMN col TYPE bytea USING
  decode(replace(col, '\\', '\\\\'), 'escape');

Note that altering the column type of a column is only supported beginning with PostgreSQL version 8.0. For more information, see the manual.

byteacastconversiondatabasemodificationpostgresqlschemasqltext
Creating a Table from a Query by cygnus on Feb 10, 2005 02:12 PM

If you want to create a table that has the same structure as the result set of a query, run:

mydb> CREATE TABLE foobar AS SELECT ...;
neatpostgresqlsql
Display NULL by xinu on Jan 20, 2005 08:10 PM

If instead of a blank space you'd prefer something that tells you a particular column in a row is NULL, you can set it:

\pset null '(null)'
commandsconfigurationpostgresqlpsql
Ditch All Output Of Query by xinu on Jan 12, 2005 10:25 AM

If you want to run a query or a function and ditch the output, you can do something like this:

xinu=# select my_void_function() \g /dev/null
commandsconfigurationpostgresqlpsql
Echo Hidden by xinu on Jan 15, 2005 02:47 PM

If you want to see how a query like \du is being built:

\set ECHO_HIDDEN
analysiscommandsconfigurationpostgresqlpsqlsql
Eliminating Aggregates by cygnus on Mar 31, 2005 08:49 AM

If you have a query like this:

SELECT max(some_column) FROM mytable WHERE ...;

and you want to eliminate the sequence scan that kills the query's performance, an alternative is:

SELECT some_column FROM mytable WHERE ... ORDER BY some_column DESC LIMIT 1;

MIN and MAX aggregates can be eliminated this way.

aggregatesmaxminperformancepostgresqlqueriesscansql
Fixing Munged Template DB by xinu on Jan 12, 2005 10:24 AM

If you've munged your template1 database, you can rebuild it from the template0 database.

First, you must set the datistemplate attribute of template1 to false so we can drop it:

xinu=# UPDATE pg_database SET datistemplate = false WHERE datname = 'template1';
UPDATE 1
xinu=# DROP DATABASE template1;
DROP DATABASE

Next, re-create template1 using template0 as a template:

xinu=# CREATE DATABASE template1 WITH TEMPLATE = template0;
CREATE DATABASE

Then restore the datistemplate attribute of template1:

xinu=# UPDATE pg_database SET datistemplate = true WHERE datname = 'template1';
UPDATE 1
debugginggotchapostgresqlpsqlrecoveryrestoresqltemplate
Incorrect Checksum by xinu on Jan 12, 2005 10:22 AM

If you see the following when trying to load a newly transplanted database, it may be due to a difference in architecture (32-bit vs. 64-bit):

FATAL:  incorrect checksum in control file.

The moral is: never do a direct copy of the data directory. It isn't considered good practice.

64bitarchitecturedebugginggotchapostgresqlrecoveryrestore
Inserting All Default Values by cygnus on Jul 28, 2005 12:08 PM

If you have a table 'mytable' whose columns all have default values defined and you want to insert a record using all default values, run this query:

INSERT INTO mytable DEFAULT VALUES;
defaultsinsertpostgresqlsql
Inserting Binaries with Psycopg by xinu on Jan 12, 2005 01:41 PM

Use the psycopg.Binary() function to escape the binary data:

>>> db = psycopg.connect("dbname=%s user=%s" % (database, user))
>>> db.autocommit(True)
>>> cursor = db.cursor()
>>> file = "/path/to/binary_file.jpg"
>>> fd = open(file, "r")
>>> contents = fd.read(os.stat(file)[6])
>>> fd.close()
>>> cursor.execute("INSERT INTO pr0n (image) VALUES (%s)", (psycopg.Binary(contents)))
binarybyteacursordbapiescapeinteractivelanguagespostgresqlprogrammingpsycopgpython
MySQL-style output border by cygnus on Jan 12, 2005 10:25 AM

In your ~/.psqlrc, add this to emulate MySQL-style borders in query results:

\pset border 2
commandsconfigurationmysqloutputpostgresqlpsqlpsqlrc
Order By Array Subscript by cygnus on Jan 12, 2005 10:25 AM

It is possible to order by an array element as follows:

SELECT * FROM (SELECT ARRAY[1, 2, 3] AS foo) sub ORDER BY foo[1];
arrayobscurepostgresqlsqlsyntax
PostgreSQL Authentication by xinu on Apr 29, 2008 08:28 PM

If you intend to use passwords for local database authentication, you'll need to make an adjustment to the pg_hba.conf file:

# TYPE  DATABASE    USER        IP-ADDRESS        IP-MASK        METHOD

#local   all         all                                          ident sameuser
local   all         all                                          password
authenticationidentlocalpasswordpostgresqlpsql
PostgreSQL Client by xinu on Jan 19, 2005 07:35 AM

If you just need the client and not the binary, you can modify the behavior of the port installation like so:

# cd /usr/ports/postgresql7
# make install clean WITHOUT_SERVER=yes

To see the other tasty build options:

# make build
bsdcommandsfreebsdmakepostgresql
PostgreSQL Dump & Restore by xinu on Mar 25, 2008 11:46 PM

In 7.4 (and possibly 8.1), the routine to dump and restore the entire database cluster follows:

# su - postgres
$ pg_dumpall > postgresql.dmp

Transfer the postgresql.dmp file to the target system, and then do the import:

# su - postgres
$ psql template1 < postgresql.dmp
dumpmigratepg_dumpallpostgresqlpsqlrestore
PostgreSQL Tables by xinu on Aug 16, 2005 11:46 AM

You can use this query to find the table information:

SELECT
  a.attnum,
  a.attname AS field,
  t.typname AS type,
  a.attlen AS length,
  a.atttypmod AS lengthvar,
  a.attnotnull AS notnull
FROM
  pg_class c,
  pg_attribute a,
  pg_type t
WHERE
  c.relname = 'your_table_name'
  AND a.attnum > 0
  AND a.attrelid = c.oid
  AND a.atttypid = t.oid
  ORDER BY a.attnum
debugginginternalsmetadatapostgresqlqueriessql
Psycopg1 Compatibility Mode by cygnus on Dec 31, 2005 04:07 PM

If you're using the psycopg Python module to connect to Postgres, you may find that you have old scripts that use version 1 of psycopg but you have version 2 installed and don't want to modify your scripts. At the time of this writing (and according to http://initd.org/tracker/psycopg/wiki/Migration), the version 2 module provides a very easy "compatibility mode". Just import the psycopg1 submodule and alias it, or fall back to the version 1 module if version 2 is not available:

try:
    # Try importing the compatibility submodule, which will only
    # work if psycopg version 2 is available.
    import psycopg.psycopg1 as psycopg
except Exception, e:
    # Fall back to version 1.
    import psycopg
aliasdbapiimportlanguagespostgresqlprogrammingpsycopgpsycopg2python
Query performance analysis by cygnus on Jan 12, 2005 10:14 AM

Use explain:

EXPLAIN SELECT * FROM some_table WHERE condition;
analysisexplainperformancepostgresqlsql
Query Timing by xinu on Jan 19, 2005 09:43 AM

If you want to see how long a query is taking to run, you can either run explain analyze or enable timing for the client by typing \timing.

analysisconfigurationdebuggingdurationoutputperformancepostgresqlpsqlruntimesqltiming
Random Rows from SQL by http://www.tylermac.net/ on Mar 10, 2007 02:36 PM

It is sometimes useful when working with SQL to grab random rowsets, for debugging stored procedures or simply grabbing a random ad for an ad rotator. Either way, different databases use different functions to select this randomization.

  • Select a random row with MySQL:

    SELECT * FROM mytable
    ORDER BY RAND()
    LIMIT 1;
    
  • Select a random row with PostgreSQL:

    SELECT * FROM mytable
    ORDER BY RANDOM()
    LIMIT 1;
    
  • Select a random row with Microsoft SQL Server:

    SELECT TOP 1 column FROM table
    ORDER BY NEWID();
    
generatormssqlmysqlpostgresqlrandomsql
Returning Multiple Records by cygnus on Mar 22, 2005 09:10 AM

You can use RETURNS SETOF record in your plpgsql functions to return a set of rows. Here is an example of using pgsql to build a query using a table name parameter:

CREATE OR REPLACE FUNCTION test(text) RETURNS SETOF record AS '
DECLARE
  _table ALIAS FOR $1;
  _mycursor refcursor;
  _row record;
BEGIN
  OPEN _mycursor FOR EXECUTE ''SELECT * FROM '' || _table;
  FETCH _mycursor INTO _row;
  WHILE FOUND LOOP
    RETURN NEXT _row;
    FETCH _mycursor INTO _row;
  END LOOP;
  RETURN;
END
' LANGUAGE plpgsql;

Here is an example of calling such a function (the names and types of the result columns are required when returning SETOF record):

mydb=> SELECT * FROM test('mytable') AS (col1 integer, col2 text, col3 date);
functionsobscureplpgsqlpostgresqlpsqlsetofsql
Select a Random Row by xinu on Jan 12, 2005 10:22 AM

Select a random row from a table with the following query:

SELECT * FROM table ORDER BY random() LIMIT 1;
obscurepostgresqlrandomsql
Setting the Prompt in 'psql' by cygnus on Jan 14, 2005 12:34 PM

Set the PROMPT1 variable using psql prompt escape sequences:

\set PROMPT1 '[%n@%M:%/]=%# '
  • %n - User
  • %M - Host ([local] or the domain or IP address of the server)
  • %/ - Database
  • %# - # if superuser, $ if regular user.
commandsconfigurationescapepostgresqlpsqlvariables
Show Tables Without Primary Keys by cygnus on Jul 21, 2005 01:13 PM

Use the following query to show all tables without primary keys:

SELECT
  nspname AS schema,
  relname AS table
FROM pg_class
  LEFT JOIN pg_constraint ON
    pg_constraint.contype = 'p' AND
    conrelid = pg_class.oid
  JOIN pg_namespace ON
    pg_namespace.oid = pg_class.relnamespace
WHERE
  relkind = 'r' AND contype IS NULL;
internalskeysmetadatapostgresqlprimarysql
Stop On Errors by xinu on Jan 12, 2005 11:07 AM

Set this variable in your ~/.psqlrc to stop on error when psql is used to run non-interactive scripts (e.g. cat file | psql ...):

\set ON_ERROR_STOP 1

Or use it from the command line:

$ psql ... -v ON_ERROR_STOP=1 ...
commandsconfigurationdebugginginteractivepipepostgresqlpsqlpsqlrc
Update a Domain Constraint by xinu on Jan 12, 2005 10:23 AM

Use these queries to change the structure of a DOMAIN constraint:

ALTER DOMAIN foo DROP CONSTRAINT bar;
ALTER DOMAIN foo ADD CONSTRAINT bar CHECK (VALUE = 'baz'::text);
constraintsdomainpostgresqlqueriesschemasql

Given a table MYTABLE in namespace MYNAMESPACE, you can use this SQL to retrieve the primary key column name(s) for the table:

SELECT
  attname::text
FROM
  pg_attribute
JOIN
  pg_class ON pg_attribute.attrelid = pg_class.oid
JOIN
  pg_namespace ON pg_namespace.oid = pg_class.relnamespace
LEFT JOIN
  pg_constraint ON conrelid = pg_class.oid AND pg_constraint.contype = 'p'
WHERE
  pg_namespace.nspname = 'MYNAMESPACE' AND
  pg_class.relname = 'MYTABLE' AND
  pg_attribute.attnum = ANY (pg_constraint.conkey)
ORDER BY
  pg_attribute.attnum;
columnsinternalskeymetadatapostgresqlprimarysql
Using Your Own Template Database by cygnus on Jun 29, 2006 07:36 PM

If you have a database mytemplate that you'd like to use as a template, update its datistemplate attribute in the pg_database relation:

UPDATE pg_database SET datistemplate = 't' WHERE datname = 'mytemplate';

Then you can use it as a template when creating other databases:

CREATE DATABASE otherdb TEMPLATE mytemplate;
debugginggotchainternalspostgresqlrecoveryrestoresqltemplate
RSS