Sunday, April 26. 2009
Printer Friendly
Different Linux distros have their preferred place of where stuff goes and of course the default location on windows is completely different from that too. So there isn't really one default location where you can find PostgreSQL data cluster. Of course user's can pick their
locations as well. So what is a casual DBA supposed to do?
The pg_settings table
PostgreSQL has a convenient system table view called pg_settings that stores a lot of information. It stores the location of the data cluster, the pg_hbafile and other conf files.
In additon to that you can interogate it to find out information you will find in the postgresql.conf file. Why sift thru that postgresql.conf file (assuming you can already query your postgresql server) when you can find the answers you are looking
for with an SQL query?
Continue reading "Where is my data and other stuff"
Saturday, March 28. 2009
Printer Friendly
Even though others have blogged about this in the past and its well-documented in the docs, its a frequently enough asked question, that we thought we'd post it here again
with a couple of additional twists.
How to determine the size of a database on disk
SELECT pg_size_pretty(pg_database_size('somedatabase')) As fulldbsize;
How to determine the size of a database table on disk
NOTE: There are two functions in PostgreSQL - pg_relation_size and pg_total_relation_size. The pg_relation_size just measures the size of the actual table where
as the pg_total_relation_size includes both the table and all its toasted tables and indexes.
SELECT pg_size_pretty(pg_total_relation_size('someschema.sometable')) As fulltblsize,
pg_size_pretty(pg_relation_size('someschema.sometable')) As justthetblsize;
How to determine the size of a database schema
When you are doing a lot of data loading for later massaging to dump into your production tables, you will often generate junk tables in the process and worse yet other
people may generate junk tables period. As a general practice, we like to create a schema just for junk. The reason for that is that it is extremely easy to
exclude schemas from being backed up and to load individual schemas. So with this approach we create or move all tables we are not quite sure if they are needed or we only need temporarily
into a schema
called junk, scratch, possiblejunk or whatever you want to call the schema and exclude it from our backups.
Something of the form:
CREATE SCHEMA scratch;
ALTER TABLE someschema.joeyplaying SET SCHEMA scratch;
After saving the world from junk, you would like to know how much space you have saved your backups from backing up. So you would employ a query something of the form:
SELECT pg_size_pretty(pg_database_size('mycurrentdb')) As fullprod,
pg_size_pretty(CAST(pg_database_size('mycurrentdb') - (SELECT SUM(pg_total_relation_size(table_schema || '.' || table_name) )
FROM information_schema.tables WHERE table_schema = 'scratch') As bigint)) As tobebackedup_size,
pg_size_pretty(CAST((SELECT SUM(pg_total_relation_size(table_schema || '.' || table_name) )
FROM information_schema.tables
WHERE table_schema = 'scratch') As bigint) ) As junk_size;
Size of Geometries in PostGIS tables
PostGIS has a companion function for measuring geometry size which is useful when you want to get a sense of how much space your geometries are taking up on disk
for a set of records.
SELECT ST_Mem_Size(ST_GeomFromText('LINESTRING(220268 150415,220227 150505,220227 150406)'));
SELECT pg_size_pretty(CAST(SUM(ST_Mem_Size(the_geom)) As bigint) ) as totgeomsum
FROM sometable WHERE state = 'MA';
Saturday, December 20. 2008
Printer Friendly
This question was asked on the PostgreSQL novice newsgroup recently and Tom Lane fielded the question. Its a common thing to want to know if you are using the new 8.3 Enum feature. So we felt it useful to restate it.
Question: Given an enum, is there a query you can run to list out all the elements allowed by the enum?
Answer: Yes.
Below is just a regurgitation of the news item http://archives.postgresql.org/pgsql-novice/2008-12/msg00043.php
CREATE TYPE myenum as enum ('red','green','blue');
SELECT enumlabel
FROM pg_enum
WHERE enumtypid = 'myenum'::regtype
ORDER BY oid;
enumlabel
-----------
red
green
blue
(3 rows)
http://www.postgresql.org/docs/8.3/static/catalog-pg-enum.html
Tuesday, December 16. 2008
Printer Friendly
In our Product Showcase section of this issue, we introduced Fusion Charts which is a flash-based
charting product that makes beautiful flash charts. It comes in both a free and a non-free more bells and whistles
version.
In this 3-part series article we shall demonstrate using this with a PostgreSQL database, building a simple dashboard
with ASP.NET and PHP. We shall demonstrate both C# and VB.NET both using the PostgreSQL NPGSQL driver.
For this first part we shall simply load the database, do a quick analysis of what we've got to report on and create some views to help
us with our PHP and ASP.NET apps that will follow in parts 2 and 3.
We will be testing this on 8.3, but since the database is an old one, it should work just fine on older versions of
PostgreSQL. We'll try to refrain from using new features of PostgreSQL.
Continue reading "Fusion Charts and PostgreSQL Part 1: Database Analysis of USDA DB"
Wednesday, November 26. 2008
Printer Friendly
We had the pleasure of doing a fresh install of PostgreSQL 8.3.5 on RedHat EL4 box and when using the Yum repository, we noticed a couple of changes from last time we did this.
This could have been an oversight in our documentation before.
Changes to Yum Install for 8.3.5?
In our April 2008 issue we had An Almost Idiot's Guide to PostgreSQL YUM
and that article still seems to be surprisingly popular.
In the first step we had:
yum install postgresql
and that as I recall installed the postgresql server in addition to some client libraries.
For 8.3.5 fresh install it seems they are separated and to get the postgresql server you need to do:
yum install postgresql
yum install postgresql-server
Continue reading "Yum addendum for 8.3.5 and PgAgent"
Monday, November 17. 2008
Printer Friendly
Sometimes when you are testing or setting up a server or just porting things to another server, you just want to install the same set of users as you had before without restoring any databases.
In PostgreSQL, the users (Login Roles) and group roles are stored at the server level and only the permissions to objects are stored at the database level.
Question: How do you restore just the users and roles without having to do a full pg_dumpall of your server?
Continue reading "Backing up Login Roles aka Users and Group Roles"
Tuesday, October 28. 2008
Printer Friendly
The age old question of why or why is my table index not being used
is probably the most common question that ever gets asked even by expert database users.
In this brief article we will cover the most common reasons and try to order by statistical significance.
Continue reading "Why is my index not being used"
Printer Friendly
In first part Guide to Writing PLPGSQL functions, we covered the plpgsql function anatomy and basic IF and FOR loops. In this second part of our PLPGSQL Quick Guide series, we shall delve more into control flow. As we mentioned in the previous part, the following control flow constructs exist for PLPGSQL.
- FOR somevariable IN (1 ...someendnumber) LOOP .. END LOOP;
- FOR somevariable IN REVERSE someendnumber .. 1 BY somestep LOOP .. END LOOP;
- FOR somevariable IN EXECUTE(somesqlquery) LOOP ..RETURN NEXT; .. END LOOP;
- LOOP ..logic statements EXIT .. EXIT WHEN .. CONTINUE WHEN .. END LOOP;
- WHILE ... LOOP ... END LOOP;
- EXCEPTION WHEN .... WHEN ..
- Introduced in 8.3 RETURN QUERY which can be in any LOOP like structure or stand alone. This is
covered in New Features of PostgreSQL Functions
In this section we shall demonstrate looping thru sets of records and writing a set returning function. In the next section after,
we shall delve a little into recursive functions, doing table updates, and raising notices.
Continue reading "Quick Guide to writing PLPGSQL Functions: Part 2 "
Saturday, October 11. 2008
Printer Friendly
In this series we'll go over writing PLPGSQL stored functions. We shall follow up in a later issue with a one page cheat sheet.
The Anatomy of a PLPGSQL FUNCTION
All PLPGSQL functions follow a structure that looks something like the below.
CREATE OR REPLACE FUNCTION fnsomefunc(numtimes integer, msg text)
RETURNS text AS
$$
DECLARE
strresult text;
BEGIN
strresult := '';
IF numtimes > 0 THEN
FOR i IN 1 .. numtimes LOOP
strresult := strresult || msg || E'\r\n';
END LOOP;
END IF;
RETURN strresult;
END;
$$
LANGUAGE 'plpgsql' IMMUTABLE
SECURITY DEFINER
COST 10;
--To call the function we do this and it returns ten hello there's with
carriage returns as a single text field.
SELECT fnsomefunc(10, 'Hello there');
Continue reading "Quick Guide to writing PLPGSQL Functions: Part 1"
Wednesday, September 24. 2008
Printer Friendly
We've been playing around with the snapshot builds of PgAdmin III 1.9 and would like to summarize some
of the new nice features added. PgAdmin III 1.9 has not been released yet, but has a couple of neat features brewing.
For those interested in experimenting with the snapshot builds and src tarballs, you can download them from http://www.pgadmin.org/snapshots/
Continue reading "PgAdmin III 1.9 First Glance"
Sunday, September 14. 2008
Printer Friendly
OpenJump is a Java Based, Cross-Platform open source GIS analysis and query tool. We've been using it a lot lately, and I would
say out of all the open source tools (and even compared to many commercial tools) for geospatial analysis, it is one of the best out there.
While it is fairly rich in functionality in terms of doing statistical analysis on ESRI shapefile as well as PostGIS and other formats and also has numerous geometry manipulation features and plugins in its tool belt,
we like the ad-hoc query ability the most. The ease and simplicity of that one tool makes it stand out from the pack. People not comfortable with SQL may not appreciate that feature as much as we do though.
In this excerpt we will quickly go thru the history of project and the ties between the PostGIS group and OpenJump group,
how to install, setup a connection to a PostGIS enabled PostgreSQL database and doing some ad-hoc queries.
Quick History Lesson
- OpenJump is descended from Java Unified Mapping Platform - JUMP which was incubated by Vivid Solutions.
- OpenJump and the whole JUMP family tree have Java Topology Suite (JTS) as a core foundation of their functionality.
- GEOS which is a core foundation of PostGIS functionality and numerous other projects, is a C++ port of JTS. New Enhancements often are created in JTS and ported to GEOS and a large body of GEOS work has been incubated
by Refractions Research, the PostGIS incubation company.
- For more gory details about how all these things are intertwined, check out Martin Davis' recount of the history of GEOS and JTS.
Continue reading "OpenJump for PostGIS Spatial Ad-Hoc Queries"
Sunday, September 07. 2008
Printer Friendly
One of the nice things about the PostgreSQL command-line restore tool is the ease with which you can restore
select objects from a backup. We tend to use schemas for logical groupings which are partitioned by context, time, geography etc.
Often times when we are testing things, we just want to restore one schema or set of tables from our backup because restoring a 100 gigabyte database
takes a lot of space, takes more time and is unnecessary for our needs. In order to be able to accomplish such a feat, you need to
create tar or compressed (PG custom format) backups. We usually maintain PG custom backups of each of our databases.
Continue reading "How to restore select tables, select objects, and schemas from Pg Backup"
Sunday, July 27. 2008
Printer Friendly
Backup and Restore is probably the most important thing to know how to do when you have a database with data you care about.
The utilities in PostgreSQL that accomplish these tasks are pg_restore, pg_dump, pg_dumpall, and for restore of plain text dumps - psql.
A lot of the switches used by pg_dump, pg_restore, pg_dumpall are common to all three and on rare cases, the switches used by each overlap but mean different things.
pg_dump and pg_restore are complementary. You use pg_dump to do hot backups of a database and pg_restore to restore it either to another database or to recover portions of a database.
Rather than trying to keep track of which switch works with which, we decided to combine all into a single cheat sheet with a column denoting which utility the switch is supported in.
Pretty much all the text is compiled from the --help switch of each.
Below is a Thumbnail view of the PostgreSQL 8.3 Dump Restore cheat sheet
that covers PostgreSQL 8.3 pg_dump, pg_dumpall, pg_restore utilities.
PDF Portrait version 8.5 x 11" of this cheatsheet is available at PostgreSQL 8.3 Dump Restore 8.5 x 11 and also available in
PDF A4 format and HTML.
Monday, July 14. 2008
Printer Friendly
Programming Design Patterns define recommended approaches of solving common application problems. Within design patterns is a subset of design patterns called Idioms.
Idioms you can think of as a strategy for expressing recurring constructs or if you will sub-problems and often take advantage of the special features of a language.
They tend to be specific to a programming language and can not be reused
in other languages they were not specifically designed for. To demonstrate the differences lets compare two design patterns we commonly use.
Continue reading "SQL Idiom Design Patterns"
Sunday, July 06. 2008
Printer Friendly
In our April Issue An Almost Idiot's Guide to PostgreSQL YUM
we covered using the new PostgreSQL Yum repository to install the PostgreSQL 8.3.1 release on Fedora, RedHat Enterprise, and CentOS. We also received numerous useful feedback from others on issues they
ran into and how they overcame them. The blog comments are definitely worth a read.
Now that 8.3.3 has come out, many of you should be considering upgrading if you haven't already since there are a couple of bug fixes as outlined in
http://www.postgresql.org/docs/8.3/static/release-8-3-2.html, http://www.postgresql.org/docs/8.3/static/release-8-3-3.html,
and for those running 8.3.0 you will need to reindex your tables after as noted in http://www.postgresql.org/docs/8.3/static/release-8-3-1.html.
If you are running version 8.3.1 and above then
this is a fairly painless upgrade that just requires you to backup your data as a precautionary measure, but doesn't require a dump reload.
Continue reading "YUM 2: Almost Idiot's Guide to upgrade from PostgreSQL 8.3.1 to 8.3.3"
|