Postgres OnLine Journal - new in postgresql

Using procedures for batch geocoding and other batch processing

nospam@example.com (Leo Hsu and Regina Obe) — Sun, 08 Jul 2018 01:40:00 -0400

One of the features we are looking forward to in upcoming PostgreSQL 11 is the introduction of procedures via the CREATE PROCEDURE ANSI-SQL construct. The major benefit that sets apart procedures from functions is that procedures are not wrapped in an outer transaction and can have COMMITs within them. This means it's not an all or nothing like it is with functions. Even if you stop a procedure in motion, whatever work has been done and committed is saved. In the case of functions, a stop or failure would roll-back all the work. It also means you can see work in progress of a stored procedure since the work will already have been committed. This is a huge benefit for batch processing. Batch processing covers a lot of use-cases of PostGIS users since a good chunk of PostGIS work involves doing some kind of batch processing of data you get from third-parties or machines.

Continue reading "Using procedures for batch geocoding and other batch processing"

CASE with set returning functions in PostgreSQL 10

nospam@example.com (Leo Hsu and Regina Obe) — Tue, 09 May 2017 03:18:00 -0400

One of the changes coming in PostgreSQL 10 is the ability for the CASE .. WHEN statement to return multiple rows if the expression contains a set returning function. To demonstrate the feature, we'll use the classic generate_series function:

Since this article was written, this feature has been removed, because it caused old logic that had sets from doing something different without warning. Refer to this discussion for details.

Continue reading "CASE with set returning functions in PostgreSQL 10"

PostgreSQL 9.6 phrase text searching how far apart can you go

nospam@example.com (Leo Hsu and Regina Obe) — Mon, 30 May 2016 18:48:00 -0400

We've been playing around with new phrase text feature of full-text PostgreSQL 9.6. In doing so, I was curious how big a number one can designate as max that words can be apart. I discovered thru trial and error, that the magic number is 16384 which is much bigger than I had suspected.

Continue reading "PostgreSQL 9.6 phrase text searching how far apart can you go"

PLV8 binaries for PostgreSQL 9.6 windows both 32-bit and 64-bit

nospam@example.com (Leo Hsu and Regina Obe) — Fri, 13 May 2016 18:56:00 -0400

To celebrate recent release of PostgreSQL 9.6beta1, we've started to experiment with our favorite extensions. For starters, PLV8 (aka PL/JavaScript) binaries listed below and upcoming PostGIS 2.3.0 and ogr_fdw detailed here

Continue reading "PLV8 binaries for PostgreSQL 9.6 windows both 32-bit and 64-bit"

PostGIS 2.2.0 windows bundle for PostgreSQL 9.5beta2 32 and 64 bit

nospam@example.com (Leo Hsu and Regina Obe) — Sun, 22 Nov 2015 01:47:00 -0500

We just pushed out installers for PostGIS 2.2.0 for PostgreSQL 9.5beta2 windows both 32-bit and 64-bit on Application Stackbuilder. These installers are also available as standalone listed on PostGIS windows page. This is the first PostGIS 2.2.0 release for the PostgreSQL 9.5 32-bit and a rerelease for PostgreSQL 9.5 x 64-bit (this time compiled against beta2 instead of beta1).

On quick testing the PostGIS 2.2 beta1 release and pgRouting 2.1.0 worked fine on 9.5beta2, however you may want to reinstall anyway just to be safe. You can just reinstall over your existing install, no need to uninstall first. Similarly just upgrading a PostgreSQL 9.5beta1 to 9.5beta2 seemed to not require pg_upgrade or dump/restore, so safe to just upgrade from 9.5beta1 to 9.5beta2. Other notes about this 9.5beta2 PostGIS 2.2.0 release:

The FDW API changed between PostgreSQL 9.5beta1 and PostgreSQL 9.5beta2, so the OGR_FDW, if you don't reinstall the bundle, will crash and burn in PostgreSQL 9.5beta2 (using PostGIS 2.2. beta1 executables). Similarly this newly compiled OGR_FDW will not work on PostgreSQL 9.5beta1 (so upgrade to 9.5beta2 first).
The PostgreSQL 9.5betas (that includes both beta1 and beta2), are compiled against the pointcloud 1.1 master branch. This was required because the released pointcloud 1.0.1, does not compile against PostgreSQL 9.5
The PostgreSQL 9.5beta2 PostGIS 2.2.0 release comes packaged with SFCGAL 1.2.2 (instead of 1.2.0 like the others versions) which fixes a crasher with ST_StraightSkeleton as noted in ticket - https://trac.osgeo.org/postgis/ticket/3324. Newer SFCGAL will be packaged with upcoming PostGIS 2.2.1, but if you are on an older edition and are using SFCGAL, you can always copy latest SFCGAL.dll binaries from the 2.2.1dev packages on PostGIS windows page http://postgis.net/windows_downloads/.

PostgreSQL 9.5 Grouping Sets with PostGIS spatial aggregates

nospam@example.com (Leo Hsu and Regina Obe) — Mon, 03 Aug 2015 03:01:26 -0400

One of the features coming in PostgreSQL 9.5 is the triumvirate GROUPING SETS, CUBE, and ROLLUP nicely covered in Bruce's recent slide deck. The neatest thing about PostgreSQL development is that when improvements happen, they don't just affect the core, but can be taken advantage of by extensions, without even lifting a finger. Such is the case with these features.

One of the things I was curious about with these new set of predicates is Would they work with any aggregate function?. I assumed they would, so decided to put it to the test, by using it with PostGIS ST_Union function (using PostGIS 2.2.0 development). This feature was not something the PostGIS Development group planned on supporting, but by the magic of PostgreSQL, PostGIS accidentally supports it. The grouping sets feature is particularly useful if you want to aggregate data multiple times, perhaps for display using the same dataset. It allows you to do it with a single query that in other PostgreSQL versions would require a UNION query. This is a rather boring example but hopefully you get the idea.

Continue reading "PostgreSQL 9.5 Grouping Sets with PostGIS spatial aggregates"

PostGIS 2.2 leveraging power of PostgreSQL 9.5

nospam@example.com (Leo Hsu and Regina Obe) — Sat, 23 May 2015 03:11:00 -0400

Things are shaping up nicely in PostGIS 2.2 development. We are going to hit feature freeze around June 30th 2015, and plan to ship late August or early September to be in line with PostgreSQL 9.5 release. So far we have committed a couple of neat features most itemized in PostGIS 2.2 New Functions. Many of the really sort after ones will require PostgreSQL 9.5 and GEOS 3.5. The geography measurement enhancements will require Proj 4.9.0+ to take advantage of. Things I'd like to highlight and then later dedicate full-length articles in our BostonGIS Waiting for PostGIS 2.2 series once they've been stress tested.

Continue reading "PostGIS 2.2 leveraging power of PostgreSQL 9.5"

PostGIS Day synopsis

nospam@example.com (Leo Hsu and Regina Obe) — Fri, 21 Nov 2014 16:16:00 -0500

Yesterday was PostGIS day or as some may call it, Post GIS day and a couple of interesting things happened this day:

PostgreSQL 9.4 RC1 came out.
There were parties and unconferences, many summarized on http://2014.postgisday.rocks
I managed to entertain myself with a Conway's game of life PostGIS raster Map algebra style and pondered how wonderful PostGIS would be if it could generate animated gifs with some sort of aggregate function; to which I was politely called crazy by some of my fellow PSC friends.

But what was greatest of all and took the cake were these pictures:

Continue reading "PostGIS Day synopsis"

PostgreSQL 9.4beta1 and PostGIS 2.2.0 dev on Windows

nospam@example.com (Leo Hsu and Regina Obe) — Mon, 19 May 2014 21:39:00 -0400

PostgreSQL 9.4beta1 was released last week and windows binaries for both 32-bit and 64-bit are already available to try it out from http://www.postgresql.org/download/windows. Since this is a beta release, there are no installers yet, just the zip binary archive. To make the pot a little sweeter, we've setup the PostGIS windows build bot (Winnie) to automatically build for 9.4 - PostGIS 2.2.0 development branch and pgRouting 2 branches whenever there is a change in the code. We also have the pointcloud extension in the extras folder. If you are on 9.3, we've got 2.2 binaries for that as well. The PostGIS/pgRouting related stuff you can find at http://postgis.net/windows_downloads in the 9.4 folder.

For the rest of this article we'll discuss a couple of stumbling blocks you may run into.

Much of what we'll describe here is windows specific, but thanks to the beauty of extensions and GUCs, the extension install and GUC setting part for PostGIS is applicable to all operating systems.

Continue reading "PostgreSQL 9.4beta1 and PostGIS 2.2.0 dev on Windows"

PostGIS 2.1.1 released, windows installer coming soon

nospam@example.com (Leo Hsu and Regina Obe) — Sat, 09 Nov 2013 13:00:47 -0500

PostGIS 2.1.1 maintenance release was released just yesterday. Details are on the PostGIS.net site: PostGIS 2.1.1 release notes.

For windows users, we have binaries available for PostgreSQL 9.0-9.3 (64-bit) and 9.2-9.3 (32-bit). Which are available in the winnie bot section of download page. We plan to have the windows installers available for 9.2 and 9.3 (both 32 and 64-bit) in the next week or so.

PostGIS 2.1 windows bundle

nospam@example.com (Leo Hsu and Regina Obe) — Mon, 09 Sep 2013 15:35:00 -0400

PostgreSQL 9.3 came out today and we are excited to start using the new features. PostGIS 2.1.0 came out about 2 weeks ago and pgRouting just cut the RC 3 release. For windows PostGIS users who are impatient to try the new suite, binaries can be found on the Unreleased versions of PostGIS.net windows page.

We are planning an official release sometime probably next week on StackBuilder. We are waiting for release of pgRouting 2.0 before we do which should be out next week. This new 2.1 release will be dubbed the PostGIS 2.1 Bundle since it will have more than just PostGIS. It will include postgis extensions (postgis which includes geometry,raster, geography) , postgis_topology, postgis_tiger_geocoder), address_standardizer extension (a companion to tiger geocoder), and pgRouting 2.0.

For those people running PostGIS 2.0 and PostgreSQL 9.0+, especially (raster and geography) users, I highly recommend you jump to PostGIS 2.1. PostGIS 2.1 is a soft upgrade from 2.0. For raster there are enormous speed improvements and new functions. The ones we are most excited about in raster are the much much much faster ST_Clip and ST_Union functions (which now does multi-band in addition to being faster). These two functions are highly important since they are generally the first step in many raster workflows. Geography has speed improvements for point in poly and a ST_Segmentize function done on the spheroid (important for long range). Geometry has a couple of new functions. The Enhanced 3D functionality provided by SFCGAL is brand new and probably won't be distributed by many package maintainers until PostGIS 2.2 where it will garner a few more features and stability improvements.

Continue reading "PostGIS 2.1 windows bundle"

PostGIS 2.1.0 released

nospam@example.com (Leo Hsu and Regina Obe) — Sat, 17 Aug 2013 23:46:00 -0400

The 2.1.0 release of PostGIS is now available.

The PostGIS development team is proud to release PostGIS 2.1.0. As befits a minor release, the focus is on speed improvements, more features, and bug fixes.

If you are currently using PostGIS 2.0+, you can go the soft upgrade path:

ALTER EXTENSION postgis UPDATE TO "2.1.0";

If you are running 2.1.0 of beta or an unreleased version (and the above step fails with error “extension postgis has no update path from version x.x.x to version 2.1.0”) , you need to first copy the file in share\extensions\postgis—2.0.3—2.1.0.sql and change the 2.0.3 to the x.x.x noted in the error you are running. Then follow the above upgrade step again.

Users of 1.5 and below will need to go the hard-upgrade path documented in manual: http://postgis.net/docs/manual-2.1/postgis_installation.html#hard_upgrade

Best served with a bottle of GEOS 3.4.1 and PostgreSQL 9.3beta2 (planned release September).

Continue reading "PostGIS 2.1.0 released"

Materialized geometry_columns using Event Triggers

nospam@example.com (Leo Hsu and Regina Obe) — Thu, 15 Aug 2013 00:18:00 -0400

One of the big changes in PostGIS 2.0 was that the geometry_columns table became a read only view. This change was not without sacrifice. On the plus it meant a table no longer needed to be maintained which was a relief for those who had to constantly ensure the sideline table was up to date. Also on the plus it meant we could base the views on the current user so that if a user connected, they would only see tables they had access to. The sacrifice made was a bit of speed. In most use cases, the speed difference is hardly noticeable and only noticeable if you have a query constantly polling this table. However if you have a database of 15,000 geometry tables it could take as long as 450ms as noted in this ticket.

So just playing with ideas, could the new PostgreSQL 9.3 event triggers and materialized views provide a solution. Keep in mind this is just food for thought. We're not condoning people go out and hack their PostGIS install. We suspect if we implement such a thing in PostGIS core it will change from what we propose here. If you saw our prior article Caching data with materialized views and statement level triggers you know where this is going. We'll add yet another concept to this cocktail and that is what we'll call schema hiding which by that we mean counting on the order of a search_path to hide a named table/view you don't want.

Continue reading "Materialized geometry_columns using Event Triggers"

Caching data with materialized views and statement level triggers

nospam@example.com (Leo Hsu and Regina Obe) — Tue, 13 Aug 2013 22:49:00 -0400

One exciting new feature coming in PostgreSQL 9.3 is materialized views. Materialized views is really a mechanism for caching data of a query. It is especially useful if you have long running queries where the answers change infreqently. There are many things unfortunately that materialized views won't do where you are still better off with regular views.

They don't refresh themselves automatically. Thus requiring a cron job/pgagent job or a trigger on something to refresh.
They can't be user dependent or time dependent. For example if you have a view that does something like WHERE user=current_user(), then a materialized view is out of the question.

Continue reading "Caching data with materialized views and statement level triggers"

PostGIS 2.1.0rc2 released

nospam@example.com (Leo Hsu and Regina Obe) — Sat, 03 Aug 2013 15:17:00 -0400

The 2.1.0rc2 release of PostGIS is now available.

The PostGIS development team is proud to release a release candidate version of upcoming PostGIS 2.1.0. As befits a minor release, the focus is on speed improvements, more features, and bug fixes. We expect this to be the final release candidate before we officially release 2.1.0 slated for Aug 12th, 2013. We’d appreciate it if you test it before final release and report back with any issues you run into so we can have a smooth release.

Continue reading "PostGIS 2.1.0rc2 released"