PostGIS in Action
PostgreSQL: Up and Running
Book Store
About the Authors
Consulting
PostgreSQL
PostGIS
![]() ![]()
Sunday, July 15. 2012Building on MingW and deploying on VC compiled PostgreSQL Why and Why notPrinter FriendlyWe are the windows package maintainers of PostGIS. We build these packages using MingW chain of tools. For other packages we fancy that do not come packaged with the windows VC++ builds, we also build these under mingw. We've described some of these already in File FDW family. For windows 32 builds we build with the commonly known MSys/Mingw32 chain (but an older version 1.0.11) because of issues we have building with the newer msys/mingw 32. For windows 64-bit installs, we build with the mingw-w64 chain and in fact we like the ming-w64 chain so much that we plan to migrate our Mingw32 to mingw64. We have PostgreSQL 9.2 and PostgreSQL 9.3 successfully installing under the mingw-w64 for windows 32 just fine (older PostgreSQL we experience a winsock.h something or other error which we are working on troubleshooting. For 64-bit we use ming-w64 for building extensions for PostgreSQL 9.0-9.2 and soon 9.3 with some minor issues. Some people have asked us, why put yourself thru this torture? Why not just build on MS VC++ for everything? Originally we had started on mingw because PostGIS needed a Unix like environment to compile and run thru the battery of tests. This is still the case, though PostGIS is planning a CMake move with help from Mateusz Lostkot which hopefuly will provide a better cross-platform experience and allows us to integrate pgRouting (which already is on CMake). Paul Ramsey rewrote many of the regression test scripts to be completely Perl based and not require sh. The other reality is we just prefer mingw and can't really stomach having to work with VC++. I'll describe why and why not build with mingw and deploy on VC++ compiled PostgreSQL. Continue reading "Building on MingW and deploying on VC compiled PostgreSQL Why and Why not"
Posted by Leo Hsu and Regina Obe
in contrib spotlight, PLR, postgis
at
23:59
| Comments (0)
| Trackbacks (0)
Monday, December 20. 2010PL/R Part 3: Sharing Functions across PL/R functions with plr_modulePrinter FriendlyRecommended Books: Free R reference card / Cheat Sheet PostGIS In Action R in Action R In a Nutshell In Part 2 of PL/R we covered how to build PL/R functions that take arrays and output textual outputs of generated R objects. We then used this in an aggregate SQL query using array_agg. Often when you are building PL/R functions you'll have R functions that you want to reuse many times either inside a single PL/R function or across various PL/R functions. Unfortunately, if you wanted to call a PL/R function from another PL/R function, this is not possible unless you are doing it from spi.execute call. There is another way to embed reusable R code in a PostgreSQL database. In order to be able to share databases stored R code across various PL/R functions, PL/R has a feature called a plr_module. In this tutorial we'll learn how to create and register shareable R functions with plr_module. In the next part of this series we'll start to explore generating graphs with PL/R. Continue reading "PL/R Part 3: Sharing Functions across PL/R functions with plr_module"
Posted by Leo Hsu and Regina Obe
in 8.4, 9.0, 9.1, intermediate, pl programming, PLR
at
13:37
| Comments (0)
| Trackbacks (0)
Friday, December 10. 2010PL/R Part 2: Functions that take arguments and the power of aggregationPrinter FriendlyRecommended Books: Free R reference card / Cheat Sheet PostGIS In Action R in Action R In a Nutshell In Intro to PL/R and R, we covered how to enable PL/R language in the database and wrote some PL/R functions that rendered plain text reports using the R environment. What makes combining R and PostgreSQL in PL/R most powerful is when you can start writing SQL summary queries that use R functions like any other SQL function. In this next example, we'll be using PostGIS test runs from tests we autogenerated from the Official PostGIS documentation (Documentation Driven Testing (DDT)) as described in the Garden Test section of the PostGIS Developer wiki. We've also updated some of our logging generator and test patterns so future results may not represent what we demonstrated in the last article. On a side note: Among the changes in the tests was to introduce more variants of the Empty Geometry now supported by PostGIS 2.0. Our beloved PostGIS 2.0 trunk is at the moment somewhat unstable when working with these new forms of emptiness and stuffing geometries in inappropriate places. At the moment it doesn't survive through the mindless machine gun battery of tests we have mercilessly inflicted. It's been great fun trying to build a better dummy while watching Paul run around patching holes to make the software more dummy proof as the dummy stumbles across questionable but amusing PostGIS use cases not gracefully handled by his new serialization and empty logic. On yet another side note, it's nice to see that others are doing similar wonderful things with documentation. Check out Euler's comment on catalog tables where he uses the PostgreSQL SGML documentation to autogenerate PostgreSQL catalog table comments using OpenJade's OSX to convert the SGML to XML and then XSL similar to what we did with PostGIS documentation to autogenerate PostGIS function/type comments and as a platform for our test generator. For our next exercises we'll be using the power of aggregation to push data into R instead of pg.spi.execute. This will make our functions far more reusable and versatile. Continue reading "PL/R Part 2: Functions that take arguments and the power of aggregation"
Posted by Leo Hsu and Regina Obe
in 8.4, 9.0, 9.1, intermediate, pl programming, PLR
at
01:38
| Comments (0)
| Trackbacks (0)
Sunday, November 28. 2010Quick Intro to R and PL/R - Part 1Printer FriendlyRecommended Books: Free R reference card / Cheat Sheet PostGIS In Action R in Action R In a Nutshell In this article we'll provide a summary of what PL/R is and how to get running with it. Since we don't like repeating ourselves, we'll refer you to an article we wrote a while ago which is still fairly relevant today called Up and Running with PL/R (PLR) in PostgreSQL: An almost Idiot's Guide and just fill in the parts that have changed. We should note that particular series was more geared toward the spatial database programmer (PostGIS in particular). There is a lot of overlap between the PL/R, R, and PostGIS user-base which is comprised of many environmental scientists and researchers in need of powerful charting and stats tools to analyse their data who are high on the smart but low on the money human spectrum. This series will be more of a general PL/R user perspective. We'll follow more of the same style we did with Quick Intro to PL/Python. We'll end our series with a PL/R cheatsheet similar to what we had for PL/Python. As stated in our State of PostGIS article, we'll be using log files we generated from our PostGIS stress tests. These stress tests were auto-generated from the PostGIS official documentation. The raster tests are comprised of 2,095 query executions exercising all the pixel types supported. The geometry/geograpy tests are comprised of 65,892 spatial SQL queries exercising every PostGIS geometry/geography supported in PostGIS 2.0 -- yes this includes TINS, Triangles,Polyhedral Surfaces, Curved geometries and all dimensions of them. Most queries are unique. If you are curious to see what these log tables look like or want to follow along with these exercises, you can download the tables from here. What is R and PL/R and why should you care?R is both a language and an environment for doing statistics and generating graphs and plots. It is GNU-licensed and a common favorite of Universities and Research institutions. PL/R is a procedural language for PostgreSQL that allows you to write database stored functions in R. R is a set-based and domain specific language similar to SQL except unlike the way relational databases treat data, it thinks of data as matrices, lists and vectors. I tend to think of it as a cross between LISP and SQL though more experienced Lisp and R users will probably disagree with me on that. This makes it easier in many cases to tabulate data both across columns as well as across rows. The examples we will show in these exercises, could be done in SQL, but they are much more succinct to write in R. In addition to the language itself, there are a whole wealth of statistical and graphing functions available in R that you will not find in any relational database. These functions are growing as more people contribute packages. Its packaging system called Comprehensive R Archive (CRAN) is similar in concept to Perl's CPAN and the in the works PGXN for PostgreSQL. Continue reading "Quick Intro to R and PL/R - Part 1"
Posted by Leo Hsu and Regina Obe
in 9.0, gis, intermediate, pl programming, PLR
at
14:19
| Comments (3)
| Trackbacks (0)
Tuesday, November 23. 2010The State of PostGIS, Joys of Testing, and PLR the PrequelPrinter FriendlyRecommended Books: PostGIS In Action R in Action I've always enjoyed dismantling things. Deconstruction was a good way of analyzing how things were built by cataloging all the ways I could dismantle or destroy them. I experimented with mechanical systems, electrical circuitry, chemicals and biological systems sometimes coming close to bodily harm. In later years I decided to play it safe and just stick with programming and computer simulation as a convenient channel to enjoy my destructive pursuits. Now getting to the point of this article. In later articles, I'll start to demonstrate the use of PL/R, the procedural language for PostgreSQL that allows you to program functions in the statistical language and Environment R. To make these examples more useful, I'll be analyzing data generated from PostGIS tests I've been working on for stress testing the upcoming PostGIS 2.0. PostGIS 2.0 is a major and probably the most exciting release for us. Paul Ramsey did a summary talk recently of Past, Present, Future of PostGIS at State of PostGIS FOSS4G Japan http://www.ustream.tv/recorded/10667125 which provides a brief glimpse of what's in store in 2.0. Continue reading "The State of PostGIS, Joys of Testing, and PLR the Prequel"
Posted by Leo Hsu and Regina Obe
in 9.0, editor note, gis, PLR, postgis
at
18:22
| Comments (0)
| Trackbacks (0)
Sunday, February 07. 2010PostGIS 1.5.0 out and PLR working on Windows 8.3-8.4 installsPrinter FriendlyRecommended Books: PostGIS In Action PostgreSQL 8.4 Official The SQL Language PostgreSQL 8.4 Server Administration PostGIS 1.5.0 is finally outI'm happy to report that after a long haul, we have finally released PostGIS 1.5.0. Two months late, but there it is, and its a really great release I think; Perhaps the best release ever. Details on what makes this release so special. The geodetic support. Summary excerpted from Paul's slightly premature announcement February 4, 2010 The PostGIS development team has, after a long course of reflection and a detailed self-examination of our various personal failings, decided to release PostGIS 1.5.0 to the public. http://postgis.org/download/postgis-1.5.0.tar.gz This new version of PostGIS includes a new "geography" type for managing geodetic (lat/lon) data, performance-enhanced distance calculations, GML and KML format readers, an improved shape loading GUI, and other new features as well. Especial thanks to: * Dave Skea for algorithms and mathematics necessary to support spherical geometry * Nicklas Avén for the new performance enhanced distance calculations and other distance-related functions * Sandro Santilli for new buffering features (end caps and style options) * Olivier Courtin for GML/KML input functions * Guillaume Lelarge for support for the upcoming PgSQL 9.0 * George Silva for an example implementation of history tables * Vincent Picavet for Hausdorff distance calculations * The maintainers of GEOS, Proj4, and LibXML, without whom we would have less of a spatial database Love, the PostGIS project steering committee, Mark Cave-Ayland Kevin Neufeld Regina Obe Paul Ramsey Continue reading "PostGIS 1.5.0 out and PLR working on Windows 8.3-8.4 installs"
Posted by Leo Hsu and Regina Obe
in 8.3, 8.4, pl programming, PLR, postgis
at
00:58
| Comments (2)
| Trackbacks (0)
Saturday, May 10. 2008Choosing the right Database Procedural Language PLPrinter FriendlyOne of the great selling points of PostgreSQL is its pluggable PL language architecture. MySQL is known for its pluggable storage and PostgreSQL is known for its pluggable PL language architecture. From Monty's notes on slide 12 looks like MySQL may be working on a pluggable PL language architecture of their own. The most common of these languages are the all-purpose languages SQL and C (these are built-in and not really PLs like the others, but we'll throw them in there), PLPgSQL which is also built-in but not always enabled, PL/Perl, PL/Python, and the domain specific languages PL/R, PL/SH and gaining popularity Skype released PL/Proxy. There are others in the family such as PL/Tcl, PL/PHP, PL/Ruby, PL/Scheme (a dialect of Lisp), PL/Java, PL/Lua and PL/LOLCode (for kicks and as a reference implementation. Think of LOLCode as PostgreSQL Pluggable PL equivalent of MySQL's BLACK HOLE storage engine.) . The other interesting thing about the PostgreSQL PL language architecture is that it is a fairly thin wrapper around these languages. This means the kind of code you write in those languages is pretty much what you would write if you were doing general programming in those languages minus some spi calls. Since the handler is a just a thin wrapper around the environment, the language environment must be installed on the database server before you can use the PL language handler. This means you can have these functions utilized in your SQL statements and you can write in a language you feel comfortable with if you can get the darn PL compiled for your environment or someone has already kindly compiled it for your environment or that it is even compilable for your environment. The pluggable PL architecture means you can write a PL Handler for your favorite language or invent your own language that you can run in the database. In the end the barrier between code,data, and semantic constructs is more of a constraint imposed by compilers. If you have any doubts about the above statement, you need only look at some javascript injection attacks to bring the statement home. One of my fantasies is developing a language that morphs itself, that utilizes the database as its morphing engine and its OS and that breaks the illusion of data being data, code being code, and lacks rigid semantics. Of the languages we have worked with, SmallTalk comes closest to a language that satisfies these ideals and Lisp to a much lesser extent. Lisp lacked the semantic elegance of SmallTalk among other things. Most people are used to having their procedural language push their data around. PL code living in PostgreSQL allows your data to push your procedural code around in a set-based way. This is a simple but pretty powerful feature since data is in general more fluid than code. For interpretated/just-in time compiled languages it can live in the database, for compiled it has to call compiled functions. Continue reading "Choosing the right Database Procedural Language PL"
Posted by Leo Hsu and Regina Obe
in beginner, mysql, pl programming, plperl, plpgsql, PLR, sql functions, sql server
at
06:58
| Comments (4)
| Trackback (1)
Friday, November 30. 2007Language Architecture in PostgreSQLPrinter FriendlyPerhaps one of the most unique and exciting things that makes PostgreSQL stand out from other database systems, are the numerous choices of languages one can use to create database functions, triggers and define new aggregate functions with. Not only can you use various languages to write your database stored functions with, but often times the code you write lives right in the database. You have no idea how cool this is until you see it in action. The other interesting thing about the PostgreSQL language architecture is the relative ease with which new languages can be incorporated in the system. Native Languages of PostgreSQLThere are 3 languages that come packaged with PostgreSQL (2 non-PL ones are installed automatically and not even listed as languages (C and SQL) in the languages section of a db). The defacto PL/PgSQL procedural language is available for install in all PostgreSQL distributions, but need not be installed in a db by default .
The PL languagesAside from PL/pgSQL there are numerous other procedural languages that one can use to create database stored functions and triggers. Some of these languages are fairly stable and even more are experimental. Some are only supported on Unix/Linux, but many are supported on Unix/Linux/MacOS/windows. In any case there are 3 key components needed before you can start using a new language:
Registering a language in a DatabaseFor pl/pgsql items 1 and 2 are already done if you have a working PostgreSQL install. In order to accomplish item 3, you may need to do the following from psql or PgAdmin III query window.
Alternatively you can run createlang plpgsql somedb from commandline. Note createlang is a command line program that is located in the bin folder of your PostgreSQL install. To see a list of procedural languages that you already have call handlers registered for in PostgreSQL. These are the languages you can register in your specific database - do a SELECT * FROM pg_catalog.pg_pltemplate
A Flavor of the Procedural Languages (PLs)In this section, we'll show a brief sampling of what functions look like written in various PLs. These are not to suggest they are the only ones that exist. For these examples, I'm going to use the $ quoting syntax introduced in PostgreSQL 8.0 which allows for not having to escape out single quotes. SQL - the not PL languageFor basic CRUD stuff,selects and simple functions, nothing hits the spot like just plain old SQL. Since this is such a common choice and often the best choice - here are 3 examples.
For details on using out parameters, check out Robert Treat's out parameter sql & plpgsql examples PLPGSQL - a real PL LanguageFor more complex logic and massaging of results before sending back. You need something more powerful than standard SQL. Below are some examples using PLPGSQL.
Using PL/Perl
Using PL/R a language and environment for statisticsOne of my favorite PL languages to program is PL/R. The reason for this is that the R statistical environment is such a rich environment for doing statistical processing. It now is also supported on windows as well as Mac and Linux. To learn more about R and installing PL/R. Check out our Boston GIS article PLR Part 1: Up and Running with PL/R (PLR) in PostgreSQL: An almost Idiot's Guide Below is the classic median aggregate function in R. It uses the native median function in the R environment to create a PostgreSQL aggregate median function
We will be covering PLR in greater detail in another article.
Posted by Leo Hsu and Regina Obe
in intermediate, pl programming, plpgsql, PLR, sql functions
at
00:00
| Comments (4)
| Trackbacks (3)
(Page 1 of 1, totaling 8 entries)
|
QuicksearchCalendar
CategoriesArchivesSyndicate This BlogBlog AdministrationShow tagged entriesRemote RSS/OPML-Blogroll FeedNo RSS/OPML feed selected
|
|||||||||||||||||||||||||||||||||||||||||||||||||




