Tuesday, January 29, 2013

Redis FDW update.


Today I've published a major revision to the Redis Foreign Data Wrapper code for use with PostgreSQL versions 9.2 and later. There are two new features: support for Redis structured data types and support for restricted keyspace search.

The structured data type support is enabled by using the table option "tabletype", which can take one of the values "hash", "list", "set" or "zset". If this option is used the data comes back as an array rather than as a simple scalar value. In the case of "hash" tables, the values are a sequence of key/value pairs. For the other types they are simply the elements of the relevant structure. If the values column of these tables is defined to be of type "text[]"(i.e. array of text) then an actual array is returned. If the column is a plain text column an array literal is returned. Hash table arrays in particular can be turned into records via hstore's "populate_record" functionality, or transformed in other useful ways.

Restricted keyspace search is enabled by using one of the table options "tablekeyprefix" or "tablekeyset". These are mutually exclusive. "tablekeyprefix" restricts the search to keys with the given prefix. However, in a very large Redis database this might still be expensive. In that case, it might be better to keep a list of the keys in a separate set, and this is supported using the "tablekeyset" option. When this is used the global keyspace isn't searched at all, and the list of keys is simply taken as the members of the set.

The new functionality is demonstrated in the test script (which is also new).

I am also working on some facilities to push data into Redis and similar things, but these fall outside the current possibility of a Foreign Data Wrapper, and will be announced separately.

 Credit note: this work was supported by IVC

No comments:

Post a Comment