orc\_fdw

September 22, 2014 ยท View on GitHub

Foreign data wrapper for reading ORC formatted files.

Installation

  1. Clone this repo to the contrib directory of postgresql source code with the command git clone https://github.com/gokhankici/orc_fdw.git

  2. Install protobuf-c library. First protobuf then protobuf-c library should be installed. Both can be installed simply by downloading and extracting them and then issuing the following commands in their root folder:

    ./configure
make
sudo make install

  3. Run sh init.sh in the orc_fdw folder to convert the ORC protobuf definitions into C source code.

  4. Run make install in the orc_fdw folder to compile and install the extension.

Converting To ORC Format

To convert your plain text files into the ORC format, a sample Java program in the converter folder can be used. It's a maven project, so maven should be installed on your system. Hive v0.12 is needed for the fdw, so the provided hive-exec package should be used to compile the code (it isn't added as a maven dependency since it isn't contained in the repos). Eclipse could be used to add the hive-exec package as an external jar file and compile/run the project.