2010-12-01 #1

Created by michaelwever. Last edited by michaelwever, 2 years and 171 days ago. Viewed 634 times. #1

[edit] [rdf]

labels

attachments

Hadoop + Sybase

Hadoop has the ability to write its output to a relational database using the DBOutputFormat.

Unfortunately its RecordWriter isn't so friendly to Sybase as it uses the semicolon command deliminator and Sybase bombs on this.

A simple fix is to extend DBOutputFormat like

public class SybaseDBOutputFormat extends DBOutputFormat {    @Override
    public String constructQuery(String table, String[] fieldNames) {
        return super.constructQuery(table, fieldNames).replace(";", "").toString();
    }
    public static void setOutput(Job job, String tableName, int fieldCount) throws IOException {
        DBOutputFormat.setOutput(job, tableName, fieldCount);
        // override
        job.setOutputFormatClass(SybaseDBOutputFormat.class);
    }
}