The Semb + Wever Family
[ start | index | login ]
start > 2010-12-01 > 1

2010-12-01 #1

Created by michaelwever. Last edited by michaelwever, 4 years and 108 days ago. Viewed 1,068 times. #1
[edit] [rdf]

Hadoop + Sybase

Hadoop has the ability to write its output to a relational database using the DBOutputFormat.

Unfortunately its RecordWriter isn't so friendly to Sybase as it uses the semicolon command deliminator and Sybase bombs on this.

A simple fix is to extend DBOutputFormat like

public class SybaseDBOutputFormat extends DBOutputFormat {

@Override public String constructQuery(String table, String[] fieldNames) { return super.constructQuery(table, fieldNames).replace(";", "").toString(); }

public static void setOutput(Job job, String tableName, int fieldCount) throws IOException { DBOutputFormat.setOutput(job, tableName, fieldCount); // override job.setOutputFormatClass(SybaseDBOutputFormat.class); } }

no comments | post comment
Describe here what your SnipSnap is about!

Configure this box!

  1. Login in
  2. Click here: snipsnap-portlet-2
  3. Edit this box

>>The only thing worse than using cracked/stolen software is actually paying for it.
>>Liberate yourself. Free Software Foundation.

XHTML 1.0 validated CSS validated RSS 2.0 validated SnipSnap Get Firefox! RSS Feed
Powered by Gentoo Linux/GNU

Rainforest Site