The Semb + Wever Family
[ start | index | login ]
start > 2010-12-01 > 1

2010-12-01 #1

Created by michaelwever. Last edited by michaelwever, 3 years and 333 days ago. Viewed 1,004 times. #1
[edit] [rdf]
labels
attachments

Hadoop + Sybase

Hadoop has the ability to write its output to a relational database using the DBOutputFormat.

Unfortunately its RecordWriter isn't so friendly to Sybase as it uses the semicolon command deliminator and Sybase bombs on this.

A simple fix is to extend DBOutputFormat like

public class SybaseDBOutputFormat extends DBOutputFormat {

@Override public String constructQuery(String table, String[] fieldNames) { return super.constructQuery(table, fieldNames).replace(";", "").toString(); }

public static void setOutput(Job job, String tableName, int fieldCount) throws IOException { DBOutputFormat.setOutput(job, tableName, fieldCount); // override job.setOutputFormatClass(SybaseDBOutputFormat.class); } }

no comments | post comment
Semb Wever Family:
>>Family Tree, >>Wedding, & >>Photos
>>Music & Movies
>>Hosted GnuPG Keys

< October 2014 >
SunMonTueWedThuFriSat
1234
567891011
12131415161718
19202122232425
262728293031

Corporate links:
Finn
Bokklubben

Opensource projects:
Apache Tiles
Scarab
NetBeans


Describe here what your SnipSnap is about!

Configure this box!

  1. Login in
  2. Click here: snipsnap-portlet-2
  3. Edit this box



>>The only thing worse than using cracked/stolen software is actually paying for it.
>>Liberate yourself. Free Software Foundation.


XHTML 1.0 validated CSS validated RSS 2.0 validated SnipSnap Get Firefox! RSS Feed
Powered by Gentoo Linux/GNU






Rainforest Site