NoSQL/Hadoop
hadoop text file 읽기
세모데
2017. 3. 13. 16:02
하둡에 gzipped file, sequence files and avro datafile들을 command line 상에서 읽기를 원할때
아래와 같이 command를 수행하면 됨
hadoop fs -text /tmp/test/test.avro | more
hadoop fs -text /tmp/test/test.seq | more
sequence file sort 테스트
hadoop jar \
$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar \
sort -r 1 \
-inFormat org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat \
-outFormat org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat \
-outKey org.apache.hadoop.io.IntWritable \
-outValue org.apache.hadoop.io.Text \
test.seq sorted