免费观看又色又爽又黄的小说免费_美女福利视频国产片_亚洲欧美精品_美国一级大黄大色毛片

Hadoop學(xué)習(xí)--MapperReduce--day08

mapper類的代碼:

專業(yè)領(lǐng)域包括網(wǎng)站制作、成都網(wǎng)站設(shè)計(jì)、電子商務(wù)商城網(wǎng)站建設(shè)、微信營銷、系統(tǒng)平臺(tái)開發(fā), 與其他網(wǎng)站設(shè)計(jì)及系統(tǒng)開發(fā)公司不同,成都創(chuàng)新互聯(lián)公司的整合解決方案結(jié)合了幫做網(wǎng)絡(luò)品牌建設(shè)經(jīng)驗(yàn)和互聯(lián)網(wǎng)整合營銷的理念,并將策略和執(zhí)行緊密結(jié)合,為客戶提供全網(wǎng)互聯(lián)網(wǎng)整合方案。

實(shí)現(xiàn)Mapper類的方法

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

public class MyMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

// constant

private static final int MISSING = 9999;

// map function

@Override

protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, IntWritable>.Context context)

throws IOException, InterruptedException {

// per line

String line = value.toString();

// get year

String year = line.substring(15, 19);

// get airtemp

int airTemperature;

if (line.charAt(87) == '+') {

airTemperature = Integer.parseInt(line.substring(88, 92));

} else {

airTemperature = Integer.parseInt(line.substring(87, 92));

}

// valid air temp data

String quality = line.substring(92, 93);

if (airTemperature != MISSING && quality.matches("[01459]")) {

context.write(new Text(year), new IntWritable(airTemperature));

}

}

}

Reduce類的代碼:

實(shí)現(xiàn)Reducer類的方法

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Reducer;

public class MyReducer extends Reducer<Text, IntWritable, Text, IntWritable> {

@Override

protected void reduce(Text key, Iterable<IntWritable> values, Context context)

throws IOException, InterruptedException {

// max

int maxValue = Integer.MIN_VALUE;

// for

for (IntWritable value : values) {

maxValue = Math.max(maxValue, value.get());

}

// output

context.write(key, new IntWritable(maxValue));

}

}

主方法的代碼:

import mapper類實(shí)現(xiàn).MyMapper;

import reducer類實(shí)現(xiàn).MyReducer;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Job;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class MyMapperApp {

/**

* @param args

* @throws Exception

* @throws IllegalArgumentException

*/

public static void main(String[] args) throws IllegalArgumentException, Exception {

// new job

Job job = Job.getInstance();

// find jar by ClassName

job.setJarByClass(MyMapper.class);

// job name

job.setJobName("Max temperature");

FileInputFormat.addInputPath(job, new Path("file:///mnt/hgfs/test-ncdc-data"));

FileOutputFormat.setOutputPath(job, new Path("file:///home/hadoop/mr/"));

job.setMapperClass(MyMapper.class);

job.setReducerClass(MyReducer.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(IntWritable.class);

System.exit(job.waitForCompletion(true) ? 0 : 1);

}

}

網(wǎng)站標(biāo)題:Hadoop學(xué)習(xí)--MapperReduce--day08
轉(zhuǎn)載注明:http://m.newbst.com/article4/jhssoe.html

成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián),為您提供微信公眾號(hào)外貿(mào)網(wǎng)站建設(shè)網(wǎng)站排名全網(wǎng)營銷推廣品牌網(wǎng)站制作軟件開發(fā)

廣告

聲明:本網(wǎng)站發(fā)布的內(nèi)容(圖片、視頻和文字)以用戶投稿、用戶轉(zhuǎn)載內(nèi)容為主,如果涉及侵權(quán)請(qǐng)盡快告知,我們將會(huì)在第一時(shí)間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場,如需處理請(qǐng)聯(lián)系客服。電話:028-86922220;郵箱:631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載,或轉(zhuǎn)載時(shí)需注明來源: 創(chuàng)新互聯(lián)

成都網(wǎng)頁設(shè)計(jì)公司