zzg/pj2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
pj2/doc/edu/rit/pjmr/MapperTask.html
T
Zack Zatkin-Gold c2c773e0c3 initial revision
2016-04-27 01:07:35 -04:00

351 lines
19 KiB
HTML
Raw Permalink Blame History

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!-- NewPage -->
<html lang="en">
<head>
<!-- Generated by javadoc (version 1.7.0_51) on Wed Apr 06 13:11:06 EDT 2016 -->
<title>MapperTask (Parallel Java 2 Library Documentation)</title>
<meta name="date" content="2016-04-06">
<link rel="stylesheet" type="text/css" href="../../../stylesheet.css" title="Style">
</head>
<body>
<script type="text/javascript"><!--
if (location.href.indexOf('is-external=true') == -1) {
parent.document.title="MapperTask (Parallel Java 2 Library Documentation)";
}
//-->
</script>
<noscript>
<div>JavaScript is disabled on your browser.</div>
</noscript>
<!-- ========= START OF TOP NAVBAR ======= -->
<div class="topNav"><a name="navbar_top">
<!-- -->
</a><a href="#skip-navbar_top" title="Skip navigation links"></a><a name="navbar_top_firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/MapperTask.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../index-files/index-1.html">Index</a></li>
<li><a href="../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../edu/rit/pjmr/MapperConfig.html" title="class in edu.rit.pjmr"><span class="strong">Prev Class</span></a></li>
<li><a href="../../../edu/rit/pjmr/MapperTaskConfig.html" title="class in edu.rit.pjmr"><span class="strong">Next Class</span></a></li>
</ul>
<ul class="navList">
<li><a href="../../../index.html?edu/rit/pjmr/MapperTask.html" target="_top">Frames</a></li>
<li><a href="MapperTask.html" target="_top">No Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_top">
<li><a href="../../../allclasses-noframe.html">All Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_top");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li><a href="#fields_inherited_from_class_edu.rit.pj2.Task">Field</a>&nbsp;|&nbsp;</li>
<li><a href="#constructor_summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method_summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor_detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method_detail">Method</a></li>
</ul>
</div>
<a name="skip-navbar_top">
<!-- -->
</a></div>
<!-- ========= END OF TOP NAVBAR ========= -->
<!-- ======== START OF CLASS DATA ======== -->
<div class="header">
<div class="subTitle">edu.rit.pjmr</div>
<h2 title="Class MapperTask" class="title">Class MapperTask&lt;IK,IV,OK,OV extends <a href="../../../edu/rit/pj2/Vbl.html" title="interface in edu.rit.pj2">Vbl</a>&gt;</h2>
</div>
<div class="contentContainer">
<ul class="inheritance">
<li><a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">java.lang.Object</a></li>
<li>
<ul class="inheritance">
<li><a href="../../../edu/rit/pj2/Task.html" title="class in edu.rit.pj2">edu.rit.pj2.Task</a></li>
<li>
<ul class="inheritance">
<li>edu.rit.pjmr.MapperTask&lt;IK,IV,OK,OV&gt;</li>
</ul>
</li>
</ul>
</li>
</ul>
<div class="description">
<ul class="blockList">
<li class="blockList">
<dl><dt><span class="strong">Type Parameters:</span></dt><dd><code>IK</code> - Mapper input key data type (data record ID).</dd><dd><code>IV</code> - Mapper input value data type (data record contents).</dd><dd><code>OK</code> - Mapper output key data type.</dd><dd><code>OV</code> - Mapper output value data type; must implement interface
<a href="../../../edu/rit/pj2/Vbl.html" title="interface in edu.rit.pj2">Vbl</a>.</dd></dl>
<hr>
<br>
<pre>public class <span class="strong">MapperTask&lt;IK,IV,OK,OV extends <a href="../../../edu/rit/pj2/Vbl.html" title="interface in edu.rit.pj2">Vbl</a>&gt;</span>
extends <a href="../../../edu/rit/pj2/Task.html" title="class in edu.rit.pj2">Task</a></pre>
<div class="block">Class MapperTask provides a mapper task in the Parallel Java Map Reduce
Framework. Do not create mapper tasks directly; rather, define mapper tasks
as part of a <a href="../../../edu/rit/pjmr/PjmrJob.html" title="class in edu.rit.pjmr">PjmrJob</a>.
<P>
Each mapper task in a PJMR job does the following, using the configuration
specified in the PJMR job. For further information, refer to the
documentation for the classes mentioned below.
<OL TYPE=1>
<P><LI>
For each of the mapper task's configured <a href="../../../edu/rit/pjmr/Source.html" title="interface in edu.rit.pjmr">Source</a>s, create the
source object and call its <TT>open()</TT> method.
<P><LI>
For each source's configured <a href="../../../edu/rit/pjmr/Mapper.html" title="class in edu.rit.pjmr">Mapper</a>s, create the mapper object.
<P><LI>
Create the mapper task's configured <a href="../../../edu/rit/pjmr/Combiner.html" title="class in edu.rit.pjmr">Combiner</a>. This is the global
combiner.
<P><LI>
If the mapper task was configured with a <a href="../../../edu/rit/pjmr/Customizer.html" title="class in edu.rit.pjmr">Customizer</a>, create the
customizer object and call its <TT>start()</TT> method, passing in any
configured argument strings plus the global combiner.
<P><LI>
Run the mapper(s) in parallel, each in its own thread. Each mapper thread
does the following:
<OL TYPE=a>
<P><LI>
Create a thread-local copy of the global combiner.
<P><LI>
Call the mapper's <TT>start()</TT> method, passing in any configured argument
strings plus the thread-local combiner.
<P><LI>
Repeatedly call the mapper's source's <TT>next()</TT> method to obtain the
next data record from the source, and call the mapper's <TT>map()</TT>
method, passing in the data record's (key, value) plus the thread-local
combiner. (Note that if there is more than one mapper for the same source,
all the mappers call the source's <TT>next()</TT> method concurrently; the
<TT>next()</TT> method is multiple thread safe.)
<P><LI>
When there are no more data records, call the mapper's <TT>finish()</TT>
method, passing in the thread-local combiner.
</OL>
<P><LI>
After all the mapper threads have finished, reduce the thread-local combiners
into the global combiner.
<P><LI>
If the mapper task was configured with a customizer, call the customizer's
<TT>finish()</TT> method, passing in the global combiner.
<P><LI>
Call each source's <TT>close()</TT> method.
<P><LI>
Send the global combiner to the <a href="../../../edu/rit/pjmr/ReducerTask.html" title="class in edu.rit.pjmr">ReducerTask</a>.
</OL></div>
</li>
</ul>
</div>
<div class="summary">
<ul class="blockList">
<li class="blockList">
<!-- =========== FIELD SUMMARY =========== -->
<ul class="blockList">
<li class="blockList"><a name="field_summary">
<!-- -->
</a>
<h3>Field Summary</h3>
<ul class="blockList">
<li class="blockList"><a name="fields_inherited_from_class_edu.rit.pj2.Task">
<!-- -->
</a>
<h3>Fields inherited from class&nbsp;edu.rit.pj2.<a href="../../../edu/rit/pj2/Task.html" title="class in edu.rit.pj2">Task</a></h3>
<code><a href="../../../edu/rit/pj2/Task.html#ALL_CORES">ALL_CORES</a>, <a href="../../../edu/rit/pj2/Task.html#ALL_GPUS">ALL_GPUS</a>, <a href="../../../edu/rit/pj2/Task.html#ANY_NODE_NAME">ANY_NODE_NAME</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_CHUNK">DEFAULT_CHUNK</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_CORES">DEFAULT_CORES</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_GPUS">DEFAULT_GPUS</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_NODE_NAME">DEFAULT_NODE_NAME</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_SCHEDULE">DEFAULT_SCHEDULE</a>, <a href="../../../edu/rit/pj2/Task.html#DEFAULT_THREADS">DEFAULT_THREADS</a>, <a href="../../../edu/rit/pj2/Task.html#dynamic">dynamic</a>, <a href="../../../edu/rit/pj2/Task.html#fixed">fixed</a>, <a href="../../../edu/rit/pj2/Task.html#guided">guided</a>, <a href="../../../edu/rit/pj2/Task.html#leapfrog">leapfrog</a>, <a href="../../../edu/rit/pj2/Task.html#proportional">proportional</a>, <a href="../../../edu/rit/pj2/Task.html#STANDARD_CHUNK">STANDARD_CHUNK</a>, <a href="../../../edu/rit/pj2/Task.html#THREADS_EQUALS_CORES">THREADS_EQUALS_CORES</a></code></li>
</ul>
</li>
</ul>
<!-- ======== CONSTRUCTOR SUMMARY ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor_summary">
<!-- -->
</a>
<h3>Constructor Summary</h3>
<table class="overviewSummary" border="0" cellpadding="3" cellspacing="0" summary="Constructor Summary table, listing constructors, and an explanation">
<caption><span>Constructors</span><span class="tabEnd">&nbsp;</span></caption>
<tr>
<th class="colOne" scope="col">Constructor and Description</th>
</tr>
<tr class="altColor">
<td class="colOne"><code><strong><a href="../../../edu/rit/pjmr/MapperTask.html#MapperTask()">MapperTask</a></strong>()</code>&nbsp;</td>
</tr>
</table>
</li>
</ul>
<!-- ========== METHOD SUMMARY =========== -->
<ul class="blockList">
<li class="blockList"><a name="method_summary">
<!-- -->
</a>
<h3>Method Summary</h3>
<table class="overviewSummary" border="0" cellpadding="3" cellspacing="0" summary="Method Summary table, listing methods, and an explanation">
<caption><span>Methods</span><span class="tabEnd">&nbsp;</span></caption>
<tr>
<th class="colFirst" scope="col">Modifier and Type</th>
<th class="colLast" scope="col">Method and Description</th>
</tr>
<tr class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><strong><a href="../../../edu/rit/pjmr/MapperTask.html#main(java.lang.String[])">main</a></strong>(<a href="http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>[]&nbsp;args)</code>
<div class="block">Mapper task main program.</div>
</td>
</tr>
</table>
<ul class="blockList">
<li class="blockList"><a name="methods_inherited_from_class_edu.rit.pj2.Task">
<!-- -->
</a>
<h3>Methods inherited from class&nbsp;edu.rit.pj2.<a href="../../../edu/rit/pj2/Task.html" title="class in edu.rit.pj2">Task</a></h3>
<code><a href="../../../edu/rit/pj2/Task.html#actualThreads()">actualThreads</a>, <a href="../../../edu/rit/pj2/Task.html#addTupleListener(edu.rit.pj2.TupleListener)">addTupleListener</a>, <a href="../../../edu/rit/pj2/Task.html#chunk()">chunk</a>, <a href="../../../edu/rit/pj2/Task.html#chunk(int)">chunk</a>, <a href="../../../edu/rit/pj2/Task.html#cores()">cores</a>, <a href="../../../edu/rit/pj2/Task.html#cores(int)">cores</a>, <a href="../../../edu/rit/pj2/Task.html#coresRequired()">coresRequired</a>, <a href="../../../edu/rit/pj2/Task.html#getMatchingTuple(int)">getMatchingTuple</a>, <a href="../../../edu/rit/pj2/Task.html#gpus()">gpus</a>, <a href="../../../edu/rit/pj2/Task.html#gpus(int)">gpus</a>, <a href="../../../edu/rit/pj2/Task.html#gpusRequired()">gpusRequired</a>, <a href="../../../edu/rit/pj2/Task.html#groupSize()">groupSize</a>, <a href="../../../edu/rit/pj2/Task.html#initialChunk()">initialChunk</a>, <a href="../../../edu/rit/pj2/Task.html#initialSchedule()">initialSchedule</a>, <a href="../../../edu/rit/pj2/Task.html#initialThreads()">initialThreads</a>, <a href="../../../edu/rit/pj2/Task.html#matchingTupleCount()">matchingTupleCount</a>, <a href="../../../edu/rit/pj2/Task.html#nodeName()">nodeName</a>, <a href="../../../edu/rit/pj2/Task.html#nodeName(java.lang.String)">nodeName</a>, <a href="../../../edu/rit/pj2/Task.html#nodeNameRequired()">nodeNameRequired</a>, <a href="../../../edu/rit/pj2/Task.html#parallelDo(int, edu.rit.pj2.Section)">parallelDo</a>, <a href="../../../edu/rit/pj2/Task.html#parallelDo(edu.rit.pj2.Section...)">parallelDo</a>, <a href="../../../edu/rit/pj2/Task.html#parallelFor(int, int)">parallelFor</a>, <a href="../../../edu/rit/pj2/Task.html#parallelFor(long, long)">parallelFor</a>, <a href="../../../edu/rit/pj2/Task.html#parallelFor(edu.rit.pj2.WorkQueue)">parallelFor</a>, <a href="../../../edu/rit/pj2/Task.html#putTuple(int, edu.rit.pj2.Tuple)">putTuple</a>, <a href="../../../edu/rit/pj2/Task.html#putTuple(edu.rit.pj2.Tuple)">putTuple</a>, <a href="../../../edu/rit/pj2/Task.html#readTuple(T)">readTuple</a>, <a href="../../../edu/rit/pj2/Task.html#schedule()">schedule</a>, <a href="../../../edu/rit/pj2/Task.html#schedule(edu.rit.pj2.Schedule)">schedule</a>, <a href="../../../edu/rit/pj2/Task.html#takeTuple(T)">takeTuple</a>, <a href="../../../edu/rit/pj2/Task.html#taskRank()">taskRank</a>, <a href="../../../edu/rit/pj2/Task.html#threads()">threads</a>, <a href="../../../edu/rit/pj2/Task.html#threads(int)">threads</a>, <a href="../../../edu/rit/pj2/Task.html#tryToReadTuple(T)">tryToReadTuple</a>, <a href="../../../edu/rit/pj2/Task.html#tryToTakeTuple(T)">tryToTakeTuple</a>, <a href="../../../edu/rit/pj2/Task.html#workerFor()">workerFor</a></code></li>
</ul>
<ul class="blockList">
<li class="blockList"><a name="methods_inherited_from_class_java.lang.Object">
<!-- -->
</a>
<h3>Methods inherited from class&nbsp;java.lang.<a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true" title="class or interface in java.lang">Object</a></h3>
<code><a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone()" title="class or interface in java.lang">clone</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals(java.lang.Object)" title="class or interface in java.lang">equals</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize()" title="class or interface in java.lang">finalize</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass()" title="class or interface in java.lang">getClass</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode()" title="class or interface in java.lang">hashCode</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify()" title="class or interface in java.lang">notify</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notifyAll()" title="class or interface in java.lang">notifyAll</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#toString()" title="class or interface in java.lang">toString</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait()" title="class or interface in java.lang">wait</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait(long)" title="class or interface in java.lang">wait</a>, <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait(long, int)" title="class or interface in java.lang">wait</a></code></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div class="details">
<ul class="blockList">
<li class="blockList">
<!-- ========= CONSTRUCTOR DETAIL ======== -->
<ul class="blockList">
<li class="blockList"><a name="constructor_detail">
<!-- -->
</a>
<h3>Constructor Detail</h3>
<a name="MapperTask()">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>MapperTask</h4>
<pre>public&nbsp;MapperTask()</pre>
</li>
</ul>
</li>
</ul>
<!-- ============ METHOD DETAIL ========== -->
<ul class="blockList">
<li class="blockList"><a name="method_detail">
<!-- -->
</a>
<h3>Method Detail</h3>
<a name="main(java.lang.String[])">
<!-- -->
</a>
<ul class="blockListLast">
<li class="blockList">
<h4>main</h4>
<pre>public&nbsp;void&nbsp;main(<a href="http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true" title="class or interface in java.lang">String</a>[]&nbsp;args)
throws <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Exception.html?is-external=true" title="class or interface in java.lang">Exception</a></pre>
<div class="block">Mapper task main program.</div>
<dl>
<dt><strong>Specified by:</strong></dt>
<dd><code><a href="../../../edu/rit/pj2/Task.html#main(java.lang.String[])">main</a></code>&nbsp;in class&nbsp;<code><a href="../../../edu/rit/pj2/Task.html" title="class in edu.rit.pj2">Task</a></code></dd>
<dt><span class="strong">Parameters:</span></dt><dd><code>args</code> - Array of command line argument strings.
<UL>
<LI><TT>args[0]</TT> = Node name
</UL></dd>
<dt><span class="strong">Throws:</span></dt>
<dd><code><a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Exception.html?is-external=true" title="class or interface in java.lang">Exception</a></code> - The <TT>main()</TT> method can throw any exception.</dd></dl>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
</div>
<!-- ========= END OF CLASS DATA ========= -->
<!-- ======= START OF BOTTOM NAVBAR ====== -->
<div class="bottomNav"><a name="navbar_bottom">
<!-- -->
</a><a href="#skip-navbar_bottom" title="Skip navigation links"></a><a name="navbar_bottom_firstrow">
<!-- -->
</a>
<ul class="navList" title="Navigation">
<li><a href="../../../overview-summary.html">Overview</a></li>
<li><a href="package-summary.html">Package</a></li>
<li class="navBarCell1Rev">Class</li>
<li><a href="class-use/MapperTask.html">Use</a></li>
<li><a href="package-tree.html">Tree</a></li>
<li><a href="../../../deprecated-list.html">Deprecated</a></li>
<li><a href="../../../index-files/index-1.html">Index</a></li>
<li><a href="../../../help-doc.html">Help</a></li>
</ul>
</div>
<div class="subNav">
<ul class="navList">
<li><a href="../../../edu/rit/pjmr/MapperConfig.html" title="class in edu.rit.pjmr"><span class="strong">Prev Class</span></a></li>
<li><a href="../../../edu/rit/pjmr/MapperTaskConfig.html" title="class in edu.rit.pjmr"><span class="strong">Next Class</span></a></li>
</ul>
<ul class="navList">
<li><a href="../../../index.html?edu/rit/pjmr/MapperTask.html" target="_top">Frames</a></li>
<li><a href="MapperTask.html" target="_top">No Frames</a></li>
</ul>
<ul class="navList" id="allclasses_navbar_bottom">
<li><a href="../../../allclasses-noframe.html">All Classes</a></li>
</ul>
<div>
<script type="text/javascript"><!--
allClassesLink = document.getElementById("allclasses_navbar_bottom");
if(window==top) {
allClassesLink.style.display = "block";
}
else {
allClassesLink.style.display = "none";
}
//-->
</script>
</div>
<div>
<ul class="subNavList">
<li>Summary:&nbsp;</li>
<li>Nested&nbsp;|&nbsp;</li>
<li><a href="#fields_inherited_from_class_edu.rit.pj2.Task">Field</a>&nbsp;|&nbsp;</li>
<li><a href="#constructor_summary">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method_summary">Method</a></li>
</ul>
<ul class="subNavList">
<li>Detail:&nbsp;</li>
<li>Field&nbsp;|&nbsp;</li>
<li><a href="#constructor_detail">Constr</a>&nbsp;|&nbsp;</li>
<li><a href="#method_detail">Method</a></li>
</ul>
</div>
<a name="skip-navbar_bottom">
<!-- -->
</a></div>
<!-- ======== END OF BOTTOM NAVBAR ======= -->
<p class="legalCopy"><small><FONT SIZE="-2">Copyright &copy; 2013&ndash;2016 by Alan Kaminsky. All rights reserved. Send comments to ark</FONT><FONT SIZE="1" COLOR="#FFFFFF">&shy;</FONT><FONT SIZE="-2">&#64;</FONT><FONT SIZE="1" COLOR="#FFFFFF">&shy;</FONT><FONT SIZE="-2">cs.rit.edu.</FONT></small></p>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink