Skip to content

Commit

Permalink
spark.default.partitioner docs
Browse files Browse the repository at this point in the history
  • Loading branch information
markhamstra committed Jan 9, 2015
1 parent 466c8cf commit 8e01a62
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
11 changes: 11 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,17 @@ Apart from these, the following properties are also available, and may be useful
(<code>groupByKey</code>, <code>reduceByKey</code>, etc) when not set by user.
</td>
</tr>
<tr>
<td><code>spark.default.partitioner</code></td>
<td>hash</td>
<td>
Implementation to use for partitioning key-value data pairs. There are two implementations
available:<code>hash</code> and <code>byteswap</code>. Both are based on the <code>hashCode</code> of
the keys <code>mod</code> the number of partitions, but the <code>byteswap</code> partitioner also
applies <code>byteswap32</code> to the hash codes, which helps guarantee that all partitions are used
even when the hash codes are divisible by a factor of the number of partitions.
</td>
</tr>
<tr>
<td><code>spark.broadcast.factory</code></td>
<td>org.apache.spark.broadcast.<br />TorrentBroadcastFactory</td>
Expand Down
2 changes: 1 addition & 1 deletion mllib/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent</artifactId>
<version>1.1.1--csd-1-SNAPSHOT</version>
<version>1.1.1-csd-1-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down

0 comments on commit 8e01a62

Please sign in to comment.