big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-7122. Use of ThreadLocal<Random> results in poor block placement. (wang)

Browse Source
This commit is contained in:
Andrew Wang 2014-09-29 14:50:28 -07:00
parent e10eeaabce
commit c6c3247dc0
2 changed files with 7 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetworkTopology.java
View File

@ -673,27 +673,11 @@ protected boolean isSameParents(Node node1, Node node2) {
return node1.getParent()==node2.getParent();
}
private static final ThreadLocal<Random> r = new ThreadLocal<Random>();
/**
* Getter for thread-local Random, which provides better performance than
* a shared Random (even though Random is thread-safe).
*
* @return Thread-local Random.
*/
protected Random getRandom() {
Random rand = r.get();
if (rand == null) {
rand = new Random();
r.set(rand);
}
return rand;
}
private static final Random r = new Random();
@VisibleForTesting
void setRandomSeed(long seed) {
Random rand = getRandom();
rand.setSeed(seed);
r.setSeed(seed);
}
/** randomly choose one node from <i>scope</i>
@ -745,7 +729,7 @@ private Node chooseRandom(String scope, String excludedScope){
"Failed to find datanode (scope=\"" + String.valueOf(scope) +
"\" excludedScope=\"" + String.valueOf(excludedScope) + "\").");
}
int leaveIndex = getRandom().nextInt(numOfDatanodes);
int leaveIndex = r.nextInt(numOfDatanodes);
return innerNode.getLeaf(leaveIndex, node);
}
@ -918,11 +902,10 @@ public void sortByDistance(Node reader, Node[] nodes, int activeLen) {
list.add(node);
}
Random rand = getRandom();
int idx = 0;
for (List<Node> list: tree.values()) {
if (list != null) {
Collections.shuffle(list, rand);
Collections.shuffle(list, r);
for (Node n: list) {
nodes[idx] = n;
idx++;

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -532,6 +532,9 @@ Release 2.6.0 - UNRELEASED
HDFS-6865. Byte array native checksumming on client side
(James Thomas via todd)
HDFS-7122. Use of ThreadLocal<Random> results in poor block placement.
(wang)
BUG FIXES
HDFS-6823. dfs.web.authentication.kerberos.principal shows up in logs for