big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,158 Commits 2 Branches 2 Tags 264 MiB
35a303dfbe
Commit Graph

1181 Commits

Author SHA1 Message Date
yliu
f4d96be6c6 HADOOP-12367. Move TestFileUtil's test resources to resources folder. (wang via yliu) 2015-09-01 16:20:56 +08:00
Andrew Wang
7ad3556ed3 HADOOP-12368. Mark ViewFileSystemBaseTest and ViewFsBaseTest as abstract. 2015-08-31 18:17:14 -07:00
Xiaoyu Yao
48774d0a45 HADOOP-12325. RPC Metrics : Add the ability track and log slow RPCs. Contributed by Anu Engineer 2015-08-24 14:31:24 -07:00
Zhe Zhang
6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Robert Kanter
1e06299df8 HADOOP-12317. Applications fail on NM restart on some linux distro because NM container recovery declares AM container as LOST (adhoot via rkanter) 2015-08-19 19:00:51 -07:00
Zhe Zhang
1d37a88121 HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su. 2015-08-13 10:04:45 -07:00
cnauroth
6cc8e38db5 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 2015-08-12 16:44:53 -07:00
Andrew Wang
1ea1a8334e HDFS-8887. Expose storage type and storage ID in BlockLocation. 2015-08-11 23:25:33 -07:00
Jian He
f59612edd7 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula 2015-08-05 16:12:45 -07:00
Colin Patrick Mccabe
6f83274afc HADOOP-12280. Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe) 2015-08-05 09:42:38 -07:00
Zhihai Xu
a628a6f126 HADOOP-12268. AbstractContractAppendTest#testRenameFileBeingAppended misses rename operation. Contributed by Zhihai Xu 2015-07-31 21:32:45 -07:00
Colin Patrick Mccabe
454da959c7 HADOOP-12183. Annotate the HTrace span created by FsShell with the command-line arguments passed by the user (Masatake Iwasaki via Colin P. McCabe) 2015-07-31 15:14:18 -07:00
Colin Patrick Mccabe
21d10ccc6e HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 2015-07-31 15:01:34 -07:00
Allen Wittenauer
a890a31529 HADOOP-10854. unit tests for the shell scripts (aw) 2015-07-31 14:34:48 -07:00
Jakob Homan
0712a8103f HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak. 2015-07-28 21:03:31 -07:00
Colin Patrick Mccabe
69b095730b HADOOP-12175. FsShell must load SpanReceierHost to support tracing (Masatake Iwasaki via Colin P. McCabe) 2015-07-28 12:56:15 -07:00
Robert Kanter
ee233ec95c HADOOP-12259. Utility to Dynamic port allocation (brahmareddy via rkanter) 2015-07-24 09:41:53 -07:00
Jakob Homan
ab3197c204 HADOOP-12009: Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. (J.Andreina via jghoman) 2015-07-23 17:46:13 -07:00
Andrew Wang
6736a1ab70 HADOOP-12189. Improve CallQueueManager#swapQueue to make queue elements drop nearly impossible. Contributed by Zhihai Xu. 2015-07-23 14:42:35 -07:00
Arpit Agarwal
adfa34ff99 HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula) 2015-07-23 10:13:04 -07:00
Jing Zhao
29495cb8f6 HADOOP-12060. Fix ByteBuffer usage for raw erasure coders. Contributed by Kai Zheng. 2015-07-20 10:15:14 -07:00
Steve Loughran
9141e1aa16 HADOOP-12209 Comparable type should be in FileStatus. (Yong Zhang via stevel) 2015-07-20 12:32:44 +01:00
Haohui Mai
d96bbe152c HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 2015-07-16 15:21:53 -07:00
Tsuyoshi Ozawa
90bda9c611 HADOOP-12240. Fix tests requiring native library to be skipped in non-native profile. Contributed by Masatake Iwasaki. 2015-07-16 16:04:55 +09:00
Chris Douglas
a431ed9075 HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl 2015-07-13 15:36:11 -07:00
Chris Douglas
1a0752d85a HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl 2015-07-09 17:48:43 -07:00
Karthik Kambatla
ac6048372a HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) 2015-07-09 09:56:40 -07:00
Akira Ajisaka
6b7692c468 HADOOP-12200. TestCryptoStreamsWithOpensslAesCtrCryptoCodec should be skipped in non-native profile. Contributed by Masatake Iwasaki. 2015-07-09 13:14:53 +09:00
Chris Douglas
559425dcb9 HADOOP-5732. Add SFTPFileSystem. Contributed by Ramtin Boustani and Inigo Goiri 2015-07-07 10:08:59 -07:00
Varun Vasudev
c40bdb56a7 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 2015-07-07 16:59:29 +05:30
Vinayakumar B
99c8c5839b HADOOP-12117. Potential NPE from Configuration#loadProperty with allowNullValueProperties set. (Contributed by zhihai xu) 2015-07-07 16:11:27 +05:30
Chris Douglas
47a69ec7a5 HADOOP-12185. NetworkTopology is not efficient adding/getting/removing nodes. Contributed by Inigo Goiri 2015-07-06 15:03:22 -07:00
cnauroth
ed1e3ce482 HADOOP-12045. Enable LocalFileSystem#setTimes to change atime. Contributed by Kazuho Fujii. 2015-07-06 13:40:15 -07:00
Vinayakumar B
233cab89ad HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times (Contributed by zhihai xu) 2015-07-06 15:39:43 +05:30
Lei Xu
532e38cb7f HADOOP-12164. Fix TestMove and TestFsShellReturnCode failed to get command name using reflection. (Lei Xu) 2015-06-30 17:41:59 -07:00
Robert Kanter
aaafa0b2ee HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter) 2015-06-30 16:49:27 -07:00
Andrew Wang
9ee7b6e6c4 HADOOP-12158. Improve error message in TestCryptoStreamsWithOpensslAesCtrCryptoCodec when OpenSSL is not installed. 2015-06-30 15:04:33 -07:00
Steve Loughran
076948d9a4 Revert "HADOOP-12009 Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. (J.Andreina via stevel)" 
This reverts commit 3dfa8161f9.
 2015-06-30 11:54:12 +01:00
Ming Ma
8e1bdc17d9 HADOOP-12107. long running apps may have a huge number of StatisticsData instances under FileSystem (Sangjin Lee via Ming Ma) 2015-06-29 14:37:38 -07:00
Vinayakumar B
c815344e2e HADOOP-12119. hadoop fs -expunge does not work for federated namespace (Contributed by J.Andreina) 2015-06-29 15:58:54 +05:30
Steve Loughran
3dfa8161f9 HADOOP-12009 Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. (J.Andreina via stevel) 2015-06-28 19:14:00 +01:00
Benoy Antony
a815cc157c HADOOP-12049. Control http authentication cookie persistence via configuration. Contributed by Huizhi Lu. 2015-06-24 15:59:39 -07:00
Aaron T. Myers
49dfad9429 HDFS-6440. Support more than 2 NameNodes. Contributed by Jesse Yates. 2015-06-23 17:26:11 -07:00
Colin Patrick Mccabe
d112d18324 HDFS-8582. Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P. McCabe) 2015-06-19 19:14:05 -07:00
Vinayakumar B
295d678be8 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 2015-06-18 14:39:00 +05:30
Akira Ajisaka
b578807b99 HADOOP-12095. org.apache.hadoop.fs.shell.TestCount fails. Contributed by Brahma Reddy Battula. 2015-06-16 13:38:03 -07:00
Vinod Kumar Vavilapalli
722aa1db1f HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search. Contributed by Patrick White. 2015-06-15 14:22:34 -07:00
Akira Ajisaka
e4489d97e5 HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki. 2015-06-12 07:25:15 -07:00
Vinayakumar B
48ca23def1 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 2015-06-10 11:05:58 +05:30
Arpit Agarwal
3107434031 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:39 -07:00
Colin Patrick Mccabe
fc2ed4a1f9 HADOOP-11347. RawLocalFileSystem#mkdir and create should honor umask (Varun Saxena via Colin P. McCabe) 2015-06-08 17:49:31 -07:00
Arpit Agarwal
84ba1a75b6 HADOOP-12054. RPC client should not retry for InvalidToken exceptions. (Contributed by Varun Saxena) 2015-06-08 15:45:23 -07:00
Kai Zheng
c41b02cc00 HADOOP-12065 Using more meaningful keys in EC schema. Contributed by Kai Zheng 2015-06-08 20:24:17 +08:00
Andrew Wang
bc11e158b1 HADOOP-12056. Use DirectoryStream in DiskChecker#checkDirs to detect errors when listing a directory. Contributed by Zhihai Xu. 2015-06-05 13:52:21 -07:00
Andrew Wang
2dbc40e608 HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey. 2015-06-05 13:11:01 -07:00
Gera Shegalov
9f6d67844d HADOOP-11924. Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init. (Tsuyoshi Ozawa via gera) 2015-06-04 11:38:28 -07:00
Kai Zheng
0799e1e4b6 HADOOP-12011 Allow to dump verbose information to ease debugging in raw erasure coders. Contributed by Kai Zheng 2015-06-02 22:05:16 +08:00
Andrew Wang
374ddd9f9e HADOOP-12043. Display warning if defaultFs is not set when running fs commands. Contributed by Eddy Xu. 2015-05-29 17:15:58 -07:00
cnauroth
860b8373c3 HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contributed by Larry McCay. 2015-05-28 15:01:42 -07:00
Colin Patrick Mccabe
246cefa089 HDFS-8429. Avoid stuck threads if there is an error in DomainSocketWatcher that stops the thread. (zhouyingchao via cmccabe) 2015-05-28 11:52:28 -07:00
Allen Wittenauer
5df1fadf87 HADOOP-7947. Validate XMLs if a relevant tool is available, when using scripts (Kengo Seki via aw) 2015-05-28 11:01:00 -07:00
Kai Zheng
4ad484883f HADOOP-11847 Enhance raw coder allowing to read least required inputs in decoding. Contributed by Kai Zheng 2015-05-26 12:07:16 -07:00
Vinayakumar B
5a391e1d25 HADOOP-12029. Remove chunkSize from ECSchema as its not required for coders (Contributed by Vinayakumar B) 2015-05-26 12:07:16 -07:00
Kai Zheng
b30e96bfb4 HDFS-8382 Remove chunkSize and initialize from erasure coder. Contributed by Kai Zheng 2015-05-26 12:07:15 -07:00
Kai Zheng
4ae32abdf4 HADOOP-12013 Generate fixed data to perform erasure coder test. Contributed by Kai Zheng 2015-05-26 12:07:14 -07:00
Zhe Zhang
343c0e76fc HADOOP-11938. Enhance ByteBuffer version encode/decode API of raw erasure coder. Contributed by Kai Zheng. 2015-05-26 12:02:31 -07:00
Zhe Zhang
b64f6745a4 HADOOP-11566. Add tests and fix for erasure coders to recover erased parity units. Contributed by Kai Zheng. 2015-05-26 12:02:31 -07:00
Zhe Zhang
09c3a375ba HADOOP-11920. Refactor some codes for erasure coders. Contributed by Kai Zheng. 2015-05-26 12:02:31 -07:00
Zhe Zhang
9c7a78c874 HADOOP-11921. Enhance tests for erasure coders. Contributed by Kai Zheng. 2015-05-26 12:02:31 -07:00
Zhe Zhang
e8df2581c3 HDFS-8156. Add/implement necessary APIs even we just have the system default schema. Contributed by Kai Zheng. 2015-05-26 11:59:53 -07:00
Tsz-Wo Nicholas Sze
909632dd90 HADOOP-11841. Remove unused ecschema-def.xml files. 2015-05-26 11:59:12 -07:00
Kai Zheng
17f7cdc047 HADOOP-11805 Better to rename some raw erasure coders. Contributed by Kai Zheng 2015-05-26 11:55:14 -07:00
Zhe Zhang
e54a74b566 HADOOP-11740. Combine erasure encoder and decoder interfaces. Contributed by Zhe Zhang. 2015-05-26 11:55:13 -07:00
Zhe Zhang
d9af36b9bd HADOOP-11664. Loading predefined EC schemas from configuration. Contributed by Kai Zheng. 2015-05-26 11:43:43 -07:00
Kai Zheng
df297245a7 HADOOP-11647. Reed-Solomon ErasureCoder. Contributed by Kai Zheng 2015-05-26 11:43:41 -07:00
Kai Zheng
c3bc083405 HADOOP-11706 Refine a little bit erasure coder API 2015-05-26 11:42:04 -07:00
Kai Zheng
68caf8728e Fixed a compiling issue introduced by HADOOP-11705. 2015-05-26 11:32:26 -07:00
drankye
292e367d07 HADOOP-11705. Make erasure coder configurable. Contributed by Kai Zheng 2015-05-26 11:32:25 -07:00
Vinayakumar B
8f89d7489d HADOOP-11646. Erasure Coder API for encoding and decoding of block group ( Contributed by Kai Zheng ) 2015-05-26 11:32:25 -07:00
drankye
c0945a8971 HADOOP-11643. Define EC schema API for ErasureCodec. Contributed by Kai Zheng 2015-05-26 11:32:24 -07:00
drankye
dae27f6dd1 HADOOP-11542. Raw Reed-Solomon coder in pure Java. Contributed by Kai Zheng 2015-05-26 11:07:15 -07:00
Kai Zheng
b29f3bde4d HADOOP-11541. Raw XOR coder 2015-05-26 11:03:35 -07:00
Akira Ajisaka
56996a685e HADOOP-8751. NPE in Token.toString() when Token is constructed using null identifier. Contributed by kanaka kumar avvaru. 2015-05-26 16:16:21 +09:00
Arun Suresh
fd3cb533d2 HADOOP-11973. Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs. (Gregory Chanan via asuresh) 2015-05-19 11:35:57 -07:00
Vinayakumar B
d4a2830b63 HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey) 2015-05-19 14:41:05 +05:30
Akira Ajisaka
3b50dcdce4 HADOOP-10971. Add -C flag to make hadoop fs -ls print filenames only. Contributed by Kengo Seki. 2015-05-19 17:25:27 +09:00
Arpit Agarwal
a2190bf15d HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:36:29 -07:00
Akira Ajisaka
a46506d99c HADOOP-10582. Fix the test case for copying to non-existent dir in TestFsShellCopy. Contributed by Kousuke Saruta. 2015-05-18 16:31:41 +09:00
cnauroth
09fe16f166 HADOOP-11713. ViewFileSystem should support snapshot methods. Contributed by Rakesh R. 2015-05-14 14:55:58 -07:00
Robert Kanter
315074bb08 HADOOP-9737. JarFinder#getJar should delete the jar file upon destruction of the JVM (jbonofre via rkanter) 2015-05-08 16:05:04 -07:00
Robert (Bobby) Evans
cc17c83585 HADOOP-6842. "hadoop fs -text" does not give a useful text representation of MapWritable objects 2015-05-08 11:57:14 -05:00
Akira Ajisaka
305e4733d6 HADOOP-10387. Misspelling of threshold in log4j.properties for tests in hadoop-common-project. Contributed by Brahma Reddy Battula. 2015-05-07 16:53:57 +09:00
Xiaoyu Yao
7a26d174af HDFS-8310. Fix TestCLI.testAll 'help: help for find' on Windows. (Kiran Kumar M R via Xiaoyu Yao) 2015-05-06 10:24:46 -07:00
Colin Patrick Mccabe
90b3845648 HADOOP-11912. Extra configuration key used in TraceUtils should respect prefix (Masatake Iwasaki via Colin P. McCabe) 2015-05-05 17:40:31 -07:00
Robert Kanter
9809a16d3c MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:43:28 -07:00
Andrew Wang
05adc76ace HADOOP-11120. hadoop fs -rmr gives wrong advice. Contributed by Juliet Houghland. 2015-05-05 08:37:37 -07:00
Tsuyoshi Ozawa
338e88a19e HADOOP-11916. TestStringUtils#testLowerAndUpperStrings failed on MAC due to a JVM bug. Contributed by Ming Ma. 2015-05-05 12:39:24 +09:00
Gera Shegalov
f343f8657e HADOOP-11491. HarFs incorrectly declared as requiring an authority. (Brahma Reddy Battula via gera) 2015-05-01 18:18:55 -07:00
Colin Patrick Mccabe
b82567d455 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 2015-05-01 11:19:40 -07:00
Jonathan Eagles
2d7363b273 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 2015-05-01 13:11:50 -05:00