big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,350 Commits 2 Branches 2 Tags 264 MiB
8b4b3d6fe6
Commit Graph

24350 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-09-25 10:19:30 +09:00
Vivek Ratnavel Subramanian
a3463812ef HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse sometimes fails with out of memory error (#1509) 2019-09-24 16:28:25 -07:00
Bharat Viswanadham
6917754ba7
HDDS-2172.Ozone shell should remove description about REST protocol support. Contributed by Siddharth Wagle. 2019-09-24 16:06:10 -07:00
Jonathan Hung
c2731d4b63 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 2019-09-24 13:51:32 -07:00
Ayush Saxena
66400c1cbb HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena. 2019-09-25 01:31:15 +05:30
Ayush Saxena
eb96a3093e HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:16:30 +05:30
Anu Engineer
f16cf877e5
HDDS-2170. Add Object IDs and Update ID to Volume Object (#1510) 2019-09-24 11:01:05 -07:00
Inigo Goiri
afa1006a53 HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor. 2019-09-24 09:58:42 -07:00
Inigo Goiri
816d3cb908 HDFS-14837. Review of Block.java. Contributed by David Mollitor. 2019-09-24 09:54:09 -07:00
Wei-Chiu Chuang
43203b466d HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun. 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-24 09:38:58 -07:00
Márton Elek
91f50b98ca
HDDS-2167. Hadoop31-mr acceptance test is failing due to the shading 
Closes #1507
 2019-09-24 17:52:36 +02:00
Mukund Thakur
51c64b357d
HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running 
This uses the length of the file known at the start of the copy to determine the amount of data to copy.

* If a file is appended to during the copy, the original bytes are copied.
* If a file is truncated during a copy, or the attempt to read the data fails with a truncated stream,
  distcp will now fail. Until now these failures were not detected.

Contributed by Mukund Thakur.

Change-Id: I576a49d951fa48d37a45a7e4c82c47488aa8e884
 2019-09-24 11:23:24 +01:00
avijayanhwx
8f1a135118 HDDS-2081. Fix TestRatisPipelineProvider#testCreatePipelinesDnExclude. Contributed by Aravindan Vijayan. (#1506) 2019-09-24 12:59:51 +05:30
Duo Zhang
e8e7d7b4cd
HADOOP-16561. [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes (#1500) 
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
 2019-09-24 08:52:12 +08:00
Duo Zhang
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496) 
Signed-off-by:  Vinayakumar B <vinayakumarb@apache.org>
 2019-09-24 08:50:36 +08:00
Hanisha Koneru
0a716bd3a5 HDDS-2159. Fix Race condition in ProfileServlet#pid. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-23 17:21:19 -07:00
Xiaoyu Yao
6cbe5d3809 HDDS-2160. Add acceptance test for ozonesecure-mr compose. Contributed by Xiaoyu Yao. (#1490) 2019-09-23 13:28:38 -07:00
dchitlangia
3fd3d746fc HDDS-2161. Create RepeatedKeyInfo structure to be saved in deletedTable 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-23 13:15:47 -07:00
Jonathan Hung
3d78b1223d YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 2019-09-23 13:12:16 -07:00
Ayush Saxena
c30e495557 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-23 21:22:50 +05:30
Siddharth Seth
2b5fc95851
HADOOP-16591 Fix S3A ITest*MRjob failures. 
Contributed by Siddharth Seth.

Change-Id: I7f08201c9f7c0551514049389b5b398a84855191
 2019-09-23 14:58:03 +01:00
Gabor Bota
aa664d7259
HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302). Contributed by Gabor Bota. 
Change-Id: I2f637865c871e400b95fe7ddaa24bf99fa192023
 2019-09-23 13:29:01 +02:00
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B. 2019-09-23 12:37:47 +05:30
Rohith Sharma K S
4c0a7a9e13 Make upstream aware of 3.2.1 release. 2019-09-23 06:20:54 +00:00
Rakesh Radhakrishnan
659c88801d HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He. 2019-09-22 22:02:54 +05:30
Hanisha Koneru
a94aa1ff46 HDDS-2150. Update dependency versions to avoid security vulnerabilities. (#1472) 2019-09-20 23:21:53 -07:00
Siddharth Seth
e02b1023c2
HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB (#1332) 2019-09-21 11:50:45 +05:30
Vivek Ratnavel Subramanian
dbdc612b3b HDDS-2163. Add 'Replication factor' to the output of list keys (#1493) 2019-09-20 22:59:45 -07:00
Vinayakumar B
efed4450bf
HADOOP-16589. [pb-upgrade] Update docker image to make 3.7.1 protoc as default (#1482). Contributed by Vinayakumar B. 2019-09-21 10:07:26 +05:30
Ayush Saxena
aa938662f9 HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena. 2019-09-21 09:53:13 +05:30
Siyao Meng
d7d6ec8fae HDDS-2128. Make ozone sh command work with OM HA service ids (#1445) 2019-09-20 14:22:55 -07:00
Doroszlai, Attila
53637301d1 HDDS-2157. checkstyle: print filenames relative to project root (#1485) 2019-09-20 14:05:35 -07:00
Inigo Goiri
3f223bebfa HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun. 2019-09-20 11:45:01 -07:00
Nanda kumar
b3173e1f58 HDDS-2001. Update Ratis version to 0.4.0. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-20 10:03:46 -07:00
Anu Engineer
5553887d95 HDDS-1949. Missing or error-prone test cleanup. 
Contributed by Doroszlai, Attila.
 2019-09-20 09:57:08 -07:00
Yuval Degani
dd0834696a HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
In the existing implementation, the ValueQueue::getAtMost() method will only trigger a refill on a key queue if it has gone empty, instead of triggering a refill when it has gone below the watermark. Revise the test suite to correctly verify this behavior.
 2019-09-20 09:55:48 -07:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432) 
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
 2019-09-20 16:08:30 +05:30
Xiaoyu Yao
d072d3304c HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-19 22:43:00 -07:00
Vivek Ratnavel Subramanian
9be448b336 HDDS-2156. Fix alignment issues in HDDS doc pages 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-19 22:32:19 -07:00
Takanobu Asanuma
298f07abce HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 2019-09-20 13:55:34 +09:00
Vivek Ratnavel Subramanian
b7ae8a96cd HDDS-2101. Ozone filesystem provider doesn't exist (#1473) 2019-09-19 16:28:29 -07:00
Elek, Márton
126ef77a81 HDDS-2154. Fix Checkstyle issues (#1475) 2019-09-19 11:30:33 -07:00
Elek, Márton
da1c67e0c2 HDDS-1054. List Multipart uploads in a bucket (#1277) 2019-09-19 11:06:02 -07:00
Inigo Goiri
a79f286c6f HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang. 2019-09-19 11:00:10 -07:00
Doroszlai, Attila
1ada99b0bd HDDS-2151. Ozone client logs the entire request payload at DEBUG level (#1477) 2019-09-19 10:58:33 -07:00
Márton Elek
f260b5aa5b HDDS-2127. Detailed Tools doc not reachable 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-19 09:50:21 -07:00
Márton Elek
f6d884cd11 HDDS-2110. Arbitrary file can be downloaded with the help of ProfilerServlet 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-19 09:41:55 -07:00
Malcolm Taylor
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM. 
Contributed by Malcolm Taylor.

Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
 2019-09-19 16:00:05 +01:00
Kihwal Lee
d4205dce17 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 2019-09-19 08:23:35 -05:00
Doroszlai, Attila
c9900a01d1
HDDS-2141. Missing total number of operations 
Closes #1462
 2019-09-19 14:11:44 +02:00