big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,144 Commits 2 Branches 2 Tags 264 MiB
cde1f3af21
Commit Graph

26144 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Neil
2f49eec5dd
HDFS-16655. OIV: print out erasure coding policy name in oiv Delimited output (#4541). Contributed by Max Xie. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2022-07-25 17:39:25 +08:00
Gautham B A
8f83d9f56d
HDFS-16680. Skip libhdfspp Valgrind tests on Windows (#4611) 
* The CMake test libhdfs_mini_stress_valgrind
  requires Valgrind.
* This PR skips this test on Windows since
  Valgrind isn't available.
 2022-07-23 23:22:13 +05:30
Gautham B A
7de9b5ee27
HDFS-16467. Ensure Protobuf generated headers are included first (#4601) 
* This PR ensures that the Protobuf generated headers
  are always included first, even when these headers
  are included transitively.
* This problem is specific to Windows only.
 2022-07-23 23:20:15 +05:30
slfan1989
63db1a85e3
YARN-11203. Fix typo in hadoop-yarn-server-router module. (#4510). Contributed by fanshilun. 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-23 20:28:45 +05:30
xuzq
2c96357051
HDFS-15079. RBF: Namenode needs to use the actual client Id and callId when going through RBF proxy. (#4530) 2022-07-23 22:19:37 +08:00
slfan1989
5c84cb81ba
YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs (#4543) 2022-07-22 17:06:38 -07:00
Masatake Iwasaki
221eb2d68d Make upstream aware of 3.2.4 release. 
(cherry picked from commit 817b8fdd384a5c28f83bc257c389ceedd38070c5)
 2022-07-22 04:08:36 +00:00
Masatake Iwasaki
3cce41a1f6 Make upstream aware of 3.2.4 release. 
(cherry picked from commit e1637a57dfd41385dbce5de90620c48a45abb263)
 2022-07-22 02:27:19 +00:00
slfan1989
2f6916a313
HDFS-16605. Improve Code With Lambda in hadoop-hdfs-rbf moudle. (#4375) 2022-07-21 18:42:55 -07:00
wangzhaohui
08a940d5dd
HDFS-16640. RBF: Show datanode IP list when click DN histogram in Router (#4488) 2022-07-21 16:21:31 -07:00
slfan1989
838020ce3b
YARN-11160. Support getResourceProfiles, getResourceProfile API's for Federation (#4540) 2022-07-21 11:57:24 -07:00
Szilard Nemeth
f4b635c4dc YARN-11211. QueueMetrics leaks Configuration objects when validation API is called multiple times. Contributed by Andras Gyori 2022-07-21 14:20:34 +02:00
ashutoshpant
bac2219e3c
HADOOP-18330. S3AFileSystem removes Path when calling createS3Client (#4572) 
Adds a new parameter object in s3ClientCreationParameters that holds 
the full s3a path URI

Contributed by Ashutosh Pant
 2022-07-21 10:16:39 +01:00
Gautham B A
d07256a96d
HDFS-16667. Use malloc for buffer allocation in uriparser2 (#4576) 
* Windows doesn't support variables for specifying
  the array size.
* This PR uses malloc to fix this issue.
 2022-07-20 21:57:28 +05:30
Ashutosh Gupta
e664f81ce7
HADOOP-18333.Upgrade jetty version to 9.4.48.v20220622 (#4553) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-21 00:15:39 +08:00
Gautham B A
6415eb04e8
HDFS-16665. Fix duplicate sources for HDFS test (#4573) 
* The library target hdfspp_test_shim_static is
  built using the following sources, which
  causes duplicate symbols to be defined -
  - hdfs_shim.c
  - ${LIBHDFSPP_BINDING_C}/hdfs.cc
* ${LIBHDFSPP_BINDING_C}/hdfs.cc is redundant
  and removing this fixes the issue.
 2022-07-19 21:39:06 +05:30
Gautham B A
4fb799e6c5
HDFS-16464. Create only libhdfspp static libraries for Windows (#4571) 
* Added the appropriate CMake flags and
  commands to enable only statically
  linked libraries and executables to
  be built on Windows.
 2022-07-19 21:37:22 +05:30
Gautham B A
21b8952125
HDFS-16666. Pass CMake args for Windows in pom.xml (#4574) 
* This PR passes the necessary CMake args in the
  pom.xml needed for building HDFS native client
  on Windows.
* These arguments are exposed as maven options
  and can be passed from the command-line.
 2022-07-19 10:45:59 +05:30
Wei-Chiu Chuang
a55ace7bc0
HADOOP-18079. Upgrade Netty to 4.1.77. (#3977) 
Upgrade netty to address

CVE-2019-20444,
CVE-2019-20445
CVE-2022-24823

Contributed by Wei-Chiu Chuang
 2022-07-18 10:41:00 +01:00
PJ Fanning
34e548cb62
HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 
This downgrades jackson from the version switched to in
    HADOOP-18033 (2.13.0), to Jackson 2.12.7.
    This removes the dependency on javax.ws.rs-api,
    so avoiding runtime problems with applications using
    jersey-core v1 and/or jsr311-api.
    
    The 2.12.7 release still contains the fix for CVE-2020-36518.
    
    Contributed by PJ Fanning
 2022-07-17 21:37:54 +05:30
Gautham B A
440f4c2b28
HDFS-16654. Link OpenSSL lib for CMake deps check (#4538) 
* The check_c_source_compiles fails on Windows
  while linking with an "unable to resolve
  external symbol" error.
* This PR links OpenSSL lib for this check to
  fix this issue.
 2022-07-17 20:47:30 +05:30
xuzq
8774f17868
HADOOP-13144. Enhancing IPC client throughput via multiple connections per user (#4542) 2022-07-15 14:18:46 -07:00
RuinanGu
9376b65989
HDFS-16566 Erasure Coding: Recovery may causes excess replicas when busy DN exsits (#4252) 2022-07-16 04:52:12 +08:00
Murali Krishna
2835174a4c
HDFS-16652. Upgrade jquery datatable version references to v1.10.19 (#4562) 2022-07-14 18:27:07 +05:30
Samrat
84ce592a85
YARN-11198. clean up numa resources from statestore (#4546) 
* YARN-11198. clean up numa resources from levelDB

Co-authored-by: Deb <dbsamrat@3c22fba1b03f.ant.amazon.com>
 2022-07-14 11:07:48 +05:30
xuzq
6f9c4359ec
HDFS-16283. RBF: reducing the load of renewLease() RPC (#4524). Contributed by ZanderXu. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-14 07:26:40 +05:30
Ashutosh Gupta
f1bd4e117e
HADOOP-18336.Tag FSDataInputStream.getWrappedStream() @Public/@Stable (#4555) 
Contributed by: Ashutosh Gupta
 2022-07-13 12:56:56 +01:00
HerCath
4c4a940da2
HADOOP-18217. ExitUtil synchronized blocks reduced. #4255 
Reduce the ExitUtil synchronized block scopes so System.exit 
and Runtime.halt calls aren't within their boundaries,
so ExitUtil wrappers do not block each other.

Enlarged catches to all Throwables (not just Exceptions).

Contributed by Remi Catherinot
 2022-07-13 12:35:44 +01:00
Ashutosh Gupta
0ca4868aa2
HADOOP-18294. Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums (#4446) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-07-12 20:15:26 +09:00
Ashutosh Gupta
4e8c0b902e
MAPREDUCE-7201.Make Job History File Permissions configurable (#4507) 
* MAPREDUCE-7201.Make Job History File Permissions configurable

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-11 11:34:52 +05:30
lmccay
e11ba5930e
HADOOP-18074 - Partial/Incomplete groups list can be returned in LDAP… (#4503) 
* HADOOP-18074 - Partial/Incomplete groups list can be returned in LDAP groups lookup
 2022-07-11 01:03:44 -04:00
Akira Ajisaka
9b1d3579b4
Revert "MAPREDUCE-7388. Remove unused variable _eof in GzipCodec.cc (#4429)" 
This reverts commit fac895828f.
 2022-07-09 03:05:42 +09:00
cfg1234
fac895828f
MAPREDUCE-7388. Remove unused variable _eof in GzipCodec.cc (#4429) 
Co-authored-by: cWX456268 <chenfengge1@huawei.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-07-09 02:51:49 +09:00
Ayush Saxena
96f8e5b6f4
HADOOP-15789. DistCp does not clean staging folder if class extends DistCp. Contributed by Lawrence Andrews. (#4534) 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-08 17:04:20 +05:30
Gautham B A
8e39e35bea
HDFS-16466. Implement Linux permission flags on Windows (#4526) 
* HDFS-16466. Implement Linux permission flags on Windows

* statinfo.cc uses POSIX permission flags.
  These flags aren't available for Windows.
* This PR implements the equivalent flags
  on Windows to make this cross platform
  compatible.
 2022-07-08 09:29:13 +05:30
Jinhu Wu
3ec4b932c1
HADOOP-18313: AliyunOSSBlockOutputStream should not mark the temporary file for deletion (#4502) 
HADOOP-18313: AliyunOSSBlockOutputStream should not mark the temporary file for deletion. Contributed by wujinhu.
 2022-07-06 14:23:46 +08:00
Ashutosh Gupta
a432925f74
HADOOP-18321.Fix when to read an additional record from a BZip2 text file split (#4521) 
* HADOOP-18321.Fix when to read an additional record from a BZip2 text file split

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com> and Reviewed by Akira Ajisaka.
 2022-07-06 10:00:14 +05:30
slfan1989
161b1fac2e
YARN-11169. Support moveApplicationAcrossQueues, getQueueInfo API's for Federation. (#4464) 2022-07-05 11:24:29 -07:00
cxzl25
ea46f49b04
HDFS-16638. Add isDebugEnabled check for debug blockLogs in BlockManager. (#4480). Contributed by dzcxzl. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-04 02:36:29 +05:30
jianghuazhu
a333674785
HDFS-16647. Delete unused NameNode#FS_HDFS_IMPL_KEY. (#4525). Contributed by JiangHua Zhu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-03 01:47:02 +05:30
Ashutosh Gupta
9e206fe0b6
HADOOP-18297. Upgrade dependency-check-maven to 7.1.1 (#4449) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-03 04:04:55 +08:00
Ashutosh Gupta
151bb31c47
YARN-9403.GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table (#4516) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:59:28 +05:30
Ashutosh Gupta
57cbde9abf
YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping (#4515) 
* YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:58:56 +05:30
Tamas Domok
3cad632709 YARN-11202. Optimize ClientRMService.getApplications. Contributed by Tamas Domok. 
Change-Id: I55ddb46fd0e4cdb644747d6d43083215f10861b5
 2022-07-01 10:50:48 +02:00
9uapaw
2d133a54ac YARN-11204. Various MapReduce tests fail with NPE in AggregatedLogDeletionService.stopRMClient. Contributed by Szilard Nemeth. 2022-06-29 15:12:04 +02:00
slfan1989
073b8ea1d5
HADOOP-18284. Remove Unnecessary semicolon ';' (#4422). Contributed by fanshilun. 2022-06-29 15:20:41 +05:30
jianghuazhu
321a4844ad
HADOOP-18314. Add some description for PowerShellFencer. (#4505) 2022-06-28 19:06:39 -07:00
hchaverr
cf33164857
HDFS-16591. Setup JaasConfiguration in ZKCuratorManager when SASL is enabled 
Fixes #4447
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2022-06-28 16:44:02 -07:00
Samrat
7eefdf8642
YARN-11195. Adding document to enable numa (#4501) 
Contributed by Samrat Deb.
 2022-06-28 17:46:43 +05:30
Ashutosh Gupta
a177232ebc
YARN-9822.TimelineCollectorWebService#putEntities blocked when ATSV2 HBase is down (#4492) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-28 09:32:07 +05:30