big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,103 Commits 2 Branches 2 Tags 264 MiB
3a52bfc5db
Commit Graph

25103 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mingliang Liu
6e5692e7e2
Revert "HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)" 
This reverts commit 4a26a61ecd.
 2021-06-11 00:34:24 -07:00
Viraj Jasani
4ef27a596f
HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087) 2021-06-11 12:15:52 +09:00
Akira Ajisaka
9a17f97133
YARN-10803. [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException. (#3068) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-06-10 14:29:47 +09:00
Takanobu Asanuma
b3c7254563
HADOOP-17750. Fix asf license errors in newly added files by HADOOP-17727. (#3083) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
 2021-06-09 15:32:07 +09:00
Viraj Jasani
81d7069316
YARN-10805. Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project (#3075) 2021-06-09 15:15:47 +09:00
Viraj Jasani
eefa664fea
HDFS-16054. Replace Guava Lists usage by Hadoop's own Lists in hadoop-hdfs-project (#3073) 2021-06-09 15:12:48 +09:00
snehavarma
4c039fafeb
HADOOP-17715 ABFS: Append blob tests with non HNS accounts fail (#3028) 2021-06-09 10:54:10 +05:30
Viraj Jasani
ca27744770
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-06-08 19:37:23 -07:00
Ayush Saxena
a1a3184171
HDFS-15916. Addendum. DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff. (#3056) 2021-06-08 19:34:45 -07:00
Viraj Jasani
00d372b663
HADOOP-17725. Improve error message for token providers in ABFS (#3041) 
Contributed by Viraj Jasani.
 2021-06-08 22:03:03 +01:00
Steve Loughran
762a83e044
HADOOP-17631. Configuration ${env.VAR:-FALLBACK} to eval FALLBACK when restrictSystemProps=true (#2977) 
Contributed by Steve Loughran.
 2021-06-08 21:56:40 +01:00
Ahmed Hussein
a2a0283c7b
HDFS-16042. DatanodeAdminMonitor scan should be delay based (#3058) 2021-06-08 11:09:31 -05:00
Prabhu Josephraj
9445abb500 YARN-10792. Set Completed AppAttempt LogsLink to Log Server URL. Contributed by Abhinaba Sarkar 2021-06-08 20:37:40 +05:30
zhuqi-lucas
ec16b1d3b9 YARN-10807. Parents node labels are incorrectly added to child queues in weight mode. Contributed by Benjamin Teke. 2021-06-08 21:03:43 +08:00
litao
c748fce17a
HDFS-16048. RBF: Print network topology on the router web (#3062) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-06-08 16:14:06 +09:00
Gautham B A
1c0b2edde9
HADOOP-17727. Modularize docker images (#3043) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-06-07 20:11:08 -07:00
Akira Ajisaka
57a3613e5d
HDFS-16050. Some dynamometer tests fail. (#3079) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-06-07 14:37:30 +09:00
Viraj Jasani
f4b24c68e7
HADOOP-17743. Replace Guava Lists usage by Hadoop's own Lists in hadoop-common, hadoop-tools and cloud-storage projects (#3072) 2021-06-07 13:24:09 +09:00
Viraj Jasani
207c92753f
MAPREDUCE-7350. Replace Guava Lists usage by Hadoop's own Lists in hadoop-mapreduce-project (#3074) 2021-06-07 11:51:29 +09:00
Hideyuki Furue
9c7b8cf54e
Fix container-executor 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-06-06 07:05:29 +09:00
July
4a26a61ecd
HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042) 
Contributed by kaifeiYi (yikf).

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-06-04 13:36:09 -07:00
Szilard Nemeth
200eec8f2e YARN-10796. Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%. Contributed by Peter Bacsko 2021-06-03 13:27:36 +02:00
Viraj Jasani
59fc4061cb
HADOOP-17152. Provide Hadoop's own Lists utility to reduce dependency on Guava (#3061) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-06-03 18:56:00 +09:00
sumangala-patki
76d92eb2a2
HADOOP-17596. ABFS: Change default Readahead Queue Depth from num(processors) to const (#2795) 
. Contributed by Sumangala Patki.
 2021-06-03 14:26:15 +05:30
Szilard Nemeth
2707f69251 YARN-10787. Queue submit ACL check is wrong when CS queue is ambiguous. Contributed by Gergely Pollak 2021-06-01 16:01:39 +02:00
Gergely Pollak
e9339aa376 YARN-10797. Logging parameter issues in scheduler package. Contributed by Szilard Nemeth 2021-06-01 15:57:22 +02:00
huhaiyang
b38b00e528
HDFS-15998. Fix NullPointException In listOpenFiles (#3036) 
Co-authored-by: huhaiyang <huhaiyang@didichuxing.com>
 2021-06-01 00:26:47 -07:00
Wei-Chiu Chuang
9a0a808338
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-31 20:34:48 -07:00
Ayush Saxena
b8ab19373d Revert "HDFS-15982. Deleted data using HTTP API should be saved to the trash (#2927)" 
This reverts commit 041488e8f5.
 2021-06-01 08:38:13 +05:30
Ayush Saxena
bce14e746b HDFS-16051. Misspelt words in DataXceiver.java line 881 and line 885. Contributed by Ning Sheng. 2021-06-01 08:34:56 +05:30
Steve Loughran
bd7b97ba13
HADOOP-17735. Upgrade AWS SDK to 1.11.1026 (#3057) 2021-05-31 21:43:26 +01:00
Jinglun
e90c41af34 HDFS-15973. RBF: Add permission check before doing router federation rename. Contributed by Jinglun. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-05-30 19:28:40 +08:00
Jinglun
2b11e160e2 Revert "HDFS-15973." because the user.name and user.email message is not correct. 
This reverts commit b7f69467c1.
 2021-05-30 19:27:10 +08:00
李经纶
b7f69467c1 HDFS-15973. RBF: Add permission check before doing router federation rename. Contributed by Jinglun. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-05-30 18:50:03 +08:00
Akira Ajisaka
9983ab8a99
HDFS-16046. TestBalancerProcedureScheduler and TestDistCpProcedure timeouts. (#3060) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-05-29 23:04:48 +09:00
zhengchenyu
d5ad181684
MAPREDUCE-7287. Distcp will delete exists file , If we use "-delete and -update" options and distcp file. (#2852) 
Contributed by zhengchenyu
 2021-05-28 20:21:37 +01:00
Konstantin V Shvachko
8ce30f51f9 HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 2021-05-27 18:47:20 -07:00
Takanobu Asanuma
53ff2dfed3
Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 0774116756.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-27 13:14:17 +09:00
Konstantin V Shvachko
1abd03d68f HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 2021-05-26 14:59:50 -07:00
Gautham B A
bcaeb1ac8c
Revert "Revert "HDFS-15971. Make mkstemp cross platform (#2898)"" (#3044) 2021-05-26 14:12:53 -07:00
Hui Fei
98a74e2351
HDFS-16041. TestErasureCodingCLI fails (#3052) 2021-05-26 20:12:17 +09:00
Viraj Jasani
986d0a4f1d
HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom (#3049) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-05-26 17:14:31 +09:00
Akira Ajisaka
8a489ce78e
MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053) 
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-05-26 15:47:56 +09:00
Narges Shadab
3fdeb7435a
HDFS-16031. Possible Resource Leak in org.apache.hadoop.hdfs.server.aliasmap#InMemoryAliasMap (#3027) 2021-05-26 14:43:41 +09:00
zhuxiangyi
e18d806212
HDFS-16024: RBF: Rename data to the Trash should be based on src loca… (#3009) 2021-05-26 13:41:49 +08:00
Jim Brennan
c4581827a9 YARN-10691. DominantResourceCalculator isInvalidDivisor should consider only countable resource types. Contributed by Bilwa S T. 2021-05-25 18:07:28 +00:00
Szilard Nemeth
b86a6eb871 YARN-10782. Extend /scheduler endpoint with template properties. Contributed by Andras Gyori 2021-05-25 18:27:53 +02:00
Szilard Nemeth
2541efa496 YARN-10783. Allow definition of auto queue template properties in root. Contributed by Andras Gyori 2021-05-25 13:55:59 +02:00
Akira Ajisaka
1cbcde04f3
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja. 2021-05-25 18:28:09 +09:00
Steve Loughran
832a3c6a89
HADOOP-17511. Add audit/telemetry logging to S3A connector (#2807) 
The S3A connector supports
"an auditor", a plugin which is invoked
at the start of every filesystem API call,
and whose issued "audit span" provides a context
for all REST operations against the S3 object store.

The standard auditor sets the HTTP Referrer header
on the requests with information about the API call,
such as process ID, operation name, path,
and even job ID.

If the S3 bucket is configured to log requests, this
information will be preserved there and so can be used
to analyze and troubleshoot storage IO.

Contributed by Steve Loughran.
 2021-05-25 10:25:41 +01:00