big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
18,238 Commits 2 Branches 2 Tags 264 MiB
ad10cbd91b
Commit Graph

7076 Commits

Author SHA1 Message Date
James
cbb3e9fe58 HDFS-10740: libhdfs++: Implement recursive directory generator. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
2a8edd4e52 HDFS-9271: libhdfs++: Implement basic NN operations. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
b1ed72e098 HDFS-10739: libhdfs++: In RpcEngine replace vector with deque for pending requests. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
649aff11fe HDFS-10672: libhdfs++: reorder directories in src/main/libhdfspp/examples, and add C++ version of cat tool. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
4cb0dad5e5 HDFS-10441: libhdfs++: HA namenode support. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James
d75c835696 Revert HDFS-10543 patch due to correctness issues it caused. Done by James Clampffer 2018-03-22 17:19:47 -04:00
James
a586ccbcfa HDFS-9890: libhdfs++: Add test suite to simulate network issues. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James
a23e6b422b HDFS-10578. libhdfs++: Silence compile warnings from URI parser. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
a0c7aee3b6 HDFS-10543: libhdfs++: hdfsRead stops at block boundary. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
Bob Hansen
d4c3cfbf47 HDFS-10524: libhdfs++: Implement chmod and chown. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
88c5768f99 HDFS-10515: libhdfs++: Implement mkdirs, rmdir, rename, and remove 2018-03-22 17:19:46 -04:00
Bob Hansen
193314dc34 HDFS-10511: libhdfs++: make error returning mechanism consistent across all hdfs operations. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
2616fe2025 HDFS-10526: libhdfs++: Add connect timeouts to async_connect calls. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
de6fce7817 HDFS-10527: libhdfs++: hdfsGetBlockLocations doesn't null terminate ip address strings. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
Bob Hansen
9ac66ccaa3 HDFS-10494: libhdfs++: Implement snapshot operations and GetFsStats. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
James
d64bbf983e Revert HDFS-9890. Needs some more work to reflect the current status of HDFS-8707. Done by James Clampffer 2018-03-22 17:19:46 -04:00
James
11b68ecebe HDFS-9890. libhdfs++: Add test suite to simulate network issues. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
Bob Hansen
50aa4ea776 HDFS-10454: libhdfspp: Move NameNodeOp to a separate file. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
0496bc5464 HDFS-10464: libhdfs++: Implement GetPathInfo and ListDirectory. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
18f4d2f42e HDFS-10465: libhdfs++: Implement GetBlockLocations. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
f1f0b8f0f8 HDFS-10366: libhdfs++: Add SASL authentication. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
93382381f6 HDFS-10188. libhdfs++: Implement debug allocators. Contributed by Xiaowei Zhu 2018-03-22 17:19:46 -04:00
James
b9eeecc477 HDFS-10332. hdfs-native-client fails to build with CMake 2.8.11 or earlier. Contributed by Tibor Kiss 2018-03-22 17:19:46 -04:00
James
cc51f2ce58 HDFS-10310. hdfsConnect hangs when given bad host or port. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
e542db66ee HDFS-10311: DatanodeConnection::Cancel should not delete the underlying socket. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
1cbfd6f962 libhdfs++: File length doesn't always count the last block if it's being written to. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
James
60c3437267 HDFS-10247: libhdfs++: Datanode protocol version mismatch fix. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
ed77d8d5df HDFS-10231: libhdfs++: Fix race conditions in RPC layer. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
015d93a715 HDFS-10222. libhdfs++: Shutdown sockets to avoid 'Connection reset by peer'. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
427edae365 HDFS-9616. libhdfs++: Add runtime hooks to allow a client application to add low level monitoring and tests. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
0f1a278dd5 HDFS-9118: libhdfs++ Add logging system. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
8f4a66ab8f HDFS-9556: libhdfs++: pull Options from default configs by default. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
Bob Hansen
f25bff50bf HDFS-9932: added uriparser2 library to HDFS-8707. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
Bob Hansen
69ee2e6a7e HDFS-9792: libhdfs++: EACCES not setting errno correctly. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
6a96f978eb HDFS-9699: libhdfs++: Add appropriate catch blocks for asio operations that throw. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
1600e6bba8 HDFS-9791: libhdfs+_+: ConfigurationLoader throws parse_exception on invalid input. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
2cd7ab5e94 HDFS-9753. libhdfs++: Client fails to pass TokenProto from LocatedBlockProto to server when reading a block. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
Bob Hansen
22ae319360 HDFS-9737: libhdfs++: Create examples of consuming libhdfs++ (pt 2). Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
f591b95e08 HDFS-9749: libhdfs++: RPC engine will attempt to close an asio socket before it's been opened. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
288a9a9ee7 HDFS-9741: libhdfs++: GetLastError not returning meaningful messages after some failures. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
d807773752 HDFS-9738: libhdfs++: Implement simple authentication. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
71b4f2ea25 HDFS-9737: libhdfs++: Create examples of consuming libhdfs++. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
7e946c7e30 HDFS-9712. libhdfs++: Reimplement Status object as a normal struct 2018-03-22 17:19:46 -04:00
Bob Hansen
ad23cf1490 HDFS-9679: Fix inconsistencies with libhdfs C API. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
166b3d49df HDFS-9643. libhdfs++: Support async cancellation of read operations. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
5cd2a1766c HDFS-9687. libhdfs++: potential segfault after teardown. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
1017ccabeb HDFS-9628. libhdfs++: implement builder apis from C bindings. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
dc335474a6 HDFS-9636. libhdfs++: for consistency, include files should be in hdfspp. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
11671c2bf6 HDFS-9627. libhdfs++: Add mechanism to retrieve human readable error messages through the C API. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
e376c022f4 HDFS-9609. libhdfs++: Allow seek to EOF. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
7d8452040d HDFS-9325. libhdfs++ Allow the location of hadoop source tree resources to be passed to CMake during a build. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
5dc2da1e6f HDFS-9487. libhdfs++ Enable builds with no compiler optimizations. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
e18db92398 HDFS-9523. libhdfs++: failure to connect to ipv6 host causes CI unit tests to fail. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
31d28e3105 HDFS-9524. libhdfs++ deadlocks in Filesystem::New if NN connection fails. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
dba377bfcc HDFS-9448. Enable valgrind for libhdfspp unit tests. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
6c80d02a03 HDFS-9538. libhdfs: load configuration from files. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
567c3ed44f HDFS-9537. libhdfs++ implement HDFSConfiguration class. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
6c33f89aea HDFS-9417. Clean up the RAT warnings in the HDFS-8707 branch. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
7cd98323ec HDFS-9489. Enable CI infrastructure to use libhdfs++ hdfsRead. Contributed by Stephen Walkauskas. 2018-03-22 17:19:46 -04:00
James
d6a04cdbdd HDFS-9460. Suppress warnings from third-party libraries. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
c59bd00a56 HDFS-9486. Fix valgrind failures when using more than 1 io_service worker thread. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
58f2c7183e HDFS-9228. libhdfs++ should respect NN retry configuration settings. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
eb70a64362 HDFS-9504. Initialize BadNodeTracker in FileSystemImpl constructor. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
8dac1094d8 HDFS-9497. move lib/proto/cpp_helpers to third-party since it won't have an ASF license. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
d7ecf396c9 HDFS-9144. Refactoring libhdfs++ into stateful/ephemeral objects. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
5ba8e8ab7b Revert HDFS-9448. 2018-03-22 17:19:46 -04:00
James
9a0a34bbfa HDFS-9448. Enable valgrind for libhdfspp unit tests. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
James
584c2a204d HDFS-9452. libhdfs++ Fix memory stomp in OpenFileForRead. Contributed by James Clampffer 2018-03-22 17:19:45 -04:00
James
87362b1c17 HDFS-9117. Config file reader / options classes for libhdfs++. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
James
a38703fdfc HDFS-9359. Test libhdfs++ with existing libhdfs tests. Contributed by Stephen Walkauskas. 2018-03-22 17:19:45 -04:00
James
64f0e78490 HDFS-9368. Implement reads with implicit offset state in libhdfs++. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
6f44d92071 HDFS-9103. Retry reads on DN failure. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
7dd29e548c HDFS-9408. Build both static and dynamic libraries for libhdfspp. Contributed by Stephen Walkauskas. 2018-03-22 17:19:45 -04:00
Haohui Mai
af9bc25ac5 HDFS-9419. Import the optional library into libhdfs++. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
Haohui Mai
a0b8f56317 HDFS-9328. Formalize coding standards for libhdfs++. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
a59714dcdb HDFS-9320. libhdfspp should use sizeof(int32_t) instead of sizeof(int) when parsing data. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
5f1d0957b2 HDFS-9340. libhdfspp fails to compile after HDFS-9207. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
dbd1d0c75b HDFS-8766. Implement a libhdfs(3) compatible API. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
422a6e4ed5 HDFS-9288. Import RapidXML 1.13 for libhdfspp. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
Haohui Mai
a14d1f741f HDFS-9265. InputStreamImpl should hold a shared_ptr of the BlockReader. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
08794423e8 HDFS-9207. Move the implementation to the hdfs-native-client module. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
9e929a7a0d HDFS-9095. RPC client should fail gracefully when the connection is timed out or reset. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
06326b37c9 HDFS-9108. InputStreamImpl::ReadBlockContinuation stores wrong pointers of buffers. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
6294141aeb HDFS-9116. Suppress false positives from Valgrind on uninitialized variables in tests. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
9480c116ac HDFS-9093. Initialize protobuf fields in RemoteBlockReaderTest. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
a559ef0243 HDFS-9025. Fix compilation issues on arch linux. Contributed by Owen O'Malley. 2018-03-22 17:19:45 -04:00
Haohui Mai
bc47acc9d9 HDFS-8952. InputStream.PositionRead() should be aware of available DNs. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
1efb677976 HDFS-8774. Implement FileSystem and InputStream API for libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
bccc640648 HDFS-8775. SASL support for data transfer protocol in libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
26d9f6cee3 HDFS-8788. Implement unit tests for remote block reader in libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
1bec75a13c HDFS-8764. Generate Hadoop RPC stubs from protobuf definitions. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
65b864f341 HDFS-8759. Implement remote block reader in libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
a6b2fb64c4 HDFS-8758. Implement the continuation library in libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
63eee296c7 HDFS-8745. Use Doxygen to generate documents for libhdfspp. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
40a1f3631d HDFS-8737. Initial implementation of a Hadoop RPC v9 client. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
95b479b8c1 HDFS-8725. Use std::chrono to implement the timer in the asio library. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
eb6c865516 HDFS-8724. Import third_party libraries into the repository. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Haohui Mai
d61e844c20 HDFS-8723. Integrate the build infrastructure with hdfs-client. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Inigo Goiri
e196d158a2 HDFS-12792. RBF: Test Router-based federation using HDFSContract. Contributed by Inigo Goiri. 2018-03-22 13:32:57 -07:00
Wei-Chiu Chuang
f738d75a86 HDFS-11900. Hedged reads thread pool creation not synchronized. Contributed by John Zhuge. 2018-03-22 11:29:31 -07:00