增加Router详解
This commit is contained in:
parent
bb49a67830
commit
adba43d154
@ -9,7 +9,9 @@
|
|||||||
|
|
||||||
# 子模块
|
# 子模块
|
||||||
|
|
||||||
## State Store模块
|
## 一、State Store模块
|
||||||
|
|
||||||
|
当前模块主要用于保存Router状态信息,提供了文件系统(HDFS、本地)、Mysql、Zookeeper。个人觉得一般使用Zookeeper比较合理。
|
||||||
|
|
||||||
### 初始化
|
### 初始化
|
||||||
|
|
||||||
@ -149,7 +151,7 @@ MySQLStateStoreHikariDataSourceConnectionFactory(Configuration conf) {
|
|||||||
|
|
||||||
##### initRecordStorage
|
##### initRecordStorage
|
||||||
|
|
||||||
zai StateStoreMySQLImpl当中,每个state store 对应一张表。建表语句如下:
|
在StateStoreMySQLImpl当中,每个state store 对应一张表。建表语句如下:
|
||||||
```sql
|
```sql
|
||||||
CREATE TABLE <className> (
|
CREATE TABLE <className> (
|
||||||
recordKey VARCHAR (255) NOT NULL,
|
recordKey VARCHAR (255) NOT NULL,
|
||||||
@ -163,10 +165,43 @@ CREATE TABLE <className> (
|
|||||||
|
|
||||||
##### initDriver
|
##### initDriver
|
||||||
|
|
||||||
|
对于StateStoreZooKeeperImpl,initDriver中主要是建立zk连接。核心代码如下:
|
||||||
|
```java
|
||||||
|
this.zkManager = new ZKCuratorManager(conf);
|
||||||
|
this.zkManager.start(zkHostPort);
|
||||||
|
this.zkAcl = ZKCuratorManager.getZKAcls(conf);
|
||||||
|
```
|
||||||
|
对于异步模式,会创建线程池,用于异步保存状态。
|
||||||
|
```java
|
||||||
|
ThreadFactory threadFactory = new ThreadFactoryBuilder()
|
||||||
|
.setNameFormat("StateStore ZK Client-%d")
|
||||||
|
.build();
|
||||||
|
this.executorService = new ThreadPoolExecutor(numThreads, numThreads,
|
||||||
|
0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(), threadFactory);
|
||||||
|
LOG.info("Init StateStoreZookeeperImpl by async mode with {} threads.", numThreads);
|
||||||
|
```
|
||||||
|
|
||||||
##### initRecordStorage
|
##### initRecordStorage
|
||||||
|
|
||||||
|
当前函数的实现比较简单,主要是在zk上面创建状态保存的目录。
|
||||||
|
```java
|
||||||
|
try {
|
||||||
|
String checkPath = getNodePath(baseZNode, className);
|
||||||
|
zkManager.createRootDirRecursively(checkPath, zkAcl);
|
||||||
|
return true;
|
||||||
|
} catch (Exception e) {
|
||||||
|
LOG.error("Cannot initialize ZK node for {}: {}",
|
||||||
|
className, e.getMessage());
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 二、ActiveNamenodeResolver
|
||||||
|
具体实现有配置项dfs.federation.router.namenode.resolver.client.class指定。默认为MembershipNamenodeResolver.class。目前最新的也只支持这一种。
|
||||||
|
|
||||||
|
### MembershipNamenodeResolver
|
||||||
|
|
||||||
|
|
||||||
## ActiveNamenodeResolver
|
|
||||||
|
|
||||||
|
|
||||||
## subclusterResolver
|
## subclusterResolver
|
||||||
|
Loading…
Reference in New Issue
Block a user