HBase源码分析(从Flush到HFile)

Get Started. It's Free
or sign up with your email address
Rocket clouds
HBase源码分析(从Flush到HFile) by Mind Map: HBase源码分析(从Flush到HFile)

1. HFileWriterV2 append(kv)

1.1. 检查key顺序

1.2. key不同时checkBlockBoundary()

1.2.1. 检查是否到达block边界(默认64K)

1.2.2. 到达边界finishBlock()

1.2.2.1. 获得data block的偏移量

1.2.2.2. writeHeaderAndData()写数据

1.2.2.2.1. ensureBlockReady

1.2.2.2.2. 写入onDiskChecksum

1.2.2.3. 写入索引dataBlockIndexWriter.addEntry()

1.2.2.3.1. add()

1.2.2.4. 缓存块doCacheOnWrite()

1.2.3. 写入索引块writeInlineBlocks

1.2.3.1. 缓存块cacheOnWrite()

1.2.3.2. 写非root索引块writeInlineBlock()

1.2.3.2.1. curInlineChunk.writeNonRoot(out)

1.2.3.2.2. 保存索引块的firstkey,用于父节点索引

1.2.3.3. writeHeaderAndData()

1.2.3.4. blockWritten()把非root索引块信息添加到root索引

1.2.3.5. 缓存块cacheOnWrite()

1.2.4. newBlock()准备写入新block

1.2.4.1. startWriting()

1.2.4.1.1. 初始化offset、block类型、buffer

1.2.4.1.2. header数据占位

1.2.4.1.3. 创建DataOutputStream

1.3. 如果不在写入创建新block

1.4. 写入key和value的长度和内容

1.5. block的firstkey后续作为index block的entry属性

2. writer.appendMetadata()

3. writer.close()

3.1. finishBlock()

3.2. 写索引块数据

3.3. writeIndexBlocks()

3.3.1. 如果rootChunk大于maxChunkSize,分裂成子block

3.3.2. writeRoot()

3.3.3. writeHeaderAndData()

3.4. 写meta block

3.5. 写file info

3.6. 写trailer

4. HRegion flusher.commit()

4.1. commitFile()

4.1.1. 获得HFile临时文件名

4.1.2. 重命名临时文件

4.2. updateStorefiles()

4.2.1. 把StoreFile更新至列表中

4.2.2. StoreFile列表排序

4.2.3. clearSnapshot()