zihangdai/xlnet: XLNet: Generalized Autoregressive Pretraining for Language Understanding
XLNet 的核心思想：PermutationLM 使用双向上下文 + Transformer-XL 对架构进行改进。
BERT 忽略了屏蔽位置之间的依赖关系，会有预训练和 Fine-tuning 效果的差异。
XLNet：
此外，XLNet 还将最先进的自回归模型 Transformer-XL 的思想整合到预训练中。
上一章提到了数据结构的重要性，这章开始我们就逐步深入理解一下数据结构相关的内容。最常用也是最常见的数据结构都是线性结构，线性结构简单来说就是一条序列，中间有若干元素，“线性” 也是和后面要说的树、图相比而言的。
需要注意的是，数据在计算机中有两种物理存储方式，分别是链式存储和顺序存储，顺序存储是利用计算机中连续一块内存进行存储，链式存储利用指针指向不同的地址进行存储；而上面列出的这些数据结构都是逻辑结构，存储方式不同数据结构的实现方法和复杂度也不同，但效果是可以完全相同的。
存储方式和结构之间没有关系，任何一种结构都可以使用顺序或链式存储实现，要看具体的场景和目的。
Given a string, find the length of the longest substring without repeating characters.
Elasticsearch 建立在 Lucene 上，它不仅仅是一个全文搜索引擎：
Install Elasticsearch with Docker | Elasticsearch Reference [7.2] | Elastic
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.2.0
curl http://127.0.0.1:9200/_cat/health
curl 'http://localhost:9200/?pretty'
Running Kibana on Docker | Kibana User Guide [7.2] | Elastic
docker run --link YOUR_ELASTICSEARCH_CONTAINER_NAME_OR_ID:elasticsearch -p 5601:5601 {docker-repo}:{version}
存储数据到 Elasticsearch 的行为叫做 索引，但在索引一个文档之前，需要确定将文档存储在哪里。
一个 Elasticsearch 集群可以 包含多个 索引 ，相应的每个索引可以包含多个 类型 。 这些不同的类型存储着多个 文档 ，每个文档又有 多个 属性 。
This chapter focuses on the structures assigned by context-free grammars. Context-free grammars don’t specify how the parse tree for a given sentence should be computed. We therefore need to specify algorithms that employ these grammars to efficiently produce correct trees. They are useful in applications such as grammar checking, semantic analysis, question answering and information extraction.
This chapter is devoted to the topic of context-free grammars. They are integral to many computational applications, including grammar checking, semantic interpretation, dialogue understanding, and machine translation.
Problematic of the sliding window in general NN:
RNN is a class of networks designed to address these problems by processing sequences explicitly as sequences, allowing us to handle variable length inputs without the use of arbitrary fixed-sized windows.
Parts-of-speech (also known as POS, word classes, or syntactic categories) are useful because they reveal a lot about a word and its neighbors. Useful for:
labeling named entities
coreference resolution
speech recognition or synthesis