diff options
| author | Clinton Gormley <clint@traveljury.com> | 2017-07-05 12:30:19 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2017-07-05 12:30:19 +0200 |
| commit | 0170e0e8d3fa210a539737c91251dbdd520bb458 (patch) | |
| tree | 57cc9140612ce9a13f4287f560820e5008c9ac4d | |
| parent | fefcae3d45814950c29c0b1929489ea0fb4d79bd (diff) | |
Remove usage of multi-types from the docs and added a page explaining type removal (#25543)
Closes #25401
42 files changed, 638 insertions, 790 deletions
diff --git a/docs/build.gradle b/docs/build.gradle index e0de246135..1f7b418fa6 100644 --- a/docs/build.gradle +++ b/docs/build.gradle @@ -51,6 +51,7 @@ buildRestTests.expectedUnconvertedCandidates = [ 'reference/indices/recovery.asciidoc', 'reference/indices/segments.asciidoc', 'reference/indices/shard-stores.asciidoc', + 'reference/mapping/removal_of_types.asciidoc', 'reference/search/field-stats.asciidoc', 'reference/search/profile.asciidoc', ] @@ -438,7 +439,7 @@ buildRestTests.setups['stored_scripted_metric_script'] = ''' id: "my_init_script" body: { "script": { "lang": "painless", "source": "params._agg.transactions = []" } } - match: { acknowledged: true } - + - do: put_script: id: "my_map_script" diff --git a/docs/reference/aggregations/bucket/children-aggregation.asciidoc b/docs/reference/aggregations/bucket/children-aggregation.asciidoc index 57ea9ce56c..c5f706f080 100644 --- a/docs/reference/aggregations/bucket/children-aggregation.asciidoc +++ b/docs/reference/aggregations/bucket/children-aggregation.asciidoc @@ -1,74 +1,91 @@ [[search-aggregations-bucket-children-aggregation]] === Children Aggregation -A special single bucket aggregation that enables aggregating from buckets on parent document types to buckets on child documents. +A special single bucket aggregation that selects child documents that have the specified type, as defined in a <<parent-join,`join` field>>. -This aggregation relies on the <<mapping-parent-field,_parent field>> in the mapping. This aggregation has a single option: +This aggregation has a single option: -* `type` - The child type that the buckets in the parent space should be mapped to. +* `type` - The child type that should be selected. + +For example, let's say we have an index of questions and answers. The answer type has the following `join` field in the mapping: -For example, let's say we have an index of questions and answers. The answer type has the following `_parent` field in the mapping: [source,js] -------------------------------------------------- PUT child_example { - "settings": { - "mapping.single_type": false - }, - "mappings": { - "answer" : { - "_parent" : { - "type" : "question" - } + "mappings": { + "doc": { + "properties": { + "join": { + "type": "join", + "relations": { + "question": "answer" + } } + } } + } } -------------------------------------------------- // CONSOLE -The question typed document contain a tag field and the answer typed documents contain an owner field. With the `children` +The `question` document contain a tag field and the `answer` documents contain an owner field. With the `children` aggregation the tag buckets can be mapped to the owner buckets in a single request even though the two fields exist in two different kinds of documents. -An example of a question typed document: +An example of a question document: + [source,js] -------------------------------------------------- -PUT child_example/question/1 +PUT child_example/doc/1 { - "body": "<p>I have Windows 2003 server and i bought a new Windows 2008 server...", - "title": "Whats the best way to file transfer my site from server to a newer one?", - "tags": [ - "windows-server-2003", - "windows-server-2008", - "file-transfer" - ] + "join": { + "name": "question" + }, + "body": "<p>I have Windows 2003 server and i bought a new Windows 2008 server...", + "title": "Whats the best way to file transfer my site from server to a newer one?", + "tags": [ + "windows-server-2003", + "windows-server-2008", + "file-transfer" + ] } -------------------------------------------------- // CONSOLE // TEST[continued] -Examples of `answer` typed documents: +Examples of `answer` documents: + [source,js] -------------------------------------------------- -PUT child_example/answer/1?parent=1&refresh +PUT child_example/doc/2?routing=1 { - "owner": { - "location": "Norfolk, United Kingdom", - "display_name": "Sam", - "id": 48 - }, - "body": "<p>Unfortunately you're pretty much limited to FTP...", - "creation_date": "2009-05-04T13:45:37.030" + "join": { + "name": "answer", + "parent": "1" + }, + "owner": { + "location": "Norfolk, United Kingdom", + "display_name": "Sam", + "id": 48 + }, + "body": "<p>Unfortunately you're pretty much limited to FTP...", + "creation_date": "2009-05-04T13:45:37.030" } -PUT child_example/answer/2?parent=1&refresh + +PUT child_example/doc/3?routing=1&refresh { - "owner": { - "location": "Norfolk, United Kingdom", - "display_name": "Troll", - "id": 49 - }, - "body": "<p>Use Linux...", - "creation_date": "2009-05-05T13:45:37.030" + "join": { + "name": "answer", + "parent": "1" + }, + "owner": { + "location": "Norfolk, United Kingdom", + "display_name": "Troll", + "id": 49 + }, + "body": "<p>Use Linux...", + "creation_date": "2009-05-05T13:45:37.030" } -------------------------------------------------- // CONSOLE @@ -117,10 +134,18 @@ Possible response: [source,js] -------------------------------------------------- { - "timed_out": false, "took": 25, - "_shards": { "total": 5, "successful": 5, "failed": 0 }, - "hits": { "total": 3, "max_score": 0.0, hits: [] }, + "timed_out": false, + "_shards": { + "total": 5, + "successful": 5, + "failed": 0 + }, + "hits": { + "total": 3, + "max_score": 0.0, + "hits": [] + }, "aggregations": { "top-tags": { "doc_count_error_upper_bound": 0, diff --git a/docs/reference/cat/nodes.asciidoc b/docs/reference/cat/nodes.asciidoc index 9562111091..60e204410c 100644 --- a/docs/reference/cat/nodes.asciidoc +++ b/docs/reference/cat/nodes.asciidoc @@ -184,7 +184,7 @@ segments |1.4kb |`segments.version_map_memory` |`svmm`, `segmentsVersionMapMemory` |No |Memory used by version map |1.0kb |`segments.fixed_bitset_memory` |`sfbm`, `fixedBitsetMemory` |No -|Memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields |1.0kb +|Memory used by fixed bit sets for nested object field types and type filters for types referred in `join` fields |1.0kb |`suggest.current` |`suc`, `suggestCurrent` |No |Number of current suggest operations |0 |`suggest.time` |`suti`, `suggestTime` |No |Time spent in suggest |0 |`suggest.total` |`suto`, `suggestTotal` |No |Number of suggest operations |0 diff --git a/docs/reference/docs/bulk.asciidoc b/docs/reference/docs/bulk.asciidoc index 513a5ec61b..b29afd146e 100644 --- a/docs/reference/docs/bulk.asciidoc +++ b/docs/reference/docs/bulk.asciidoc @@ -217,14 +217,6 @@ Each bulk item can include the routing value using the index / delete operation based on the `_routing` mapping. [float] -[[bulk-parent]] -=== Parent - -Each bulk item can include the parent value using the `_parent`/`parent` -field. It automatically follows the behavior of the index / delete -operation based on the `_parent` / `_routing` mapping. - -[float] [[bulk-wait-for-active-shards]] === Wait For Active Shards diff --git a/docs/reference/docs/delete.asciidoc b/docs/reference/docs/delete.asciidoc index f26a7fc64d..fe9a58b254 100644 --- a/docs/reference/docs/delete.asciidoc +++ b/docs/reference/docs/delete.asciidoc @@ -85,23 +85,6 @@ specified, the delete api will throw a `RoutingMissingException` and reject the request. [float] -[[delete-parent]] -=== Parent - -The `parent` parameter can be set, which will basically be the same as -setting the routing parameter. - -Note that deleting a parent document does not automatically delete its -children. One way of deleting all child documents given a parent's id is -to use the <<docs-delete-by-query,Delete By Query API>> to perform a - index with the automatically generated (and indexed) -field _parent, which is in the format parent_type#parent_id. - -When deleting a child document its parent id must be specified, otherwise -the delete request will be rejected and a `RoutingMissingException` will be -thrown instead. - -[float] [[delete-index-creation]] === Automatic index creation diff --git a/docs/reference/docs/get.asciidoc b/docs/reference/docs/get.asciidoc index fef929cdbd..2a252595dd 100644 --- a/docs/reference/docs/get.asciidoc +++ b/docs/reference/docs/get.asciidoc @@ -59,14 +59,6 @@ changed since the last refresh visible. In order to disable realtime GET, one can set the `realtime` parameter to `false`. [float] -[[type]] -=== Optional Type - -The get API allows for `_type` to be optional. Set it to `_all` in order -to fetch the first document matching the id across all types. - - -[float] [[get-source-filtering]] === Source filtering @@ -178,7 +170,7 @@ The result of the above get operation is: Field values fetched from the document it self are always returned as an array. Since the `counter` field is not stored the get request simply ignores it when trying to get the `stored_fields.` -It is also possible to retrieve metadata fields like `_routing` and `_parent` fields: +It is also possible to retrieve metadata fields like the `_routing` field: [source,js] -------------------------------------------------- @@ -223,7 +215,7 @@ will fail. [float] [[_source]] -=== Getting the _source directly +=== Getting the +_source+ directly Use the `/{index}/{type}/{id}/_source` endpoint to get just the `_source` field of the document, @@ -283,15 +275,15 @@ replicas. The `preference` can be set to: -`_primary`:: +`_primary`:: The operation will go and be executed only on the primary shards. -`_local`:: +`_local`:: The operation will prefer to be executed on a local allocated shard if possible. -Custom (string) value:: +Custom (string) value:: A custom value will be used to guarantee that the same shards will be used for the same custom value. This can help with "jumping values" when hitting different shards in different refresh diff --git a/docs/reference/docs/index_.asciidoc b/docs/reference/docs/index_.asciidoc index 2f042aec13..4c6f69137e 100644 --- a/docs/reference/docs/index_.asciidoc +++ b/docs/reference/docs/index_.asciidoc @@ -1,6 +1,8 @@ [[docs-index_]] == Index API +IMPORTANT: See <<removal-of-types>>. + The index API adds or updates a typed JSON document in a specific index, making it searchable. The following example inserts the JSON document into the "twitter" index, under a type called "tweet" with an id of 1: @@ -269,41 +271,6 @@ and set to be `required`, the index operation will fail if no routing value is provided or extracted. [float] -[[parent-children]] -=== Parents & Children - -A child document can be indexed by specifying its parent when indexing. -For example: - -[source,js] --------------------------------------------------- -PUT blogs -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "tag_parent": {}, - "blog_tag": { - "_parent": { - "type": "tag_parent" - } - } - } -} - -PUT blogs/blog_tag/1122?parent=1111 -{ - "tag" : "something" -} --------------------------------------------------- -// CONSOLE - -When indexing a child document, the routing value is automatically set -to be the same as its parent, unless the routing value is explicitly -specified using the `routing` parameter. - -[float] [[index-distributed]] === Distributed diff --git a/docs/reference/docs/reindex.asciidoc b/docs/reference/docs/reindex.asciidoc index 64ea0cdc48..eaefa1a07a 100644 --- a/docs/reference/docs/reindex.asciidoc +++ b/docs/reference/docs/reindex.asciidoc @@ -297,7 +297,6 @@ change: * `_index` * `_version` * `_routing` - * `_parent` Setting `_version` to `null` or clearing it from the `ctx` map is just like not sending the version in an indexing request. It will cause that document to be diff --git a/docs/reference/docs/update.asciidoc b/docs/reference/docs/update.asciidoc index 6a3a39cf5c..37091c47e0 100644 --- a/docs/reference/docs/update.asciidoc +++ b/docs/reference/docs/update.asciidoc @@ -66,8 +66,8 @@ POST test/type1/1/_update // TEST[continued] In addition to `_source`, the following variables are available through -the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing`, -`_parent`, and `_now` (the current timestamp). +the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing` +and `_now` (the current timestamp). We can also add a new field to the document: @@ -286,13 +286,6 @@ Routing is used to route the update request to the right shard and sets the routing for the upsert request if the document being updated doesn't exist. Can't be used to update the routing of an existing document. -`parent`:: - -Parent is used to route the update request to the right shard and sets the -parent for the upsert request if the document being updated doesn't exist. -Can't be used to update the `parent` of an existing document. -If an alias index routing is specified then it overrides the parent routing and it is used to route the request. - `timeout`:: Timeout waiting for a shard to become available. diff --git a/docs/reference/getting-started.asciidoc b/docs/reference/getting-started.asciidoc index ee3b30d704..41aaed1b31 100755 --- a/docs/reference/getting-started.asciidoc +++ b/docs/reference/getting-started.asciidoc @@ -47,7 +47,6 @@ A node can be configured to join a specific cluster by the cluster name. By defa In a single cluster, you can have as many nodes as you want. Furthermore, if there are no other Elasticsearch nodes currently running on your network, starting a single node will by default form a new single-node cluster named `elasticsearch`. -[sect2] [float] === Index @@ -58,7 +57,9 @@ In a single cluster, you can define as many indexes as you want. [float] === Type -Within an index, you can define one or more types. A type is a logical category/partition of your index whose semantics is completely up to you. In general, a type is defined for documents that have a set of common fields. For example, let's assume you run a blogging platform and store all your data in a single index. In this index, you may define a type for user data, another type for blog data, and yet another type for comments data. +deprecated[6.0.0,See <<removal-of-types>>] + +A type used to be a logical category/partition of your index to allow you to store different types of documents in the same index, eg one type for users, another type for blog posts. It is no longer possible to create multiple types in an index, and the whole concept of types will be removed in a later version. See <<removal-of-types>> for more. [float] === Document @@ -359,13 +360,11 @@ You might also notice that the customer index has a yellow health tagged to it. === Index and Query a Document -Let's now put something into our customer index. Remember previously that in order to index a document, we must tell Elasticsearch which type in the index it should go to. - -Let's index a simple customer document into the customer index, "external" type, with an ID of 1 as follows: +Let's now put something into our customer index. We'll index a simple customer document into the customer index, with an ID of 1 as follows: [source,js] -------------------------------------------------- -PUT /customer/external/1?pretty +PUT /customer/doc/1?pretty { "name": "John Doe" } @@ -378,7 +377,7 @@ And the response: -------------------------------------------------- { "_index" : "customer", - "_type" : "external", + "_type" : "doc", "_id" : "1", "_version" : 1, "result" : "created", @@ -394,7 +393,7 @@ And the response: -------------------------------------------------- // TESTRESPONSE[s/"_seq_no" : 0/"_seq_no" : $body._seq_no/ s/"_primary_term" : 1/"_primary_term" : $body._primary_term/] -From the above, we can see that a new customer document was successfully created inside the customer index and the external type. The document also has an internal id of 1 which we specified at index time. +From the above, we can see that a new customer document was successfully created inside the customer index. The document also has an internal id of 1 which we specified at index time. It is important to note that Elasticsearch does not require you to explicitly create an index first before you can index documents into it. In the previous example, Elasticsearch will automatically create the customer index if it didn't already exist beforehand. @@ -402,7 +401,7 @@ Let's now retrieve that document that we just indexed: [source,js] -------------------------------------------------- -GET /customer/external/1?pretty +GET /customer/doc/1?pretty -------------------------------------------------- // CONSOLE // TEST[continued] @@ -413,7 +412,7 @@ And the response: -------------------------------------------------- { "_index" : "customer", - "_type" : "external", + "_type" : "doc", "_id" : "1", "_version" : 1, "found" : true, @@ -451,11 +450,11 @@ Before we move on, let's take a closer look again at some of the API commands th [source,js] -------------------------------------------------- PUT /customer -PUT /customer/external/1 +PUT /customer/doc/1 { "name": "John Doe" } -GET /customer/external/1 +GET /customer/doc/1 DELETE /customer -------------------------------------------------- // CONSOLE @@ -481,18 +480,18 @@ We've previously seen how we can index a single document. Let's recall that comm [source,js] -------------------------------------------------- -PUT /customer/external/1?pretty +PUT /customer/doc/1?pretty { "name": "John Doe" } -------------------------------------------------- // CONSOLE -Again, the above will index the specified document into the customer index, external type, with the ID of 1. If we then executed the above command again with a different (or same) document, Elasticsearch will replace (i.e. reindex) a new document on top of the existing one with the ID of 1: +Again, the above will index the specified document into the customer index, with the ID of 1. If we then executed the above command again with a different (or same) document, Elasticsearch will replace (i.e. reindex) a new document on top of the existing one with the ID of 1: [source,js] -------------------------------------------------- -PUT /customer/external/1?pretty +PUT /customer/doc/1?pretty { "name": "Jane Doe" } @@ -504,7 +503,7 @@ The above changes the name of the document with the ID of 1 from "John Doe" to " [source,js] -------------------------------------------------- -PUT /customer/external/2?pretty +PUT /customer/doc/2?pretty { "name": "Jane Doe" } @@ -520,7 +519,7 @@ This example shows how to index a document without an explicit ID: [source,js] -------------------------------------------------- -POST /customer/external?pretty +POST /customer/doc?pretty { "name": "Jane Doe" } @@ -538,7 +537,7 @@ This example shows how to update our previous document (ID of 1) by changing the [source,js] -------------------------------------------------- -POST /customer/external/1/_update?pretty +POST /customer/doc/1/_update?pretty { "doc": { "name": "Jane Doe" } } @@ -550,7 +549,7 @@ This example shows how to update our previous document (ID of 1) by changing the [source,js] -------------------------------------------------- -POST /customer/external/1/_update?pretty +POST /customer/doc/1/_update?pretty { "doc": { "name": "Jane Doe", "age": 20 } } @@ -562,7 +561,7 @@ Updates can also be performed by using simple scripts. This example uses a scrip [source,js] -------------------------------------------------- -POST /customer/external/1/_update?pretty +POST /customer/doc/1/_update?pretty { "script" : "ctx._source.age += 5" } @@ -580,7 +579,7 @@ Deleting a document is fairly straightforward. This example shows how to delete [source,js] -------------------------------------------------- -DELETE /customer/external/2?pretty +DELETE /customer/doc/2?pretty -------------------------------------------------- // CONSOLE // TEST[continued] @@ -597,7 +596,7 @@ As a quick example, the following call indexes two documents (ID 1 - John Doe an [source,js] -------------------------------------------------- -POST /customer/external/_bulk?pretty +POST /customer/doc/_bulk?pretty {"index":{"_id":"1"}} {"name": "John Doe" } {"index":{"_id":"2"}} @@ -609,7 +608,7 @@ This example updates the first document (ID of 1) and then deletes the second do [source,sh] -------------------------------------------------- -POST /customer/external/_bulk?pretty +POST /customer/doc/_bulk?pretty {"update":{"_id":"1"}} {"doc": { "name": "John Doe becomes Jane Doe" } } {"delete":{"_id":"2"}} diff --git a/docs/reference/glossary.asciidoc b/docs/reference/glossary.asciidoc index edf84f3b4c..3559cb8985 100644 --- a/docs/reference/glossary.asciidoc +++ b/docs/reference/glossary.asciidoc @@ -45,7 +45,7 @@ [[glossary-id]] id :: The ID of a <<glossary-document,document>> identifies a document. The - `index/type/id` of a document must be unique. If no ID is provided, + `index/id` of a document must be unique. If no ID is provided, then it will be auto-generated. (also see <<glossary-routing,routing>>) [[glossary-field]] field :: @@ -64,8 +64,8 @@ [[glossary-index]] index :: An index is like a _table_ in a relational database. It has a - <<glossary-mapping,mapping>> which defines the <<glossary-field,fields>> in the index, - which are grouped by multiple <<glossary-type,type>>. + <<glossary-mapping,mapping>> which contains a <<glossary-type,type>>, + which contains the <<glossary-field,fields>> in the index. + An index is a logical namespace which maps to one or more <<glossary-primary-shard,primary shards>> and can have zero or more @@ -74,8 +74,8 @@ [[glossary-mapping]] mapping :: A mapping is like a _schema definition_ in a relational database. Each - <<glossary-index,index>> has a mapping, which defines each <<glossary-type,type>> - within the index, plus a number of index-wide settings. + <<glossary-index,index>> has a mapping, which defines a <<glossary-type,type>>, + plus a number of index-wide settings. + A mapping can either be defined explicitly, or it will be generated automatically when a document is indexed. @@ -179,11 +179,6 @@ [[glossary-type]] type :: - A type represents the _type_ of document, e.g. an `email`, a `user`, or a `tweet`. - The search API can filter documents by type. - An <<glossary-index,index>> can contain multiple types, and each type has a - list of <<glossary-field,fields>> that can be specified for - <<glossary-document,documents>> of that type. Fields with the same - name in different types in the same index must have the same <<glossary-mapping,mapping>> - (which defines how each field in the document is indexed and made searchable). + A type used to represent the _type_ of document, e.g. an `email`, a `user`, or a `tweet`. + Types are deprecated and are in the process of being removed. See <<removal-of-types>>. diff --git a/docs/reference/indices/aliases.asciidoc b/docs/reference/indices/aliases.asciidoc index 3f839465e8..447873a595 100644 --- a/docs/reference/indices/aliases.asciidoc +++ b/docs/reference/indices/aliases.asciidoc @@ -244,8 +244,6 @@ GET /alias2/_search?q=user:kimchy&routing=2,3 // CONSOLE // TEST[continued] -If an index operation that uses index routing alias also has a parent routing, the -parent routing is ignored. [float] [[alias-adding]] diff --git a/docs/reference/indices/create-index.asciidoc b/docs/reference/indices/create-index.asciidoc index 6537fa74cb..8089745844 100644 --- a/docs/reference/indices/create-index.asciidoc +++ b/docs/reference/indices/create-index.asciidoc @@ -74,7 +74,7 @@ that can be set when creating an index, please check the [[mappings]] === Mappings -The create index API allows to provide a set of one or more mappings: +The create index API allows to provide a type mapping: [source,js] -------------------------------------------------- @@ -121,7 +121,7 @@ PUT test [[create-index-wait-for-active-shards]] === Wait For Active Shards -By default, index creation will only return a response to the client when the primary copies of +By default, index creation will only return a response to the client when the primary copies of each shard have been started, or the request times out. The index creation response will indicate what happened: @@ -135,15 +135,15 @@ what happened: -------------------------------------------------- // TESTRESPONSE -`acknowledged` indicates whether the index was successfully created in the cluster, while +`acknowledged` indicates whether the index was successfully created in the cluster, while `shards_acknowledged` indicates whether the requisite number of shard copies were started for -each shard in the index before timing out. Note that it is still possible for either -`acknowledged` or `shards_acknowledged` to be `false`, but the index creation was successful. -These values simply indicate whether the operation completed before the timeout. If +each shard in the index before timing out. Note that it is still possible for either +`acknowledged` or `shards_acknowledged` to be `false`, but the index creation was successful. +These values simply indicate whether the operation completed before the timeout. If `acknowledged` is `false`, then we timed out before the cluster state was updated with the -newly created index, but it probably will be created sometime soon. If `shards_acknowledged` -is `false`, then we timed out before the requisite number of shards were started (by default -just the primaries), even if the cluster state was successfully updated to reflect the newly +newly created index, but it probably will be created sometime soon. If `shards_acknowledged` +is `false`, then we timed out before the requisite number of shards were started (by default +just the primaries), even if the cluster state was successfully updated to reflect the newly created index (i.e. `acknowledged=true`). We can change the default of only waiting for the primary shards to start through the index @@ -171,5 +171,5 @@ PUT test?wait_for_active_shards=2 // CONSOLE // TEST[skip:requires two nodes] -A detailed explanation of `wait_for_active_shards` and its possible values can be found +A detailed explanation of `wait_for_active_shards` and its possible values can be found <<index-wait-for-active-shards,here>>. diff --git a/docs/reference/indices/put-mapping.asciidoc b/docs/reference/indices/put-mapping.asciidoc index 5fc4f0d9cc..a1bc842855 100644 --- a/docs/reference/indices/put-mapping.asciidoc +++ b/docs/reference/indices/put-mapping.asciidoc @@ -7,20 +7,7 @@ fields to an existing type: [source,js] -------------------------------------------------- PUT twitter <1> -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "tweet": { - "properties": { - "message": { - "type": "text" - } - } - } - } -} +{} PUT twitter/_mapping/user <2> { @@ -31,19 +18,19 @@ PUT twitter/_mapping/user <2> } } -PUT twitter/_mapping/tweet <3> +PUT twitter/_mapping/user <3> { "properties": { - "user_name": { - "type": "text" + "email": { + "type": "keyword" } } } -------------------------------------------------- // CONSOLE -<1> <<indices-create-index,Creates an index>> called `twitter` with the `message` field in the `tweet` <<mapping-type,mapping type>>. +<1> <<indices-create-index,Creates an index>> called `twitter` without any type mapping. <2> Uses the PUT mapping API to add a new mapping type called `user`. -<3> Uses the PUT mapping API to add a new field called `user_name` to the `tweet` mapping type. +<3> Uses the PUT mapping API to add a new field called `email` to the `user` mapping type. More information on how to define type mappings can be found in the <<mapping,mapping>> section. @@ -139,80 +126,3 @@ PUT my_index/_mapping/user Each <<mapping-params,mapping parameter>> specifies whether or not its setting can be updated on an existing field. -[float] -[[merging-conflicts]] -=== Conflicts between fields in different types - -Fields in the same index with the same name in two different types must have -the same mapping, as they are backed by the same field internally. Trying to -<<updating-field-mappings,update a mapping parameter>> for a field which -exists in more than one type will throw an exception, unless you specify the -`update_all_types` parameter, in which case it will update that parameter -across all fields with the same name in the same index. - -TIP: The only parameters which are exempt from this rule -- they can be set to -different values on each field -- can be found in <<field-conflicts>>. - -For example, this fails: - -[source,js] ------------------------------------ -PUT my_index -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "type_one": { - "properties": { - "text": { <1> - "type": "text", - "analyzer": "standard" - } - } - }, - "type_two": { - "properties": { - "text": { <1> - "type": "text", - "analyzer": "standard" - } - } - } - } -} - -PUT my_index/_mapping/type_one <2> -{ - "properties": { - "text": { - "type": "text", - "analyzer": "standard", - "search_analyzer": "whitespace" - } - } -} ------------------------------------ -// CONSOLE -// TEST[catch:request] -<1> Create an index with two types, both of which contain a `text` field which have the same mapping. -<2> Trying to update the `search_analyzer` just for `type_one` throws an exception like `"Merge failed with failures..."`. - -But this then running this succeeds: - -[source,js] ------------------------------------ -PUT my_index/_mapping/type_one?update_all_types <1> -{ - "properties": { - "text": { - "type": "text", - "analyzer": "standard", - "search_analyzer": "whitespace" - } - } -} ------------------------------------ -// CONSOLE -// TEST[continued] -<1> Adding the `update_all_types` parameter updates the `text` field in `type_one` and `type_two`. diff --git a/docs/reference/ingest/ingest-node.asciidoc b/docs/reference/ingest/ingest-node.asciidoc index 7759dd7643..1621183933 100644 --- a/docs/reference/ingest/ingest-node.asciidoc +++ b/docs/reference/ingest/ingest-node.asciidoc @@ -498,7 +498,7 @@ The following example sets the `_id` metadata field of a document to `1`: -------------------------------------------------- // NOTCONSOLE -The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`, `_parent`. +The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`. [float] [[accessing-ingest-metadata]] @@ -1186,7 +1186,7 @@ that is generally written for humans and not computer consumption. This processor comes packaged with over https://github.com/elastic/elasticsearch/tree/master/modules/ingest-common/src/main/resources/patterns[120 reusable patterns]. -If you need help building patterns to match your logs, you will find the {kibana-ref}/xpack-grokdebugger.html[Grok Debugger] tool quite useful! The Grok Debugger is an {xpack} feature under the Basic License and is therefore *free to use*. The Grok Constructor at <http://grokconstructor.appspot.com/> is also a useful tool. +If you need help building patterns to match your logs, you will find the {kibana-ref}/xpack-grokdebugger.html[Grok Debugger] tool quite useful! The Grok Debugger is an {xpack} feature under the Basic License and is therefore *free to use*. The Grok Constructor at <http://grokconstructor.appspot.com/> is also a useful tool. [[grok-basics]] ==== Grok Basics diff --git a/docs/reference/mapping.asciidoc b/docs/reference/mapping.asciidoc index 59e49c317a..bb4d375dbe 100644 --- a/docs/reference/mapping.asciidoc +++ b/docs/reference/mapping.asciidoc @@ -17,13 +17,14 @@ are stored and indexed. For instance, use mappings to define: [float] [[mapping-type]] -== Mapping Types +== Mapping Type -Each index has one or more _mapping types_, which are used to divide the -documents in an index into logical groups. User documents might be stored in a -`user` type, and blog posts in a `blogpost` type. +Each index has one _mapping type_ which determines how the document will be +indexed. -Each mapping type has: +deprecated[6.0.0,See <<removal-of-types>>]. + +A mapping type has: <<mapping-fields,Meta-fields>>:: @@ -34,11 +35,8 @@ treated. Examples of meta-fields include the document's <<mapping-types,Fields>> or _properties_:: -Each mapping type contains a list of fields or `properties` pertinent to that -type. A `user` type might contain `title`, `name`, and `age` fields, while a -`blogpost` type might contain `title`, `body`, `user_id` and `created` fields. -Fields with the same name in different mapping types in the same index -<<field-conflicts,must have the same mapping>>. +A mapping type contains a list of fields or `properties` pertinent to the +document. [float] @@ -91,14 +89,12 @@ causing a mapping explosion: == Dynamic mapping Fields and mapping types do not need to be defined before being used. Thanks -to _dynamic mapping_, new mapping types and new field names will be added -automatically, just by indexing a document. New fields can be added both to -the top-level mapping type, and to inner <<object,`object`>> and -<<nested,`nested`>> fields. +to _dynamic mapping_, new field names will be added automatically, just by +indexing a document. New fields can be added both to the top-level mapping +type, and to inner <<object,`object`>> and <<nested,`nested`>> fields. -The -<<dynamic-mapping,dynamic mapping>> rules can be configured to -customise the mapping that is used for new types and new fields. +The <<dynamic-mapping,dynamic mapping>> rules can be configured to customise +the mapping that is used for new fields. [float] == Explicit mappings @@ -107,70 +103,33 @@ You know more about your data than Elasticsearch can guess, so while dynamic mapping can be useful to get started, at some point you will want to specify your own explicit mappings. -You can create mapping types and field mappings when you -<<indices-create-index,create an index>>, and you can add mapping types and +You can create field mappings when you +<<indices-create-index,create an index>>, and you can add fields to an existing index with the <<indices-put-mapping,PUT mapping API>>. [float] -== Updating existing mappings +== Updating existing field mappings -Other than where documented, *existing type and field mappings cannot be +Other than where documented, *existing field mappings cannot be updated*. Changing the mapping would mean invalidating already indexed documents. Instead, you should create a new index with the correct mappings -and reindex your data into that index. - -[[field-conflicts]] -[float] -== Fields are shared across mapping types - -Mapping types are used to group fields, but the fields in each mapping type -are not independent of each other. Fields with: - -* the _same name_ -* in the _same index_ -* in _different mapping types_ -* map to the _same field_ internally, -* and *must have the same mapping*. - -If a `title` field exists in both the `user` and `blogpost` mapping types, the -`title` fields must have exactly the same mapping in each type. The only -exceptions to this rule are the <<copy-to>>, <<dynamic>>, <<enabled>>, -<<ignore-above>>, and <<properties>> parameters, which may have different -settings per field. - -Usually, fields with the same name also contain the same type of data, so -having the same mapping is not a problem. When conflicts do arise, these can -be solved by choosing more descriptive names, such as `user_title` and -`blog_title`. +and <<docs-reindex,reindex>> your data into that index. [float] == Example mapping -A mapping for the example described above could be specified when creating the -index, as follows: +A mapping could be specified when creating an index, as follows: [source,js] --------------------------------------- PUT my_index <1> { - "settings": { - "mapping.single_type": false - }, "mappings": { - "user": { <2> + "doc": { <2> "properties": { <3> "title": { "type": "text" }, <4> "name": { "type": "text" }, <4> - "age": { "type": "integer" } <4> - } - }, - "blogpost": { <2> - "properties": { <3> - "title": { "type": "text" }, <4> - "body": { "type": "text" }, <4> - "user_id": { - "type": "keyword" <4> - }, + "age": { "type": "integer" }, <4> "created": { "type": "date", <4> "format": "strict_date_optional_time||epoch_millis" @@ -182,13 +141,15 @@ PUT my_index <1> --------------------------------------- // CONSOLE <1> Create an index called `my_index`. -<2> Add mapping types called `user` and `blogpost`. -<3> Specify fields or _properties_ in each mapping type. +<2> Add a mapping type called `doc`. +<3> Specify fields or _properties_. <4> Specify the data `type` and mapping for each field. -- +include::mapping/removal_of_types.asciidoc[] + include::mapping/types.asciidoc[] include::mapping/fields.asciidoc[] diff --git a/docs/reference/mapping/dynamic-mapping.asciidoc b/docs/reference/mapping/dynamic-mapping.asciidoc index 99dcf1339e..b4b9198b17 100644 --- a/docs/reference/mapping/dynamic-mapping.asciidoc +++ b/docs/reference/mapping/dynamic-mapping.asciidoc @@ -16,14 +16,10 @@ PUT data/counters/1 <1> <1> Creates the `data` index, the `counters` mapping type, and a field called `count` with datatype `long`. -The automatic detection and addition of new types and fields is called +The automatic detection and addition of new fields is called _dynamic mapping_. The dynamic mapping rules can be customised to suit your purposes with: -<<default-mapping,`_default_` mapping>>:: - - Configure the base mapping to be used for new mapping types. - <<dynamic-field-mapping,Dynamic field mappings>>:: The rules governing dynamic field detection. @@ -36,50 +32,9 @@ TIP: <<indices-templates,Index templates>> allow you to configure the default mappings, settings and aliases for new indices, whether created automatically or explicitly. +include::dynamic/field-mapping.asciidoc[] -[float] -=== Disabling automatic type creation - -Automatic type creation can be disabled per-index by setting the `index.mapper.dynamic` -setting to `false` in the index settings: - -[source,js] --------------------------------------------------- -PUT data/_settings -{ - "index.mapper.dynamic":false <1> -} --------------------------------------------------- -// CONSOLE -// TEST[continued] - -<1> Disable automatic type creation for the index named "data". - -Automatic type creation can also be disabled for all indices by setting an index template: - -[source,js] --------------------------------------------------- -PUT _template/template_all -{ - "index_patterns": ["*"], - "order":0, - "settings": { - "index.mapper.dynamic": false <1> - } -} --------------------------------------------------- -// CONSOLE -// TEST[continued] - -<1> Disable automatic type creation for all indices. - -Regardless of the value of this setting, types can still be added explicitly -when <<indices-create-index,creating an index>> or with the -<<indices-put-mapping,PUT mapping>> API. - +include::dynamic/templates.asciidoc[] include::dynamic/default-mapping.asciidoc[] -include::dynamic/field-mapping.asciidoc[] - -include::dynamic/templates.asciidoc[] diff --git a/docs/reference/mapping/dynamic/default-mapping.asciidoc b/docs/reference/mapping/dynamic/default-mapping.asciidoc index 1483672744..4dbd723f9c 100644 --- a/docs/reference/mapping/dynamic/default-mapping.asciidoc +++ b/docs/reference/mapping/dynamic/default-mapping.asciidoc @@ -1,89 +1,14 @@ [[default-mapping]] === `_default_` mapping -The default mapping, which will be used as the base mapping for any new -mapping types, can be customised by adding a mapping type with the name +deprecated[6.0.0,See <<removal-of-types>>] + +The default mapping, which will be used as the base mapping for a new +mapping type, can be customised by adding a mapping type with the name `_default_` to an index, either when <<indices-create-index,creating the index>> or later on with the <<indices-put-mapping,PUT mapping>> API. +The documentation for this feature has been removed as it no longer makes +sense in 6.x where there can be only a single type per index. -[source,js] --------------------------------------------------- -PUT my_index -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "_default_": { <1> - "_source": { - "enabled": false - } - }, - "user": {}, <2> - "blogpost": { <3> - "_source": { - "enabled": true - } - } - } -} --------------------------------------------------- -// CONSOLE -<1> The `_default_` mapping defaults the <<mapping-source-field,`_source`>> field to disabled. -<2> The `user` type inherits the settings from `_default_`. -<3> The `blogpost` type overrides the defaults and enables the <<mapping-source-field,`_source`>> field. - -NOTE: When updating the `_default_` mapping with the -<<indices-put-mapping,PUT mapping>> API, the new mapping is not merged with -the existing mapping. Instead, the new `_default_` mapping replaces the -existing one. - -While the `_default_` mapping can be updated after an index has been created, -the new defaults will only affect mapping types that are created afterwards. - -The `_default_` mapping can be used in conjunction with -<<indices-templates,Index templates>> to control dynamically created types -within automatically created indices: - - -[source,js] --------------------------------------------------- -PUT _template/logging -{ - "index_patterns": ["logs-*"], <1> - "settings": { "number_of_shards": 1 }, <2> - "mappings": { - "_default_": { - "_field_names": { <3> - "enabled": false - }, - "dynamic_templates": [ - { - "strings": { <4> - "match_mapping_type": "string", - "mapping": { - "type": "text", - "fields": { - "raw": { - "type": "keyword", - "ignore_above": 256 - } - } - } - } - } - ] - } - } -} - -PUT logs-2015.10.01/event/1 -{ "message": "error:16" } --------------------------------------------------- -// CONSOLE -<1> The `logging` template will match any indices beginning with `logs-`. -<2> Matching indices will be created with a single primary shard. -<3> The `_field_names` field will be disabled by default for new type mappings. -<4> String fields will be created with a `text` main field, and a `keyword` `.raw` field. diff --git a/docs/reference/mapping/dynamic/templates.asciidoc b/docs/reference/mapping/dynamic/templates.asciidoc index 90fc5de5c4..c8a24ba461 100644 --- a/docs/reference/mapping/dynamic/templates.asciidoc +++ b/docs/reference/mapping/dynamic/templates.asciidoc @@ -396,34 +396,3 @@ PUT my_index <1> Like the default dynamic mapping rules, doubles are mapped as floats, which are usually accurate enough, yet require half the disk space. - -[[override-default-template]] -=== Override default template - -You can override the default mappings for all indices and all types -by specifying a `_default_` type mapping in an index template -which matches all indices. - -For example, to disable the `_field_names` field by default for all types in all -new indices, you could create the following index template: - -[source,js] --------------------------------------------------- -PUT _template/disable_field_names -{ - "order": 0, - "index_patterns": ["*"], <1> - "mappings": { - "_default_": { <2> - "_field_names": { <3> - "enabled": false - } - } - } -} --------------------------------------------------- -// CONSOLE -<1> Applies the mappings to an `index` which matches the pattern `*`, in other - words, all new indices. -<2> Defines the `_default_` type mapping types within the index. -<3> Disables the `_field_names` field by default. diff --git a/docs/reference/mapping/fields.asciidoc b/docs/reference/mapping/fields.asciidoc index cc803ef566..9361c6b26d 100644 --- a/docs/reference/mapping/fields.asciidoc +++ b/docs/reference/mapping/fields.asciidoc @@ -49,11 +49,7 @@ can be customised when a mapping type is created. All fields in the document which contain non-null values. [float] -=== Routing meta-fields - -<<mapping-parent-field,`_parent`>>:: - - Used to create a parent-child relationship between two mapping types. +=== Routing meta-field <<mapping-routing-field,`_routing`>>:: @@ -67,7 +63,6 @@ can be customised when a mapping type is created. Application specific metadata. - include::fields/all-field.asciidoc[] include::fields/field-names-field.asciidoc[] @@ -78,8 +73,6 @@ include::fields/index-field.asciidoc[] include::fields/meta-field.asciidoc[] -include::fields/parent-field.asciidoc[] - include::fields/routing-field.asciidoc[] include::fields/source-field.asciidoc[] diff --git a/docs/reference/mapping/fields/meta-field.asciidoc b/docs/reference/mapping/fields/meta-field.asciidoc index befaf4e8df..c59877f231 100644 --- a/docs/reference/mapping/fields/meta-field.asciidoc +++ b/docs/reference/mapping/fields/meta-field.asciidoc @@ -1,7 +1,7 @@ [[mapping-meta-field]] === `_meta` field -Each mapping type can have custom meta data associated with it. These are not +A mapping type can have custom meta data associated with it. These are not used at all by Elasticsearch, but can be used to store application-specific metadata, such as the class that a document belongs to: diff --git a/docs/reference/mapping/fields/parent-field.asciidoc b/docs/reference/mapping/fields/parent-field.asciidoc deleted file mode 100644 index 873ed888da..0000000000 --- a/docs/reference/mapping/fields/parent-field.asciidoc +++ /dev/null @@ -1,169 +0,0 @@ -[[mapping-parent-field]] -=== `_parent` field - -A parent-child relationship can be established between documents in the same -index by making one mapping type the parent of another: - -[source,js] --------------------------------------------------- -PUT my_index -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "my_parent": {}, - "my_child": { - "_parent": { - "type": "my_parent" <1> - } - } - } -} - -PUT my_index/my_parent/1 <2> -{ - "text": "This is a parent document" -} - -PUT my_index/my_child/2?parent=1 <3> -{ - "text": "This is a child document" -} - -PUT my_index/my_child/3?parent=1&refresh=true <3> -{ - "text": "This is another child document" -} - -GET my_index/my_parent/_search -{ - "query": { - "has_child": { <4> - "type": "my_child", - "query": { - "match": { - "text": "child document" - } - } - } - } -} --------------------------------------------------- -// CONSOLE -<1> The `my_parent` type is parent to the `my_child` type. -<2> Index a parent document. -<3> Index two child documents, specifying the parent document's ID. -<4> Find all parent documents that have children which match the query. - - -See the <<query-dsl-has-child-query,`has_child`>> and -<<query-dsl-has-parent-query,`has_parent`>> queries, -the <<search-aggregations-bucket-children-aggregation,`children`>> aggregation, -and <<parent-child-inner-hits,inner hits>> for more information. - -The value of the `_parent` field is accessible in aggregations -and scripts, and may be queried with the -<<query-dsl-parent-id-query, `parent_id` query>>: - -[source,js] --------------------------- -GET my_index/_search -{ - "query": { - "parent_id": { <1> - "type": "my_child", - "id": "1" - } - }, - "aggs": { - "parents": { - "terms": { - "field": "_parent", <2> - "size": 10 - } - } - }, - "script_fields": { - "parent": { - "script": { - "source": "doc['_parent']" <3> - } - } - } -} --------------------------- -// CONSOLE -// TEST[continued] - -<1> Querying the id of the `_parent` field (also see the <<query-dsl-has-parent-query,`has_parent` query>> and the <<query-dsl-has-child-query,`has_child` query>>) -<2> Aggregating on the `_parent` field (also see the <<search-aggregations-bucket-children-aggregation,`children`>> aggregation) -<3> Accessing the `_parent` field in scripts - - -==== Parent-child restrictions - -* The parent and child types must be different -- parent-child relationships - cannot be established between documents of the same type. - -* The `_parent.type` setting can only point to a type that doesn't exist yet. - This means that a type cannot become a parent type after it has been - created. - -* Parent and child documents must be indexed on the same shard. The `parent` - ID is used as the <<mapping-routing-field,routing>> value for the child, - to ensure that the child is indexed on the same shard as the parent. - This means that the same `parent` value needs to be provided when - <<docs-get,getting>>, <<docs-delete,deleting>>, or <<docs-update,updating>> - a child document. - -==== Global ordinals - -Parent-child uses <<eager-global-ordinals,global ordinals>> to speed up joins. -Global ordinals need to be rebuilt after any change to a shard. The more -parent id values are stored in a shard, the longer it takes to rebuild the -global ordinals for the `_parent` field. - -Global ordinals, by default, are built eagerly: if the index has changed, -global ordinals for the `_parent` field will be rebuilt as part of the refresh. -This can add significant time the refresh. However most of the times this is the -right trade-off, otherwise global ordinals are rebuilt when the first parent-child -query or aggregation is used. This can introduce a significant latency spike for -your users and usually this is worse as multiple global ordinals for the `_parent` -field may be attempt rebuilt within a single refresh interval when many writes -are occurring. - -When the parent/child is used infrequently and writes occur frequently it may -make sense to disable eager loading: - -[source,js] --------------------------------------------------- -PUT my_index -{ - "settings": { - "mapping.single_type": false - }, - "mappings": { - "my_parent": {}, - "my_child": { - "_parent": { - "type": "my_parent", - "eager_global_ordinals": false - } - } - } -} --------------------------------------------------- -// CONSOLE - -The amount of heap used by global ordinals can be checked as follows: - -[source,sh] --------------------------------------------------- -# Per-index -GET _stats/fielddata?human&fields=_parent - -# Per-node per-index -GET _nodes/stats/indices/fielddata?human&fields=_parent --------------------------------------------------- -// CONSOLE diff --git a/docs/reference/mapping/fields/routing-field.asciidoc b/docs/reference/mapping/fields/routing-field.asciidoc index ebb659af34..96a5de1c61 100644 --- a/docs/reference/mapping/fields/routing-field.asciidoc +++ b/docs/reference/mapping/fields/routing-field.asciidoc @@ -6,8 +6,7 @@ formula: shard_num = hash(_routing) % num_primary_shards -The default value used for `_routing` is the document's <<mapping-id-field,`_id`>> -or the document's <<mapping-parent-field,`_parent`>> ID, if present. +The default value used for `_routing` is the document's <<mapping-id-field,`_id`>>. Custom routing patterns can be implemented by specifying a custom `routing` value per document. For instance: @@ -133,5 +132,5 @@ less than `index.number_of_shards`. Once enabled, the partitioned index will have the following limitations: -* Mappings with parent-child relationships cannot be created within it. +* Mappings with <<parent-join,`join` field>> relationships cannot be created within it. * All mappings within the index must have the `_routing` field marked as required.
\ No newline at end of file diff --git a/docs/reference/mapping/fields/type-field.asciidoc b/docs/reference/mapping/fields/type-field.asciidoc index 5eb620b642..f2ff963163 100644 --- a/docs/reference/mapping/fields/type-field.asciidoc +++ b/docs/reference/mapping/fields/type-field.asciidoc @@ -1,6 +1,8 @@ [[mapping-type-field]] === `_type` field +deprecated::[6.0.0,See <<removal-of-types>>] + Each document indexed is associated with a <<mapping-type-field,`_type`>> (see <<mapping-type>>) and an <<mapping-id-field,`_id`>>. The `_type` field is indexed in order to make searching by type name fast. @@ -11,28 +13,17 @@ scripts, and when sorting: [source,js] -------------------------- # Example documents -PUT my_index -{ - "settings": { - "mapping.single_type": false - } -} - -PUT my_index/type_1/1 -{ - "text": "Document with type 1" -} -PUT my_index/type_2/2?refresh=true +PUT my_index/doc/1?refresh=true { - "text": "Document with type 2" + "text": "Document with type 'doc'" } GET my_index/_search { "query": { - "terms": { - "_type": [ "type_1", "type_2" ] <1> + "term": { + "_type": "doc" <1> } }, "aggs": { diff --git a/docs/reference/mapping/fields/uid-field.asciidoc b/docs/reference/mapping/fields/uid-field.asciidoc index 7ab97a305d..68b5c82ab0 100644 --- a/docs/reference/mapping/fields/uid-field.asciidoc +++ b/docs/reference/mapping/fields/uid-field.asciidoc @@ -1,9 +1,7 @@ [[mapping-uid-field]] === `_uid` field -deprecated[6.0.0, Now that types have been removed, documents are uniquely -identified by their `_id` and the `_uid` field has only been kept as a view -over the `_id` field for backward compatibility.] +deprecated[6.0.0, Now that types have been removed, documents are uniquely identified by their `_id` and the `_uid` field has only been kept as a view over the `_id` field for backward compatibility.] Each document indexed is associated with a <<mapping-type-field,`_type`>> (see <<mapping-type>>) and an <<mapping-id-field,`_id`>>. These values are diff --git a/docs/reference/mapping/params/dynamic.asciidoc b/docs/reference/mapping/params/dynamic.asciidoc index 985e5a63df..9653a2ca43 100644 --- a/docs/reference/mapping/params/dynamic.asciidoc +++ b/docs/reference/mapping/params/dynamic.asciidoc @@ -82,6 +82,5 @@ PUT my_index <2> The `user` object inherits the type-level setting. <3> The `user.social_networks` object enables dynamic mapping, so new fields may be added to this inner object. -TIP: The `dynamic` setting is allowed to have different settings for fields of -the same name in the same index. Its value can be updated on existing fields +TIP: The `dynamic` setting can be updated on existing fields using the <<indices-put-mapping,PUT mapping API>>. diff --git a/docs/reference/mapping/params/enabled.asciidoc b/docs/reference/mapping/params/enabled.asciidoc index 9d7dc05b38..e14e727d6c 100644 --- a/docs/reference/mapping/params/enabled.asciidoc +++ b/docs/reference/mapping/params/enabled.asciidoc @@ -92,7 +92,6 @@ GET my_index/_mapping <3> <2> The document can be retrieved. <3> Checking the mapping reveals that no fields have been added. -TIP: The `enabled` setting is allowed to have different settings for fields of -the same name in the same index. Its value can be updated on existing fields +TIP: The `enabled` setting can be updated on existing fields using the <<indices-put-mapping,PUT mapping API>>. diff --git a/docs/reference/mapping/params/format.asciidoc b/docs/reference/mapping/params/format.asciidoc index 3619017141..530807589c 100644 --- a/docs/reference/mapping/params/format.asciidoc +++ b/docs/reference/mapping/params/format.asciidoc @@ -31,8 +31,7 @@ Many APIs which support date values also support <<date-math,date math>> expressions, such as `now-1m/d` -- the current time, minus one month, rounded down to the nearest day. -TIP: The `format` setting must have the same setting for fields of the same -name in the same index. Its value can be updated on existing fields using the +TIP: The `format` setting can be updated on existing fields using the <<indices-put-mapping,PUT mapping API>>. @@ -174,7 +173,7 @@ The following tables lists all the defaults ISO formats supported: `date_time` or `strict_date_time`:: - A formatter that combines a full date and time, separated by a 'T': + A formatter that combines a full date and time, separated by a 'T': `yyyy-MM-dd'T'HH:mm:ss.SSSZZ`. `date_time_no_millis` or `strict_date_time_no_millis`:: diff --git a/docs/reference/mapping/params/index.asciidoc b/docs/reference/mapping/params/index.asciidoc index 270c930e28..e097293d14 100644 --- a/docs/reference/mapping/params/index.asciidoc +++ b/docs/reference/mapping/params/index.asciidoc @@ -4,4 +4,3 @@ The `index` option controls whether field values are indexed. It accepts `true` or `false`. Fields that are not indexed are not queryable. -NOTE: For the legacy mapping type <<string,`string`>> the `index` option only accepts legacy values `analyzed` (default, treat as full-text field), `not_analyzed` (treat as keyword field) and `no`. diff --git a/docs/reference/mapping/params/position-increment-gap.asciidoc b/docs/reference/mapping/params/position-increment-gap.asciidoc index d9d37e2108..addd8aadbb 100644 --- a/docs/reference/mapping/params/position-increment-gap.asciidoc +++ b/docs/reference/mapping/params/position-increment-gap.asciidoc @@ -85,6 +85,3 @@ GET my_index/groups/_search <2> The phrase query matches our document which is weird, but its what we asked for in the mapping. -TIP: The `position_increment_gap` setting is allowed to have different settings -for fields of the same name in the same index. Its value can be updated on -existing fields using the <<indices-put-mapping,PUT mapping API>>. diff --git a/docs/reference/mapping/params/search-analyzer.asciidoc b/docs/reference/mapping/params/search-analyzer.asciidoc index d31a997430..26cda7afa1 100644 --- a/docs/reference/mapping/params/search-analyzer.asciidoc +++ b/docs/reference/mapping/params/search-analyzer.asciidoc @@ -78,6 +78,5 @@ GET my_index/_search See {defguide}/_index_time_search_as_you_type.html[Index time search-as-you- type] for a full explanation of this example. -TIP: The `search_analyzer` setting must have the same setting for fields of -the same name in the same index. Its value can be updated on existing fields +TIP: The `search_analyzer` setting can be updated on existing fields using the <<indices-put-mapping,PUT mapping API>>. diff --git a/docs/reference/mapping/removal_of_types.asciidoc b/docs/reference/mapping/removal_of_types.asciidoc new file mode 100644 index 0000000000..5a9a13fed9 --- /dev/null +++ b/docs/reference/mapping/removal_of_types.asciidoc @@ -0,0 +1,419 @@ +[[removal-of-types]] +== Removal of mapping types + +IMPORTANT: Indices created in Elasticsearch 6.0.0 or later may only contain a +single <<mapping-type,mapping type>>. Indices created in 5.x with multiple +mapping types will continue to function as before in Elasticsearch 6.x. +Mapping types will be completely removed in Elasticsearch 7.0.0. + +[float] +=== What are mapping types? + +Since the first release of Elasticsearch, each document has been stored in a +single index and assigned a single mapping type. A mapping type was used to +represent the type of document or entity being indexed, for instance a +`twitter` index might have a `user` type and a `tweet` type. + +Each mapping type could have its own fields, so the `user` type might have a +`full_name` field, a `user_name` field, and an `email` field, while the +`tweet` type could have a `content` field, a `tweeted_at` field and, like the +`user` type, a `user_name` field. + +Each document had a `_type` meta-field containing the type name, and searches +could be limited to one or more types by specifying the type name(s) in the +URL: + +[source,js] +---- +GET twitter/user,tweet/_search +{ + "query": { + "match": { + "user_name": "kimchy" + } + } +} +---- +// NOCONSOLE + +The `_type` field was combined with the document's `_id` to generate a `_uid` +field, so documents of different types with the same `_id` could exist in a +single index. + +Mapping types were also used to establish a +/guide/en/elasticsearch/reference/5.4/mapping-parent-field.html[parent-child relationship] +between documents, so documents of type `question` could be parents to +documents of type `answer`. + +[float] +=== Why are mapping types being removed? + +Initially, we spoke about an ``index'' being similar to a ``database'' in an +SQL database, and a ``type'' being equivalent to a +``table''. + +This was a bad analogy that led to incorrect assumptions. In an SQL database, +tables are independent of each other. The columns in one table have no +bearing on columns with the same name in another table. This is not the case +for fields in a mapping type. + +In an Elasticsearch index, fields that have the same name in different mapping +types are backed by the same Lucene field internally. In other words, using +the example above, the `user_name` field in the `user` type is stored in +exactly the same field as the `user_name` field in the `tweet` type, and both +`user_name` fields must have the same mapping (definition) in both types. + +This can lead to frustration when, for example, you want `deleted` to be a +`date` field in one type and a `boolean` field in another type in the same +index. + +On top of that, storing different entities that have few or no fields in +common in the same index leads to sparse data and interferes with Lucene's +ability to compress documents efficiently. + +For these reasons, we have decided to remove the concept of mapping types from +Elasticsearch. + +[float] +=== Alternatives to mapping types + +[float] +==== Index per document type + +The first alternative is to have an index per document type. Instead of +storing tweets and users in a single `twitter` index, you could store tweets +in the `tweets` index and users in the `user` index. Indices are completely +independent of each other and so there will be no conflict of field types +between indices. + +This approach has two benefits: + +* Data is more likely to be dense and so benefit from compression techniques + used in Lucene. + +* The term statistics used for scoring in full text search are more likely to + be accurate because all documents in the same index represent a single + entity. + +Each index can be sized appropriately for the number of documents it will +contain: you can use a smaller number of primary shards for `users` and a +larger number of primary shards for `tweets`. + +[float] +==== Custom type field + +Of course, there is a limit to how many primary shards can exist in a cluster +so you many not want to waste an entire shard for a collection of only a few +thousand documents. In this case, you can implement your own custom `type` +field which will work in a similar way to the old `_type`. + +Let's take the `user`/`tweet` example above. Originally, the workflow would +have looked something like this: + +[source,js] +---- +PUT twitter +{ + "mappings": { + "user": { + "properties": { + "name": { "type": "text" }, + "user_name": { "type": "keyword" }, + "email": { "type": "keyword" } + } + }, + "tweet": { + "properties": { + "content": { "type": "text" }, + "user_name": { "type": "keyword" }, + "tweeted_at": { "type": "date" } + } + } + } +} + +PUT twitter/user/kimchy +{ + "name": "Shay Banon", + "user_name": "kimchy", + "email": "shay@kimchy.com" +} + +PUT twitter/tweet/1 +{ + "user_name": "kimchy", + "tweeted_at": "2017-10-24T09:00:00Z", + "content": "Types are going away" +} + +GET twitter/tweet/_search +{ + "query": { + "match": { + "user_name": "kimchy" + } + } +} +---- +// NOCONSOLE + +You could achieve the same thing by adding a custom `type` field as follows: + +[source,js] +---- +PUT twitter +{ + "mappings": { + "doc": { + "properties": { + "type": { "type": "keyword" }, <1> + "name": { "type": "text" }, + "user_name": { "type": "keyword" }, + "email": { "type": "keyword" }, + "content": { "type": "text" }, + "tweeted_at": { "type": "date" } + } + } + } +} + +PUT twitter/doc/user-kimchy +{ + "type": "user", <1> + "name": "Shay Banon", + "user_name": "kimchy", + "email": "shay@kimchy.com" +} + +PUT twitter/doc/tweet-1 +{ + "type": "tweet", <1> + "user_name": "kimchy", + "tweeted_at": "2017-10-24T09:00:00Z", + "content": "Types are going away" +} + +GET twitter/_search +{ + "query": { + "bool": { + "must": { + "match": { + "user_name": "kimchy" + } + }, + "filter": { + "match": { + "type": "tweet" <1> + } + } + } + } +} +---- +// NOCONSOLE +<1> The explicit `type` field takes the place of the implicit `_type` field. + +[float] +==== Parent/Child without mapping types + +Previously, a parent-child relationship was represented by making one mapping +type the parent, and one or more other mapping types the children. Without +types, we can no longer use this syntax. The parent-child feature will +continue to function as before, except that the way of expressing the +relationship between documents has been changed to use the new +<<parent-join,`join` field>>. + + +[float] +=== Schedule for removal of mapping types + +This is a big change for our users, so we have tried to make it as painless as +possible. The change will roll out as follows: + +Elasticsearch 5.6.0:: + +* Setting `index.mapping.single_type: true` on an index will enable the + single-type-per-index behaviour which will be enforced in 6.0. + +* The <<parent-join,`join` field>> replacement for parent-child is available + on indices created in 5.6. + +Elasticsearch 6.x:: + +* Indices created in 5.x will continue to function in 6.x as they did in 5.x. + +* Indices created in 6.x only allow a single-type per index. Any name + can be used for the type, but there can be only one. + +* The `_type` name can no longer be combined with the `_id` to form the `_uid` + field. The `_uid` field has become an alias for the `_id` field. + +* New indices no longer support the old-style of parent/child and should + use the <<parent-join,`join` field>> instead. + +* The `_default_` mapping type is deprecated. + +Elasticsearch 7.x:: + +* The `type` parameter in URLs are optional. For instance, indexing + a document no longer requires a document `type`. + +* The `GET|PUT _mapping` APIs support a query string parameter + (`include_type_name`) which indicates whether the body should include + a layer for the type name. It defaults to `true`. 7.x indices which + don't have an explicit type will use the dummy type name `_doc`. + +* The `_default_` mapping type is removed. + +Elasticsearch 8.x:: + +* The `type` parameter is no longer supported in URLs. + +* The `include_in_type` parameter defaults to `false`. + +Elasticsearch 9.x:: + +* The `include_in_type` parameter is removed. + +[float] +=== Migrating multi-type indices to single-type + +The <<docs-reindex,Reindex API>> can be used to convert multi-type indices to +single-type indices. The following examples can be used in Elasticsearch 5.6 +or Elasticsearch 6.x. In 6.x, there is no need to specify +`index.mapping.single_type` as that is the default. + +[float] +==== Index per document type + +This first example splits our `twitter` index into a `tweets` index and a +`users` index: + +[source,js] +---- +PUT users +{ + "settings": { + "index.mapping.single_type": true + }, + "mappings": { + "user": { + "properties": { + "name": { + "type": "text" + }, + "user_name": { + "type": "keyword" + }, + "email": { + "type": "keyword" + } + } + } + } +} + +PUT tweets +{ + "settings": { + "index.mapping.single_type": true + }, + "mappings": { + "tweet": { + "properties": { + "content": { + "type": "text" + }, + "user_name": { + "type": "keyword" + }, + "tweeted_at": { + "type": "date" + } + } + } + } +} + +POST _reindex +{ + "source": { + "index": "twitter", + "type": "user" + }, + "dest": { + "index": "users" + } +} + +POST _reindex +{ + "source": { + "index": "twitter", + "type": "tweet" + }, + "dest": { + "index": "tweets" + } +} +---- +// NOCONSOLE + +[float] +==== Custom type field + +This next example adds a custom `type` field and sets it to the value of the +original `_type`. It also adds the type to the `_id` in case there are any +documents of different types which have conflicting IDs: + +[source,js] +---- +PUT new_twitter +{ + "mappings": { + "doc": { + "properties": { + "type": { + "type": "keyword" + }, + "name": { + "type": "text" + }, + "user_name": { + "type": "keyword" + }, + "email": { + "type": "keyword" + }, + "content": { + "type": "text" + }, + "tweeted_at": { + "type": "date" + } + } + } + } +} + + +POST _reindex +{ + "source": { + "index": "twitter" + }, + "dest": { + "index": "new_twitter" + }, + "script": { + "source": """ + ctx._source.type = ctx._type; + ctx._id = ctx._type + '-' + ctx._id; + ctx._type = 'doc'; + """ + } +} +---- +// NOCONSOLE + diff --git a/docs/reference/mapping/types.asciidoc b/docs/reference/mapping/types.asciidoc index 7cd2010726..8ef7c3b2bf 100644 --- a/docs/reference/mapping/types.asciidoc +++ b/docs/reference/mapping/types.asciidoc @@ -78,8 +78,6 @@ include::types/numeric.asciidoc[] include::types/object.asciidoc[] -include::types/string.asciidoc[] - include::types/text.asciidoc[] include::types/token-count.asciidoc[] diff --git a/docs/reference/mapping/types/parent-join.asciidoc b/docs/reference/mapping/types/parent-join.asciidoc index 63bdea1dc4..e0a8b5f022 100644 --- a/docs/reference/mapping/types/parent-join.asciidoc +++ b/docs/reference/mapping/types/parent-join.asciidoc @@ -60,7 +60,7 @@ WARNING: It is required to index the lineage of a parent in the same shard so yo always route child documents using their greater parent id. -For instance the following index two children documents pointing to the same parent `1 +For instance the following index two children documents pointing to the same parent `1` with a `routing` value equals to the `id` of the parent: [source,js] diff --git a/docs/reference/mapping/types/string.asciidoc b/docs/reference/mapping/types/string.asciidoc deleted file mode 100644 index c4be060cc4..0000000000 --- a/docs/reference/mapping/types/string.asciidoc +++ /dev/null @@ -1,18 +0,0 @@ -[[string]] -=== String datatype - -The `string` field is unsupported for indexes created in 5.x in favor of the -`text` and `keyword` fields. Attempting to create a string field in an index -created in 5.x will cause Elasticsearch to attempt to upgrade the `string` into -the appropriate `text` or `keyword` field. It will return an HTTP `Warning` -header telling you that `string` is deprecated. This upgrade process isn't -always perfect because there are some combinations of features that are -supported by `string` but not `text` or `keyword`. For that reason it is better -to use `text` or `keyword`. - -Indexes imported from 2.x *only* support `string` and not `text` or `keyword`. -To ease the migration from 2.x Elasticsearch will downgrade `text` and `keyword` -mappings applied to indexes imported from 2.x into `string`. While long lived -indexes will eventually need to be recreated against 5.x before eventually -upgrading to 6.x, this downgrading smooths the process before you find time for -it. diff --git a/docs/reference/mapping/types/text.asciidoc b/docs/reference/mapping/types/text.asciidoc index 16110e6b21..2f0d2dbce4 100644 --- a/docs/reference/mapping/types/text.asciidoc +++ b/docs/reference/mapping/types/text.asciidoc @@ -128,8 +128,3 @@ The following parameters are accepted by `text` fields: Whether term vectors should be stored for an <<mapping-index,`analyzed`>> field. Defaults to `no`. -NOTE: Indexes imported from 2.x do not support `text`. Instead they will -attempt to downgrade `text` into `string`. This allows you to merge modern -mappings with legacy mappings. Long lived indexes will have to be recreated -before upgrading to 6.x but mapping downgrade gives you the opportunity to do -the recreation on your own schedule. diff --git a/docs/reference/query-dsl/joining-queries.asciidoc b/docs/reference/query-dsl/joining-queries.asciidoc index 6a467fe539..e40b865506 100644 --- a/docs/reference/query-dsl/joining-queries.asciidoc +++ b/docs/reference/query-dsl/joining-queries.asciidoc @@ -13,8 +13,8 @@ fields are used to index arrays of objects, where each object can be queried <<query-dsl-has-child-query,`has_child`>> and <<query-dsl-has-parent-query,`has_parent`>> queries:: -A <<mapping-parent-field,parent-child relationship>> can exist between two -document types within a single index. The `has_child` query returns parent +A <<parent-join,`join` field relationship>> can exist between +documents within a single index. The `has_child` query returns parent documents whose child documents match the specified query, while the `has_parent` query returns child documents whose parent document matches the specified query. diff --git a/docs/reference/query-dsl/parent-id-query.asciidoc b/docs/reference/query-dsl/parent-id-query.asciidoc index 1a110f4147..eaf4a26ba1 100644 --- a/docs/reference/query-dsl/parent-id-query.asciidoc +++ b/docs/reference/query-dsl/parent-id-query.asciidoc @@ -1,85 +1,68 @@ [[query-dsl-parent-id-query]] === Parent Id Query -added[5.0.0] - The `parent_id` query can be used to find child documents which belong to a particular parent. Given the following mapping definition: [source,js] -------------------------------------------- -PUT /my_index +PUT my_index { - "settings": { - "mapping.single_type": false - }, - "mappings": { - "blog_post": { - "properties": { - "name": { - "type": "keyword" - } - } - }, - "blog_tag": { - "_parent": { - "type": "blog_post" - }, - "_routing": { - "required": true - } + "mappings": { + "doc": { + "properties": { + "my_join_field": { + "type": "join", + "relations": { + "my_parent": "my_child" + } } + } } + } } --------------------------------------------- -// CONSOLE -// TESTSETUP +PUT my_index/doc/1?refresh +{ + "text": "This is a parent document", + "my_join_field": "my_parent" +} -[source,js] --------------------------------------------------- -GET /my_index/_search +PUT my_index/doc/2?routing=1&refresh { - "query": { - "parent_id" : { - "type" : "blog_tag", - "id" : "1" - } - } + "text": "This is a child document", + "my_join_field": { + "name": "my_child", + "parent": "1" + } } --------------------------------------------------- -// CONSOLE -The above is functionally equivalent to using the following -<<query-dsl-has-parent-query, `has_parent`>> query, but performs -better as it does not need to do a join: +-------------------------------------------- +// CONSOLE +// TESTSETUP [source,js] -------------------------------------------------- GET /my_index/_search { "query": { - "has_parent": { - "parent_type": "blog_post", - "query": { - "term": { - "_id": "1" - } - } + "parent_id": { + "type": "my_child", + "id": "1" } } } -------------------------------------------------- // CONSOLE + ==== Parameters This query has two required parameters: [horizontal] -`type`:: The **child** type. This must be a type with `_parent` field. - -`id`:: The required parent id select documents must referrer to. +`type`:: The **child** type name, as specified in the <<parent-join,`join` field>>. +`id`:: The ID of the parent document. `ignore_unmapped`:: When set to `true` this will ignore an unmapped `type` and will not match any documents for this query. This can be useful when querying multiple indexes diff --git a/docs/reference/redirects.asciidoc b/docs/reference/redirects.asciidoc index 2ee6bc7b88..1e3c428731 100644 --- a/docs/reference/redirects.asciidoc +++ b/docs/reference/redirects.asciidoc @@ -55,6 +55,11 @@ directory. Instead, mappings should be created using the API with: * <<indices-put-mapping>> * <<indices-templates>> +[role="exclude",id="mapping-parent-field"] +=== `_parent` field + +The `_parent` field has been removed in favour of the <<parent-join,`join` field>>. + [role="exclude",id="modules-memcached"] === memcached @@ -465,7 +470,7 @@ index that make warmers not necessary anymore. The index time boost mapping has been replaced with query time boost (see <<mapping-boost>>). [role="exclude",id="modules-scripting-native"] -=== Native scripting +=== Native scripting Native scripts have been replaced with writing custom `ScriptEngine` backends (see <<modules-scripting-engine>>). diff --git a/docs/reference/search/request/highlighting.asciidoc b/docs/reference/search/request/highlighting.asciidoc index 73cf54046b..367ce1b786 100644 --- a/docs/reference/search/request/highlighting.asciidoc +++ b/docs/reference/search/request/highlighting.asciidoc @@ -39,8 +39,7 @@ be used for highlighting if it mapped to have `store` set to `true`. ================================== The field name supports wildcard notation. For example, using `comment_*` -will cause all <<text,text>> and <<keyword,keyword>> fields (and <<string,string>> -from versions before 5.0) that match the expression to be highlighted. +will cause all <<text,text>> and <<keyword,keyword>> fields that match the expression to be highlighted. Note that all other fields will not be highlighted. If you use a custom mapper and want to highlight on a field anyway, you have to provide the field name explicitly. diff --git a/docs/reference/search/request/inner-hits.asciidoc b/docs/reference/search/request/inner-hits.asciidoc index 72d13d2d11..e97f07c13f 100644 --- a/docs/reference/search/request/inner-hits.asciidoc +++ b/docs/reference/search/request/inner-hits.asciidoc @@ -190,7 +190,7 @@ So in the above example only the comment part is returned per nested hit and not document that contained the comment. [[nested-inner-hits-source]] -==== Nested inner hits and _source +==== Nested inner hits and +_source+ Nested document don't have a `_source` field, because the entire source of document is stored with the root document under its `_source` field. To include the source of just the nested document, the source of the root document is parsed and just diff --git a/docs/reference/search/request/stored-fields.asciidoc b/docs/reference/search/request/stored-fields.asciidoc index b2a64c7e14..2feb9313d8 100644 --- a/docs/reference/search/request/stored-fields.asciidoc +++ b/docs/reference/search/request/stored-fields.asciidoc @@ -40,8 +40,7 @@ GET /_search If the requested fields are not stored (`store` mapping set to `false`), they will be ignored. -Stored field values fetched from the document itself are always returned as an array. On the contrary, metadata fields like `_routing` and -`_parent` fields are never returned as an array. +Stored field values fetched from the document itself are always returned as an array. On the contrary, metadata fields like `_routing` are never returned as an array. Also only leaf fields can be returned via the `field` option. So object fields can't be returned and such requests will fail. |