diff options
author | Adrien Grand <jpountz@gmail.com> | 2016-06-20 18:06:31 +0200 |
---|---|---|
committer | Adrien Grand <jpountz@gmail.com> | 2016-06-22 08:35:54 +0200 |
commit | db9af54ec0a7c8b3ffad036ab1acea819999ae9e (patch) | |
tree | 3891e4c8502632dbdc97fea0cc5edb279b390a79 /docs/reference | |
parent | 5dc88ffd2664869fa3ef84d37623865599cbc506 (diff) |
Remove `_timestamp` and `_ttl` on 5.x indices. #18980
This removes the ability to use `_timestamp` and `_ttl` on indices created on
or after 5.0.
Closes #18280
Diffstat (limited to 'docs/reference')
-rw-r--r-- | docs/reference/docs/bulk.asciidoc | 20 | ||||
-rw-r--r-- | docs/reference/docs/index_.asciidoc | 64 | ||||
-rw-r--r-- | docs/reference/docs/reindex.asciidoc | 2 | ||||
-rw-r--r-- | docs/reference/docs/update.asciidoc | 2 | ||||
-rw-r--r-- | docs/reference/index-modules.asciidoc | 5 | ||||
-rw-r--r-- | docs/reference/ingest/ingest-node.asciidoc | 3 | ||||
-rw-r--r-- | docs/reference/mapping/fields.asciidoc | 8 | ||||
-rw-r--r-- | docs/reference/mapping/fields/timestamp-field.asciidoc | 97 | ||||
-rw-r--r-- | docs/reference/mapping/fields/ttl-field.asciidoc | 112 | ||||
-rw-r--r-- | docs/reference/migration/migrate_5_0/mapping.asciidoc | 8 | ||||
-rw-r--r-- | docs/reference/modules/indices.asciidoc | 4 | ||||
-rw-r--r-- | docs/reference/modules/indices/ttl_interval.asciidoc | 16 |
12 files changed, 10 insertions, 331 deletions
diff --git a/docs/reference/docs/bulk.asciidoc b/docs/reference/docs/bulk.asciidoc index 6b0a5b2e40..1fbd0419e2 100644 --- a/docs/reference/docs/bulk.asciidoc +++ b/docs/reference/docs/bulk.asciidoc @@ -128,26 +128,6 @@ field. It automatically follows the behavior of the index / delete operation based on the `_parent` / `_routing` mapping. [float] -[[bulk-timestamp]] -=== Timestamp - -deprecated[2.0.0,The `_timestamp` field is deprecated. Instead, use a normal <<date,`date`>> field and set its value explicitly] - -Each bulk item can include the timestamp value using the -`_timestamp`/`timestamp` field. It automatically follows the behavior of -the index operation based on the `_timestamp` mapping. - -[float] -[[bulk-ttl]] -=== TTL - -deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version] - -Each bulk item can include the ttl value using the `_ttl`/`ttl` field. -It automatically follows the behavior of the index operation based on -the `_ttl` mapping. - -[float] [[bulk-consistency]] === Write Consistency diff --git a/docs/reference/docs/index_.asciidoc b/docs/reference/docs/index_.asciidoc index aa62b65292..e1c260ae48 100644 --- a/docs/reference/docs/index_.asciidoc +++ b/docs/reference/docs/index_.asciidoc @@ -291,70 +291,6 @@ to be the same as its parent, unless the routing value is explicitly specified using the `routing` parameter. [float] -[[index-timestamp]] -=== Timestamp - -deprecated[2.0.0,The `_timestamp` field is deprecated. Instead, use a normal <<date,`date`>> field and set its value explicitly] - -A document can be indexed with a `timestamp` associated with it. The -`timestamp` value of a document can be set using the `timestamp` -parameter. For example: - -[source,js] --------------------------------------------------- -PUT twitter/tweet/1?timestamp=2009-11-15T14:12:12 -{ - "user" : "kimchy", - "message" : "trying out Elasticsearch" -} --------------------------------------------------- -// CONSOLE - -If the `timestamp` value is not provided externally or in the `_source`, -the `timestamp` will be automatically set to the date the document was -processed by the indexing chain. More information can be found on the -<<mapping-timestamp-field,_timestamp mapping -page>>. - -[float] -[[index-ttl]] -=== TTL - -deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version] - - -A document can be indexed with a `ttl` (time to live) associated with -it. Expired documents will be expunged automatically. The expiration -date that will be set for a document with a provided `ttl` is relative -to the `timestamp` of the document, meaning it can be based on the time -of indexing or on any time provided. The provided `ttl` must be strictly -positive and can be a number (in milliseconds) or any valid time value -as shown in the following examples: - -[source,js] --------------------------------------------------- -PUT twitter/tweet/1?ttl=86400000ms -{ - "user": "kimchy", - "message": "Trying out elasticsearch, so far so good?" -} --------------------------------------------------- -// CONSOLE - -[source,js] --------------------------------------------------- -PUT twitter/tweet/1?ttl=1d -{ - "user": "kimchy", - "message": "Trying out elasticsearch, so far so good?" -} --------------------------------------------------- -// CONSOLE - -More information can be found on the -<<mapping-ttl-field,_ttl mapping page>>. - -[float] [[index-distributed]] === Distributed diff --git a/docs/reference/docs/reindex.asciidoc b/docs/reference/docs/reindex.asciidoc index bb2a06c023..525552ae85 100644 --- a/docs/reference/docs/reindex.asciidoc +++ b/docs/reference/docs/reindex.asciidoc @@ -272,8 +272,6 @@ change: * `_version` * `_routing` * `_parent` - * `_timestamp` - * `_ttl` Setting `_version` to `null` or clearing it from the `ctx` map is just like not sending the version in an indexing request. It will cause that document to be diff --git a/docs/reference/docs/update.asciidoc b/docs/reference/docs/update.asciidoc index 39261c5d21..85e5bf2f67 100644 --- a/docs/reference/docs/update.asciidoc +++ b/docs/reference/docs/update.asciidoc @@ -57,7 +57,7 @@ curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{ In addition to `_source`, the following variables are available through the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing`, -`_parent`, `_timestamp`, `_ttl`. +`_parent`. We can also add a new field to the document: diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc index 3cc95e5af9..ff1a3c62c7 100644 --- a/docs/reference/index-modules.asciidoc +++ b/docs/reference/index-modules.asciidoc @@ -131,11 +131,6 @@ specific index module: Set to `true` to disable index metadata reads and writes. -`index.ttl.disable_purge`:: - - experimental[] Disables the purge of <<mapping-ttl-field,expired docs>> on - the current index. - `index.max_refresh_listeners`:: Maximum number of refresh listeners available on each shard of the index. diff --git a/docs/reference/ingest/ingest-node.asciidoc b/docs/reference/ingest/ingest-node.asciidoc index c8a5a05002..2d906fd861 100644 --- a/docs/reference/ingest/ingest-node.asciidoc +++ b/docs/reference/ingest/ingest-node.asciidoc @@ -439,8 +439,7 @@ The following example sets the `_id` metadata field of a document to `1`: } -------------------------------------------------- -The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`, `_parent`, -`_timestamp`, and `_ttl`. +The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`, `_parent`. [float] [[accessing-ingest-metadata]] diff --git a/docs/reference/mapping/fields.asciidoc b/docs/reference/mapping/fields.asciidoc index 90d75b9784..1cd886dde0 100644 --- a/docs/reference/mapping/fields.asciidoc +++ b/docs/reference/mapping/fields.asciidoc @@ -48,14 +48,6 @@ can be customised when a mapping type is created. All fields in the document which contain non-null values. -<<mapping-timestamp-field,`_timestamp`>>:: - - A timestamp associated with the document, either specified manually or auto-generated. - -<<mapping-ttl-field,`_ttl`>>:: - - How long a document should live before it is automatically deleted. - [float] === Routing meta-fields diff --git a/docs/reference/mapping/fields/timestamp-field.asciidoc b/docs/reference/mapping/fields/timestamp-field.asciidoc deleted file mode 100644 index 104b07cfee..0000000000 --- a/docs/reference/mapping/fields/timestamp-field.asciidoc +++ /dev/null @@ -1,97 +0,0 @@ -[[mapping-timestamp-field]] -=== `_timestamp` field - -deprecated[2.0.0,The `_timestamp` field is deprecated. Instead, use a normal <<date,`date`>> field and set its value explicitly] - -The `_timestamp` field, when enabled, allows a timestamp to be indexed and -stored with a document. The timestamp may be specified manually, generated -automatically, or set to a default value: - -[source,js] ------------------------------------- -PUT my_index -{ - "mappings": { - "my_type": { - "_timestamp": { <1> - "enabled": true - } - } - } -} - -PUT my_index/my_type/1?timestamp=2015-01-01 <2> -{ "text": "Timestamp as a formatted date" } - -PUT my_index/my_type/2?timestamp=1420070400000 <3> -{ "text": "Timestamp as milliseconds since the epoch" } - -PUT my_index/my_type/3 <4> -{ "text": "Autogenerated timestamp set to now()" } - ------------------------------------- -// CONSOLE - -<1> Enable the `_timestamp` field with default settings. -<2> Set the timestamp manually with a formatted date. -<3> Set the timestamp with milliseconds since the epoch. -<4> Auto-generates a timestamp with <<date-math,now()>>. - -The behaviour of the `_timestamp` field can be configured with the following parameters: - -`default`:: - - A default value to be used if none is provided. Defaults to <<date-math,now()>>. - -`format`:: - - The <<mapping-date-format,date format>> (or formats) to use when parsing timestamps. Defaults to `epoch_millis||strictDateOptionalTime`. - -`ignore_missing`:: - - If `true` (default), replace missing timestamps with the `default` value. If `false`, throw an exception. - - -The value of the `_timestamp` field is accessible in queries, aggregations, scripts, -and when sorting: - -[source,js] --------------------------- -GET my_index/_search -{ - "query": { - "range": { - "_timestamp": { <1> - "gte": "2015-01-01" - } - } - }, - "aggs": { - "Timestamps": { - "terms": { - "field": "_timestamp", <2> - "size": 10 - } - } - }, - "sort": [ - { - "_timestamp": { <3> - "order": "desc" - } - } - ], - "script_fields": { - "Timestamp": { - "script": "doc['_timestamp']" <4> - } - } -} --------------------------- -// CONSOLE -// TEST[continued] - -<1> Querying on the `_timestamp` field -<2> Aggregating on the `_timestamp` field -<3> Sorting on the `_timestamp` field -<4> Accessing the `_timestamp` field in scripts (inline scripts must be <<enable-dynamic-scripting,enabled>> for this example to work) diff --git a/docs/reference/mapping/fields/ttl-field.asciidoc b/docs/reference/mapping/fields/ttl-field.asciidoc deleted file mode 100644 index 26760d5aca..0000000000 --- a/docs/reference/mapping/fields/ttl-field.asciidoc +++ /dev/null @@ -1,112 +0,0 @@ -[[mapping-ttl-field]] -=== `_ttl` field - -deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version] - -Some types of documents, such as session data or special offers, come with an -expiration date. The `_ttl` field allows you to specify the minimum time a -document should live, after which time the document is deleted automatically. - -[TIP] -.Prefer index-per-timeframe to TTL -====================================================== - -With TTL , expired documents first have to be marked as deleted then later -purged from the index when segments are merged. For append-only time-based -data such as log events, it is much more efficient to use an index-per-day / -week / month instead of TTLs. Old log data can be removed by simply deleting -old indices. - -====================================================== - -The `_ttl` field may be enabled as follows: - -[source,js] -------------------------------- -PUT my_index -{ - "mappings": { - "my_type": { - "_ttl": { - "enabled": true - } - } - } -} - -PUT my_index/my_type/1?ttl=10m <1> -{ - "text": "Will expire in 10 minutes" -} - -PUT my_index/my_type/2 <2> -{ - "text": "Will not expire" -} -------------------------------- -// CONSOLE -<1> This document will expire 10 minutes after being indexed. -<2> This document has no TTL set and will not expire. - -The expiry time is calculated as the value of the -<<mapping-timestamp-field,`_timestamp`>> field (or `now()` if the `_timestamp` -is not enabled) plus the `ttl` specified in the indexing request. - -==== Default TTL - -You can provide a default `_ttl`, which will be applied to indexing requests where the `ttl` is not specified: - -[source,js] -------------------------------- -PUT my_index -{ - "mappings": { - "my_type": { - "_ttl": { - "enabled": true, - "default": "5m" - } - } - } -} - -PUT my_index/my_type/1?ttl=10m <1> -{ - "text": "Will expire in 10 minutes" -} - -PUT my_index/my_type/2 <2> -{ - "text": "Will expire in 5 minutes" -} -------------------------------- -// CONSOLE -<1> This document will expire 10 minutes after being indexed. -<2> This document has no TTL set and so will expire after the default 5 minutes. - -The `default` value can use <<time-units,time units>> like `d` for days, and -will use `ms` as the default unit if no time unit is provided. - -You can dynamically update the `default` value using the put mapping -API. It won't change the `_ttl` of already indexed documents but will be -used for future documents. - -==== Note on documents expiration - -Expired documents will be automatically deleted periodically. The following -settings control the expiry process: - -`indices.ttl.interval`:: - -How often the purge process should run. Defaults to `60s`. Expired documents -may still be retrieved before they are purged. - -`indices.ttl.bulk_size`:: - -How many deletions are handled by a single <<docs-bulk,`bulk`>> request. The -default value is `10000`. - -==== Note on `detect_noop` -If an update tries to update just the `_ttl` without changing the `_source` of -the document it's expiration time won't be updated if `detect_noop` is `true`. -In 2.1 `detect_noop` defaults to `true`. diff --git a/docs/reference/migration/migrate_5_0/mapping.asciidoc b/docs/reference/migration/migrate_5_0/mapping.asciidoc index 84dea729ec..2d0ca4b9fb 100644 --- a/docs/reference/migration/migrate_5_0/mapping.asciidoc +++ b/docs/reference/migration/migrate_5_0/mapping.asciidoc @@ -76,6 +76,14 @@ PUT my_index Also the `precision_step` parameter is now irrelevant and will be rejected on indices that are created on or after 5.0. +==== `_timestamp` and `_ttl` + +The `_timestamp` and `_ttl` fields were deprecated and are now removed. As a +replacement for `_timestamp`, you should populate a regular date field with the +current timestamp on application side. For `_ttl`, you should either use +time-based indices when applicable, or cron a delete-by-query with a range +query on a timestamp field + ==== `index` property On all field datatypes (except for the deprecated `string` field), the `index` diff --git a/docs/reference/modules/indices.asciidoc b/docs/reference/modules/indices.asciidoc index 309d449012..34a1ccdf8f 100644 --- a/docs/reference/modules/indices.asciidoc +++ b/docs/reference/modules/indices.asciidoc @@ -30,10 +30,6 @@ Available settings include: Control the resource limits on the shard recovery process. -<<indices-ttl,TTL interval>>:: - - Control how expired documents are removed. - include::indices/circuit_breaker.asciidoc[] include::indices/fielddata.asciidoc[] diff --git a/docs/reference/modules/indices/ttl_interval.asciidoc b/docs/reference/modules/indices/ttl_interval.asciidoc deleted file mode 100644 index 5e3069302d..0000000000 --- a/docs/reference/modules/indices/ttl_interval.asciidoc +++ /dev/null @@ -1,16 +0,0 @@ -[[indices-ttl]] -=== TTL interval - -Documents that have a <<mapping-ttl-field,`ttl`>> value set need to be deleted -once they have expired. How and how often they are deleted is controlled by -the following dynamic cluster settings: - -`indices.ttl.interval`:: - - How often the deletion process runs. Defaults to `60s`. - -`indices.ttl.bulk_size`:: - - The deletions are processed with a <<docs-bulk,bulk request>>. - The number of deletions processed can be configured with - this settings. Defaults to `10000`. |