Remove `_timestamp` and `_ttl` on 5.x indices. #18980

This removes the ability to use `_timestamp` and `_ttl` on indices created on
or after 5.0.

Closes #18280

12 files changed, 10 insertions, 331 deletions

diff --git a/docs/reference/docs/bulk.asciidoc b/docs/reference/docs/bulk.asciidoc
index 6b0a5b2e40..1fbd0419e2 100644
--- a/docs/reference/docs/bulk.asciidoc
+++ b/docs/reference/docs/bulk.asciidoc
@@ -128,26 +128,6 @@ field. It automatically follows the behavior of the index / delete
 operation based on the `_parent` / `_routing` mapping.
 
 [float]
-[[bulk-timestamp]]
-=== Timestamp
-
-deprecated[2.0.0,The `_timestamp` field is deprecated.  Instead, use a normal <<date,`date`>> field and set its value explicitly]
-
-Each bulk item can include the timestamp value using the
-`_timestamp`/`timestamp` field. It automatically follows the behavior of
-the index operation based on the `_timestamp` mapping.
-
-[float]
-[[bulk-ttl]]
-=== TTL
-
-deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version]
-
-Each bulk item can include the ttl value using the `_ttl`/`ttl` field.
-It automatically follows the behavior of the index operation based on
-the `_ttl` mapping.
-
-[float]
 [[bulk-consistency]]
 === Write Consistency
 
diff --git a/docs/reference/docs/index_.asciidoc b/docs/reference/docs/index_.asciidoc
index aa62b65292..e1c260ae48 100644
--- a/docs/reference/docs/index_.asciidoc
+++ b/docs/reference/docs/index_.asciidoc
@@ -291,70 +291,6 @@ to be the same as its parent, unless the routing value is explicitly
 specified using the `routing` parameter.
 
 [float]
-[[index-timestamp]]
-=== Timestamp
-
-deprecated[2.0.0,The `_timestamp` field is deprecated.  Instead, use a normal <<date,`date`>> field and set its value explicitly]
-
-A document can be indexed with a `timestamp` associated with it. The
-`timestamp` value of a document can be set using the `timestamp`
-parameter. For example:
-
-[source,js]
---------------------------------------------------
-PUT twitter/tweet/1?timestamp=2009-11-15T14:12:12
-{
-    "user" : "kimchy",
-    "message" : "trying out Elasticsearch"
-}
---------------------------------------------------
-// CONSOLE
-
-If the `timestamp` value is not provided externally or in the `_source`,
-the `timestamp` will be automatically set to the date the document was
-processed by the indexing chain. More information can be found on the
-<<mapping-timestamp-field,_timestamp mapping
-page>>.
-
-[float]
-[[index-ttl]]
-=== TTL
-
-deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version]
-
-
-A document can be indexed with a `ttl` (time to live) associated with
-it. Expired documents will be expunged automatically. The expiration
-date that will be set for a document with a provided `ttl` is relative
-to the `timestamp` of the document, meaning it can be based on the time
-of indexing or on any time provided. The provided `ttl` must be strictly
-positive and can be a number (in milliseconds) or any valid time value
-as shown in the following examples:
-
-[source,js]
---------------------------------------------------
-PUT twitter/tweet/1?ttl=86400000ms
-{
-    "user": "kimchy",
-    "message": "Trying out elasticsearch, so far so good?"
-}
---------------------------------------------------
-// CONSOLE
-
-[source,js]
---------------------------------------------------
-PUT twitter/tweet/1?ttl=1d
-{
-    "user": "kimchy",
-    "message": "Trying out elasticsearch, so far so good?"
-}
---------------------------------------------------
-// CONSOLE
-
-More information can be found on the
-<<mapping-ttl-field,_ttl mapping page>>.
-
-[float]
 [[index-distributed]]
 === Distributed
 
diff --git a/docs/reference/docs/reindex.asciidoc b/docs/reference/docs/reindex.asciidoc
index bb2a06c023..525552ae85 100644
--- a/docs/reference/docs/reindex.asciidoc
+++ b/docs/reference/docs/reindex.asciidoc
@@ -272,8 +272,6 @@ change:
  * `_version`
  * `_routing`
  * `_parent`
- * `_timestamp`
- * `_ttl`
 
 Setting `_version` to `null` or clearing it from the `ctx` map is just like not
 sending the version in an indexing request. It will cause that document to be
diff --git a/docs/reference/docs/update.asciidoc b/docs/reference/docs/update.asciidoc
index 39261c5d21..85e5bf2f67 100644
--- a/docs/reference/docs/update.asciidoc
+++ b/docs/reference/docs/update.asciidoc
@@ -57,7 +57,7 @@ curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{
 
 In addition to `_source`, the following variables are available through
 the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing`,
-`_parent`, `_timestamp`, `_ttl`.
+`_parent`.
 
 We can also add a new field to the document:
 
diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc
index 3cc95e5af9..ff1a3c62c7 100644
--- a/docs/reference/index-modules.asciidoc
+++ b/docs/reference/index-modules.asciidoc
@@ -131,11 +131,6 @@ specific index module:
 
     Set to `true` to disable index metadata reads and writes.
 
-`index.ttl.disable_purge`::
-
-    experimental[] Disables the purge of <<mapping-ttl-field,expired docs>> on
-    the current index.
-
 `index.max_refresh_listeners`::
 
     Maximum number of refresh listeners available on each shard of the index.
diff --git a/docs/reference/ingest/ingest-node.asciidoc b/docs/reference/ingest/ingest-node.asciidoc
index c8a5a05002..2d906fd861 100644
--- a/docs/reference/ingest/ingest-node.asciidoc
+++ b/docs/reference/ingest/ingest-node.asciidoc
@@ -439,8 +439,7 @@ The following example sets the `_id` metadata field of a document to `1`:
 }
 --------------------------------------------------
 
-The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`, `_parent`,
-`_timestamp`, and `_ttl`.
+The following metadata fields are accessible by a processor: `_index`, `_type`, `_id`, `_routing`, `_parent`.
 
 [float]
 [[accessing-ingest-metadata]]
diff --git a/docs/reference/mapping/fields.asciidoc b/docs/reference/mapping/fields.asciidoc
index 90d75b9784..1cd886dde0 100644
--- a/docs/reference/mapping/fields.asciidoc
+++ b/docs/reference/mapping/fields.asciidoc
@@ -48,14 +48,6 @@ can be customised when a mapping type is created.
 
     All fields in the document which contain non-null values.
 
-<<mapping-timestamp-field,`_timestamp`>>::
-
-    A timestamp associated with the document, either specified manually or auto-generated.
-
-<<mapping-ttl-field,`_ttl`>>::
-
-    How long a document should live before it is automatically deleted.
-
 [float]
 === Routing meta-fields
 
diff --git a/docs/reference/mapping/fields/timestamp-field.asciidoc b/docs/reference/mapping/fields/timestamp-field.asciidoc
deleted file mode 100644
index 104b07cfee..0000000000
--- a/docs/reference/mapping/fields/timestamp-field.asciidoc
+++ /dev/null
@@ -1,97 +0,0 @@
-[[mapping-timestamp-field]]
-=== `_timestamp` field
-
-deprecated[2.0.0,The `_timestamp` field is deprecated.  Instead, use a normal <<date,`date`>> field and set its value explicitly]
-
-The `_timestamp` field, when enabled, allows a timestamp to be indexed and
-stored with a document. The timestamp may be specified manually, generated
-automatically, or set to a default value:
-
-[source,js]
-------------------------------------
-PUT my_index
-{
-  "mappings": {
-    "my_type": {
-      "_timestamp": { <1>
-        "enabled": true
-      }
-    }
-  }
-}
-
-PUT my_index/my_type/1?timestamp=2015-01-01 <2>
-{ "text": "Timestamp as a formatted date" }
-
-PUT my_index/my_type/2?timestamp=1420070400000 <3>
-{ "text": "Timestamp as milliseconds since the epoch" }
-
-PUT my_index/my_type/3 <4>
-{ "text": "Autogenerated timestamp set to now()" }
-
-------------------------------------
-// CONSOLE
-
-<1> Enable the `_timestamp` field with default settings.
-<2> Set the timestamp manually with a formatted date.
-<3> Set the timestamp with milliseconds since the epoch.
-<4> Auto-generates a timestamp with <<date-math,now()>>.
-
-The behaviour of the `_timestamp` field can be configured with the following parameters:
-
-`default`::
-
-    A default value to be used if none is provided.  Defaults to <<date-math,now()>>.
-
-`format`::
-
-    The <<mapping-date-format,date format>> (or formats) to use when parsing timestamps.  Defaults to `epoch_millis||strictDateOptionalTime`.
-
-`ignore_missing`::
-
-    If `true` (default), replace missing timestamps with the `default` value.  If `false`, throw an exception.
-
-
-The value of the `_timestamp` field is accessible in queries, aggregations, scripts,
-and when sorting:
-
-[source,js]
---------------------------
-GET my_index/_search
-{
-  "query": {
-    "range": {
-      "_timestamp": { <1>
-        "gte": "2015-01-01"
-      }
-    }
-  },
-  "aggs": {
-    "Timestamps": {
-      "terms": {
-        "field": "_timestamp", <2>
-        "size": 10
-      }
-    }
-  },
-  "sort": [
-    {
-      "_timestamp": { <3>
-        "order": "desc"
-      }
-    }
-  ],
-  "script_fields": {
-    "Timestamp": {
-      "script": "doc['_timestamp']" <4>
-    }
-  }
-}
---------------------------
-// CONSOLE
-// TEST[continued]
-
-<1> Querying on the `_timestamp` field
-<2> Aggregating on the `_timestamp` field
-<3> Sorting on the `_timestamp` field
-<4> Accessing the `_timestamp` field in scripts (inline scripts must be <<enable-dynamic-scripting,enabled>> for this example to work)
diff --git a/docs/reference/mapping/fields/ttl-field.asciidoc b/docs/reference/mapping/fields/ttl-field.asciidoc
deleted file mode 100644
index 26760d5aca..0000000000
--- a/docs/reference/mapping/fields/ttl-field.asciidoc
+++ /dev/null
@@ -1,112 +0,0 @@
-[[mapping-ttl-field]]
-=== `_ttl` field
-
-deprecated[2.0.0,The current `_ttl` implementation is deprecated and will be replaced with a different implementation in a future version]
-
-Some types of documents, such as session data or special offers, come with an
-expiration date. The `_ttl` field allows you to specify the minimum time a
-document should live, after which time the document is deleted automatically.
-
-[TIP]
-.Prefer index-per-timeframe to TTL
-======================================================
-
-With TTL , expired documents first have to be marked as deleted then later
-purged from the index when segments are merged.  For append-only time-based
-data such as log events, it is much more efficient to use an index-per-day /
-week / month instead of TTLs.  Old log data can be removed by simply deleting
-old indices.
-
-======================================================
-
-The `_ttl` field may be enabled as follows:
-
-[source,js]
--------------------------------
-PUT my_index
-{
-  "mappings": {
-    "my_type": {
-      "_ttl": {
-        "enabled": true
-      }
-    }
-  }
-}
-
-PUT my_index/my_type/1?ttl=10m <1>
-{
-  "text": "Will expire in 10 minutes"
-}
-
-PUT my_index/my_type/2 <2>
-{
-  "text": "Will not expire"
-}
--------------------------------
-// CONSOLE
-<1> This document will expire 10 minutes after being indexed.
-<2> This document has no TTL set and will not expire.
-
-The expiry time is calculated as the value of the
-<<mapping-timestamp-field,`_timestamp`>> field (or `now()` if the `_timestamp`
-is not enabled) plus the `ttl` specified in the indexing request.
-
-==== Default TTL
-
-You can provide a default `_ttl`, which will be applied to indexing requests where the `ttl` is not specified:
-
-[source,js]
--------------------------------
-PUT my_index
-{
-  "mappings": {
-    "my_type": {
-      "_ttl": {
-        "enabled": true,
-        "default": "5m"
-      }
-    }
-  }
-}
-
-PUT my_index/my_type/1?ttl=10m <1>
-{
-  "text": "Will expire in 10 minutes"
-}
-
-PUT my_index/my_type/2 <2>
-{
-  "text": "Will expire in 5 minutes"
-}
--------------------------------
-// CONSOLE
-<1> This document will expire 10 minutes after being indexed.
-<2> This document has no TTL set and so will expire after the default 5 minutes.
-
-The `default` value can use <<time-units,time units>> like `d` for days, and
-will use `ms` as the default unit if no time unit is provided.
-
-You can dynamically update the `default` value using the put mapping
-API. It won't change the `_ttl` of already indexed documents but will be
-used for future documents.
-
-==== Note on documents expiration
-
-Expired documents will be automatically deleted periodically. The following
-settings control the expiry process:
-
-`indices.ttl.interval`::
-
-How often the purge process should run. Defaults to `60s`. Expired documents
-may still be retrieved before they are purged.
-
-`indices.ttl.bulk_size`::
-
-How many deletions are handled by a single <<docs-bulk,`bulk`>> request. The
-default value is `10000`.
-
-==== Note on `detect_noop`
-If an update tries to update just the `_ttl` without changing the `_source` of
-the document it's expiration time won't be updated if `detect_noop` is `true`.
-In 2.1 `detect_noop` defaults to `true`.
diff --git a/docs/reference/migration/migrate_5_0/mapping.asciidoc b/docs/reference/migration/migrate_5_0/mapping.asciidoc
index 84dea729ec..2d0ca4b9fb 100644
--- a/docs/reference/migration/migrate_5_0/mapping.asciidoc
+++ b/docs/reference/migration/migrate_5_0/mapping.asciidoc
@@ -76,6 +76,14 @@ PUT my_index
 Also the `precision_step` parameter is now irrelevant and will be rejected on
 indices that are created on or after 5.0.
 
+==== `_timestamp` and `_ttl`
+
+The `_timestamp` and `_ttl` fields were deprecated and are now removed. As a
+replacement for `_timestamp`, you should populate a regular date field with the
+current timestamp on application side. For `_ttl`, you should either use
+time-based indices when applicable, or cron a delete-by-query with a range
+query on a timestamp field
+
 ==== `index` property
 
 On all field datatypes (except for the deprecated `string` field), the `index`
diff --git a/docs/reference/modules/indices.asciidoc b/docs/reference/modules/indices.asciidoc
index 309d449012..34a1ccdf8f 100644
--- a/docs/reference/modules/indices.asciidoc
+++ b/docs/reference/modules/indices.asciidoc
@@ -30,10 +30,6 @@ Available settings include:
 
     Control the resource limits on the shard recovery process.
 
-<<indices-ttl,TTL interval>>::
-
-    Control how expired documents are removed.
-
 include::indices/circuit_breaker.asciidoc[]
 
 include::indices/fielddata.asciidoc[]
diff --git a/docs/reference/modules/indices/ttl_interval.asciidoc b/docs/reference/modules/indices/ttl_interval.asciidoc
deleted file mode 100644
index 5e3069302d..0000000000
--- a/docs/reference/modules/indices/ttl_interval.asciidoc
+++ /dev/null
@@ -1,16 +0,0 @@
-[[indices-ttl]]
-=== TTL interval
-
-Documents that have a <<mapping-ttl-field,`ttl`>> value set need to be deleted
-once they have expired.  How and how often they are deleted is controlled by
-the following dynamic cluster settings:
-
-`indices.ttl.interval`::
-
-    How often the deletion process runs.  Defaults to `60s`.
-
-`indices.ttl.bulk_size`::
-
-    The deletions are processed with a <<docs-bulk,bulk request>>.
-    The number of deletions processed can be configured with
-    this settings. Defaults to `10000`.