-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to set date format in Ingest node's date processor #22982
Comments
This is because ingest is not using the same date format as specified here https://www.elastic.co/guide/en/elasticsearch/reference/5.2/mapping-date-format.html @talevy is ingest using joda? Could we support the standard date formats available in ES? |
"formats": [ "yyyy-MM-dd'T'HH:mm:ss" ] also throws an exception because of 'T' in the middle (isn't it a valid Joda pattern?) |
What is your conclusion so far? Is it a bug or am I doing something wrong? I need to parse dates in "2017-02-07T10:53:22" format in Ingest node processor. |
Hi @john16, I just tried this with no problems. Although we do not have support for the Elasticsearch here is an example curl request:
The error you are seeing is because of the escaping necessary for single quotes For Kibana's console, you could use this: POST /_ingest/pipeline/_simulate
{
"pipeline": {
"processors" : [
{
"date": {
"field": "my_date",
"formats": ["yyyy-MM-dd'T'HH:mm:ss"]
}
}
]
},
"docs": [
{
"_source": {
"my_date": "2017-02-07T10:53:22"
}
}
]
} |
@clintongormley I don't see a reason why we wouldn't be able to support the built-in date format names. I'll look into it |
Then it's a documentation issue as we say that we support Joda patterns in https://www.elastic.co/guide/en/elasticsearch/reference/current/date-processor.html (see format) |
Hmmm. I misread https://www.elastic.co/guide/en/elasticsearch/reference/5.2/mapping-date-format.html. Ignore my previous comment. |
@dadoonet, I do like the idea of supporting Elasticsearch's date-format built-ins. I've begun a PR to see how it would fit. Since it is used during mapping/indexing, there isn't too much of a reason it shouldn't be supported |
Yeah! Same here and I think we should then update the ingest doc to link to https://www.elastic.co/guide/en/elasticsearch/reference/5.2/mapping-date-format.html or something like this. |
@dadoonet, I've opened up an issue for this. ^^^. |
Thanks! (May be mention this in docs? Because the same pattern works without escaping in Logstash and if you move from Logstash you need to add escaping). |
Closing this in favor of #23032 |
Elasticsearch version: 5.1.2
Plugins installed: []
JVM version: OpenJDK
OS version: Ubuntu 16.04
Description of the problem including expected versus actual behavior:
Steps to reproduce:
1.
2.
3.
Provide logs (if relevant):
See https://discuss.elastic.co/t/ingest-node-processor-date-unable-to-parse-date
The text was updated successfully, but these errors were encountered: