Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP ignores messages without trailing \n v5.6.5 #99

Closed
slatemine opened this issue Dec 14, 2017 · 3 comments
Closed

TCP ignores messages without trailing \n v5.6.5 #99

slatemine opened this issue Dec 14, 2017 · 3 comments

Comments

@slatemine
Copy link

slatemine commented Dec 14, 2017

I suspect this is related to issue #90 and #95 but in moving from logstash 5.4.0 to 5.6.5 we hit this issue, unless a message is newline terminated it seems to get ignored.

The udp input continued to work as before.


  • Version: 5.6.5
  • Operating System: ubuntu 16.04
  • Config File (if you have sensitive info, please remove it):
input {
  tcp {
    port => "514"
    type => "syslog"
  }
  udp {
    port => "514"
    type => "syslog"
  }
}
  • Steps to Reproduce:
    Toggle comment on line #msg = "%s\n"%(msg)
#!/usr/bin/env python
import json, datetime
import socket
import logging
from collections import OrderedDict
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

ts = datetime.datetime.now()
time = ts.isoformat()
obj = {"another":"panda", "field": time}
evt = json.dumps(obj)
logger.info(evt)

dic = OrderedDict()
dic['Event'] = obj

msg = json.dumps(dic, sort_keys=False, ensure_ascii=True)
#msg = "%s\n"%(msg)
msg = msg.encode()
logger.info((msg,time))

socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    socket.connect(('localhost',514))
    socket.send(msg)
finally:
    socket.close()
@slatemine
Copy link
Author

slatemine commented Dec 14, 2017

Codec seems to be lines, even when I try to force it to plain, I wonder it that is related.

I see this in debug every time I send a msg using the above.

[DEBUG][logstash.codecs.line     ] config LogStash::Codecs::Line/@charset = "UTF-8"
[DEBUG][logstash.codecs.line     ] config LogStash::Codecs::Line/@id = "line_b369f23e-4ff4-4656-a3a7-0515fb865645"
[DEBUG][logstash.codecs.line     ] config LogStash::Codecs::Line/@enable_metric = true
[DEBUG][logstash.codecs.line     ] config LogStash::Codecs::Line/@delimiter = "\n"

@danhermann
Copy link
Contributor

This issue should be resolved as of the commit referenced above and released in v5.0.3 of the TCP input plugin.

@slatemine
Copy link
Author

v5.0.3 confirmed to be working correctly in ES 5.6.5, but I had to update Gemfile to allow the input to advance passed v4xx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants