Description
In read mode if exit_after_read is set to true and file_completed_action to log (i.e., the files are not removed from their path after being fully read) logstash will not properly halt execution if all files have already been fully read during a previous run.
This is because files which are fully read (according to sincedb) are marked as in-active and are thus skipped in the Processor.process_active file loop and left in a watched state thus the pipeline never becomes empty.
Calling common_detach_when_allread before skipping in-active files, when exit_after_read is set fixes this issue.
Information
- Version: 4.2.4
- Operating System: Ubuntu Bionic (18.04)
- Config File:
# pipeline config
input {
file {
path => "/tmp/test.log"
mode => "read"
sincedb_path => "/tmp/sincedb"
file_completed_action => "log"
file_completed_log_path => "/tmp/done.log"
exit_after_read => true
}
}
output {
stdout {}
}
- Sample Data:
/tmp/test.log
Line 1
Line 2
Line 3
Line 4
Line 5
- Steps to Reproduce:
- Run the pipeline the first time (this will create the sincedb file)
- See that the logstash process correctly stops after reading "Line 5"
- Run the pipeline again
- See that the logstash process correctly recognizes
test.log as fully read, but does not stop execution.
Description
In read mode if
exit_after_readis set totrueandfile_completed_actiontolog(i.e., the files are not removed from their path after being fully read) logstash will not properly halt execution if all files have already been fully read during a previous run.This is because files which are fully read (according to sincedb) are marked as in-active and are thus skipped in the
Processor.process_activefile loop and left in a watched state thus the pipeline never becomesempty.Calling
common_detach_when_allreadbefore skipping in-active files, whenexit_after_readis set fixes this issue.Information
/tmp/test.log
test.logas fully read, but does not stop execution.