Server Log a bit too verbose

[Scott D 69]

The last few days I have noticed the server log files growing to unusable sizes and spiking the hard drive activity.  This morning, the log file was rotated at midnight and at 6:29 am it appears to have hit its limit and a new one was started.

 

During that timeframe, only one user was watching OTA tv provided by HDHomeRun.

 

I wish I could attach the server log files, as will be requested, but they are 515Mb in size.  They tend to gag word/note pad when trying to redact private information.

 

Attached are screenshots of the …/logs directory and the activity from the dashboard.

 

System information:

Server 4.2.1.0

Win-ders 10 Home (Microsoft Windows NT 6.2.9200.0)

Client Roku Ultra 3.0.175

 

Would it be possible to add a few things to the log file options?

1.  A check box option to add/remove/redact all private information.

2.  A verbosity setting that would provide a little less detail when unchecked.  Or change "debug logging" to a scale of 0-5, where 0 = no log creation, 1 = basic system info of plug-ins started, system configuration, system capabilities, etc. and 5 is crazy amount of information.

 

If available, and trying to troubleshoot a problem, the options can be adjusted to provide information as is typically requested and when things are running smoothly, there would be less activity creating unused log information.

 

[scb99 245]

The last few days I have noticed the server log files growing to unusable sizes and spiking the hard drive activity.  This morning, the log file was rotated at midnight and at 6:29 am it appears to have hit its limit and a new one was started.

 

During that timeframe, only one user was watching OTA tv provided by HDHomeRun.

 

I wish I could attach the server log files, as will be requested, but they are 515Mb in size.  They tend to gag word/note pad when trying to redact private information.

 

Attached are screenshots of the …/logs directory and the activity from the dashboard.

 

System information:

Server 4.2.1.0

Win-ders 10 Home (Microsoft Windows NT 6.2.9200.0)

Client Roku Ultra 3.0.175

 

Would it be possible to add a few things to the log file options?

1.  A check box option to add/remove/redact all private information.

2.  A verbosity setting that would provide a little less detail when unchecked.  Or change "debug logging" to a scale of 0-5, where 0 = no log creation, 1 = basic system info of plug-ins started, system configuration, system capabilities, etc. and 5 is crazy amount of information.

 

If available, and trying to troubleshoot a problem, the options can be adjusted to provide information as is typically requested and when things are running smoothly, there would be less activity creating unused log information.

 

Funny enough I was thinking to make a post along the same lines, but only about the private info.  Actually I think the log should -by default- hash out at least external IP addresses and ports, and the User ID part of windows profile paths, and rather require a switch to positively include this info. I see so many people posting logs here and when I look at the log out of curiosity, it's not info I would personally want posted.

Edited September 29, 2019 by scb99

[Luke 42085]

Make sure you haven't enabled debug logging and then it shouldn't be too bad.

 

Please note that plugins may be contributing to what you're seeing as well.

[Scott D 69]

Make sure you haven't enabled debug logging and then it shouldn't be too bad.

 

Please note that plugins may be contributing to what you're seeing as well.

 

Debug is/has been unchecked.  I believe the only plug-ins are those installed by the system.

 

Attached is the start-up portion of the log file.  If you see anything that is out of line, let me know.

startup.txt

[Luke 42085]

That log looks fine to me. You'e added the playback reporting plugin, auto-organize, and others.

[Happy2Play 9784]

And with 19 transcoded/remuxed files most of the information resides in the server log also along with session information.  Without seeing the server log it would appear to be normal.

[Scott D 69]

I guess my point is, that with all of the logging that goes on when nothing seems to be wrong is simply utilizing services and disk space only to be cleared out when the log files are deleted.  I walked in on the process of writing the log file and found my hard drive sitting with a 95% utilization.  I watched the log file jump from 400m to over 500 m in a very short period of time.  

 

I understand the importance of log files when trying to troubleshoot an issue.  But, it seems a bit of a resource hog when things seem to be running smoothly and no active viewing of the logs are necessary.  Adding an option to perform a varied level of logging might be more useful.  Maybe log hard errors by default and warnings, anomalies and debug on request.

 

I don't have the most powerful system on the market, but I do have a 500g SSD that is used for logging, transcoding and the like.  There is over 250g of free space most of the time.  I have found the drive to be near capacity with log files and temp transcoding files on a fairly regular basis.  In a 15 hour period there was over a gig of log files alone.  Just saying.

 

As I have noted in another current posting, the best way I have found to document a problem is to clear all logs, restart the server, and then perform the function in question.  Then a clean log file is produced and digging through 100's of pages of repetitive entries is not necessary. 

 

I wish I could have uploaded the server log but with a 90mb limit, that would not have worked.  Tried to parse through the file on my own, but opening in word/note pad was a waste of a morning.

 

Just throwing my perspective out there.  I have no idea what it would take to implement, but to me it seems like a lot of data being slung around that ends up in the bin.

[Luke 42085]

We'd have to look at an example. Thanks.

[scb99 245]

Also the other point made by the OP, that private info should be anonymised by default, seems to have been overlooked.

[ebr 16195]

Also the other point made by the OP, that private info should be anonymised by default, seems to have been overlooked.

 

It hasn't been overlooked.  There is an existing feature request for this.

 

It should be pointed out, however, that doing that is in direct contrast with his primary complaint here (that logging is taking too many resources) as attempting to obfuscate logs on the fly would consume even more resources.  It also isn't always completely obvious (from a machine perspective) what should be obfuscated so a user doing this themselves is the safest way.

 

Thanks.