Include log in /config folder

[PhuriousGeorge]

enhancement req

To help with troubleshooting, it would be helpful to easily retrieve log data (and differentiate log-level). I haven't entered the container to see if a log is kept internally, but reviewing via docker logs tubesync or within an admin interface ie: Portainer, most of what is seen is output stating what it's collecting to generate the web interface.

[meeb]

Yep a reasonable suggestion. The only reason this wasn't done already is I didn't want to have to handle log rotation manually in the container with yet another process, without it the log could get massive pretty quickly.

[PhuriousGeorge]

Understood. I've not developed much in Linux beyond some caveman scripts, but isn't log rotation typically managed by the logrotate package and a cron job?

[meeb]

Yes, via logrotate on the host machine. However the container can't assume logrotate is set up and configured on the host to manage the log rotations properly (e.g. via Docker on Windows or deploying onto a Kubernetes cluster which have no ability to have logrotate for files running). Without some in-container log rotation management the logs TubeSync would write would have by default (from docker run...) potentially no maximum size. The Python logging library supports rotation by day, but not zipping etc. features that logrotate does. It's perfectly possible, just a bit of extra work to implement as it has to work entirely out of the box.

[PhuriousGeorge]

Since the /config folder is exposed to the guest, I would have thought logratate within the container should be able to manage it? I'm still trying to figure out Docker, but I've managed this in an lxc container with an attached directory.

[meeb]

Yes, logrotate can handle it if configured on the host quite easily. There's no way for logrotate no easily notify the container that the log has been rotated, but that's not too much of an issue.

I just don't want to have to require setting up logrotate on the host for people who may not know how to set it up or configure it, or potentially add a "landmine" option where "enabling logging" can do something like fill up your volume on a k8s deployment with no obvious solution where logrotate may not even be available.

The "docker/container" way to handle this would be to just use docker for logging including just to log container stdout/stderr to the hosts syslog (for example with --log-driver=syslog) and then configure syslog to write container logs to a logfile if you want them that way by filtering them with a syslog facility. I can see why a plain text file is useful but for ease of use and universal working out of the container without any additional magic required on the host it'll need some extra log rotation handling work.

[Code-Slave]

what ive seen done is set a linecount rotate logs when it hits that count

[meeb]

Cheers for the suggestion. You can handle this in Python with TimedRotatingFileHandler or similar and rotate based on file size or per day etc. with the standard logging library.

However, this would mean different logs for the front end and each worker processes (as they spawn as completely different processes sharing a single fd would be more complicated than its worth) and no logs for the web server (as nginx has no built in logging rotation by design). For nginx again would require logrotate to be set up externally outside of the container as well as sending a USR1 to nginx in the container once rotated or some additional log handler process in the container. This is possible, but would be a mess, not to mention 5 different log files by v1.0.

In the immediate short term probably just using the centralised logging already in container hosting / Docker / etc. (which is designed to handle this) is probably the easiest way for now. I would think if you're capable of setting up custom logrotate configs you're capable of setting up a syslog facility to write to a file for a tubesync.log. The entire design really of containers is you just write to stdout/stderr and let some centralised handler manage it in most cases.

I'll sort internal logging to logfiles out at some point, but it's not that high up my personal priority tree right now given the rest of the work queued up in the other milestones.

[tcely]

My testing experience was greatly improved by simply removing the nginx logs.

access_log /config/log/nginx/access.log.gz combined gzip=9 flush=10m;

If I need to look for anything I can run zless /config/log/nginx/access.log.gz and search for whatever request I'm checking.

My testing container has been running for days now and that file is about 150k, so rotating isn't a pressing concern, at least for me.

[tcely]

I've created a pull request (#713) for logs from nginx to be written as files in /config/log/nginx because I found this useful. It's better this way than continuing to use my custom script for reconfiguring nginx logging.

Rotation is handled at container start before nginx runs the first time. Logs from the previous 10 container starts should remain available.

Because of this, I recommend using the default restart policy of no or on-failure:3 to prevent logs being lost because of a failing container.