Damn, if you took out the “Kubernetes” part, and made it generalized, it looks like you built something that I have wanted to see for a long time. I think log explorers work best as a GUI, and that they need deep integration with structured logs. Basically I just want the DataDog log explorer but locally, and able to simply intake from some files. Some have tried, but they are always too simple, not parsing out properties of structured logs and building good filtering on top of that. I think setting up Grafana/Loki/Whatever is way too heavy for such a simple ask.
Anyway all that is anecdotal, what you made here is really cool!
Look into https://logdy.dev . I've played around with it a little bit, but really putting some work into learning it and integrating it with OTEL traces is my next side project.
Another alternative is https://openobserve.ai/ . It needs to run as a daemon to ingest logs (instead of opening a file), but it has a really nice UI.
Thanks! Part of what enables us to make a helpful lightweight solution is that we're leveraging the Kubernetes API to give logs context without requiring extra configuration. It'd be great to generalize Kubetail but a lot of that depends on how cloud platforms evolve over the next few years. Do you use Kubernetes?
Sorry for bringing up my own side-project on a "Show HN", but I'm making humanlog.io which does exactly what you want. Local-first log query engine (and tracing too, soon). You feed it your logs and you can search them, aggregate them, and soon make some graphs and dashboards with them. It started as just a CLI tool to parse and make structured logs pretty, and now I'm turning it into a full observability tool on your machine.
It's very WIP but I would love to help you get started if you want to try it out.
I don't use kubernetes but the GUI looks great, I'm looking for something sililar for heroku (currently storing logs in papertrail but haven't found a great way to search through achieved logs, especially locally).
Great demo video btw. Would you mind detailing which software you used to make it? I'd like to do similarly for some of my projects.
Thanks! I used Screen Studio for the demos (https://screen.studio). I had to pay for it but I couldn't find the zoom+highlight feature anywhere else.
Regarding Heroku - I loved Heroku the first time I used it so I set up my Kubernetes cluster to work in a similar way (`git push` to deploy to production). I know K8s has a reputation for being useful only for large deployments/large teams but in my experience this isn't true. You can run a single node cluster and use it to host multiple projects all sharing the same load balancer, or you can scale up to thousands of machines with many microservices working together internally. Would it be helpful for you to see a HOWTO for setting up a Heroku-like K8s cluster?
Wow, this is exactly what I’ve been missing—juggling a dozen kubectl logs windows and still losing context. Seeing all container logs merged in real time is a game-changer for debugging multi-pod workloads. Love that it runs locally against the API—no more sending sensitive logs offsite. Big thanks to the author for saving my sanity here!
While the search offered is handy, I watch logs on multi-pod workloads via:
kubectl logs -f -l app=api --max-log-requests=50
This follows along all pods with the given label (app: api) for up to 50 pods or however many you want. Quite useful when I'm looking for specific output such as ERROR logs to just pipe it to grep like this:
Thanks! Your comment made my day! It sounds like your use-case is similar to mine when I started working on the project. Now we have a community of contributors working on Kubetail so if you have time, stop by our Discord and let us know what else we can do to help (https://discord.gg/CmsmWAVkvX).
I was about to comment about Stern. It's an established well known OSS project - I wonder if the author knew about it, and if so.. what was his inspiration for creating an almost identical clone, albeit with a UI on top?
+1 for Stern especially since it only has Go dependencies whereas Kubetail does not. Ease of integration with an existing stack is a bigger addition than the lack of a web UI is a subtraction.
If you get a chance to try out the Kubetail CLI, I'd love to hear your thoughts! There's a lot of overlap with stern at the moment but we're planning on adding new features soon that will be unique (e.g. remote grep, system logs).
"Search" feels like a bit of a stretch to me--that suggests that it plays in the same space as OpenSearch or Splunk. There's no index here that I can tell.
Thanks! Kubetail doesn't cache or store logs itself. By default, it uses the Kubernetes API to fetch logs from your cluster and send them directly to your client (browser or terminal). If the "Kubetail Cluster API" is installed then it uses Kubetail's custom agent to do the same.
Kubetail is more lightweight than Loki/Grafana. It fetches logs using the Kubernetes API which means you can only see the current state of the cluster but you can use it without installing any additional software or provisioning storage. Our new search feature greps container log files on the nodes themselves as compared to Loki/Grafana that builds a full-text index in the background and queries that at search time (I think).
Stern is a CLI tool which more closely compares to the Kubetail CLI tool (as opposed to the Kubetail web interface). Currently, there's a lot of overlap between the two tools but Kubetail gives you more control over source filters and time. For example with the Kubetail CLI tool you can do queries like this:
I'm not too familiar with stern though so please correct me if I'm wrong. In any case, soon we're going to add more features to the Kubetail CLI tool that will be unique (e.g. remote grep, system logs).
Anyway all that is anecdotal, what you made here is really cool!
Another alternative is https://openobserve.ai/ . It needs to run as a daemon to ingest logs (instead of opening a file), but it has a really nice UI.
It's very WIP but I would love to help you get started if you want to try it out.
Great demo video btw. Would you mind detailing which software you used to make it? I'd like to do similarly for some of my projects.
Regarding Heroku - I loved Heroku the first time I used it so I set up my Kubernetes cluster to work in a similar way (`git push` to deploy to production). I know K8s has a reputation for being useful only for large deployments/large teams but in my experience this isn't true. You can run a single node cluster and use it to host multiple projects all sharing the same load balancer, or you can scale up to thousands of machines with many microservices working together internally. Would it be helpful for you to see a HOWTO for setting up a Heroku-like K8s cluster?
https://github.com/stern/stern
"Filter" sounds more accurate.
Would be really cool to install it into k8s and just hit a hosted web endpoint with all the logs and grep/exploration capabilities kubetail has.
https://github.com/johanhaleby/kubetail
Stern is a CLI tool which more closely compares to the Kubetail CLI tool (as opposed to the Kubetail web interface). Currently, there's a lot of overlap between the two tools but Kubetail gives you more control over source filters and time. For example with the Kubetail CLI tool you can do queries like this:
I'm not too familiar with stern though so please correct me if I'm wrong. In any case, soon we're going to add more features to the Kubetail CLI tool that will be unique (e.g. remote grep, system logs).