[ vra troubleshooting ]

Working with vRA 8.x logs

With the release of vRA 8 we are no longer able to simply tail a single vcac log file during investigations. We now need to work with Kubernetes and how containers handle logging.

Kubernetes comes with a tool called kubectl, which can be used to follow logs. The kubectl logs command can be used with the --follow parameter to tail the pods logs.

For example, to follow the postgres pods logs we can use kubectl logs --namespace [vra namespace] --follow [pod name]

kubectl logs --namespace prelude --follow postgres-0

Since this is a kubectl command it works across nodes in a clustered environment. Regardless if the pod is running locally or on another node in the cluster. Unfortunately, there are some drawbacks when leveraging this approach.

For example, one of my recent cases, 1/3 nodes had a networking issues which brought down the postgres pod. This meant we were unable to review the logs with kubectl

Fortunately the pods running on a node store their logs locally in /services-logs/prelude/ So, we can simply tail all the log files in the folder with tail -f /services-logs/prelude/postgres-0/*.log

To further expand on this, lets say we want to tail all of the logs for all of the pods in the prelude namespace on a node. We can leverage find /services-logs/prelude/ -name "*.log" | xargs tail -f


vRA 8.x


vRA 8.2 has a new section regarding (Legacy) Pod logs

Happy Logging :)



DISCLAIMER This is a personal blog. Any views or opinions represented in this blog are personal and belong solely to the blog owner and do not represent those of people, institutions or organizations that the owner may or may not be associated with in professional or personal capacity, unless explicitly stated. Any views or opinions are not intended to malign any religion, ethnic group, club, organization, company, or individual.
All content provided on this blog is for informational purposes only. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of this information.
Photos
Unless stated, all photos are the work of the blog owner and are licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. If used with watermark, no need to credit to the blog owner. For any edit to photos, including cropping, please contact me first.
Recipes
Unless stated, all recipes are the work of the blog owner and are licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. Please credit all recipes to the blog owner and link back to the original blog post.
Downloadable Files
Any downloadable file, including but not limited to pdfs, docs, jpegs, pngs, is provided at the user’s own risk. The owner will not be liable for any losses, injuries, or damages resulting from a corrupted or damaged file.
Comments
Comments are welcome. However, the blog owner reserves the right to edit or delete any comments submitted to this blog without notice due to
– Comments deemed to be spam or questionable spam
– Comments including profanity
– Comments containing language or concepts that could be deemed offensive
– Comments containing hate speech, credible threats, or direct attacks on an individual or group
The blog owner is not responsible for the content in comments.
This policy is subject to change at anytime. disclamer c/o http://kaloferov.com/