databricks dbx execute - how to check cluster standard error log

Question

When I use databricks connect I can see standard error log via my local shell. Now I am using databricks dbx, only shows dbx log... Is there way to check cluster log easily(standard error)? Standard error/log4j out of databricks cluster

Update

when I miss argument :

databricks connect log

main.py: error: the following arguments are required: args1

dbx log(only about dbx)

⠼ Running the entrypoint file[dbx][2023-01-19 10:39:57.445] Execution failed, please follow the given error
╭────────── Traceback (most recent call last) ───────────╮
│ d:\...s\execute.py:144 in execute                      │
│                                                        │
│   141 │   │   upload_via_context=upload_via_context,   │
│   142 │   │   pip_install_extras=pip_install_extras,   │
│   143 │   )                                            │
│ ❱ 144 │   controller_instance.run()                    │
│   145                                                  │
│   146                                                  │
blablabla...
RuntimeError: Command execution failed. Traceback from cluster:
An exception has occurred, use %tb to see the full traceback.

I found follow link, https://github.com/databrickslabs/dbx/issues/112 it says databricks cluster cannot export log right now... is this true?

Answer 1

Have you tried these steps , method in this document: Diagnostic logging in Azure Databricks ?

Diagnostic settings:

You can configure the collection of following data and connect to respective environment as shown in the below image

For example if you want to connect log Analytics then, configure the diagnostic setting, you can go to Log Analytics Workspace -> Logs -> Log Management-> you will find Databricks notebook -> Run the below query to get the details about the notebook.

DatabricksNotebook 

| where TimeGenerated > ago(24h) 

| limit 10

Update:

You can access these files from the Driver logs tab on the data bricks cluster.

Go to cluster -> driver logs -> Click the name of a log file and download it.