@Mohammad Saber​ :

If you are not seeing the expected logs in the log files, it's possible that either logging was not properly configured or the logs have been rotated out of the active log file and into an archive file. Here are some suggestions you can try to locate the logs for the queries you ran:

1. Check if logging was properly configured: Ensure that logging was properly configured when you set up the cluster. You can check the cluster's logging configuration by going to the cluster configuration page and looking under the "Advanced Options" section.
2. Check the archive files: As you noted, logs are periodically archived and stored in separate gzipped files. Check the archive files for the relevant time period to see if they contain the logs you are looking for. You can access the archive files by navigating to the DBFS folder where the logs are stored and searching for files with names like "log4j-YYYY-MM-DD-HH.log.gz". Note that you may need to unzip the files to view their contents.
3. Check the audit logs: Databricks provides an audit logging feature that records all user activity in the workspace. This includes information such as who ran a query, when it was run, and which tables were accessed. You can access the audit logs by navigating to the "Audit Logs" section of the workspace.
4. Check the metastore logs: If your tables are stored in Hive Metastore, you may be able to find information about table usage and queries in the metastore logs. These logs can typically be found in the same directory as the metastore database, and may contain information about table creation, modification, and usage.

If none of these suggestions help you locate the logs you are looking for, you may need to consult with the Databricks support team for further assistance.