Hi, surprisingly we have found 2 new folders with some short logfiles in our Databricks workspace:
ls -lFr sparse_logs/ tensorboard/
tensorboard/:
-rwxrwxrwx 1 root root 88 Sep 2 11:26 events.out.tfevents.1725275744.0830-063833-n68nsxoq-10-139-64-10.2071.0*
-rwxrwxrwx 1 root root 88 Sep 2 07:26 events.out.tfevents.1725261966.0830-063833-n68nsxoq-10-139-64-10.2411.0*
-rwxrwxrwx 1 root root 88 Sep 2 06:52 events.out.tfevents.1725259952.0830-063833-n68nsxoq-10-139-64-10.4812.0*
-rwxrwxrwx 1 root root 88 Aug 30 07:34 events.out.tfevents.1725000798.0830-063833-n68nsxoq-10-139-64-12.2804.0*
-rwxrwxrwx 1 root root 88 Aug 30 06:30 events.out.tfevents.1724998741.0828-073605-8ilf5p15-10-139-64-11.121675.0*
-rwxrwxrwx 1 root root 88 Aug 30 06:09 events.out.tfevents.1724997141.0828-073605-8ilf5p15-10-139-64-11.117151.0*
-rwxrwxrwx 1 root root 88 Aug 30 05:41 events.out.tfevents.1724995015.0828-073605-8ilf5p15-10-139-64-11.112695.0*
-rwxrwxrwx 1 root root 88 Aug 30 05:08 events.out.tfevents.1724939336.0828-073605-8ilf5p15-10-139-64-11.4013.0*
sparse_logs/:
-rwxrwxrwx 1 root root 43 Aug 30 07:34 30-08-2024_06.53.17.log*
-rwxrwxrwx 1 root root 43 Aug 30 06:30 30-08-2024_06.19.01.log*
-rwxrwxrwx 1 root root 43 Aug 30 06:09 30-08-2024_05.52.20.log*
-rwxrwxrwx 1 root root 43 Sep 6 08:49 30-08-2024_05.16.55.log*
-rwxrwxrwx 1 root root 43 Sep 25 12:42 29-08-2024_13.48.55.log*
-rwxrwxrwx 1 root root 43 Sep 25 12:47 02-09-2024_11.15.43.log*
-rwxrwxrwx 1 root root 43 Sep 25 12:44 02-09-2024_07.26.06.log*
We do some RAG-developement and thereby we serve an embedding-model (bge_large_en_v1_5-1) and llm (llama_2_7b_chat_hf-3, llama3_70b_endpoint), but actually we can't reproduce, what or which process has created these files. After deleteing them they never appeared again. Has anybody an idea, what it was?
Content sparse-logs was: manager stage: Model structure initialized
Thanks 😊
